This page is intended for use with Hipchat server and explains additional export/import info, required logs to troubleshoot export/import issues, recommendations, common warnings you see from imports, etc. The main export/import documentation can be found here.

Additional export/import info

• Deactivated users are included in the export
• Archived rooms are included in the export
• While the export/import process is running, some intermediate files will be stored under /file_store/scissortail. They will be removed once the job finishes, regardless of the result of the job. The actual export files are stored on cumulus.
• Starting from Hipchat server v1.3 build 1.3.1, export notifications can be sent to multiple email recipients via hipchat export --notify her@example.com him@example.com and it handles file attachments from deleted accounts for consistency
• Starting from Hipchat server v1.3 build 1.3.3, exports can be interrupted at the command line with hipchat export --cancel
• As of Hipchat Server v1.3 build 1.3.7, export of 1-to-1 chat history includes 1-to-1 chats of deactivated users.

Recommendations

• When exporting and importing between two Hipchat Server instances, make sure they are configured with different FQDNs prior to importing to avoid potential routing issues
• It's highly recommended to upgrade server to the latest available version before doing an export/import mainly because latest version has bugs addressed
• If you're migrating to another server to have user authentication with external directory, follow these steps:
  1. Generate the export

  2. In parallel, configure SSL/network on the server that you want to migrate to using a local admin account that's unique (not associated with any existing accounts in the other server or external directory - this helps to prevent admin lockout issue)
  3. Import the generated export to the new server (You can tail -f /var/log/hipchat/hcapp.log to watch the progress via the CLI)
  4. Check that the email addresses used by the imported users match the ones coming from external directory, and update if necessary
  5. Add directory configuration - synchronisation will merge existing accounts with directory accounts if the email addresses matched

  6. Install and configure integrations

Export/import logs

The following logs are essential in troubleshooting export/import issues:

• /var/log/hipchat/hcapp.log
• /var/log/hipchat/coral.log
• /var/log/hipchat/runtime.log

It would be good to send them in when you raise an issue to Support along with Hipchat server version and the type of import (from cloud to server, from server to server). In a certain scenario, /var/log may be required for further investigation. You can filter the lines that have 'scissortail-worker' in hcapp.log to learn more about the export/import processes and you can see the actions triggered.

Validation

Export completion through /var/log/hipchat/hcapp.log:

• Start

  2017-09-14T03:08:53.678359+00:00 hipchat.example.com gunicorn.scissortail-web.access: [4712] - - "GET /status HTTP/1.1" 200 244 "-" "python-requests/2.8.1"
  2017-09-14T03:08:53.682430+00:00 hipchat224 scissortail-web:4712:INFO GID: 1 JID: eaa5a498-450e-4f66-83d9-3f020c786a8e - get_export_job_status eaa5a498-450e-4f66-83d9-3f020c786a8e
  2017-09-14T03:08:53.684103+00:00 hipchat224 scissortail-web:4712:INFO GID: 1 JID: eaa5a498-450e-4f66-83d9-3f020c786a8e - Coral calls made during job eaa5a498-450e-4f66-83d9-3f020c786a8e: 256, coral calls$
  2017-09-14T03:08:53.690176+00:00 hipchat.example.com gunicorn.scissortail-web.access: [4712] - - "GET /export/group/1/latest HTTP/1.1" 200 851 "-" "python-requests/2.8.1"
  2017-09-14T03:08:53.726906+00:00 hipchat224 scissortail-web:4712:INFO GID: 1 JID: a4accbe6-6163-4941-858c-46883aaeaed8 - Created rq job, a4accbe6-6163-4941-858c-46883aaeaed8
  2017-09-14T03:08:53.727035+00:00 hipchat224 scissortail-web:4712:INFO GID: 1 JID: a4accbe6-6163-4941-858c-46883aaeaed8 - Created export job, a4accbe6-6163-4941-858c-46883aaeaed8
  2017-09-14T03:08:53.730827+00:00 hipchat224 scissortail-web:4712:INFO GID: 1 JID: a4accbe6-6163-4941-858c-46883aaeaed8 - Created notif job, f93fbabc-f87b-47b4-be3f-b84bc41796f8: scissortail.workers.app.n$
  2017-09-14T03:08:53.731089+00:00 hipchat224 scissortail-web:4712:INFO GID: 1 JID: a4accbe6-6163-4941-858c-46883aaeaed8 - get_export_job_status a4accbe6-6163-4941-858c-46883aaeaed8
  2017-09-14T03:08:53.732041+00:00 hipchat224 scissortail-web:4712:INFO GID: 1 JID: a4accbe6-6163-4941-858c-46883aaeaed8 - Created export jid-a4accbe6-6163-4941-858c-46883aaeaed8 for gid-1 uid-1 | include_$
  2017-09-14T03:08:53.739005+00:00 hipchat224 scissortail-worker:5419:INFO GID: 1 JID: a4accbe6-6163-4941-858c-46883aaeaed8 - Starting export of gid-uid: 1-1 | include_attachments True | include_oto True |$

• End

  2017-09-14T03:08:58.364650+00:00 hipchat224 scissortail-worker:5419:INFO GID: 1 JID: a4accbe6-6163-4941-858c-46883aaeaed8 - Deleted /file_store/scissortail/tmpo_dk25jp/data
  2017-09-14T03:08:58.365027+00:00 hipchat224 scissortail-worker:5419:INFO GID: 1 JID: a4accbe6-6163-4941-858c-46883aaeaed8 - Encrypting /file_store/scissortail/tmpo_dk25jp/hipchat-2017-09-14_03-08-53.tar.$
  2017-09-14T03:08:58.369820+00:00 hipchat224 scissortail-worker:5419:INFO GID: 1 JID: a4accbe6-6163-4941-858c-46883aaeaed8 - Deleted /file_store/scissortail/tmpo_dk25jp/hipchat-2017-09-14_03-08-53.tar.gz
  2017-09-14T03:08:58.371603+00:00 hipchat224 scissortail-worker:5419:INFO GID: 1 JID: a4accbe6-6163-4941-858c-46883aaeaed8 - Config[site] => loading from '/hipchat/config/site.json'
  2017-09-14T03:08:58.372020+00:00 hipchat224 scissortail-worker:5419:INFO GID: 1 JID: a4accbe6-6163-4941-858c-46883aaeaed8 - Making directories as necessary /file_store/local/files/1/1/9rMJw1V51wxONTX/ fo$
  2017-09-14T03:08:58.372486+00:00 hipchat224 scissortail-worker:5419:INFO GID: 1 JID: a4accbe6-6163-4941-858c-46883aaeaed8 - Saved /file_store/local/files/1/1/9rMJw1V51wxONTX/hipchat-2017-09-14_03-08-53.t$
  2017-09-14T03:08:58.373146+00:00 hipchat224 scissortail-worker:5419:INFO GID: 1 JID: a4accbe6-6163-4941-858c-46883aaeaed8 - Finished export. Result is available at https://hipchat.example.com$
  2017-09-14T03:08:58.373784+00:00 hipchat224 scissortail-worker:5419:INFO GID: 1 JID: a4accbe6-6163-4941-858c-46883aaeaed8 - coral calls made during job: 256, coral calls rate: 55.22 calls/second
  2017-09-14T03:08:58.385930+00:00 hipchat224 scissortail-worker:5421:INFO GID: None JID: f93fbabc-f87b-47b4-be3f-b84bc41796f8 - Notification job starting: Settings {'recipients': ['hipchat@atlassian.com']$

Import completion through /var/log/hipchat/hcapp.log:

2017-06-14T02:12:57.149226+00:00 ip-10-0-140-192 scissortail-worker:12656:INFO GID: 1 JID: ba5515e8-1468-4406-9033-9509cc1d0684 - Finished import task, found 2 errors and warnings
2017-06-14T02:12:57.149334+00:00 ip-10-0-140-192 scissortail-worker:12656:ERROR GID: 1 JID: ba5515e8-1468-4406-9033-9509cc1d0684 - A user you're importing already exists. The user 1 is now mapped to 1.
2017-06-14T02:12:57.171950+00:00 ip-10-0-140-192 scissortail-worker:12656:INFO GID: 1 JID: ba5515e8-1468-4406-9033-9509cc1d0684 - Finished job ba5515e8-1468-4406-9033-9509cc1d0684 successfully
2017-06-14T02:12:57.185799+00:00 ip-10-0-140-192 scissortail-worker:12671:INFO GID: None JID: b027460a-dc7b-45ac-8bff-9c368783763a - Notification job starting: Settings {'recipients': ['hipchat@example.com'], 'type': 'email'}

Warnings/errors and what do they mean

Your HipChat Server export job failed with the following errors and warnings: Something went wrong. Try again in a few minutes.

This normally means that a process died and didn't recover, the server is out of disk, or the server hit into an exporter bug. The coral.log or the hcapp.log log should have more information about this. When ElasticSearch (stored chat messages and files) doesn't work well, a lot of things could happen e.g. messages are not stored properly, exports don't work. It's good to double check server disk space (df -h) because insufficient disk space is a contributing factor. Hipchat server also doesn't automatically delete the exports, so if the user did a few of those it might be filling up the disk space and cause the problem.

User 110 had the following bad or non-unique values: mention_name, replaced them with defaults: {'timezone': 'America/New_York', 'is_deleted': True, 'title': 'Manager', 'is_guest': False, 'created': '2015-01-27T23:50:41+00:00', 'mention_name': 'johndoe', 'email': 'johndoe@example.com', 'name': 'John Doe', 'is_group_admin': False}

This means the server has a user with the same mention_name as the one in the export. It could also be that there was another user in the server, then it was deleted, and a new user had the same mention_name. This error with bad mention_name could also be that the data came that way from a cloud export.

User 895 had the following bad or non-unique values: name, replaced them with defaults: {'is_deleted': False, 'timezone': 'America/New_York', 'email': 'johndoe@atlassian.com', 'mention_name': 'jdoe', 'created': '2015-08-25T19:29:00+00:00', 'title': 'Senior Director', 'is_group_admin': False, 'is_guest': False, 'name': 'John [Imported]'}

In this particular case, what happened is that the user coming from the export had a name with no last name. You can see this message in the hcapp.log - 'User 895 had the following bad or non-unique values: name, replaced them with defaults: {'name': 'John [Imported]'. It's a requirement to have a name with at least two elements (First Last). So, if the user doesn't have it, we just put [Imported] as the last name, so the APIs don't break.

A room you're importing already exists. The room 1 is now mapped to 1.

This means there's a conflict with a room which also indicates that the server already has existing data and possibly with a room with the same name. In this case, the existing room will get the messages that come from the importer.

Issues found when importing the private chat history of user 328:
Failed to import file: Job exceeded maximum timeout value (14400 seconds).
Issues found when importing the private chat history of user 330:
Failed to import file: 413 <Response [413]>

413 means Request Entity Too Large. The server is refusing to process a request because the request entity is larger than the server is willing or able to process. The server may close the connection to prevent the request from continuing on. The importer is supposed to be able to upload the same size of files as it downloaded previously, but something went wrong here. Unfortunately, the side effect to this is that some 1-1 chat history (for the mentioned user IDs) were not successfully imported to a new server. No message should be left behind and we have fixed this in Hipchat Server v1.3 build 1.3.7.

Issues found when importing the private chat history of user 735:
Failed to import file: 400 {'error': {'message': 'File is too large', 'code': 400, 'type': 'Bad Request'}}
Failed to import file: 413 <Response [413]>

This issue is related to the '413' error above and we have fixed this in Hipchat Server v1.3 build 1.3.7.

Issues found when importing the chat history of room 1248:
Import file /file_store/scissortail/tmp9o0e6gnr/rooms/1147/files/Ky8iuc8ycufyB8n/Files_-%25D8%25B3%25D9%2586%25D8%25BA%25D8%25A7%25D9%2581%25D9%2588%25D8%25B1%25D8%25A9%25D9%2585%25D8%25B7%25D8%25A7%25D8%25B1_%25D8%25B4%25D8%25A7%25D9%2586%25D8%25BA%25D9%258A_SIN__%25D8%25B3%25D9%2586%25D8%25BA%25D8%25A7%25D9%2581%25D9%2588%25D8%25B1%25D8%25A9.png is missing from the package

This is a known issue as we don't export files with very long names and non-ascii characters. The only impact is that the specific file is not imported. We have improved this in Hipchat Server v1.3 build 1.3.8. 

Couldn't download https://hipchat.atlassian.com/files/1/330/8GgNa3l9lGA0Qdz/hipchat.csv.gz into /file_store/scissortail/tmpoy2_d0xk/data/users/files/8GgNa3l9lGA0Qdz/hipchat.csv.gz: Couldn't download https://hipchat.atlassian.com/files/1/330/8GgNa3l9lGA0Qdz/hipchat.csv.gz into /file_store/scissortail/tmpoy2_d0xk/data/users/files/8GgNa3l9lGA0Qdz/hipchat.csv.gz: ('Received response with content-encoding: gzip, but failed to decode it.', error('Error -3 while decompressing data: incorrect header check',))

Export is missing some files with .gz file extension with error 'Received response with content-encoding: gzip, but failed to decode it'. We have fixed this in Hipchat Server v1.3 build 1.3.7.

Issues found when importing the private chat history of user 154:
Failed to import file: Job exceeded maximum timeout value (14400 seconds).
Failed to import file: 405 {'error':
Unknown macro: {'message'}

This is a known issue. When the job reaches 14,400 seconds it logs that error and interrupts the current action. The only impact is the specific file is not imported. We have fixed this in Hipchat Server v1.3 build 1.3.7.

Issues found when importing the private chat history of user 136:
Failed to import file: 405 {'error': {'message': '<p>The method is not allowed for the requested URL.</p>', 'code': 405, 'type': 'Method Not Allowed'}}
Failed to import file: 500 {'error':

The issue is we tried to upload messages as if they had files, even when they don't. We have fixed this in Hipchat Server v1.3 build 1.3.7.

Couldn’t decrypt the archive file. Make sure you’re entering the correct password.

Although this clearly states it's an issue with the password, it will pop-up if you're importing using the full archive URL and the download of *.aes files is not allowed on your network. The .aes extension should be whitelisted. 

Known Limitations

• Deleted rooms are not included in the export.
• User uploaded emoticons are not included in the export.
• The importer doesn't do message de-duplication, so multiple imports into the same server will result in a server with the same messages multiple times.
• At the moment, we don't support incremental export/import.