Removing invalid characters from XML backups

Usage FAQ

On this page

Still need help?

The Atlassian Community is here for you.

Ask the community

This applies to Jira 6.4.x and above.

JIRA 3.1 and above should not suffer from this problem unless migrating to postgreSQL from another database such as MySQL. Invalid characters otherwise are automatically stripped from imported data.

In older versions of JIRA it was possible to cut & paste text containing control characters into JIRA issue fields. This causes problems, because JIRA's backup format is XML, and XML does not allow for the storage of most control characters.When XML containing control characters is imported into JIRA, the import fails with an error:

To fix this, the control characters will need to be removed from the JIRA backup file. This can be done with the following:

  1. Download atlassian-xml-cleaner-0.1.jar to the same folder as entities.xml
  2. Open a command prompt and locate the XML or ZIP backup file on your computer, ensuring that it is extracted if it's within a ZIP file. In this example, we will use entities.xml.
  3. Run the application with the below:

    java -jar atlassian-xml-cleaner-0.1.jar entities.xml > entities-clean.xml

    This will create a copy of entities.xml as entities-clean.xml with the invalid characters replaced with a replacement character

  4. Copy the entities-clean.xml file into another directory, rename it back to entities.xml and create a new ZIP with the newly created entities.xml file and the activeobjects.xml file.

    Make sure there's no subfolders inside the zip archive, and that the files are named exacly entities.xml and activeobjects.xml. Otherwise, Jira will throw an error as explained in https://confluence.atlassian.com/jirakb/unable-to-find-jira-backup-entities-xml-inside-of-zip-file-error-779160740.html

  5. Import the new ZIP file, ensuring that it contains both XML files.

If you are seeing an error specifically with 0xffff as the affected character, please use this perl command to fix the file:

perl -i -pe 's/\xef\xbf\xbf//g' entities.xml

And if experiencing the error with 0xfffe, use the below perl command:

perl -i -pe 's/\xef\xbf\xbe//g' entities.xml

And in case you are running Windows and the above Perl command doesn't work, here's a Power Shell script to fix the problem:

$yourfile = "PATH_TO_THE_XML\entities.xml"
$outputfile = "PATH_TO_SAVE_NEW_XML\entities_clean.xml"
get-content -path $yourfile | out-file $outputfile -encoding utf8


The problem could be caused due to bug  JRA-62240 - Getting issue details... STATUS

If you are having problem with handling 4-byte characters due to MySQL limitation with JIRA, then please refer to XML backup from MySQL fails to import to Jira server.
Last modified on Jun 17, 2022

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.