Backing up and restoring FishEye data

FishEye data can be backed up from the admin interface or command line. This page contains the command syntax, options and the required procedure to back up and restore your FishEye instance.

If you are migrating to a different machine, please follow the instructions on our Migrating FishEye Between Servers page.

Using the admin interface to back up

  1. Navigate to the FishEye Admin area (click Administration in the footer of any FishEye page).
  2. Click Backup (under 'System Settings').
  3. On the Backup screen, you can edit the name of the backup file that will be generated.
  4. Choose the components, described in the table below, that should be included in the backup file.
    Note that the back up will always include the configuration data (config.xml), your license file and the FishEye user data.
    (info) Repository and application caches contain temporary data stored from repository scans and library caches that improve startup time. Both will be recreated automatically by re-scanning the source repositories, so the backup files can be significantly reduced by excluding these (if the cost of re-scanning is acceptable).
  5. Click Create Backup Now.

Component

Purpose

ActiveObjects Data that is stored by plugins

Repository and application caches

The cache contains data that reflects the state of FishEye's repositories. Without it, FishEye must re-scan its repositories after a backup is restored. The cache also contains OSGI library data that increases startup time. These too can be excluded and will be generated automatically when the application is started.

Plugins

Plugins are 3rd-party extensions that you may have installed, and configuration for all plugins (this includes configuration for Crucible's set of standard plugins).

SQL Database

Refers to the SQL content database (used by both FishEye and Crucible and containing all user profile data, reviews and their comments).

Web templates

In this context, these are custom freemarker templates that you or your users have created. They live in FISHEYE_INST/template.

Uploaded files

In this context, uploads refers to files which are added to Crucible via the web interface (such as patch file reviews). It also includes each repository-backed file that went under review, when Crucible is configured to make a local copy of every reviewed file.


Screenshot: The FishEye Backup screen

Using the command line to back up

(info) Your FishEye instance must be running during the back up.

  1. Open a terminal or command prompt on the FishEye server computer.
  2. Navigate to the <FishEye home directory>/bin/ directory.
  3. Run the backup command on the command line using any desired options (described in the table below). By default, all backup components, except repository cache data, is included in the backup.

The backup is created as a new Zip archive file and placed in the FISHEYE_INST/backup/ directory.
(info) Note that if your FishEye instance uses a custom FISHEYE_INST directory, make sure the environment variable is properly set when running the backup command.

Option

Switch

Default setting

Quiet mode

-q, --quiet

No

Output filename

-f, --file

FISHEYE_INST/backup/ is the default directory.

Compression level

-c, --compression

Yes (6)

Anonymise

-a, --anonymise

No

Cache Backup

--cache

No. By default, the cache data is excluded from backups.

 

Advanced backup command line settings

In some cases it might be preferable to only back up a limited set of items. This could be useful when your instance uses an external database such as MySQL or PostgreSQL and your DBA has already configured automatic back ups in the database. The commands below allow this.

Option

Switch

Description

Default

Exclude Plugins

--no-plugins

Excludes plugins from the backup.

No. By default, plugins are included in every backup.

Exclude Templates

--no-templates

Excludes templates from the backup.

No. By default, templates are included in every backup.

Exclude Uploads

--no-uploads

Excludes uploaded files (such as patch reviews, stored in Crucible's internal database) from the backup. (info) This item only applies when using Crucible with FishEye.

No. By default, uploads are included in every backup.

Exclude SQL Database

--no-sql

Excludes the SQL content database used by both FishEye and Crucible.

No. By default, this data is included in every backup.

Backup command line examples

These examples are for use in a Linux-like operating system. When using these commands on Windows, use the filename fisheyectl.bat and use the correct slashes. Run the command from the <FishEye home directory>/bin/ directory.

The basic syntax of the backup command is as follows:

To see inline help for all backup options, run the following command in the <FishEye home directory>/bin/ directory:

Backing up with compression of 9, quiet mode and setting an output location:

Back up including cache data (also includes all default components):

Known limitations

Please note that the limitations below are common to any Java-based backup tool.

Archives Containing Over 65535 Files
Versions of Java earlier than v1.6 (b25) are incapable of handling zip files that contain more than 65,535 files. The solution for this problem is to either upgrade to a version of Java later than v1.6 (b25), or ensure that the archive does not exceed the threshold (contains less than 65,535 files). The FishEye cache (not included in backups by default) can be a contributor of many small files. Hence, exclude the cache from backups if this is likely to be a concern.

Archives Larger Than 4GB
Java has trouble reading and writing zip files that are larger than 4GB. As of release 1.5 Java appears capable of reliably creating archives that are over 4GB, but remains unable to extract them. For details see Sun's bug report. Also be aware of the fact that some file systems (including FAT32) have trouble with files larger than 4GB.

As a workaround, make sure you do not create archives that are larger than 4GB. The FishEye cache (not included in backups by default) can be a contributor of a lot of small files (although these tend to compress very well). If you still want to archive everything and end up with an achive that is too large, consider creating separate backups for the FishEye cache and uploaded files respectively.

Scheduling FishEye back ups

To set a schedule for automatic back ups:

  1. Go to the Admin area and click Backup (under 'System Settings').
  2. Click Manage Scheduled Backups at the bottom of the page.
  3. Click Edit, set the desired options, then click Save.

The options for scheduled back ups are detailed in the table below:

Option name

Description

Allowed Values

Disable Scheduled Backups

Stops regular back ups from taking place.

On (disabled) or Off (enabled)

Backup path

The path where the backup .zip file will be stored.

Any system or network path that FishEye or Crucible can access. This cannot be changed using the FishEye interface. Please edit your config.xml, and add the <path> attribute according to this example:

<backup>
<schedule enabled="true">
<path>path_to_backup_location</path>
<prefix>fisheye_backup_</prefix>
<date-pattern>yyyy_MM_dd</date-pattern>
<frequency>WEEKLY</frequency>
<time>07:00</time>
<items> <item name="sql"/>
<item name="templates"/>
<item name="cache"/>
<item name="ao"/>
<item name="plugins"/>
<item name="uploads"/>
</items>
</schedule>
</backup>

Save changes, and restart your FishEye instance.

Backup file prefix

Characters that will be added to the beginning of the backup file name.

Any string of characters that can be used as part of a filename on the local operating system.

Backup file date pattern

Sets a date for the next (or initial) back up to take place.

Any valid date in the format yyyy_MM_dd (year, month, day of the month).

Backup frequency

Sets how often the back up will take place.

Can be set to every day, every Sunday, Monday to Friday and first day of the month.

Backup time (HH:mm)

The time when the back up will take place.

Any valid 24-hour time in the format HH:mm (hours, minutes).

Include

Specifies which items must be included in the backups (these components are explained at the top of this page).

As per the options for regular on-demand back up (These components are explained at the top of this page).

Screenshot: Scheduling back ups in FishEye

(info) Be aware that scheduled back ups can fill up disks unless you regularly move or delete old archives.

Restoring FishEye data

Using the command line to restore FishEye data

There is currently no way to restore a backup from the admin interface because FishEye must be shut down during a data restore.

(warning) Restoring a backup will irreversibly overwrite the data of your installation with the data from the backup archive. Note that you cannot restore data into versions of FishEye which are older than the version that created the backup.

  1. Install FishEye into a new, empty directory (this must be the same version that the backup was created from, or later).
  2. Make sure the FishEye instance is not running.
  3. Open a command line interface on the FishEye server computer.
  4. Run the restore command on the command line with any desired options.
  5. The specified elements will be restored.
  6. Start the FishEye instance.
  7. When using FishEye integrated with Crucible, you will need to re-index your repositories after restoring data, unless the backup archive was created with the --cache option.

Restore command line options

By default, the restore process will restore all items found in the backup archive (so if you included the caches using the --cache option, these will automatically be restored). However, you can specify a partial restore, by explicitly specifying the item names on the command line.

If you are using an external database (as opposed to the default HSQL database), make sure the JDBC driver file is present in the FISHEYE_INST/lib directory when running restore.

Furthermore, if you are restoring to a new FishEye instance and home directory, and select --dbtype of mysql, you must download the JDBC driver and accept the license agreement before proceeding with the restore.

The options available for use with the restore command are listed in the following table:

Option

Switch

Description

Choose file to restore from

-f PATH/FILENAME,

--file PATH/FILENAME

(Required) Restore the backup from PATH/FILENAME.

Repository and application caches

--cache

Restore the repository cache backup.

Plugins

--plugins

Restore 3rd-party plugins and their configuration data.

Web templates

--templates

Restore freemarker templates from the backup (the restored instance will use the built-in templates).

Uploaded files

--uploads

Restore uploads (e.g. patch files uploaded into Crucible and contents of files under review). (info) This item only applies when using Crucible with FishEye.

SQL Database

--sql

Restore the SQL database containing user profiles, reviews and review comments.

ActiveObjects --ao Restore ActiveObjects data stored by plugins.
List backup contents -l, --list List the contents of the backup file, and exit.

Set database type

-t, --dbtype

SQL database type (built-in, mysql, oracle, postgresql, sqlserver2005, sqlserver2008, sqlserver2012). Only required when restoring to a database location different to that used at back up time.

Set JDBC URL

-j, --jdbcurl

JDBC URL of the SQL database. Only required when restoring to a database location different to that used at used at back up time (not applicable for built-in).

Set JDBC username

-u, --username

JDBC username of the SQL database. Only required when restoring to a database location different to that used at used at back up time (not applicable for built-in).

JDBC password

-p, --password

JDBC password of the SQL database. Only required when restoring to a database location different to that used at used at back up time (not applicable for built-in).

JDBC class

-d, --driver

Optionally, specify the JDBC driver class name needed to access the SQL database. Only required when restoring to a database location different to that used at used at back up time and when using a different JDBC driver than the standard driver associated with the database specified though --dbtype. (Not applicable for built-in.)

JDBC driver source -s, --driver-source Optionally, specify the JDBC driver file. The default ('bundled') is to use the bundled JDBC driver file; 'user' = use the driver from the FISHEYE_INST/lib directory.

Suppress output

-q, --quiet

Suppress the output messages from the restore program on the command line.

Overwrite the existing DB  --force Overwrite the existing database, without warning.
Don't batch SQL inserts --no-batch-sql Do not batch SQL inserts. This can be useful when diagnosing errors.
Display the help -h, --help Display the help, and exit.

(info) These examples are for use in a Linux-like operating system. When using these commands on Windows, use the filename fisheyectl.bat and use the correct slashes. Run the command from the <FishEye home directory>/bin/ directory.

The basic syntax of the restore command is as follows:

To see inline help for all backup options, run the following command in the <FishEye home directory>/bin/ directory:

Restoring a backup with cache data (also restores all default components):

Migrating backup data

When the process restores a SQL database, it looks at the configuration data (config.xml) included in the backup archive to learn which database product was used and how to connect to it. When FishEye uses the built-in HSQLDB database (which is the default), the restored instance will also use that.
However, when the restored instance will use a different database than the backed up instance (for instance, HSQLDB was used at the time the backup was created, but it needs to be restored on MySQL ), you should use the command line options to point the process to the new database which must exist before running the restore.  The restore will only populate the database, it will not create the database.

Command line example: migrating backup data to MySQL

(info) These examples are for use in a Linux-like operating system. When using these commands on Windows, use the filename fisheyectl.bat and use the correct slashes. Run the command from the <FishEye home directory>/bin/ directory.

Restoring to a FishEye instance that uses a different database (ensure the MySQL driver jar file is present in the FISHEYE_INST/lib directory)

Was this helpful?

Thanks for your feedback!

9 Archived comments

  1. User avatar

    Aivo Kuhlberg

    ensure the MySQL driver jar file is present in the FISHEYE_INST/lib directory

    Should it be <Fisheye home directory> instead?

    09 Apr 2013
    1. User avatar

      Paul Watson [Atlassian]

      Hi Aivo,

      The documentation is correct (wink). See also Migrating to MySQL.

      The MySQL library (including the JDBC driver) will work in both locations, but is now kept in the instance directory so that it is not over-written when FishEye is upgraded, as it would be if it was in the home (install) directory. This allows you to maintain that library yourself. Note that the MySQL JDBC driver is no longer shipped with FishEye due to licensing reasons.

      regards, Paul

      10 Apr 2013
  2. User avatar

    David Chou [Intuit]

    I also want to share that if you are including 'Repository and application caches' in your backup, you Fisheye instance will not be accessible.  Users will see a maintenance page when visiting Fisheye.  

    Furthermore, if you have JIRA/Fisheye integration, while the backup with cache is running the source tab in JIRA will show an error, which users may not comprehend.  I've put in an improvement request to make it a friendlier error  FE-4611 - Display nicer error in JIRA Source tab when Fisheye Repo/App Cache is being backed up Open

    Note: Probably not going to run this often, but noting this for others. 

    11 Apr 2013
    1. User avatar

      Emi

      This is true regardless of the "Repository and application caches" is included or not, the instance is not accessible during the backup process. 

      28 Oct 2014
  3. User avatar

    Rz Toopr

    Is there a way to merge two Fisheye Projects together?

    We have stuff for 1 client in Fisheye1

    Different stuff for another client in Fisheye2

    Can I move the stuff from Fisheye2 into Fisheye1 without getting rid of the existing data in Fisheye1? (Be able to access both brances through Fisheye 1)

    14 Mar 2014
    1. User avatar

      Daniel Rohan [Atlassian]

      Hello Michael,

      You can move the FishEye index data from one instance to another but I have to mention that you cannot move Crucible review data between instances.  In order to successfully move index data you simply need to make sure that the repository data in the configuration file is exactly the same between instances. The second item you will need is the actual index data. For the first item, find the repo data within the $FISHEYE_INST/config.xml in instance A and add it to the config.xml file of instance B – make sure instance B is not running during this edit. Second, move over the $FISHEYE_INST/var/cache/<repo_name_here> folder from instance A to instance B. The configuration file change and the actual index folder are all you need to see the FishEye data of one instance in the other instance. If you need further assistance, please open a support request at https://support.atlassian.com.

      14 Mar 2014
  4. User avatar

    Ryan Goodwin

    It sort of blows my mind that Atlassian uses the install and home directory in the EXACT OPPOSITE MANNER for Fecru as it does for the rest of it's applications. Fisheye home directory = installation files? Come on ...

    13 Jun 2014
    1. User avatar

      Richard Stephens [Atlassian]

      Hi Ryan,

      Unfortunately the FISHEYE_INST and FISHEYE_HOME directories have been named that way since before Atlassian acquired Cenqua and the FishEye/Crucible product

      Kind Regards,
      Richard Stephens 

      19 Jun 2014
  5. User avatar

    Fahad

    Hi Team,

    The restoring scripts is confusing, we tried to restore the prod backup to QA, but it imported the details to Pord DB only since the DB URL in the export file is pointing to the prod DB only.

    Please mention that not only if the user wants to restore the data to a different type of database, ie from HSQLDB to MySQL also if the user is trying to restore from a MySQL DB to a different MySQL DB, the user needs to specify the DB details in the command line options.

    We had lost some data due to this. Please update the documentation so that any other users wont face this issue.

     

    Thanks

    Fahad 

    23 Jun 2015
Powered by Confluence and Scroll Viewport