Backing up and restoring Crucible data
Crucible data can be backed up from the admin interface or command line. This page contains the command syntax, options and the required procedure to backup and restore your Crucible instance.
If you are migrating to a different machine, please follow the instructions on our Migrating Fisheye Between Servers page.
Backing up using admin interface
- Navigate to the Crucible Admin area (click the Administration link in the footer of any Crucible page).
- In the left navigation bar, click Backup.
- On the Backup screen, you can edit the name of the backup file that will be generated.
- Choose the components, described in the table below, that should be included in the backup file.
Note that the back up will always include the configuration data (config.xml
), your license file and the Fisheye user data.
Repository and application caches contain temporary data stored from repository scans and library caches that improve startup time. Both will be recreated automatically by re-scanning the source repositories, so the backup files can be significantly reduced by excluding these (if the cost of re-scanning is acceptable). - Click Create Backup Now .
Component | Purpose |
---|---|
ActiveObjects | Data that is stored by plugins |
Repository and application caches | The cache contains data that reflects the state of Crucible's repositories. Without it, Crucible must re-scan its repositories after a backup is restored. The cache also contains OSGI library data that increases startup time. These too can be excluded and will be generated automatically when the application is started. |
Plugins | Plugins are 3rd-party extensions that you may have installed, and configuration for all plugins (this includes configuration for Crucible's set of standard plugins). |
SQL Database | Refers to the SQL content database (used by both Fisheye and Crucible and containing all user profile data, reviews and their comments). |
Web templates | In this context, these are custom freemarker templates that you or your users have created. They live in |
Uploaded files | In this context, uploads refers to files which are added to Crucible via the web interface (such as patch file reviews). It also includes each repository-backed file that went under review, when Crucible is configured to make a local copy of every reviewed file. |
Using the command line to back up
Your Crucible instance must be running during the backup.
- Open a command line interface on the Crucible server computer.
- Navigate to
<Crucible installation dir>/bin/
- Run the backup command on the command line using any desired options (described in the table below). By default, all backup components, except repository cache data, is included in the backup.
The backup is created as a new Zip archive file and placed in the <Crucible installation dir>
/ backup/
directory.
Note that if your Crucible instance uses a custom FISHEYE_INST
directory; make sure the environment variable is properly set when running the backup command.
Option | Switch | Default setting |
---|---|---|
Quiet mode | -q, --quiet | No |
Output filename | -f, --file |
|
Compression level | -c, --compression | Yes (6) |
Anonymize | -a, --anonymize | No |
Cache Backup | --cache | No. By default, the cache data is excluded from backups. |
Debug | --debug | Print extra information to the debug log. |
Advanced backup command line settings
In some cases it might be preferable to only backup a limited set of items. This could be useful when your instance uses an external database such as MySQL or PostgreSQL and your DBA has already configured automatic backups in the database. The commands below allow this.
Option | Switch | Description | Default |
---|---|---|---|
Exclude Plugins | --no-plugins | Excludes plugins from the backup. | No. By default, plugins are included in every backup. |
Exclude Templates | --no-templates | Excludes templates from the backup. | No. By default, templates are included in every backup. |
Exclude Uploads | --no-uploads | Excludes uploaded files (such as patch reviews, stored in Crucible's internal database) from the backup. | No. By default, uploads are included in every backup. |
Exclude SQL Database | --no-sql | Excludes the SQL content database used by both Fisheye and Crucible. | No. By default, this data is included in every backup. |
Show help | -h, --help | Shows inline help on the command line. | No |
Backup command line examples
These examples are for use in a Linux-like operating system. When using these commands on Windows, use the filename fisheyectl.bat
and use the correct slashes. Run the command from
<Crucible installation dir>/bin/
.
The basic syntax of the backup command is as follows:
$ ./fisheyectl.sh backup [OPTIONS]
Backing up with compression of 9, quiet mode and setting an output location
$ ./fisheyectl.sh backup --compression 9 -q -f /application_backups/fisheye/20090215.zip
Backup including cache data (also includes all default components)
$ ./fisheyectl.sh backup --cache
Known limitations
Archives Larger Than 4GB
Be aware of the fact that some file systems (like FAT32) have trouble with files larger than 4GB. If you still want to archive everything and end up with an archive that is too large, consider creating separate backups for the Crucible cache and uploaded files respectively.
Scheduling Crucible backups
To set a schedule for automatic backups:
- Go to the Admin area and click Backup (under 'System Settings').
- Click Manage Scheduled Backups at the bottom of the page.
- Click Edit, set the desired options, then click Save.
The options for scheduled backups are detailed in the table below.
Option name | Description | Allowed Values |
---|---|---|
Disable Scheduled Backups | Stops regular backups from taking place. | On (disabled) or Off (enabled) |
Backup path | The path where the backup .zip file will be stored. | Any system or network path that Fisheye or Crucible can access. This cannot be changed using the Crucible interface. Please edit your <backup> Save changes, and restart your Crucible instance. |
Backup file prefix | Characters that will be added to the beginning of the backup file name. | Any string of characters that can be used as part of a filename on the local operating system. |
Backup file date pattern | Sets a date for the next (or initial) backup to take place. | Any valid date in the format |
Backup frequency | Sets how often the backup will take place. | Can be set to 'every day', 'every Sunday', 'Monday to Friday' and 'first day of the month'. |
Backup time (HH:mm) | The time when the backup will take place. | Any valid 24-hour time in the format |
Include | Specifies which items must be included in the backups (these components are explained at the top of this page). | As per the options for regular on-demand backup (These components are explained at the top of this page). |
Be aware that scheduled backups can fill up disks unless you regularly move or delete old archives.
Restoring Crucible data
There is currently no way to restore a backup from the web interface because Crucible must be shut down during a data restore.
Restoring a backup will irreversibly overwrite the data of your installation with the data from the backup archive.
If you made a backup from production which connected to an external database, and restore this backup to a test server without specifying another database to restore to, you will drop and restore to your production database. Thus when restoring to a test server, always ensure you specify the correct database to restore to (or restore to an in-built database).
Using the command line to restore Crucible data
- Install Crucible into a new, empty directory (this must be the same version that the backup was created from, or later).
Note that you cannot restore data into versions of Crucible which are older than the version that created the backup. - Make sure the Crucible instance is not running.
- Open a command line interface on the Crucible server computer.
- Run the restore command on the command line with the desired options.
- The specified elements will be restored.
- Start the Crucible instance.
- When using Fisheye integrated with Crucible, you will need to re-index your repositories after restoring data, unless the backup archive was created with the
--cache
option.
Command line restore options
By default, the restore process will restore all items found in the backup archive (so if you included the caches using the --cache
option, these will automatically be restored). However, you can specify a partial restore, by explicitly specifying the item names on the command line.
If you are using an external database (as opposed to the default HSQL database), make sure the JDBC driver file is present in the FISHEYE_INST/lib
directory when running restore.
Furthermore, if you are restoring to a new Crucible instance and install directory, and select --dbtype of mysql, you must download the JDBC driver and accept the license agreement before proceeding with the restore.
The options available for use with the restore command are listed in the following table:
Option | Switch | Description |
---|---|---|
Choose file to restore from | -f PATH/FILENAME, --file PATH/FILENAME | (Required) Restore the backup from PATH/FILENAME. |
Repository and application caches | --cache | Restore the repository cache backup. |
Plugins | --plugins | Restore 3rd-party plugins and their configuration data. |
Web templates | --templates | Restore freemarker templates from the backup (the restored instance will use the built-in templates). |
Uploaded files | --uploads | Restore uploads (e.g. patch files uploaded into Crucible and contents of files under review). This item only applies when using Crucible with Fisheye. |
SQL Database | --sql | Restore the SQL database containing user profiles, reviews and review comments. |
ActiveObjects | --ao | Restore ActiveObjects data stored by plugins. |
List backup contents | -l, --list | List the contents of the backup file, and exit. |
Set database type | -t, --dbtype | SQL database type (hsql, |
Set JDBC URL | -j, --jdbcurl | JDBC URL of the SQL database. Only required when restoring to a database location different to that used at used at back up time (not applicable for hsql). |
Set JDBC username | -u, --username | JDBC username of the SQL database. Only required when restoring to a database location different to that used at used at back up time (not applicable for |
JDBC password | -p, --password | JDBC password of the SQL database. Only required when restoring to a database location different to that used at used at back up time (not applicable for |
JDBC class | -d, --driver | Optionally, specify the JDBC driver class name needed to access the SQL database. Only required when restoring to a database location different to that used at used at back up time and when using a different JDBC driver than the standard driver associated with the database specified though |
JDBC driver source | -s, --driver-source | Optionally, specify the JDBC driver file. The default ('bundled') is to use the bundled JDBC driver file; 'user' = use the driver from the FISHEYE_INST/lib directory. |
Suppress output | -q, --quiet | Suppress the output messages from the restore program on the command line. |
Overwrite the existing DB | --force | Overwrite content of the database without warning. The --force flag checks your current database versions and will let you overwrite the existing database only if it matches the schema version. If you want to downgrade Crucible, you must use an empty database. |
Don't batch SQL inserts | --no-batch-sql | Do not batch SQL inserts. This can be useful when diagnosing errors. |
Display the help | -h, --help | Display the help, and exit. |
These examples are for use in a Linux-like operating system. When using these commands on Windows, use the filename fisheyectl.bat
and use the correct slashes. Run the command from <Crucible installation dir>/bin/
.
The basic syntax of the restore command is as follows:
$ ./fisheyectl.sh restore -f /path/to/backup_2009-10-02_1138.zip [OPTIONS]
To see inline help for all backup options, run the following command in
<Crucible installation dir>/bin/
$ ./fisheyectl.sh restore --help
Restoring a backup with cache data (also restores all default components):
$ ./fisheyectl.sh restore --cache
Restores a Fisheye/Crucible backup instance.
If you are using an external database (as opposed to the default built-in database), make sure the JDBC driver file is present in the FISHEYE_INST/lib
directory when running restore.
Restoring a backup of SQL Server database
Crucible 4.7.1 (and higher) uses new scheme for SQL Server JDBC URL. In order to restore a backup generated from a version prior to 4.7.1, database connection properties (database type, URL, username and password) have to be set explicitly in a command line. Otherwise, restore command will fail as config.xml
in the backup file contains an invalid URL scheme.
Change the scheme in the following manner:
from jdbc:jtds:sqlserver://
to jdbc:sqlserver://
To use any custom connection parameters, such as domain-based authentication, you have to change them to the ones supported by Microsoft JDBC (see properties supported by jTDS and by Microsoft JDBC).
Example for Linux operating system:
$ ./fisheyectl.sh restore \
--username 'john' \
--password 'smith' \
--jdbcurl 'jdbc:sqlserver://localhost:1433;databaseName=crucible;' \
--dbtype 'sqlserver2012' \
--file '/path/to/backup_2009-10-02_1138.zip' \
--force
Migrating backup data
When the process restores a SQL database, it looks at the configuration data (config.xml
) included in the backup archive to learn which database product was used and how to connect to it. When Crucible uses the built-in HSQLDB database (which is the default), the restored instance will also use that.
However, when the restored instance will use a different database than the backed up instance (for instance, HSQLDB was used at the time the backup was created, but it needs to be restored on MySQL ), you should use the command line options to point the process to the new database which must exist before running the restore. The restore will only populate the database, it will not create the database.
Command line example: migrating backup data to MySQL
This example is for use in a Linux-like operating system. When using these commands on Windows, use the filename fisheyectl.bat
and use the correct slashes. Run the command from <Crucible installation dir>
/ bin/
We recommend to put values in quotes to avoid evaluation of special characters by the shell (e.g. ';' in JDBC URL may be treated as command separator on Linux).
Use the '--force' option also for an empty database (otherwise restore
command won't proceed).
Restoring to a Fisheye instance that uses a different database (ensure the MySQL driver jar file is present in the
FISHEYE_INST/lib
directory)
$ ./fisheyectl.sh restore \
--username 'john' \
--password 'smith' \
--jdbcurl 'jdbc:mysql://localhost:3306/crucible' \
--dbtype 'mysql' \
--file '/path/to/backup_2009-10-02_1138.zip' \
--force