Migration guide for Git internally managed repositories
Internally managed Git repositories were deprecated in Fisheye and Crucible 2.8, and support for these was removed for the Fisheye and Crucible 3.2 releases. See End of Support Announcement for Internally Managed Repositories.
This page provides migration strategies and procedures for those who have internally managed Git repositories and wish to continue using those after upgrading to Fisheye/Crucible 3.2.
What does this mean for users?
If you used managed repositories to share your code with others, you will no longer be able to push or pull changes from those repositories. Your Fisheye administrator will need to arrange access, or perhaps migrate those repositories and provide you with the new connection details so hat you can update the remotes in your local clones.
What does this mean for sysadmins?
As an administrator, you will need to decide whether you want to remove your internally managed repositories or convert them to standard repositories. When you upgrade to Fisheye/Crucible 3.2, conversion will happen automatically and the repositories will be fully browsable and searchable, but any Git interaction with them through Fisheye will be no longer possible. This means that users will not be able to share their code anymore, until you arrange that. There are a few options available for arranging Git access:
- Migrate the repository to different a Git hosting solution, for example Bitbucket Server or Bitbucket Cloud.
- Migrate the repository to another server that can be accessed from Fisheye, and by all users.
- Leave the repository in the same location.
For all these options, permissions will need to be arranged so that all the expected users can access the repositories and share their code. Detailed guides for those options are described below.
Note that, as of 21 September 2015, Stash is now known as Bitbucket Server.
On this page:
Related pages:
How to check if you have managed repositories
If you using a Fisheye/Crucible version prior to 3.2 you can check if you have managed repositories by going to the admin area. In the Repositories list, the icons at the left show whether the repository is internally managed or not (a badged icon indicates an internally managed repository):
Alternatively, you can use the REST service http://fecru-host:port/CONTEXT/rest-service-fe/managed-repositories-v1/git – it will list all managed repositories.
Preparing to upgrade to Fisheye 3.2
Before upgrading, you should check if you have any internally managed Git repositories and decide whether you want to keep using those repositories or not. If not, you may simply delete them.
If you want to continue to use them, you may need to migrate them elsewhere and you will need to arrange access rights, so that users of those repositories will be able to share their code (that is, to push and pull to and from the repositories).
It is recommended that you upgrade your Fisheye instance first, so as to have the repositories converted to standard ones, and to be able to change the repository location without the need to reindex it.
Upgrading to Fisheye 3.2
Follow the standard FishEye upgrade guide and Crucible upgrade guide.
When Fisheye/Crucible 3.2 is restarted and detects one or more managed repositories, it will stop the upgrade (before making any changes to your data) and will show the following warning in the log:
If you are happy with having the repositories converted, add the -Dfisheye.managed.repos.convert=true option to the FISHEYE_OPTS environment variable, as explained in Command-line options.
If you run the upgrade now, with that system property defined, your repositories will be automatically converted:
The conversion task will:
- mark the repository as standard (rather than managed).
- set the polling period to default for managed repositories that had that disabled.
Conversion will not migrate repositories to an external Git hosting solution such as Bitbucket Server. This has to be done manually and is explained below.
Migrating a managed repository from Fisheye 3.2 (and later)
In Fisheye/Crucible version 3.2 and later it is no longer possible to interact with managed repositories using Git. For developers to collaborate with others and push/pull their changes to a repository, the repository has to be made accessible to everybody. There are a few options available:
- Bitbucket Server - Git Repository Management for Enterprise Teams
- Bitbucket - hosting site for the Git and Mercurial distributed version control systems (DVCS)
- and many others.
To migrate a repository:
- Create a repository in the remote destination (for example, Bitbucket Server).
- For Bitbucket, choose Create Repository > I have an existing project to push up
- Configure permissions for the repository in the remote destination so that all expected users have appropriate access.
- Copy (or write down) the URL for the new repository (referred to here as DEST_URL).
- Use the ssh: protocol instead of https: if you want to generate a key pair for authentication purposes. If you attempt to use https: with a key pair you will get the error:
You must not use http(s) with SSH key authentication, use ssh instead
- Use the ssh: protocol instead of https: if you want to generate a key pair for authentication purposes. If you attempt to use https: with a key pair you will get the error:
Run the following commends on the machine that hosts your Fisheye instance:
# REPOSITORY - name of the repository # It is assumed the repo is in the $FISHEYE_INST/managed-repos/ folder, but this location might have been changed in Admin > Global Settings > Server > Managed Repository Settings > Repositories Root. If unsure, check the Location attribute of your repository in Admin / Repositories cd $FISHEYE_INST/managed-repos/REPOSITORY.git git push --all DEST_URL git push --tags DEST_URL chmod -R a-w $FISHEYE_INST/managed-repos/REPOSITORY.git # don't allow changes anymore; main repo is DEST_URL now.- In the Fisheye admin area, click Repositories (under 'Repository Settings') and then the repository name and then SCM Details.
- Update Repository Location to DEST_URL. (Be careful here, pointing to the wrong repository will result in inconsistencies in the Fisheye/Crucible data – the repository will need to be fully reindexed to recover.)
- Configure Git Authentication settings on the same 'SCM Details' page. Generate an SSH key pair and then:
- For Bitbucket Server, upload the public key as a user key (Manage account > SSH keys and click Add key) or as a repository access key.
- For Bitbucket Cloud, upload the public key as a user key (Profile > Manage Account > SSH Keys) or as a repository Deployment Key.
- Click Test Connection to validate access.
- Click Save. You will be prompted to restart the repository. You may need to trigger an incremental scan manually if you have polling disabled.
Your repository will be recloned from the DEST_URL location (this may take a few minutes) but no reindex will be necessary. Once the reclone completes the repository should be ready for use in Fisheye/Crucible.
You will need to notify users of the new location of the repository (for example, in Bitbucket) so that they can update remotes in their local clones.
Migrating a managed repository from Fisheye 3.1 (and earlier)
This section describes how to migrate managed repositories from Fisheye 3.1 to Bitbucket Server or Bitbucket Cloud.
Note that we recommend that you first upgrade to Fisheye 3.2 and then perform the migration, as described in the preceding section.
Restoring a backup of Fisheye 3.1 or earlier that contains managed repositories
The Fisheye backup/restore mechanism ignores internally managed Git repositories, so files under the FISHEYE_INST/managed-repos/ folder had to be manually backed up and restored. Thus, if you attempt to restore a backup of Fisheye 3.1 (or earlier) on Fisheye 3.2 (or later), it will restore the backup keeping configuration of managed repositories in place. But on the first start of the restored instance, Fisheye will check the existence of managed repositories and will issue a warning if any managed repositories are found. As explained in the section above, the system administrator may now use the -Dfisheye.managed.repos.convert=true system property to convert those repositories to standard ones so that they cn then be easily migrated elsewhere.
Guidance for developers who used to interact with managed repos using Git
You used to use Git to push or pull changes to a Fisheye internally managed repository but now you get the following error:
$ git pull
fatal: repository 'http://localhost:6060/foo/git/managedGit1_1.git/' not foundThis is because your repository has been migrated elsewhere and your local clone still points to its old location, on the Fisheye instance. You need to find the new location – you can either ask your Fisheye administrator, or simply find it in Fisheye:
Once you know the repository location you can either update your local clone with the new location or purge your local clone and create a new one from the remote location. Note that the latter action will lose all your local changes. To preserve your local clone and simply update it with the new location, follow this procedure:
cd YOUR_LOCAL_CLONE
# check existing remotes
$ git remote -v
origin http://localhost:6060/foo/git/managedGit1_1.git (fetch)
origin http://localhost:6060/foo/git/managedGit1_1.git (push)
# in this case there is only one remote (origin). It points to the old location (my FishEye instance) so I am going to update it with the new location for the migrated repository:
git remote set-url origin ssh://git@bitbucketserver.yourcompany.com:7999/test/managedGit1_1.git
# now I can pull from/push to that repository.
$ git pull
Already up-to-date.Questions and Answers
Q: Will I lose my managed repository if I upgrade to Fisheye 3.2?
A: No, Fisheye/Crucible won't delete any data. The managed Git repository will stay intact in its original location.
Q: How long will it take to migrate a managed repository elsewhere?
A: The slowest step of the whole operation is cloning the SCM repository to the external location and then allowing Fisheye/Crucible to reclone its local clone from the same repository location. So, it is mostly limited by the available network bandwidth. In our testing, a 61MB Git repository was migrated in approximately 90 seconds, over a fast network link to a Bitbucket server 200ms apart.
Q: Will the upgrade process reindex all my repositories?
A: No. Fisheye/Crucible will need to reclone the repository from the new location once the location changes, but no reindex will be necessary. After all, it is the same repository so all internal Fisheye/Crucible caches can be easily reused.
Q: What kind of SCMs did internally managed repositories support?
A: Only Git was supported by internally managed repositories.
Q: What if I updated the repository location with a path to the wrong repository, during migration? (Imagine you have separate repos A and B, you migrated them to ARepo and BRepo, but then updated repo A with the BRepo repository location by mistake).
A: Unfortunately such an operation will cause inconsistencies in the Fisheye/Crucible data. Once this happens it can be recovered by either restoring this repository from a backup or requesting a full reindex on that repository.