Indexer Paused with "Too many open files" Error
While indexing a repository the indexer stops and the following error can be seen in the logs (example below for an SVN repository):
The indexer has hit the the maximum permitted number of open files in Linux/UNIX operating systems.
The current values for open file descriptors and maximum allowed file descriptors can be determined by turning on debug logging under
Administration >> Global Settings >> Server >> Debug Logging
After debug logging as been turned on, the
Resource Monitor will provide values for
ofd (open file descriptors) and
mfd (maximum file descriptors):
The above output indicates that FishEye can have a maximum of 4096 file descriptors and currently is using 4080 file descriptors.
There are two possible resolutions:
- Optimize the Search Indexes on restart.
More information can be found on the following page: Optimizing the Search Indexes on Restart
Increase the number of allowed file handles per process (and possibly for the entire system).
There are two file handle limits that can affect FishEye/Crucible: the total number of file handles allowed per process and the number of file handles allowed on the entire system.
FishEye/Crucible can hold a high number of open file handles which often exceeds common per-process default limits – usually between 1000 and 4000 open files. If these limits are reached we recommend increasing the limit to 40000. The total number of file handles allowed on the entire system is usually set high enough by default but on an server with many other running applications this too may need to be increased.
Setting the permitted number of file handles per process
To check the number of currently open file handles, run
sudo lsof -u FISHEYE_USER(restricted to the FisheEye running user) or
sudo lsof -p FISHEYE_PROCESS(restricted to the fisheye process).
To check the permitted number of file handles per process, run
To modify the permitted number of file handles per process, edit
/etc/security/limits.confand add (replace FISHEYE_USER with the name of the user that will run FishEye/Crucible or use * to make the change for all users):
Single user: FISHEYE_USER
Setting the permitted number of file handles over the whole system
To check the number of open file handles, run
To check the total number of open file permitted in the whole system, run
To modify the total number of open file permitted in the whole system, edit
/etc/sysctl.confand add the line:
Note: After making these changes the OS will need to be restarted. For further information please see this page.
Was this helpful?
Thanks for your feedback!