Summary

How to monitor the Directory Import process referred to in the Importing Users from One Crowd Directory into Another knowledge article.

Environment

Crowd 5.2.2

Solution

Please follow these steps to monitor this process: 

• In Crowd, Enable Debug logging for the below classes in Setting > Logging and Properties. At the end, add each package one by one, select New Level as DEBUG, and click Add. Once all are added, click on update logging.

  com.atlassian.crowd.manager.directory.DirectoryManagerGeneric
  com.atlassian.crowd.dao.user.UserDAOHibernate
  com.atlassian.crowd.dao.group.GroupDAOHibernate
  com.atlassian.crowd.dao.membership.MembershipDAOHibernate

• Start the Directory import process.

  
  When the import process starts, it will print the details about each stage and the number of users and groups it will import. However, it will not print a count of user-to-group memberships and group-to-group memberships. To find out the progress of each stage, follow the next steps:

  2024-03-19 09:57:08,839 http-nio-8095-exec-35 url: /crowd/console/secure/dataimport/directoryconfirmation!update.action; user: admin INFO [crowd.importer.importers.BaseImporter] Importing 6 groups.
  2024-03-19 09:57:08,887 http-nio-8095-exec-35 url: /crowd/console/secure/dataimport/directoryconfirmation!update.action; user: admin INFO [crowd.importer.importers.BaseImporter] Importing 2 users.
  2024-03-19 09:57:08,928 http-nio-8095-exec-35 url: /crowd/console/secure/dataimport/directoryconfirmation!update.action; user: admin INFO [crowd.importer.importers.BaseImporter] Importing user-to-group memberships.

• Find out the directory id's for the source and destination directory.

  select count(*) from cwd_directory where directory_id = 'source_directory_id';
  
  select count(*) from cwd_directory where directory_id = 'destination_directory_id';

• To find out the count of user-group memberships that will be imported, run the following SQL query:

  select count(*) from cwd_membership where directory_id = "source_directory_id";

• Find out the thread ID of the thread running the Directory Import by taking a thread dump, and noting down the thread ID for the thread having the below stack trace:

  "http-nio-8095-exec-3" daemon prio=5 tid=0x000000000000008c nid=0 runnable 
     java.lang.Thread.State: RUNNABLE
  	at java.base@11.0.22/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  	at java.base@11.0.22/java.lang.reflect.Method.invoke(Method.java:566)
  	at org.hibernate.property.access.spi.GetterMethodImpl.get(GetterMethodImpl.java:42)
  	at org.hibernate.tuple.entity.AbstractEntityTuplizer.getPropertyValues(AbstractEntityTuplizer.java:574)
  	at org.hibernate.tuple.entity.PojoEntityTuplizer.getPropertyValues(PojoEntityTuplizer.java:154)
  	at org.hibernate.persister.entity.AbstractEntityPersister.getPropertyValues(AbstractEntityPersister.java:5169)
  	at org.hibernate.event.internal.DefaultFlushEntityEventListener.getValues(DefaultFlushEntityEventListener.java:206)
  	at org.hibernate.event.internal.DefaultFlushEntityEventListener.onFlushEntity(DefaultFlushEntityEventListener.java:162)
  	at org.hibernate.event.internal.AbstractFlushingEventListener$$Lambda$787/0x00000008414a9840.accept(Unknown Source)
  	at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:99)
  	at org.hibernate.event.internal.AbstractFlushingEventListener.flushEntities(AbstractFlushingEventListener.java:229)
  	at org.hibernate.event.internal.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:93)
  	at org.hibernate.event.internal.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:50)
  	at org.hibernate.internal.SessionImpl$$Lambda$836/0x00000008414b1040.accept(Unknown Source)
  	at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:99)
  	at org.hibernate.internal.SessionImpl.autoFlushIfRequired(SessionImpl.java:1327)
  	at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1870)
  	at org.hibernate.internal.CriteriaImpl.list(CriteriaImpl.java:370)
  	at org.hibernate.internal.CriteriaImpl.uniqueResult(CriteriaImpl.java:392)
  	at com.atlassian.crowd.dao.group.GroupDAOHibernate.findByNameInternal(GroupDAOHibernate.java:150)
  	at com.atlassian.crowd.dao.group.GroupDAOHibernate.findByName(GroupDAOHibernate.java:138)
  	at com.atlassian.crowd.dao.group.GroupDAOHibernate.findByName(GroupDAOHibernate.java:44)
  	at com.atlassian.crowd.directory.AbstractInternalDirectory.findGroupByName(AbstractInternalDirectory.java:710)
  	at com.atlassian.crowd.directory.AbstractInternalDirectory.findGroupByName(AbstractInternalDirectory.java:65)
  	at com.atlassian.crowd.directory.AbstractForwardingDirectory.findGroupByName(AbstractForwardingDirectory.java:147)
  	at com.atlassian.crowd.manager.directory.DirectoryManagerGeneric.findGroupByName(DirectoryManagerGeneric.java:541)
  	at com.atlassian.crowd.manager.directory.DirectoryManagerGeneric.addUserToGroup(DirectoryManagerGeneric.java:728)
  	at com.atlassian.crowd.manager.directory.DirectoryManagerGeneric.addAllUsersToGroup(DirectoryManagerGeneric.java:1288)
  ....
  	at com.atlassian.crowd.importer.importers.BaseImporter.importUserMemberships(BaseImporter.java:132)
  	at com.atlassian.crowd.importer.importers.BaseImporter.importUserToGroupMemberships(BaseImporter.java:226)
  	at com.atlassian.crowd.importer.importers.BaseImporter.importUsersGroupsAndMemberships(BaseImporter.java:210)
  	at com.atlassian.crowd.importer.importers.DirectoryImporter.importUsersGroupsAndMemberships(DirectoryImporter.java:30)
  ...
  	at com.atlassian.crowd.importer.manager.ImporterManagerImpl.performImport(ImporterManagerImpl.java:49)
  	at com.atlassian.crowd.console.action.dataimport.DirectoryConfirmation.update(DirectoryConfirmation.java:35)
  .....
  	at java.base@11.0.22/java.lang.Thread.run(Thread.java:829)

• In the grep  command, substitute the thread ID which is running the directory import and directory ID to get the count of records completed for each stage. The command has to executed from the <crowd-home>/logs  directory which has atlassian-crowd.log  

  // To check the user import status.
  grep DEBUG atlassian-crowd.log*|grep "http-nio-8095-exec-3"|grep "crowd.dao.user.UserDAOHibernate"|grep 45678922|grep "Saving Object"|wc -l
  
  //To check the group import status
  grep DEBUG atlassian-crowd.log*|grep "http-nio-8095-exec-3"|grep "crowd.dao.group.GroupDAOHibernate"|grep 45678922|grep "Saving Object"|wc -l
  
  //To check User-group membership.
  grep DEBUG atlassian-crowd.log*|grep "http-nio-8095-exec-3"|grep "com.atlassian.crowd.dao.membership.MembershipDAOHibernate"grep 45678922|grep "Saving Object"|wc -l

• Once the directory import process completes, it will print below message:

  2022-03-21 19:14:47,100 http-nio-8095-exec-12  url: /crowd/console/secure/dataimport/directoryconfirmation!update.action; user: admin INFO [crowd.importer.importers.BaseImporter] Completed importing 13456 user-to-group memberships.
  2022-03-21 15:27:36,210 http-nio-8095-exec-12  url: /crowd/console/secure/dataimport/directoryconfirmation!update.action; user: admin INFO [crowd.importer.importers.BaseImporter] Completed importing 54 group-to-group memberships.