Unknown Users appears in Mentions, permissions, etc due to duplicates in the user_mapping table
Platform notice: Server and Data Center only. This article only applies to Atlassian products on the Server and Data Center platforms.
Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Duplication of users appears in various places in Confluence due to bugs as reported in:
For example, when trying @ mention users, Confluence is returning 'Unknown Users' in the search results, or existing @ mention or restriction/permission entries are showing as 'Unknown Users', you might run into duplicates in the database, specifically the user_mapping
table.
Environment
Confluence is connected to an external LDAP user directory.
Cause
This issue is observed when there are duplicated users that originated from across different user directories. For example, if Confluence is connected to LDAP, and also connected to Crowd that's connected to the same LDAP with the same sets of users.
One of the possible steps to reproduce the issue:
- Set a user directory connection to LDAP A with username attribute: SAMAccountname
- Set another user directory connection to LDAP A with username attribute: mail
- Edit the second user directory's username attribute to SAMAccountname
Diagnosis
First, try a content reindex. If it doesn't help, run this query (diagnosis 1) below, if it returns any results, please proceed to the next diagnosis step.
Diagnosis 1 - NULL users
SELECT * FROM user_mapping where lower_username is null;
Now you need to determine if you have single records or duplicated records, and which scenario types you are affected with. The workaround you perform will depend on this.
Diagnosis 2 - Determine Single/Duplicated records
select u.* from user_mapping u where lower(u.username) in (select lower(nullrecord.username) from user_mapping nullrecord where lower_username is null) order by u.username;
Below are 4 possible scenarios of how the query output from Diagnosis 2 would look like:
Scenario type 1 - Single user record, no lower username exists (Single Record)
user_key username lower_username ff8080814a5a97df014a5a97fb240001 userA Scenario type 2 - Duplicated users with no lower username exists, regardless of casings in the username column (Duplicated Records)
user_key username lower_username ff8080814a5a97df014a5a97fb240001 userA ff8080814a5a97df014a5a97fb240002 userA ff8080814a5a97df014a5a97fb240003 usera Scenario type 3 - Duplicated users with lower_username (Duplicated Records)
username lower_username ff8080814a5a97df014a5b16c37c0008 userA usera ff8080814a5a97df014a5a97fb240001 userA Scenario type 4 - Duplicated users with lower_username, regardless of casings in the username column (Duplicated Records)
user_key username lower_username ff8080814a5a97df014a5b16c37c0008 userA usera ff8080814a5a97df014a5a97fb240001 UserA ff8080814a5a97df014a5a97fb240002 userA
Please proceed to the appropriate workaround based on whether you see single or duplicated records. If you have both, apply the single records workaround first, then duplicated records.
Workaround
SINGLE RECORDS Workaround (scenario type 1)
Set the lower_username
value for single records only. This will not affect nulls for duplicate records.
update user_mapping set lower_username = lower(username) where lower_username is null and lower(username) in
(select LOWER(nullrecords.username) from user_mapping as nullrecords group by LOWER(nullrecords.username) having count(*) = 1);
This query will search for all single records and update its lower_username with a lowercase of its username.
Now run the diagnosis query (diagnosis 1) again to ensure you are no longer affected by the single records issue/Scenario 1 type issue, the query should return zero result.
DUPLICATED RECORDS Workaround (scenario type 2)
Please run the following SQL query, this query will search for records that have a scenario type 2 issue, and give one of the records to have a lower_username
. Then, please proceed with #1 - Clean up null user records.
select username, lower_username, count(*) as count into temp1 from user_mapping group by username, lower_username having count(*) > 1;
select username into temp2 from temp1 where lower(username) not in (select lower(username) from user_mapping where lower_username is not null);
select username, max(user_key) as keeping into uniqueone FROM user_mapping where username in (select username FROM temp2) group by username;
update user_mapping set lower_username = lower(username) where user_key in (select keeping from uniqueone);
drop table temp1;
drop table temp2;
drop table uniqueone;
DUPLICATED RECORDS Workaround (scenario type 3 and scenario type 4)
Please proceed with #1 - Clean up null user records.
#1 - Clean up null user records (without content associated).
First, try to remove the null values. If there is no content that has been created by the users, this set of queries will work. If you run into any Foreign Key constraint errors during this process, please proceed to #2 - If the null user records have content associated with them.
delete from imagedetails where attachmentid in
(select avatar.contentid from content avatar where avatar.pageid in
(select userinfo.contentid from content userinfo
where userinfo.contenttype = 'USERINFO'
and userinfo.username in
(select user_key from user_mapping where lower_username is null)));
delete from contentproperties where contentid in
(select avatar.contentid from content avatar where avatar.pageid in
(select userinfo.contentid from content userinfo
where userinfo.contenttype = 'USERINFO'
and userinfo.username in
(select user_key from user_mapping where lower_username is null)));
delete from content avatar where avatar.pageid in
(select userinfo.contentid from content userinfo
where userinfo.contenttype = 'USERINFO'
and userinfo.username in
(select user_key from user_mapping where lower_username is null));
delete from content userinfo
where userinfo.contenttype = 'USERINFO'
and prevver is not null
and userinfo.username in
(select user_key from user_mapping where lower_username is null);
delete from content userinfo
where userinfo.contenttype = 'USERINFO'
and prevver is null
and userinfo.username in
(select user_key from user_mapping where lower_username is null);
delete from logininfo where username in
(select user_key from user_mapping where lower_username is null);
delete from user_mapping where lower_username is null;
#2 - If the null user records have content associated with them.
Step One: Find the affected users (Keypair Generator)
First, run the below query, and save the results. You'll need the oldKey and newKey from the results.
SELECT
user_key AS oldKey,
(SELECT
user_key
FROM
user_mapping u
WHERE
lower(u.username) = u.lower_username
AND lower(u.username) = lower(um.username)) AS newKey,
um.username,
um.lower_username
FROM
user_mapping um
WHERE
user_key IN (SELECT
user_key
FROM
user_mapping
WHERE
(lower(username) != lower_username))
OR (lower_username is null)
Step Two: Validate your Keys:
Check if the result of the Keypair Generator produces any NULL values for the newkey column. NULL newkey values could exist when the scenario type 2 issue still exists in the database. Please follow SINGLE RECORDS Workaround (scenario type 1) and DUPLICATED RECORDS Workaround (scenario type 2) to ensure that scenario 1 and 2 has been ruled out.
Only proceed with Step Three when there are no NULL values in the newkey column.
Step Three: Individual/Mass fixes
3.1 - Individual cleanup
Run the following SQL queries to map @oldkey into @newkey. This workaround is ideal when the Keypair generator only returns 1-5 users.
If the KeyPair generator resulted in a large number of users, you can use a Python script provided in the next step to generate a set of SQL queries.
Change all Occurrence of oldkey and newkey in the SQL query template below, with the oldkey and newkey Keypair obtained from KeyPair Generator, either by using:
- Declare method
- Simple text editor
To use the Declare method, simply change the Declare Variables below to match the Key Pair obtained from the KeyPair Generator. Continue with running the SQL queries:
-- Declare Variables:
SET @oldKey := 'foo'; SET @newKey := 'bar';
Or, use a simple text editor to 'find and replace' all occurrences of @oldkey and @newkey, from the SQL query template below, for each Key Pairs obtained from the KeyPair Generator.
-- TRACKBACKLINKS (Not required for version 7 or above)
UPDATE TRACKBACKLINKS SET LASTMODIFIER = @newKey WHERE LASTMODIFIER = @oldKey;
UPDATE TRACKBACKLINKS SET CREATOR = @newKey WHERE CREATOR = @oldKey;
-- SPACES
UPDATE SPACES SET LASTMODIFIER = @newKey WHERE LASTMODIFIER = @oldKey;
UPDATE SPACES SET CREATOR = @newKey WHERE CREATOR = @oldKey;
-- SPACE PERMISSIONS
UPDATE SPACEPERMISSIONS SET LASTMODIFIER = @newKey WHERE LASTMODIFIER = @oldKey;
UPDATE SPACEPERMISSIONS SET CREATOR = @newKey WHERE CREATOR = @oldKey;
UPDATE SPACEPERMISSIONS SET PERMUSERNAME = @newKey WHERE PERMUSERNAME = @oldKey;
-- SPACEGROUPS ( Not required for version 6 and above)
UPDATE SPACEGROUPS SET LASTMODIFIER = @newKey WHERE LASTMODIFIER = @oldKey;
UPDATE SPACEGROUPS SET CREATOR = @newKey WHERE CREATOR = @oldKey;
-- SPACEGROUPPERMISSIONS ( Not required for version 6 and above)
UPDATE SPACEGROUPPERMISSIONS SET PERMUSERNAME = @newKey WHERE PERMUSERNAME = @oldKey;
-- PAGETEMPLATES
UPDATE PAGETEMPLATES SET LASTMODIFIER = @newKey WHERE LASTMODIFIER = @oldKey;
UPDATE PAGETEMPLATES SET CREATOR = @newKey WHERE CREATOR = @oldKey;
-- NOTIFICATIONS
UPDATE NOTIFICATIONS SET LASTMODIFIER = @newKey WHERE LASTMODIFIER = @oldKey;
UPDATE NOTIFICATIONS SET CREATOR = @newKey WHERE CREATOR = @oldKey;
UPDATE NOTIFICATIONS SET USERNAME = @newKey WHERE USERNAME = @oldKey;
-- LINKS
UPDATE LINKS SET LASTMODIFIER = @newKey WHERE LASTMODIFIER = @oldKey;
UPDATE LINKS SET CREATOR = @newKey WHERE CREATOR = @oldKey;
-- LIKES
UPDATE LIKES SET USERNAME = @newKey WHERE USERNAME = @oldKey;
-- LABEL
UPDATE LABEL SET OWNER = @newKey WHERE OWNER = @oldKey;
-- FOLLOW_CONNECTIONS
UPDATE FOLLOW_CONNECTIONS SET FOLLOWER = @newKey WHERE FOLLOWER = @oldKey;
UPDATE FOLLOW_CONNECTIONS SET FOLLOWEE = @newKey WHERE FOLLOWEE = @oldKey;
-- EXTRNLINKS (Not required for version 7 or above)
UPDATE EXTRNLNKS SET CREATOR = @newKey WHERE CREATOR = @oldKey;
UPDATE EXTRNLNKS SET LASTMODIFIER = @newKey WHERE LASTMODIFIER = @oldKey;
-- CONTENT_PERM
UPDATE CONTENT_PERM SET CREATOR = @newKey WHERE CREATOR = @oldKey;
UPDATE CONTENT_PERM SET LASTMODIFIER = @newKey WHERE LASTMODIFIER = @oldKey;
UPDATE CONTENT_PERM SET USERNAME = @newKey WHERE USERNAME = @oldKey;
-- CONTENT_LABEL
UPDATE CONTENT_LABEL SET OWNER = @newKey WHERE OWNER = @oldKey;
-- CONTENT
UPDATE CONTENT SET LASTMODIFIER = @newKey WHERE LASTMODIFIER = @oldKey;
UPDATE CONTENT SET USERNAME = @newKey WHERE USERNAME = @oldKey;
UPDATE CONTENT SET CREATOR = @newKey WHERE CREATOR = @oldKey;
-- USERCONTENT_RELATION ( Only required for version 6 and above)
UPDATE USERCONTENT_RELATION SET CREATOR = @newKey WHERE CREATOR = @oldKey;
UPDATE USERCONTENT_RELATION SET SOURCEUSER = @newKey WHERE SOURCEUSER = @oldKey;
UPDATE USERCONTENT_RELATION SET LASTMODIFIER = @newKey WHERE LASTMODIFIER = @oldKey;
-- ATTACHMENTS ( Not required for version 5.7 and above)
UPDATE ATTACHMENTS SET LASTMODIFIER = @newKey WHERE LASTMODIFIER = @oldKey;
UPDATE ATTACHMENTS SET CREATOR = @newKey WHERE CREATOR = @oldKey;
-- LOGIN INFO AND USER_MAPPING
DELETE FROM logininfo WHERE USERNAME = @oldKey;
DELETE FROM user_mapping where user_key = @oldKey;
3.2 Mass Cleanup Using the mass duplicate fixer
If you have a large number of duplicates, you may choose to use the mass-dup-fixer.zipto generate the SQL queries. You'll need Python (tested on Python 2.7 and Python 3) installed to run the script:
- Ensure that Python is installed
- Run the KeyPair Generator
Save the output of that query in a tab-separated format, oldKey<tab>newKey
Example2c9d81ef8450b135018450b2025c0000 2c9d81ef8450b135018450b269730001 2c9d81ef8450b135018454c05f670002 2c9d80eb8473fa8c018474460f7a0001
- Name the file
keypairs.txt
- Extract "mass-dup-fixer.zip" to a directory,
- Place your
keypairs.txt
file into this directory. - Run
make-mass-dup-fixer.py
from the directory you extracted everything to - It will generate a file called output.sql
- Run the SQL queries inside the output.sql against Confluence database