Symptoms

As an administrator, you're unable to delete users as you are presented with an error dialog that says Bamboo: Unauthorized.  Removing users from all groups does not alleviate this issue.

 Causes

This is due to the user having been a Bamboo user and the administrator not also having admin permissions in Bamboo. 

Solution

The workaround for this is to add yourself to the administrators-bamboo group or equivalent.  If you navigate to the Bamboo admin page at <URL>/builds/admin/viewGlobalPermissions.action, you will see what group as the Restricted Admin functionality - you just need to be a part of that group.