"Ghost" notifications are being sent by Bamboo for builds that didn't happen
Platform Notice: Data Center Only - This article only applies to Atlassian products on the Data Center platform.
Note that this KB was created for the Data Center version of the product. Data Center KBs for non-Data-Center-specific features may also work for Server versions of the product, however they have not been tested. Support for Server* products ended on February 15th 2024. If you are running a Server product, you can visit the Atlassian Server end of support announcement to review your migration options.
*Except Fisheye and Crucible
Summary
Bamboo sends emails and/or other type of notifications for builds that never really happened.
Environment
Any supported version of Bamboo with notifications configured.
Diagnosis
In what appears to be a random behavior, normal notifications will be sent reporting build results. This can be a Slack message, or an email, or similar.
Let's take as an example a Slack message reporting:
Slack
1
10:10 My Project › My Plan › #18 failed. Changes by ... Responsible Users: ...The best way to diagnose what Bamboo was doing is by looking into the server logs for that particular day, at 10:10. A normal entry for a sent notification will be:
atlassian-bamboo.log
1
2021-03-04 10:10:00,083 INFO [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-20] [ChainStatusChangeNotificationType] Plan status changed to successful sending a notification Depending on the type of notification, more debugging log entries can be enabled, but they are not needed at this point of the investigation.
What is important to identify in the logs is:
If the build (identifying plan and build number) happened and finished approximately at that time. There's no queue, thread limitation or other theories that could justify performance issues with notifications, so events should happen almost immediately.
If there was an instruction to send a notification at that time.
Cause
If server logs don't report the notification from being created and sent, then that instance of Bamboo was not responsible for that notification. Reported scenarios were this behavior was observed included:
A recent migration of Bamboo, with the original instance still running.
A second environment (i.e. staging, testing, etc) running simultaneously.
Any upgrade/test/other scenario were a second instance was created and was left running
In every example given, there was more than one instance of Bamboo running, and the unchecked instance was the one sending the notifications.
Solution
There are many ways to mitigate this problem, and all involve addressing the secondary/obsolete instance. Valid approaches could be:
Shut down that instance.
Remove the configuration for the email server/IM server.
Remove notifications.
Block builds from happening (through disabling agents, running Bamboo on safe mode, etc).
Other options that address builds or notifications.
Was this helpful?