Rackspace hosted Exchange suffered a disastrous failure beginning December 2, 2022 and is still continuous since 12:37 AM December fourth. At first referred to as connection and login problems, the guidance was eventually updated to announce that they were handling a security event.
Rackspace Hosted Exchange Issues
The Rackspace system decreased in the morning hours of December 2, 2022. Initially there was no word from Rackspace about what the issue was, much less an ETA of when it would be resolved.
Consumers on Buy Twitter Verification reported that Rackspace was not reacting to support emails.
This has been rather the day with #Rackspace. Every hosted exchange client has actually been down for 14 hours or two. Assistance isn’t reading/responding to tickets. Updates are unhelpful.
I am concerned now that they came down with something bad like the ProxyNotShell PoC hack. https://t.co/jchKsAO3Z7
— Joe Sinkwitz (@CygnusSEO) December 2, 2022
A Rackspace customer independently messaged me over social networks on Friday to relate their experience:
“All hosted Exchange clients down over the past 16 hours.
Unsure the number of business that is, but it’s significant.
They’re serving a 554 long delay bounce so people emailing in aren’t knowledgeable about the bounce for numerous hours.”
The official Rackspace status page provided a running upgrade of the outage however the initial posts had no information other than there was an outage and it was being examined.
The very first official upgrade was on December 2nd at 2:49 AM:
“We are investigating a problem that is affecting our Hosted Exchange environments. More details will be published as they appear.”
Thirteen minutes later Rackspace started calling it a “connectivity concern.”
“We are examining reports of connection issues to our Exchange environments.
Users might experience an error upon accessing the Outlook Web App (Webmail) and syncing their email customer(s).”
By 6:36 AM the Rackspace updates described the ongoing problem as “connection and login issues” then later on that afternoon at 1:54 PM Rackspace announced they were still in the “examination phase” of the interruption, still trying to determine what failed.
And they were still calling it “connectivity and login problems” in their Cloud Workplace environments at 4:51 PM that afternoon.
Rackspace Recommends Migrating to Microsoft 365
Four hours later Rackspace described the situation as a “significant failure”and began using their customers complimentary Microsoft Exchange Strategy 1 licenses on Microsoft 365 as a workaround up until they understood the issue and could bring the system back online.
The main guidance stated:
“We experienced a substantial failure in our Hosted Exchange environment. We proactively closed down the environment to avoid any additional concerns while we continue work to restore service. As we continue to resolve the root cause of the problem, we have an alternate option that will re-activate your capability to send out and receive emails.
At no charge to you, we will be providing you access to Microsoft Exchange Strategy 1 licenses on Microsoft 365 till further notification.”
Rackspace Hosted Exchange Security Event
It was not up until almost 24 hours later at 1:57 AM on December 3rd that Rackspace officially announced that their hosted Exchange service was struggling with a security event.
The announcement even more exposed that the Rackspace specialists had actually powered down and disconnected the Exchange environment.
“After further analysis, we have actually identified that this is a security incident.
The recognized effect is isolated to a part of our Hosted Exchange platform. We are taking needed actions to evaluate and safeguard our environments.”
Twelve hours later that afternoon they upgraded the status page with more info that their security team and outside professionals were still working on fixing the failure.
Was Rackspace Service Affected by a Vulnerability?
Rackspace has actually not released details of the security occasion.
A security occasion generally involves a vulnerability and there are two serious vulnerabilities presently in the wile that were covered in November 2022.
These are the 2 most present vulnerabilities:
Microsoft Exchange Server Server-Side Request Forgery (SSRF) Vulnerability
A Server Side Demand Forgery (SSRF) attack allows a hacker to read and change information on the server.
Microsoft Exchange Server Remote Code Execution Vulnerability
A Remote Code Execution Vulnerability is one in which an assailant is able to run harmful code on a server.
An advisory published in October 2022 explained the impact of the vulnerabilities:
“An authenticated remote opponent can perform SSRF attacks to escalate opportunities and carry out arbtirary PowerShell code on vulnerable Microsoft Exchange servers.
As the attack is targeted against Microsoft Exchange Mail box server, the attacker can possibly gain access to other resources via lateral motion into Exchange and Active Directory site environments.”
The Rackspace failure updates have actually not shown what the particular issue was, just that it was a security event.
The most current status upgrade since December fourth specified that the service is still down and clients are encouraged to migrate to the Microsoft 365 service.
Rackspace posted the following on December 4, 2022 at 12:37 AM:
“We continue to make progress in addressing the incident. The availability of your service and security of your data is of high significance.
We have devoted substantial internal resources and engaged first-rate external knowledge in our efforts to decrease unfavorable impacts to consumers.”
It’s possible that the above kept in mind vulnerabilities relate to the security occurrence affecting the Rackspace Hosted Exchange service.
There has been no announcement of whether client details has actually been jeopardized. This occasion is still ongoing.
Featured image by Best SMM Panel/Orn Rin