Cloudflare Hacked Using Auth Tokens Stolen In Okta Attack
Cloudflare disclosed today that its internal Atlassian server was breached by a suspected ‘nation state attacker’ who accessed its Confluence wiki, Jira bug database, and Bitbucket source code management system.
The threat actor first gained access to Cloudflare’s self-hosted Atlassian server on November 14 and then accessed the company’s Confluence and Jira systems following a reconnaissance stage.
“They then returned on November 22 and established persistent access to our Atlassian server using ScriptRunner for Jira, gained access to our source code management system (which uses Atlassian Bitbucket), and tried, unsuccessfully, to access a console server that had access to the data center that Cloudflare had not yet put into production in São Paulo, Brazil,” said Cloudflare CEO Matthew Prince, CTO John Graham-Cumming, and CISO Grant Bourzikas,
To access its systems, the attackers used one access token and three service account credentials stolen during a previous compromise linked to Okta’s breach from October 2023 that Cloudflare failed to rotate (out of thousands were leaked during the Okta compromise).
Cloudflare detected the malicious activity on November 23, severed the hacker’s access in the morning of November 24, and its cybersecurity forensics specialists began investigating the incident three days later, on November 26.
While addressing the incident, Cloudflare’s staff rotated all production credentials (over 5,000 unique ones), physically segmented test and staging systems, performed forensic triage on 4,893 systems, reimaged and rebooted all systems on the company’s global network, including all Atlassian servers (Jira, Confluence, and Bitbucket) and machines accessed by the attacker.
The threat actors also tried hacking into Cloudflare’s data center in São Paulo—which isn’t yet used in production—but these attempts failed. All equipment in Cloudflare’s Brazil data center was later returned to the manufacturers to ensure that the data center was 100% secure.
Remediation efforts ended almost one month ago, on January 5th, but the company says that its staff is still working on software hardening, as well as credential and vulnerability management.
The company says that this breach did not impact Cloudflare customer data or systems; its services, global network systems, or configuration were also unaffected.
“Even though we understand the operational impact of the incident to be extremely limited, we took this incident very seriously because a threat actor had used stolen credentials to get access to our Atlassian server and accessed some documentation and a limited amount of source code,” said Prince, Graham-Cumming, and Bourzikas.
“Based on our collaboration with colleagues in the industry and government, we believe that this attack was performed by a nation state attacker with the goal of obtaining persistent and widespread access to Cloudflare’s global network.
“Analyzing the wiki pages they accessed, bug database issues, and source code repositories, it appears they were looking for information about the architecture, security, and management of our global network; no doubt with an eye on gaining a deeper foothold.”
On October 18, 2023, Cloudflare’s Okta instance was breached using an authentication token stolen from Okta’s support system. The hackers who breached Okta’s customer support system also gained access to files belonging to 134 customers, including 1Password, BeyondTrust, and Cloudflare.
After the October 2023 incident, the company said that its Security Incident Response Team’s quick response contained and minimized the impact on Cloudflare systems and data and that no Cloudflare customer information or systems were impacted.
Another attempt to breach Cloudflare’s systems was blocked in August 2022 after attackers tried using employee credentials stolen in a phishing attack but failed because they didn’t have access to the victims’ company-issued FIDO2-compliant security keys.