Kaseya, an IT solutions developer for MSPs and enterprise clients, announced that it had become the victim of a cyberattack on July 2, over the American Independence Day weekend.
It appears that attackers have carried out a supply chain ransomware attack by leveraging a vulnerability in Kaseya’s VSA software against multiple managed service providers (MSP) – and their customers.
What is Kaseya?
Kaseya provides IT solutions including VSA, a unified remote-monitoring and management tool for handling networks and endpoints. In addition, the company provides compliance systems, service desks, and a professional services automation platform.
What Happened?
On July 2 at 2:00 PM EDT, Kaseya CEO, Fred Voccola announced, “a potential attack against the VSA that has been limited to a small number of on-premise customers.”
At the same time, out of an abundance of caution, Voccola urged clients to immediately shut down their VSA servers. “It’s critical that you do this immediately because one of the first things the attacker does is shut off administrative access to the VSA,” the executive said. Customers were notified of the breach via email, phone, and online notices.
As Kaseya’s Incident Response team investigated, the vendor also decided to proactively shut down its SaaS servers and pull its data centers offline. By July 4, the company had revised its thoughts on the severity of the incident, calling itself the “victim of a sophisticated cyberattack.” Cyber forensics experts from FireEye’s Mandiant team, alongside other security companies, have been pulled in to assist. “Our security, support, R&D, communications, and customer teams continue to work around the clock in all geographies to resolve the issue and restore our customers to service,” Kaseya said, adding that more time is needed before its data centers are brought back online.
Once the SaaS servers are operational, Kaseya will publish a schedule for distributing a security patch to on-prem clients. In a July 5 update, Kaseya said that a fix has been developed and would first be deployed to SaaS environments, once testing and validation checks are complete.
“We are developing the new patch for on-premises clients in parallel with the SaaS Data Center restoration,” the company said. “We are deploying in SaaS first as we control every aspect of that environment. Once that has begun, we will publish the schedule for distributing the patch for on-premises customers.” The ransomware attack, explained
The FBI described the incident succinctly: a “supply chain ransomware attack leveraging a vulnerability in Kaseya VSA software against multiple MSPs and their customers.”
Huntress (1,2) has tracked 30 MSPs involved in the breach and believes with “high confidence” that the attack was triggered via an authentication bypass vulnerability in the Kaseya VSA web interface.
According to the cybersecurity firm, this allowed the attackers to circumvent authentication controls, gain an authenticated session, upload a malicious payload, and execute commands via SQL injection, achieving code execution in the process.
“Some of the functionality of a VSA Server is the deployment of software and automation of IT tasks,” Sophos noted. “As such, it has a high level of trust on customer devices. By infiltrating the VSA Server, any attached client will perform whatever task the VSA Server requests without question. This is likely one of the reasons why Kaseya was targeted.”
The vendor has also provided an in-depth technical analysis of the attack.
Security expert Kevin Beaumont said that ransomware was pushed via an automated, fake, and malicious software update using Kaseya VSA dubbed “Kaseya VSA Agent Hot-fix”.
“This fake update is then deployed across the estate — including on MSP client customers’ systems — as it [is] a fake management agent update,” Beaumont commented. “This management agent update is actually REvil ransomware. To be clear, this means organizations that are not Kaseya’s customers were still encrypted.”
With a tip from RiskIQ, Huntress is also investigating an AWS IP address that may have been used as a launch point for the attack.
On July 5, Kaseya released an overview of the attack, which began on July 2 with reports of ransomware deployment on endpoints.
“In light of these reports, the executive team convened and made the decision to take two steps to try to prevent the spread of any malware: we sent notifications to on-premises customers to shut off their VSA servers and we shut down our VSA SaaS infrastructure,” the company says.
According to the firm, zero-day vulnerabilities were exploited by the attackers to trigger a bypass authentication and for code execution, allowing them to infect endpoints with ransomware. However, Kaseya emphasizes that there is no evidence of the VSA codebase being “maliciously modified”.
Who has been Impacted?
Over the weekend, Kaseya said that SaaS customers were “never at risk” and current estimates suggest that fewer than 40 on-prem clients worldwide have been affected.
However, it should be noted that while a small number of Kaseya clients may have been directly infected, as MSPs, SMB customers further down the chain relying on these services could be impacted in their turn.
According to reports, 800 Coop supermarket chain stores in Sweden had to temporarily close as they were unable to open their cash registers.
Huntress said in a Reddit explainer that an estimated 1,000 companies have had servers and workstations encrypted. The vendor added that it is reasonable to suggest “thousands of small businesses” may have been impacted.
“This is one of the farthest-reaching criminal ransomware attacks that Sophos has ever seen,” commented Ross McKerchar, Sophos VP. “At this time, our evidence shows that more than 70 managed service providers were impacted, resulting in more than 350 further impacted organizations. We expect the full scope of victim organizations to be higher than what’s being reported by any individual security company.”
On July 5, Kaseya revised previous estimates to “fewer than 60” customers, adding that “we understand the total impact thus far has been to fewer than 1,500 downstream businesses.”
Now, on July 6, the estimate is between 50 direct customers, and between 800 and 1,500 businesses down the chain.
When it comes to SaaS environments, Kaseya says, “We have not found evidence that any of our SaaS customers were compromised.”
In a press release dated July 6, Kaseya has insisted that “while impacting approximately 50 of Kaseya’s customers, this attack was never a threat nor had any impact to critical infrastructure.”
What is Ransomware?
Ransomware is a type of malware that specializes in the encryption of files and drives.
In what has become one of the most severe and serious security problems modern businesses now face, ransomware is used by threat actors worldwide to hijack systems and disrupt operations.
Once a victim’s system or network has been encrypted, cyber criminals will place a ransom note on the system, demanding payment in return for a decryption key (which may, or may not, work).
Today’s ransomware operators may be part of Ransomware-as-a-Service (RaaS), when they ‘subscribe’ to access and use a particular type of ransomware. Another emerging trend is double extortion, in which a victim will have their information stolen during a ransomware raid.
If they refuse to pay up, they may then face the prospect of their data being sold or published online.
Common and well-known ransomware families include REvil, Locky, WannaCry, Gandcrab, Cerber, NotPetya, Maze, and Darkside.
Who is Responsible?
The cyberattack has been attributed to the REvil/Sodinikibi ransomware group, which has claimed responsibility on its Dark Web leak site, “Happy Blog.”
In an update over the weekend, the operators claimed that more than “a million” systems have been infected.
REvil has offered a decryption key, allegedly universal and, therefore, able to unlock all encrypted systems, for the ‘bargain’ price of $70 million in the bitcoin (BTC) cryptocurrency.
REvil has been previously linked to ransomware attacks against companies, including JBS, Travelex, and Acer.
Ransomware Payment Terms
The ransomware note claims that files are “encrypted, and currently unavailable.” A file extension .csruj has reportedly been used. Operators are demanding payment in return for a decryption key and one ‘freebie’ file decryption is also on the table to prove the decryption key works.
The operators add (spelling unchanged):
“Its just a business. We absolutely do not care about you and your deals, except getting benefits. If we do not do our work and liabilities – nobody will not cooperate with us. Its not in our interests. If you will not cooperate with our service –for us, its does not matter. But you will lose your time and data, cause just we have the private key. In practice – time is much more valuable than money.”
Sophos malware analyst Mark Loman shared a screenshot on Twitter of a ransomware note planted on an infected endpoint demanding $44,999.
John Hammond, senior security researcher at Huntress, said that the company has already seen ransom demands of up to $5 million.
Kevin Beaumont says that, unfortunately, he has observed victims “sadly negotiating” with the ransomware’s operators.
Fabian Wosar, CTO of Emsisoft, has also explained in a Twitter thread why using a key obtained by a single organization paying up is unlikely to be a viable path for unlocking all victims.
“REvil absolutely has the capability of decrypting only a single victim without these purchased decryption tools being applicable for other victims hit by the same campaign public key,” the security expert noted.
Reactions
At the time of the breach, Kaseya notified law enforcement and cybersecurity agencies, including the Federal Bureau of Investigation (FBI) and US Cybersecurity and Infrastructure Security Agency (CISA).
The FBI and CISA have released a joint statement on the security incident and are urging customers to run a tool provided by Kaseya to determine the risk of exploit, and to both enable and enforce multi-factor authentication (MFA) on enterprise accounts, wherever possible.
Kaseya has been holding meetings with the FBI and CISA “to discuss systems and network hardening requirements prior to service restoration for both SaaS and on-premises customers.”
The White House is asking organizations to inform the Internet Crime Complaint Center (IC3) if they suspect they have been compromised.
On Saturday, US President Biden said he has directed federal intelligence agencies to investigate.
“Targeting [an] MSP platform (that is managing many customers at once) was very well thought and planned,” Amit Bareket, CEO of Perimeter 81. “What’s unique is that hackers are becoming more strategic and targeting platforms that will filtrate down to many companies with one shot. RMMs [remote monitoring and management] are basically keys to many many companies, which amount to the kingdom for bad actors.”
Recovery Plans
As of July 4, Kaseya says the company has now moved on from a root cause analysis of the attack to recovery and patch plans, consisting of:
- Communication of our phased recovery plan with SaaS first followed by on-premises customers.
- Kaseya will be publishing a summary of the attack and what we have done to mitigate it.
- Some lightly-used legacy VSA functionality will be removed as part of this release out of an abundance of caution. A specific list of the functionality and its impact on VSA capabilities will be outlined in the release notes.
- There will be new security measures implemented including enhanced security monitoring of our SaaS servers by FireEye and enablement of enhanced WAF capabilities.
- We have successfully completed an external Vulnerability Scan, checked our SaaS Databases for Indicators of Compromise, and have had external security experts review our code to ensure a successful service restart.
Data centers starting with the EU will be restored, followed by the UK, APAC, and then North American systems.
By late evening on July 5, Kaseya said a patch has been developed and it is the firm’s intention to bring back VSA with “staged functionality” to hasten the process. The company explained:
The first release will prevent access to functionality used by a very small fraction of our user base, including:
- Classic Ticketing
- Classic Remote Control (not LiveConnect).
- User Portal
What can Customers do?
Kaseya has released a tool, including Indicators of Compromise (IoC), which can be downloaded via Box. There are two PowerShell scripts for use: one on a VSA server, and the other has been designed for endpoint scanning.
The self-assessment scripts should be used in offline mode. They were updated on July 5 to also scan for data encryption and REvil’s ransom note.
However, the scripts are only for potential exploit risk detection and are not security fixes. Kaseya will release patches as quickly as it can, but in the meantime, customers simply have to wait.
“All on-premises VSA Servers should continue to remain offline until further instructions from Kaseya about when it is safe to restore operations,” the firm said. “A patch will be required to be installed prior to restarting the VSA.”
Cado Security has provided a GitHub repository for responders, including malware samples, IoCs, and Yara Rules.
Kaseya has also warned that “customers who experienced ransomware and receive a communication from the attackers should not click on any links — they may be weaponized.”
Summary
Special thank you to ZDNet and other sources for the in depth information on this attack. One of the key reasons why this attack was executed was the lack of infrastructure hardening. If better Secure Infrastructure security controls were put in place, the behavior of this attack would have been made visible and would have stopped this breach from materializing.
If you would like to learn more about how you can protect against these types of cyber events, please contact Sniper Watch. https://go.oncehub.com/sniperwatch-intro
