Virtual appliances are a popular way for software vendors to distribute their products to enterprise customers as they contain all the necessary pre-configured software stacks their applications need to function and can be deployed in public clouds or private data centers with ease. Unfortunately, enterprises are at risk of deploying images that are vulnerable out-of-the-box according to a new study. It found that many vendors, including well-established ones, do a poor job of patching flaws and updating the software components in their virtual appliances.
Few virtual appliances get good security grades
Orca Security, a cloud security company, scanned more than 2,200 virtual appliance images from 540 vendors that were being distributed through the public marketplaces of common cloud platforms including VMware, Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform. The appliances were both commercial and free-to-use, contained both proprietary software and open-source, and were supplied by both security and non-security vendors.
The company created a scoring system from 0 to 100 that took into account whether the appliances were running supported or no longer supported operating system versions, contained one or more of 17 high-profile and high-risk vulnerabilities such as Heartbleed, EternalBlue and DirtyCOW, contained one or more other vulnerabilities rated above CVSS 9 (critical), or had one or more vulnerabilities rated between CVSS 7 and 9.
A grading system from A+ (exemplary) to F (failed) was also used. A virtual appliance would automatically fail the test if it had an unsupported operating system, contained four of the 16 high-profile vulnerabilities, had 20 or more flaws with CVSS 9 and higher, had 100 or more flaws with CVSS 7 to 9, or had more than 400 unique vulnerabilities. Fifteen percent of the tested appliances received an F and the lowest recorded score was 6 out of 100. Another 16% received a D rating (poor), 25% received a C (mediocre) and 12% a B (above average). Only 8% received an A+ and 24% an A.
In total, Orca's scanning identified 401,571 vulnerabilities across 2,218 appliances. The subsequent notification of affected vendors resulted in 287 products being updated and 53 being removed from distribution. Some vendors had products at both ends of the spectrum. Some vendors were responsive, but others argued it was customers' responsibility to update the appliance's software and patch any existing flaws after deployment.
Infrequent virtual appliance updates
As expected, the number of vulnerabilities discovered per appliance was directly tied to how frequently the appliance was being updated by its publisher. Almost half hadn't been updated by vendors over the past year and only 2.8% (64) had been updated within the month before Orca's scans. Another 14% (312) had been updated within the previous three months.
Virtual appliances from traditional security vendors scored above average (83 instead of 79), but there were cases of poor image maintenance in this category as well. For example, one security company was distributing a 26-month-old appliance that contained a critical vulnerability that the vendor itself discovered and reported to the industry back in 2018. Upon notification, the appliance was removed.
"Poor processes account for the product age problem in many cases," Orca said in its report. "Out-of-date products remain available after they’ve reached their end-of-life. The overall product is no longer supported, the operating systems may be unsupported, and/or updates and patches are no longer being applied. As a result of Orca Security’s research, 39 products have been removed from distribution."
Commercial appliances scored about the same on average as free and open-source ones, with the latter having a slight advantage. However, hardened virtual appliances whose operating systems and software stacks had been stripped down to minimize attack surface, scored much higher than all other appliances -- 94.2 on average.
Over half of tested appliances came from system integrators. These images have all the necessary components to run certain Web applications -- for example an image with WordPress, but also the Apache Web server and MySQL database and the OpenSSL security library. Their average score was 77.6, which is close to the overall average score for all appliances, but lower than those from security vendors.
Vendor response mixed
According to Orca, all vendors were contacted by email but only 80 responded and the responses ranged from polite and professional to derogatory and even threatening. The company named Cisco, Intel, Dell, IBM, TrendMicro, Qualys and HailBytes among those who were professional in their responses. Dell issued a critical security advisory for one of its appliances and others updated or removed their vulnerable images.
On the negative side, 24 vendors said the vulnerabilities are not exploitable and declined to take action even though sometimes flaws can become exploitable at a later time in combination with newly discovered vulnerabilities. Patching all flaws is always good practice. Thirty-two vendors said it was the customers' responsibility to patch the flaws and a few responded with legal threats.
The main takeaway from the report is that even well-established software vendors, including companies from the security space, can ship virtual appliances that are not updated in a timely manner or which are not retired when the products they deliver reach end-of-life. Another conclusion is that choosing appliances from vendors who charge more does not mean they are more secure or better maintained, but neither are the open-source ones. Even with hardened virtual appliances, some had a C rating. This ultimately means enterprises should treat virtual appliance use with caution and run their own testing to ensure they are free of critical flaws before deploying them in production and giving them access to sensitive corporate data.
Orca recommends these steps to reduce security risks related to virtual appliances:
- Asset management can provide you with an understanding of the virtual appliances deployed across your organization’s IT estate. This must include both internal platforms and the public cloud. Don't overlook informal deployments (shadow IT), as it’s too easy for end users to access and deploy their own virtual appliances.
- Vulnerability management tools can discover virtual appliances and scan for known vulnerabilities and other security issues. Make sure the vulnerability management process in your organization scans all virtual appliances; you cannot assume they're safe to use as supplied by vendors.
- The vulnerability management process should prioritize actions to be taken by identifying the most severe vulnerabilities. In the short-term there are two choices: Fix a product or immediately stop using it.
- In the longer-term for those appliances kept running, approach the respective vendors, understand their support process and how arising vulnerabilities are fixed—if at all. Seek an alternative if a given vendor’s support processes are not satisfactory.