It's been a rough couple of weeks for providers of networking and computing infrastructure left to explain why they should be confident their IT systems haven't been compromised by the US's National Security Agency (NSA).
Ever since the late December disclosure in Germany's Der Spiegel that the NSA's Tailored Access Operations (TAO) group was planting back doors in just about every leading vendor's gear, US partners' phones have been ringing off the hook with urgent questions about the security and integrity of installed systems.
Given that Der Spiegel, reporting on a security conference address by independent security specialist and hacker Jacob Appelbaum, named the likes of Dell, HP, Cisco, Juniper, Apple, Microsoft, Western Digital, Huawei, Seagate, Samsung, Maxtor, and Oracle, the crisis in confidence touches just about every reseller and their customers (see list of specific gear said to be compromised by the NSA at the bottom of this page).
Most of the vendors have been quick to distance themselves from this NSA mess, issuing terse denials that they knew about the NSA intrusions or assisted in them in any way. The responses are cold comfort to providers that need to tell clients what happened and if anything can be done about it.
None of the vendors on the record so far is offering any prescription for dealing with equipment that according to the German magazine were sometimes diverted by the NSA during shipping to secret TAO workshops -- where malware and rogue hardware components were installed to facilitate agency access.
Hacks into the parcel-shipping businesses of carriers like UPS, FedEx and the US Postal Service are among the NSA's "most productive operations" when it comes to infiltrating IT systems worldwide, according to documents in Appelbaum's presentation.
In reality, most US businesses served by the channel are likely at little risk from the cloak-and-dagger stuff revealed by NSA leakers.
As distasteful as the NSA TAO tactics sound, the leaked documents appear to be aimed at what the agency considered "target" persons, agencies or companies.
Unless a company or its principals are on some sort of secret, no-fly list for IT purchases, there appears to be little chance their equipment ever got the TAO treatment.
For those partners who want to craft client responses that specifically reference the vendors' comments, here is a rundown of which kit was alleged by Der Spiegel to be affected, and the official vendor responses we have received since New Year's Eve.
Named by Der Spiegel: Cisco Pix and ASA (Adaptive Security Appliance) firewalls, 5505, 5510, 5540, 5550 (firmware implant)
Cisco's chief security officer John Stewart came out strongly in a blog post that categorically denied any collusion with the NSA to crack its gear. "We are deeply concerned with anything that may impact the integrity of our products or our customers' networks and continue to seek additional information," wrote Stewart.
"At this time, we do not know of any new product vulnerabilities, and will continue to pursue all avenues to determine if we need to address any new issues. If we learn of a security weakness in any of our products, we will immediately address it. We do not work with any government to weaken our products for exploitation, nor to implement any so-called security 'back doors' in our products."
Cisco has since referred customers seeking additional information on the NSA issue to its Cisco Security Response unit.
If there is any levity at all in this whole sordid affair, it may have been in a slide detailing how an NSA program called ARKSTREAM was taking advantage of security weaknesses in Microsoft's error and crash reporting features to gather data about user's machines. The slide poked fun at Redmond's own error message, replacing it with the words "This information may be intercepted by a foreign SIGINT system to gather detailed information and better exploit your machine."
In an email statement sent to the press, Microsoft, which is on the record demanding more transparency from the NSA, said it "does not provide any government with direct or unfettered access to our customer's data. We would have significant concerns if the allegations about government actions are true".
Named by Der Spiegel: HP ProLiant 380DL G5 servers (hardware implant)
HP executives said they were never aware of NSA efforts to compromise their gear, adding that the vendor has "no reason to believe that the HP ProLiant G5 server mentioned was ever compromised as suggested in the article".
HP said it was not aware of any of the information presented in the Der Spiegel article.
"HP's privacy and security policies are quite clear: we do not knowingly develop products to include security vulnerabilities," it said in a statement. "We are also active in testing and updating our products regularly to eliminate threats and make our products more secure. HP takes the privacy and security of our customer information with great seriousness. We will continue to put in place measures to keep our customers' information confidential and secure."
Named by Der Spiegel: Dell PowerEdge 1850 / 2850 / 1950 / 2950 RAID servers with BIOS versions A02, A05, A06, 1.1.0, 1.2.0, or 1.3.7 (BIOS exploits)
Dell PowerEdge 1950 / 2950 servers (hardware implant, JTAG interface)
While the NSA claimed the ability to hack two outdated Dell servers, the Dell PowerEdge 1950 and Dell PowerEdge 2950, the Texas vendor dismissed the idea that its equipment was at risk and categorically denied assisting the NSA.
Dell's VP of global security John McClurg said in a blog post that Dell "has a long-standing commitment to design, build and ship secure products and quickly address instances when issues are discovered. Our highest priority is the protection of customer data and information, which is reflected in our robust and comprehensive privacy and information security programme and policies.
"We take very seriously any issues that may impact the integrity of our products or customer security and privacy," McClurg wrote. "Should we become aware of a possible vulnerability in any of Dell's products we will communicate with our customers in a transparent manner as we have done in the past. Dell does not work with any government - United States or otherwise - to compromise our products to make them potentially vulnerable for exploit. This includes 'software implants' or so-called 'back doors' for any purpose whatsoever."
Named by Der Spiegel: Juniper Netscreen ns5xt, ns25, ns50, ns200, ns500, and ISG 1000 firewalls
Juniper SSG 500 and SSG 300 firewalls (320M, 350M, 520, 550, 520M, 550M).
JUNOS (Juniper's customised version of FreeBSD) on all J-Series, M-Series, T-Series routers
Part of Appelbaum's disclosure in Der Spiegel involved the NSA's claimed ability to insert malicious instruction into equipment BIOS, making the hacks persistent and difficult to remove through conventional security methods. The BIOS hacking hit vendors like Juniper hard and featured prominently in the company's denials.
"Juniper Networks is not aware of any so-called "BIOS implants" in our products and has not assisted any organisation or individual in the creation of such implants," the company said in a statement sent to the press. "Juniper maintains a Secure Development Lifecycle, and it is against Juniper policy to intentionally include 'back doors' that would potentially compromise our products or put our customers at risk.
"We take allegations of this nature very seriously and are working actively to address any possible exploit paths. As a company that consistently operates with the highest of ethical standards, we are committed to maintaining the integrity and security of our products. We are also committed to the responsible disclosure of security vulnerabilities, and if necessary, will work closely with customers to implement any mitigation steps."
Named by Der Spiegel: Huawei Eudemon 200, 500, and 100 series firewalls (installed as a boot ROM upgrade).
Moreover, the document says that Huawei routers are targeted, as part of a joint operation between the NSA and the CIA to exploit Huawei equipment (project: TURBOPANDA).
If there's a winner on the tech vendor side in the NSA mess, it might just be the much-maligned Huawei, which has spent the past year fending off allegations that it might be a true threat to security IT networks. As a result of their quarrels with the US government, Chinese-owned Huawei doesn't have to work as hard to prove it is not cooperating with the NSA. But they did manage to have a dig at their critics in their standard email response to the Der Spiegel affair.
"As we have said in the past, and as the media reports seem to validate, threats to network and data integrity can come from any and many sources," Huawei spokespeople wrote. "While the security assurance programmes we have in place are designed to deter and detect such malicious activity, we will conduct appropriate audits to determine if any compromise has taken place and to implement and communicate any fixes as necessary."
In response to reports that the NSA had compromised the ubiquitous iPhone and could now pull call logs, text messages, contact lists and other files, Apple released a statement saying it never cooperated with the spy agency on "any of our products, including iPhone".
"Whenever we hear about attempts to undermine Apple's industry-leading security, we thoroughly investigate and take appropriate steps to protect our customers," the Apple statement said. "We will continue to use our resources to stay ahead of malicious hackers and defend our customers from security attacks, regardless of who's behind them."
Lest you think the lowly hard drive was immune from NSA infiltration, the Der Spiegel article goes into some detail on how the agency could infect the devices with malicious firmware that allowed remote compromise and additional malware loading.
On behalf of itself and its newly acquired asset Maxtor, Western Digital issued a fairly tepid email response to media inquiries:
"Western Digital has no knowledge of, nor has it participated in, the development of technology by government entities that create 'implants' on Western Digital hard drives, as Der Spiegel described," the company said.
As part of our special editorial relationship, CRN is republishing this article from Channelnomics
Security firm set to become part of acquisitive Shearwater Group
Distributor merges three northern sites into one new hub in Warrington
Activist investor puts forward five director candidates as turmoil continues at security giant
Nima Green asks what is driving public cloud uptake in Germany