Situation Report: Water Facility Hack Underscores Resilience, Redundancy, and Deterrence

On Friday, February 5^th, an unknown hacker exploited a remote access software vulnerability utilized by the water treatment plant servicing Oldsmar City, Florida, in an attempt to poison the city’s water supply.

Fortunately, since the water utility employees witnessed the hacker’s actions in real time – increasing the amount of sodium hydroxide from 100 to 11,100 parts per million – they were able to take immediate action by intervening and reversing the harmful manipulation.

The water utility informed local law enforcement and federal agencies such as the FBI are now investigating the breach.  Florida Senator Marco Rubio called the incident, “a matter of national security.”

The hack is further evidence that criminals and adversaries are exploiting known vulnerabilities in America’s critical infrastructure.

The March 2020 Cyberspace Solarium Commission, chartered under the 2019 National Defense Authorization, was specifically directed to address cybersecurity. The introduction to the Commission’s report includes a fictional scenario of the results of hacked water systems:

“The rainbow of colors in the window paints how everything went so wrong, so fast. The water in the Potomac still has that red tint from when the treatment plants upstream were hacked, their automated systems tricked into flushing out the wrong mix of chemicals…That’s what you get from deciding in the 18th century to put your capital city in low-lying swampland and then in the 21st century wiring up all its infrastructure to an insecure network.… What can we really do? No matter what legislation we pass now, after everything that’s happened, we’re too late.”

Perhaps it’s not surprising that the Solarium Commission chose a water utility as target of their “attention gaining” cyberattack story. The Department of Homeland Security (DHS) Office of Cyber and Infrastructure Analysis (OCIA) conducted an analysis of vulnerability assessments that revealed:

“[A]mong surveyed critical infrastructure that depend upon water for core operations, services are degraded 50 percent or more within eight hours of losing drinking water services. The same holds true for a loss of wastewater treatment services. For example, the OCIA analysis noted that nearly all hospital functions could be degraded within two hours due to a loss of external wastewater discharge services. Yet, many infrastructure owners and operators do not have alternative sources of water or wastewater services.”

Meanwhile, the nation’s industries and human population rely primarily upon community water systems (CWS) to supply water and wastewater services.

“According to the Environmental Protection Agency (EPA), in fiscal year (FY) 2016, more than 300 million Americans – roughly 95% of the U.S. population – got at least some of their drinking water from a CWS,” explains Steve Bieber, Water Resources Program Director for the Metropolitan Washington Council of Governments.

These CWS facilities rely on a wide array of industrial control systems (ICS) to operate their facilities, many of which are vulnerable to external hacking.

Joe Weiss, a cybersecurity expert points to a “recent biannual risk report published by Claroty which showed, ‘[t]here were hundreds of industrial control system (ICS) vulnerabilities identified last year and more than 70% of them were remotely exploitable. Vulnerabilities were most prevalent in the critical manufacturing, energy, water and wastewater, and commercial facilities sectors.’”

Mr. Weiss noted, “This is not the first-time water systems have been hacked with intent to cause damage – Maroochy Shire in Australia in 2000, the Illinois water hack from Russia in 2011, and recently an undisclosed domestic US drinking water system where pumps and control setpoints were changed. This type of incident should be expected as many small water systems depend on remote access.”

However, it’s not just remote exploitation that threatens these CWS facilities and other critical infrastructures.  Weiss has consistently warned of supply chain attacks.

A supply chain attack is where the utility unknowingly installs equipment that is either counterfeit or contain built-in backdoors that could be exploited by an adversary.

“These vulnerabilities were all Internet-Protocol (IP) network-related and do not address any hardware backdoors…nor does it address the lack of cyber security in non-IP networks,” Weiss stated.

While Mr. Weiss’ warnings pertained to a supply chain attack surrounding a massive Chinese-built electric transformer, the same can be said about threats to community water facilities.

Because of these vulnerabilities to CWS systems, critical infrastructure owners – and people for that matter – should recognize the need for all-hazards resilient and redundant water supplies.

“Despite significant interdependencies on the water sector, many critical infrastructure owners and operators do not have adequate plans for alternative sources of water or wastewater service,” contributing author Steve Bieber warns in Powering Through: Building Critical Infrastructure Resilience.

Infrastructure targets are less lucrative to attack by criminals or adversaries when resilience and redundancies are built in, offering a form of deterrence.

“All of us are just one hack away from water insecurity if we don’t have multiple measures in place. Reminder that layers of redundancy are as important now as the day after 9/11,” notes the Honorable Ben Grumbles, Maryland Secretary of the Environment.

Because cyber threats routinely outpace cyber defenses it’s vital to establish significant deterrence as the Solarium Commission advocates: “a new strategic approach to cybersecurity: layered cyber deterrence.”