The Department of Energy, CISA, the FBI, and the NSA jointly issued an advisory describing a sophisticated piece of malware called Pipedream that’s designed to attack a wide range of industrial control systems. This is clearly from a government, but no attribution is given. There’s also no indication of how the malware was discovered. It seems not to have been used yet.
More information. News article.
Ted • April 14, 2022 12:08 PM
@RealFakeNews
Maybe you could contact the FBI and tell them they’re wrong?
Douglas Turner • April 14, 2022 12:51 PM
I have to wonder if this is something we developed that was found in the wild.
Clive Robinson • April 14, 2022 12:52 PM
@ Bruce, ALL,
This is clearly from a government, but no attribution is given.
Sorry but that realy is an assumption.
There is a reason Prof Ross J. Anderson came up with the three levels of Attackers and made Level thre “State level and others”.
There are depending on who’s list you use around 200 states in the World. The majority of which have a GDP below that of the Silicon Valley Corps.
There is more money to be made in working in industry than for Government. Untill fairly recently a large number of “companies” providing the smaller nation states with “technical expertise”, that the respective states education systems were not provided their Governments with. Thus hundreds of millions are known to have changed hands and like as not ten to a hundred times that.
A mistake many ICT Analysts, gurus and spokes persons have been making for years is the assumption that Government writes covert APT and others write low hanging fruit criminal malware…
That is just not the case. Some of the more interesting vulnarability exploiting code, comes via “Private Enterprise”, where a lot of money can be made very quickly.
Some of the vulnerability discovery and exploit code comes out of Argentina, not from the Government but individuals, who can sell their vulnerabilities without having it stolen or most of the money stolen by a Government.
Unfortunately certain very disreputable types got involved. We know them better as “Venture Capitalists” thair aim is effectively a “Pump and Dump” ?exercise. They basically buy into a small technology company throw money at them to build them up to appearing to be way more than they realy are then selling them on.
This leaves the managment having to meet expectations they realistically can not legally…
Which is why things are turning a bit sticky currently.
But “Programable Logic Controlers”(PLCs) are about hardware reliability not applications and functionality.
Many PLC’s are little more than “Ladder Logic Sequencers” that would have once been programed with “wire wrap guns” all that has realy changed at that level is that the wrap frames and counters have been replaced with programable logic and basic state machines.
All the “modern stuff” is not down at the PLC nore how it interfaces with “Supervisory Control and Data Acquisition”(SCADA) systems, that’s not realy changed fundementally this century (except for MUX). Where change has been is where SCADA front ends for “Human Computer Interface”(HCI) etc has been designed to use PC components and connect in like an “Office PC”, with all the insecurities that brings.
I was working on the “bleeding edge” of that change four decades back when PC’s were still 16bit. And the 16bit CMOS 8086 Intrinsically Safe “Remote Telemetry Unit”(RTU) I designed was very much not just “state of the art” but “world leading technology” (to my surprise it was still advertised as for sale a couple of years back).
The problem with all such systems is they tend to be designed by Engineers for Engineers and Technicians, and simplicity and human readable protocols tend to be the name of the game not security, or even AuthN / AuthZ. That is once you are below a certain level it’s assumed you should be there and know what you are doing (an issue that went horribly wrong for mainframes and mini-computers in the 60’s and 70’s, and PC’s in the 80’s and 90’s.
It looks to many that “Industrial Control Systems”(ICS) are only just “catching up”. But… there are other systems such ad Avionics and AeroSpace BackPlanes that are even further behind…
So expect to be seeing the same old same old for the next decade or three.
Oh and please do not talk about IoT systems for ICS, that realy is a state of madness the world could definitely do with out… But unfortunatly “bean coubters” and “venture capatalists” will no doubt force on us… As they have already done wirh some “Security and Access Control”(SAC) Systems for “physical security”.
John • April 14, 2022 2:11 PM
@Clive,
One system I am familiar was turning on and off a gas cutting torch. The processor of choice had an undocumented instruction that did not increment the PC.
The engineers called it the loop and catch fire instruction!
Not sure what they did to ‘solve’ that problem!
Designing reliable systems that keep running is non-trivial.
I understand that working safely under fault and no data conditions was the force pushing ADA into certain military systems.
One of my friends justifies resetting her phone by saying it was just a computer that needed to be reset from time to time!!
John
Ted • April 14, 2022 2:20 PM
@Clive
What do you make of Dragos’ assessment?
Dragos assesses with high confidence that this capability was developed by a state-sponsored adversary with the intention to leverage PIPEDREAM in future operations.
Dragos designated the activity group CHERNOVITE.
Ted • April 14, 2022 3:05 PM
Mandiant thinks this activity is consistent with Russia’s interest in ICS. They think INCONTROLLER (aka PIPEDREAM) “poses the greatest threat to Ukraine, NATO member states, and other states actively responding to Russia’s invasion of Ukraine.”
https://www.mandiant.com/resources/incontroller-state-sponsored-ics-tool
SpaceLifeForm • April 14, 2022 4:31 PM
@ Ted, Clive
CVE-2022-26809 is probably related. See recent squid.
Ted • April 14, 2022 5:30 PM
@SpaceLifeForm, Clive
Over 700k Windows machines have port 445 open to the internet? Yeesh. Unpatched this would be awful for ransomware.
https://isc.sans.edu/forums/diary/An+Update+on+CVE202226809+MSRPC+Vulnerabliity+PATCH+NOW/28550/
The CISA ICS advisory mentioned an ASRock motherboard driver that could compromise Windows-based engineering workstations. Lateral movement. 👀
Clive Robinson • April 14, 2022 5:50 PM
@ SpaceLifeForm, Ted,
Re : CVE-2022-26809
A logrithmic risk rating of 9.8 out of 10, whilst not quite an end stopper, is certainly a “shoe stopper” to put it another way “it’s open door day in downtown Chicago”…
@ Ted,
I gave up,trying to understand Mandient’s resoning years ago. They were at one time just mouthing the US Gov “1 from the ‘Must be one of China, Iran, North Korea, Russian’ list” political favourite of the day, which was frankly ludicrous.
It got embarasing when it was fairly obvious wrong…
As for Dragos’ assessment, “insufficient information to make a valid assesment”.
I follow basic scientific reasoning. The first part of which is,
1, full disclosure of source material.
2, Full disclosure of source aquirment and measurands.
3, Full disclosure of methodology.
And a few more steps.
Anything that basically says “looks like XYZ” gets zero rated as far as I am concerned. Because it’s easy to just “copy or fake” because you know what you want it to look like. Look on it as a similar method to malware writers running their code over and over through AV software with little tweaks untill they get it to pass right through.
If you write a filter that says “Russian” and people can run their code through it over and over then it will come out “looking Russian by those measures” not that it is Russian.
Atribution is hard especially when vulnerability exploitation can be launched from anywhere and made to look the same as other code “by style”. Style is famously unreliable at even the best of times as it is either too subjective or too easy to fake.
If you want to get atribution right you have to go “old school” and put “boots on the ground with HumInt” or some other “side channel” that is directly applicable to individuals.
Of all the attribution methods so far made public it takes only a few moments thought as to how to either render them usless or fake them up.
So attribution is much like the 1980’s ECM, to ECCM to ECCCM battle. With,
“Each Counter measure geting counter measured”.
There are known ways to deal with it upto a point, but as the cost grows exponentially with each step it quickly became way too expensive as a method.
Ted • April 14, 2022 8:36 PM
All attribution aside, I was hoping someone might comment on how noteworthy the malware is.
PIPEDREAM is the seventh known ICS-specific malware following STUXNET, HAVEX, BLACKENERGY2, CRASHOVERRIDE/ INDUSTROYER, TRISIS/TRITON, and INDUSTROYER2.
Apparently, it’s the first to be identified before it was used.
Roland • April 14, 2022 11:36 PM
@Ted
That RPC remote code execution vulnerability could have existed where it is courtesy of Microsoft. But thank you for posting. Reading that info gave me an idea of what to do.
Clive Robinson • April 15, 2022 9:22 AM
@ Ted, ALL,
Apparently, it’s the first to be identified before it was used.
You left the word “publicly” out before “idebtified”.
There have been many attacks on the public facing communications interfaces and systems that suround SCADA front ends.
Most fail without comment or investigation. Those that do get investigated are those that AV companies pull from their “samples repositories”.
As there is way way more in those repositories than there are people to look at them, then a certain prioritisation is to br expected.
So I would expect to find way down any priority list malware aimed at ICS and SCADA systems, waiting to be investigated “someday, maybe”.
Remember “known evidence of existance” is most certainly not required for “existance” that’s the way it works in the real world of pathogens… From prions, through viruses, bacteria, and all sorts of larger beasties as large as insects and some vertibrates there are things out there that kill not just other animals and us as well that we know little or nothing about. Heck there are new species of tape worms being discovered just hanging around in the most unpleasent of circumstances… As for those that lay eggs in hosts…
Ted • April 15, 2022 10:36 AM
@Clive
Yeah, Mandiant’s discovery methods seem resource-intensive. But what do I know.
A lot of your – and @SpaceLifeForm’s – advice for mitigating these risks seems to jive with the nature of ICS protection.
What is unnerving to me is seeing that the malware is using industrial network protocols, such as OPC UA; Modbus; Codesys.
Dragos believes the most likely targets are equipment in liquefied natural gas (LNG) and electric power environments. Does preventing an attack avoid escalation? Who knows.
Clive Robinson • April 15, 2022 1:08 PM
@ Ted,
What is unnerving to me is seeing that the malware is using industrial network protocols, such as OPC UA; Modbus; Codesys.
I’ve mentioned the reason for why ICS comms protocols are targeted by attackers. Usually,
1, They are simple plaintext you can drive from a Serial Terninal or equivalent.
2, They either totally lack or have very poor AuthN and AuthZ.
3, They are very close to the hardware thus have direct control advantages.
Thus from an attackers point of view it’s as if not easier than getting into an EMail server back in the 1980’s…
Also almost as importantly even very large ICS are broken down into very small related blocks maybe as little as a valve and a flow guage, or a motor and a rev counter, with a simple f(x) mapping model that provides linearised control. You can do a simple “straight line aproximation” with just a couple of linrs to even power law systems and get “close enough” (as it’s Easter, think 20mins per pound plus 20mins to roast a leg of lamb in an oven starting at 200C/Gas7 for 20mins then turn down to 180/Gas5 as a simple example)…
anon • April 16, 2022 7:09 PM
@Clive
PLCs…ugh. I have not had to network them in quite some time, but in the 1990’s and early 2000’s you could DoS a PLC by pinging it at a rate of 10pps…
name.withheld.for.obvious.reasons • April 16, 2022 9:56 PM
No surprise here, have reported extensively about industrial systems, and especially power generation and transmission systems. Did I tell you the one about taking a whole site down (30MWatts), by affecting one power interface (the primary and backup)? Slammed the plant down with a thundering crash–you could feel the shutdown. It’s the hardest shutdown possible on a generation station–it is never done intentionally. But the real concern is the level of visibility into operational security elements. From the networks, hosts, servers, control interfaces, PLC, and surrounding administrative and maintenance elements. This along with the physical plant security, the level of vulnerability is above what I would consider nominal.
Marc • April 18, 2022 8:16 PM
Why would this be a Government and not Industrial Espionage? Whenever cyber attacks start to involve systems outside of the IT realm I get suspicious of attributions. Also every APT, particularly Russian, always has to have a government agency involved.
Private industry and business seem to have a lot more available infrastructure and the ability to host 100-1000s of civilians outside of government reach. The overall reach of APTs in 2022 is FAR FAR more than all the specialized cyber agencies could possibly manage. It’s becoming ridiculous. And assuming a hell of a lot of competence on the part of the agencies. You could imagine maybe 2 projects per agency (for competition purposes) realistically. Because we are receiving the information from mostly US private business, who really knows.
Clive Robinson • April 19, 2022 3:43 AM
@ Marc,
Why would this be a Government and not Industrial Espionage?
If you take the very reasonable view that “espionage” is a “technical activity” then the actions of a “directing mind” are fairly irrelevant when you are building defences.
Quite a few years ago now Prof Ross J. Anderson defined three technical levels of attacker.
Level three got named by many as “state level” rather than “those with resources equivalent to or exceading those of a small state”.
As we’ve seen the resources needed are actually quite minimal in terms of “a brain and a computer and aquired knowledge”. The, real expense comes from maintaining the body that maintains the brain over what could be several years.
Thus “the market” built around paying those who “got lucky” and found a zero day first.
With something approaching 10,000 vulnerabilities a month getting “officially” recorded, you have to ask the obvious question of how many “unofficial” vulnerabilities there are being discovered on top of that…
You can also assume that outside of “ego food” and “CV guilding” a goodly percentage of those unofficial vulnerabilities are changing hands for quite large sums of money. So the insentive to not only keep them “unofficial” but as a lucrative income stream is very definitely there.
More importantly if you assume you will only get one valuable vulnarabiliry every few years, you are not going to want to be working in a tax regime that assumes “regular income” that massively hits any “windfall” for close to 90%.
So the “purchase market” is going to move out of the West, and into small almost invisable entities in second world countries, where a dollar goes way way further than it does in NY/Washington etc.
Which surprise surprise is what we are seeing. Places like Argentina, Brazil and Mexico, may not be many peoples idea of a good place to have a family etc… But the reality is if your idea of happiness does not involve “glitz, glamour, Celeb” then you can live very comfortably and quite anonymously in pleasant weather and suroundings etc. Similar with many parts of Eastern Europe.
Clint • April 27, 2022 3:20 AM
An industrial control system is a great tool for boosting the industry in the region. Moreover, implementing such solutions enables industrial companies to drive sustainability in the region by eliminating paperwork, saving man-hours on inspections, etc. Here is a great success story of renewables implementing ICS – https://fluix.io/industry-renewable-energy
Subscribe to comments on this entry
Sidebar photo of Bruce Schneier by Joe MacInnis.
RealFakeNews • April 14, 2022 11:14 AM
So far, the “attribution” of any given attack or software to do so hasn’t failed in how it is mis-used for propaganda effect.
Just yesterday you posted how a cyber attack was discovered before it was even launched against Ukraine and absolutely knew without any doubt whatsoever that it was Russia “that did it”, and even who wrote it.
Yet, here we are… a piece of malware for attacking industrial systems has been discovered (again, before use), but mysteriously “no attribution” beyond “it appears to be a Government”.
Well… if enough is known to know “it is a Government”, but no attribution is given, then the logical conclusion is it is by someone we don’t want to upset (5-eyes, Israel, or China). India perhaps also, but otherwise who is left on the very short list of Governments with the time, money, and importantly, inclination to develop such attacks?
Seeing as so little ois actually known, and the fact it wasn’t used yet, what is even the point in reporting it was found? It is safe to assume that certain Governments are always finding ways to attack systems. Might as well throw darts at a dartboard.