btw the attacker here has a screen name that sure sounds chinese. My guess is that this person is not in fact chinese. I talk to a lot of security teams who want to block their app teams from using open source components that have chinese contributors (as an example), and they were thinking of a scenario exactly like this, and oh man they are going to absolutely jump to the wrong conclusions from this.
like, they’re implicitly assuming random attacker is going to sign his commit “[email protected]” when trying to slip a backdoor in a PR.
guys I don’t want to be alarmist but yall really don’t know how close we came to a worldwide meltdown on this thing. we only avoided it because some total freak noticed a slight dip in his database benchmark times using the latest test packages, and made a decision to investigate (the timing variation was extremely slight and he absolutely could have just shrugged it off).
This particular backdoor was insanely bad (basically would have allowed unlimited remote access on any systemd-based linux system running sshd, which is like all of them) and was extremely sophisticated (it was slipped in via a compression library that sshd doesn’t even link to directly (systemd bridges the gap here). The account that slipped it in had been building up credibility over three years of work on this project, it’s an extremely long con. None of the automated QA processes caught this. This was mostly restricted to upstream and a couple of bleeding edge testing distros that nobody runs in production, but if it had gone unnoticed, it would be rolling out to production systems in a couple of months.
now for the really scary part. this is 100% a state actor, it’s not a lone wolf, there are already sockpuppet accounts being dug up, it is a mortal lock that other projects have also been infiltrated by sleepers even if they haven’t had any compromise yet.
I’m not exaggerating when I say this dude probably saved us from literally TRILLIONS of dollars of economic damage, and possibly even prevented WW3. Dude is basically Stanislav Petrov, he noticed something that didn’t make sense, make a judgement call, and saved the world.
I guess when this guy says “foreign government” it’s not 100% clear if that means “foreign to America” (looks like he’s Polish but has worked for American companies) but I don’t see why we should assume USA#1 shouldn’t be on the list of possible state actors behind this sort of thing. (I wouldn’t put us at the top of the list for a couple reasons, but we’re definitely on it!)
yeah good catch, I’ve been discussing this with a bunch of people and it’s absolutely possible this was a US operation but everyone agrees some nation state that is hostile to the US is more likely (not sure exactly if this is just vibes or what, that sort of assessment is outside of my area by a long ways).
This shit is why I hate having to white knuckle the last 10 years of my career hoping the Russians or Chinese or whoever doesn’t want to take my operation. Really wish the globalists or whatever elite cabal runs the world would do something to disincentivize this stuff.
Yeah vibes for me too, like I assume we would be less likely than other countries to put a backdoor like this that makes everyone (including, as you noted, American companies in particular) vulnerable, but it’s certainly not out of the realm of possibility.
vi is an old school Linux text editor, like a command line Notepad.
It is notorious because it takes some arcane key combinations to get into various edit modes, and you need to use the right arcane key combinations to exit the program. Sometimes novices will feel trapped in there and can’t get out, and throwing their computer in the garbage seems like the only way to escape its clutches.
Compared with doing shit in emacs, vi’s syntax is positively elementary, but I grant that it is unique in that it’s the only “basic” text editor where if you just start typing, nothing gets written, and also, quitting isn’t obvious.
It is a text editor with modes, to enter text you need to be in edit or input mode, to exit the program you need to be in some other mode whose name I never learned. Here are the steps to enter “hello world” in VI or VIM:
Open Vi
type i to get into insert mode
type hello world
type esc to get our of insert mode
type : to get to, I don’t know, command mode
type q! to quit without saving
In trying to come up with those instructions I locked up my terminal program and had to restart it.
