On Dec. 9, word of a newly discovered computer bug in a hugely popular piece of computer code started rippling around the cybersecurity community. By the next day, nearly every major software company was in crisis mode, trying to figure out how its products were affected and how it could patch the hole.
The descriptions used by security experts to describe the new vulnerability in an extremely common section of code called log4j border on the apocalyptic.
“The log4j vulnerability is the most serious vulnerability I have seen in my decades-long career,” Jen Easterly, security director of the U.S. Cybersecurity and Infrastructure Security Agency, said in an interview last week on CNBC.
So why is this obscure piece of software causing so much panic, and should regular computer users be worried?
Log4j is a chunk of code that helps software applications keep track of their past activities. Instead of reinventing a “logging” — or record-keeping — component each time developers build new software, they often use existing code such as log4j instead. It’s free on the Internet and very widely used, appearing in a “big chunk” of Internet services, according to Asaf Ashkenazi, chief operating officer of security company Verimatrix.
Each time log4j is asked to log something new, it tries to make sense of that new entry and add it to the record. A few weeks ago, the cybersecurity community realized that by simply asking the program to log a line of malicious code, it would execute that code in the process, effectively letting bad actors grab control of servers that are running log4j.
Reports differ when it comes to who first raised the alarm about the vulnerability. Some people say it surfaced in a forum dedicated to the video game Minecraft. Others point to a security researcher at Chinese tech company Alibaba. But experts say it’s the biggest software vulnerability of all time in terms of the number of services, sites and devices exposed.
The fact that log4j is such a ubiquitous piece of software is what makes this such a big deal. Imagine if a common type of lock used by millions of people to keep their doors shut was suddenly discovered to be ineffective. Switching a single lock for a new one is easy, but finding all the millions of buildings that have that defective lock would take time and an immense amount of work.
Log4j is part of the Java programming language, which is one of the foundational ways software has been written since the mid-90s. Huge swaths of the computer code that modern life runs on uses Java and contains log4j.
Cloud storage companies such as Google, Amazon and Microsoft, which provide the digital backbone for millions of other apps, are affected. So are giant software sellers whose programs are used by millions, such as IBM, Oracle and Salesforce. Devices that connect to the Internet such as TVs and security cameras are at risk, as well. Even NASA’s Ingenuity helicopter on Mars uses it.
Hackers who try to break into digital spaces to steal information or plant malicious software suddenly have a massive new opportunity to try to get into nearly anywhere they want to. That doesn’t mean everything will be hacked, but it just got a lot easier to do so — just as if the locks on half of the homes and businesses in a city suddenly stopped working all at once.
On top of all that, the vulnerability is straightforward to take advantage of. In the Minecraft video game, it’s as easy as typing a line of malicious code into the public chat box during a game. On Twitter, some people changed their display names to strings of bad code, Wired reported.
The vulnerability also gives hackers access to the heart of whatever system they’re trying to get into, cutting past all the typical defenses software companies throw up to block attacks. Overall, it’s a cybersecurity expert’s nightmare.
Computer programmers and security experts have been working night and day since the vulnerability was publicized to fix it in whatever piece of software they’re responsible for. At Google alone, more than 500 engineers had been going through reams and reams of code to make sure it was safe, according to one employee. That process was being repeated at all kinds of tech companies, spawning an entire new genre of memes from coders lamenting the hellish week they’ve been through.
“Some of the people didn’t see sleep for a long time, or they sleep like three hours, four hours, and wake back up,” Ashkenazi said. “We were working around the clock. It’s a nightmare since it was out. It’s still a nightmare.”
Hackers have been working just as hard as the security experts to exploit log4j before the bug gets patched. Cybersecurity software company Checkpoint said in a blog post that it saw hackers send out 60 different variations of the original exploit in a single 24-hour period. Hackers have already tried to use it to get into nearly half of all corporate networks around the world, Checkpoint said.
Most of the hacking has focused on hijacking computers to run bitcoin mining software, a tactic that hackers have used for years to make money, but on Dec. 15, Checkpoint said Iranian state-backed hackers used the vulnerability to try to break into Israeli government and business targets.
Though the bug was present for years, it’s unlikely criminal hackers have known about it until now, because if they had security experts, they would have spotted it being used before, said William Malik, vice president at cybersecurity company Trend Micro. That doesn’t mean that more sophisticated government hackers, such as those working for the U.S., Russia, Israel or China, haven’t used it before, he said.
CISA has given federal civilian agencies a Dec. 24 deadline for patching log4j. But even with engineers working around the clock to meet that deadline, hackers will have potentially found their way into hundreds of thousands of services and sites by then, Ashkenazi said. In some cases, hackers will install “back doors” or malicious code that stays put even after the initial log4j problem gets fixed. Identifying and removing those back doors will be a whole separate task for security experts.
And not everyone will fix the problem in the first place. Getting an entire industry to update a specific piece of software quickly is next to impossible. Many companies won’t end up doing it, or will think they aren’t affected when really they are.
That means log4j could be a problem for years to come, leaving a door open for hackers who want to run ransomware attacks or steal people’s personal information.
To take advantage of the vulnerability, hackers have to deliver malicious code to a service running log4j. Phishing emails — those messages that try to trick you into clicking a link or opening an attachment — are one way to do so. Keep an eye out for an influx of phishing messages in the coming days, Malik said, as hackers scramble to plant bad code in as many places as possible.
If you get an email saying that your account has been compromised or your package failed to deliver, don’t open any links or attachments. First, make sure you actually have an account with that company or were expecting mail from that carrier. Then, find a real customer service number or address online and reach out that way.
The best thing regular computer users can do is make sure the apps they use are updated to their most recent versions, Malik said. Developers will be sending out patches over the coming days to fix any log4j issues, and downloading those quickly will be important.
For the most part, consumers should just wait and let the experts fix their software programs.
“Sit back, take a deep breath. It’s not the end of the world,” Malik said. “It’s going to be very busy the next few days for security folks.”