Stellar Cyber: Log4j Vulnerability and Exploitation Detection

1. Introduction

In the past several days, a serious Log4j vulnerability (CVE-2021-44228, CVE-2021-45046) has nearly led to a perfect storm in the Internet world. As a widely used Java logging utility with an easily exploitable vulnerability, Log4j has no doubt made IT professionals and companies nervous and many questions have been raised – What is this vulnerability? How can I know if our system is vulnerable? Has my IT infrastructure already been breached? What can I do to prevent future attacks leveraging this vulnerability?

At Stellar Cyber, we have been closely monitoring the situation, and we are here to provide our takeaways and advice to our current and prospective customers and partners as they navigate through the uncertainties brought by this Log4j vulnerability.

2. Impact and Mitigation

According to CVE-2021-44228, any Apache Log4j2 prior to v2.15.0 is affected by the vulnerability due to an unchecked string interpolation with Java Naming and Directory Interface (JNDI) lookup. An attacker who has the knowledge to inject data into log messages can craft and inject a special formatted JNDI interpolation expression to load and execute arbitrary code from the specified JNDI endpoint (e.g., an LDAP server) when the interpolation expression is evaluated by Log4j.

The impact of this vulnerability is expected to be broad due to the wide use of Log4j in many popular software applications and its easy exploitability. The security community has been tracking websites, software, open-source components, and other manufacturers using Log4j, and confirmed that even some big-brand companies were vulnerable.

To mitigate this vulnerability, we advise our customers and partners to update their existing Log4j to v2.15.0, in which the JNDI-related behavior is disabled by default. If immediate upgrade is not an option, another mitigation is to set system property log4j2.formatMsgNoLookups to true (applicable to v2.10+) or to remove the JndiLookup class from the classpath (applicable to Log4j prior to v2.10 via zip -q -d log4j-core-\.jar org/apache/logging/log4j/core/lookup/JndiLookup.class*).

In addition, we recommend that our customers and partners update their existing Java to at least version 6u211, 7u201, 8u191, or 11.0.1 in which JNDI remote class loading is disabled by default.

For specific software applications using Log4j and affected by this vulnerability, please reach out to the software vendors to apply any appropriate patches as soon as possible.

3. Technical Details

In CVE-2021-44228, there are two essential and relevant parts that constitute the vulnerability: Java Naming and Directory Interface (JNDI) with remote method invocation functionality, and Log4j’s support for log interpolation with JNDI lookup.

Java Naming and Directory Interface (JNDI, Figure 1) is an ancient but still useful feature dating back to Java 2 v1.3. It provides naming and directory functionality to Java applications, so an application may invoke through JNDI various service providers such as Lightweight Directory Access Protocol (LDAP). For instance, a Java application may pass a URL ldap://some-ldap-server:389/o=SomeObjectID through JNDI to a supported LDAP server some-ldap-server to find and invoke a remote object SomeObject.

On the Log4j side, JNDI lookup support was first introduced in v2.0 back in 2013 as requested by its users for sophisticated logging features. Since then, Log4j has allowed developers to use string interpolation with JNDI in log messages. For instance, one may write logger.error("${jndi:ldap://some-ldap-server:389/o=SomeObjectID}") , and Log4j
would evaluate the interpolation expression and invoke JNDI accordingly to retrieve data that constitute the log message from a remote server.

Finally, eight years later on November 30, 2021, the Log4j team was made aware of the remote code execution vulnerability as a result of the combination of log interpolation with JNDI lookup. One week later, nearly everyone in the security community and IT industry got informed and started to panic one way or another.

The consequence of this vulnerability would be particularly bad not only because Log4j is so widely used, but also because people have been using Log4j in a way that is too perfect and too easy for attackers to exploit.

As we can see from the aforementioned example of logger.error("${jndi:ldap://some-ldap-server:389/o=SomeObjectID}")
, to successfully exploit this vulnerability, an attacker needs to know what kind of injected data will eventually be logged somewhere by a victim system using Log4j. In reality, it is low-hanging fruit for any attackers with minimal familiarity with modern network-based applications, in which there is a common practice to log every request including its URL, query string, and user agent.

At Stellar Cyber, we have been monitoring exploit attempts made to our customers and partners in the past week. Our machine learning-powered User Agent Anomaly Detection has already detected anomalous user agent strings such as:

Among the those samples, we find that attackers have been investigating different ways to exploit the vulnerability – not only by using the vanilla form, but also by coming up with different variations to evade rule-based Web Application Firewalls that try to match ${jndi:ldap*.

Although it has not been a known case where attackers exploited the vulnerability and successfully caused severe consequences, it is still notable that even a simple probing, e.g., an attempt to exploit the vulnerability with DNSLog to probe whether a target system is vulnerable, could still be damaging since attackers may be collecting future assets at the moment.

4. How Stellar Cyber Can Help

At Stellar Cyber, we provide multiple tools to our customers and partners to defend against attacks exploiting CVE-2021-44228. Through our threat intelligence provider, Stellar Cyber Security Sensor (SDS) already has the up-to-date signatures to detect typical exploitations of CVE-2021-44228.

4.1 Stellar Alerts on Exploitation Detections

We currently offer the following alert types that detect a new IDS signature or a spike of IDS signatures:

  • – Public to Private Exploit Anomaly

  • – Private to Public Exploit Anomaly

  • – Private to Private Exploit Anomaly

  • – Public to Public Exploit Anomaly

Please watch for these alerts with "log4j" in the alert description or ids.signature field.

4.2 ATH to Keep Track of Matched Apache Log4j Signatures

If you would like to keep track of all instances of matched Apache Log4j signatures, you may consider creating the following rules (Caution: it might create a large number of alerts):

  • – Search ML-IDS/Malware Sandbox Events index with the query ids.signature: log4j
    .

4.3 URL Reconnaissance and User Agent Anomaly Detections

With attackers exploring different variations of exploitations, our AI-powered anomaly detections (Figure 2) also help detect indirect or unusual attempts.

We find some indirect attempts from URL Reconnaissance Anomaly Detection with Log4j in part of URLs to exploit the vulnerability through Java web applications. Such attempts would usually involve an attacker scanning for pages to exploit, leading to an anomalous number of HTTP 4xx errors, which can be detected by URL Reconnaissance Anomaly Detection.

Since User-Agent strings are a major means of exploitation according to our observations, our User Agent Anomaly Detection is also a useful tool. It detects User-Agent strings that have never been seen before or have been seen very rarely, thereby being able to provide additional defense against the rising trend of exploitation. To search "log4j" directly in all alerts might be reasonable. (Caution: this is a very expensive query, please do not try on raw data which might lead to data lake overload.)

4.4 Incident Correlation

An attempt to exploit CVE-2021-44228 may just be a single action resulting in a single alert from our detections. In order to further investigate and keep track of potential ongoing attacks related to the easily exploitable Log4j, we provide an additional Incidents feature (Figure 3) to correlate a set of multiple correlated alerts and entities constituting a potential unified attack (i.e., an incident) for better security visibility and manageability. We use machine-learning capabilities to generate incidents automatically, grouping related alerts into a unified incident for improved attack resolution.

As we can see in Figure 4, which shows a detailed timeline view of an incident with CVE-2021-44228 exploitation, our Incident Correlation automatically puts together three highly correlated alerts from three different detections ordered by the time they happened: first a User-Agent anomaly where an attacker attempted to inject a malicious User-Agent string from 10.11.191.95 to 10.11.190.88, followed by an abnormal process spawning detected on 10.11.190.88, and ending with a command anomaly executing a dangerous command xargs -r -0 rm -f

5. Conclusions

In this post, we have shared our takeaways of CVE-2021-44228 and have offered advice to our current and prospective customers and partners, and we have also shown how Stellar Cyber can come to the rescue in this uncertain period of time. As a final reminder, be sure to patch and update any impacted software, including Log4j (to v2.15.0+) and Java (to at least 6u211, 7u201, 8u191, or 11.0.1)