Antivirus Event Analysis Cheat Sheet v1.7

We’ve just released an updated version of our Antivirus Event Analysis cheat sheet. You can download version 1.7 here.

The major changes are:

  • Updated AV signature lists
  • Split AV signature cells into two columns to save space
  • Fixed and added some directory names
  • Extended file extension list
  • Google filename search
  • More Virustotal checks

SPARK uses Sigma Rules in Eventlog Scan

Sigma is a rule format for threat detection in log files. It is for log data what “Snort rules” are for network traffic or “YARA signatures” are for file data. It is easy to write and read. Writing a Sigma rule is a matter of minutes.

On the right you can see a simple Sigma rule that checks the “System” eventlog for traces of password dumper activity. The detection section contains 1+ identifiers (selection, keywords, quarkspwdump) that can be defined freely by the rule author. These selectors are used in the condition to build the rule.

It also contains a description, references, possible false positives and a level.

Analysts use Sigma to generate search queries for their SIEM or log management solution. The Sigma repo contains a converter that allows to convert the generic rules to ElasticSearch, Splunk, QRadar, Logpoint, Windows Defender ATP (WDATP) and ArcSight.

Wouldn’t it be great if you could apply Sigma rules on the endpoint?

Well, the upcoming version 1.14 of SPARK, which will be released at the end of July,  does that. It applies Sigma rules to the local Eventlog. This way you’re able to apply searches that you have once defined for your SIEM to the local Eventlogs.

This way you are able “query” the standalone systems that are not connected to your SIEM and uncover otherwise common blind spots in your environment.

 

We ship the current rule set, which is part of the public Sigma repository and contains more than 200 rules with our SPARK program package in an encrypted form. (*.yms)

You can add your own Sigma rules to the “./custom-signatures/sigma/” folder in the SPARK program directory.

To activate Sigma scanning, use the new “–sigma” parameter.

Currently only SPARK supports this feature and there are no plans to implement this in THOR as well.

The feature is currently free for all customers but may become a premium feature that has to be licensed separately by the end of the year depending on the customer’s plan. 

See the comparison table for a complete overview on all features.

New Antivirus Event Analysis Cheat Sheet Version 1.2

Today we release a new version of our “Antivirus Event Analysis” Cheat Sheet that helps you with the analysis of Antivirus events by providing a clear decision matrix.

We’ve updated many of the sections, added new VirusTotal online analysis checks and brought it in a new format.

You can download the PDF version here.

The Best Possible Monitoring with Sigma Rules

Some of you may already have heard of Sigma, a generic approach for signatures used in SIEM systems. Its main purpose is to provide a structured form in which researchers or analysts can describe their once developed detection methods and make them shareable with others.
Today I would like to describe another use case that helps us build the best possible signatures for any application.
In typical security monitoring projects, people integrate different log sources and threats/use cases they want to monitor in these logs. The selection of the log sources is determined by importance, availability and operational requirements. The central question is:

“What do I want to detect?”

There are several white papers and guides for the most common log sources like the Windows Eventlog. What you have to do is to read the documents, extract the interesting detection methods and define searches, panels and dashboards in your SIEM system. One of the main purposes of Sigma is to relieve you from this time consuming process. These public descriptions of detection methods are already part of Sigma or will be integrated by one of the contributors.
But today I’d like to point out a different use case that is less obvious but very useful.

The Best Possible Monitoring

Security analysts try to define useful searches and statistical analyses on the log data from all different log sources. They typically follow the public guides and create many different “failed logon” rules for all kinds of operating systems and applications.
But what happens when start integrating log sources and types that are not covered by public guidelines?
There are different approaches to this problem:

  • Cover failed authentication attempts
  • Monitor any failure event (with a base-lining)
  • Use statistic deviations (many new events of a certain type)
  • Look through the log data and select interesting messages
  • Talk to the application owners / administrators (often rather disappointing)
  • Talk with the application developers (in case of inhouse development)

All of these methods are valid and can help you setup useful searches on the available data. However, in order to create the best possible signatures we have to consider the following:

  • The most interesting and relevant fault conditions do not happen under normal conditions and therefore do not appear in everyday log data (the log data we have to select interesting events from)
  • Developers know the rarest or most relevant error conditions and the corresponding error IDs and messages
  • All possible error messages of that application appear in its source code

Imagine that we can extract the most interesting messages that the application is able to generate and define rules for these conditions. Imagine that this process is an easy task that any developer is able to perform.
Well, we have at our fingertips all the tools that are needed.

An Example: vsftpd

Let’s make this approach clearer. Under optimal conditions we ask the developers to extract the most relevant or abnormal error messages that the applications produces under conditions that indicate a manipulation or attack. Alternatively, we can do it ourselves and cross-check the results with the available log data to avoid false positives.
I checked the source code of the FTP server ‘vsftpd’ for error messages that indicate rare error conditions and fatal failures.

The resulting Sigma rule looks like this:

title: Suspicious VSFTPD error messages
description
: Detects suspicious VSFTPD error messages that indicate a fatal or suspicious error that could be caused by exploiting attempts
reference
: https://github.com/dagwieers/vsftpd/
author
: Florian Roth
date
: 2017/07/05
logsource
:
    product
: linux
    service
: vsftpd
detection
:
    keywords
:
        - 'Connection refused
: too many sessions for this address.'
        - 'Connection refused
: tcp_wrappers denial.'
        - 'Bad HTTP verb.'
        - 'port and pasv both active'
        - 'pasv and port both active'
        - 'Transfer done (but failed to open directory).'
        - 'Could not set file modification time.'
        - 'bug
: pid active in ptrace_sandbox_free'
        - 'PTRACE_SETOPTIONS failure'
        - 'weird status:'
        - "couldn't handle sandbox event"
        - 'syscall * out of bounds'
        - 'syscall not permitted:'
        - 'syscall validate failed:'
        - 'Input line too long.'
        - 'poor buffer accounting in str_netfd_alloc'
        - 'vsf_sysutil_read_loop'
    condition
: keywords
falsepositives
:
   - Unknown
level
: medium

After some testing we can now share the rule with the world in order to build a large signature database for the all different types of applications and services.
We could also try to convince developers to provide Sigma rules with their applications to help organisations establish the best possible monitoring of their software. I was thinking about an error message extractor that would parse a software project and extract error message strings that contain certain keywords like “failure”, “fatal”, “permission denied”, “refused” and others to facilitate this process for the rule writer.
If you want to support the project, create a pull request or send me your rules as gists or via pastebin to @cyb3ops. You could also become a contributor by providing a new backend for the rule converter “Sigmac” or become a member of our Slack team for threat hunters (references required).

Not All IOC Scanning Is the Same

In the recent months I had several talks with friends and coworkers about IOC scanning and how to integrate IOCs from threat intel feeds into our scanners or other products that our customers already use. People often tell me that EDR or client management product X already does IOC scanning and that we don’t need to check for these indicators a second time. Especially when it comes to network wide sweeps for traces of activity due to an ongoing incident I recommend scanning a second time with one of our scanners or a tool of similar quality.
This blog post explains why.
People usually spend a fair amount of time on selecting threat intel feeds and interesting indicators for their scans. However when it comes to the actual application of these indicators they seem to be satisfied with the simplest form of checks.
Especially when we look at C2 or Filename IOCs I can easily explain the difference between the “compulsory” and “freestyle” methods of IOC scanning.
IOC scanning automation
A plain “compulsory” filename IOC check would walk the disk or query a database looking for a certain filename, right?
However if you think about it for a second and ask yourself “where else could we check for that filename?” you’ll realize that the following elements could also contain the malicious filename:

  • Eventlog entries (e.g. process starts, service installs with image path, access failures …)
  • Log files (local Antivirus log file, access to file in web root > web server access log, backup errors, PowerShell history …)
  • Registry (recently opened files, shell bags, service image path, other caches …)
  • MFT (deleted entry)
  • Archive content (packed in ZIP file)
  • WMI (scripts – e.g. see this PoC by Matt Graeber)
  • Crash dumps
  • Windows Error Report (WER – file names and content)
  • Free disk space (filename as content of batch files or other scripts, scheduled tasks …)

Actually we often see that during lateral movement attackers access systems, run their tools remotely, copy the output, delete the output files and leave no file system traces behind. We use the locations that I mentioned above and others to detect them using their tools although all the files have been removed from disk. That’s the “freestyle” method.
The same counts for the C2 IOCs. The “compulsory” plain method would check the system’s network connections. The “freestyle” method also includes checking for these C2 IOCs in the following locations:

  • Process memory (C2 strings loaded and decrypted in process memory)
  • Log files (web server access logs, Windows firewall log file, AV module log file …)
  • Hosts file
  • Files (in backdoor config files on disk)
  • Registry (hard coded C2 server in registry key)

I am sure that digital forensics experts would come up with other fruitful locations. It is just sad to see those great indicators feed into tools that do “IOC scanning” only to get another check mark in a product comparison table – aka the “compulsory” way.

If all you have is a hammer, everything looks like a nail.

So – the next time when someone tells you that their tool checks for IOCs on the endpoint, your question should be “How and where do you check for these IOCs?”.

How to Fall Victim to Advanced Persistent Threats

During the last four years, I was engaged on incident response teams for several large advanced persistent threat (APT) cases involving different German corporations. In this time, we have developed methods and tools to detect compromised systems, while also planning and performing remediation. During the course of these investigations, I noticed that certain circumstances supported the chance that a corporation would fall victim to advanced persistent threats. In this article I want to focus on favorable preconditions that nearly ensure a successful APT attack.
The recommendations in this article were discussed and extended with the help of red team leaders and fellow CERT team members. This is an article inspired by a book named “Anleitung zum Herzinfarkt” by Bernhard Ludwig. This book provides serious guidance on how to increase the risk of dying from heart attack in someone’s early years. I found that this principle of a ‘humorous reversed guide’ could be useful to describe the typical pitfalls and mistakes that we regard as crucial for the development of advanced persistent threats with the aim to help organizations, which have not been hit by this type of attacks so far.

Don’t Create Network Segments

At first, strictly avoid placing systems in different network segments. Instead – keep it simple. Let standard Windows workstations, admin workstations, server systems, print servers, industrial control systems, backup servers, network management systems, monitoring servers, terminal servers, mobile management servers, development systems, IP cameras, house automation, and SIP telephones be in a single huge network in order to avoid firewalling issues.
That principle applies also to subsidiaries and affiliates. After the acquisition of other companies, don’t waste time asking them for compliance with your security policies. Save time and connect their network directly with your backbone! Make sure to allow any type of connection and create domain trusts as fast as possible to enable cross-domain resource access to your data center in Berlin from your user workstations in Brazil, as well as the call center in Bangladesh.
If you are required to create network zones for some reason, interconnect these zones with mutual Windows domain trust relationships. (see Titanic as an example)
why-titanic-sank
In order to maintain an environment of mutual trust and respect, never monitor the gateways between segments for port scanning, network sweeps, or other types of suspicious network activity. Also, please do not commit the new business unit to provide technical support in cases of suspicious activity and incident response.

Don’t Limit Privileged Account Usage

You trust your administrator, don’t you? So, let administrators use their privileged accounts to surf the web and read email while they are connected via SSH and RDP to their domain controllers and Internet facing servers in DMZ networks. Do not use hardened and intensely monitored jump servers from special admin workstations to manage your most important server networks.
Don’t waste precious resources creating administrative roles. If someone needs to install a text editor on a server, add him to the Domain Administrators group and make sure that no one ever figures out why you did this.
Do not use Microsoft’s solution called LAPS to ensure that all computers have different, complex, local administrator passwords or Privileged Admin Workstations (PAW) to provide a dedicated operating system for sensitive tasks. This would give attackers an unnecessarily hard time finding highly privileged credentials and fewer opportunities to use them.

Don’t Collect and Analyze Useful Log Data in a Central Location

Attackers tend to make themselves familiar with the environment and therefore cause strange peaks in log volumes that could be easily detected by an over-attentive employee – so, don’t log security events. If you do log events for any reason, make sure that all of your systems keep their logs in a local log file and do not transmit them to a central SIEM system.
IMG_6213
If you have to use a SIEM system, use Active Directory for authentication, so that attackers can find all the fine log data in a central place. If you have any problems on a server, such as with disk space due to the fact that you configured your golden image for Windows servers to use only 20GB of space on drive ‘C:’, don’t hesitate to reduce the log file size to a minimum. That may cause the log to overwrite itself every 30 minutes, but hey, each line that is overwritten can help prevent successful detection.
On Linux systems configure your log file rotation to keep 7 days, not more. Your IT staff typically needs a couple of days investigate the system owner and have some meaningful discussion with your data protection officer.
If you have to comply with certain policies that require the collection of log data, you still can do a lot to make sure that breaches remain undetected for months. Here is our Top 10:

  1. Only log and report high amounts of failed logins, because attackers tend to use valid credentials after taking the first hop
  2. Disregard all antivirus events that have the status “deleted” or “moved to quarantine”. They are gone, so they won’t trouble you.
  3. Do not collect the logs of client workstations to avoid detecting zero-day attacks. Detecting them would cause pressure to take action.
  4. Do not search for anomalies in your log files because hacker activities generate some very special or completely new log types. If you look for them you could find them – so again, don’t do that.
  5. Do only collect the log files at system level and disregard the logs of applications running higher up.
  6. Do not spend time understanding your organization’s structure, logging completeness, and logging behavior of your assets. It’s usually pretty complex and the less you know the better.
  7. Use the default logging configuration on your systems. Maybe you miss the most relevant event types, but if they really were relevant, wouldn’t they be in the default anyway?
  8. Monitor and report attacks on your Internet-facing firewall to tie up valuable resources with useless pie charts.
  9. Do not collect the logs of SysInternals Sysmon, AppLocker, Windows Defender, or Microsoft EMET. The protection provided from their use, especially when combined, is far too effective.
  10. Let the data protection officers and workers’ council decide on what you’re allowed to log and analyze.

Use your Active Directory for All Types of Authentication

Centralization is good. It saves resources and simplifies user administration. Use Active Directory authentication for everything: the logins to your proxy servers, network devices, online certificate authorities, virtual machine consoles, administrative jump hosts, security monitoring servers, VPN servers, SIEM system, and last but not least – backup servers.
This ensures that attackers take over the complete infrastructure after compromising a single outdated member server that has domain admin logon sessions. Remediation becomes much more exciting when attackers have access to proxies, DNS servers, and mail gateways all at once!

Don’t Regard the Client Workstations as the First Line of Defense

Since corporate workstations operate deep in the internal network, which are well protected by several firewalls and proxy servers, consider them to be far outside of the danger zone. Don’t audit them as frequently as you do with DMZ servers. There are so many more clients than servers that it is certainly not efficient to scan them all. Also, random samples never give a valid result, so let’s just drop the whole thing.
Don’t patch workstation software like Microsoft Office, PDF viewers, JAVA and Flash plugins, media players, and archivers as soon as a patch is published. Don’t use exploit protection, like Microsoft EMET, in order to increase the risk of zero-day attacks and finally,  don’t deactivate content in document viewers, such as JavaScript in Acrobat Reader or VBA in Microsoft Office.
Simplify administration of user workstations by granting standard users local administrative rights.
Also allow workstations to access the Internet directly on any service port. Don’t use a proxy server and do not monitor dropped connections on typical Trojan back connect ports. Developers and administrators are highly skilled professionals that tend to download and install suspicious software from the shadiest websites. Remember: You trust them and if they think that they need that (suspicious) software than of course they do. To increase the probability of such events, block Sourceforge, Github, and the whole category “Software Downloads” on your Internet proxy server.

Don’t Mind Antivirus Alerts

As mentioned before, do not consider antivirus alerts that have the status “deleted”, “cleaned”, or “moved to quarantine”. A deeper analysis of these events could reveal a Trojan that had control of the system for weeks before the right signature returned a match on components of a hack tool set that attackers moved from server to server. Therefore, do only check for errors and unresolved operational issues.
Make sure that no one pays special attention to antivirus events that report “Hack Tools”, “Password Dumpers”, or “Scanners”. Tell everyone that this would cause too many false positives because system administrators need these tools to find their assets and regain access to them.
FullSizeRender
Also avoid rating antivirus events according to an evaluation method. That’s so bureaucratic.
To allow attackers the greatest possible leeway, create vast exclusion lists and don’t use special Antivirus functions like PUA scanning or application controls that block password dumpers from accessing the memory.

Handle Web Servers Like Other Server Types

Regard web servers like any other server type. Frequently patching the web server service is perfectly sufficient. Do not audit the applications running on that web server; if you have to due to corporate policies, do it once, print the report, and archive it.
Place the web servers behind reverse proxy servers and tell everyone that this will protect them. If you repeat that constantly everyone will believe it one day. Do not protect the web servers with costly Web Application Firewalls and don’t collect the logs of such a system for central attack detection and analysis.
Allow developers to access the management interfaces, like JMX or Tomcat Manager, from remote locations. Don’t log access to these applications and don’t ban source IPs on security violations. Tell everyone that each developer may access the servers anytime, from anywhere, to reduce operational risks.
If you have to run Apache or Tomcat on a server, choose Windows as the operating system. Do not use limited user accounts to run the web server services in order to maximize the impact of a successful attack. Last but not least: Avoid annoying security features like SELinux.

A Few Last Words

I know that it is hard to guarantee a successful APT attack and that all of these recommendations require a certain amount of stubbornness and resistance to advice; however, even if this advice does not guarantee falling victim to advanced persistent threats, chances increase exponentially the more of my advice you apply. Good luck – I’ll keep my fingers crossed.
If you have further ideas that you want to share, please comment on this article or contact me on Twitter @cyb3rops.

Credits

Many thanks to Stephan Kaiser for the idea, Julia Stolz and Jeff for major reviews and Matthias Kaiser (@matthias_kaiser), Daniel Sauder (@DanielX4v3r), Thomas Patzke (@blubbfiction), Claas Rettinghausen, Robert Haist (@SleuthKid), Alexander Döhne for their valuable feedback.

YARA Rules to Detect Uncommon System File Sizes

YARA is an awesome tool especially for incident responders and forensic investigators. In my scanners I use YARA for anomaly detection on files. I already created some articles on “Detecting System File Anomalies with YARA” which focus on the expected contents of system files but today I would like to focus on the size of certain system files.
I did a statistical analysis in order to rate a suspicious “csrss.exe” file and noticed that the size of the malicious file was way beyond the typical file size. I thought that I should do this for other typically abused file names based on this blog post by @hexacorn.
I used my VT Intelligence access and burned some searches to create this list.

System Files and Sizes

System Files and Sizes


You can find a spread sheet of this list here. It can be edited by everyone.
I created some YARA rules that use the external variable “filename” to work. LOKI and THOR use the “filename” and other external variables by default.
UPDATE 23.12.15 4:50pm:
I’ll update the list on the LOKI github page. For a current version of the YARA signatures visit this page.

rule Suspicious_Size_explorer_exe {
    meta:
        description = "Detects uncommon file size of explorer.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-21"
    condition:
        uint16(0) == 0x5a4d
        and filename == "explorer.exe"
        and ( filesize < 1000KB or filesize > 3000KB )
}
rule Suspicious_Size_chrome_exe {
    meta:
        description = "Detects uncommon file size of chrome.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-21"
    condition:
        uint16(0) == 0x5a4d
        and filename == "chrome.exe"
        and ( filesize < 500KB or filesize > 1300KB )
}
rule Suspicious_Size_csrss_exe {
    meta:
        description = "Detects uncommon file size of csrss.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-21"
    condition:
        uint16(0) == 0x5a4d
        and filename == "csrss.exe"
        and ( filesize > 18KB )
}
rule Suspicious_Size_iexplore_exe {
    meta:
        description = "Detects uncommon file size of iexplore.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-21"
    condition:
        uint16(0) == 0x5a4d
        and filename == "iexplore.exe"
        and ( filesize < 75KB or filesize > 910KB )
}
rule Suspicious_Size_firefox_exe {
    meta:
        description = "Detects uncommon file size of firefox.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-21"
    condition:
        uint16(0) == 0x5a4d
        and filename == "firefox.exe"
        and ( filesize < 265KB or filesize > 910KB )
}
rule Suspicious_Size_java_exe {
    meta:
        description = "Detects uncommon file size of java.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-21"
    condition:
        uint16(0) == 0x5a4d
        and filename == "java.exe"
        and ( filesize < 140KB or filesize > 900KB )
}
rule Suspicious_Size_lsass_exe {
    meta:
        description = "Detects uncommon file size of lsass.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-21"
    condition:
        uint16(0) == 0x5a4d
        and filename == "lsass.exe"
        and ( filesize < 13KB or filesize > 45KB )
}
rule Suspicious_Size_svchost_exe {
    meta:
        description = "Detects uncommon file size of svchost.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-21"
    condition:
        uint16(0) == 0x5a4d
        and filename == "svchost.exe"
        and ( filesize < 14KB or filesize > 40KB )
}
rule Suspicious_Size_winlogon_exe {
    meta:
        description = "Detects uncommon file size of winlogon.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-21"
    condition:
        uint16(0) == 0x5a4d
        and filename == "winlogon.exe"
        and ( filesize < 279KB or filesize > 510KB )
}
rule Suspicious_Size_igfxhk_exe {
    meta:
        description = "Detects uncommon file size of igfxhk.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-21"
    condition:
        uint16(0) == 0x5a4d
        and filename == "igfxhk.exe"
        and ( filesize < 200KB or filesize > 265KB )
}
rule Suspicious_Size_servicehost_dll {
    meta:
        description = "Detects uncommon file size of servicehost.dll"
        author = "Florian Roth"
        score = 60
        date = "2015-12-23"
    condition:
        uint16(0) == 0x5a4d
        and filename == "servicehost.dll"
        and filesize > 150KB
}
rule Suspicious_Size_rundll32_exe {
    meta:
        description = "Detects uncommon file size of rundll32.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-23"
    condition:
        uint16(0) == 0x5a4d
        and filename == "rundll32.exe"
        and ( filesize < 30KB or filesize > 60KB )
}
rule Suspicious_Size_taskhost_exe {
    meta:
        description = "Detects uncommon file size of taskhost.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-23"
    condition:
        uint16(0) == 0x5a4d
        and filename == "taskhost.exe"
        and ( filesize < 45KB or filesize > 85KB )
}
rule Suspicious_Size_spoolsv_exe {
    meta:
        description = "Detects uncommon file size of spoolsv.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-23"
    condition:
        uint16(0) == 0x5a4d
        and filename == "spoolsv.exe"
        and ( filesize < 50KB or filesize > 800KB )
}
rule Suspicious_Size_smss_exe {
    meta:
        description = "Detects uncommon file size of smss.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-23"
    condition:
        uint16(0) == 0x5a4d
        and filename == "smss.exe"
        and ( filesize < 40KB or filesize > 140KB )
}
rule Suspicious_Size_wininit_exe {
    meta:
        description = "Detects uncommon file size of wininit.exe"
        author = "Florian Roth"
        score = 60
        date = "2015-12-23"
    condition:
        uint16(0) == 0x5a4d
        and filename == "wininit.exe"
        and ( filesize < 90KB or filesize > 250KB )
}

I ran this rule set over my goodware database and got only a few false positives. Feel free to use these rules wherever you like but please share new rules or statistical analyses on other system files.

Yara System File Checks - False Positives

False Positives

Splunk Threat Intel IOC Integration via Lookups

Today most security teams have access to a lot of different information sources. On the one hand they collect log data from different sources and try to correlate them in a useful way in so-called SIEM systems. On the other hand they receive threat information from different sources like APT reports, public or private feeds or derive those indicators from their own investigations and during incident response.
Therefore one of the main tasks of security monitoring today is to combine these different data sources, which means to apply the threat intel information to the data that is already available in SIEM systems or scan for it on-demand using tools like my free IOC scanner LOKI or our APT Scanner THOR.
In this article I would like to describe a method to apply threat intel information to log data in Splunk using simple lookup definitions.
(more…)

Detect System File Manipulations with SysInternals Sysmon

SysInternals Sysmon is a powerful tool especially when it comes to anomaly detection. I recently developed a method to detect system file manipulations, which I would like to share with you.
We know how to track processes with the standard Windows audit policy option “Audit process tracking”, but Sysmon messages contain much more information to evaluate. By using Sysmon on many systems within the network and collecting all the logs in a central location you’ll get a database full of interesting attributes and Metadata which can be statistically analyzed in order to identify anomalies.
Carlos Perez wrote a really good article on Sysmon, which you should check out if you’re new to Sysmon and its capabilities.

Anomaly Detection

In recent years “anomaly detection” has often been used as marketing buzzword and as a result lost some of its shine. I am still a strong believer and often phrase sentences like “anomaly detection is the only method to detect yet unknown threats”. In security monitoring we call it anomaly detection, Antivirus vendors call it heuristics and SPAM appliances evaluate it in a “X-Spam-Score”.

Anomaly detection requires the ability to describe what is normal and exclude it from the evaluation.

With the data collected from the different Sysmon sources, this is an easy task to do. Sysmon provides the executable hash as MD5, SHA1 or SHA256 in the log entries that enables an analyst to identify the few different versions of a certain system executable. A hash of a system program like “cmd.exe” executed on the different systems on your domain should always be the same on all systems running the same version of Windows. But let me give you some examples.
A sane system environment analysis for the “cmd.exe” would look like this:

Hash - Image - Count
3C77C39347A6FA560A74587B0498FE84 - C:\WINDOWS\system32\cmd.exe - 56
AD7B9C14083B52BC532FBA5948342B98 - C:\Windows\System32\cmd.exe - 34

The following analysis includes an anomaly, which is worth to be investigated:

Hash - Image - Count
3C77C39347A6FA560A74587B0498FE84 - C:\WINDOWS\system32\cmd.exe - 56
AD7B9C14083B52BC532FBA5948342B98 - C:\Windows\System32\cmd.exe - 34
D8B7B276710127D233ABCDB7313AAC36 - C:\WINDOWS\system32\cmd.exe - 1

Let’s take a look at two analysis examples in which I use this method to identify different anomalies.

Anomaly 1: “StickyKeys” backdoor and the like

I use my favorite log analysis system for the analysis, which is Splunk. Getting the Sysmon data into splunk is easy as there is already a Sysmon Add-on available in the App Store. Just use the deployment manager to push the Add-on to the Splunk Forwarders and install Sysmon. (see my other blog post on Sysmon for more appropriate configuration options)
Then you can do things like that:

SOURCE="WinEventLog:Microsoft-Windows-Sysmon/Operational" NOT Image=*Sysmon.exe | dedup host,Image | stats distinct_count(Image) AS different_names,VALUES(Image),VALUES(host) BY Hash | sort -different_names

It gives you an overview of files with the same hash but different names. It is pretty easy to spot the manipulation.

StickyKey Backdoor Detection with Splunk

StickyKey Backdoor Detection with Splunk and Sysmon


We detected a so called “StickyKeys” backdoor, which is a system’s own “cmd.exe” copied over the “sethc.exe”, which is located in the same folder and provides the Sticky Keys functionality right in the login screen. Replacing it with a system command line establishes a shell running as LOCAL_SYSTEM that pops up when you RDP to a server and press 5 times shift consecutively. (see this blog post for more information on this backdoor)

Anomaly 2: The Black Sheep

If you create the statistics by “Image” instead of “Hash” you’ll get an overview of the different versions of system files in use and are able to identify system file versions that are unique.
Look at the following example to get an impression what can be done with this method.

SOURCE="WinEventLog:Microsoft-Windows-Sysmon/Operational" NOT Image=*Sysmon.exe | dedup host,Image | rex FIELD=Image "(?<executable>[^\\\]+)$" | eval Executable=LOWER(Executable) | stats COUNT BY Executable,Hash | sort +COUNT

I am sorry but I can’t give you a nice screenshot on what would it look like in a big environment. These are the results from 3 different demo systems only (Win2003, Win7 and Win8), but in order to see what it would look like in a environment with hundreds or thousands of systems, see the listing below.

Sysmon Detection Splunk

Sysmon Anomaly Detection with Splunk


The result would look like this:

Hash - Image - Count
AD7B9C14083B52BC532FBA5948342B98 - cmd.exe - 1480
3C77C39347A6FA560A74587B0498FE84 - cmd.exe - 256
D8B7B276710127D233ABCDB7313AAC36 - cmd.exe - 2

Consider the image files with a low count as anomalies and try to figure out, why the hash of the system executable is different from the variants on the other systems.
I would google the hash of the black sheep, which is “D8B7B276710127D233ABCDB7313AAC36” and see if I can get more details. An empty google result is NOT a good sign as some may be inclined to believe. If the google results are ambiguous you should try to figure out if these systems are somehow special – e.g. certain readout system on embedded OS versions, systems that do not receive patches. If the findings are still suspicious you should drop the samples in a sandbox and see how they behave.
Hope you liked it. Please give me feedback if you actually tested this method in your environment so that I can improve the search statements or handle false positive conditions.