THOR 8.53 Feature: Diff Mode

With the upcoming version 8.53 of THOR, we’re testing a new feature called “Difference” or “Diff” mode (–diff).

The idea behind “Diff” mode is that a scan could be much faster, if it would only consider elements that have been created or changed since the last scan on that system. We can apply this principle to various modules and increase scan speed massively.

Diff mode is currently supported in the long running modules

  • Filesystem – files with MAC timestamps older than the last scan (start) will be skipped
  • Registry – registry keys with last modification dates older than the last scan (start) will be skipped
  • Eventlog – runs until it reaches eventlog entries with timestamps older than the last scan (start)

Diff mode requires the use of THOR DB, which is the default but could have been disabled with “–nothordb”. This is necessary to determine information from the last scan, e.g. “when did it start” but also “which modules were used in the last scan”.

The main advantage is an incredible fast scan. Our tests showed that scans in “Diff” mode complete within 5 and 15 minutes. In “Diff” mode, the longest running module is “ProcessCheck” with run times between 3 and 6 minutes.

The main disadvantage of “Diff” mode is the inability to detect Timestomping attacks, in which attackers or malware changes the timestamps of files and other elements.

Important Update Process Changes

As we have announced in May, the old “thor-upgrade.exe” is already out-of-support and the old update servers accessed by “thor-upgrade.exe” will be decommissioned at the end of October.

The new all-round utility “thor-util.exe” now supports all of the features provided by the old “thor-upgrade.exe” including NTLM Authentication with corporate proxy servers.

You can use “thor-util.exe upgrade –help” to see all options of the “upgrade” feature.

Also note that “thor-util” has an “encrypt” feature that allows you to encrypt custom signature files and the “report” feature that creates HTML reports from plain text log files. 

The only valid update servers that should be accessible to get updates from November onward are:

  • update1.nextron-systems.com
  • update2.nextron-systems.com

The “thor-util” utility is part of the THOR and SPARK packages and can also be downloaded from the Customer Portal in the “Downloads” section.

If you are a customer and don’t have access to the Customer Portal yet, please contact us or the respective partner.

New Feature: THOR-util and SPARK-Core-util Signature Encryption

The new THOR-util version 1.2.4 supports the encryption of your custom signatures so that you can deploy your own IOC files and YARA rules in an encrypted form.

We use a public key in the utilities to encrypt the files for our scanners so that admins, Antivirus engines and attackers won’t be able to read the contents of the files.

 

The feature is also available in SPARK Core, our free scanner.

After encryption, place the encrypted IOC files in the “./custom-signatures” directory and the encrypted YARA rules in the “./custom-signatures/yara” directory.

The use of the function is simple. Just point it to a file, a list of files or use wildcards to select a set of files for encryption. The extension of the output file depends on the extension of the input file.

  • IOC Files: .txt > .dat
  • YARA Rules: .yar > .yas
  • Sigma Rules: .yml > .yms

Examples:

thor-util.exe encrypt case44.yar
thor-util.exe encrypt case44-hashes.txt
thor-util.exe encrypt case44-hashes.txt case44.yar
thor-util.exe encrypt case44.*

You can use the “upgrade” feature in both tools to get the newest version of the utility.

thor-util upgrade

ASGARD IOC Management

The upcoming ASGARD version 1.5 comes with a IOC management section in which you can manage your own set of IOCs in text files, YARA and Sigma rules.

You can then select each of the folders when creating a new scan run with THOR or SPARK. Selecting one of these folders will not include the sub folders.

You can schedule and run scans with different IOC, Sigma and YARA rule sets. You can review the included custom signatures in the scan details. 

The following features are not yet implemented in v1.5 but on the roadmap for ASGARD v1.6:

  • Signature verification
  • Exclude the standard rule set (shipped with THOR and SPARK)

THOR Version 8.49.0 Changes

There are a few relevant changes in the upcoming THOR version 8.49.0 that we would like to announce.

Interpreter and Module Upgrades

The integrated Python interpreter will be upgraded to Version 2.7.15. We have also upgraded several modules. All our tests showed no signs of problems even with the oldest Windows version like Windows 2003 Server. (officially unsupported)

If you encounter any issues, please let us know.

4th Generation License Format Support

THOR 8.49.0 supports the newest license format which allows us to:

  • set a start date for the period of validity
  • enable or disable certain modules and features in THOR and SPARK
    (e.g. we could license a SPARK version that only scans endpoint logs with Sigma rules)

THOR-util Report Generation

The new included THOR-util version 1.2 allows to generate HTML reports from scan log files. It can also generate reports for a directory that contains THOR or SPARK scan logs (up to 50 per HTML report). We’ve discussed this feature in detail in a previous blog post.

Noresume Becomes the New Default

The Scan Resume feature has caused many problems during incident response engagements in the past. The feature activates a journal in THOR DB that tracks the state of the scan and resumes the scan automatically if it was interrupted by a user or terminated due to a system shutdown. This feature seemed to be helpful but actually caused some problems.

THOR logs are created in “write” (w) mode, not in “append” (a) mode. When an administrator started THOR on a system, terminated the scan and then restarted it shortly after, the first part of the local log file was overwritten by the second scan. Sometimes a scan was interrupted on a system due to different reasons. When an administrator received the order to start a new scan on that system, the scan resumed the last scan and the log file and report contained only info of the resumed part of the scan.

We therefore decided to not resume scans by default. If you still want to maintain the old behaviour, please use the new “–resume” parameter. The old “–noresume” parameter is still valid but has no effect and is marked “obsolete” in the help.

Analysis Cockpit Web Session

We’ve just recently published a web session that gives an overview on our whole product portfolio and describes the features of our Analysis Cockpit in detail. (18 minutes, English language)

The main features of the Analysis Cockpit are:

  • THOR / SPARK Log Baselining
  • Automatic case creation based on similarities of the events
  • Filtered Forwarding of Logs to a SIEM system

THOR-Util with HTML Report Generation

The new version of “thor-util” (used with THOR/SPARK) / “spark-core-util” (used with SPARK Core) support a feature that allows a user to convert any scanner log file into a convenient report. 

  • Convert THOR / SPARK / SPARK Core scan logs into HTML reports
  • Convert a single text log file into an HTML report
  • Convert multiple log files (50 max.) in a directory into a single HTML report 
  • Provide a file with filters to suppress false positives in the reports
  • Even LOKI logs can be converted (no support)
  • Hash values linked to Virustotal searches
  • IP values linked to VirusTotal searches
  • Header sections linked to elements via ankers

You can access this feature in the upcoming enterprise products (THOR 8.47.2 and SPARK 1.13) and the free product SPARK Core (SPARK Core 1.13). 

The following screenshot shows a typical text log file. It can be processed in log analysis solutions but it is difficult to read for an analyst. Most analysts search these log files for “(Alert|Warning):” or use grep to get the most relevant messages.

Our tools “thor-util” and “spark-core-util” will help you with this task. 

Generate an HTML report for a single log file

<br /> thor-util report --logfile PROMETHEUS_thor.log<br />

Generate an HTML report for multiple log files

thor-util report --logdir ./logs

You can also provide a file with regular expressions that are applied during log parsing as filters to suppress false positives in the reports. 

The new tools will be in all productive packages at the end of this week. 

THOR Util Replaces THOR-Upgrade

We are currently upgrading our update infrastructure in many different ways.

We have added 2 new dedicated update servers – update1 (Karlsruhe, Germany) and update2 (Lenexa, USA). The old update locations will still be supported for a few months but have to be regarded as obsolete.

As a customer, please make sure to allow the following update servers in your proxy / firewall:

  • update1.nextron-systems.com (443/tcp)
  • update2.nextron-systems.com (443/tcp)

In this regard, our old utility called “thor-upgrade.exe” will be out-of-support by the end of July 2018.Please make sure to use the “THOR util” for all update tasks.

Major changes:

  • Supports all download types (THOR, SPARK for Windows, Linux, macOS)
  • Verifies Download via RSA signature
  • Runs on all platforms (Windows, Linux, macOS)
  • Allows updates and the download of a full program packages with config files
  • No support for proxy NTLM authentication

It is already part of all download packs.

Since THOR v8.46.9 and SPARK v1.11 all binaries are signed with a 2048 bit RSA key. The signatures are integrated in the download packs as separate “*.sig” files.

The new version 1.1.6 of THOR util checks the signatures during the upgrade / download and interrupts the process if an invalid signature is found.

You can verify the signatures yourself, by using the the new “verify” function.
These changes make our updates more reliable and secure.

If you have any question, don’t hesitate to contact us via support@nextron-systems.com

THOR 8.44 features TLS Syslog Transmission & ZIP YARA Scanning

The new THOR version 8.44 comes with some interesting new features.

TLS/SSL Syslog Transmission

THOR version 8.44.0 supports the Syslog log transmission in an SSL/TLS encrypted form. Just set the value “TCPTLS” as protocol in the 4th position of the target definition.

thor.exe -s mysyslogserver:6514:SYSLOG:TCPTLS

The documentation has been updated accordingly.

TLS Syslog Log Transmission

ZIP YARA Scanning

Until today the ZIP file checks were limited to file name IOC or anomaly checks. The new version 8.44.2 supports the scanning of ZIP file contents with the YARA rule base. However, for the time being the ZIP YARA scanning has some limitations:

  1. The feature is limited to files which decompressed size does not exceed the defined maximum file size (default 4.5 Megabytes)
  2. The feature is limited to certain scan modes: –intense, –fsonly, –dropzone

If the feature proves to be stable, we will activate it in the default scan mode in a future minor release.

ZIP YARA Scanning

New THOR / SPARK License Packs

We have just recently released new, flexible and practice-oriented license packs for our scanners THOR and SPARK. These license packs will help you to get started as quickly as possible in case of an incident response, digital forensics engagement or compromise assessment.

Most packs include a short-term but unrestricted enterprise license that allows you to run THOR or SPARK on any end system within an organisation. (the default licensing includes only host-based licenses; unrestricted enterprise licenses are more expensive)

Each license pack is offered at an attractively low price.

(more…)

Write YARA Rules to Detect Embedded EXE Files in OLE Objects

This is the first blog post published on our new website. If you followed my blog on www.bsk-consulting.de you should consider subscribing to the RSS feed of this blog or the “Nextron Systems Newsletter”.

This is one of the YARA related blog posts showcasing a special use case. Last year I noticed that I wrote many rules for hex encoded strings found in OLE objects embedded in MS Office documents and RTF files.

I did most of the encoding and decoding work on the command line or with the help of CyberChef, an online tool provided by GCHQ. I also thought about a new YARA keyword that would allow us to write rules without encoding the strings.

Today, rules contain strings in a hex encoded form. I usually add the decoded string as a comment.

$s1 = "68007400740070003a002f002f00" /* http:// */

Rules with the new keyword would look like this:

$s1 = "http://" wide hex

Neat, isn’t it? I already forwarded that feature request to Wesley Shields (@wxs) but it seems to be no low hanging fruit. I’ll keep you informed about this feature via Twitter.

A tweet by Kevin Beaumont reminded me of the work that I’ve done and while looking at the tool by Rich Warren. I thought that I should create a illustrative example of a more generic YARA rule that explains why the “hex” keyword would be very useful.

The tool creates weaponized RTF files with hex encoded payloads.

I derived some strings for a new rule from the decoded object.

/* Hex encoded strings */
/* This program cannot be run in DOS mode */
$a1 = "546869732070726f6772616d2063616e6e6f742062652072756e20696e20444f53206d6f6465" ascii
/* C:fakepath */
$a2 = "433a5c66616b65706174685c" ascii

To further improve the rule I went to my goodware directory and ran the following command to generate a list of the most frequent PE file headers in a hex encoded form.

neo$ find ./ -type f -name "*.exe" -exec xxd -ps -l 14 {} ; | sort | uniq -c | sort -k 1 | tail -10
4 4d5a87000300000020000000ffff
4 4d5aae010300000020000000ffff
4 4d5abf000300000020000000ffff
4 4d5add000300000020000000ffff
4 4d5aeb000300000020000000ffff
6 213c73796d6c696e6b3e2f757372
8 4d5a72010200000020001700ffff
88 4d5a40000100000006000000ffff
116 4d5a50000200000004000f00ffff
5852 4d5a90000300000004000000ffff

Then I used these hex encoded strings in a YARA rule that looks for these strings in the OLE objects of an RTF file.

rule MAL_RTF_Embedded_OLE_PE {
   meta:
      description = "Detects a suspicious string often used in PE files in a hex encoded object stream"
      author = "Florian Roth"
      reference = "https://github.com/rxwx/CVE-2018-0802/blob/master/packager_exec_CVE-2018-0802.py"
      date = "2018-01-22"
   strings:
      /* Hex encoded strings */
      /* This program cannot be run in DOS mode */
      $a1 = "546869732070726f6772616d2063616e6e6f742062652072756e20696e20444f53206d6f6465" ascii
      /* KERNEL32.dll */
      $a2 = "4b45524e454c33322e646c6c" ascii
      /* C:fakepath */
      $a3 = "433a5c66616b65706174685c" ascii
      /* DOS Magic Header */
      $m3 = "4d5a40000100000006000000ffff"
      $m2 = "4d5a50000200000004000f00ffff"
      $m1 = "4d5a90000300000004000000ffff"
   condition:
      uint32be(0) == 0x7B5C7274 /* RTF */
      and 1 of them
}

The first analysis of the coverage looks pretty good. I see only clear matches in munin‘s output.

The few questionable matches look fishy enough to release my rule.

If you have further ideas to improve the rule, ping me via Twitter.