STIXv2 Support in SPARK

SPARK Version 1.17.0 adds extensive STIXv2 support.

This allows you to easily extend SPARK’s signature bases with IOCs from any sandbox, analysis or threat intel platforms that support STIXv2 export by placing the exported *.json files in the ./custom-signatures folder.

For now, the supported observable object types are:

  • file:name with = != LIKE and MATCHES
  • file:parent_directory_ref.path with = != LIKE and MATCHES
  • file:hashes.sha-256 / file:hashes.sha256 with = and !=
  • file:hashes.sha-1 / file:hashes.sha1 with = and !=
  • file:hashes.md-5 / file:hashes.md5 with = and !=
  • file:size with < <= > >= = !=
  • file:created with < <= > >= = !=
  • file:modified with < <= > >= = !=
  • file:accessed with < <= > >= = !=
  • win-registry-key:key with = != LIKE and MATCHES
  • win-registry-key:values.name with = != LIKE and MATCHES
  • win-registry-key:values.data with = != LIKE and MATCHES
  • win-registry-key:values.modified_time with < <= > >= = !=

These types are applied in different modules:

  • FileScan: file:*
  • Registry: win-registry-key:* and file:name (applied to data field)

You can find a list of products that support the STIX data exchange format here.

Important Update Process Changes

As we have announced in May, the old “thor-upgrade.exe” is already out-of-support and the old update servers accessed by “thor-upgrade.exe” will be decommissioned at the end of October.

The new all-round utility “thor-util.exe” now supports all of the features provided by the old “thor-upgrade.exe” including NTLM Authentication with corporate proxy servers.

You can use “thor-util.exe upgrade –help” to see all options of the “upgrade” feature.

Also note that “thor-util” has an “encrypt” feature that allows you to encrypt custom signature files and the “report” feature that creates HTML reports from plain text log files. 

The only valid update servers that should be accessible to get updates from November onward are:

  • update1.nextron-systems.com
  • update2.nextron-systems.com

The “thor-util” utility is part of the THOR and SPARK packages and can also be downloaded from the Customer Portal in the “Downloads” section.

If you are a customer and don’t have access to the Customer Portal yet, please contact us or the respective partner.

Feature: SPARK Sample Quarantine via Bifrost

The new SPARK v1.14.16 supports the sample quarantine protocol named Bifrost.

With Bifrost you’re able to send suspicious samples that THOR or SPARK  detect on endpoints directly to a central server for analysis.

A Bifrost server is shipped in form of a Python script with THOR and SPARK. (./tools sub folder)
You can also activate the Bifrost server on our ASGARD platform.

All samples that have a score higher than the given limit are dropped into a given directory and are available for further post-processing – e.g. drop them into a sandbox or static analysis.

New Feature: THOR-util and SPARK-Core-util Signature Encryption

The new THOR-util version 1.2.4 supports the encryption of your custom signatures so that you can deploy your own IOC files and YARA rules in an encrypted form.

We use a public key in the utilities to encrypt the files for our scanners so that admins, Antivirus engines and attackers won’t be able to read the contents of the files.

 

The feature is also available in SPARK Core, our free scanner.

After encryption, place the encrypted IOC files in the “./custom-signatures” directory and the encrypted YARA rules in the “./custom-signatures/yara” directory.

The use of the function is simple. Just point it to a file, a list of files or use wildcards to select a set of files for encryption. The extension of the output file depends on the extension of the input file.

  • IOC Files: .txt > .dat
  • YARA Rules: .yar > .yas
  • Sigma Rules: .yml > .yms

Examples:

thor-util.exe encrypt case44.yar
thor-util.exe encrypt case44-hashes.txt
thor-util.exe encrypt case44-hashes.txt case44.yar
thor-util.exe encrypt case44.*

You can use the “upgrade” feature in both tools to get the newest version of the utility.

thor-util upgrade

ASGARD IOC Management

The upcoming ASGARD version 1.5 comes with a IOC management section in which you can manage your own set of IOCs in text files, YARA and Sigma rules.

You can then select each of the folders when creating a new scan run with THOR or SPARK. Selecting one of these folders will not include the sub folders.

You can schedule and run scans with different IOC, Sigma and YARA rule sets. You can review the included custom signatures in the scan details. 

The following features are not yet implemented in v1.5 but on the roadmap for ASGARD v1.6:

  • Signature verification
  • Exclude the standard rule set (shipped with THOR and SPARK)

SPARK uses Sigma Rules in Eventlog Scan

Sigma is a rule format for threat detection in log files. It is for log data what “Snort rules” are for network traffic or “YARA signatures” are for file data. It is easy to write and read. Writing a Sigma rule is a matter of minutes.

On the right you can see a simple Sigma rule that checks the “System” eventlog for traces of password dumper activity. The detection section contains 1+ identifiers (selection, keywords, quarkspwdump) that can be defined freely by the rule author. These selectors are used in the condition to build the rule.

It also contains a description, references, possible false positives and a level.

Analysts use Sigma to generate search queries for their SIEM or log management solution. The Sigma repo contains a converter that allows to convert the generic rules to ElasticSearch, Splunk, QRadar, Logpoint, Windows Defender ATP (WDATP) and ArcSight.

Wouldn’t it be great if you could apply Sigma rules on the endpoint?

Well, the upcoming version 1.14 of SPARK, which will be released at the end of July,  does that. It applies Sigma rules to the local Eventlog. This way you’re able to apply searches that you have once defined for your SIEM to the local Eventlogs.

This way you are able “query” the standalone systems that are not connected to your SIEM and uncover otherwise common blind spots in your environment.

 

We ship the current rule set, which is part of the public Sigma repository and contains more than 200 rules with our SPARK program package in an encrypted form. (*.yms)

You can add your own Sigma rules to the “./custom-signatures/sigma/” folder in the SPARK program directory.

To activate Sigma scanning, use the new “–sigma” parameter.

Currently only SPARK supports this feature and there are no plans to implement this in THOR as well.

The feature is currently free for all customers but may become a premium feature that has to be licensed separately by the end of the year depending on the customer’s plan. 

See the comparison table for a complete overview on all features.

THOR-Util with HTML Report Generation

The new version of “thor-util” (used with THOR/SPARK) / “spark-core-util” (used with SPARK Core) support a feature that allows a user to convert any scanner log file into a convenient report. 

  • Convert THOR / SPARK / SPARK Core scan logs into HTML reports
  • Convert a single text log file into an HTML report
  • Convert multiple log files (50 max.) in a directory into a single HTML report 
  • Provide a file with filters to suppress false positives in the reports
  • Even LOKI logs can be converted (no support)
  • Hash values linked to Virustotal searches
  • IP values linked to VirusTotal searches
  • Header sections linked to elements via ankers

You can access this feature in the upcoming enterprise products (THOR 8.47.2 and SPARK 1.13) and the free product SPARK Core (SPARK Core 1.13). 

The following screenshot shows a typical text log file. It can be processed in log analysis solutions but it is difficult to read for an analyst. Most analysts search these log files for “(Alert|Warning):” or use grep to get the most relevant messages.

Our tools “thor-util” and “spark-core-util” will help you with this task. 

Generate an HTML report for a single log file

<br /> thor-util report --logfile PROMETHEUS_thor.log<br />

Generate an HTML report for multiple log files

thor-util report --logdir ./logs

You can also provide a file with regular expressions that are applied during log parsing as filters to suppress false positives in the reports. 

The new tools will be in all productive packages at the end of this week. 

SPARK Core – Free IOC and YARA Scanning

It is done! Our new free scanner SPARK Core has been released.

After weeks of planning, development and testing, we’re proud to provide the community with a new and powerful multi-platform scanner.

SPARK Core is a reduced version of our successful scanner SPARK.

The main differences are the Open Source signature base and the reduced set of modules. It uses LOKI’s open source “signature-base” instead of the big signature set that is used in THOR and SPARK. It also lacks some of the modules, like the SHIM cache, Registry, Eventlog and DeepDive modules.

This overview explains how SPARK Core fits in our current scanner portfolio:

Some key points:

  • Free scanner for Windows, Linux and macOS
  • Precompiled and encrypted open source signature set
  • Update utility (spark-core-util) to download tested versions with signature updates
  • Documentation
  • Custom IOCs and signatures (just add them to the ./custom-signatures/ folder)
  • Different output formats: text log, SYSLOG (udp/tcp/tcp+tls), JSON to file, JSON via Syslog
  • Scan throttling to limit the CPU usage

All we ask for is a SPARK Core Newsletter subscription, which is a requirement for the automatic license renewal. Each subscriber receives a personal licenses file that is valid for 1 year and allows to run SPARK Core on as many systems as he wishes.

Support is not guaranteed but we provide the possibility to submit issues via our github page.

More information and download can be found on the product page.

We hope that you can use SPARK Core to catch some bad guys.

New THOR / SPARK License Packs

We have just recently released new, flexible and practice-oriented license packs for our scanners THOR and SPARK. These license packs will help you to get started as quickly as possible in case of an incident response, digital forensics engagement or compromise assessment.

Most packs include a short-term but unrestricted enterprise license that allows you to run THOR or SPARK on any end system within an organisation. (the default licensing includes only host-based licenses; unrestricted enterprise licenses are more expensive)

Each license pack is offered at an attractively low price.

(more…)

Write YARA Rules to Detect Embedded EXE Files in OLE Objects

This is the first blog post published on our new website. If you followed my blog on www.bsk-consulting.de you should consider subscribing to the RSS feed of this blog or the “Nextron Systems Newsletter”.

This is one of the YARA related blog posts showcasing a special use case. Last year I noticed that I wrote many rules for hex encoded strings found in OLE objects embedded in MS Office documents and RTF files.

I did most of the encoding and decoding work on the command line or with the help of CyberChef, an online tool provided by GCHQ. I also thought about a new YARA keyword that would allow us to write rules without encoding the strings.

Today, rules contain strings in a hex encoded form. I usually add the decoded string as a comment.

$s1 = "68007400740070003a002f002f00" /* http:// */

Rules with the new keyword would look like this:

$s1 = "http://" wide hex

Neat, isn’t it? I already forwarded that feature request to Wesley Shields (@wxs) but it seems to be no low hanging fruit. I’ll keep you informed about this feature via Twitter.

A tweet by Kevin Beaumont reminded me of the work that I’ve done and while looking at the tool by Rich Warren. I thought that I should create a illustrative example of a more generic YARA rule that explains why the “hex” keyword would be very useful.

The tool creates weaponized RTF files with hex encoded payloads.

I derived some strings for a new rule from the decoded object.

/* Hex encoded strings */
/* This program cannot be run in DOS mode */
$a1 = "546869732070726f6772616d2063616e6e6f742062652072756e20696e20444f53206d6f6465" ascii
/* C:fakepath */
$a2 = "433a5c66616b65706174685c" ascii

To further improve the rule I went to my goodware directory and ran the following command to generate a list of the most frequent PE file headers in a hex encoded form.

neo$ find ./ -type f -name "*.exe" -exec xxd -ps -l 14 {} ; | sort | uniq -c | sort -k 1 | tail -10
4 4d5a87000300000020000000ffff
4 4d5aae010300000020000000ffff
4 4d5abf000300000020000000ffff
4 4d5add000300000020000000ffff
4 4d5aeb000300000020000000ffff
6 213c73796d6c696e6b3e2f757372
8 4d5a72010200000020001700ffff
88 4d5a40000100000006000000ffff
116 4d5a50000200000004000f00ffff
5852 4d5a90000300000004000000ffff

Then I used these hex encoded strings in a YARA rule that looks for these strings in the OLE objects of an RTF file.

rule MAL_RTF_Embedded_OLE_PE {
   meta:
      description = "Detects a suspicious string often used in PE files in a hex encoded object stream"
      author = "Florian Roth"
      reference = "https://github.com/rxwx/CVE-2018-0802/blob/master/packager_exec_CVE-2018-0802.py"
      date = "2018-01-22"
   strings:
      /* Hex encoded strings */
      /* This program cannot be run in DOS mode */
      $a1 = "546869732070726f6772616d2063616e6e6f742062652072756e20696e20444f53206d6f6465" ascii
      /* KERNEL32.dll */
      $a2 = "4b45524e454c33322e646c6c" ascii
      /* C:fakepath */
      $a3 = "433a5c66616b65706174685c" ascii
      /* DOS Magic Header */
      $m3 = "4d5a40000100000006000000ffff"
      $m2 = "4d5a50000200000004000f00ffff"
      $m1 = "4d5a90000300000004000000ffff"
   condition:
      uint32be(0) == 0x7B5C7274 /* RTF */
      and 1 of them
}

The first analysis of the coverage looks pretty good. I see only clear matches in munin‘s output.

The few questionable matches look fishy enough to release my rule.

If you have further ideas to improve the rule, ping me via Twitter.