APT Anomaly Detection – Part 1

APT Anomaly Detection – Part 1: Eliminating the Noise

The rapid discovery of a breach is key to minimizing the damage of a targeted attack. Context-aware anomaly detection improves an organizations security accuracy and efficiency by bringing relevant suspect events into focus and thus helps eliminate distracting noise.

Improve security analyst efficiency, reduce operational overhead and cost by eliminating noise

In APT Anomaly Detection – Part 1 we present a primer on the various options for Network Behavior Analysis as a complement to other core technologies and tools, adding to the capability to detect and investigate targeted attacks. The series then digs into and focuses upon improving the accuracy of events through triage to improve detection precision as well as eliminate the noise.

Signal to Noise Ratio

It’s a known fact that a lot of time is typically wasted on analyzing false positives generated by technology that is not correctly baselined, customized, tuned, optimized. Depending upon the environment, false positives can often be numerous and very difficult to verify, costing analysts a valuable time determining whether or not something is an event the analyst should be worried about.

Security Event Signal to Noise Ratio

Organizations today are exposed to a greater volume and variety of network attacks than ever before. Adversaries are exploiting zero-day vulnerabilities, taking advantage of risks introduced by cloud and mobile computing, and applying social engineering tactics to compromise user accounts. Advanced attackers are both patient and clever, evading detection at the network level. Security professionals wrestle with efficiently detecting these threats and effectively resolving them.

Reportedly Neiman Marcus experienced 60,000 alerts during their latest breach and Target was flooded with alerts. In both cases, the alerts failed to generate proper action. Relying on a tool (or tools) for alerts is useless if it generates too much noise and not enough signal. Too many alerts without the proper context fail to guide the right response.

Insider attacks are on the rise. To monitor and act on internal abuse, as well as comply with data protection regulations, organizations need to tie network security events to local systems and user credentials. Correlating threat information from intrusion prevention systems with actual user identities (logged on to local systems) allows security professionals to identify breaches of policy and fraudulent activity more accurately within the internal network.

Context-Aware Security

Traditional defenses, such as signature-based anti-malware tools and stateful inspection firewall technology, are less and less effective against new threats, they have no knowledge of applications in use, normal traffic patterns or user activity in the context of a network’s normal behavior patterns. New approaches to security, such as those focusing on context awareness and security intelligence, will provide the next generation technology to cope with evolving threats.

Leveraging Context-Aware Security

Context-aware security is the use of supplemental information to improve security decisions at the time the decision is made. The goal? More-accurate security decisions capable of supporting more-dynamic business and IT environments as well as providing better protection against advanced threats.

If possible, all information security infrastructure must become context-aware – endpoint protection platforms, access control systems, network firewalls, IPS systems, security information and event management (SIEM) systems, secure web gateways, secure email gateways, data loss prevention (DLP) systems, and so on.

The shift to incorporate “application awareness”, “identity awareness”, “virtualization awareness”, “location awareness”, “content awareness” and so on are all facets of the same underlying shift in information security infrastructure to become context-aware.

Why Context-Aware Security is Needed

To understand contextual security, organizations should understand the signature of a typical attack. A common type of APT attack involves embedding Trojan horse code in PDF documents delivered as an email attachment. When the unsuspecting email recipient clicks on the attachment, malicious code is unleashed, but it doesn’t immediately execute, delaying until any antimalware program is no longer watching. When the Trojan does finally execute, it discretely begins collecting data and sending GET requests to commonly visited sites to test network connectivity. If it detects an active network connection, the Trojan initiates a status beacon message to a command-and-control node.

The Signature of an Advanced Targeted Threat

As malware authors continue to introduce new antivirus evasion techniques, organizations must learn how to detect attacks that have slipped through the net and are living on the network. As the Mandiant APT1 report illustrated to the security community, attackers are capable of staying inside an organization’s network for years if organizations lack robust measures to detect and remediate attacks.

Network Baseline and Behavior Analysis

Network Behavior Anomaly Detection (NBAD) techniques were originally developed to monitor network traffic thresholds for shifts or changes that could indicate attacks or signal specific issues with systems. Over time, NBAD evolved into Network Behavior Analysis (NBA) which focuses on the establishment of a comprehensive network baseline. This overall baseline is then continually monitored for deviations or exceptions that may alert an analyst to an issue.

Behavior-based Anomaly Detection

There are three main components of a network behavior monitoring strategy for use in information security and network operations:

• Traffic Flow Patterns and Data: Network flow data such as NetFlow, sFlow, and Jflow.
• Network Performance Data: Simple Network Management Protocol (SNMP) events and Quality of Service (QoS) data for system and network performance.
• Passive Traffic Analysis: Passive analysis tools can continually monitor traffic for protocol anomalies, tunneled protocols, use of non-standard ports and protocol field values, etc.

Ultimately, these tools can also provide a much higher degree of visibility into what systems and applications are communicating with one another and where they are on the network, which provides intrusion prevention systems with much needed environmental context.

Forensic NetFlow and IPFIX analysis tools are ideal security layers with which to detect and investigate APTs. Network flows provide a complete account of all network activity both at the perimeter of the network as well as the network core. Advanced flow analysis solutions trigger alarms by monitoring for suspect behavioral patterns within the network flows. Identifying suspicious traffic patterns involves automated correlation of different types of contextual information then, deciphering the intent and danger associated.

One of the best ways to detect if internal hosts are communicating with other external APT launch points, is to compare NetFlow data to a host reputation list. By sending NetFlow from the Internet facing routers to a NetFlow collector that can compare all flows to the host reputation database, internal machines talking with known compromised Internet hosts, can be identified.

Getting started with Contextual Security

In order to combat these increasingly common scenarios, organizations must implement four lines of defense.

Rule Sets: Usually in conjunction with an intrusion detection system such as Snort.

Formulating effective rule sets is a fundamental portion of the contextual approach to network security. Rule sets are typically reactionary since they are usually only formulated after an attack vector has been identified but are still an important tool.

Also see, the APT Detection Indicators series which discusses Indicators of Compromise (IoCs) that can be used to develop and correlate rules.

Statistical Correlation: Utilize statistical and correlation methods to analyze the latest trends in malware.

This is the key that ties all of the other methods together since it meshes rule sets, log examination and data exfiltration monitoring. Correlation methods are used to examine whatever alerts are currently configured and to look for relationships between each alert that is triggered. These relationships can be with regard to type of alert, port number or any other type of selector configured by the security analyst. Statistical methods do not rely on prior knowledge of an attack vector, but rather on the time and frequency of a set of alerts.

Monitoring: Monitor for unusual data exfiltration attempts.

The most important portion of a context-aware security paradigm, examining and blocking data exfiltration attempts is the last line of defense when attempting to combat APT attacks. It is important for an organization to know what should and should not be leaving the network.

Log Analysis: Strongly emphasize the need to manually examine logs.

Automating log reviews with tools such as Splunk is a popular technique, and when operating in a highly trafficked network, automation is indeed a necessity. However, when attempting to discover new attacks against a network, nothing is as effective as human observation and intuition. Human intuition, along with informed experience should alert the security administrator to any site that looks suspicious, which could then spawn a new network monitoring rule to block that avenue of attack in the future.

Context Reduces Noise

As attackers become better at hiding out on networks, organizations need to be aware of the context surrounding security events to better sniff out APTs and reduce the noise. This means setting up the right kind of alerts based on Indicators of Compromise (IoCs) as well as previous attack vectors and correlating the information between triggered alerts. Most importantly this means having some human eyes monitoring data leaving the network and looking over logs to become familiar with the network and spot interesting traffic that may not be coded yet as triggers.

If an organization cannot connect all the dots across its network, it will be unable to fend off a new breed of persistent, stealthy malware. The organization needs to consider is this something that they build and operate in-house since security is mission critical?, whether they partner with consulting or a service to co-source both monitoring and skilled resources in a Hybrid SoC? or, outsource completely to a managed service since security is just not a core competency – although this needs strong process integration in terms of contextual awareness of the internal business operations as well as strict SLAs to ensure preparedness to respond.

Conclusion

Protecting an organizations data from APT invasion is an ongoing and daily task. Vigilance and healthy paranoia is a good defense against the possible insurgence. Many experts combating APTs suggest that organizations always be on the alert, that is assuming an APT is always present or already underway and to operate defensively rather than passively. Use Red Teams (see: APT Red Teams) to keep skills current and hone capabilities and response.

Improving communications visibility with evolving contextual anomaly detection is one of the best ways to detect internal malware that has circumvented the traditional firewalls. Many APTs have no trouble sneaking right past even the best security appliances, however, they have a habit of exhibiting the same suspicious behaviors, see Defensible Security Posture for details on the signature of an APT and Cyber Kill Chain.

In APT Anomaly Detection – Part 2 we will expand upon the above topics in more detail as well as discuss the options to add contextual sources, as well as fine tune and improve detection precision to improve analyst efficiency and reduce operational overhead and cost. This post is complemented by the APT Detection Indicators blog series which discusses Indicators of Compromise (IoCs) as well as useful open source tools and techniques to detect APTs.

Thanks for your interest!

Nige the Security Guy.