Despite many improvements to security technology, information sharing, ease-of-use, and reduction in cost over time, we have yet to see a meaningful reduction in breaches. Why? The answer may have something to do with how organizations respond to a breach beyond the moment it’s discovered.
Incident response dominates much of the security conversation, and for good reason. It’s imperative that organizations act quickly once a single isolated threat has been detected. However, this is just one part of a holistic risk prevention strategy – it’s arguably more critical to have a full view of events and analyze the entire lifespan of an attack. When organizations have visibility into a threat’s entry point and history, it’s possible to uncover what else on the network may have been compromised. To that end, some organizations favor monitoring adversaries within their environment to identify what assets are being targeted. Understanding the motivation behind an attack, and asking the right questions at the start of an investigation, allows security teams to focus their defenses and ultimately improve protections.
Why You Should Play Detective
What many organizations don’t realize is that a comprehensive response strategy begins the same way as so many other complex processes: with a plan. Instead, many security teams turn to less effective, knee-jerk reactions that lead to tearing compromised hosts offline…and possibly even destroying evidence as they fumble for a solution. You learn nothing from missing or destroyed evidence, precluding any kind of reliable prevention. Preserving those systems and learning from them is a necessity and requires what I refer to as “the speed of haste.” Decisions should be made quickly, but carefully. A decision made without due consideration – or, made in haste – is almost always the wrong one.
Instead, organizations should adopt a response process that enables them to gather necessary information before it disappears or is destroyed. More mature organizations will look at the information they’ve gathered and compare it to their attack model, looking for major gaps. By chasing down those gaps as investigative objectives, organizations are better informed about what happened and what to do about it.
Your incident response plan should also cover more than just an order of operations. It should define the personnel who are doing the investigative work, which means you should be hiring for capability and not just headcount. Additionally, these informed practitioners need access to data and a means of acting on it. Technology naturally play big roles in both areas – improving visibility, force-multiplying your human team members, providing technical capabilities that people lack, and providing assurance of integrity.
How to Ask the Right Questions
When sitting down to triage an isolated event, an analyst usually doesn’t know what the scope or nature of the problem is. An alert doesn’t necessarily indicate an intrusion, and an intrusion may not result in a single alert. For these reasons, when an analyst approaches the challenge of scoping and responding to a breach, they need to develop broad investigative questions to pursue in the form of leads. Some will be generic, such as:
1. How many distinct groups of activity are known?
2. How is each group of activity being identified?
3. How many systems in the enterprise are infected as part of this intrusion?
4. How many systems were accessed but not infected?
5. Which accounts was the attacker using during the compromise?
6. What was the most recent activity?
7. Is the adversary active and, if so, at what cadence?
Other questions tend to be more specific and often flesh out the details of the investigation. These might be very specific points, such as whether they used a unique fraudulent certificate with each of their malware payloads and which certificate authorities they chose to generate them with. But it’s the combination of answers from both generic and specific questions that enable the investigator to conclude their investigation and provide direction for preventing future intrusions.
Example: Threat History Detection in Action
One of the common mistakes I see across immature security organizations is failing to look at the intrusion as a whole entity first, before seeking the missing pieces to their investigative puzzle. Let’s attempt to illustrate this using one of the attack models we’ve mentioned:
If you subscribe to the FireEye intrusion lifecycle model, there are seven phases that represent major adversary objectives during a breach:
1. Initial compromise (such as being phished with a macro-enabled document lure)
2. Establish a foothold (execute a non-persistent script-based backdoor)
3. Escalate privileges (user happens to be a member of local admins already) which may also involve stealing valid local or domain credentials
4. Perform internal reconnaissance (adversaries have to learn about the environment and if they show you that they don’t they may have been there before)
5. Move laterally (Remote Desktop Protocol is one familiar method)
6. Maintain a presence (deploying additional and usually different backdoors, sometimes a variety of payloads and methods)
7. Complete the mission (compress data, stage it for theft, steal the data)
This is an extreme example, but let’s say you’re four weeks into an investigation and you know a few things:
• You self-detected at the final stage, noting that 2.4GB of legal documents were stolen from a file share where they were also staged in a compressed archive.
• You were able to identify recently-installed persistent backdoors on three systems, installed within five seconds of a domain administrator logging in via RDP and before the same session was terminated.
• The adversary used several accounts in the domain admins group and accessed these infected systems from a fourth system in a desktop support business unit.
• All domain users are in the local administrators group on their own system, so they automatically escalated privileges at the moment of initial infection.
• You found the phishing lure they used and confirmed that three users downloaded malware and all of these users workstations were also infected.
If you map those things to the attack model above, the major missing piece is how they conducted reconnaissance. The exercise of going back and specifically trying to fill in that blank is essential to the response process. One reason that IR firms charge so much per hour isn’t just for expertise. It’s the experience of those consultants that recognize the essential questions to ask. An experienced incident responder will see that part of the puzzle is blank and work to fill it in.
If your breach response process is to delete malware from infected systems and change a few passwords, that’s a missed opportunity. When you’re fighting fire with fire in that way, it’s challenging to understand the life cycle of an intrusion and difficult to value the evidence that’s lost. Organizations should plan beyond initial compromise and invest in technologies, personnel and methods that foster a more complete understanding. Only then can they move from being a spectator to becoming a contender.
Devon is a principal researcher at Endgame, focusing on detection and response technologies. Formerly a Mandiant incident response and remediation lead, Devon has over 6 years of experience in security professional services where he has worked with clients in a nearly every conceivable industry. He has significant experience helping Fortune 500 organizations with the detection, response, and containment of advanced targeted threat actors and has led large-scale network and application architecture reviews, post-incident strategic planning, and regulatory gap assessments. He has delivered a range of technical presentations for security conferences, industry organizations, and the United States Department of Defense. Prior to his career in information security, Devon spent 15 years in operations roles as a system administrator and network engineer.
Tags: 
