Threat Hunting Is Not a Magical Unicorn

Threat hunting, like most market buzz terms, started with a concept or an idea, and then got overused and misused by every vendor, blogger, and Twitter account with an opinion. This has led to a lot of confusion for security teams that want to build a threat hunting capability. So what is threat hunting and how do you do it?

For more threat hunting best practices from Joe Moles, watch an on-demand webinar with Carbon Black: How to Build Threat Hunting into Your Security Operations

What Threat Hunting Is…and Isn’t

First let’s understand what threat hunting is not. Threat hunting is not a product, it is not automated, and it is not something you can put in a script or flow chart for a trained SOC monkey to follow. Threat hunting is not alerts generated by a solution and it’s not running a scan for IOCs. Hunting is not a passive activity. One thing I like to tell people when discussing strategies and techniques for finding threats is: if I can put it in a script, I can automate it. I will caveat my statement by saying automated processes and tools can help improve the efficiency of your hunting effort.

All right, then: what is threat hunting? The simplest definition of threat hunting (or as I prefer to call it, just “hunting”) is finding anomalous activity that has a negative security implication. Now that is a very broad definition, and you can easily say that some automated, whizbang, next-gen, ML, AI tool does that. A more in-depth definition would be: augmenting your prevention and detection capabilities by actively looking for anomalous activity that has not been identified by your existing toolsets by searching through various sources of data.

The key is that hunting is a tool and a method for augmenting and improving your existing capabilities.

Hunting should be a core part of the feedback and continuous improvement process. It should be one of the core activities of any good SOC analyst. The goal should be to find ways to quickly and potentially automatically find threats. The basic cycle should be: look for bad thing, find bad thing, and figure out how to find that bad thing faster next time.

How to Develop a Threat Hunting Capability

So how does a team go about this? It starts very similarly to any other kind of hunting or detective work.

1. Begin by doing your homework.

   For example, one way to begin involves generating an idea or hypothesis by asking questions like “How would I break into my organization” or “What would it look like if I did X bad thing?” Those ideas can be an original thought of your own or driven by something you have seen or read recently.

2. Break it down.

   Break that concept down into behaviors or steps.

3. Now figure out what that behavior would look like in your environment.

   What information sources do you have to be able to see that activity? Is there anything unique about your environment that would cause the behavior to look a little different?

4. Look for those behaviors.

   Once you have adequately defined how the attacker would behave, now the fun starts. Go look for those behaviors (mind blown!). You’ll have a start, but identifying the behaviors you defined is not the end goal. You need to follow any thread that you surface. When you find the initial behavior, start looking around at what else was happening. Is there enough data to make a decision if this is anomalous or not?

5. Look for ways to find it faster.

   If you find little or no activity that matches the behavior, you’re not done. Just because the attackers have not tried that technique today doesn’t mean they won’t tomorrow. Determine how to create an alert to find the behavior in the future. Spend the extra time now because it will pay dividends down the road. (This is how you should be thinking about hunting: automating your tech to quickly surface potential threats and using human capital to find new and more advanced activity.)

6. Keep going.

   Once you have the logic built, move on to the next idea and continue expanding your alerting capabilities.

As these new alerts start coming in, you can use the aggregate of those behaviors to refine the baseline of your environment and more efficiently identify bad behaviors. But you are not done yet! You will need to start tracking the accuracy of each alert and using that information to drive your efforts to tune those alerts. While this is very high level, this is how every organization should be thinking about improving their coverage: by regularly engaging with their data and continuously re-evaluating visibility and the value of the alerting source.

Dig deeper into threat hunting best practices: How to Build Threat Hunting into Your Security Operations

Another Threat Hunting Approach

What I just outlined is very systematic and every SOC should put a process in place to continually improve detection and hunting. The other way you can hunt for things is to just jump in the deep end.

1. Start with a very broad idea.

   For example, look at every process that resulted in a binary being introduced in the last 7 days. Or, every network connection to an external address that is not connecting on 80 or 443.

2. Filter.

   Filter out the activity that you know is normal and then scan through the results to see if something anomalous stands out.

3. Go deeper.

   Keep peeling back the data until you cannot filter any more or you find something you need to take action on.

4. Incorporate feedback.

   If you do find something, obviously you want to address it first. But don’t forget that part of your response process has to include feeding it back into the system. How would you find it faster next time? Create a rule or alert to do the heavy lifting for you, then move on to the next idea.

Threat Hunting Is the Act of Continuous Improvement

I said this before but I want to stress it again: not finding something on the first go should not be considered a failure. The activity of engaging with information about your environment still helps you build a better understanding of “normal.” And during every investigation and hunt, you should always be thinking: how do I automate this? Automation should not be your final goal. Meaning, you shouldn’t get all of your other processes in place and then think about automation. You should start with the intent of automating to find it faster and with less effort next time.

“The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency.” -Bill Gates

There is an inherent risk in not having coverage for all known threats, but to build a sophisticated and efficient security operation you have to start with base coverage and build up through a continuous feedback and review.

Key Takeaways for Threat Hunters

Threat hunting is not a new concept, it has just become a popular buzzword to describe what many people have been doing for a long time. It is something your team and your providers should be doing, and it is a core part of continuing to expand and improve your security posture. I hope you can see that a tool or application is not going to do this for you, as it is not something you can automate away. However, it can be something that does and should lead to improved automation.

Whether your team is internal or external, you should have a solid process for continuous improvement based on threat hunting. Make sure you can answer questions like: Is there a continuous improvement process? Is feedback built into alerts? How do we look for new threats and dynamically expand to provide coverage? Threat hunting is not a standalone activity. Detection requires a good mechanism for engaging with your data to understand your environment and find the threats unique to it.

Looking for more threat hunting best practices? Watch Now: How to Build Threat Hunting into Your Security Operations