Evaluating EDR Effectiveness

EDR

EDR stands for Endpoint Detection + Response. This is the tool that your company will probably spend a chunky amount of money to buy, deploy an agent to the user’s endpoint for visibility, detection, and response. The agent collect the device’s telemetry and send back data to the server. If something seems odd or suspicious, it’ll send an alert. Sometimes, these platform may even comes with an AV capability for you to check the compliance box.

Recently, I got the fun opportunity to test out a few different EDR vendors and their effectiveness. This seemed like a fun task, since it means I can:

1. Get to learn to use different vendor’s tools
2. See the different popular EDR platform, and actually get to test how it works
3. See its capabilities, what it catches, what it doesn’t catches, and whether or not if I can bypass it.

At heart, I am a blue teamer. However, most of my training has been on the offensive side. There’s nothing more fun than pwning a machine, only to realize how much footprints that was left behind. I like the ability to learn on how to be stealth and bypass current detection in places in order to defend it better.

Evaluation Criteria

• Usability - ask your analysts how was their experience in navigating around working a case

• Detection piece

  • Does it detect AND alert on the malware? The payload? The key that was added to registry?

• Response piece

  • Does it stop the obvious maliciousness that’s happening?
  • Did it stop the weird behavior?

The test

The attacker’s side:

• Deploy a malware, and or a payload. Do some fun stuffs. Record what you did.

  • Initial compromise
  • Exploitation
  • Persistence
  • Privilege Escalation
  • Lateral Movement
  • Exfiltration
  • Hiding the tracks

The analyst’s side:

• Can they find it?
• What evidences can they find?
• How easy was it to find?
• How fast was it to pivot?

The tools

• Empire

  • Straight powershell/python payload
  • Payload embedded in a worddoc
  • Payload embedded in a file (.lnk, hta)

• Custom malware

  • Maybe reuse the payload, obfuscate it a bit more
  • Changing up some variable, recompile the binaries, changing the name

• Living off the land

  • Using Microsoft signed binaries such as certutil, mshta, wmi to download and execute the payload

The methodology

Preparation

Each platform get a week of testing. Among other things

• Pre-planning: Identify folks who would be happy to be your test users. Beg some devs. Bribe some others with coffee, beers, donuts…whatever it takes.

  • People (baseline): baseline the environment, see how much false positives are generated and alerted. Give you a good perspective of how much work it’ll take to tune the product (post purchase)
  • Test machines (known known): Deploy custom payload
  • Test VMs (known unknown): detonate malware, easily revert back. Do this on a MIFI. Off corporate, unjoined domain with just the EDR agent on.
  • A scorecard: identify what’s important to you. Test the EDR against your own team’s standard. Give it a weight. That way, no one is biased at final results.

Test week

• Day 1: Get info about platform. Install agents on test machines (Computer/VMs/Server)
• Day 2: Deploy payload 1-3
• Day 3: Deploy malware; Start triage process
• Day 4: Triage + Analyze + Investigate
• Day 5: Evaluation

The results

• Fill out the score card.
• Sit with your team to determine the pros/cons of each platform

Final findings may surprise you. Some EDR vendors are better than others. Each has its own pros and cons. I have absolutely no recommendation other than for you to evaluate what’s important to you in your own environment. It all depends on your inventory, budget, skills, and styles.