跳到正文
DEV Community· Chad Ray·· 4 小時前AI 評分34

展示推理依據:Logster 與可解釋警報及未觸發原因的偵測設計

Show Your Work: Logster and Detection That Explains Its Alerts and Its Silences

AI 導讀

Logster 將 Windows 和 Linux 端點活動整合成事件全貌,並為每項偵測判斷提供書面解釋。警報和抑制規則都應連結至實際觀察到的活動;排除條件須範圍狹窄、清晰可見,並可在生效前測試,以免寬泛規則成為攻擊者藏身之處。

正文

Every maths teacher has written the same note in the margin: show your work. A correct answer with no working earns partial credit. A wrong answer with clear working can still be fixed, because the teacher can see where the reasoning went wrong.

Security teams rarely get that courtesy from their tools. An alert arrives with a severity, a technique ID and perhaps a score. The analyst then does the working backwards, reconstructing why the system reached that conclusion before deciding whether they agree with it.

The same problem runs the other way, and there it is quieter and more dangerous. When a detection is tuned to stop firing on something benign, the tuning usually shows even less of its reasoning than the alert did. Nobody reviews an alert that never fires.

This article looks at both halves of the problem: verdicts that can't show their work, and silences that can't either.

Key Takeaways

  • An alert is only as useful as the evidence an analyst can check behind it; a score alone forces them to reconstruct the reasoning.
  • Suppressions and exclusions are detection logic too. Broad ones can become places for an attacker to hide.
  • A good system ties every verdict to specific observed activity and keeps each suppression narrow, visible and testable before it takes effect.

Alerts that can't show their reasoning

Picture two alerts for the same host.

The first reads: Suspicious PowerShell activity. Severity high.

The second reads: A document viewer launched PowerShell with an encoded command. That PowerShell process wrote a script to a temporary folder, and a scheduled task registered a minute later points to that script. Each clause links to the process, the file and the task it refers to.

Both might be correct. Only the second can be checked in a minute. With the first, the analyst still has to open the process tree, find the parent, look for file writes, and search for persistence, which is the investigation the alert was supposed to start.

Pattern-matching rules have always had a partial answer here: the rule itself is the explanation. If a rule fires on "Office application spawns a shell", the analyst knows what it matched. That transparency is one reason rules remain valuable.

Model-driven detection makes the problem harder. A statistical score says how sure the model is, not why. A language model can produce a fluent explanation, but a fluent paragraph is not evidence. The model may describe a connection that isn't in the records. The paragraph and the telemetry need to be tied together, or the analyst is simply trading one kind of reconstruction for another: checking whether the prose is true.

So the useful question isn't "Does the detector explain itself?" It is "Can each step of the explanation be traced to something that was actually observed?"

Suppressions that hide more than they should

Every production detection program tunes. A backup agent that reads thousands of files every night, or a software deployment tool that launches scripts across the fleet, will generate noise until someone suppresses it. That work is necessary; an untuned system buries real findings.

The trouble is the shape suppressions tend to take under pressure. The fastest fix is broad: exclude a directory, a process name or a whole host. Such rules often contain wildcards and are rarely reviewed again.

Attackers understand this. MITRE ATT&CK documents Impair Defenses: Disable or Modify Tools (T1562.001), which covers adversaries modifying or disabling security tooling to avoid detection. (MITRE ATT&CK, T1562.001) Adding an exclusion is one quiet way to do that. One who finds an existing exclusion that is broad enough doesn't need to add anything.

Consider a hypothetical suppression that matches any process whose path contains a trusted vendor's folder name. It was written to silence an updater. It also matches a binary an attacker drops into a subfolder of that directory, and anything that binary launches. The rule matched a name. It had no way to notice that the behaviour around that name had changed.

CISA's joint guidance on living-off-the-land techniques recommends establishing and maintaining baselines of normal network, user and administrative activity, and comparing current activity against them. (CISA and partners, February 2024) A suppression is a baseline written down, and a baseline is only safe if it describes what normal looks like, not merely who is usually present.

What "showing your work" would require

Put the two halves together and a set of requirements emerges. None is exotic on its own. The difficulty is getting all of them in one system.

Requirement For verdicts For suppressions
Grounding Every reasoning step points to observed entities Every rule matches observed attributes, not substrings of them
Scope The verdict covers a connected set of activity, not a single event A match covers only benign activity, not whatever that activity spawned
Visibility The analyst can see what the conclusion relied on The team can see what a rule would suppress before it does
Honest failure "Could not decide" is distinct from "benign" A rule that stops matching stops suppressing

The common thread is that both the alert and the silence should be decisions about structure: which process did what, to what, and in what order. That is also the information an analyst needs to judge either one.

One way to build it

This is the design behind Logster, an AI-driven endpoint threat detection platform that the team behind these articles builds. Here is what it offers for each requirement, and where the approach has limits.

Logster connects the activity on each Windows and Linux endpoint into a single picture of what happened, so related actions are judged together rather than one event at a time. Each verdict comes with a written explanation of how it was reached.

Grounding the verdict. When an analyst investigates an attack verdict, each step of the AI model's explanation is linked to the specific processes, files and connections it refers to. Clicking a step highlights them. The linking annotates the verdict; it never changes it. The prose now points to evidence, and the analyst can check whether that evidence says what the prose claims.

Scoping the silence. Analysts suppress known-benign activity by approving patterns of behaviour rather than names. A rule matches what it was written for, not everything that happens to resemble it, so a rule written for a vendor's updater doesn't accidentally cover a file dropped beside it. Anything broader has to be written explicitly, where a reviewer can see it.

The subtle part is what happens when approved activity changes. If the trusted updater suddenly launches a shell that nothing approved, that activity stays in view and is still assessed. The exclusion covers the behaviour it was written for and stops there.

Making silence visible. A new rule can be trialled before it takes effect: the team sees what it would have suppressed while nothing is hidden. Once it is live, every verdict records what was suppressed. Before a suppression goes live, and long after it does, the team can see exactly what it is hiding.

Failing honestly. If the AI model can't reach a verdict, Logster reports a failure, not a clean result. A finding set aside by tuning stays on record with the reason, so reviewers can still find it rather than losing it entirely.

Where the working still runs out

Showing your work does not guarantee the answer is right.

Linking an explanation to evidence ties it to real activity, but an explanation can cite real activity and still draw the wrong conclusion from it. The highlighting makes that easier to catch, not impossible to happen. Model confidence is not a calibrated probability until someone measures it against labelled data.

Any verdict also rests only on what was collected. If telemetry is missing or the activity that explains an event happened earlier than what was assessed, the verdict rests on less than the full picture. A narrow suppression rule is still only as good as the analyst's judgement about what counts as benign, and a pattern that is normal today can be abused tomorrow.

Architecture describes intent. Detection quality, false-positive rates and time saved still need to be measured against real attacks and real workloads.

Mark the paper

The next time you review your detection stack, grade both columns.

For a recent alert, ask: could an analyst verify it from the alert alone, or did they have to rebuild the story? Does every claim in the explanation point to a specific event?

For your suppressions, ask harder questions. Which of them match on a substring or wildcard? What would happen if an allowed process launched something new? When did anyone last look at what each one is hiding?

A detection system earns trust the way a student does: by showing work that holds up when someone checks it. That applies to the answers it gives and to the questions it has been told to stop asking.

來源:DEV Community · dev.to