Understanding the Log Pipeline: From Raw Event to Alert

You cannot write effective detection rules without understanding what happens between "something occurs on the endpoint" and "an alert appears in the dashboard." The pipeline has four stages and every stage matters. Most people skip this and then wonder why their rules don't fire correctly.

The full pipeline

Something happens on the endpoint
          │
          ▼
  RAW LOG — Windows Event XML blob
          │
          ▼
  DECODER — extracts structured fields
  (windows_eventchannel decoder)
          │
          ▼
  RULES ENGINE — matches decoded fields
  against rule conditions
          │
          ▼
  ALERT — stored in index, visible in dashboard

Reading a real event

A failed login was triggered deliberately using a non-existent account. Windows Event ID 4625 — a failed logon — was generated and appeared in the dashboard as a decoded alert with these fields:

"win.system.eventID":            "4625"
"win.eventdata.targetUserName":  "fakeuser"
"win.eventdata.subjectUserName": "REDACTED"
"win.eventdata.subStatus":       "0xc0000064"
"win.eventdata.logonType":       "2"
"decoder.name":                  "windows_eventchannel"
"rule.id":                       "60122"
"rule.level":                    5
"rule.firedtimes":               4

What each field means

Field	Value	Meaning
`win.system.eventID`	4625	Failed logon event
`win.eventdata.targetUserName`	fakeuser	Account being attacked
`win.eventdata.subStatus`	0xc0000064	Username does not exist
`win.eventdata.logonType`	2	Interactive — at the keyboard
`rule.level`	5	Medium severity
`rule.firedtimes`	4	Rule fired 4 times total

The subStatus code matters. 0xc0000064 means the account doesn't exist. 0xc000006a means wrong password on a real account. These generate different event structures which is important for writing precise rules.

The gap that correlation rules fill

The built-in rule 60122 fired at level 5 on every failed login. It tracked how many times it fired but it never escalated. There was no logic for: "if this fires 5 times in 120 seconds, this is a brute force attack."

That gap is exactly what a correlation rule fills.

Key fields to know for security events

win.system.eventID           — Windows Event ID (4624, 4625, 4672 etc.)
win.system.channel           — Which log channel (Security, Application, System)
win.system.severityValue     — AUDIT_SUCCESS or AUDIT_FAILURE
win.eventdata.targetUserName — Account being targeted
win.eventdata.ipAddress      — Source IP of the attempt
win.eventdata.logonType      — How the logon occurred
win.eventdata.subStatus      — Specific failure reason code

DQL search syntax in OpenSearch Dashboards

data.win.system.eventID: 4625
data.win.system.channel: Security
agent.name: my-laptop
data.win.system.eventID: 4625 and agent.name: my-laptop

Use lowercase and / or / not. Field names include the data. prefix in dashboard searches — but that prefix does not exist inside rule XML. This distinction trips up everyone early on.

What enrichment adds (future state)

The decoder extracts what's already in the log. Enrichment tools add external context:

VirusTotal — takes file hashes and returns malware verdicts
AbuseIPDB — takes source IPs and returns reputation scores
MISP — matches against known threat intelligence indicators

That's for later. First — write the detection rules.