Mambark scores every event; AI agent investigates only suspicious ones

AI agents can investigate suspicious security activity by pulling in context from across an environment and explaining whether an event represents a genuine threat. But applying that kind of reasoning to every security event is prohibitively expensive.

If you're building AI agents that operate over large streams of events, you've likely encountered the same trade-off we faced in security: You can't afford to apply expensive AI reasoning to every event, but you also can't afford to miss the ones that matter. To make AI-driven security detection practical at fleet scale, we built a two-stage detection pipeline. Mambark, a small state-space model pretrained on sequences of audit logs, scores every event and forwards only the most suspicious ones to an AI agent for deeper investigation. We're currently testing this detection mode with a small group of Datadog Cloud SIEM design partners.

In this post, we explain why a model like Mambark is necessary, how we built it, how it performs on public benchmarks, and how it fits together with AI agents in a detection pipeline that's inexpensive enough to run at fleet scale and precise enough to perform investigations.

Why agents alone can't watch your logs

LLM agents are among the strongest reasoning systems available today. Given a suspicious authentication event, an agent can retrieve the user's recent activity, check an asset's patch level, consult a threat intelligence feed, and explain whether the event represents a genuine threat. That's the kind of investigation a security analyst performs, and the kind of work that doesn't scale to a raw event stream.

For a large enterprise, scoring every security-relevant event with a frontier LLM could cost tens of millions of dollars a day. Reasoning is powerful, but it's too expensive to apply to the entire event stream.

You might wonder whether the agent could simply query the logs interactively instead of reading every event. In principle, it can. But interactive search finds only what you already know to look for. Spotting a single anomalous event hidden among billions, when no rule or query describes it in advance, is a ranking problem rather than a search problem. Someone has to score every straw to find the needle, and that someone cannot be a frontier LLM.

The solution is a classic two-stage design borrowed from information retrieval: a fast, inexpensive retriever narrows the field, and a slower, more expensive reader reasons over the shortlist. In security terms, the agent is the reader. It needs a retriever that has high recall—one that scores every event before passing only the most suspicious candidates to the agent. That retriever is Mambark.

Mambark: A small language model that scores every event

Mambark starts from a simple observation: Any next-token language model is already an anomaly detector. A model trained to predict the next event in a stream assigns each observed event a probability. If the model assigns a low probability because it did not expect the event given everything that came before, the event is, by definition, surprising. Its negative log-likelihood becomes an anomaly score, and it requires no labels to compute.

To turn that observation into a fleet-scale event scorer, we made two key design choices.

First, the architecture and pretraining. Mambark is a Mamba selective state-space model with about 96.9 million parameters. Starting from the open mamba-130m backbone, we pretrained it for next-event prediction across hundreds of billions of security events, teaching it the structure of normal activity. Unlike a transformer, whose attention cost grows quadratically with sequence length, a state-space model runs in linear time. This matters because real attacks often unfold slowly: A session can span thousands of events, and a slow-and-low intrusion many more. Mambark can condition on contexts spanning tens of thousands of events on a single GPU, where a comparable transformer would not fit.

Second, a uniform representation. Security telemetry data comes in many forms, including syslog, authentication records, network flows, and insider-threat activity. We normalize every source the same way: We group events by entity, flatten each event into a fixed field structure, and hash every field into a shared vocabulary. The model never sees which data source produced a token. The only thing that changes from one source to the next is the set of fields extracted from each event. The architecture, tokenizer, and training recipe remain identical across every benchmark, with no hand-engineered features.

Mambark delivers strong results across public benchmarks

Mambark's role in our detection pipeline is to score every event and forward only the most suspicious candidates to an AI agent for deeper investigation. The public benchmarks in this section evaluate how well a single backbone generalizes across diverse security datasets. The next section shows how Mambark fits into the larger detection pipeline.

Benchmark	Metric	Mambark	SOTA	SOTA source
Thunderbird	F1	1.000	0.990	Guo et al. 2024
BGL	F1	0.999	0.990	Guo et al. 2024
UNSW-NB15	F1	0.994	0.990	More et al. 2024
LANL Cyber1	AUC	0.987	0.990	King and Huang, 2022
HDFS (block)	F1	0.941	0.980	Guo et al. 2024
CERT Insider	AUC	0.879	0.910	Le et al. 2024

Despite being designed for efficiency and versatility, Mambark matches or exceeds previously published results on Thunderbird (F1 1.000), BGL (F1 0.999), and UNSW-NB15 (F1 0.994), while remaining competitive on LANL Cyber1 (AUC 0.987), HDFS (F1 0.941, AUC 1.000), and CERT Insider Threat (AUC 0.879).

Across the established log anomaly benchmarks, including Thunderbird, BGL, HDFS, and UNSW-NB15, Mambark reaches perfect or near-perfect performance. On the more challenging datasets such as LANL Cyber1 and CERT, where a small number of malicious events hide among millions of benign ones, it remains competitive with specialized, hand-tuned architectures. It achieves these results using the same backbone across every modality, demonstrating that a single backbone can generalize across diverse security tasks.

Mambark and the agent: A two-stage pipeline

In production, the two components work together as a single detection pipeline.

Stage one is Mambark. It scores every event in the stream in a single forward pass and promotes only the most surprising ones—on the order of 10,000 candidates a day out of 10 billion events. That sweep costs on the order of tens of dollars per day on commodity GPUs.

Stage two is the agent. It receives the shortlist, runs a full multistep investigation on each candidate, and produces an explained, actionable verdict. Because it operates only on a prescreened shortlist, its cost also remains in the tens of dollars per day, roughly four orders of magnitude less than applying an agent to the raw event stream.

The value of this architecture extends beyond cost. Mambark provides recall at scale: Every event is scored, rather than relying solely on human-authored rules to surface suspicious activity. The agent provides precision and explanation: Each surfaced candidate is investigated with the full reasoning capacity of a frontier model. Running the agent only on prescreened candidates also bounds the blast radius of an overconfident or hallucinated action. Neither component is sufficient alone. Together they reach an operating point that signature rules, statistical UEBA, and standalone LLM agents struggle to match.

Why this architecture works

The agent is what you see, but the model underneath it is what makes it practical to apply a reasoning agent to your security stream. Mambark follows the recipe that made GPT work (Radford et al., 2018): Pretrain a generative model at scale (here, by predicting the next event in a log stream), then fine-tune it for each downstream task (here, by classifying which events are worth the agent's attention). That single backbone transfers to system logs, authentication traces, network flows, and insider-threat activity with only a short fine-tune and a per-source field list—no feature engineering—and it runs efficiently enough to score every event. That model is what makes this approach to security detection faster, broader, and more economical than rule-and-threshold systems or agent-only approaches.

This detection mode is being tested with a small group of Datadog Cloud SIEM design partners and is not yet a generally available product. We are sharing it now because it points toward where our security products are headed: a next generation in which a small, universal model reads every event and an agent reasons over what it surfaces.

Interested in building systems like Mambark? Check out our open engineering roles and help shape the next generation of AI-powered security detection.

References

• Generative pretraining + fine-tuning (the recipe). Radford, Narasimhan, Salimans, and Sutskever, Improving Language Understanding by Generative Pre-Training (GPT-1), OpenAI, 2018. https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf.

• Mamba (the model architecture). Gu and Dao, Mamba: Linear-Time Sequence Modeling with Selective State Spaces, COLM, 2024. https://arxiv.org/abs/2312.00752.

• BGL and Thunderbird logs. Oliner and Stearley, What Supercomputers Say: A Study of Five System Logs, IEEE/IFIP DSN, 2007. https://doi.org/10.1109/DSN.2007.103.

• HDFS logs. Xu et al., Detecting Large-Scale System Problems by Mining Console Logs, ACM SOSP, 2009. https://doi.org/10.1145/1629575.1629587.

• UNSW-NB15 network intrusion. Moustafa and Slay, UNSW-NB15: A Comprehensive Data Set for Network Intrusion Detection Systems, MilCIS, 2015. https://doi.org/10.1109/MilCIS.2015.7348942.

• LANL Cyber1 authentication events. Kent, Comprehensive, Multi-Source Cyber-Security Events, Los Alamos National Laboratory, 2015. https://csr.lanl.gov/data/cyber1/.

• CERT insider-threat data. Glasser and Lindauer, Bridging the Gap: A Pragmatic Approach to Generating Insider Threat Data, IEEE Security and Privacy Workshops, 2013. https://doi.org/10.1109/SPW.2013.37.

• Euler (lateral-movement state of the art). King and Huang, Euler: Detecting Network Lateral Movement via Scalable Temporal Link Prediction, NDSS, 2022. https://doi.org/10.14722/ndss.2022.24107.

• LogFormer (log-anomaly state of the art). Guo et al., LogFormer: A Pre-train and Tuning Pipeline for Log Anomaly Detection, AAAI, 2024. https://doi.org/10.1609/aaai.v38i1.27764.

• UNSW-NB15 evaluation. More et al., Enhanced Intrusion Detection Systems Performance with UNSW-NB15 Data Analysis. Algorithms 17 (2): 64. https://doi.org/10.3390/a17020064.

• CERT insider-threat detection. Le and Zincir-Heywood, Anomaly Detection for Insider Threats Using Unsupervised Ensembles. IEEE Transactions on Network and Service Management 18 (2): 1152–64. https://doi.org/10.1109/TNSM.2021.3071928.