Most production incidents follow the same painful script. An alert fires at 3 AM. Someone gets paged. They spend 20 minutes staring at dashboards, trying to figure out what changed right before things broke. Then they make a decision.
Datadog just changed the economics of that entirely.
The company’s engineering team fine-tuned Qwen3.5-9B — a 9-billion-parameter open-source language model — specifically on the task of change attribution during incident investigations. Change attribution means answering one question: what deployment, configuration change, or code push happened right before this alert fired?
The results are concrete. The specialized smaller model achieved 87% of the accuracy of the much larger teacher model it was trained from. Cost per investigation: $0.003. A single 40GB GPU can handle roughly 100,000 investigations per week. That’s not a marginal improvement — it’s a structural shift in how on-call engineering scales.
Why Model Size Matters
General-purpose AI models are expensive to run at scale because they’re designed to do everything. A large frontier model can write poetry, explain quantum physics, and debug code. That breadth costs money per query.
When you only need a model to do one thing — look at a set of traces and identify what changed — you don’t need all that capability. You need depth in a narrow domain.
Fine-tuning is the process of taking a general model and training it further on specific examples of the task you care about. Datadog trained Qwen3.5-9B on thousands of real investigation traces from their larger GLM-5.3 model, essentially teaching the smaller model to replicate the reasoning patterns of the larger one — but for this specific problem only. The result is a model that’s dramatically cheaper to run while doing the one job it needs to do nearly as well.
What This Means for Engineering Teams
AI-powered incident response is already here. Tools like Datadog Bits AI can receive an alert, correlate telemetry across services, and surface a probable root cause — all before a human opens their laptop. The Qwen3.5-9B breakthrough means that capability can run at enterprise scale without an enterprise-sized inference bill.
For smaller teams, the implications are even bigger. A two-person engineering team can’t staff a 24/7 on-call rotation. AI-powered first-pass investigation doesn’t replace engineers — it gives them back the 20 minutes of dashboard archaeology before every actual decision. That’s the difference between a rested engineer making a focused call and an exhausted one making a frantic one.
There’s also a cost argument that’s hard to ignore. At $0.003 per investigation, a team running 500 incident investigations a month spends $1.50 on AI-assisted triage. The economics of on-call are changing. Fast.
The companies adopting this kind of AI-augmented operations today are building a durable advantage: faster incident resolution, lower toil, and on-call rotations that don’t burn out your best engineers.
Want to explore how AI-powered monitoring could reduce incident response time for your team? Let’s talk.