What are silent LLM API failures?

Silent LLM API failures are errors that return a successful HTTP status (usually 200) while the actual response is broken. Examples include OpenAI's finish_reason: content_filter with empty content, Anthropic's stop_reason: refusal, and a 529 overloaded_error delivered mid-stream after a 200. Because the transport status says success, status-code monitoring never fires and the failure is invisible until a user reports it.

Why do some LLM API errors return HTTP 200?

Content filtering and moderation run as part of the model inference pipeline, not as request validation. Because the request itself was accepted and processed, the HTTP status stays 200. The failure is reported as a field in the response body (finish_reason, stop_reason) or as an event inside a stream, rather than as an HTTP error status. To detect these you must inspect the response body, not the status code.

How do I monitor for LLM failures my dashboard misses?

Log the response body on every call, not just non-2xx responses, and capture the fields that carry the real outcome: finish_reason, stop_reason, stream error events, and content filter results. Alert on the rate of these outcomes over time rather than individual occurrences, since a spike usually means a prompt pattern, model change, or provider capacity event. Status-code dashboards alone cannot see any of these.

Which LLM providers have these invisible errors?

All major providers. OpenAI has finish_reason: content_filter and invalid_prompt moderation flags; Anthropic has stop_reason: refusal and mid-stream overloaded_error; Azure OpenAI has per-deployment content_filter behavior that differs between deployments. The common thread is that the HTTP status reports success or hides the real cause, so the failure lives in the response body.

🚫 Silent Failures

The LLM API Errors Your Monitoring Can’t See

200 OK is a lie. Some of the most damaging failures in LLM apps arrive with a successful HTTP status: a response that is silently filtered, a refusal in the body, a 529 that shows up after the 200. Your uptime monitor sees green. Your users see a broken experience. This is the class of error status-code dashboards are structurally blind to, and it is exactly the gap Dstl8 exists to close.

Providers: OpenAI · Anthropic · Azure OpenAI Common thread: HTTP says success, the application failed Verified: July 2026 (third-party API behavior, re-checked quarterly)

TL;DR: Uptime monitors watch HTTP status codes. A whole family of LLM API failures returns 200 while withholding, refusing, or truncating the actual response. If your monitoring only looks at status codes, these are invisible. Below: the specific errors, why each one hides, and how to see them.

Silent failures (200 OK, broken UX)

The response looks successful at the transport layer. The failure is a field in the body, or an event in the stream, that status-code monitoring never inspects.

False-positive moderation blocks

These fail loudly with a non-2xx status, but the reason is a content verdict dressed as a format or policy error, so naive error handling never catches the real pattern.

Why this whole class is invisible

Uptime checks, health probes, and status dashboards all key on HTTP status. A 200 reads as healthy. Detecting any of the errors above requires inspecting the response body, or the stream, on every call, which almost no team logs or alerts on. The tooling teams trust to tell them something broke is looking at the one layer where nothing looks broken.

How Dstl8 sees what your monitor can’t

Dstl8 analyzes the actual text of every log message, regardless of severity or status code, so a silent content_filter, a body-level refusal, or a mid-stream 529 shows up as a recurring pattern you can watch over time, instead of hiding as a 200 in your access log. When one of these spikes after a deploy or model change, Möbius flags it and points at the correlated cause.

Frequently asked questions

What are silent LLM API failures?: Errors that return a successful HTTP status (usually 200) while the actual response is broken: finish_reason: content_filter with empty content, stop_reason: refusal, or a 529 delivered mid-stream after a 200. Status-code monitoring never fires, so the failure is invisible until a user reports it.
Why do some LLM API errors return HTTP 200?: Content filtering and moderation run as part of the model inference pipeline, not request validation. The request was accepted and processed, so the status stays 200. The failure is reported as a field in the body or an event in the stream. You have to inspect the body, not the status code.
How do I monitor for LLM failures my dashboard misses?: Log the response body on every call, not just non-2xx, and capture finish_reason, stop_reason, stream error events, and content filter results. Alert on the rate over time, since a spike usually means a prompt pattern, model change, or capacity event.
Which LLM providers have these invisible errors?: All major providers. OpenAI (finish_reason: content_filter, invalid_prompt), Anthropic (stop_reason: refusal, mid-stream overloaded_error), and Azure OpenAI (per-deployment content_filter differences). The common thread: the HTTP status reports success or hides the cause.