Skip to content
All projects
LLM Infrastructure/2026

OpenIncident

A production-grade MCP server suite that lets AI agents debug incidents without compromising security.

Status
MIT licensed · 91 passing tests
Role
Solo — server design, policy gateway, threat model, evaluation harness
Timeline
2026

At a glance

Four specialised Model Context Protocol servers behind a policy gateway, built so an on-call AI agent can safely query production databases, metrics, logs, and Kubernetes during an incident. Capability scoping, hash-chained audit trails, and safe-by-construction mutations — validated by 91 tests against real backends.

  • mcp-postgres-safe: schema-aware, cost-limited SQL with PII redaction and role-based table access.
  • mcp-k8s-guarded: pod operations that default to dry-run and require explicit confirmation tokens.
  • Gateway spawns each backend as a stdio subprocess and re-exposes every tool with an explicit, hand-written signature.
  • Outbound responses are scanned for secrets; every call is written to a hash-chained audit log.
  • Threat model ships with executable demos, including live prompt-injection blocking.

Key numbers

4

MCP servers

postgres-safe, observability, runbooks, k8s-guarded

91 passing

Tests

Against real backends, not mocks

10 incidents

Eval harness

Benchmarked with Qwen2.5-7B locally

Hash-chained

Audit trail

Tamper-evident by construction

Tech stack

Core
Python 3.11+uvstdio subprocess modelHand-written tool schemas
Backends
Embedded PostgreSQLSQLite + FTS5Prometheus queriesKubernetes API
Gateway
OAuth2Token-bucket rate limitingOutbound secret scanningHash-chained audit log
Assurance
pytest (real backends)MCP conformance suiteOpenTelemetry tracingPrompt-injection demos

The problem

During an incident, the fastest responder is often the one who can query the database, read the metrics, search the logs, and check the pods — all at once. That is also exactly the access you do not want to hand to an autonomous agent without controls.

Most MCP projects wrap an API and stop. OpenIncident is about the three things that come after: capability scoping, auditability, and mutations that are safe by construction.

The four servers

Each server is narrow and hardened rather than general.

  • mcp-postgres-safe — schema-aware SQL with query-cost limits, PII redaction, and per-role table access.
  • mcp-observability — Prometheus metric queries and structured log search.
  • mcp-runbooks — full-text search over incident procedures and post-mortems.
  • mcp-k8s-guarded — pod operations with dry-run defaults and explicit confirmation tokens for anything destructive.

The policy gateway

The gateway runs as an HTTP server and spawns each backend as a stdio subprocess, then re-exposes every tool with an explicit, hand-written signature. That design choice matters: the tool schema is the real API surface, not a generic pass-through that leaks capability.

On top of that it adds OAuth2 authentication, per-user capability policies, token-bucket rate limiting, outbound secret scanning, response-size enforcement, and a hash-chained audit trail that makes tampering detectable.

Evidence it works

91 tests run against real embedded backends rather than mocks. A conformance suite checks all four servers against the MCP specification. An evaluation harness benchmarks the system against 10 seeded incidents using a local Qwen2.5-7B model, and security demos show prompt-injection attempts being blocked in real time.

What I took from it

Giving a model a tool is easy. Giving it a tool that a security review would approve — scoped, audited, and impossible to misuse into a destructive action — is the actual engineering, and it is most of the work.

Want the details?

The full source, READMEs, evaluation harnesses, and results are on GitHub.

github.com/dariyaDeepanshu/OPENINCIDENT