Log Clustering and Triage Agent over MCP

What This Builds

There is a point in every incident where the problem is not finding logs but understanding them. Formats drift, error messages mutate, and dashboards describe yesterday’s system. Engineers keep asking: what broke, how widespread is it, where do we look first.

This recipe builds a triage layer that answers those questions. A small, linear pipeline turns raw logs into severity-aware clusters of similar failures (stable across parameter noise like request ids and timestamps), keeps a representative exemplar per cluster, and computes a coverage metric that tells you whether parsing kept up. Those clustered views are exposed as read-only MCP tools, and a LangGraph agent calls them to answer plain-English questions.

The agent answers fast because it is not guessing over raw text — it calls deterministic tools backed by tables you already computed. The artifacts (CSV/JSON clusters, a Markdown report) are the audit trail; the agent is the interface.

The Stack

• Fenic is a PySpark-inspired DataFrame framework built for AI/agentic apps. You manipulate logs like a table but get first-class text extraction, embeddings, and LLM operators with explicit, batch-friendly config — avoiding both brittle regex and unpredictable one-click LLM summarization.
• Model Context Protocol exposes the computed views (clusters, exemplars, coverage, raw-lines-by-cluster) as read-only tools the agent can call.
• LangGraph runs the agent that maps a natural-language question to the right MCP tool calls.

Step-by-Step Outline

1. Parse without brittleness: define a few Fenic templates (named fields, not capture groups), unnest them, and coalesce to a canonical schema (timestamp, level, service, message).
2. Cluster: embed messages, then group similar failures so parameter noise collapses into one cluster. Keep an exemplar line per cluster.
3. Score coverage: compute the fraction of lines that parsed into the canonical schema so you know whether the signal is trustworthy today.
4. Expose tools over MCP: “top clusters above WARN in the last hour”, “only ERRORs for a given service”, “raw lines for cluster N”, “coverage check”. All read-only.
5. Wire the agent: point a LangGraph agent at the MCP server so questions like “only errors in payments” resolve to deterministic tool calls.
6. Persist artifacts: write clusters to CSV/JSON and a compact Markdown report each run for dashboards and human review.

Source

• Typedef: Build an LLM Agent for Log Clustering and Triage (runnable Colab and typedef-ai/fenic-examples repo)