FilingsIQ — Verifiable RAG over SEC Filings

LLMs answer financial questions fluently and sometimes wrongly — a fabricated revenue figure stated with total confidence is worse than no answer at all. Grounding every claim in a source, and actually checking derived numbers rather than trusting the model's arithmetic, is what turns a fluent guesser into something a financial analyst could rely on.

Real 10-K filings are pulled from EDGAR and parsed with table-atomic extraction, so a balance sheet round-trips into a clean markdown grid with row labels and fiscal-year columns intact rather than dissolving into unstructured text.

Retrieval is hybrid (SQLite + BM25) with hard metadata filtering applied before retrieval, not after — including fiscal-year-end edge cases, like resolving a bare 'in 2024' for NVIDIA (fiscal year ends late January) to FY2025, the label that actually covers most of calendar 2024. Generation and embeddings both run locally — Qwen2.5-3B-Instruct and BAAI/bge-small-en-v1.5 — with no API key required.

The verifier is the core of the system: every derived claim (growth rate, margin, delta) is independently recomputed in plain Python and checked against what the model claimed. A claim that fails verification gets exactly one regeneration attempt with the failure as feedback; if it still fails, the system refuses rather than looping.

Verified live against real filings (Apple, Microsoft, Nvidia), not synthetic fixtures. The verifier is proven correct on four cases: a correct claim passes, a fabricated number is caught by the provenance check, a correct derived arithmetic claim (YoY growth) passes, and a hallucinated arithmetic claim is caught by recomputation. An unrecognised formula name fails closed rather than being trusted.

The company/fiscal-year metadata filter is a SQL WHERE clause, so retrieval precision holds as the corpus scales to more companies — cross-company questions without a named company are the harder case that gets worse as the pool grows, which is documented rather than glossed over.

A verifier that recomputes numbers instead of trusting the model's arithmetic catches exactly the failure mode that matters in finance. Refusing to answer when the corpus doesn't support a claim is a feature worth designing for, not an edge case to patch over later.