probabilistic outputs
The same input gives different answers
Pass/fail testing was built for deterministic software. An AI system needs its quality described as a distribution — and monitored as one.
AI governance · business intelligence for AI systems
The hard part of AI isn’t building it.
It’s knowing it works.
AI outputs are probabilistic, ground truth is scarce, and quality drifts without anyone noticing. ai-gov is Andrena’s governance platform: the measurement layer that turns your AI systems into something you can monitor, audit, and improve with confidence.
scarce ground truth
Nobody tells you the right answer
Correctness is not in the logs. It has to be deliberately collected — from your experts, your users, and your outcomes — before any metric means anything.
silent drift
Quality decays without an alarm
Models get swapped, prompts get edited, users change behavior. Last quarter’s spot-check says nothing about today unless something is still measuring.
01The evaluator language
A language for saying what “good” means
Every AI system carries expectations — accurate, grounded, on-brand, safe. Most teams state them in prose and hope. We express them in a precise evaluator language: each expectation becomes a named, versioned check with a defined scale.
“Replies must be grounded in the customer’s thread” stops being a hope and becomes a measurement — one your dashboards, reports, and audits can hold the system to, release after release.
expectations, made measurable
“Replies stay grounded in the thread” groundedness · pass/fail per output
“Summaries stay under 200 words” response-length · limit: 200
“Drafts read like our approved replies” similarity-to-approved · score 0–1
“Staff would actually send this” staff-decision · human judgment
02Lifecycle reporting
One report, from first prototype to production
Every phase of a system’s life — pre-launch testing, pilot, live traffic — produces the same records and rolls up into the same report: what was expected, what was measured, what still needs evidence, and who owes it. One self-contained page, no login, no build step — it survives being emailed to a stakeholder.
A live report, rendered by our SDK from a real test run. Scroll inside the frame — the pending evidence, the review queue and the compliance checks are all part of the document.
03Human-in-the-loop
Human judgment, routed to where it counts
Some calls only people can make. The platform collects those judgments deliberately: outputs are queued for review, shared with the experts who can actually label them, and tracked until enough evidence exists to settle the question.
Automated checks watch everything; people are reserved for the decisions that need them — and every judgment they give becomes ground truth that sharpens the automated side.
review queue · direct-response-agent
“Thanks for flagging this — the outage affected accounts created before May. Your data was not impacted, and here’s how to confirm…”
✓ would send edit: tone edit: facts would not send
reviewed 18 of 40
04Data analysis & ML models
Data science fundamentals, not vibes
Measurement is statistics, and we treat it that way. Quality is a distribution, not an anecdote. Consistency is measured across repeated runs, not assumed. Classification quality is scored against collected ground truth, and confidence models are calibrated on real outcomes.
The result is business intelligence you can defend in a room of skeptics — and a foundation solid enough to train on.
similarity-to-approved · 64 outputs
0.71 mean 0.74 p50 0.88 p90
0.91 consistency 0.86 precision 0.79 recall
05Developer tooling
Tooling your engineers will actually use
Governance that lives outside the development loop gets ignored. Ours ships as an SDK: simulate a system on a laptop before launch, run the same checks in CI, and graduate to the platform without changing shape.
That is how we support teams through rapid prototyping, hardening, and long-term maintenance — one set of rails from the first experiment to the system nobody is allowed to break.
local-first
$ pip install ai-gov-sdk
$ python simulate.py --html
✓ 64 outputs recorded
✓ 210 measurements collected
✓ report.html writtenSee what this looks like in practice
The platform tour shows the dashboards, reports, and review tools your team would live in.