Testing & Verification
The classification contract is "not valid until proven by automated tests in CI." This page lists the test layers, their results, and how to reproduce them.
Test layers
| Layer | Covers | Latest result |
|---|---|---|
| Classification contract tests | Positive, negative and override cases for QueryType and risk signals | Passing in CI (mutation-survival enforced) |
| Mutation tests ("Survival Proof") | Removing any rule (e.g. IntoClause detection) fails red; restoring passes green | Enforced in CI |
| Evasion corpus | Comments, dollar strings, Unicode confusables, mixed encodings, multi-statement payloads, extended-protocol parameter bypass attempts | Growing in CI |
| Regression suite (core) | End-to-end decision paths over the HTTP API | 31/31 passed |
| Chaos suite (core) | DB down, slow backend, cancel, reconnect, policy failure | 10/10 passed |
| Audit restart tests | Evidence continuity across restarts | Passing |
| Wire regression (lab) | Simple Query intercept and enforce end-to-end | 65/65 passed (lab) |
| Wire chaos (lab) | Backend down, wrong target, slow backend, cancel, reconnect | 14/14 passed (lab) |
| Real driver matrix | psycopg3, asyncpg, Prisma, pgJDBC single-statement via AXIS | 37 passed each (2026-07-05) |
| Stress / throughput | Rate limits, oversized inputs, concurrency | Passing within configured limits |
Operator verification drill
Health: GET /health → status: ok.
Policy: GET /policy/status → expected version, operator_auth_enabled.
Evidence: GET /evidence/verify → status: ok.
Decision drill: dry-run a known-write and a known-block SQL; confirm decisions and would_execute: false.
Approval drill: submit a REQUIRE_APPROVAL write; resolve it; retry with the approval_id; confirm execution and evidence.
Shadow check: compare decisions between shadow and enforce for the same corpus (decisions must match).
Reproducing test results
- Core and chaos suites run via the project's test harness against a local stack; see the Installation page for prerequisites.
- The driver matrix requires live drivers (psycopg3, asyncpg, Prisma, JDBC) against a running AXIS + PostgreSQL stack.
- Lab wire tests require
AXIS_PGWIRE_ENABLED=truewithAXIS_PGWIRE_LAB_MODE=trueand the RFC harness.
What tests do not prove
- Driver tests prove single-statement enforcement; they do not prove universal ORM compatibility or batch/copy support.
- Lab wire tests are for the POC; they do not certify the wire path for production.
- No external audit, compliance certification, or third-party penetration test is claimed.
Run the verification drill before and after every policy activation, and include it in the pilot acceptance checklist.
Related: Pilot · Security Review · Changelog