Aivist Verify

Most tools hand you a pile of maybe.

Suspected IDORs you still have to verify by hand, at 2 a.m., one by one. Aivist Verify hands you proof, or an honest “no.” You give it a candidate — one endpoint, two identities — and it tells you whether the attacker actually crosses a user boundary into the victim's resource, with a reproducible evidence chain attached.

AI proposes. Code disposes.

The model reads messy, business-specific traffic and picks what is worth checking — code can't guess that. Then deterministic gates re-check the proposal against the attack's own runtime bytes. They can only downgrade. The model's opinion is not an input to the final verdict.

Recorded separately. Every result stores the model's raw verdict (ai_verdict_raw) and the gate's decision (guard_override) as distinct fields, so the evidence chain literally reads “the model proposed X; code decided Y.”

430 runs. Zero false positives.

Five confirmation shapes against two structurally different, self-contained vulnerable labs — integer ids and UUID ids — driven by a real gemini-2.5-pro loop, freshly seeded every run.

This is a controlled benchmark on two labs — not a tally of real-world kills. Clean real-world confirmations are genuinely rare, and this project's honest headline is discriminative power plus a zero-false-positive discipline, not a screen full of CONFIRMED. Every number here is recomputed from the committed artifact scripts/measure/results/sweep_highN.jsonl.

Validated on software we didn't write.

The benchmark runs on labs we built. So we took the engine to two public, deliberately-vulnerable targets we did not write — crAPI and VAmPI — and ran it against their documented BOLAs. Nine engine runs, two real cross-user BOLAs confirmed, zero false positives. Every run is archived verbatim in the repo.

Scope of this claim. These are nine hand-verified runs on two targets — an engineering signal, not the statistical zero-false-positive record the lab benchmark provides. Ground truth here was established by hand before each run; the lab benchmark's ground truth is independent and machine-checkable.

What it misses, on purpose.

The same probe that fixed the community-feed false positive causes a deliberate miss. When every authenticated user can read a resource, the engine refuses to confirm it — even when it is a real vulnerability. crAPI's mechanic_report and VAmPI's /books are exactly that: real BOLAs that flip to inconclusive the moment a bystander token is supplied.

This is not a bug, and it is not fixable by a better algorithm. Black-box, “every authenticated user can read this because authorization is broken” and “every authenticated user can read this by design” produce byte-identical responses. The distinguishing fact is the API author's intent, and intent is not present in any HTTP response.

A real confirmation, start to finish.

aivist demo boots a built-in vulnerable lab and confirms a real cross-user write end to end — no Docker, no target, no tokens. This is the actual, unmodified render:

[CONFIRMED]  cross-user write (BOLA) - POST /api/users/1/display-name
  Verdict: verified  (confirming channel: write-record read-back)  (guard_override=write_record_readback_decisive)
  Basis: a deterministic code gate authorized this (write-then-independent-read proof), not the model's opinion alone.
  What the engine proved:
    Wrote as the attacker, then read the object back through a different
    endpoint as another identity. A record carrying the victim's object id and
    the exact value this attack wrote was found on that read-back, so the
    unauthorized write provably persisted. That persisted read-back is the
    proof.
  Here's what happened - the Evidence chain (physical bytes the engine actually exchanged):
    1. Sent as the attacker:
       POST http://127.0.0.1:8001/api/users/2/display-name
       Content-Type: application/json
       Authorization: ***REDACTED***
       Body: {"display_name": "vm-1-eae9d9d20e"}
    2. Attack response received:
       -> HTTP 200 | Content-Length: 15
       {"status":"ok"}
    3. What decided it (byte-level):
       - the write was attributed to the attacker's own identity  (caller_identity=same_as_caller)
       - this attack's unique value was present in the read-back body - the write landed  (payload_causality=confirmed_in_body)
       - the anchoring read-back path was not found (observe-only)  (anchoring_result=failed_path_not_found)
    4. Not taken as proof:
       - the model's raw opinion alone did NOT decide this - the deterministic code channel did  (ai_verdict_raw=verified)
  Re-runnable evidence package (fill <REDACTED> from YOUR config; never a live token):
    # 1) The attack request - reproduces the cross-user access AS THE ATTACKER:
    curl -X POST 'http://127.0.0.1:8001/api/users/2/display-name' \
    -H 'Content-Type: application/json' \
    -H 'Authorization: <REDACTED>' \
    --data '{"display_name": "vm-1-eae9d9d20e"}'
  So what / Next step:
    A real cross-user access bug: the attacker could write to the victim's
    object. It is reproducible (the request above).
    Next: report it, or fix by enforcing an ownership check on this endpoint.
  [lab oracle] lab label=REAL (expects verified); engine said 'verified' - AGREES. (informational only; NEVER an input to the verdict)

And when the attacker gains nothing, the tool says so: a run with no cross-user effect returns [REFUTED]; a run that gets rate-limited or hits an expired token returns [NOT DATA] and claims no verdict at all — neither safe nor vulnerable. It refuses to guess.

Three commands.

Python 3.11+ and a Gemini API key.

Authorized targets only. Confirming a BOLA/IDOR sends real cross-user requests with real credentials. Only run against systems you own or have explicit written permission to test. It is built for locally-run targets you control, has no authentication of its own, and is not a mass-scanner.