We used to grade AI vendors. Here is why we stopped.

Until 8 September 2026, Vannus published a letter grade for the AI tools in its catalog, from A+ to F. Beside it sat a tier, one of five bands named Sovereign, Durable, Moderate, Fragile and Wrapper, and behind both a score that combined nine criteria. We withdrew all three on 8 September. This page records what they were, why they went, what replaced them, and every place they lingered after that date, with the day each came off. We are not taking the history down, because a record that quietly rewrote its own past would be a strange thing to ask anyone to trust.

What we published

The grade was built from nine criteria: data sovereignty, allied infrastructure, training privacy, conditional privacy, compliance standard, operational resilience, exit portability, real-world utility, and a caution flag. Each tool got a letter, a band and a place in the catalog’s order.

What went wrong before it went

Twice in July the grading went wrong at scale, and both times we said so in public. Those posts stay up, labeled as history:

• 20 July. The heaviest criterion, meant to measure whether a tool runs its own model or resells someone else’s, credited a tool by whether its name appeared on a list. It never read what the tool runs. Fixing it changed 233 of 287 grades. We re-graded every AI tool →
• 27 July. Adding 69 tools, the first attempt would have published 67 failing grades on real companies because of gaps in our own data, and the second an F on the most privacy-respecting vendor in the batch. We held them back twice →

Why we stopped

Fixing the inputs was not the problem. The problem was what a grade is. A grade was our arithmetic, presented in the record’s own voice, and it could not be quoted from anything. A vendor could dispute it forever, a buyer could not check it against the vendor’s own documents, and an auditor could not use it, because nothing the vendor published produced it: it was our weighting of our reading. A record that exists to be cited cannot rest on a number the vendor’s own words do not support. We decided on 7 September and withdrew the grade, the tier and the score on 8 September.

What replaced it

A determination: the legal entity a buyer actually contracts with, taken from the vendor’s own terms with a source and the date it was read, that entity’s country, the governing law, what the answer depends on, and whether US legal reach follows under a rule we publish. Every field is either a fact with a citation or marked as not established. It is the unit of the record now, field by field, and it is served as data, for example /determination/github-copilot.

Where it lingered, and the day each came off

Withdrawing a grading system is not one edit. It had reached pages, emails, API fields and a paid report, and several of those kept showing it after 8 September. Each is listed here with the date it came off and the commit that took it off.

• 9 September. The search page’s metadata still described grades (3b0433ca). Our own self-assessment page still showed Vannus’s own D (fcb6f858). The $299 report’s description still told buyers it contained grades (bf5ca569). The Terms, the catalog’s buttons and the contract wording had not been updated (60988212). A blog post was still presenting the grading system to search engines (394a035e).
• 13 September. The sovereignty assessment behind each tool page still returned a trust tier, a risk score, a badge and a recommendation (b16fd709).
• 18 September. The free stack check still showed an empty “Verdict” heading and a “Sovereign/Durable 0%” tile (8ed90fcb).
• 19 September. Every tool page still pointed at “the grade above”, and our machine-readable file for AI assistants still told them to quote one (6699c04f).
• 21 September. A public API route still computed a vendor trust score on request, from inputs the caller supplied; it was closed to the public when every route was put behind our access check unless chosen public (0e35a067).
• 22 September. Two public API routes still answered with a risk level and a compliance verdict, beside eleven retired catalogues; all were closed to the public (ee5541ae).
• 24 September. The $299 report still printed an exposure band and Keep, Replace or Drop verdicts, the free check an amber band, and the Concierge page nine-criterion scorecards (46b67b82).
• 29 September. The catalog’s default order was still the retired score, under the name “Vannus Rank”. Every tool page still offered to publish a vendor’s reply “next to our grade”. And the methodology file served to AI assistants still carried the retired weights and tier thresholds (f2fd78cd). The same day, the public catalog API stopped declaring four grade fields that had been empty since 8 September.

If you hold a Vannus grade, tier or score, from a cached page, a search snippet or an old report, it is withdrawn. Please do not cite it, and do not describe a vendor as graded by Vannus. If you find one still being served, tell us and we will add it to the list above with the date it came off.

This post is a record, not a dashboard. Its dates are facts as of the date above. Where one moves, this post is corrected in place with a dated line here — never edited silently.
Corrections: none as of 29 September 2026.

Scope. This post describes Vannus’s own methodology and its history. It is not legal advice and makes no statement about any vendor. Vannus is operated by PRAXIS AI LLC.