The Shadow AI Audit — a 90-minute checklist for ops leaders.
Most ops leaders running a 20–200 person team already know the headline: their team is using AI tools the company doesn't pay for, and paying for AI tools the team doesn't use. What they don't have is a 90-minute checklist that produces an actual inventory — not a Notion doc full of guesses.
This post is that checklist. It assumes you have no SaaS-management platform (you don't need one for this), no procurement system (this is upstream of procurement), and no IT helpdesk handing you reports (it doesn't help here either). What you need is a calendar block, three tabs open, and the willingness to ask a slightly awkward question.
Why "shadow AI" is the harder half of the audit
Your AP system can tell you what AI tools you pay for. That's the easy half. The harder half is what your team actually uses — including the personal ChatGPT account three of your engineers run, the Claude Pro subscription one of them expenses through a different category, the Grammarly Premium your marketing lead has billed to her personal card, and the Otter.ai bot two members of your sales team installed in their personal Zoom accounts.
None of those show up in procurement. All of them are part of your AI surface area — with the same training-privacy, sovereignty, and exit-portability risks the contracted tools have. The audit isn't worth the calendar block if it skips them.
The 90-minute, 4-step checklist
Run these in order. Don't skip Step 1 to get to the consolidation conversation — the inventory is what makes Step 4 honest.
Pull what AP shows.
Open your accounting system or expense tool and filter the last 6 months for vendor names that include “AI,” “GPT,” “Copilot,” “Claude,” “Notion,” “Otter,” “Grammarly,” “Zapier,” “Make.” Then run a second pass for every line item under $50/month: that's where the personal-card-reimbursed AI subscriptions hide.
You want three columns: tool name, monthly cost, who signed up. The last column is the one most ops leaders skip and the one that makes Step 2 work.
Ask the awkward question.
Send three Slack DMs — one to your engineering lead, one to your marketing or content lead, one to your customer-facing lead (sales or support). Identical message:
Quick one. Doing a 30-day AI tool inventory. Can you reply with every AI tool you or your team uses at least weekly? Include personal accounts. No judgment, just trying to get to ground truth.
You will get three things back: (1) names of tools that match what's in AP, (2) names of tools that don't, (3) at least one tool you didn't know existed. The third bucket is where the audit pays for itself. Add everything to your inventory with a note: “reported by [name], not in AP.”
Score against the trust dimensions.
For each tool in the inventory, mark four things:
- Used daily, weekly, monthly, or never. (Self-reported from Step 2, cross-checked against any usage logs your tools expose.)
- Sovereignty: US-hosted, allied-hosted, or untracked? Tools without a clear answer get flagged.
- Training privacy: contractual zero-training, opt-out available, or default-on training? Most teams don't know — that's fine, mark it “unknown” and move on.
- Has SOC 2 Type II? Yes / no / unknown.
Don't try to be exhaustive in 15 minutes. You're flagging which tools need a follow-up review, not producing a final score. If you want this step automated, paste your tool list into the Rate my AI stack tool — it returns the same four answers for every tool already in our catalog, instantly.
Decide three things, in writing.
You should now have a list of 8–15 tools across four columns: name, cost, who uses it, four trust flags. Before you close the document, write three decisions:
- Cancel by next renewal: tools paid for but reported “never used” in Step 2, plus tools used “monthly” or less by a single person.
- Migrate within 60 days: tools used weekly but with a clear sovereignty or training-privacy gap. Replacement candidates exist in the catalog for almost every category.
- Keep and consolidate spend onto: tools used daily across multiple people with a clean trust posture. These are your anchors. Renegotiate them at renewal using the consolidated footprint as leverage.
Save the document. Calendar a 30-minute repeat for 90 days from today. Done.
What this gets you in 90 minutes
If you're at a typical 50–100 person team that's been adding AI subscriptions ad-hoc for the last 12 months, the inventory you've now built is worth roughly $4,000–$12,000 in annualized savings. Most of it comes from three sources: cancelling the never-used tools, downgrading tiers on tools used by fewer people than the plan supports, and renegotiating the kept tools at renewal with leverage that wasn't there before.
The biggest single number you'll find is almost always a wrapper subscription: a UI layer over GPT-4-class foundation models that costs $49–$99/mo and gets used twice a week, when the team's existing ChatGPT or Claude subscription absorbs the same use case at marginal cost. Audit those first. The math is brutal and the migration cost is near zero.
What this doesn't get you
Three things this checklist is honest about not producing:
- The full 9-dimension scoring per tool. Step 3 flags risks. The full scoring (sovereignty, allied infrastructure, training privacy, conditional privacy, compliance, operational resilience, exit portability, real-world utility, caution flag) is what the Vannus manifesto publishes and what powers the catalog. A 90-minute self-audit can't get there.
- Replacement shortlists with migration-effort estimates per “Replace” tool. You'll know which tools to migrate. You won't know to what, or how long it'll take, without research.
- An action plan with owner placeholders and dollar impact per step. Three decisions in Step 4 is the floor. A real 30/60/90-day plan is the next level.
Those three gaps are what the Vannus Concierge audit ($7,500, 14 days) closes — for the teams where the 90-minute version revealed enough waste that the deeper version pays for itself. For everyone else, the 90 minutes you just spent is the highest-leverage ops work you'll do this quarter.
Part 1 (this post): The Shadow AI Audit — a 90-minute checklist.
Part 2: The Free-Tier Trap (coming soon).
Part 3: When NOT to consolidate — the counterargument.
Part 4: AI Procurement RFP template for mid-market teams.
About Vannus — we're an AI tool selection platform built around an elimination-first methodology. No vendor influence, no paid placements. We're paid by buyers (subscriptions and audits), not by the vendors we evaluate. Read the methodology.