Insight · Yalla2x
Writing an AI RFP that vendors cannot game
Score for provability, not polish
Ask every bidder to demonstrate that a specific past decision can be replayed and shown unaltered. Vendors who built on verifiable foundations can do this in minutes; vendors who did not will produce a slide. That single requirement reorders most shortlists.
Require the uncomfortable disclosures
Put these in the mandatory-response section:
- A decision the system got wrong, and how it was detected.
- Exactly what the buyer owns versus rents, including data and keys.
- Where inference runs and under whose jurisdiction.
- Who is accountable, in writing, if a decision causes harm.
Weight honesty in the scoring
Give explicit credit for a vendor that states its limits, in the spirit of an honest scoreboard. A bidder who can tell you what they have not proven is usually safer than one who claims everything works. The procurement questions behind this live in our pre-signing checklist.
Frequently asked
How do I stop AI vendors from gaming our RFP?
Require things a demo cannot fake: replaying a specific past decision and proving it was unaltered, a plain statement of what you own versus rent, where inference runs, and written accountability. Score provability and honesty, not feature counts.
Should accuracy percentages be the main RFP criterion?
No. Self-reported accuracy is easy to assert and hard to verify. Weight the ability to prove and audit decisions, the ownership model, and the vendor's honesty about limitations more heavily than quoted numbers.
What is the single most revealing RFP requirement?
Ask the vendor to replay a real past decision and show it has not been altered. Those built on verifiable foundations do it quickly; the rest produce a slide, which tells you what you need to know.