This is for a founder about to sign with an AI development agency and wants to avoid the two most common outcomes: a relabeled web-dev shop that bolts an API call onto a CRUD app, or a vendor that quietly hands your project to juniors. By the end you will have a concrete checklist and the red flags that predict a bad engagement.
How do I vet an AI development agency before signing?
Ask for one case study close to your use case, one reference you can actually call, and a sample of how they scope work — then verify all three. Most vetting mistakes happen because founders take a portfolio page at face value instead of asking "show me the thing, and show me who built it." A five-minute reference call catches more problems than an hour of reading a website.
- Ask to see a live product, not just a case study screenshot. If they can't demo something real, that's your answer.
- Ask who, by name or role, will be writing your code — and whether that's the same person in your kickoff call.
- Ask for one reference from a client with a similar budget to yours, not their biggest logo.
- Ask how they handle a wrong model choice or a blown architecture assumption discovered in week 3.
- Ask exactly what you own at the end: code, prompts, fine-tuned weights, accounts.
A scorecard you can use during the sales call
Score each agency 1–3 on these five areas right after the call, while the conversation is fresh. Anything averaging below 2 across two or more agencies you're comparing is worth a second, harder conversation before you sign.
| Area | 1 (weak signal) | 2 (okay) | 3 (strong signal) |
|---|---|---|---|
| Technical depth | Buzzwords only ("we use AI") | Names tools and models used | Explains a specific trade-off they made and why |
| Scoping process | Quotes a fixed price on the first call | Asks some clarifying questions first | Proposes a paid discovery phase before a firm quote |
| Proof | Logo wall, no demos | One or two case study write-ups | Live demo of a comparable product, named client |
| Ownership terms | Vague ("you'll have access") | Mentions you own the code | Explicit in writing: code, IP, repo, accounts, all yours |
| Communication cadence | "We'll keep you posted" | Weekly updates promised | Specific cadence named (e.g. daily written updates, weekly demos) |
How do I know if an "AI agency" is actually just a general dev shop rebranded?
Look at their portfolio for AI-specific decisions, not just an AI feature bolted onto a standard app. A real AI shop can tell you why they picked RAG over fine-tuning for a given problem, what they do about hallucination rates, and roughly what a user query costs them in API fees. A rebranded dev shop talks about "integrating ChatGPT" and stops there — no mention of evaluation, guardrails, latency, or cost per request, because they haven't had to solve those problems yet.
A second tell: ask what percentage of their current work is AI versus general software. Directories like Clutch publish this breakdown for listed agencies — firms genuinely focused on AI development typically show 30–100% of work as AI, not 5–10% (source).
What red flags indicate an AI agency will underdeliver?
The biggest red flag is a fixed timeline with no discovery phase. Any agency that quotes an exact price and date for your AI MVP before understanding your data, your users and your constraints is guessing, and you will pay for that guess in scope changes later. A real scoping conversation, even a short one, should come before a firm number.
- No discovery or scoping step. Serious agencies spend time understanding the problem before quoting; see what a discovery sprint actually covers.
- Vague ownership language. "You'll have access to the code" is not the same as "you own the code." Push for the exact words in the contract.
- No mention of model/API costs separate from dev fees. If they don't bring this up, you'll discover it on your first OpenAI invoice.
- Portfolio full of logos, no live demos. A logo wall proves nothing about whether the product worked or still exists.
- Pressure to sign immediately. A legitimate agency with a healthy pipeline does not need to rush you.
- Junior-only delivery team with no senior review. Ask directly who reviews architecture and AI-specific decisions.
What certifications should an AI agency have?
None are strictly required to ship good AI software, but SOC 2 Type II and ISO 27001 are worth asking about if your product touches regulated or sensitive data (health records, financial data, anything under GDPR or similar regimes), because they indicate a vendor has formal security processes, not just good intentions. For an early-stage MVP with no sensitive data yet, don't let the absence of a certification rule out an otherwise strong, smaller team — most agencies at the MVP stage, including boutique and India-based studios, won't have SOC 2 yet, and that's normal rather than disqualifying. What matters more at that stage is whether they can describe concrete practices: least-privilege access to your accounts, NDAs signed before any data changes hands, and a plan for removing their access once the engagement ends. We cover this in more depth in data security when outsourcing AI development.
Should I run a small paid test project before committing to a full build?
It can be useful, but weigh it against the cost of delay. A one- or two-week paid discovery or scoping phase — not a full build — is the better version of this idea: it's short, it's priced as part of the real engagement (ideally credited toward the build if you continue), and it tells you how the team thinks about your actual problem rather than a generic test task. Avoid asking agencies to do free "trial" work; agencies worth hiring will decline, and the ones willing to work for free on spec are rarely the strongest ones. If a short paid discovery phase surfaces a mismatch, you've lost a small, bounded amount rather than a full project's budget.
What's a reasonable minimum project size for a reputable AI agency?
Most Clutch-listed AI agencies set a minimum project size of $25,000 or more, with some premium US firms at $50,000–$100,000+ (source). That floor reflects real AI work: architecture, evaluation, integration, and testing take time regardless of how "simple" the feature looks. Be skeptical of anyone quoting a few hundred dollars for a custom AI build — that price point usually means a thin wrapper around a public API with no real engineering behind it, common among commodity offshore shops (example pricing).
Fixed scope, time-and-materials, or retainer — which pricing model should I choose?
Fixed scope is the right default for a first AI MVP: you know the total cost before you start, and it forces the agency to actually scope the work rather than discover problems on your clock. Time-and-materials makes more sense for ongoing work where the task list changes week to week — iterating on a live product, for instance — because forcing a fixed scope onto open-ended work usually produces padded estimates on both sides. A monthly retainer fits a partnership that has moved past the initial build into steady support, monitoring and incremental features. We go through the trade-offs of each in more depth in fixed price vs. time and materials.
Whichever model an agency proposes, confirm what happens when the scope changes mid-project — a near-certainty on any real AI build once you see the first working version. A fixed-scope contract should say explicitly how change requests are priced, not leave it to a conversation later when you have less leverage, and should name a rough per-hour or per-change-request rate rather than leaving it open-ended.
What should I ask them directly?
We've grouped a full set of 20+ questions, with what a strong answer actually sounds like, in questions to ask an AI development agency. Use it alongside this checklist in your first call.
How do I audit an agency's portfolio before trusting it?
Click through to the actual product, not just the case study page, and look for three things: does it still work, does it look like the screenshots, and can you find evidence someone other than the agency uses it. A surprising number of agency portfolios link to products that have since been shut down, rebuilt by someone else, or never left a staging environment. If a "case study" doesn't link anywhere you can visit, treat it as a claim, not proof.
For AI-specific work, go one step further: ask what the agency would do differently if they rebuilt the project today. A team that actually shipped the thing will have a real answer — a model they'd swap, a cost they'd optimize, an evaluation step they'd add. A team that is describing someone else's work, or work that barely shipped, tends to give a generic answer like "it's working well, no complaints."
Does location matter — US agency vs. India vs. elsewhere?
Less than most founders assume, and more about process than geography. A well-run India-based team with written daily updates and overlapping working hours can outperform a disorganized US agency, and vice versa. We go through the honest trade-offs, including rates and how to verify quality, in outsourcing AI development to India.
How we approach this at Venzpire
We'd rather you vet us hard than find out six weeks in that we weren't the right fit. Every engagement starts with a $1,500 fixed discovery sprint — credited to the build if you continue — so you see how we think before committing to a full build. You can check our case studies at work, see the stack we actually use, and ask us anything from this checklist directly at apply.
Building something like this?
I read every application myself and reply within one business day, even when we’re not the right fit.