The 7 Questions Your AI Consultant Should Be Able to Answer (If They Can't, Walk Away)
Here's a scene I hear about more often than I'd like: a business owner spends six months and six figures with an AI consulting firm. They get decks. They get demos. They get a system that works beautifully in the controlled environment of a conference room presentation, and then quietly falls apart three months after go-live. Nobody explains why. The consultant has already moved on to the next engagement.
This isn't a rare story. By some estimates, more than 80% of AI projects fail to deliver their intended business value, roughly twice the failure rate of comparable IT projects that don't use AI at all [1]. A 2025 S&P Global Market Intelligence survey found that 42% of companies abandoned most of their AI initiatives before reaching production, up from just 17% a year earlier [2]. That's not a bad luck problem. That's a bad consultant problem, or more precisely, a buyer-didn't-know-what-to-ask problem.
I started building systems in 1995. That's when I launched adoption.com, one of the first internet-scale platforms in the world, before Google existed, before most people knew what a URL was. I did it with dial-up infrastructure and no roadmap except "build something that works." Since then I've run operations across seven countries, including orphanages and humanitarian logistics in Ethiopia, Kenya, and Haiti, places where systems either work or people suffer real consequences. I'm not a technologist who learned business. I'm a business operator who mastered technology.
I founded Verity Agentic because I got tired of watching businesses get sold AI hype when what they actually needed was AI that runs. And the single best thing I can do for you right now, before you hire anyone, is give you the seven questions that separate a real operator from a very confident presenter.
Ask every one. Don't accept a dodge on any of them.
Why These 7 Questions, Specifically
I didn't pull these from a generic consulting checklist. I pulled them from the patterns I've watched repeat across real engagements: the questions that expose whether someone has actually been responsible for a live system over time, or whether they've just been involved in the fun parts of building one.
There's a critical difference between a consultant who builds AI and one who's maintained AI. Between someone who defines success at kickoff and someone who waits to see what success looks like afterward. Between a firm that's thought carefully about human adoption and one that treats deployment day as the finish line.
Every question below probes one of those fault lines.
Question 1: Can You Show Me an AI System You Built That's Still Running 12 Months Later, and Connect Me with That Client?
This is the most important question on the list, so I'm putting it first.
Anyone can build something that works in demo conditions. I can show you a beautiful AI workflow running on clean test data in a controlled environment in about an afternoon. What I can't fake is a client reference who's been living with that system for a year, who can tell you what broke, what got fixed, and whether they'd hire me again.
The 12-month mark matters for a specific technical reason: AI models drift. A model can launch with 95% accuracy and drop to around 70% within six months as real-world data diverges from its training baseline [3]. In dynamic environments like fraud detection, customer behavior prediction, or content moderation, meaningful drift can emerge within weeks. If a consultant has never managed a system through its first drift cycle, they haven't really shipped one.
What a Good Answer Sounds Like
A good consultant gives you a specific system, a specific client, and a contact name with a phone number or email. They tell you when drift first became noticeable, what they did about it, and how monitoring is handled today. They might even mention a specific failure and how they recovered.
What a Bad Answer Sounds Like
"We have many satisfied clients" with no specifics. "Our NDA prohibits us from sharing client names." (A real client who's happy with you will talk. If none of them will talk, ask yourself why.) Or the enthusiastic pivot to a new feature they're building that you didn't ask about.
My answer, when you ask me this: I'm honest that I don't have a roster of multi-year AI consulting clients yet. What I do have is a Cap Gemini background helping businesses develop internet strategy before Google existed, 30 years of building systems that had to work in the real world, and hands-on experience building AI agents and web apps with Claude Code and Codex. I know what a well-maintained system looks like, and I know what drift looks like, because I've been responsible for systems that couldn't afford to degrade quietly. If you ask me this question, I'll tell you exactly what I've built, what's running, and what I can and can't claim. If I'm not the right fit, I'll tell you that too.
Question 2: What's the Most Common Reason Your AI Builds Don't Perform as Planned?
This is a honesty probe. Every experienced consultant has a pattern they've learned to watch for. Anyone who says their builds always perform as planned is either very new or not being straight with you.
RAND's 2024 qualitative study, based on interviews with 65 experienced data scientists and engineers, identified the root causes of AI project failure. They cluster into eight patterns that show up across industries, company sizes, and budget levels [4]:
Data quality and availability. Bad, missing, or siloed data is the single most consistent predictor of underperformance. A system is only as good as what it's trained on and what it sees in production.
Misaligned requirements. What got built versus what the business actually needed. This usually happens when discovery is rushed or skipped entirely.
Low adoption and change management failure. The system works. People don't use it. Or they use it wrong. This kills more projects than technical failure does.
No monitoring after go-live. Models drift. Integrations break when upstream systems update. Without active monitoring, a system that launched at 95% accuracy can quietly degrade to 70% while nobody notices.
Unrealistic expectations set in demos. Pilots succeed in controlled conditions with clean data. Production doesn't look like that. The gap between demo and reality is where most projects collapse.
Scope expansion without updating the success definition. The project grows after kickoff, but the original success metric doesn't get updated to reflect what's actually being built. At the end, nobody can agree on whether it worked.
Organizational readiness gaps. The business wasn't prepared to integrate AI into its actual workflows. Leadership buy-in was assumed rather than confirmed. The infrastructure to support the system wasn't in place before go-live.
Technology-first thinking. "Agentic AI" and "large language models" are being written into proposals right now whether or not they're the right tool for the job. The technology drove the solution instead of the problem driving the technology.
Why This Question Matters
When a consultant identifies their own common failure mode, they're telling you what they've learned to guard against. That's a sign they're paying attention. It also sets up a useful conversation: ask them how they catch these patterns early with a new client. What's their diagnostic process? How long does discovery take?
A consultant who pitches a solution in your first meeting, before they've toured your facility, reviewed your workflows, or talked to the people who'll use the system daily, is a consultant who's not going to catch the real problems [5].
My Answer
The failure mode I watch most closely is the gap between what a system was designed to handle and what actually shows up in production. Real-world data is messier, more ambiguous, and more variable than any test environment. I learned this the hard way building systems that ran across three continents on infrastructure that was sometimes intermittent and staff that rotated constantly.
My approach now is to stress-test with real data samples before go-live, build monitoring in from day one (not bolt it on afterward), and assume the first 90 days in production will surface edge cases nobody anticipated. I tell clients all eight failure modes on the list are real. I've seen versions of most of them in my own work, and I'd rather surface them honestly upfront than pretend my builds are immune.
Question 3: How Do You Handle the Human Side of Adoption?
This is the question most AI consultants are least prepared for, and it's the one that kills more projects than any technical failure.
A 2025 WRITER survey found that 79% of organizations face significant challenges adopting AI, a double-digit increase from the prior year [6]. Only 7% of respondents said AI was fully deployed and integrated across their organizations. The barriers aren't mostly technical. They're human: lack of skilled workers, lack of managerial buy-in, and the fundamental problem that the cost of learning a new system feels personal and immediate to the people being asked to use it, while the benefits feel abstract and distant.
Only about one-third of companies prioritized change management and training as part of their AI rollouts in late 2024 [7]. That means the other two-thirds deployed a system and then wondered why nobody was using it.
What to Look For
A consultant who's thought about this will have specific methods: a rollout sequence that phases users in rather than flipping a switch, a training approach tailored to the actual role (not a generic "here's how to use AI" seminar), a way of collecting early user feedback, and a plan for handling the staff member who's convinced the new system is going to eliminate their job.
A consultant who hasn't thought about this will tell you that the system is intuitive and training is really the client's responsibility.
My Answer
I've built systems in organizations where staff turnover is high, where English is a second language for most users, and where the people being asked to adopt new technology had deep skepticism that anything would actually improve their daily work. That's not hypothetical. It's what humanitarian operations look like across three continents, and it's what I was navigating at adoption.com from day one.
The discipline I bring to adoption comes from that experience, not from a long list of AI consulting engagements. My approach includes three checkpoints in the first 90 days: a usage rate review, a friction inventory (what's confusing or frustrating users), and a course-correction meeting with whoever owns the process. Deployment day is the beginning of the work, not the end. That's true whether the organization is a humanitarian nonprofit or a B2B services firm.
Question 4: What Happens If the AI Makes a Mistake? Who's Responsible and How Do You Catch It?
This question makes people uncomfortable, which is exactly why you should ask it.
AI systems make mistakes. That's not pessimism, it's engineering reality. A 71% majority of technology leaders told Kyndryl's 2025 AI Readiness Report that they don't trust their own organizations to manage future AI risks effectively [8]. And when something goes wrong, the legal and operational question of who carries accountability is genuinely complicated.
Under current frameworks, accountability for AI errors is distributed across the model's creators, the consultant who deployed it, and the organization that operates it [9]. The EU AI Act, effective August 2026, will formalize many of these responsibilities by requiring documented safety standards and testing before AI systems are placed on the market. But even before that, any serious consulting engagement should specify in writing who's responsible for what when the system produces a wrong output.
What a Good Answer Looks Like
A good consultant will distinguish between different error types. A hallucination in a summary is different from a wrong decision in a routing system. They'll have a monitoring approach that catches performance degradation before it causes real harm. They'll have a documented escalation process. And their contract will be clear about liability, not because they're planning for failure, but because clarity about failure modes is part of building anything serious.
Red Flags
"Our AI is very accurate" is not an answer to this question. Neither is a vague reference to the AI provider's terms of service. The question isn't whether the AI will be perfect. It won't. The question is what the protocol is when it isn't.
My Answer
My background is in medical technology, where you don't just trust the analyzer. You run quality controls, you track drift, and you have escalation protocols for out-of-range results. I bring that same discipline to AI systems.
Every system I build includes human-in-the-loop checkpoints for high-stakes decisions. I build alerting so the client knows when output confidence drops below a defined threshold. I document the error-handling protocol before go-live, not in response to the first incident. And my engagements are clear about which failure modes are within my scope to remediate and which require the client to make the call. That clarity isn't pessimism. It's what accountable systems design looks like.
Question 5: How Do You Define Success Before You Start, in Measurable Terms?
Vague success definitions are how projects get to "completion" without ever delivering value.
McKinsey's research on AI ROI consistently points to the same pattern: organizations that define specific, measurable outcomes before starting implementation are dramatically more likely to achieve them. Yet only about 29% of executives say they can measure AI ROI confidently [10]. That gap exists because most engagements start with outcomes described in aspirational language ("improve efficiency," "enhance the customer experience," "leverage AI capabilities") rather than in numbers.
Here's how I'd put it: if you can't disagree about whether success was achieved, your success definition isn't specific enough.
The Right Success Definition
A good success definition includes a specific metric (cost per transaction, processing time, error rate, customer satisfaction score), a baseline measurement taken before the project starts, a target value with a timeframe, and an agreed measurement methodology. It might look like: "Reduce average invoice processing time from 4.2 minutes to under 90 seconds by the 90-day mark, measured by timestamp data from the ERP system."
That definition is hard to fake and easy to verify. Both parties know exactly what they're working toward. And if the project isn't tracking toward that outcome at the 45-day checkpoint, there's a concrete basis for a course-correction conversation.
What I've Seen Go Wrong
The most common failure mode here is that "success" gets defined at the end, looking backward at whatever actually happened. "We delivered the system" becomes the success metric because nobody wrote down something better at the start. This is how consultants collect payment for projects that don't actually move the needle.
My Answer
I won't start an engagement without a signed success definition. That discipline comes directly from my Cap Gemini background, where vague objectives were treated as a project risk, not a starting point. Consulting without a measurable outcome isn't consulting. It's expensive guessing.
If a client can't define what measurable success looks like, that's a signal we need more time on discovery before we touch any technology. That conversation is part of the service, not a delay. It's where most of the real value gets established.
Question 6: What Does Your Maintenance and Support Model Look Like After Go-Live?
Deployment day is when the real work begins. Most AI consulting proposals are weighted toward the build phase. The ongoing support model is often buried in small print or left vague until after you've signed.
This matters because AI systems aren't like traditional software that stays put. Models drift as real-world data changes. Integrations break when the upstream systems they connect to get updated. New edge cases emerge that weren't anticipated in the training data. A system that runs beautifully on day one can quietly degrade over the following months without anyone noticing until something goes conspicuously wrong.
The gold standard for post-deployment care involves continuous monitoring of model outputs, regular retraining cycles as production data accumulates, a clear escalation path for incidents, documented governance covering who's responsible for what decisions, and SLA-backed uptime commitments [11].
Questions Within the Question
When you ask about post-go-live support, follow up with specifics. What's the SLA for a critical bug fix? How often will you review model performance together? What triggers a retraining cycle? Who monitors the system over weekends? Is monitoring included in the engagement fee or billed separately?
A consultant who's designed their business model around ongoing retainers has an incentive to keep you dependent. A consultant who's designed their model around clean handoffs has an incentive to train your team to manage the system independently. Both models can work, but you should know which one you're buying.
My Answer
I offer two post-go-live models. The first is a clean handoff: I build the system, train your team to monitor and maintain it, and document everything thoroughly enough that you're not dependent on me long-term. The second is an ongoing operations agreement where I'm responsible for monitoring, retraining, and incident response on a defined schedule.
I'm honest about which systems need which model. Simple automations can usually handle their own maintenance with quarterly checkpoints. Complex AI systems with real drift risk need ongoing attention, and I'll tell you that before you sign, not after go-live. That kind of transparency is something I learned early: the organizations I built for in Ethiopia and Haiti couldn't afford to discover a dependency they hadn't planned for. Neither can your business.
Question 7: What Kind of Business Are You NOT a Good Fit For?
This is the final question, and it's the one that tells you the most about a consultant's integrity.
Every consultant has a specialty and a genuine fit zone. The AI equivalent of a pediatrician isn't qualified to do cardiac surgery, and pretending otherwise doesn't serve the patient. A consultant who'll tell you honestly when they're not the right fit for your situation is a consultant you can trust when they tell you they are.
The consultants who can't answer this question are the ones who'll take any engagement, regardless of fit, and figure out the rest afterward. That's not a business model that serves clients. It's a business model that generates the horror stories that end up on the AI failure statistics.
What You're Listening For
A genuine answer names specific business types, industry contexts, or project characteristics where the consultant doesn't have strong track record or genuine expertise. Maybe they're excellent at automating back-office workflows but don't have healthcare compliance expertise. Maybe they're brilliant at customer-facing AI but haven't built warehouse management systems. Maybe they're best suited to companies with more than 50 employees and stable data infrastructure, and a solo operator with messy spreadsheets isn't their sweet spot.
The more specific the answer, the more trustworthy the consultant.
My Answer
I'm not the right fit if you want a quick proof-of-concept that's never meant to run in production. That's a legitimate thing to want, but it's not what I build. I build systems intended to run, and I'm not interested in building something that looks good in a demo and gets shelved six weeks later.
I'm also not the right fit if you're not prepared to invest in the change management side. Technically excellent systems fail because organizations weren't ready to adopt them. I've watched it happen, and I don't want to be part of that story. If you're not prepared to give your team the time and support they need to actually use what we build, that's a conversation we need to have before we start.
And I'm not the right fit if your data infrastructure is in serious disrepair and you want to skip the foundational work. The honest answer is that we need to fix the foundation first, and that's not always what someone hoping for a quick AI win wants to hear. The businesses I respect most are the ones that can hear that answer and work with it.
The Bigger Picture
The AI consulting market is growing fast and it's not well regulated. Anyone can call themselves an AI consultant. There's no bar exam, no licensing requirement, no minimum track record you have to demonstrate before hanging out a shingle. That means the quality range is enormous, from people with genuine multi-year operational experience to people who completed a six-week course and learned the vocabulary.
The seven questions above are your filter. They don't require you to understand the technical internals of AI systems. They require a consultant to demonstrate operational maturity: that they've shipped real things, kept them running, learned from their failures, and built honest practices around measurement and accountability.
Thirty years of building systems that have to survive contact with reality has taught me one thing above everything else: the gap between "this should work" and "this is working" is where most of the real engineering lives. The consultants who've crossed that gap, and can prove it with references and specific examples, are the ones worth hiring.
The ones who can't answer Question 1 probably haven't.
A Final Word on Timing
Don't wait until you've signed to ask these questions. Ask them during the proposal stage. Ask them before you've committed to anything. A consultant who's annoyed by due diligence is a consultant who's not used to accountability. A consultant who welcomes the questions and answers them in specific, concrete terms is a consultant who's built things and lived with the consequences.
You deserve to know which one you're talking to before the contract is signed.
Sources
[1] RAND Corporation, "The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed: Avoiding the Anti-Patterns of AI," https://www.rand.org/pubs/research_reports/RRA2680-1.html, 2024
[2] S&P Global Market Intelligence, "Generative AI Shows Rapid Growth but Yields Mixed Results," https://www.spglobal.com/market-intelligence/en/news-insights/research/2025/10/generative-ai-shows-rapid-growth-but-yields-mixed-results, 2025
[3] Fulcrum Digital, "AI Model Drift in Production: What Enterprises Must Monitor," https://fulcrumdigital.com/blogs/ai-model-drift-in-production-what-enterprises-must-monitor/, 2025
[4] RAND Corporation, "Why AI Projects Fail," https://www.rand.org/pubs/presentations/PTA2680-1.html, 2024
[5] AI Assembly Lines, "AI Consulting Firm Red Flags: 7 Warning Signs to Check," https://aiassemblylines.com/post/ai-consulting-red-flags, 2025
[6] WRITER, "Enterprise AI Adoption in 2026: Why 79% Face Challenges Despite High Investment," https://writer.com/blog/enterprise-ai-adoption-2026/, 2026
[7] Chronus, "Why AI Adoption Fails: Enterprise Barriers AI Leaders Ignore," https://chronus.com/blog/why-ai-adoption-fails-enterprise-barriers-to-ai-leaders-ignore, 2025
[8] Kyndryl, "2025 AI Readiness Report," cited in riskinfo.ai, "Who Is Responsible When AI Makes a Mistake?," https://www.riskinfo.ai/post/who-is-responsible-when-ai-makes-a-mistake, 2025
[9] Taylor Wessing, "AI Liability: Who Is Accountable When Artificial Intelligence Malfunctions?," https://www.taylorwessing.com/en/insights-and-events/insights/2025/01/ai-liability-who-is-accountable-when-artificial-intelligence-malfunctions, 2025
[10] Master of Code, "AI ROI: Why Only 5% of Enterprises See Real Returns in 2026," https://masterofcode.com/blog/ai-roi, 2026
[11] iMaintain, "Essential Guide to AI Model Maintenance: Managing Your AI Models Post-Deployment," https://imaintain.uk/essential-guide-to-ai-model-maintenance-managing-your-ai-models-post-deployment/, 2025