There’s a version of the AI story that goes like this: the board demands an AI roadmap, investors want to see the AI concept, competitors announce AI features, and suddenly the entire company is pressured to release something, anything with the “AI” prefix. This story isn’t hypothetical, it’s unfolding within product divisions right now, and the results are disappointing.The pressure is understandable. According to McKinsey’s “State of AI” report, AI adoption in enterprises has grown from 55% in 2023 to 78% by the end of 2024. But adoption rates and value creation are not the same thing. BCG’s landmark October 2024 study, “Where’s the Value in AI?”, surveyed over 1,000 senior executives in 59 countries and found that 74% of companies still struggle to achieve and scale value from AI investments. Only 26% of companies have developed the capabilities needed to move from proof-of-concept to production systems that deliver measurable results. Meanwhile, 70-85% of AI initiatives fail to meet expectations, and 42% of companies will abandon most of their AI projects by 2025, up from 17% the year before.💡Often, the real issue isn’t the technology itself but the decision-making process that happens before implementation even begins. Successful product organizations start with a problem and ask whether AI is the right solution. The less mature organizations tend to start with the technology itself and then look for where it adds value. This article is about how to be first and why the price of second is higher than most teams realize.5 questions to ask your AI agent vendor before you buyMost AI agent failures don’t happen during the demo. They happen when APIs fail, context windows explode, costs spiral, and nobody can explain why the agent made a decision. Here are five questions that separate production-ready platforms from expensive experiments.AI Accelerator InstituteAndrew Lovell1. Two categories, two different decisionsBroadly speaking, AI initiatives tend to fall into two categories: customer-facing AI or internal operational AI.Customer-facing AI is the type of AI where customers use the product: copilot, recommendation engines, smart search, generative content tools, natural language interfaces and AI assistants. Because customers see all aspects of the user experience, failures regarding trust and quality are immediately apparent and can be very damaging.Internal operational AI is the type of AI that improves how the organization operates internally: research assistants, document summarization, coding copilots, analytical interpretation, automated workflow routing and internal decision support. Value exists in increasing efficiency, improving the quality of decisions and increasing scalability.Investments in these types of AI cannot be compared or evaluated in the same manner. If a customer-facing AI fails to perform well, it will lead to decreased customer trust and increased customer churn. If an internal operational AI tool performs poorly, it may cost time and damage credibility with your team; however, the negative effects are ultimately recoverable. The stakes involved, the standards of performance expected, and the level of acceptable risk tolerance are fundamentally different.2. If AI is a product feature, implementing it still requires disciplineAdding AI to your product will not automatically make you innovative; AI features still need to be assessed with the same product discipline as any other feature. The questions remain:What specific user problems does this solve, and how do we truly know these are valid problems?At what points in the user experience is there currently friction?Will the users actually use this new feature without having to fundamentally alter their behavior?Does this improve the users’ retention, conversion rates, satisfaction ratings, or monetization opportunities, and if so, by how many percentage points?The bar for deploying AI features should be higher than deploying traditional product features, due to both the increased costs associated with developing and implementing the technology as well as increased technical and operational overheads associated with deploying and maintaining it.Klarna’s deployment of an OpenAI-powered customer service application is a prime example. Klarna revealed they had deployed a system which resulted in approximately 2.3 million conversations during its initial month alone. Resolution times were reduced from 11 minutes down to less than two minutes. Customer satisfaction equaled that of human support agents, repeat enquiry rates decreased by 25%, and Klarna projected $40 million in savings in 2024 based on a $2-3 million investment.Then it began to fall apart. Cost became a “predominant evaluation criterion” by mid-2025. Although the AI performed well on low-hanging fruit, there was no correlation to quality of experience for customers dealing with more complex, emotionally charged issues. Customer satisfaction was decreasing for those who escalated. Contact frequency from repeat customers was increasing.By early 2026, Klarna had quietly begun to rebuild its customer service personnel, adopting a hybrid model with AI handling routine, high-volume questions and humans handling complex enquiries requiring judgment and emotional intelligence.The lesson is not that AI doesn’t belong in customer service. It is that success must be defined by what truly matters to the business, not by what is easiest to measure.3. AI features require better onboarding, rules of engagement and economic modelsThere are three ways an AI feature could fail differently than a traditional feature, even if it solves the right problem.Onboarding: The interaction models of AI, including copilots and other assistants, remain foreign to most end-users. Unlike adding a new button or dashboard view, users trying out an AI feature for the first time do not know what to ask for, what they will receive as output, or when to rely upon it. One unhelpful experience is often enough to permanently categorize the feature as “broken.”Effective onboarding must answer four questions at the point of first use: “What can I ask the AI?” “What will I receive in return?” “What am I able to do with the output?” and “When should I trust the output versus verifying it?”Token economics: Customer-facing AI functionality has a cost structure that grows with use. Each interaction is charged by token. The diagnostic question every product organization must answer before launching: will the added value created by this AI feature exceed the additional costs at expected levels of user engagement, at scale? If the answer is uncertain, the business plan is incomplete.Governance: Conventional features are static. AI output is dynamic, created on demand each time a user interacts. Users can experience hallucinations, irrelevant output, privacy violations, and legal liability. These failures require active management through testing, monitoring, human review processes, and escalation pathways. Treating these as after-market considerations rather than front-end planning items is one of the most expensive mistakes organizations currently make.How RLHF encodes bias through alignment tampering | AIAIYour reward model is learning exactly what your annotators prefer. The problem is that “better” and “unbiased” are two different things, and RLHF has no way to tell them apart.AI Accelerator InstituteAndrew Lovell4. For internal AI, the greatest value is measurable leverage (not just saving time)There are two types of internal AI leverage.Execution efficiency means accomplishing the same task in less time. A summary document that used to take one hour now takes ten minutes. This is useful, but there is a limitation to its value.Decision leverage means making better decisions sooner than had been possible: combining competitive intelligence from fifty sources before a key strategic meeting instead of five, running pricing scenario models that would have taken analysts a week, surfacing trends buried within customer feedback. This is where AI has the potential to change what an organization is capable of.BCG data confirms this distinction: among companies that successfully scaled AI, the most valuable applications concentrate in operations (23% of total AI value), marketing and sales (20%), and R&D (13%), all areas where decision quality directly impacts results. Support functions, despite early adoption, account for only 38% of value.💡The diagnostic question for internal AI: does it create levers that would otherwise be impossible, or does it simply automate what already works adequately? The first can justify meaningful investment. The second may bring efficiency, but not strategic advantage.5. AI won’t repair a faulty workflow. It’ll amplify the faultinessThe most consistently ignored axiom in enterprise AI adoption: applying AI to a poorly designed process does not fix it. It produces the same flawed output faster, and at greater volume.Camunda’s 2025 State of Process Orchestration and Automation Report found that 82% of organizations surveyed reported concerns about “digital chaos,” the condition where the interconnectivity of automated processes exceeds the organization’s capacity to govern them. Of those, 77% reported an increased risk of failure for core business processes. Automating without first defining good process logic is commonly cited as the root cause.Before beginning any AI automation initiative, teams should be able to answer yes to all of the following:Is the workflow logical from start to finish?Is responsibility clearly defined at each stage?Is the source data reliable and structured?Does the process produce the correct outcome when executed well by humans?Have unnecessary steps been identified and eliminated?Only when all five questions are answered positively is AI the right next investment. Sometimes the first step is to improve the underlying process, which can give faster and lower-risk returns.6. Context beats capabilityMany organizations acquire the most advanced standalone AI tool available, deploy it independently of existing business processes, and are then disappointed by its usage. The reason is context.When AI operates outside existing systems, users must reconstruct context manually in every session: exporting data, providing backstory, managing document versions across multiple systems, and maintaining yet another subscription beyond the current technical stack. Each of these is a potential abandonment point.The most effective internal AI implementations are often less about technical sophistication and more about how deeply they are embedded in the area where actual work takes place: collaboration software, design tools, project management tools, documentation tools. The AI inherits the context already inherent in those platforms: the discussion thread, the current design iteration, the project timeline.Morgan Stanley’s deployment of GPT-4 as a knowledge assistant for financial advisors illustrates this. Rather than creating a new research interface, Morgan Stanley installed the AI within the interfaces advisors were already using. The AI had direct access to the institutional knowledge base and was able to provide value without needing context to be recreated each session.Similar results have been observed at Klarna, IKEA, and Unilever: AI tools embedded in existing workflows consistently outperform standalone tools with greater raw capability but significantly more friction.Multi-turn reasoning is broken in a way nobody saw comingMulti-turn reasoning is broken in a way nobody saw coming. The question is; what can we do to fix it?AI Accelerator InstituteAndrew Lovell7. The true cost of AI includes opportunity costMost organizations focus their AI cost discussions on software spend: licensing, API tokens, and compute. This is not fully representative of what needs to be considered.The total cost includes everything the organization did not do while pursuing the AI initiative:Engineering time redirected from core productsLeadership time consumed by vendor evaluations and steering committeesSecurity and compliance teams diverted to AI risk assessmentsCapital not deployed on hiring, partners, or infrastructureThe average financial services firm with revenues above $5B spent $22.1 million on AI in 2024, money that could have gone somewhere else. Before approving an AI initiative, organizations should be able to answer:Is this the highest-ROI use of engineering and product resources available right now?Would fixing the underlying process create more value, faster, with less risk?Would hiring strong operators or domain experts solve the actual problem more reliably?Are we pursuing this because it solves a genuine priority, or because there is pressure to have an AI story?The final question can be the toughest to ask, but also the most critical. Board expectations, competitive anxiety, and market trends are real pressures, but none of them replace a product-based rationale.8. Vendor dependence is a strategic risk, not a technical issueMost organizations building AI products rely on third-party APIs (OpenAI, Anthropic, Google, Microsoft Azure) because training foundation models is out of reach for all but a few hyperscalers. The dependency created is frequently underestimated in product and business strategies.OpenAI’s global outage in June 2025, lasting over 15 hours, made this clear. Thousands of businesses whose customer service, internal processes, and decision-making relied on a single provider had no fallback. Downstream applications including Zendesk and Perplexity also experienced errors and high latency.💡Zapier’s 2026 Enterprise AI Survey found that 81% of enterprise leaders have concerns about AI vendor dependency, 47% stated a primary business function would cease to operate under a material outage or pricing change, and only 6% believed they could replace their primary vendor without material operational disruption.The risk dimensions include service availability, pricing changes (Azure’s OpenAI pricing doubled for some enterprise customers in early 2025), model deprecation (DALL-E 3 was discontinued in May 2026 with limited notice), policy and access changes, and legal liability, as illustrated by the New York Times vs. OpenAI case around prompt data ownership.Gartner predicts that by 2028, 70% of companies building AI applications across two or more LLMs will implement AI gateways to manage dependency risk. Building contingency strategies now (alternative vendor relationships, multi-model routing, emergency override procedures, and business continuity tests) is significantly less expensive than building them after an incident.What separates strategic AI from reactive AI?AI is a genuine capability shift. Targeted, well-reasoned applications have produced meaningful productivity gains, better decisions, and scalable handling of high-volume routine work. But AI is not a product strategy.Understanding your customers’ problems, identifying the underlying issues within your organization’s operational models, and developing effective solutions: these are all required before AI enters the conversation.Organizations that derive lasting competitive advantage over the next five years will be those that apply the same discipline to every AI decision: beginning with the customer or operator problem, and holding AI to the same standard of proof as any other strategic investment.The more useful question is not “How do we add AI?” but “Where can AI deliver meaningful value, and where might a simpler solution be the better choice?”💡When applied with discipline, AI creates speed, leverage, and better decisions. Without it, AI creates cost, vulnerability, and strategic distraction dressed up as innovation. Product judgment continues to matter far more than the hype.