Artificial intelligence is often framed as a technology race defined by cutting-edge models, governance frameworks, and trillion-dollar infrastructure bets. But for thousands of mid-market companies, the real barrier to AI adoption is far more basic. It is not model selection, talent shortages, or regulatory uncertainty. It is legacy systems.
A global survey by Futurum Group found that 35% of organizations identified legacy system integration as a major barrier to AI adoption, underscoring how outdated infrastructure, not AI capability, is often the real constraint.
The question, then, is not whether legacy systems create friction, but when they shift from being a technical inconvenience to becoming a structural barrier that materially limits a company’s ability to generate measurable ROI from AI.
Not Strategic Levers but Isolated Experiments
The shift happens when legacy infrastructure stops enabling operational scalability and starts actively constraining strategic execution, says Cesar DOnofrio, CEO and co-founder of Making Sense, who has worked for over two decades with the digital transformation of US mid-market organizations and recognizes that legacy modernization is now a recurring structural constraint barrier to AI driving real economic impact in this segment.
“AI cannot generate measurable ROI if it operates outside the core systems of record. To deliver tangible business impact, AI must be integrated into the processes that directly drive revenue, operational efficiency, and customer experience.” – Cesar DOnofrio, CEO and co-founder of Making Sense
“When legacy systems limit access to reliable data, slow down integration across workflows, or make change deployment complex and time-consuming, AI initiatives stop being strategic levers and become isolated experiments. Organizations may be able to run pilots, but they cannot operationalize or scale them,” he explains.
Many mid-market companies still rely on platforms originally designed to support stable, predictable operations. AI, however, requires a fundamentally different foundation, connected data ecosystems, flexible architectures, and the ability to embed intelligence directly into core workflows.
“When data must be manually extracted, inconsistencies reconciled across disconnected systems, or business processes depend on workarounds, the cost and complexity of making AI operational increases exponentially,” he adds.
At that point, he says, the issue ceases to be technical debt and becomes a structural growth constraint.
“AI cannot generate measurable ROI if it operates outside the core systems of record. To deliver tangible business impact, AI must be integrated into the processes that directly drive revenue, operational efficiency, and customer experience,” he says.
Enforced Data Silos by Design
In many mid-market companies, critical operational data remains fragmented across siloed enterprise resource planning systems, spreadsheets, disconnected applications, and, in some cases, paper-based workflows. Information may exist, but it is not accessible in a usable, unified form. This makes AI deployment technically possible but operationally ineffective.
Srinivas Devarakonda, Principal Data Scientist at Nisum, says the shift happens the moment your infrastructure enforces data silos by design rather than by accident.
“We see the ROI floor drop out when organizations spend 80% of their budget on bespoke middleware just to get fragmented systems to talk to each other. At that point, you aren’t investing in intelligence; you are paying a legacy tax to keep the lights on. If your system design doesn’t allow for a multi-agent approach to coordinate tasks across IT and finance, you will never see the compounding returns that define successful AI automation. The barrier is reached when the cost of integration consistently outweighs the marginal gain of the insight.” – Srinivas Devarakonda, Principal Data Scientist at Nisum
“You can tolerate technical inconvenience when it simply means a slower release cycle. However, it becomes a structural barrier when the latency between a business event and your model’s ability to ingest that event exceeds the window of decision-making,” he states.
In high-volume environments such as retail or logistics, he adds, legacy systems that cannot provide real-time visibility into inventory or customer behavior render AI ineffective.
“We see the ROI floor drop out when organizations spend 80% of their budget on bespoke middleware just to get fragmented systems to talk to each other. At that point, you aren’t investing in intelligence; you are paying a legacy tax to keep the lights on. If your system design doesn’t allow for a multi-agent approach to coordinate tasks across IT and finance, you will never see the compounding returns that define successful AI automation. The barrier is reached when the cost of integration consistently outweighs the marginal gain of the insight,” he warns.
Siloed Data That Cannot Communicate
When data is siloed, AI remains confined to isolated pilots. Instead of transforming workflows, it produces reports, summaries, or predictions that cannot be embedded into real-time operational decision-making.
Matt Roberts, Founder of Happy Operators, an AI consultancy, says the tipping point is when your data lives in silos that can’t talk to each other.
“AI needs data like a restaurant needs ingredients. You can have the best chef in the world, but if your suppliers are delivering to 12 different doors in 47 different formats, you’re going to run into chaos pretty quickly.” – Matt Roberts, Founder of Happy Operators
“AI needs data like a restaurant needs ingredients. You can have the best chef in the world, but if your suppliers are delivering to 12 different doors in 47 different formats, you’re going to run into chaos pretty quickly,” he warns.
He adds that access, not availability, is often the hidden constraint, “More often than not, the right data and systems are available. The barrier is that only developers can touch them, so potential use cases never get discovered. The people closest to the business problems are locked out of the shed.”
Larry Adams, Executive Chairman at AI software development firm Chromatics.AI, who has rebuilt systems inside AT&T, scaled Vimeo from a niche platform into a real subscription business, and helped launch HBO Max, says he has seen the same pattern repeatedly.
“Legacy becomes a real barrier when leadership wants faster decisions, smarter customer experiences, and better margins, but the company is still operating off disconnected databases, manual reports, and weekly reconciliations. You cannot move at modern speed if your information is stitched together by hand.” – Larry Adams, Executive Chairman at Chromatics.AI
“The strategy was not the problem. The ambition was not the problem. The systems were,” he says. “Legacy becomes a real barrier when leadership wants faster decisions, smarter customer experiences, and better margins, but the company is still operating off disconnected databases, manual reports, and weekly reconciliations. You cannot move at modern speed if your information is stitched together by hand.”
At Vimeo, for example, the company needed real-time visibility into subscriber behavior to grow revenue. But when billing data, product usage, and marketing performance existed in disconnected systems, progress slowed.
“In that environment, advanced analytics look impressive in a deck but struggle to drive real change,” he explains. “Legacy stops being a nuisance when it starts costing you time. In today’s market, time is a competitive advantage. If others are adjusting daily and you are adjusting quarterly, the gap widens fast. That is not a talent issue. It is a systems issue,” he reiterates.
Connecting What Matters Most
Modernization does not always require replacing entire systems at once. Often, the fastest progress comes from connecting the most critical data flows. Roberts explains that at HBO Max, meaningful progress began when subscriber behavior, content data, and marketing signals could finally be viewed together without weeks of manual reconciliation.
“Once leaders could see the business clearly, decisions improved quickly,” he says.
Jonathan Selby, Tech & Media Practice Lead at Founder Shield, says the shift from “inconvenience” to “barrier” occurs when a company attempts to move from AI experimentation to live, operational integration.
“If your AI can’t access clean, live data because of a 20-year-old ERP, you aren’t just losing efficiency; you’re generating “hallucinations” or insights based on stale info (creating massive compliance and liability risks!). At that point, the cost of maintaining the legacy system outweighs the projected ROI of the AI initiative, effectively capping the company’s competitive growth.” – Jonathan Selby, Tech & Media Practice Lead at Founder Shield
“Unfortunately, when this shift happens, a ‘data traffic jam’ often unfolds,” he says.
Many mid-market companies operate with data trapped in localized physical servers or disconnected software systems.
“These dynamics create the ‘silo’ problem, which prevents real-time processing. If your AI can’t access clean, live data because of a 20-year-old ERP, you aren’t just losing efficiency; you’re generating “hallucinations” or insights based on stale info (creating massive compliance and liability risks!). At that point, the cost of maintaining the legacy system outweighs the projected ROI of the AI initiative, effectively capping the company’s competitive growth.”
The Real Risk for Mid-Market Companies Is Delay
Skylar Roebuck, CTO at AI-first advisory and digital engineering firm Solvd, says legacy infrastructure becomes a structural barrier the moment it slows AI experimentation to a crawl, or worse, when leaders stop entertaining AI ideas because they assume their organization is ‘too immature.’
“The companies that see the most value balance quick wins with high-conviction, game-changing use cases backed by strong business metrics. AI adoption isn’t about a single initiative, but a collection of bets guided by a stubborn AI vision.” – Skylar Roebuck, CTO at Solvd
“That mindset is often more limiting than the technology itself. Traditional modernization tends to over-index on protecting how things work today rather than building for what’s next. AI capability is compounding rapidly, and the real risk for mid-market companies is delay,” he warns.
Workflow digitization often delivers early momentum because it produces visible gains quickly. But scalable AI requires deeper, systemic integration.
“The companies that see the most value balance quick wins with high-conviction, game-changing use cases backed by strong business metrics. AI adoption isn’t about a single initiative, but a collection of bets guided by a stubborn AI vision,” he says.
Across manufacturing firms, logistics providers, healthcare operators, and private equity-backed portfolio companies, AI ambition is running headlong into decades-old infrastructure. These systems, built long before cloud computing, real-time analytics, or machine learning existed, were designed for a different operational reality. Today, they have become structural constraints, limiting not just AI deployment, but the ability of organizations to modernize at all.
The result is a widening gap between AI potential and AI readiness.
