Table of Contents
The conversation around artificial intelligence in the workplace has shifted dramatically. It is no longer about the hype of what might be possible, but rather the practical reality of implementation. Organizations globally are experimenting with AI, yet many struggle to translate high-level ambition into measurable business impact. The hurdle isn't typically the capability of the AI models themselves; rather, it is how companies prepare their infrastructure to deploy these tools safely, efficiently, and at scale.
In a recent discussion for Intelligence Squared, Ben Kus, Chief Technology Officer of Box, outlined a strategic roadmap for businesses navigating this transition. From auditing legacy data architectures to embracing "agentic" workflows, the path to AI readiness requires a shift in how organizations manage their most valuable asset: their information. Below is a comprehensive look at the five essential steps leaders must take to implement and scale AI effectively.
Key Takeaways
- Data architecture is the primary bottleneck: Most "AI problems" are actually data problems, specifically regarding the accessibility of unstructured data like documents and emails.
- Governance is non-negotiable: AI models do not keep secrets; robust permission structures and a single source of truth are required to prevent data leakage.
- Start small to scale fast: Success comes from targeting specific bottlenecks and empowering mid-level managers, rather than attempting to overhaul entire enterprise systems overnight.
- Move beyond the chatbot: The future lies in "Agentic AI"—systems that don't just answer questions but perform asynchronous work loops to complete complex tasks.
- Measure tangible process metrics: Focus on immediate time-savings in specific workflows rather than trailing, high-level company KPIs.
1. Auditing Data Architecture: The Foundation of AI
Before an organization can deploy an intelligent agent, it must address the environment in which that agent will operate. A common realization among CTOs is that they do not have an AI problem; they have a data problem. Enterprise data is often bifurcated into structured data (databases, CRMs like Salesforce) and unstructured data (files, emails, Slack messages, contracts). While structured data is often centralized, unstructured data—which Generative AI is uniquely suited to process—frequently lives in silos, legacy servers, or disconnected platforms.
If an AI model cannot access the relevant data, its sophistication is irrelevant. Kus offers a compelling analogy for this limitation:
"Think of AI agents as a new employee. Imagine you bring in the smartest employee you have... and you say, 'I want you to go work on something.' And they say, 'Okay, where's that data?' And you say, 'I don't know, I won't give you access to it.' In which case, no matter how intelligent these AI models are... they're going to struggle to deliver real business value."
The Top-Down Audit
Successful implementation begins with a top-down audit. IT leaders must identify where critical data resides and whether those systems are modern enough to communicate with AI models via APIs. This often necessitates revisiting legacy systems and moving data from on-premise file servers to cloud platforms that support modern integration standards.
2. Establishing a Single Source of Truth and Governance
Once data is located, the challenge shifts to security and access control. A fundamental risk in AI deployment is that standard Large Language Models (LLMs) do not inherently understand organizational hierarchy or clearance levels. If an AI is given unfettered access to a company's data lake to answer a query, it will retrieve everything it finds—including sensitive HR files or executive compensation packages.
Why AI "Doesn't Keep Secrets"
In a manual workflow, a human employee knows they cannot access a folder they don't have permissions for. AI, however, is designed to be helpful above all else. If it has access to the data, it will use it to answer the prompt. Therefore, the governance layer must be applied before the AI interacts with the data.
This requires a "single source of truth" where permissions are rigidly enforced. When a user queries an AI agent, the system must respect the user's specific access control list (ACL). It should only synthesize answers from documents that specific user is authorized to view. This approach ensures that while the AI system technically has access to the enterprise's knowledge, it only reveals what is appropriate for the individual prompter.
3. Strategy: Have Big Ambitions, But Start Small
A frequent cause of failure in AI projects is the desire to "boil the ocean"—attempting to automate the most complex, high-value business process immediately. These ambitious projects often stall due to their complexity. A more effective strategy is to identify specific bottlenecks where manual work is slowing down productivity.
Targeting Bottlenecks
For example, in the financial sector, loan origination often involves reviewing disparate unstructured documents—scanned PDFs, emails, and forms. Instead of trying to automate the entire decision-making process, a "start small" approach would focus solely on data extraction: using AI to read the documents and standardize the data into a structured format. This provides immediate value by saving hours of manual entry and sets the stage for more advanced automation later.
"Automating this internal audit step or this procurement step... from the person who's responsible for it, who's driving that project... is usually where we see people who approach it this way getting more successful projects."
This approach also aids in change management. By pushing the problem down to managers closer to the workflow, organizations foster internal champions who understand the nuances of the task and can verify the AI's output effectively.
4. Pushing Beyond the Chat Interface
The popularization of tools like ChatGPT has conditioned the market to view AI primarily as a conversational interface—a "chatbot." While valuable, conversation is merely the interface layer. The true potential of business AI lies in Agentic AI.
Understanding Agentic Workflows
Unlike a standard chatbot that responds to a prompt in real-time, an AI agent is designed to achieve an objective. It can reason, plan, and execute tasks asynchronously. This is often referred to as the Agentic Experience (AX).
In an agentic workflow, a user might instruct an AI to "prepare a briefing for next Tuesday's client meeting." The agent does not simply spit out a generic template. Instead, it might:
- Access the CRM to review recent client interactions.
- Analyze relevant emails and financial statements.
- Identify missing information and flag it.
- Spend hours "thinking" and processing in the background.
- Deliver a comprehensive report hours later.
This shift from "talk to the bot" to "assign work to the agent" represents the next phase of productivity. It allows AI to act as a skilled coworker capable of handling complex, multi-step workflows without constant human supervision.
5. Measuring Real Business Value
The final step in scaling AI is rigorous measurement. However, leaders should avoid the trap of looking immediately for high-level ROI on trailing indicators, such as overall company profitability. These metrics move too slowly to be useful for iterating on nascent technology.
Instead, measurement should focus on the specific process being automated. If a compliance review typically takes 10 hours and AI reduces it to 20 minutes, that is a verifiable metric. When these micro-efficiencies are replicated across hundreds of processes within an enterprise, they ladder up to significant organizational impact.
The Ecosystem of the Future
Looking toward 2026 and beyond, the business landscape will likely be defined by ecosystems of cooperating agents. We will move away from singular AI interactions toward environments where specialized agents—finance agents, legal agents, creative agents—collaborate to solve complex problems. Companies that have laid the proper data foundations and governance structures today will be the ones positioned to orchestrate this new digital workforce effectively.
Conclusion
Implementing AI is a journey of infrastructure as much as it is innovation. By auditing data architectures, enforcing strict governance, and moving toward agentic workflows, leaders can turn the promise of AI into tangible business results. The technology is moving fast—what was impossible six months ago is standard today—but the principles of clean data and clear security remain constant.
