IBM: Enterprises Deploy 'Everything' in AI, Now Face LLM-to-Use-Case Challenge

Throughout the past century, IBM has witnessed numerous technology trends emerge and fade. The approaches that endure tend to be those that offer flexibility and choice.

At VB Transform 2025, Armand Ruiz, IBM's Vice President of AI Platform, outlined the company's perspective on generative AI and how its enterprise customers are practically implementing the technology. Ruiz stressed that the current landscape is not about selecting one exclusive large language model provider or solution. Businesses are increasingly moving away from single-vendor AI strategies, opting instead for multi-model frameworks that pair specialized LLMs with specific operational needs.

While IBM offers its own open-source AI models through the Granite series, it does not present them as a universal solution for every application. This customer behavior is shaping IBM's role; the company aims to serve as a central control hub for AI operations, rather than competing directly as a foundation model provider.

"When I meet with customers, they are leveraging every tool at their disposal," Ruiz noted. "For coding tasks, many prefer Anthropic. For reasoning and logic, they might use o3. For tailoring LLMs with their own data via fine-tuning, options range from our Granite models to Mistral's smaller models, or even Llama. It's about selecting the right model for the specific task. We also assist them by offering recommendations."

A Strategy for Multi-LLM Connectivity

To address this market dynamic, IBM has introduced a new model gateway. This system gives enterprises a unified API to seamlessly transition between various LLMs, while ensuring consistent oversight and governance across all implementations.

The technical design enables clients to operate open-source models on their own inference infrastructure for sensitive data, while also utilizing public APIs such as AWS Bedrock or Google Cloud's Gemini for less critical functions.

"This gateway offers our clients a single interface with one API to switch between different LLMs, incorporating full observability and governance throughout the process," Ruiz stated.

This method stands in contrast to vendor tactics that often seek to lock customers into proprietary systems. IBM is not alone in advocating for a multi-vendor model selection. Recently, several tools have appeared for model routing, designed to assign workloads to the most suitable model.

Agent Coordination Protocols Become Essential Infrastructure

In addition to managing multiple models, IBM is addressing the growing need for agent-to-agent communication through open standards.

The company created the Agent Communication Protocol and has submitted it to the Linux Foundation. ACP is a counterpart to Google's Agent2Agent protocol, which Google also contributed to the Linux Foundation this week.

Ruiz mentioned that both protocols are designed to streamline agent communication and minimize the need for custom coding. He anticipates that the different methodologies will eventually merge, with the current distinctions between A2A and ACP being primarily technical in nature.

These agent coordination protocols establish uniform methods for AI systems to interact across diverse platforms and providers.

The technical importance is evident at an enterprise level: some IBM clients are already testing over 100 agents in pilot projects. Without standard communication protocols, each interaction between agents demands custom development, leading to an unmanageable integration workload.

AI Transforms Workflows and Operational Methods

Regarding AI's current influence on businesses, Ruiz believes its impact must extend beyond basic chatbot functions.

"If your AI initiatives are limited to chatbots or mere cost reduction, you haven't fully embraced AI," Ruiz commented. "True AI involves fundamentally reshaping workflows and how work is accomplished."

The difference between basic AI use and full transformation lies in how thoroughly the technology is embedded into business operations. IBM's internal HR case demonstrates this evolution: instead of staff consulting chatbots for HR details, specialized agents now manage standard inquiries about pay, recruitment, and advancement, automatically connecting to relevant systems and involving human staff only when essential.

"I previously spent significant time consulting HR representatives for various matters. Now, an HR agent handles most of my requests," Ruiz explained. "Based on the inquiry—whether it's about compensation, employee separation, hiring, or promotions—these tasks link to different internal HR systems, each managed by dedicated agents."

This signifies a basic structural change from human-computer interaction to computer-driven workflow automation. Instead of employees learning to use AI tools, the AI is learning to perform entire business processes from start to finish.

The practical consequence: companies must progress beyond simple API connections and prompt design, advancing toward comprehensive process integration that enables AI agents to independently execute multi-stage workflows.

Strategic Considerations for Corporate AI Investment

IBM's deployment observations point to several key changes for enterprise AI planning:

Move beyond a chatbot-centric approach: Companies should pinpoint entire workflows for overhaul, rather than just adding conversational front-ends to current systems. The objective is to remove manual steps, not just enhance human-computer interaction.

Build for multi-model adaptability: Instead of tying themselves to单一 AI suppliers, businesses require integration platforms that allow switching between models according to task needs while upholding governance.

Prioritize communication standards: Organizations should favor AI solutions that support emerging protocols like MCP, ACP, and A2A, avoiding proprietary integration methods that lead to vendor dependency.

"There is immense potential to develop, and I continually emphasize that everyone must learn about AI. Business leaders, in particular, need to adopt an AI-first mindset and grasp the fundamental concepts," Ruiz concluded.