- Most companies struggle to implement agent AI effectively, and legacy infrastructure is one of the main reasons, the report finds
- Google surveyed IT leaders, with 83% stating that infrastructure upgrades are necessary
- IT managers are also concerned about the hidden costs of agent AI, such as increased power consumption and operational complexity
If there was a single takeaway from this article, it’s that the infrastructure every business relies on today is not built to handle agent AI.
Google surveyed over 1,400 senior IT leaders about their AI ambitions and found that 83% of organizations say they need infrastructure upgrades to take full advantage of production-grade agent AI.
In addition, many respondents also see unexpected costs arising from trying to run agent AI on legacy infrastructure. 62% said they had seen a significant bottom line tax driven by data egress fees, bloated storage and idle specialized hardware, along with 82% who said scaling AI introduces hidden operational costs. 79% also cite security, governance and MLOs as a key barrier to scaling agent AI.
Upgrades required for full agent AI benefit
To combat these limitations, Google has several recommendations for organizations hampered by legacy infrastructure.
Leveraging fluid computing “to dynamically match the right silicon to the right task while minimizing operational costs” is Google’s first recommendation, providing computing power for agentic AI tasks without reducing capacity for general workloads, avoiding the need for excess memory consumption to run agentic workloads that use large context windows.
For those struggling with agent sprawl caused by a cascade of new tasks across platforms and teams, Google recommends using enterprise quality management tools, usually available through the cloud partners companies already use. Google offers its own platform, Agent Gateway, as an example of a solution that provides visibility and visibility into how agents communicate, the data they access, and their workloads.
Organizing data more efficiently prevents AI agents from drawing more computing power when running heavy queries trying to access siled data. Organizations looking to improve the effectiveness of agent AI should work toward using a unified data layer that automatically annotates unstructured data so agents can understand where the data is without having to navigate pipelines. An additional benefit of using a unified data layer is that it helps avoid duplication of data, saving on additional storage costs in the long run.
Moving your AI to the edge—by deploying agents directly on the site where they are most used—is an additional recommendation, and one that organizations are actively pursuing. 90% of organizations surveyed by Google said this was a consideration in their AI initiatives. By deploying agents on-premises in manufacturing facilities, retail stores or hospitals, agents benefit from reduced latency, greater resiliency (in the event of a centralized cloud outage), and improved cost efficiency by reducing costs per token with local, highly optimized models.
As with businesses of all sizes, energy costs are a key consideration. When choosing new hardware, 91% of managers now consider power consumption as a factor, especially when navigating power availability in regions without expanding capacity, complying with regulations and reducing the cost of ownership of AI systems.
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews and opinions in your feeds.
