Autonomous AI systems put governance to the test in physical environments

Autonomous AI systems are expanding beyond software environments into warehouses, delivery networks, and public spaces. This shift raises questions about whether existing AI regulations adequately address systems operating in physical environments.

Most current AI governance frameworks have concentrated on online harms and model outputs, such as bias, misinformation, and harmful content. Embodied AI systems introduce risks in physical settings, where failures can impact infrastructure, property, or human safety.

Singapore’s Infocomm Media Development Authority released version 1.5 of its Model AI Governance Framework for Agentic AI on May 20. The framework provides guidance for organizations deploying AI agents that can plan, make decisions, and take actions across multiple steps to achieve user-defined goals.

The framework notes that agents can interact with tools, external systems, and other agents, including systems that update databases, write files, control devices, or execute transactions. It outlines access controls, monitoring, and human approval as governance measures for deployment.

AI moves into physical systems

At an AI summit in Singapore last week, discussions on robotics and embodied AI centered on operational safety issues more commonly associated with aviation, industrial systems, and critical infrastructure oversight than conventional software regulation.

Speakers also explored whether autonomous systems can operate safely and reliably in unpredictable real-world environments over extended periods.

Dr. Ya-Qin Zhang, founding dean of the Institute for AI Industry Research at Tsinghua University, noted that embodied AI systems amplify risks already present in autonomous software. He said failures can directly affect transport systems, drones, logistics networks, and critical infrastructure.

“Any risk in the digital domain will be amplified in the physical domain, and the physical domain will have a physical consequence,” Zhang told MLex on the sidelines of the summit.

He added that vehicles, drones, smart grids, and other infrastructure could become vulnerable as AI systems are embedded more deeply into physical operations.

Speakers highlighted reliability, operational monitoring, and post-deployment assurance as governance concerns. Summit discussions pointed to deployment-based governance models built around simulation, telemetry, and iterative testing, rather than one-time certification alone.

IMDA’s framework also recommends gradual rollouts, continuous monitoring, and further testing after deployment. It notes that agents interact dynamically with their environment and not all risks can be anticipated before release.

Monitoring becomes a deployment issue

Grab, which is piloting autonomous vehicles and delivery robots in Singapore’s Punggol district, said deployment governance depends heavily on simulation, testing, and continuous monitoring.

“We do a lot of simulation, we do a lot of testing in closed courses and open courses in order to make sure our robots are reliable,” Suthen Thomas Paradatheth, Grab’s chief technology officer, said during one of the summit panels.

“Before we scale to hundreds of robots, we make sure we crack it first in simulation and with a few robots,” he added.

Grab also pointed to monitoring systems designed to track robot performance and detect unexpected failures after deployment.

“There’s a long tail of issues that could emerge,” Paradatheth said.

The IMDA framework says organizations should assess agentic AI use cases based on data access, external system access, autonomy, and task complexity. It also points to the scope and reversibility of agent actions, third-party involvement, and overall system complexity.

It also recommends limiting agent access to tools and systems, applying least-privilege permissions, and defining standard operating procedures for agent workflows. Organizations should also set mechanisms to take agents offline when they malfunction.

Accountability spreads across more actors

MLex reported that embodied AI systems can involve several parties across development, manufacturing, and deployment. These include AI developers, robotics manufacturers, semiconductor suppliers, and infrastructure operators.

MLex also noted that responsibility can be harder to assign when systems continue adapting after deployment through software updates, telemetry, and operational data.

IMDA says organizations and humans remain accountable for agent actions, even when agents operate autonomously. The framework calls for clear responsibility across the agentic AI value chain, from model and platform providers to deployers, tooling providers, and end users.

Applied Materials said large-scale robotics deployment is also tied to semiconductor economics and systems integration. Om Nalamasu, the company’s chief technology officer, said robotics systems will depend on better sensors, energy efficiency, advanced packaging, and computing architectures.

Nalamasu said robotics systems would require purpose-built designs adapted to specific industrial ecosystems rather than a single solution for all environments.

Zhao Yuli, chief strategy officer of Chinese robotics startup Galbot, said Beijing is prioritizing deployment scale and industrial commercialization through government-backed testbeds, industrial partnerships, and long-term funding initiatives.

Galbot has deployed humanoid robotics systems in retail, warehouse, and pharmaceutical operations in China. These include autonomous stores that operate around the clock. Zhao said semi-structured industrial environments are likely to become an early commercialization path because they offer more controllable operating conditions.

Japan is placing more focus on standards-setting, robotics datasets, and safety governance. Professor Yutaka Matsuo of the University of Tokyo’s Graduate School of Engineering pointed to an “AI Association” project aimed at collecting 100,000 hours of robotics data to support robotic foundation models.

Matsuo also referred to Japan’s AI Safety Institute and the Hiroshima AI Process as part of broader efforts to develop governance standards for embodied AI systems with Singapore and other Asian countries.

Singapore sets out agent controls

Singapore’s framework outlines four governance areas for agentic AI. These cover upfront risk assessment, human accountability, technical controls, and end-user responsibility. The framework describes them as an iterative process rather than a one-time assessment.

The framework says human oversight must be adapted for agentic systems because continuous review of all workflows becomes impractical at scale. It recommends human approval at significant checkpoints, including high-stakes actions, irreversible actions, and outlier behavior.

IMDA also identifies automation bias and alert fatigue as risks when humans supervise capable agents. It recommends auditing oversight through indicators such as human override rates and response times, and using automated real-time monitoring to flag unexpected behavior.

The framework says users should be informed about what actions an agent can take, what data it can access, and what responsibilities remain with the user. It also recommends employee training on human-agent interaction, oversight, and the professional skills needed to assess agent outputs.

Companies test AI in regulated workflows

JPMorgan is implementing AI tools across its global investment banking business, Paul Uren, the bank’s Asia Pacific head of investment banking, told Reuters. The bank said the tools help bankers access more information and synthesize it with internal systems. They are also being used to prepare content and support client engagement.

JPMorgan CEO Jamie Dimon told Bloomberg News that the bank would hire more AI specialists and fewer traditional bankers. Reuters reported that global banks are increasing AI investment, reshaping workforces, and changing job roles.

The bank is also among selected organizations permitted by Anthropic to use its Mythos cybersecurity model under a controlled initiative known as Project Glasswing. According to Anthropic, Mythos can detect old vulnerabilities in browsers, infrastructure, and software.

Reuters reported that Goldman Sachs, Citigroup, Bank of America, and Morgan Stanley also have access to, or are testing, Mythos, citing sources and company executives.

IMDA’s framework includes a case study from OCBC Bank of Singapore on source-of-wealth analysis. The system parses income-related documents and drafts a source-of-wealth memo. It does not make credit, onboarding, or risk decisions autonomously.

In that case, the workflow is limited to task-level autonomy and operates only when triggered by predefined workflows. Human review is required at critical decision points, and final validation remains with designated reviewers.

Robots move into industrial use

In Japan, one-third of companies are already using or considering AI-powered robots, according to a Reuters survey conducted by Nikkei Research from May 1 to May 15. The survey contacted 492 companies, with 220 responding on the condition of anonymity.

About 4% of respondents said they already use AI robots, 5% plan to deploy them, and 25% are considering doing so. The remaining 66% said they had no such plans.

Transportation equipment manufacturers were the most active group in the survey, with 80% already using AI robots or considering deployment. By comparison, 94% of wholesale sector respondents said they had no plans to deploy AI robots.

Among companies using, planning to use, or considering AI robots, 71% selected manufacturing as a use case. Another 19% selected dangerous tasks, while 11% selected customer-facing services.

The Japanese government expects AI robots to help address the country’s chronic labor shortage and support its position in industrial robotics. Japan is home to robotics companies including Fanuc, Yaskawa Electric, and Kawasaki Heavy Industries, but faces competition from China and the United States in AI-enabled robotics.

Retail agents expand beyond search

Walmart has outlined plans to use agentic AI across shopping, employee, supplier, and developer workflows.

In July 2025, the retailer announced plans for four AI-powered “super agents.” They are designed for shoppers, store employees, suppliers and sellers, and software developers. Walmart said these agents would become the main entry point for AI interactions across those groups.

One of the tools, Sparky, is already available in Walmart’s app as a generative AI-powered shopping assistant. Hari Vasudev, Walmart’s US chief technology officer, said its expanded version would be able to reorder items and plan events. It would also use computer vision to suggest recipes based on the contents of a shopper’s fridge.

Walmart is also developing an Associate super agent for store workers and corporate staff. A separate Marty agent is being built for sellers, suppliers, and advertisers. The retailer is also working on a Developer super agent for testing, building, and launching future AI tools.

The company declined to say whether the agents would replace jobs. Dave Glick, senior vice president of enterprise business systems, said the tools would create new jobs, without giving further details.

See also: OpenAI opens Singapore AI lab as IMDA updates AI framework

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.