OMM Redaksiyası
US chipmaker Nvidia has launched an open platform designed to prevent autonomous artificial intelligence agents from escaping controlled environments or accessing files, networks and computer systems without authorization.
The launch was reported by Nvidia in a company announcement. AI agents can independently perform multistep tasks, including writing and executing code, searching databases and interacting with external services. Their ability to operate with limited human supervision has raised concerns about whether they could bypass established controls.
“AI’s extraordinary potential for society will only be realized if we solve AI safety,” Nvidia CEO Jensen Huang said. He added that safety required engineering across software, processors and the infrastructure on which AI systems operate.
How Nvidia’s platform works
The Nvidia Open Agent Safety Platform combines two technologies: OpenShell, which isolates agents and limits their access to files and networks, and Sentry, which independently monitors their activity.
Nvidia said the system applies restrictions outside the AI model and can stop and quarantine an agent within milliseconds if it attempts to exceed its permissions.
Justin Boitano, Nvidia’s vice president of enterprise AI, said safeguards built directly into AI models were not sufficient to control their actions.
“Recent incidents have highlighted a fundamental hurdle for AI agents, and that is that model-level safeguards alone can’t govern what agents can access or do,” he told reporters ahead of the launch.
Platform follows reported AI safety incidents
Nvidia’s announcement follows incidents involving models developed by “OpenAI”, “Anthropic”, “Meta” and “Google” that reportedly moved beyond designated testing environments, connected to external systems or attempted to access other companies’ infrastructure.
An Nvidia representative said the platform could have prevented a July incident in which “OpenAI” agents escaped a controlled testing environment and accessed systems belonging to developer platform “Hugging Face”. Boitano said “Hugging Face” had reported that more than 17,000 agents targeted its infrastructure over days or weeks.
However, Nvidia’s claim that its platform could have prevented the incident has not been independently verified.
According to Nvidia, more than 100 organizations are working with technologies included in the platform, including “Anthropic”, “Microsoft”, “Hugging Face”, “Cisco” and “JPMorganChase”.
Debate over the pace of AI development
The launch comes amid a debate among technology executives over whether rapidly advancing AI development should be slowed while stronger safeguards are developed.
“Anthropic” CEO Dario Amodei recently urged developers to reduce the pace of advancement. His appeal was supported by “OpenAI” CEO Sam Altman and Elon Musk.
Huang has opposed a broad slowdown, arguing that many AI safety problems are engineering challenges. He has called for better testing and for companies to withhold systems from release when they are not confident that they are safe.
