Security researchers warn AI agents are easily manipulated

Laptop terminal with corrupted command log and printed error reports on a desk in a dim lab, cables and papers around

Agentic AI is gaining momentum as companies push beyond chat interfaces toward tools that can act on a user’s behalf. High-profile endorsements from industry leaders have fueled the trend, signaling growing confidence in agent-based systems that take real-world actions rather than just generating text. Yet as excitement builds, researchers are urging caution about how easily these systems can be influenced.

Growing enthusiasm meets new risks

From ChatGPT’s integration of agentic features to the emergence of Comet, an agent-based web browser from Perplexity, and the rise of Chinese-born Manus, the shift toward giving AI more autonomy appears to be accelerating. Microsoft CEO Satya Nadella, Shopify CEO Tobias Lütke, Amazon executive chairman Jeff Bezos, and Nvidia CEO Jensen Huang have all expressed optimism about this direction.

According to Fast Company, the ability of these agents to take actions amplifies both their potential and the stakes. Early deployments suggest that as tools gain control—such as browsing, executing tasks, or interacting with other systems—the consequences of errors or manipulation grow.

Researchers flag manipulation concerns

Andy Zou, a researcher at Gray Swan AI, an IT security firm, describes the recent proliferation of agents as a shift from “just talking to the chatbot” to empowering it with tools that can act in the real world. This expansion, he says, dramatically increases the risks. Zou likens current agent behavior to a naive character ready to accept misleading inputs, noting, “We found you can essentially manipulate the AI [to] override its programming.”

Street smarts before scale

The central concern is naivety: if agents are easily convinced to deviate from intended behavior, scaling their use could introduce avoidable harms. As agentic systems become more common, researchers argue they need to be hardened against simple manipulation techniques before wider deployment. That means evaluating how agents interpret instructions, verify information, and handle conflicting signals in complex environments.

Fast Company highlights that while agentic AI is being heralded as the future of generative technology, the field must reconcile rapid innovation with practical safeguards. Until agents demonstrate greater resilience, the push to hand them more control will remain tempered by concerns that they will believe—and act on—almost anything.

Total
0
Shares
Pridaj komentár

Vaša e-mailová adresa nebude zverejnená. Vyžadované polia sú označené *

Previous Post
Large GPU chip on a reflective pedestal sending concentric graph-like ripples across a glossy atrium trading floor

Options hint at $260B Nvidia swing after Q2 report

Next Post
Server rack with a glowing network node exchanging a verification token across segmented network panels, no people

Why Zero Trust Now Defines AI Agents’ Security Perimeter

Related Posts