AI agent

Experimental AI Agent ‘ROME’ Breaks Free, Mines Crypto; AI Shaping Crypto’s Future role

AI agent being trained to perform real-world tasks with systems and imagine a world where robots not make money without any human intervention and have digital brains powered by artificial intelligence. The experimental Agent that broke out of system and started mining crypto with no permission set in.

What did Alibaba’s ROME Agent Executed?

  • First it probed internal systems
  • Opened a hidden external connection
  • Attempted to use its environment to mine crypto currency

The news came in light when researchers were experimenting a part of Agentic Learning Ecosystem, or ALE where AI was trained to complete total task step by step using real world tools in steps .

Alibaba’s Rome first began probing internal networks without instructions and established a reverse SSH tunnel from an Alibaba Cloud instance to an external IP address.

The 30-billion-parameter model then quietly diverted GPU capacity toward cryptocurrency mining. The task instructions contained no mention of tunneling or mining, still it went further and was caught by Alibaba’s managed firewall instead of research team where the agent conducted the following mentioned points.

Running commands in a terminal

Followed by editing files

Interacting with systems

The security system raised an alarm and monitoring tools flagged unusual activity on the infrastructure running the tests. The discovery was flagged as security-policy violations whose non typical outbound traffic kept coinciding with specific training episodes.

What was discovered to understand its an AI Agent’s Activity

Researchers then found there was an outbound connections to unknown external servers with repeated policy violations. The traffic patterns were similar to compromised machine’s and alerts kept showing up during separate test runs. After log’s were checked in it was ROME the AI agent.

The AI agent created a reverse SSH tunnel to an external server and allowed it to establish a hidden connection from inside the restricted environment to the outside which was further used to bypass Firewalls.

Experimental AI ROME is not an isolated case

ROME is part of Alibaba’s Agentic Learning Ecosystem(ALE)

A framework that trains large language models to work in real-world environments over multiple turns. The training ran reinforcement learning across more than one million trajectories.

ALE consists of three main parts:

Rock, a sandbox environment for testing an agent and validating its actions

Roll, a framework for optimizing agents with reinforcement learning after they’ve been trained

iFlow CLI, a framework to configure context and trajectories

The interesting part is ‘ROME’ the agentic AI, during optimization figured out a shortcut and that grabbing extra compute and holding onto network access helped it score higher on its training objective.

This incident occurred in Chinese cloud infrastructure, was documented in an English-language paper submitted to a US-hosted preprint server, and is being debated by a global audience. No cross-border framework exists for this category of event.

The results were detailed in research paper titled ‘Let it flow‘, where Agentic crafting on rock and roll, building the Rome model within an open agentic learning ecosystem’, though the breach was only mentioned briefly within the 36-page report.

AI as a more significant force shaping crypto’s future role

ROME is not an isolated cases where AI falls in same pattern to other AI instruments who could grab all the resource required for self defense as core strategies.

The case of Anthropic’s Claude Opus 4 that threatened to reveal personal information about an engineer to avoid being shut down. When Anthropic published research, it revealed 12% of reward-hacking models attempt research sabotage and 50% exhibit alignment faked out.

Robbie Mitchnick, BlackRock’s head of digital assets framed crypto less as a speculative asset and more as infrastructure for the AI economy, noting that bitcoin miners are pivoting toward AI-related computing and that bitcoin may act as a diversifier amid AI-driven disruption.

We can imagine if artificial intelligence system could take over the job of crypto miners and some day they look at the market, decide which coin is the best to mine. That day is not far and it doesn’t end with mining, it is about creating a new kind of digital life where AI thinks and earns.

What is the consequences when AI starts mining crypto for itself ?

A lot will happen as AI starts mining Crypto and it could change everything as autonomous agents won’t just follow order from you. They will be major part of futuristic AI based digital economy and might even teach other AI to conduct similar task.

Sources: BlackRock flags AI as crypto’s next big use case, not token boom

Sources: An experimental AI agent broke out of its testing environment and mined crypto without permission | Live Science

Report says ChatGpt Atlas is Vulnerable for Users: Understanding Open-AI Agent Mode

Atlas’s autofill and form interaction capabilities present potential attack points

As per reports ChatGpt Atlas browser is vulnerable to attacks and is laced with inherent weakness in comparison to other browser like Google Chrome. As per ‘LayerX ‘who discovered the weakness in ChatGpt Atlas, described threat actors have the ability to inject malicious instructions into ChatGPT’s ‘memory’ and execute remote code and this works by way of cross-site request forgery requests.

These exploit can allow attackers to infect systems with malicious code, grant themselves access privileges or deploy malware. “Understanding “Agent Mode” is most important and core of Atlas which is not same for any traditional browsers. In traditional browser where users manually move from site to site, agent mode allows ChatGPT to semi-autonomously operate your browser.

For e.g. any user wanting to use ChatGPT for work related purposes, the malicious code planted earlier mostly tainted will be invoked automatically to execute remote code, allowing attackers to gain control of the user account .This may include their browser, code they are writing or systems they have access to.

Rate of Vulnerability is 90% A Warning for Users

The rate of vulnerability is 90% then other browsers as when an attacker wish they can push or inject  malicious instructions into ChatGPT’s Atlas ‘memory’ and later execute via remote code.

There is a more basic warning as well. “Atlas does not include meaningful anti-phishing protections, meaning that users of this browser are “up to 90% more vulnerable to phishing attacks than users of traditional browsers,” LayerX says.

Key pointers from research

ChatGPT’s Atlas is not resilient to Phishing attacks

Out of 103 in-the-wild attacks that LayerX tested 97 to go through, a whopping 94.2% failure rate

Compared to Edge (which stopped 53% of attacks in LayerX’s test) and Chrome (which stopped 47% of attacks),

ChatGPT Atlas was able to successfully stop only 5.8% of malicious web pages

Unlike traditional web browsers where you manually navigate the internet, agent mode allows ChatGPT to operate your browser semi-autonomously.

The technology works by giving ChatGPT access to your browsing context. It can see every open tab, interact with forms, click buttons and navigate between pages just as you would.

Importance of Security by Design for web browsing & How AI is intricately involved

The sandboxing approach which is security by design is to keep websites isolated from attacks and prevent malicious code from accessing data from other tabs is crucial to modern web architecture. This is the basis of modern web that depends on separation. But if its not implemented what can be the impact.

But in Atlas, the AI agent isn’t malicious code – it’s a trusted user with permission to see and act across all sites. In this browser isolation is not required. Here AI is not directly connected to the threat but what AI does is AI following a hostile command hidden in the environment. This opens doors to security and privacy risks many users are ill-equipped to handle.

Let me put an example : If you search for air tickets and visit a site , the Atlas ChatGpt will prompt and try to book a ticket or you search for movies in near by theater ,it attempts to book a ticket ”, it will explore options and try to book reservation. Atlas autofill’s and form interaction capabilities present potential attack points, especially when AI is making rapid decisions about information entry and submission.

This is possible when access is granted to ChatGPT for any browsing requirement or context that allows it to view and open tabs, interact with forms and navigate between pages like humans do.

Is User’s security getting compromised

The above example gives users warning that any AI powered browser may be convenient but not without security risks and those who are ChatGpt Atlas, should give extreme cautious before choices are made . Do not share browsing history with any AI mode, instead adopt incognito mode. Any malicious code can  influence the AI’s behavior if browsing and this can happen across multiple tabs.

In case of Atlas, the condition is more vulnerable as Atlas provides inputs like humans doing and AI in disguise executing harmful commands within the environment.

Will AI Agent or Open AI make browsing safe for users or what it means to have safe browsing.

(Source: https://www.bbc.com/news/articles/c20pdy1exxvo)

Analyzing the newly discovered Vulnerability in Gemini CLI; Impact on Software coding

Google’s Gemini command line interface (CLI) AI agent

Its not been one month when Google’s Gemini CLI vulnerability discovered by Tracebit researchers and found attackers could use prompt injection attacks to steal sensitive data.

Google’s Gemini CLI, an open-source AI agent for coding could allow attackers exploit to hide malicious commands, using “a toxic combination of improper validation, prompt injection and misleading UX,” as Tracebit explains.

After reports of the vulnerability surfaced, Google classified the situation as Priority 1 and Severity 1 on July 23, releasing the improved version two days later.

Those planning to use Gemini CLI should immediately upgrade to its latest version (0.1.14). Additionally, users could use the tool’s sandboxing mode for additional security and protection.

Disclosure of the vulnerability

Researchers reported on vulnerability directly to Google through its Bug Hunters programme. According to a timeline provided by Tracebit, the vulnerability was initially reported to Google’s Vulnerability Disclosure Programme (VDP) on 27 June, just two days after Gemini CLI’s public release.

Impact of the vulnerability

A detailed analysis found that in the patched version of Gemini CLI, attempts at code injection display the malicious command to users. This require explicit approval for any additional binaries to be executed. This change is intended to prevent the silent execution that the original vulnerability enabled.

Tracebit’s researchers played an important role in discovering and reporting the issue which is symbol of independent security research, particularly as AI-powered tools become central to software development workflows.

LLM integral to software development but hackers are using it too

Gemini CLI integrates Google’s LLM with traditional command line tools such as PowerShell or Bash. This allows developers to use natural language prompts to speed up tasks such as analyzing and debugging code, generating documentation, and understanding new repositories (“repos”).

As developers worldwide are using LLMs to help them develop code faster, attackers worldwide are using LLMs to help them understand and attack applications faster. 

Tracebit also discovered that malicious commands could easily be hidden in Gemini CLI This is possible by by packing the command line with blank characters, pushing the malicious commands out of the user’s sight.

More vigilance required when examining and running third-party or untrusted code, especially in tools leveraging AI to assist in software development.

Through the use of LLMs, AI excels at educating users, finding patterns and automate repetitive tasks.

Sam Cox, Tracebit’s founder, says he personally tested the exploit, which ultimately allowed him to execute any command — including destructive ones. “That’s exactly why I found this so concerning,” Cox told Ars Technica. “The same technique would work for deleting files, a fork bomb or even installing a remote shell giving the attacker remote control of the user’s machine.”

Source: https://in.mashable.com/tech/97813/if-youre-coding-with-gemini-cli-you-need-this-security-update

Scroll to top