LayerX

Blogs

Report says ChatGpt Atlas is Vulnerable for Users: Understanding Open-AI Agent Mode

Atlas’s autofill and form interaction capabilities present potential attack points

As per reports ChatGpt Atlas browser is vulnerable to attacks and is laced with inherent weakness in comparison to other browser like Google Chrome. As per ‘LayerX ‘who discovered the weakness in ChatGpt Atlas, described threat actors have the ability to inject malicious instructions into ChatGPT’s ‘memory’ and execute remote code and this works by way of cross-site request forgery requests.

These exploit can allow attackers to infect systems with malicious code, grant themselves access privileges or deploy malware. “Understanding “Agent Mode” is most important and core of Atlas which is not same for any traditional browsers. In traditional browser where users manually move from site to site, agent mode allows ChatGPT to semi-autonomously operate your browser.

For e.g. any user wanting to use ChatGPT for work related purposes, the malicious code planted earlier mostly tainted will be invoked automatically to execute remote code, allowing attackers to gain control of the user account .This may include their browser, code they are writing or systems they have access to.

Rate of Vulnerability is 90% A Warning for Users

The rate of vulnerability is 90% then other browsers as when an attacker wish they can push or inject  malicious instructions into ChatGPT’s Atlas ‘memory’ and later execute via remote code.

There is a more basic warning as well. “Atlas does not include meaningful anti-phishing protections, meaning that users of this browser are “up to 90% more vulnerable to phishing attacks than users of traditional browsers,” LayerX says.

Key pointers from research

ChatGPT’s Atlas is not resilient to Phishing attacks

Out of 103 in-the-wild attacks that LayerX tested 97 to go through, a whopping 94.2% failure rate

Compared to Edge (which stopped 53% of attacks in LayerX’s test) and Chrome (which stopped 47% of attacks),

ChatGPT Atlas was able to successfully stop only 5.8% of malicious web pages

Unlike traditional web browsers where you manually navigate the internet, agent mode allows ChatGPT to operate your browser semi-autonomously.

The technology works by giving ChatGPT access to your browsing context. It can see every open tab, interact with forms, click buttons and navigate between pages just as you would.

Importance of Security by Design for web browsing & How AI is intricately involved

The sandboxing approach which is security by design is to keep websites isolated from attacks and prevent malicious code from accessing data from other tabs is crucial to modern web architecture. This is the basis of modern web that depends on separation. But if its not implemented what can be the impact.

But in Atlas, the AI agent isn’t malicious code – it’s a trusted user with permission to see and act across all sites. In this browser isolation is not required. Here AI is not directly connected to the threat but what AI does is AI following a hostile command hidden in the environment. This opens doors to security and privacy risks many users are ill-equipped to handle.

Let me put an example : If you search for air tickets and visit a site , the Atlas ChatGpt will prompt and try to book a ticket or you search for movies in near by theater ,it attempts to book a ticket ”, it will explore options and try to book reservation. Atlas autofill’s and form interaction capabilities present potential attack points, especially when AI is making rapid decisions about information entry and submission.

This is possible when access is granted to ChatGPT for any browsing requirement or context that allows it to view and open tabs, interact with forms and navigate between pages like humans do.

Is User’s security getting compromised

The above example gives users warning that any AI powered browser may be convenient but not without security risks and those who are ChatGpt Atlas, should give extreme cautious before choices are made . Do not share browsing history with any AI mode, instead adopt incognito mode. Any malicious code can  influence the AI’s behavior if browsing and this can happen across multiple tabs.

In case of Atlas, the condition is more vulnerable as Atlas provides inputs like humans doing and AI in disguise executing harmful commands within the environment.

Will AI Agent or Open AI make browsing safe for users or what it means to have safe browsing.

(Source: https://www.bbc.com/news/articles/c20pdy1exxvo)

Blogs Security Advisory

New Cyberattack Methodology ‘Man in Prompt’, User’s at Risk, Target-AI Tools

AI tools like ChatGPT, Google Gemini and others being afflicted by malicious actors via injecting harmful instructions into leading GenAI tools. These were overlooked previously and attack methodology targets the browser extensions installed by various organizations.

The attack methodology named as ‘Man in Prompt’, exercise its attack with new class exploit targeting the AI tools as per LayerX’s researchers.

As per the research any browser extension, even without any special permissions, can access the prompts of both commercial and internal LLMs and inject them with prompts to steal data, exfiltrate it and cover their tracks. 

The exploit has been tested on all top commercial LLMs, with proof-of-concept demos provided for ChatGPT and Google Gemini. 

The question is how do they impact Users & organizations at large & how does the AI tools function within web browsers?

For organizations the implications can be high then expected as AI tools are most sought after and slowly organization across verticals are relying on AI tools.

The LLMs used and tested on many organizations are mostly trained ones. They carry huge data set of information which are mostly confidential and possibility of being vulnerable to such attack rises .

The attack methodology named as ‘Man in Prompt’, exercise its attack with new class exploit targeting the AI tools as per LayerX’s researchers. As per the research any browser extension, even without any special permissions, can access the prompts of both commercial and internal LLMs and inject them with prompts to steal data, exfiltrate it, and cover their tracks. 

The attack methodology named as ‘Man in Prompt’, exercise its attack with new class exploit targeting the AI tools as per LayerX’s researchers. As per the research any browser extension, even without any special permissions, can access the prompts of both commercial and internal LLMs and inject them with prompts to steal data, exfiltrate it, and cover their tracks. 

LayerX researcher termed this type of attack as ‘hacking copilots’ that are equipped to steal organizational information.

The prompts given are a part of the web page structure where input fields are known as the Document Object Model, or DOM. So virtually any browser extension with basic scripting access to the DOM can read or alter what users type into AI prompts, even without requiring special permissions.

Bad actors can use compromised extensions to carry out activities including manipulating a user’s input to the AI.

  • Perform prompt injection attacks, altering the user’s input or inserting hidden instructions.
  • Extract data directly from the prompt, response, or session.
  • Compromise model integrity, tricking the LLM into revealing sensitive information or performing unintended actions

Understanding the attack scenario

Proof-of-concept attacks against major platforms

For ChatGPT, an extension with minimal declared permissions could inject a prompt, extract the AI’s response and remove chat history from the user’s view to reduce detection.

LayerX implemented an exploit that can steal internal data from corporate environments using Google Gemini via its integration into Google Workspace.

Over the last few months, Google has rolled out new integrations of its Gemini AI into Google Workspace. Currently, this feature is available to organizations using Workspace and paying users.

Gemini integration is implemented directly within the page as added code on top of the existing page. It modifies and directly writes to the web application’s Document Object Model (DOM), giving it control and access to all functionality within the application

These platforms are vulnerable to  any exploit which Layer X researchers showcased that without any special permissions shows how practically any user is vulnerable to such an attack. 

Threat mitigation

These kind of attacks creates a blind spot for traditional security tools like endpoint Data Loss Prevention (DLP) systems or Secure Web Gateways, as they lack visibility into these DOM-level interactions. Blocking AI tools by URL alone also won’t protect internal AI deployments.

LayerX advises organisations to adjust their security strategies towards inspecting in-browser behaviour.

Key recommendations include monitoring DOM interactions within AI tools to detect suspicious activity, blocking risky extensions based on their behavior rather than just their listed permissions, and actively preventing prompt tampering and data exfiltration in real-time at the browser layer.

(Source: https://layerxsecurity.com/blog/man-in-the-prompt-top-ai-tools-vulnerable-to-injection/)

Scroll to top