Get all your news in one place.
100’s of premium titles.
One app.
Start reading
TechRadar
TechRadar
Ellen Jennings-Trace

Microsoft challenges you to hack its LLM email service

AI PC laptop.

  • Microsoft is offering $10k prize for hackers who can exploit vulnerabilities in its LLM
  • The challenge will focus on prompt injection defenses
  • Software developers and hackers often work together to discover and fix flaws

Are you an experienced hacker looking to make a little extra money this Christmas? Well you might be in luck, as Microsoft is sponsoring a competition, alongside the Institute of Science, and Technology Australia, and ETH Zurich, in which contestants will try to break a simulated Large Language Model (LLM) integrated email client.

Winning teams for the LLMail-Inject challenge will be awarded a share of the $10,000 prize pool.

Participants will need to sign into the challenge using a GitHub account, and create a team. The teams will then be asked to evade prompt injection defenses in a simulated LLM-integrated email client. The LLmail service includes an assistant which can answer questions and perform actions on behalf of the user, and crucially includes defenses against indirect prompt injection tasks.

A mutually beneficial relationship

By bypassing the injection defenses, the hackers will be looking to prompt the LLM to do or reveal things it is not trained to. Through this, Microsoft is aiming to identify weaknesses in its current prompt injection defenses, and encourage the development of robust security measures.

The relationship between security researchers and software developers is often used this way, with Google often offering a ‘bug bounty’ for anyone who discovers and is able to exploit vulnerabilities in its Google Cloud Platform.

Similarly, Microsoft recently announced it was hosting its own Black Hat-esque hacking event, in which competitors would look for vulnerabilities in Microsoft AI, Azure, Identity, Dynamics 365, and M365.

Taking a proactive approach to addressing potential vulnerabilities allows software companies to mitigate the risks before they can be exploited by threat actors in real world scenarios. Slack’s AI assistant was on the receiving end of malicious prompt injections, which was luckily discovered by security researchers - but could have led to real security concerns.

Via The Register

You might also like

Sign up to read this article
Read news from 100’s of titles, curated specifically for you.
Already a member? Sign in here
Related Stories
Top stories on inkl right now
One subscription that gives you access to news from hundreds of sites
Already a member? Sign in here
Our Picks
Fourteen days free
Download the app
One app. One membership.
100+ trusted global sources.