It is important to note here that prompt injection has not yet caused any disasters, or at least none that have been publicly reported. But now that there are potentially hundreds of thousands of OpenClaw agents on the Internet, early injection may seem like a more attractive strategy for cybercriminals. “Such tools are emboldening malicious actors to attack a much broader population,” Papernote says.
construction of railing
The term “prompt injection” was coined by popular LLM blogger Simon Willison in 2022, a few months before ChatGPT was released. Even at the time, it was possible to understand that LLMs would introduce an entirely new type of security vulnerability once they came into widespread use. LLMs can’t separate instructions received from users and the data used to fulfill those instructions, such as emails and web search results – in LLMs, they’re all just text. So if an attacker embeds some sentences in an email and the LLM mistakes them for instructions from its user, the attacker can do whatever he wants with the LLM.
Early injection is a tough problem, and it doesn’t seem to be going away any time soon. “We don’t really have a silver-bullet defense right now,” says Don Song, a computer science professor at UC Berkeley. But a strong academic community is working on this problem, and they have come up with strategies that could eventually make AI personal assistants safer.
Technically speaking, it is possible to use OpenClaw today without the risk of prompt injection: just do not connect it to the Internet. But restricting OpenClaw from reading your emails, managing your calendar, and doing online research defeats much of the purpose of using an AI assistant. The trick to protecting against early injection is to prevent the LLM from responding to hijacking attempts, while also giving it room to do its job.
One strategy is to train the LLM to ignore prompt injections. A key part of the LLM development process, called post-training, involves taking a model that knows how to generate realistic text and turning it into a useful assistant by “rewarding” it for answering questions appropriately and “punishing” it for failing to do so. These are reward and punishment metaphors, but the LLM learns from them like an animal. Using this procedure, it is possible to train LLMs not to respond to specific instances of prompt injection.
But there is a balance: train an LLM to reject injected commands too enthusiastically, and it may start rejecting legitimate user requests as well. And because there is a fundamental element of randomness in LLM practice, even an LLM that has been trained very effectively to resist accelerated injection will still occasionally slip up.
Another approach involves stopping the quick injection attack before it reaches the LLM. Typically, this involves using a special detector LLM to determine whether there is any spurious injection in the data being sent to the original LLM. one in recent studyHowever, even the best performing detector completely failed to catch some categories of prompt injection attacks.
The third strategy is more complex. Instead of controlling the LLM’s inputs by detecting whether they contain accelerated injection or not, the goal is to design a policy that guides the LLM’s output – that is, its behavior – and prevents it from doing anything harmful. Some defenses in this regard are quite simple: for example, if an LLM is only allowed to email certain pre-approved addresses, it certainly will not send its user’s credit card information to an attacker. But such a policy would prevent LLMs from performing many useful functions, such as conducting research on your user’s behalf and reaching out to potential professional contacts.
