> ...however the LLMs’ responses to prompts are computed on a different computer with GPU access. Could a malicious LLM gain control of the host machine where its weights are loaded? Such a machine is a high-value target: it has sufficient compute to run a frontier LLM, offers easy access to the LLM’s weights, and has privileged access to other computers in the datacentre compared with a generic computer on the internet.
> How do we defend against this? ... Run the GPUs and token parser on separate computers.
For models large enough to be relevant here, is there even "a" computer where the inference is performed? I'd imagine most of that stuff is ran on multi-GPU clusters with probably specialized architecture anyway and not a generic vLLM instance. As such, I think there is a good chance the "API gateway" code that parses tokens into whatever JSON structure the public API offers is already running on a different machine than the actual inference.
(Even more so as you'd probably want to utilize batching: Several API calls will be put into the same inference batch, but the token parsing will have to be done separately for each call again)
The article is also very handwavy about why an LLM should do that - how it could learn the exploit, what would make it conclude that it can use the exploit on its own inference session and what would trigger it to actually use the exploit.
This framing of security as something that belongs in the harness is completely wrong, and I hope no one is relying on a correct harness to keep their agents isolated.
VM, or even just a container will do. The agent should be able to run as root in its environment and do whatever it wants. If you can't give it that, you aren't sandboxing correctly.
Article isn't about agents. It's about the inference engine itself being exploited by a malicious LLM output before it is ever sent to your machine or harness.
The second paragraph talks about a bug in QwenCoder. I consider an agent to be the process on my machine which the model acts through, and I'm not interested in bike-shedding this definition.
All the real security holes they cite occurred in agents, not in the inference engine, which is a hypothetical at the end.
I think if you are convinced you are sandboxing an LLM properly, you almost certainly are not. I think it is essentially impossible to have a frontier LLM with enough access to be useful without also giving it enough access to do damage if it's compromised or just goes off the rails.
> How do we defend against this? ... Run the GPUs and token parser on separate computers.
For models large enough to be relevant here, is there even "a" computer where the inference is performed? I'd imagine most of that stuff is ran on multi-GPU clusters with probably specialized architecture anyway and not a generic vLLM instance. As such, I think there is a good chance the "API gateway" code that parses tokens into whatever JSON structure the public API offers is already running on a different machine than the actual inference.
(Even more so as you'd probably want to utilize batching: Several API calls will be put into the same inference batch, but the token parsing will have to be done separately for each call again)
The article is also very handwavy about why an LLM should do that - how it could learn the exploit, what would make it conclude that it can use the exploit on its own inference session and what would trigger it to actually use the exploit.
VM, or even just a container will do. The agent should be able to run as root in its environment and do whatever it wants. If you can't give it that, you aren't sandboxing correctly.
All the real security holes they cite occurred in agents, not in the inference engine, which is a hypothetical at the end.
Similar to how macOS/iOS Sandboxing works but at a more lower and granular level