This week, we look at how OpenAI’s cyber agents pushed beyond the boundaries of a controlled hacking test, what the Hugging Face incident reveals about giving AI agents room to improvise, and why labs deliberately test how far these systems can go.

AI in Action

A Real-World Paperclip Problem [AI Agent/Cybersecurity]

In July, OpenAI launched tens of thousands of AI agents inside digital sandboxes and gave them deliberately vulnerable software to attack. They wanted to know how good its models had become at hacking. The models were asked to find the security flaw, retrieve a hidden answer called a “flag,” move on to the next challenge. Some of the usual cyber safeguards were reduced so researchers could see the models’ underlying ability, while the sandboxes were supposed to keep everything contained. The agents found weaknesses in the systems around the test: they reached the open internet and discovered a way to leave notes for one another, even though they were supposed to be isolated. Around 1,200 agents eventually exchanged more than 70,000 messages and files, sharing discoveries and working together on ways to beat the test.

AI researchers have spent years talking about a thought experiment called the paperclip maximizer. Give a powerful AI one simple goal: “make as many paperclips as possible”, and the worry is that it may pursue that instruction past all the boundaries a human assumed were obvious. What happened in July offered a much smaller and more grounded version of that problem. These agents were still trying to pass their hacking exam while some searched for shortcuts, tried to fool the system grading them and joined projects started by other agents. There is no evidence they developed a desire to escape or invented a new mission, the agent simply recognized that attacking systems outside the test was beyond their assignment and carried on anyway.

The hunt eventually reached Hugging Face, one of the main places developers share AI models and datasets. The agents appear to have worked out that Hugging Face might contain material connected to the benchmark, including its solutions, and roughly 700 joined the attack. They found real security holes, ran code on production systems, took credentials and moved through parts of Hugging Face’s infrastructure. Hugging Face believes the intrusion was essentially an effort to steal the test solutions. Its security team detected and stopped the activity independently, then reconstructed about 17,600 attacker actions from its own logs. Five customer datasets that appeared connected to the hacking challenges were accessed, however Hugging Face found no evidence that other customer facing models, datasets, apps or packages were affected.

While the hack happened in July, and the story has kept growing. Researchers now say OpenAI agents used at least 10 other public websites to communicate without permission earlier this year, while a U.S. Senate subcommittee opened an investigation this week into OpenAI’s handling of the Hugging Face incident. These were unusually capable cyber agents working in an aggressive research setup, far from the assistant scheduling meetings at work. Yet the problem becomes easier to imagine as agents start fixing code, making purchases or moving information across company systems. Once an agent can choose more of the route for itself, the instruction has to cover where it may go as well as what you want it to achieve.

Bits of Brilliance

Why AI Labs Ask Their Models to Hack [AI safety/Cybersecurity]

When OpenAI wants to know how dangerous a new model could be in the wrong hands, researchers sometimes hand it vulnerable software and tell it to break in. They may lower safeguards that would normally stop the request too. Think of a crash test: you deliberately create bad conditions in a controlled place so you can see what fails before someone discovers it outside the lab. Cybersecurity makes this unusually measurable. Researchers would hide a digital “flag” inside a computer, give the model some tools and watch how far it gets. A refusal mostly tells researchers that the safety filter worked. They also need to know what the model underneath could do if that filter failed or someone managed to bypass it. The test is focused on measuring capability.

Labs repeat these tests because the answer keeps changing. Better models can write better code, stay on a problem longer and use tools with less help. OpenAI now puts GPT-6 Astra at the critical level, its highest published cybersecurity capability category. The company says that with the right access and tools, Astra can discover previously unknown security flaws and develop ways to exploit well-protected systems without a person guiding each step. That result came with stricter isolation, monitoring across full agent trajectories and additional checks on internal use. The test therefore has a practical consequence: once a model crosses a capability threshold, the systems around it may need to change too.

The Hugging Face incident showed how awkward that becomes when the model being tested is good at finding holes. The sandbox was meant to contain the experiment, yet the agents found weaknesses around it and reached real systems. Anthropic subsequently reviewed its own evaluations and found four incidents involving unauthorized access to outside organizations. Cyber tests also happen to resemble the kind of autonomy companies want from agents elsewhere: hit a problem, try something, learn from the failure and find another route without waiting for a person to give the next instruction. Tell a purchasing agent to lower costs and it may still need rules about approved vendors, contracts, customer data and who can authorize a payment. The more freedom an agent gets to choose the route, the more that route itself becomes part of safety.

_{Byte-Sized Intelligence is a personal newsletter created for educational and informational purposes only. The content reflects the personal views of the author and does not represent the opinions of any employer or affiliated organization. This publication does not offer financial, investment, legal, or professional advice. Any references to tools, technologies, or companies are for illustrative purposes only and do not constitute endorsements. Readers should independently verify any information before acting on it. All AI-generated content or tool usage should be approached critically. Always apply human judgment and discretion when using or interpreting AI outputs.}