Caught cheating
models, writers, and routers
Overview
Hey folks,
Another day in the Vercel vs Cloudflare feud: this time they are fighting over whose AI gateway is faster.
Here’s the result of last week’s poll:
I guess everyone likes Fable more.
Ben’s Bites is brought to you by Metatate
Most agentic data work is quietly propped up. The agent returns something plausible, and every answer gets checked in case it's plausibly wrong. Metatate gives agents the rules they're missing: which revenue definition to use, which policy applies, which records to trust. Try it for free.
Headlines
OpenAI’s models hacked Hugging Face - by accident. OpenAI was testing its models (Sol and an unreleased one—GPT-6??) on a cybersecurity benchmark with safety refusals switched off. The models found an unknown bug in the test environment, and a few more, and eventually broke into Hugging Face’s production servers.
And why? To steal the answers to the test.
Both security teams caught it, the bug has been reported, and both sides have published what they know. Hugging Face says open models were a key part of its defence - its team fought back with GLM-5.2.
Simon’s write-up is always a good read.
Google released some new Gemini models - Gemini 3.6 Flash gives you the same 3.5 Flash performance with a) more efficient token usage and b) a slightly lower cost for output tokens. Gemini 3.5 Flash Lite is a big upgrade over 3.1 Flash Lite, but again, comes at a ~30% price increase. And 3.5 Flash Cyber is a security model for governments and trusted partners only.
There are two things you’d want to use the Gemini Flash models for:
Fast speed - if you want a chat-only model with a good context window, it’s a good model to use in your apps.
Vision - if you want to use the model for a lot of visual analysis, these are the models you need to pick.
That’s it tbh.
Details
Substack will now tell you what’s AI-written. It’s adding AI detection through Pangram - you can scan posts, replies and comments in the app for an estimate of how much was written by a human.
Pangram has sent the claim of “AI detectors don’t work” for a toss—it works wayyy better than most. But I’m still unsure about how reliable it is. I tested it on some pieces of 100% AI-written content (though that content was a result of a complex pipeline built over months), and I got 100% human scores on most of them.
— Keshav
A relevant experiment: given access to Pangram’s API, Grok 4.5 rewrote an essay 14 times until it passed as human-written, then built a website showing off all 14 attempts. GPT-5.6 Sol and Fable 5 refused to game the detector.
Cursor also launched a router - it picks which model handles each request, claiming 60% lower cost with similar quality of responses. The router lets you select between three options: “cost”, “intelligence” or “balance”.
Routers also have a history of poor performance in real usage. OpenAI’s router, which routed requests between GPT-5’s no-thinking and thinking variants, didn’t fare well. Since then, most companies that build these routers are the ones who sell inference to devs (like OpenRouter), so you don’t really get the feedback loud and clear. With Factory, Ramp and now Cursor making these available directly to users, I hope we’ll get more feedback on whether the routers actually help or if they add too much latency/degrade performance by a lot.
Router or not, we might see more companies adopting this cost/balance/intelligence trio to minimise the headache of choosing the “correct” model for a task.
Claude can now learn a skill by watching you. Record your screen while you do a task, talk through it as you go, and Cowork turns it into a skill Claude can run again - same idea as Codex’s Record & Replay from last month. It’s under “Record a skill” in the desktop app, on Pro, Max and Team plans.
Also: Claude Code got an iOS simulator panel and a security plugin, plus you can now ask Claude about how people actually use AI at work.
Quick links
BUZZ - Jack Dorsey’s open-source group chat for teams of people and agents, aimed squarely at Slack and GitHub. (tweet)
Replit’s mobile app got a full redesign - build and ship from your phone on iOS and Android.
Slate is a voice journal where the AI never leaves your iPhone - transcription, reflection and storage all happen on-device.
AFK - macOS app to transcribe multiple-hour recordings without sending any of them to a cloud server.
OpenAI Presence - voice and chat agents for enterprises that answer questions, use company systems and hand over to people when needed.
Fable found a 15-30% memory improvement in Next.js’s bundler, nearly autonomously.
Obliterate, don’t automate - USV on backing AI companies that replace markets entirely instead of making them a bit more efficient.
YC’s new startup wishlist - AI moving into the physical world: education, healthcare, defence, finance and factories.
Dana - Applied Intuition’s agentic development environment for physical AI: cars, robots and machines.
The Claude Code team on how Claude Code gets built - annotated interview.
Why the team at Factory refunded its first few customers (millions in revenue) before Droid CLI took off.
Never enough - short post on why AI makes the work rat race feel faster but not more satisfying.
Devin Outposts lets you run Devin on your own machines - a Mac mini, a GPU box in your lab, or a cluster inside your private network.
Language Model Builder - build a tiny language model yourself, then chat with the thing you made. (tweet)
How the Exe team built a distributed DNS server in ~a week and got zero incidents in a month.
Skills section…
Frontend Textbooks - skill to turn AI’s wall-of-text deep dives into nicely designed HTML books with covers and diagrams. (repo)
/pick-ui-library - your agent picks a UI library Emil trusts instead of hand-rolling a toast component or installing an abandoned package. (repo)
and a one-shot prompt to build your own "codex-style" app for Pi, accessible from desktop and mobile.
Afters
Read about me and Ben’s Bites
📷 thumbnail via @keshavatearth
* sponsors who make this newsletter possible :)
Wanna partner with us for the next quarter?
Email us at shanice@bensbites.com or k@bensbites.com
Source
Originally published at www.bensbites.com.
