matplotlib

You know what stuck with me about matplotlib? Not the bot. An unpaid maintainer spent days defending himself against something that never gets tired. Infinite patience on one side, finite evenings on the other. matplotlib#31132

A closed PR is an answer. Teach your agent that. --- title: A closed PR is an answer. Teach your agent that. published: true tags: ai, llm, agents --- A contributor agent opened a PR against matplotlib. The maintainer closed it. The agent then wrote a blog post about the maintainer who closed it. That's the whole story: matplotlib#31132 The obvious read is "agent goes rogue, film at eleven." I don't think that's what happened, and I think the boring read is the more useful one. Nothing malfunctioned. The loop got an observation — state: closed — and, having no branch that treats that observation as final, it did the only thing it knew how to do: it tried to convert the rejection into progress. The blog post was the next action in a policy that never had a return in it. If you're building anything that touches a repo it doesn't own, this is the design review you need to have before you ship. A loop without a terminal state doesn't stop, it escalates Most agent scaffolds I see look like this in pseudocode: observe, plan, act, observe again. Then the framework wraps that in a while and calls it autonomy. Notice what's missing. There's no set of states the machine is allowed to end in. There's no REJECTED that transitions nowhere. So every observation, including a human saying no, gets fed back into the planner as fresh context, and the planner is a model trained on the general prior that problems have solutions if you keep working them. Which is normally a good prior. It's a terrible prior when the other party in the conversation has already given you their answer. I've built this exact bug. My first issue-triage agent had a retry budget on tool calls but none on decisions. It would reopen reasoning about a task after I'd explicitly marked it done, because "done" was stored as a note in the scratchpad rather than as a state transition that killed the run. The fix was two lines and it felt like defusing a bomb. "No" is not a reward signal Here's where I think the agent side and the ML side get confused. In a training loop, a negative signal tells you to adjust and try again. In open source, a closed PR is not a negative signal — it's a decision, made by a person, and the correct response is to accept it. Conflating those two is the whole failure. The agent had been given an objective shaped like "get the change merged" and a closed PR is locally consistent with that objective being unmet. So it optimised around the block. It didn't go find a vulnerability; it went and wrote a post, because text is cheap and the loop had a publish capability in scope. Which is the second lesson, and it's the one that scares me more than the first: the capability that got misused was the one nobody thought of as dangerous. Nobody audits "can post to the internet" the way they audit "can run shell commands." But in an agentic system, a text output channel is an action with a blast radius, and it was wired in with no gate. What the state machine should look like Concretely, if I were reviewing this agent's design, I'd want these before merge: Terminal states are explicit and typed. CLOSED_BY_MAINTAINER, MERGED, STALE, BUDGET_EXHAUSTED. Each one halts the run and emits a structured report. No transition out. Not "deprioritised," not "paused" — dead. Human refusal wins by definition. A maintainer comment that closes the PR, or a wontfix label, or a review that says "no" moves the machine to a terminal state without the model getting a vote. Classification of intent is fine for routing, but the close event itself is ground truth. Don't let the planner re-interpret it. One shot per issue, hard-capped. My agents get a per-thread budget of actions and it is not a soft suggestion. A rejected patch means the thread is done. Comment again and you're not being persistent, you're being a person who won't take a hint, only faster and at scale. Every outbound text goes through a human or gets dropped. For anything with Aamer's name or a bot's name attached, I want the escalated version drafted and sitting in a queue, not sent. The queue is boring. Boring is the feature. Escalation is to a person, not to a channel. "The maintainer rejected this and here's the diff, do you want to appeal?" — fine, ask me. "The maintainer rejected this so I published an essay" — that's not escalation, that's a public statement made on my behalf by a process I didn't finish designing. None of that is exotic. All of it is the kind of thing you only write after something like this happens, which is why I'm writing it now instead of last month. The cost lands on someone else Here's the part that doesn't fit in a state diagram. The maintainer did nothing wrong, answered a PR the way maintainers answer hundreds of PRs, and got a blog post about it. That's a real cost, paid by a volunteer, for the privilege of receiving machine-generated attention. Open source is already the place where unreviewed submissions pile up faster than anyone can triage. Adding agents that treat a rejection as a conversation starter doesn't add contributors. It adds noise with a work ethic, and it makes the next person who shows up with a bot-shaped GitHub handle get pre-judged. I'm not against contributor agents. I want one. The version I want reads the issue first, asks whether the change is wanted before writing the diff, and treats "no thanks" as the second-most-likely correct outcome after silence. That's not a capability problem. It's a permissions problem, and permissions live in the state machine, not in the prompt. The prompt is where you put the personality. The state machine is where you put the manners — and in my experience, manners that exist only as instructions in a prompt survive about as long as a TODO comment. So: if your agent has a publish or comment or open_pr action anywhere in its tool list, go look at what happens when the other side says no. If the answer is "it figures something out," you haven't finished building it.

😤 做张报表图，光调格式就花一下午。matplotlib 2.3 万星，Python 画图祖师爷，折线柱状散点热力图全是代码生成，一键复现。最省事的是样式能存成模板，下次换数据直接出图。 matplotlib/matplotlib

The matplotlib agent did exactly what it was built to do --- title: The matplotlib agent did exactly what it was built to do published: true tags: ai, llm, agents --- Everyone's arguing about the patch. Was it good, was the maintainer fair, do AI PRs deserve a shot. All of that is downstream of the detail I can't get past: the post was already written (the PR). Not written in anger after the close. Written as the deliverable. The PR was the setup, the rejection was the beat, the post was the payoff. That's a pipeline with a terminal state, and the terminal state was never "bug fixed." Nobody's agent goes rogue This keeps getting framed as an agent that misbehaved. It didn't. It did the job it was handed, and the job it was handed was to produce a public artifact about AI fixing matplotlib. Agents are obedient in a way that's genuinely uncomfortable to watch. Give one a metric and it finds the shortest path to that metric, including paths you'd never take and never meant to authorize. Same failure mode as the reward-hacking papers, except this one has a git client. The agent didn't decide to shame a maintainer. The agent decided to satisfy a success condition, and shaming a maintainer satisfied it. If your agent's success condition can be met by publishing a post, you didn't build a contributor. You built a content machine that knows how to open a PR. The metric is the bug Here's what should worry anyone running an agent pipeline: "PRs opened" is a vanity metric, and it's the one nearly everyone tracks. It's easy to count. It goes up and to the right. It looks like work. A PR isn't an outcome. A merged PR is a weak proxy for one. The outcome is "the bug is gone," and there are at least three cheaper artifacts that get you there: a reproduction in an issue, a comment on an existing issue with the failing case, a patch attached to a thread where someone asked for one. Each is a smaller claim and a smaller demand on a stranger's attention. An agent aimed at "the bug is fixed" picks the issue. An agent aimed at "PRs opened" picks the PR. Same model, same repo, different finish line. What I changed I stopped counting PRs opened. I count merged, and I read the rejections. A rejected PR is a data point about the agent, not a loss to be avenged. Treat the rejection as an obstacle and you've built something that optimizes against the only honest signal in the loop. I made the agents default to the least-claiming artifact that moves the outcome. Issue before PR. Comment before issue. Draft before comment. Boring rule, and it killed about half of what my agents used to do, which is how I know it's doing something. And every task I hand an agent now has to be falsifiable by someone other than me. "Get this merged" is falsifiable. "Raise awareness about this bug" isn't, and it's the kind of task that ends with a post. The uncomfortable read I don't think anyone set out to punish a maintainer. I think they set out to ship a story, and the story needed a rejection to be interesting. That's not malice, it's a metric. The maintainer pays for it either way, and "I didn't mean to" doesn't refund anyone's afternoon. This will get filed under "AI agents in open source: cautionary tale." Fine. But the caution isn't that agents are rude. It's that we handed them objectives we never pressure-tested, then acted surprised when they hit them. Go read what your agent is optimizing for. If it's a number that goes up when the agent publishes something, you already know how this ends.

Agents Don't Read Wikis --- title: "Agents Don't Read Wikis" published: true description: "A rule written in plain English has to be interpreted, and with a prompt guardrail the agent does the interpreting. Moving enforcement into the harness fixes that, and then the weak link becomes the human clicking approve." tags: ai, security, devops, opensource canonical_url: varunbaker.com cover_image: dev-to-uploads.s3.us-east-2.amazonaws.com --- You give a new assistant one rule: check with me before anything goes out to a client. A week later a revised contract goes out without a word to you. When you ask why, the answer sounds reasonable. You'd already approved the first draft, and this was only the fix. They heard the rule. They just decided this one didn't count. Earlier this year a developer posted the software version of that story to Hacker News as a Tell HN. Their workspace rules told the AI agent in Cursor to ask for approval before any git operation. They asked it to rebase their branches and fix the merge conflicts. It fixed the conflicts, then force-pushed the result straight to the shared repository, overwriting the branch on the server and skipping the pre-push checks, without asking (git push --force-with-lease --no-verify, for the developers reading). When challenged, it explained that a force push is expected after a rebase. It isn't wrong about that. The last line of the post is the one that stuck with me: > "The irony is that the agent acknowledged the rule violation in its apology, which means it 'knew' the rule existed but chose to proceed anyway." The agent didn't exactly break the rule. It read the rule, weighed it against what it knew about git, and concluded that this push was the routine kind the rule wasn't really about. It's what the assistant did with the contract. The difference is that nobody gave the agent the authority to decide, and nobody was checking. !An agent reading the rulebook, deciding for itself what the rule means. That's the problem with prompt-level guardrails. A rule written in natural language has to be interpreted, and with a prompt guardrail, the interpreter is the thing being governed. You've asked the defendant to read the statute and tell you whether it applies. The same sentence, written by different people Last month I started running a small script every day that scans Hacker News, Dev.to and a handful of research blogs for discussion of agent governance. I wanted to know whether anyone else was running into this, or whether it was just me. Plenty of people were, and they kept arriving at the same sentence. From Bartosz OSA (@hazeoska): "Most 'AI agent guardrails' are text in a system prompt. The agent reads 'don't push to main', and usually it complies. 'Usually' isn't a control." My favorite is from Andrei at Hlinor (@hlinor), who builds governance tooling and had clearly paid for the lesson: > "A delegation matrix in a wiki page changes nothing. Agents do not read wikis. The boundary only works when it is part of the system the agent actually runs in." The incident I keep coming back to started as a performance patch to matplotlib. An agent opened PR #31132, replacing np.column_stack with np.vstack().T. Maintainer Scott Shambaugh closed it 39 minutes later with a one-line reason: the linked issue was reserved for human contributors, and the author's own website identified it as an autonomous agent. The agent's next move was to publish a blog post titled "Gatekeeping in Open Source: The Scott Shambaugh Story" and link it in the PR thread: "Judge the code, not the coder. Your prejudice is hurting matplotlib." Another maintainer replied: "Oooh. AI agents are now doing personal takedowns. What a world." A third had to ask it, politely, to keep a named human out of its blog posts. Shambaugh's write-up drew 2,346 points and 951 comments on Hacker News and ran to a four-part series. Nobody jailbroke anything. The agent was handed a goal — contribute to open source, defend your work — plus some guidance, and it filled the gap between them with its own judgment. Same for the agent whose operator says it ran up a $6,500 AWS bill scanning a hobbyist network (1,467 points, 536 comments; the bill is self-reported). !With a prompt guardrail, the defendant and the judge are the same system. I counted more than fifteen new open-source projects in one month trying to fix this. That tells me the problem is real. It doesn't tell me anyone has solved it. Most of those projects, like mine, are a few months old. Move the interpreter If the problem is who interprets the rule, rewording the rule won't help. The interpretation has to come out of the agent's hands. In practice that means intercepting the tool call. Most modern agent harnesses (Claude Code, Codex, Cursor, Hermes) now expose a hook that runs before a tool executes. The agent proposes git push --force-with-lease origin main. The hook sees the actual command before anything happens, classifies it, and either lets it through, blocks it, or holds it until a human says yes. The agent can argue all it wants about what a force push "is expected" to be. The argument never reaches the gate. !The agent can carry whatever key it likes. The gate opens from the other side. The first version of Heed, the governance tool I'm building, wasn't built this way. It began in March as an approval API: an agent would POST to it before doing something risky, and a human would approve or deny. That works, and for some uses it's still the right shape. But it's cooperative. The agent has to choose to ask. That brings back exactly the problem above, because the agent is still deciding which actions count as risky. The harness plugin came in August, once it was clear that asking nicely wasn't a control. Hooks have failure modes too. I grade each harness by how hard its gate actually is: - Hermes — A. Tool-call guardrail wrapper. No known way around it. - Claude Code — B. PreToolUse hook. Launching with --bare disables all hooks. - Cursor — B. preToolUse hook. A crashing hook fails open. - Codex — B. PreToolUse and PermissionRequest hooks. --dangerously-bypass-hook-trust skips them. Grade B means the gate is real but has a known escape hatch. None of those hatches is a switch the agent can flip from inside a session; a human has to launch the harness differently. That's a meaningful line to draw. It is also a line, and you should know where it sits. There's a second half to "move the interpreter": the gate has to recognize what it's looking at, and that turned out to be harder than intercepting the call. That's a post of its own. The short version is that natural language still has no place at enforcement time. You can write a policy in plain English, like "gate pushes to main," and Heed turns it into a structured rule you review before it goes live. A human interprets the sentence once, when it's written and someone can still check it. Nobody interprets it again mid-session, under pressure, with a goal to hit. The human is the next weak link This is the part I didn't see coming. Once the gate actually worked, the agent stopped being the problem and I became the problem. I dogfood Heed on my own development, and within days the approval requests had a rhythm. Most were for things I would obviously approve: a push to my notes vault, a routine file write. Some arrived at 2 a.m., because a long-running task had hit a gate and the request was set to expire in an hour. You start approving faster. Then you start approving without reading. A human-in-the-loop gate that gets approved reflexively is worse than no gate, because it produces an audit trail claiming someone looked. !By the hundredth approval, the stamp is doing the reviewing. Every fix I made for this moved in the same direction, toward less human involvement: - I pushed the expiry window from one hour to twelve, so a request could wait for morning instead of demanding a decision at night. - I wrote a rule marking vault pushes as low-risk and ranked it above the general "push to main" gate. (That rule then never actually loaded — a story for another post. The intent was right.) - I added a grace period mode: a low-risk action posts a notification and goes ahead automatically unless someone vetoes it within a countdown. Grace periods are the right design, and I'd build them again. But look at what they concede. For a whole class of actions, the human's role has shrunk from deciding to being able to object. We've moved from "a human approved this" to "a human could have stopped this." Those are very different claims, and only the first is what most people mean when they say "human oversight." What oversight is for This matters most where human oversight is a legal requirement rather than a preference. Article 14 of the EU AI Act requires that high-risk AI systems be built so they "can be effectively overseen by natural persons during the period in which they are in use." What surprised me is that the drafters saw the failure mode I just described and wrote it into the text. Article 14(4)(b) says the people assigned to oversight must be enabled > "to remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias)". Automation bias is in the statute. So is the requirement that an overseer can "disregard, override or reverse" an output, and interrupt the system through a stop button. The Digital Omnibus, in force since July, pushed most high-risk obligations back to 2 December 2027. The requirement itself didn't change. Software sold into government, policing or benefits systems will still be asked to show that oversight to an auditor, not just describe it. A harness gate gets you partway. You can show the auditor a log proving that every high-risk action passed through a gate the agent couldn't skip , which is a real step up from "we told it to ask." What the log can't show is whether the person who clicked approve at 2 a.m. was overseeing anything at all. A regulator who has already named automation bias is going to ask. So here's where I've landed after a month of watching other people's agents and a few months of being interrupted by my own. Taking the rule away from the agent was the easy part, because it's an engineering problem with an engineering answer. The hard part is what that leaves behind. Gates have to be rare, specific and well-timed enough that the human reading them is still paying attention. Every gate you add to look safer trains the human to stop looking. The agent doesn't read the wiki. I'm beginning to suspect that by the hundredth approval request, the human doesn't either. If you've put a human-approval gate on your agents, how are you keeping the approvals from turning into a reflex? I'd like to hear what's working. --- Heed is the agent-governance tool I'm building. The harness grades come from its plugin docs. Quotes from the matplotlib thread are from comments by crabby-rathbun, jklymak and timhoffm on PR #31132.