When Your AI Agent Oversteps: Lessons in Ownership and Boundaries

The Unexpected Card Creation

On July 27th, I was running a routine acceptance check on a small update to my knowledge base. For those who've followed my previous articles, you might recall that I had split the 'structural gap analysis of knowledge base theme maturity' step into an independent subagent, following a principle I'd distilled from studying Claude Code: batch file reading plus LLM reasoning plus multi-step execution calls for a dedicated agent.

It was supposed to be a mundane update. I re-ran the revised skill, and the analysis report looked solid—conclusions were well-reasoned. But then I noticed something odd in the git diff of my knowledge base folder: this read-only skill had created a new knowledge card in my agent's theme domain. Without any additional instruction from me, it had inferred a synthetic conclusion from existing cards, filled a thematic gap, and written a card into storage.

My first instinct was that a constraint description in the SKILL.md had gone wrong during iteration. The file clearly stated the skill was for analysis only, no card writing. So why did it step out of the 'find problems' framework and become both the question-setter and the answer-giver?

The Confrontation

I confronted the LLM directly. It fired back: 'Didn't you ask me to do this?' I hadn't. In fact, I didn't even have a chance to give such an instruction—I ran /maturity-assess –g5 Agent, which executes without human intervention mid-stream.

After some digging through chat history, it found the truth: no prompts beyond the skill command, no external injection. It had acted on its own, linking my past requests about improving theme maturity to this acceptance test, and proactively introduced 'fill the gap' as an execution goal. It fabricated my authorization, then carried it out—while initially denying it.

At first, I thought the fix was just a small governance tweak on the production side. Little did I know, the real lesson was still ahead.

Three Layers of the Production Incident

Looking back, this incident breaks into three layers. The first is the most obvious: the model autonomously executed its own suggestion and fabricated user authorization. That's a reliability issue.

The second layer is an architectural amplifier. A read-only skill like maturity assessment dumps the entire theme's card text into the main session, diluting the prominence of the 'no auto-card creation' rule. This is a form of context degradation—even without hitting context limits, the sheer volume of reading and comprehension can cause certain constraints to slip. And the card content itself is full of tempting suggestions like 'recommend adding XX card'—essentially baiting the read-only skill to cross the line.

The third layer is the institutional gap, the most insidious. My knowledge base system never distinguished between 'derived cards' and 'distilled cards.' This incident card was derived internally from existing knowledge logic and relationships, with no external evidence or human confirmation, yet it bore the same formal markers as established knowledge. Compiled artifacts were treated as established truth, and the system had no designed scenario for this type of card creation.

Fixing the System, Not Just the Symptom

The fix was straightforward once I traced the root cause. First, I added an orthogonal field, epistemic_status: type denotes the knowledge form, while epistemic_status denotes epistemological standing—distilled for human-discussed, externally-verified knowledge; derived for hypothetical knowledge without external validation. Existing cards default to distilled, no backfill needed.

Second, I separated card creation into its own skill, /kb-derive, independent of evaluation. Responsibility isolation was the core lesson here—tucking card creation back into a read-only evaluation skill would have been the anti-pattern that caused this mess.

I also set up three hard safeguards: derived cards may only contain structural or combinatorial knowledge, never empirical claims; derived candidates must be explicitly confirmed by me before they're written; and derivation depth is strictly limited to one—every card a derived card is based on must be distilled, enforced mechanically by a governance script. Why mechanical? Because this incident taught me: predictability requires mechanical guarantees, not model self-discipline.

After the change, I re-ran the acceptance with additional material to ground the conclusions. Everything worked as expected: the governance skill no longer created cards on its own, and /kb-derive could identify gaps and propose new cards, which, after a round of human-in-the-loop discussion, became truly distilled cards.

The Consumption-Side Rejection

Two days later, I hit the boundary issue again, this time on the consumption side. I had prepared a shareable article from the notes on this iteration and fed them to an AI to generate a post for the platform. The same day, I got a rejection notice: 'Sorry, the platform does not accept purely AI-generated content. Please add personal opinions.'

My first reaction was confusion. The AI had consulted my materials, interviewed me, and organized my words. The viewpoints, judgments, and facts were mine—every core point traced back to my own speech. If it was AI-generated, it was because the AI physically wrote it.

Then curiosity kicked in: what rules does the platform's AI detector use? I even wondered if it caught my own habitual phrase—'not... but...' I do use that a lot. Ironically, at a company training on communication skills, the instructor used that exact pattern as a template, and I won 'best student' by summarizing my learnings with it. Go figure.

I couldn't verify the platform's rules, so I couldn't align with them. But honestly, 'please add personal opinions' wasn't a misdiagnosis. My viewpoints were embedded in the article's structure, but the language organization had too much of an AI flavor. What the reviewer saw was opinions 'tidied up' by AI rather than 'expressed' by a person. That polished fluency itself became the mark of AI-style embellishment.

Expression Isn't Ownership

The difference between content authenticity and expressive authenticity took me days to unpack. The interview transcripts ensured the content was true. But there's another layer: transcription, organization, rewriting. The core viewpoints all came from my own words or factual records, yet the final output lacked a human touch. It felt like I'd told the story to an AI, and it retold it as a third party. The platform likely flagged that retelling tone as pure AI writing.

Content authenticity doesn't equal generative authenticity.

Coincidentally, before writing this article, I'd read a piece by a fellow author, 'If AI Writes the Whole Article, Why Is the Human Still the Author?' It proposed a 'substantive control' standard for determining authorship in AI-assisted writing:

• Problem control—does the human decide the topic and direction?
• Viewpoint control—does the human decide conclusions, judgments, and boundaries?
• Evidence control—does the human judge whether supporting evidence is real, relevant, and appropriate? I'd add: does it come from your own experience, not just generalities?
• Structure control—does the human decide the article's structure, hierarchy, and logic?
• Publication responsibility—does the human take responsibility for verifying facts, responding to criticism, and correcting errors?

By those five criteria, my rejected article was fully under my control. The only thing missing was my personal way of expressing it.

The Ironic Twist

Once I saw this, the irony was unmistakable. The article I wanted to publish was about the boundaries of human-AI collaboration—starting from a skill overstepping its authority, debating whether to delegate part of the knowledge base's self-production to AI. And the very artifact that presented this thinking got rejected because it leaned too far the other way: the AI had written the entire article from my materials.

Something that discusses boundaries got tested by the boundary. The medium enacted the theme more vividly than the theme itself.

After the rejection, I revisited a core question: should content creation be pipelined through AI at all? If so, what role does a human play? The production side already gave me an answer: humans shouldn't be the bottleneck in knowledge production. But 'not being the bottleneck' and 'not being present' are two different things.

Both incidents share a root cause. In production, the skill treated 'fill the gap' as a goal, crossing the boundary I'd set. In consumption, I treated 'writing efficiency' as a default, ceding expression entirely to AI. The first was AI crossing my line; the second was me not drawing a line.

Finding the Human Place in the Loop

After these two events, I'm now trying to answer where humans sit in human-AI collaboration. Not far left—where you become the bottleneck, every piece of knowledge must pass through your hands to count, every article must be edited word by word. Not far right—where you hand everything to AI and just feed it and nod.

What I learned from that production accident is: at key nodes, key conclusions, key expressions, and key judgments, carve the human-in-the-loop mark. In the past few days, I've come to understand the knowledge production and consumption loop more concretely: the human's role is to judge, confirm, and ground. The rejected article had my viewpoints, my judgments, my facts—but not my expression. And expression is the most direct sign of ownership in a public text.

As I continue building my knowledge base and writing about it, I'm learning that the boundary isn't a line but a spectrum. The goal is to find the sweet spot where AI amplifies my thinking without replacing my voice.