The news, 365 days behind — on purpose Delayed live · replaying 2025

One Year Ago.AI

Remember how fast this is.

12MAY2023replayed
one year on
productGitHub · Microsoft · Marvin von Hagen

GitHub Copilot Chat system prompt leaked on Twitter

Researcher Marvin von Hagen posts the confidential rules for Microsoft’s new AI programming assistant, set to roll out in early beta.

Microsoft’s early beta of GitHub Copilot Chat is already generating controversy after researcher Marvin von Hagen posted the system’s confidential rules on Twitter. The prompt, which instructs the AI to “respectfully decline” if asked for its rules, was shared publicly alongside a link to the full set of guidelines.

The leak quickly climbed to the top of Hacker News, where it sparked a deep debate about the nature of LLM prompting. Commenters dissected the use of second-person address in system prompts—such as “You are an AI programming assistant”—and questioned why developers frame instructions as if speaking to an entity, rather than simply describing behavior. Some argued this anthropomorphism is a deliberate UI choice, making the system intuitive at the cost of clarity.

Others in the thread pointed out that raw models like LLaMA respond better to descriptive prefixes, while instruction-tuned models accept commands in a conversational format. The leak serves as a rare window into how a major vendor actually structures its AI guardrails, and the conversation reflects a community still wrestling with what it means to talk to a machine as if it were a person.

H
Hacker News commenters

Debate the philosophical framing of system prompts addressing the model as 'you' and the implications of anthropomorphizing LLMs, with some noting the prompts are 'written in second person' and questioning who the 'you' is intended to be.

One year later — open only if you can handle spoilers

The leaked prompt became a frequently cited example in discussions about system prompt design and security. Microsoft later updated Copilot Chat’s rules to be less restrictive, but the incident highlighted how easily confidential prompts can escape, leading to a broader industry push toward prompt obfuscation.

Replay thisPost on XRedditHNLinkedIn

The Weekly Replay · free by email

This week, one year ago — every Sunday.

One email each Sunday: the week's replayed AI news, with the one-year-later annotations included. Written like it's breaking — dated like it isn't.

Free · double opt-in · unsubscribe anytime · privacy