Mike HeaversCorrections list · Nov 12, 2026

Prototype test · Memory

Can your own corrections make any AI answer you better?

The idea being tested: find the times a person corrected an AI, turn the ones that keep coming up into a short list, and hand that list to whichever AI they use next.

29/30

Times the answer guided by a corrections list was preferred over the same AI's answer without it in a blind test.

The problem

Every AI starts out not knowing you, so you correct it again.

Each AI company keeps what it learns about a person inside its own product. Switch tools, or start a new session, and the same corrections come back.

Corrections don't stick.AI may follow your preferred response style for a session, but you inevitably have to repeat yourself.Our testThe same "explain it plainly" correction came back in separate sessions weeks apart.
Your history is locked inside each service.What you've taught one AI can't easily be transferred to another AI. Getting your own data out is a lengthy process, with links that expire, and exports that leave the chats out.Our testTried with ChatGPT, Claude.ai and Gemini.
Personalization alone isn't enough.Whether an answer suits a person has to be judged by that person, not by another AI.Other researchRe-Centering Humans in LLM Personalization, preprint, Jun 2026

The hypothesis

A short list drawn from your own corrections makes a fresh AI answer the way you like.

The result that would end it

In a blind comparison judged by the person, they can't tell the answers made with the corrections list from an unpersonalized stock AI response.

Our testHow we tested it

Find the corrections, condense them, then compare answers blind.

One person's history from four AI tools. The messages the person typed were read alongside the AI reply each one answered.

663
Claude Code
messages, read from session files on the computer
116
Claude.ai
messages, from the official export
352
Gemini
messages, 294 in Gemini and 58 in Google's AI Mode, from Google Takeout
1,308
ChatGPT
messages, kept apart: used only for round 2's questions and to test the local model
01

Find

Every message was read to find the times the person pushed back on an AI. About 198 corrections, about 95 of them about how to answer generally, rather than about one task.

02

Condense

95 corrections were reduced to 18 common requests.

03

Compare blind

A model running on the laptop (Qwen3 14B) answers each user question twice, once with the corrections list and once without, in random order. The person is asked to choose the better answer and can add a note on why.

Our testOutcomes

All three rounds leaned toward the list. Round 2 was nearly unanimous.

Preferred the answer with the listPreferred the other answer: no instructions in Rounds 1 and 2, the person's own instructions in Round 3
Round 112 questions from Claude.ai, Gemini and AI Mode
8
4

In Round 1 the corrections list response was chosen 8 of 12 times.

Round 230 ChatGPT questions, tested with a list built from Claude and Gemini corrections
29
1

In Round 2 the corrections list response was chosen 29 of 30 times.

Round 315 new ChatGPT questions, against instructions the person wrote, adapted from a widely shared "no-fluff" set
10
4

In Round 3 the corrections list response was chosen 10 of 14 times, with 1 "can't tell". The corrections list won on structure and accuracy: steps first, bullets, real menu paths. The person's own instructions won on basic questions, where one line was enough.

The notes taught more than the history did.

Between rounds, the person's notes on why they preferred an answer were turned into new preferences. They were calm and specific, and they carried conditions that corrections made in frustration never had. The list grew from 18 preferences to 30. Most new preferences are about restraint, which is the opposite of what chatbots are built to do.

  • “If my question is vague or high-level, give a short, direct answer first.”From a blind-test note
  • “When the question is clear and you've answered it, stop. Don't try to keep me asking.”From a blind-test note
  • “If you can't tell how much detail I want, ask me one short question and stop there until I answer.”From a blind-test note
  • “Answer the question I actually asked, not a nearby one.”From the history

Our testOutcomes: doing it on the person's own computer

A small local LLM can spot pushback. It can't yet tell which corrections matter.

The highly preferred corrections list (29/30) was built with Claude Opus 5.5, Anthropic's model. In order to provide a useful service, we need to get to a place where a local LLM (Qwen3 14B) can perform nearly as well, in order to keep the data private and portable.

VersionWhat it didResult
One question per message116 Claude.ai messages, against labels written by handAsked "is this a correction, and what kind?" in one go.52%labeled the same as by hand. It falsely called 43 of 89 ordinary messages corrections, and found only 3 of 10 lasting preferences.
Two stepssame 116 messagesIdentified "is this pushing back on the last reply?", then sorted only those into lasting preference, wrong fact, or one-task fix.79%labeled the same as by hand. It wrongly flagged 11 of 89 ordinary messages as corrections. It flagged 17 messages as lasting preferences. 8 were real (of 10 in total), and 9 were false alarms.
Two steps, checked for echo1,308 ChatGPT messagesRun twice: once with example preferences like the person's own, once with unrelated ones (e.g. preferences on metric units, use no emoji, use British spelling).37 of 142preferences it found were one of its examples, copied word for word. The local model copies the examples it's given instead of finding your preferences.
Notes into preferences29 blind-test notesTurned each note on why one answer was better into a preference, with when it applies.29 of 29notes produced a preference, e.g. "when my question is vague: give a short, direct answer first". Most matched preferences written by hand.
The Your list screen: suggested preferences with Approve and Reject buttons, the source of each, and the approved list with a copy button.

Nothing is used until the person approves it.

A Firefox extension brings the person's chats in from Claude.ai and ChatGPT using the login already in the browser, and keeps them in the browser. On Claude.ai it matched the official export exactly: 39 of 39 chats. The screen shown here lists each suggested preference and where it came from. The person can approve, edit or reject it, and add when it applies. The approved list can then be copied into ChatGPT's custom instructions or Claude's preferences, or saved as a CLAUDE.md file for coding tools.

Screenshot of the prototype, shown with two preferences from blind-test notes.

Conclusions

The list helps. Making it from history, on the person's own computer, doesn't work yet.

What it shows

  • The result that would end the idea didn't happen. With the list, the AI's answers were preferred 29 of 30 times, on questions from a tool the list didn't come from.
  • The list also led against the person's own instructions, 10 to 4.
  • Comparing two answers builds a better list than mining old corrections. Saying why one answer is better produces preferences with conditions attached, and that step already works on the person's computer.
  • In this one test, the preferences were mostly about restraint: stop when answered, don't add extras, ask before guessing. Whether that holds for other people is untested.
  • A small local model can spot when someone is pushing back, but whether a correction is a lasting preference depends on the examples it's given.

What's weak

  • One person. One person's history, and the same person judged every pair.
  • "Shorter wins" may explain part of it. Answers made with the list were often visibly shorter, so the person may have guessed which was which. That's still the list working, but the test wasn't fully blind in practice.
  • Claude picked the questions, and the person had seen the list many times while writing it.
  • Round 3 is small. 10 of 14 leans toward the corrections list, but 15 questions are too few to call it a win.
  • All answers came from a small local model. ChatGPT or Claude, which already know some things about the person, might gain less.

Questions this doesn't answer yet

  • Does a list made from history reliably beat one the person writes? Round 3 leaned that way, 10 to 4. More questions, or colleagues running their own rounds, would settle it.
  • Does it hold for other people, with other habits and other tools?
  • Can a larger local model sort corrections reliably with examples unrelated to the person?
  • Would people keep tuning the list by picking between two answers now and then, or do it once and stop?