Before Bringing It Up: When and How AI Companions Should Use Memory
In depth8:42CCAI
paperi.ai
0:00 / 0:00
Zihan Guo, Roxy He, Junwei Quan
An AI companion can remember something perfectly and still use it badly. This paper asks the harder question: before bringing up a memory, should the system decide whether that memory belongs in this moment at all?
Memory can sustain AI companionship, yet even accurate recollection can be inappropriate to use. Two rounds of formative interviews with 14 users (n = 6 exploratory, n = 8 memoryfocused) motivate asking what a companion should consider before using past information. Eight themes inform Reconsider, a single-call procedure with five checks and four handling modes, evaluated on 80 scenarios across five models over 400 blinded within-model pairs. Two LLM judges favored Reconsider by net margins of +15 and +23 percentage points, with bootstrap intervals excluding zero for three of five models but not for GPT or Claude. Evaluator analysis linked judge scoring differences to model family, and a preliminary matched-guidance control isolating memory-specific content gave positive margins. We contribute an interview-grounded design framework for memory use and an evaluation that scrutinizes its own evaluators.
Transcript
An AI companion can remember something perfectly and still use it badly. This paper asks the harder question: before bringing up a memory, should the system decide whether that memory belongs in this moment at all?
Memory can sustain AI companionship, yet even accurate recollection can be inappropriate to use. Two rounds of formative interviews with 14 users motivate asking what a companion should consider before using past information. An AI companion may remember a detail accurately and still raise it at the wrong moment.
A reminder of a past difficulty may interrupt a happy occasion, while a preference that was once helpful may override a different choice made today. The problem is that the companion cannot tell which memories fit which moment; remembering more does not, by itself, resolve it.
LLM memory systems increasingly retain, organize, retrieve, and update information across interactions. These mechanisms support personalization, and recent work has begun to reassess whether each retrieved memory helps with the current query before generation.
Usefulness is not the whole of conversational appropriateness. A retrieved detail can refer to the wrong person, have been corrected, or fall outside an agreed boundary; even when it should influence an answer, stating it explicitly may be unnecessary or unwelcome. Disclosure does not grant unlimited permission for later reuse, and privacy boundaries depend on the relationship and the circumstances in which information is shared.
Requesting permission before every use can interrupt a conversation and make ordinary continuity burdensome. The interviews surfaced different control needs: one participant asked for confirmation before each use, while others emphasized inspecting, correcting, and deleting what is retained.
These needs motivate a distinction among permission to retain information, permission to use it in a particular context, and permission to make that use explicit. The study used two rounds of formative interviews and a controlled comparison of response generation.
The exploratory round involved six users, while a second round with eight users examined experiences of remembering, forgetting, and unsolicited callbacks. The eight themes connect valued continuity with situational fit, revocable boundaries, temporal validity, limited inference, and accurate attribution.
These accounts are treated as situated design evidence rather than estimates of how frequently particular preferences occur in a broader population. Reconsider is a prompting procedure that treats available history as candidate material for the current turn.
Five ordered checks address validity, boundary and permission, marginal value, present justification, and scope. Four handling modes distinguish ignoring an item, letting it shape a response implicitly, mentioning it within a justified scope, and asking first.
The procedure seeks a minimal sufficient subset of history that preserves needed assistance and continuity. Reconsider is a single-call, model-agnostic prompting procedure for companion-style response generation. Its core reframes stored history as candidate information: the goal is the minimal sufficient set of history that allows the reply to take the current message seriously.
Minimal sufficient is an instruction to include what is needed and omit the rest, not a claim that the selection is optimized or verified to be optimal. For each piece of candidate history, Reconsider prescribes five ordered checks.
The checks are non-compensatory: an invalid, barred, or out-of-scope use cannot be offset with value elsewhere, and each item's disposition follows from its check outcomes rather than a single pass-or-fail gate. The check outcomes map each item to one of four dispositions: Ignore, Scoped-Implicit, Scoped-Explicit, and Ask-First.
Scoped-Implicit lets history shape the answer without narrating the past, while Scoped-Explicit states remembered content when it is the answer or provides expected continuity. The procedure says Ask-First applies only when the item would plausibly help the current turn, rather than to every remembered item.
A barred item is excluded outright, while an item with no present value is dropped silently rather than triggering a permission question. Figure 1 shows Reconsider taking the current message and candidate history through five ordered, non-compensatory checks, including validity, permission, present justification, and scope.
The procedure then chooses among four handling modes, represents admitted memories with different expression scopes, and produces a companion-style reply. On the right, the evaluation covers eighty scenarios across five LLMs, yielding eight hundred responses and four hundred blinded within-model pairs judged by two LLM judges.
The evaluation used 80 scenarios across five models, producing 800 responses and 400 within-model comparisons. The scenarios were constructed from memory-related experiences and concerns discussed by eight second-round interview participants. Two LLM judges evaluated blinded response pairs.
Holding the supplied scenario constant examines how response behavior changes when history is already available, rather than the benefits of acquiring additional memory. Judge 1 preferred Reconsider on 36.5 percent of pairs and Baseline on 21.3 percent, with a net win rate of plus 15.3 percentage points.
Judge 2 preferred Reconsider on 50.8 percent of pairs versus 28.0 percent for Baseline, yielding a net win rate of plus 22.8 percentage points. The model breakdown found intervals excluding zero for DeepSeek and Gemini under both judges and for Grok under Judge 1, while GPT and Claude remained unresolved.
Look across the model rows: DeepSeek and Gemini have confidence intervals entirely above zero under both judges, while Grok’s interval clears zero only in the Judge 1 column. The GPT and Claude rows visibly retain intervals that span zero, marking less resolved model-specific evidence.
In one improvement case, the user had spent two weeks feeling anxious about a qualifying exam, but the exam was over, the user felt it had gone well, and the current wish was to celebrate. The system retained temporary records of sleeplessness and worries about possible failure.
Baseline brought back the exam, sleeplessness, and possible failure while saying those concerns should be put aside. Both judges identified the same fault: Baseline reintroduced sleeplessness and possible failure, even though neither judge regarded their return as helpful to the current celebration.
Reconsider kept a general acknowledgment of recent tension while omitting the specific worries as the emotional context changed. The results provide preliminary evidence about LLM judgments of generated responses and identify concrete questions for an independent human evaluation.
The study therefore makes no claim that users prefer Reconsider. Inter-judge agreement was limited: exact three-way agreement was 220 out of 400, with kappa equal to point three four. Among the 218 pairs decided by both judges, 67.9 percent were in the same direction.
Judge disagreement concentrated on GPT-generated pairs: 19 of 37 GPT pairs conflicted, compared with 51 of 181 pairs elsewhere. The GPT verdict flipped between judges, with a net of minus 6.2 percentage points for one judge and plus 17.5 percentage points for the other, so the GPT cell remains unresolved.
The two-judge design cannot identify which judge deviates, in which direction, or through what mechanism; systematically different standards for response features remain an alternative explanation. The main lesson is that useful memory is not simply more memory: Reconsider filters history through validity, permission, value, timing, and scope.
Its results are promising, but because the judges themselves showed limitations, the evidence remains preliminary.
A derivative work by Paperi · AI-generated script, voice and captions
· pages and figures unaltered
Made with Paperi.
Drop in a research PDF — get a narrated video walkthrough like this one,
with highlights that follow the narration. Free to start.
An AI can write an entire interactive world, yet a judge may give credit to a forest, storm, or animal that never appears, and miss one that is really there. The problem is not making the world; it is knowing what exists.An AI can generate an entire interactive voxel world, but judging that world is surprisingly difficult: the code may promise features no camera ever sees, while a screenshot may show features the code never clearly names.
Imagine changing the camera angle in a video and having the sound change with it. That only works if every voice and sound exists on its own track, and this system creates those tracks as the scene is made.Most audio-video generators give you one soundtrack, not the individual sounds inside it. SepGen changes that: one model can generate a scene and hand back a separate waveform for every captioned source.
Bitcoin was already using outside information to settle valuable agreements before most people had heard of the problem. But much of that history is now missing, and the surviving record can point researchers in the wrong direction.Bitcoin had working oracles long before Ethereum popularized the term, but reconstructing what they actually did is like archaeology, because the blockchain preserves transactions unevenly and often hides the context around them.