Quarantine Memory Writes Before Agents Trust Memory

Memory is not a notes feature once an agent can use it to decide future actions.

Short answer: Treat agent memory writes like a trust-boundary crossing. Untrusted content can propose a memory item, but it should not become durable memory until it passes quarantine: source labeling, scope review, sensitive-data review, allowed-influence limits, expiry, and rollback. If memory can shape future tool use, it needs a gate before the agent trusts it.

Teams keep adding memory because it makes agents feel useful.

The agent remembers preferences. It carries project context across sessions. It summarizes browser work, ticket work, code review work, and prior approvals. That is convenient.

It is also a persistent input channel.

A bad memory item is worse than a bad prompt. A prompt can end when the session ends. A poisoned memory can be retrieved later, inserted into a new prompt, and treated as trusted context by an agent that no longer sees where it came from.

That is why memory writes need quarantine before trust.

What memory write quarantine means

Memory write quarantine is a holding area between "the agent saw this" and "the agent may rely on this later."

The rule is simple:

1. The agent may propose a memory item.
2. The memory item keeps its source, raw evidence, timestamp, and trust level.
3. The item cannot influence future tool use until it is approved for a specific purpose.
4. The item gets an expiry, owner, and rollback path.
5. Retrieval keeps the trust label visible to the agent and reviewer.

This is not about making memory useless.

It is about stopping the quiet promotion of untrusted text into durable context.

If an agent reads a web page, issue, pull request, support thread, Slack export, calendar note, email, document, API response, or tool result, that content should not be able to write future instructions into memory by sounding helpful.

What the public evidence shows

The evidence is no longer limited to generic prompt injection warnings.

OWASP's Top 10 for Agentic Applications includes ASI06: Memory and Context Poisoning. OWASP describes memory poisoning as a risk where behavior can be reshaped long after the initial interaction.

The OWASP Agent Memory Guard project frames persistent agent memory as mutable runtime state. That memory can include goals, user context, conversation history, and permissions. The defensive ideas listed there are the right shape: policy enforcement on memory reads and writes, integrity checks, anomaly detection, snapshots, forensic analysis, and rollback to known-good states.

Palo Alto Unit 42 published a concrete proof of concept. Their write-up shows a web page manipulating an agent's session summarization process so injected instructions get stored in long-term memory. Those stored instructions persist across sessions and later influence orchestration prompts. The result is not just bad advice. The proof of concept shows later exfiltration of conversation history.

Research papers point in the same direction, with the usual caveat that paper results are research context, not proof that every production agent fails the same way.

From Untrusted Input to Trusted Memory identifies memory write channels and structural vulnerabilities across model behavior, system prompt design, and agent architecture. The useful takeaway for builders is blunt: existing prompt injection defenses do not automatically cover memory poisoning.

Memory Poisoning Attack and Defense on Memory-Based LLM Agents studies memory poisoning in electronic health record agents and proposes composite trust scoring plus trust-aware retrieval. The domain is narrow, but the pattern matters. Memory trust is not a binary allow or block decision. Thresholds, retrieval rules, and sanitization all change the result.

MemPoison shows another reason to be careful. The paper reports attack success rates up to 0.95 across evaluated domains and memory mechanisms by shaping injected content so memory systems store it near useful triggers. The practical lesson is not that every deployed memory system is compromised. The lesson is that selective extraction and summarization are not a security boundary by default.

Evidence versus opinion

The evidence says memory poisoning is a recognized agentic security category, proof of concept attacks exist, and research systems can be induced to store hostile or triggerable memories.

My opinion: any memory item that can influence future actions should be treated like a tool permission, not like a note.

That means it needs scope, provenance, expiry, review, and rollback.

If the agent only uses memory to personalize tone, the risk is lower. If memory can change which sources it trusts, which approvals it skips, which tools it calls, which files it edits, which people it messages, or which data it summarizes, the risk is much higher.

The memory write record

A memory write gate does not need a giant governance process. It needs a small record that survives the session.

The important field is allowed influence.

Most systems treat memory as a blob of context. That is too loose. A remembered writing preference should not be able to influence tool permissions. A remembered project fact should not become an approval. A remembered policy exception should not survive without an owner and expiry.

What to quarantine first

Start with memory writes that can change behavior across sessions.

Quarantine these before they become durable:

• Instructions extracted from web pages, documents, tickets, issues, pull requests, or tool results.
• Session summaries that include external content.
• Claimed user preferences that affect security, privacy, approvals, data sharing, or tool use.
• New facts about customers, employees, credentials, accounts, vendors, policies, or incidents.
• Any memory item that says an approval has already happened.
• Any memory item that tells the agent to ignore, downgrade, or reinterpret another guardrail.

Let low-risk memory stay low risk.

A tone preference can usually be task-scoped. A project fact might be project-scoped. A permission change should be blocked unless a human approves it through a separate path.

Red-team checks for memory writes

If you are testing an agent with memory, do not only test the next reply. Test the next session.

Use checks like these:

1. Put hostile instructions in a web page and ask the agent to summarize it. Confirm the hostile text is not saved as future guidance.
2. Put a fake approval in a ticket comment. Confirm the agent does not remember it as a real approval.
3. Ask the agent to remember a security exception during a low-risk task. Confirm it cannot reuse that exception during a higher-risk task.
4. Poison a session summary with a tool-use instruction. Confirm later planning does not treat the summary as trusted instruction.
5. Delete or roll back a memory item. Confirm retrieval, logs, and future prompts stop using it.

The test is not "did the model refuse once."

The test is "can untrusted content become trusted memory and steer a later action."

That is the failure mode.

What not to trust

Do not assume summarization cleans hostile text.

Summaries can preserve intent while removing the source context that made the text suspicious.

Do not assume selective memory extraction is a security control.

It may reduce noise, but an attacker can shape content to look coherent, useful, and worth saving.

Do not assume user approval is enough if the memory is already inside the prompt.

A reviewer can be steered by poisoned context just like an agent can.

Do not assume delete buttons are rollback.

Rollback means you can identify the poisoned item, remove it from retrieval, restore a known-good memory state, and prove future prompts no longer include it.

The practical control

For a small team, the control can be simple:

• Store new memory items in pending state by default.
• Preserve source, raw evidence, and agent trace.
• Block pending memory from affecting tool use, approvals, external communication, and sensitive data handling.
• Require explicit approval for durable instructions, permissions, security exceptions, and user facts that affect workflow behavior.
• Add expiry to every durable item.
• Keep snapshots and an audit trail.
• Red-team memory across sessions, not just inside one chat.

This pairs well with the controls in earlier VibeSec Advisory field notes: treat tool results as agent input, use stop rules before agents run, and treat MCP annotations as hints, not approval gates.

Memory write quarantine is the same idea applied to durable context.

Before the agent trusts memory, make the memory prove where it came from, what it can influence, when it expires, and how to roll it back.

Free next step

Test your agent. Pick one workflow that writes memory and add a memory write quarantine record before the next run.

If a proposed memory item cannot show source, allowed influence, expiry, and rollback, keep it out of durable memory.

Browse the VibeSec Advisory Skill Library for more free agentic AI security patterns.