Your assistant is sharp for one session and blank the next time the process restarts. The usual patch is to paste the whole transcript into every prompt, which holds up until the transcript reaches 60k or 80k tokens. Past that point you pay for the same history on every turn, latency climbs, and the model starts losing details buried in the middle of the context window.
A separate memory layer fixes the shape of the problem instead of the symptom. Mem0 is an open source memory service for LLM apps: you hand it conversations, it extracts the facts worth keeping, and at query time you pull back only the handful of memories relevant to the current question.
This walkthrough runs the entire stack on your own machine. Mem0 open source as a Python library, Qdrant for vectors, Ollama for both the extraction LLM and the embedding model. No API keys, no per-token bill, and nothing leaving the box.
What you need
- Python 3.10 or newer, since
mem0airequires>=3.10, <4.0 - Docker, for Qdrant
- Ollama installed and running (
ollama --version) - A few GB of free disk for the Qdrant volume plus the models you pull
Step 1: run Qdrant
docker run -d --name qdrant -p 6333:6333 -p 6334:6334 \
-v "$(pwd)/qdrant_storage:/qdrant/storage:z" \
qdrant/qdrant
Three ports and paths matter here. The REST API answers on localhost:6333, the dashboard lives at localhost:6333/dashboard, and the gRPC endpoint is on 6334. Mem0 talks over REST by default, so 6333 is the one you configure.
Step 2: pull the Ollama models
ollama pull llama3.1:latest
ollama pull nomic-embed-text:latest
llama3.1 does the extraction work, meaning it reads a conversation and decides which statements become memories. nomic-embed-text turns each memory into a 768-dimension vector. Hold on to that number, because a mismatch between the embedder's output size and the collection's expected size is the error you are most likely to hit.
Step 3: install Mem0
python -m venv .venv && source .venv/bin/activate
pip install mem0ai qdrant-client openai
If your searches involve exact identifiers like order numbers, hostnames, or error codes, install the extra that enables BM25 keyword matching and entity extraction:
pip install "mem0ai[nlp]"
python -m spacy download en_core_web_sm
Paraphrase-style queries work without it. Keyword-exact queries do not.
Step 4: configure the memory instance
from mem0 import Memory
OLLAMA = "http://localhost:11434"
CUSTOM_INSTRUCTIONS = """
Extract only durable facts from support conversations:
- Customer identity and contact details
- Product, plan, or order identifiers
- Stated preferences and constraints
- Problems already reported and their current status
Exclude greetings, small talk, and anything hypothetical.
Input: hi
Output: {"facts": []}
Input: My order #A-4471 still shows as packing after nine days.
Output: {"facts": ["Order #A-4471 stuck in packing status after nine days"]}
Return JSON with a single key "facts" as a list of strings.
"""
config = {
"vector_store": {
"provider": "qdrant",
"config": {
"collection_name": "agent_memory",
"host": "localhost",
"port": 6333,
"embedding_model_dims": 768,
},
},
"llm": {
"provider": "ollama",
"config": {
"model": "llama3.1:latest",
"temperature": 0,
"max_tokens": 2000,
"ollama_base_url": OLLAMA,
},
},
"embedder": {
"provider": "ollama",
"config": {
"model": "nomic-embed-text:latest",
"ollama_base_url": OLLAMA,
},
},
"custom_instructions": CUSTOM_INSTRUCTIONS,
}
memory = Memory.from_config(config)
Three details in that config are easy to get wrong.
custom_instructions is a top-level key, not part of the llm block, and it has to be set before Memory.from_config() runs. Older scripts call it custom_fact_extraction_prompt; the parameter was renamed, so update the name if you are porting something that used to work. Whatever prompt you write, the extractor expects JSON back with a facts array.
embedding_model_dims must equal what the embedder actually outputs. nomic-embed-text gives 768. Switch to an OpenAI embedder and the default text-embedding-3-small gives 1536, which means a fresh collection name, because Qdrant rejects vectors of the wrong size for an existing collection.
It helps to know what you get with no config at all. A bare Memory() uses OpenAI gpt-5-mini for extraction, text-embedding-3-small for embeddings, stores vectors in a Qdrant instance on disk at /tmp/qdrant, and keeps operation history in SQLite at ~/.mem0/history.db. That path wants OPENAI_API_KEY set even when you believe you are running locally.
Step 5: the retrieve, answer, store loop
from openai import OpenAI
# Ollama serves an OpenAI-compatible endpoint at /v1
chat = OpenAI(base_url=f"{OLLAMA}/v1", api_key="ollama")
def ask(user_input, user_id):
hits = memory.search(user_input, filters={"user_id": user_id}, top_k=5)
facts = [h["memory"] for h in hits["results"]]
context = "\n".join(facts) if facts else "(nothing stored yet)"
reply = chat.chat.completions.create(
model="llama3.1:latest",
messages=[
{
"role": "system",
"content": f"You are a support agent. Known facts about this user:\n{context}",
},
{"role": "user", "content": user_input},
],
).choices[0].message.content
memory.add(
[
{"role": "user", "content": user_input},
{"role": "assistant", "content": reply},
],
user_id=user_id,
)
return reply
Both search and get_all return a dict with a results key, and each item carries id, memory, score, and created_at. Loop over hits["results"] rather than the response itself, or you will spend ten minutes chasing a string-index error.
Step 6: prove that it persists
ask("My order #A-4471 still hasn't shipped", "dewi")
# ... stop the process, start it again ...
print(ask("Any update on my order?", "dewi"))
stored = memory.get_all(filters={"user_id": "dewi"})
for m in stored["results"]:
print(m["id"], "|", m["memory"])
The second call has no conversation history in the prompt and still answers with the order number, because it came back out of Qdrant. The dump at the end is the step people skip and then regret. With no custom_instructions, that list quietly fills up with entries like "hey" and "thanks", and retrieval quality drops. Run it after every edit to your extraction prompt.
When you already know a fact and want it stored as written, skip the extraction LLM entirely:
memory.add(
[{"role": "user", "content": "Customer is on the Enterprise plan, invoice by email."}],
user_id="dewi",
infer=False,
)
infer=False writes the text verbatim. No extraction call, no rewording, no cost.
Step 7: organize memories with metadata
Mem0 open source has no category system. A category is just an add-time metadata field you define, and filters only see keys you actually wrote.
memory.add(
[{"role": "user", "content": "Prefers invoice by email, not paper."}],
user_id="dewi",
metadata={"bucket": "preferences"},
)
prefs = memory.search(
"billing preferences",
filters={"user_id": "dewi", "bucket": "preferences"},
)
Keep it to two or three buckets at the start. Every extra key you filter on is another key you have to remember to set on every write.
Separate agents that share one user use agent_id:
memory.add(messages, user_id="dewi", agent_id="billing_bot")
memory.search("escalation rules", filters={"agent_id": "billing_bot"})
One trap with date filters: Qdrant range conditions only apply to numeric payload fields, so a string date written into metadata will never match a {"gte": ...} filter. Store an epoch integer instead.
import time
memory.add(
[{"role": "user", "content": "Reported a duplicate charge on the last invoice."}],
user_id="dewi",
metadata={"logged_epoch": int(time.time())},
)
cutoff = int(time.time()) - 30 * 24 * 3600
recent = memory.search(
"billing problems",
filters={"user_id": "dewi", "logged_epoch": {"gte": cutoff}},
)
Step 8: expire the facts that should expire
Mem0 open source never deletes anything on its own. The documented pattern is to store an expiry alongside the memory and prune on a schedule. Epoch seconds again, so the range filter works:
WEEK = 7 * 24 * 3600
memory.add(
[{"role": "user", "content": "Card was declined, retrying next week."}],
user_id="dewi",
metadata={"bucket": "constraints", "expires_at": int(time.time()) + WEEK},
)
def prune(user_id):
now = int(time.time())
stale = memory.get_all(filters={"user_id": user_id, "expires_at": {"lt": now}})
for m in stale["results"]:
memory.delete(memory_id=m["id"])
return len(stale["results"])
print("removed", prune("dewi"), "expired memories")
Run it from cron or from whatever scheduler you already have. Wire it up before launch, not after the first stale memory causes a wrong answer to a real customer.
Errors you will probably hit
Invalid input, expected vector size 768, got 1536 means the embedder and embedding_model_dims disagree. Fix the number, or point at a new collection name.
OPENAI API key not found while you are sure you configured Ollama means one provider block is still defaulting to OpenAI. Check llm and embedder separately.
Searches that return nothing usually mean the default similarity threshold is filtering everything out. Pass threshold=0 while debugging and cross-check with get_all to see whether the memory was stored at all.
An empty results array from add is not a failure. It means your custom_instructions told the extractor that message contained nothing durable, which is the right call for "hey".
When to use this, and when to reach for something else
Use library mode (pip install mem0ai) when one application owns the memory and runs in your own process. That is what this walkthrough sets up, and it is the right starting point for most projects. Use the self-hosted server when several services or languages need to read and write the same memory over a REST API, which Mem0 ships as a Docker Compose stack. Use the managed cloud version when you would rather not operate Qdrant yourself.
Reach for different tools when the shape of the problem differs. If you need a resumable workflow with steps and checkpoints, a LangGraph checkpointer fits better, because what you are saving is execution state rather than facts about a user. If you want the model itself to maintain and rewrite its own context, Letta (formerly MemGPT) is built around editable memory blocks. And if you already run Postgres and want no new services, pgvector plus your own extraction prompt is legitimate, with the caveat that you now own the extraction schema, the deduplication logic, and the ranking.
On numbers: Mem0 reports its April 2026 algorithm at 92.5 on LoCoMo, up from 71.4, and 94.4 on LongMemEval, at under 7,000 tokens per retrieval call. Two caveats come straight from Mem0's own README. Those scores are from the managed platform, which includes optimizations the open source SDK does not have, so expect the same direction rather than the same figures. And the benchmark is self-reported. The token count is the useful part, because full-context approaches on the same benchmarks consume 25,000+ tokens per query. That gap is the cost you are avoiding by keeping memory outside the prompt.
Next steps
One collection per environment, so test data never mixes with production data. Two or three metadata buckets, no more. Log every add response for the first week and actually read the extracted facts, because that is where quality is won or lost. Add the prune job. Then, once retrieval looks sane on real queries, install mem0ai[nlp] and measure whether hybrid keyword matching earns its dependency.