Qwen3.5-0.8B-v15
Qwen3.5 with episodic memory and "sleep"
3 min readupdated
LTM-Lab ported the compact multimodal model Qwen3.5-0.8B (0.87B parameters) to the Qwen3_5V15ForConditionalGeneration architecture, giving it long-term episodic memory. The base model weights remain bit-for-bit untouched—no fine-tuning or quantization was performed—while adding memory management scripts at the KV-cache (saved attention states) and recurrent layer levels. Unlike the previous V14 release, this version drops the trainable "world model" and introduces a "sleep" mode: when the server is idle, the model reviews accumulated records on its own, merges duplicates, flags outdated facts, and links notes about the same entities. This setup is handy for developers experimenting with local assistants that retain context across past conversations without RAG or vector databases.
Key features
- Memory without RAG or training: while reading a prompt, attention keys and values are chunked into events of up to 128 tokens and saved to disk along with a recurrent state engram, then injected back during subsequent queries without inserting text into the prompt.
- Idle "sleep" cycle: after 10 minutes of inactivity (by default), the server kicks off a background review—replaying recent records, merging duplicates, archiving contradicted facts, and building "bridges" between related events.
- Ready-to-run SGLang inference: supported via the modified
lovewin10/sglang-v15Docker image featuring an OpenAI-compatible API, CUDA graph, prefix cache, and EAGLE speculative decoding. - Honest 0.8B limitations: at this small parameter count, the model acts as a fairly weak judge during its own "sleep" phase (and may mistakenly archive unrelated notes), so the release is positioned as a lightweight testbed for the mechanism.
What you can do
- Build a local bot with cross-session memory: spin up a Docker container with a mounted
/memoryfolder so the model remembers facts from one chat and recalls them in a brand-new session, even after a server restart. - Test recall and "sleep" in Python: load the model directly via
transformerswithtrust_remote_code=Trueand triggermodel.sleep()manually between conversations while tracking hits viamemory_trace.
Example
from transformers import AutoModelForImageTextToText, AutoProcessor
import torch
path = 'LTM-Lab/Qwen3.5-0.8B-v15'
model = AutoModelForImageTextToText.from_pretrained(path, trust_remote_code=True, dtype=torch.bfloat16).to('cuda').eval()
processor = AutoProcessor.from_pretrained(path)
# После нескольких вызовов ask(...) можно запустить цикл консолидации памяти:
report = model.sleep()Read more
DetailsMemory benchmark (bf16, RTX PRO 6000 Blackwell): single facts — 9/10, across 48 distractor sessions — 7/10, across 248 sessions — 6/10, two-hop retrieval after sleep — 6/6 (5/6 before sleep), fact updates after sleep — 2/6.
Hardware and requirements: SGLang requires a GPU supporting
flashinfer or fa3 backends (TP/PP/DP = 1) and a fast SSD for the /memory store; running via transformers also works on CPU.Important limitation: the
/memory store is shared across the entire server instance with no per-user isolation, and the model's own generated responses are not written to memory until they appear in a subsequent prompt.









Comments