Skip to content

Dataset · memorilla

Synthetic benchmark for checking memory and personalization

memorilla/PersonalizationV4

Fictional characters' dialogues with AI and questions to check hidden user preferences.

What's inside

A synthetic English-language benchmark with a volume of 10 to 100 thousand entries created for personalization evaluation. It is based on detailed fictional characters, each of whom had 200 short dialogues with an AI assistant. Verification questions place the user in a new situation and require combining at least two facts about them, while the character profile itself is not shown to the model—conclusions must be drawn solely from the chat history.

What you can do with it

  • Evaluating long-term memory and chatbots' ability to adapt to the user
  • Testing logical inference and matching disparate facts from dialogue history
  • Benchmarking models on personalized response generation tasks
Tasks
Question answering, Text generation
Languages
English
Size
10K–100K rows
Downloads this month
10K
Likes
4

How to load it

from datasets import load_dataset

ds = load_dataset("memorilla/PersonalizationV4")