Dataset · memorilla
Synthetic benchmark for checking memory and personalization
memorilla/PersonalizationV4
Fictional characters' dialogues with AI and questions to check hidden user preferences.
What's inside
A synthetic English-language benchmark with a volume of 10 to 100 thousand entries created for personalization evaluation. It is based on detailed fictional characters, each of whom had 200 short dialogues with an AI assistant. Verification questions place the user in a new situation and require combining at least two facts about them, while the character profile itself is not shown to the model—conclusions must be drawn solely from the chat history.
What you can do with it
- Evaluating long-term memory and chatbots' ability to adapt to the user
- Testing logical inference and matching disparate facts from dialogue history
- Benchmarking models on personalized response generation tasks
- Tasks
- Question answering, Text generation
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 10K
- Likes
- 4
How to load it
from datasets import load_dataset
ds = load_dataset("memorilla/PersonalizationV4")