Smart-Home Energy SLM
a compact language model for home energy analysis
2 min readupdated
Built on the Qwen3 architecture, this model was trained from scratch on a custom dataset of 148,363 energy-related Q&A pairs to evaluate utility meter readings. Packing just 109M parameters, it calculates electricity costs, classifies load levels, detects consumption peaks, and refuses to answer when data is missing. Because it is tuned for a strict prompt format rather than open-ended chat, it is best suited for developers exploring how to train small language models for narrow home-automation tasks.
Key features
- Qwen3 architecture featuring grouped-query attention and RMSNorm
- Trained from scratch on random weights with no pre-training
- Specialized in cost calculations and electrical load analysis
- Reliable abstention on incomplete data without hallucinating
What you can do
- Run the model locally via Ollama to process energy-consumption queries
- Integrate the model into a Python app using the transformers library
- Benchmark response accuracy on the UCI and REFIT appliance datasets
Example
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "Design-Thinking-SLM/smart-home-energy-qwen3-scratch"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
question = "Appliances: 250 Wh at 2016-05-18 19:10. Is that unusually high?"
text = tok.apply_chat_template([{"role": "user", "content": question}], tokenize=False, add_generation_prompt=True)
ids = tok(text, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=160, do_sample=False)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))
Read more
DetailsBenchmarks: pooled accuracy of 69.0% (UCI) and 43.1% (REFIT)
Hardware requirements: trained on an RTX 5060 Laptop GPU (8 GB); 379 MB file size in GGUF
Limitations: weak arithmetic when handling many numbers; intended for educational use only
Training details: 109.1M parameters, 448-token context length, 72,000 steps










Comments