0xr4j/banksathi-gemma2-2b
Offline banking assistant
2 min readupdated
BankSathi · Gemma-2 2B is a lightweight language model based on google/gemma-2-2b-it and trained with QLoRA (an efficient fine-tuning method) to help rural users navigate banking operations and stay safe from scams. The assistant runs offline directly on Android devices, understands code-switched speech (Hinglish, English, and Hindi), tolerates speech-recognition errors, and outputs strict structured JSON for seamless app integration. It is built for banking customers who need a fast, offline advisor to check limits, handle transfers, and screen suspicious calls.
Key features
- Offline on-device inference: The converted Android model takes up 1.67 GB and runs on CPU.
- Scam protection: Includes a built-in ScamGuard classifier to flag fraudulent SMS messages and calls.
- Multilingual support: Understands English, Hindi (Devanagari), and conversational Hinglish.
- High accuracy: Achieves 100% recall on ScamGuard and 98.5% intent-classification accuracy.
What you can do
- Run on a smartphone: Integrate the model into an Android app using MediaPipe LLM Inference.
- Handle banking queries: Check account limits, assist with KCC loan applications, and analyze suspicious calls.
- Run in Python: Load the LoRA adapter via the
peftlibrary for server-side pipelines.
Example
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("google/gemma-2-2b-it")
base = AutoModelForCausalLM.from_pretrained("google/gemma-2-2b-it", attn_implementation="eager")
model = PeftModel.from_pretrained(base, "0xr4j/banksathi-gemma2-2b").merge_and_unload()
Read more
DetailsParameters: 2.6B (LoRA adapter size: 83 MB)
VRAM requirements: 8.0 GB (trained on a free T4 in 70 minutes)
Dataset: 1,500 synthetic rows (no real user data)
Limitations: Requires strict JSON formatting and a deterministic fallback in the app










Comments