iTransformer Electricity with CST
Time-Series Forecasting with Atomic Weights
2 min readupdated
The model tackles multivariate time-series forecasting using the Electricity power consumption dataset. It takes the previous 96 hourly measurements across 321 time series and predicts the next 96 hours. This is an independent variation of the baseline iTransformer architecture, where standard projections are replaced with Polar Triweight Atoms (special atomic weights). The model has 15.5% fewer parameters compared to the reference Dense model and is suited for analyzing and forecasting power system metrics.
Key features
- Atomic projections: Replaces encoder FFN projections with Polar Triweight Atoms while retaining trainable bias.
- Parameter reduction: Contains 4,084,320 parameters, which is 15.5% fewer than the reference model.
- Compatibility: Supports running on GPU and CPU through a standard CLI interface.
- Complete package: Includes scaler configuration, inference scripts, and the original architecture code.
What you can do
- Power consumption forecasting: Generate a 96-hour ahead forecast from a CSV file containing historical data for 321 series.
- Configuration comparison: Run inference using the reference variant (--variant dense) to evaluate accuracy.
- Local execution: Run inference on a CPU or GPU using the provided safetensors weights.
- Training reproduction: Run training scripts on the original THUML Electricity benchmark dataset.
Example
Command to clone the repository and run model inference:
python -m pip install huggingface_hub
hf download takumiecd/cst-itransformer-electricity --local-dir itr-ecl
python -m pip install -r itr-ecl/requirements.txt
python itr-ecl/inference.py itr-ecl electricity.csv prediction.csv --device cuda
Read more
DetailsStandardized test MSE: Dense 0.14778, CST 0.14982
Standardized test MAE: Dense 0.23948, CST 0.24189
Peak VRAM usage on RTX 4090: Dense 809.4 MiB, CST 951.6 MiB
Training: Dense on A100 MIG, CST on RTX 4090










Comments