ChronosBERT-256
a time series representation model
1 min read
ChronosBERT is a pre-trained model featuring a bidirectional BERT-style encoder (transformer architecture) built for time series representation learning. This version was trained with a batch size of 256. Time series data is normalized and discretized into tokens via the Chronos tokenizer, then processed by the encoder using Masked Language Modeling (token masking). You can fine-tune the model for downstream tasks like time series classification.
Key features
- BERT-Small architecture (encoder-only) with around 17.3M parameters
- Chronos MeanScaleUniformBins tokenizer for time series data
- Pre-trained with Masked Language Modeling (predicting masked tokens)
- Larger batch size (256) compared to the standard version
What you can do
- Fine-tune the model for time series classification on custom datasets
- Extract general-purpose representations from sequential data
Example
import sys
sys.path.append("ChronosBERT/chronos_pkg/src")
from chronos_pkg.src.chronos import ChronosPipeline
pipeline = ChronosPipeline.from_pretrained(
"ddrg/ChronosBERT-256",
task="classification",
num_labels=2,
dropout_head=0.1,
TrainInnerModel=True,
)
model = pipeline.model
tokenizer = pipeline.tokenizer
Read more
DetailsArchitecture: BERT-Small (encoder-only)
Parameters: ~17.3M
Pre-training objective: Masked Language Modeling
Pre-training batch size: 256










Comments