chess-pgn-2400plus-seed1
character-level chess transformer
2 min readupdated
A compact 51-million-parameter language model by Arisp that predicts chess moves in PGN text format (the standard notation for recording games). Built on a character-level transformer architecture, it was trained on 10 million Lichess games where both players were rated 2400+. This checkpoint serves as a control run for the base chess-pgn-2400plus model, using a different random seed (seed 1) and a single cosine learning rate schedule over 250,000 steps. It is useful for researchers evaluating run-to-run variance and developers who need a lightweight move generator without heavy chess engines.
Key features
- Character-level generation: processes raw PGN notation within a 1024-character context window, inferring board state solely from move history.
- Compact footprint: only 51M parameters in safetensors format, making inference easy to run even on a CPU.
- Improved validation loss: achieved 0.3566 bits per character on a 5,000-game held-out split, compared to 0.3622 for the first version.
- Control checkpoint: helps separate real experimental gains from natural variance caused by changing the seed and learning rate schedule.
What you can do
- Build a chess bot or training tool that mimics the play style of strong club players and 2400+ masters, including realistic human-like mistakes.
- Pair it with the original
chess-pgn-2400plusas a baseline when testing new hyperparameters and fine-tuning methods for PGN models. - Generate opening and middlegame continuations in Python scripts using the built-in
generate_movesfunction and thepython-chesslibrary.
Example
import json, chess, sys
from huggingface_hub import snapshot_download
from safetensors.torch import load_file
path = snapshot_download("Arisp/chess-pgn-2400plus-seed1")
sys.path.insert(0, path)
from pgn_model import PGNTransformer, generate_moves
model = PGNTransformer(**json.load(open(f"{path}/config.json"))["architecture"])
model.load_state_dict(load_file(f"{path}/model.safetensors"))
model.eval()
board = chess.Board()
board.push_san("e4")
move = generate_moves(model, [board], temperature=0.0, dev="cpu")[0]
print(board.san(move))Read more
DetailsValidation loss: 0.3566 bits per character (0.2472 nats) across 5,000 games.
Training: 250,000 steps (checkpoint saved at step 242,500), AdamW, bf16 on a single NVIDIA B200.
Limitations: not a full chess engine, occasionally outputs illegal moves, and loses context after move 80–100 due to the 1024-character window.










Comments