C-LPNCM-2.8B
Calibrated Looped-Pointer Next-Concept Model
2 min readupdated
C-LPNCM-2.8B is a 2.89-billion-parameter hybrid language model built by researcher entity2260. Unlike standard autoregressive models, it uses entropy analysis to split text into variable-length semantic patches, quantizes them into a discrete conceptual hierarchy, and refines them iteratively through equilibrium loops. The model relies on a dual-path decoding setup that combines concept generation with pointer-based token copying. Weighing in at around 5.38 GB on disk in bfloat16, it is aimed at researchers and developers exploring alternative transformer architectures.
Key features
- CLPNCMForCausalLM architecture (a 4-tower pure transformer without diffusion).
- Roughly 3.6B effective compute parameters achieved through 8 weight-tied equilibrium loops.
- Modest hardware requirements: around 5.88 GB VRAM, running easily on a single NVIDIA T4, L4, or A100 GPU.
- Epistemic confidence calibration and precise adaptive halting of refinement loops during generation.
What you can do
- Run local inference inside a standard transformers library pipeline using the trust_remote_code=True flag.
- Test English text generation with dynamic allocation of refinement loops.
- Inspect the custom encoder, quantization, and loop implementations in modeling_cl_pncm.py.
Example
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "entity2260/cl-pncm-2.8b"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
repo_id,
trust_remote_code=True,
torch_dtype=torch.bfloat16,
device_map="auto",
)
model.eval()
prompt = "the government announced that the new economic plan will"
inputs = tokenizer(prompt, add_special_tokens=False, return_tensors="pt").to(model.device)
output_ids = model.generate(
**inputs,
max_new_tokens=8,
temperature=0.7,
top_k=10,
repetition_penalty=8.0,
)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))
Read more
DetailsBenchmarks and training: Surface CE Loss 1.4390, VQ Commitment Loss 0.0002, Brier Calibration Error 0.0216.
Hardware and VRAM: 2 sharded .safetensors files, 5.38 GB on disk, 5.88 GB VRAM in bfloat16.
Limitations: requires trust_remote_code=True, 30,522-token vocabulary size.










Comments