0c-ja-reranker
Japanese reranker for input methods
2 min readupdated
This 33.8M-parameter, 8-layer transformer encoder is built to rerank homophones for the 0C Japanese input method. By looking at the context on both sides of a keyword, it scores dictionary candidates and picks the best writing form. Trained on Wikipedia and Stack Exchange corpora, the model is exported to ONNX and ships in an int8-quantized variant for fast CPU execution.
Key features
- Specialization: Accurately selects Japanese homophones using surrounding sentence context.
- Optimization: Exported to ONNX Runtime with int8 inference for fast CPU performance.
- Metrics: Reaches 86.45% top-1 accuracy when using top-3 gating mode.
- License: Released under CC BY-SA 4.0 with an open manifest and attribution.
What you can do
- Integrate the model into a custom Japanese input method (IME) or text editor.
- Post-process and fix homophone errors in speech recognition or OCR pipelines.
- Run inference via ONNX Runtime in Python apps with a tiny memory footprint (around 62 MB delta).
Example
import json
import numpy as np
import onnxruntime as ort
import sentencepiece as spm
sp = spm.SentencePieceProcessor(model_file="vocab.model")
session = ort.InferenceSession(
"model.int8.onnx", providers=["CPUExecutionProvider"]
)
PAD, BOS, EOS = 0, 2, 3
SIDE, SEQ = 60, 124
def layout(left_ids, right_ids, candidate_id):
ids = np.full(SEQ, PAD, dtype=np.int64)
mask = np.zeros(SEQ, dtype=np.int64)
ids[0] = BOS
mask[0] = 1
left = left_ids[-SIDE:]
ids[1 + SIDE - len(left) : 1 + SIDE] = left
mask[1 + SIDE - len(left) : 1 + SIDE] = 1
ids[SIDE + 1] = EOS
ids[SIDE + 2] = candidate_id
ids[SIDE + 3] = EOS
mask[SIDE + 1 : SIDE + 4] = 1
right = right_ids[:SIDE]
ids[SIDE + 4 : SIDE + 4 + len(right)] = right
mask[SIDE + 4 : SIDE + 4 + len(right)] = 1
return ids, mask
rows = [layout(left, right, sp.piece_to_id(surface)) for surface in candidates]
ids = np.stack([row[0] for row in rows])
mask = np.stack([row[1] for row in rows])
scores = session.run(None, {"input_ids": ids, "attention_mask": mask})[0]
best = candidates[int(np.argmax(scores))]
Read more
DetailsArchitecture: D512, 8 layers, 8 attention heads, 33,864,193 parameters.
Performance (int8): 23.16 ms median latency for a batch of 5 candidates.
Memory footprint: Uses around 62 MB once loaded.
Limitations: Requires a strict positional layout (60 tokens of context on each side, 124 total sequence length).










Comments