eliasubz/biobert-drug-ner
Fast Model for Drug Entity Extraction
2 min readupdated
Based on BioBERT (108M parameters, built on top of dmis-lab/biobert-base-cased-v1.1), this model detects drug mentions in texts and classifies them into four categories: specific substance (drug), drug group (group), brand name (brand), and other names (drug_n). The model is partially distilled using the larger MedGemma-27B language model, delivering high processing speed while maintaining key accuracy metrics. This tool is useful for researchers, healthcare IT developers, and English NLP specialists.
Key features
- Extreme acceleration: Processes a single sentence in 2.9 ms on an RTX 3090, making it 530 times faster than the MedGemma-27B teacher.
- High accuracy retention: Maintains a Macro-F1 of 0.759 (89.8% of the teacher model) and a Micro-F1 of 0.888.
- Specialized training: Fine-tuned on the DDI corpus dataset with added synthetic labeled data from PubMed for rare classes.
- Compact size: Only 108 million parameters for easy deployment in production.
What you can do
- Medical text analysis: Integrate into literature processing pipelines to automatically search for drug names.
- Drug classification: Separate mentions in abstracts and articles into groups, brands, and active ingredients.
- Integration into NLP pipelines: Use the standard
transformerslibrary pipeline for rapid text annotation.
Example
from transformers import pipeline
ner = pipeline("token-classification", model="eliasubz/biobert-drug-ner",
aggregation_strategy="simple")
ner("Concomitant use of warfarin and NSAIDs such as Advil increases bleeding risk.")
Read more
DetailsMetrics on the DDI test set (1,430 sentences): Macro-F1 0.759, Micro-F1 0.888, Accuracy 0.980.
Requirements and limitations: Works only with English texts (DrugBank and MEDLINE); the
drug_n class is recognized less reliably (F1 0.34) due to unlabeled teacher data.License: Subject to the Health AI Developer Foundations terms.










Comments