hybrid-radiograph-feature-fusion
hybrid models for chest X-ray analysis
2 min readupdated
A set of pretrained PyTorch checkpoints for medical image classification, combining modern AI model architectures (ConvNeXt-Tiny and Swin Transformer-T) with classical computer vision descriptors (HOG, LBP, Gabor, and SIFT-BoVW). The model tackles binary classification of chest X-rays using the Kermany Pediatric Chest X-Ray Cohort dataset. This hybrid approach improves accuracy by extracting both deep and textural image features together. The solution will be useful for researchers and developers in medical diagnostic imaging.
Key features
- Combination of AI model backbones (ConvNeXt-Tiny and Swin-T) with classical image descriptors
- Pretrained weights for various feature fusion strategies (concatenation, Gated Cross-Modal, Token Attention)
- Tested on the pediatric chest X-ray dataset (225 test images)
- Full compatibility with the PyTorch ecosystem and Hugging Face Hub utilities
What you can do
- Integrate pretrained weights into automated X-ray analysis pipelines
- Conduct a comparative analysis of the effectiveness of deep and hybrid features in medical CV
- Run inference for the desired checkpoint using the provided loading script
Example
import torch
from huggingface_hub import hf_hub_download
from src.models import HybridMedicalClassifier
ckpt_path = hf_hub_download(
repo_id="subhan1501/hybrid-radiograph-feature-fusion",
filename="exp6_swin_sift.pth",
)
model = HybridMedicalClassifier(
backbone_name="swin_t",
fusion_type="token_attention",
hand_dim=208,
embed_dim=256,
num_classes=2,
pretrained=False,
)
model.load_state_dict(torch.load(ckpt_path, map_location="cpu"))
model.eval()Read more
Details• Test accuracy: up to 94.22% (exp1_convnext_baseline.pth)
• Dataset: Kermany Pediatric Chest X-Ray Cohort (1500 images: 70% Train / 15% Val / 15% Test)
• Preprocessing: Symmetric CLAHE (clipLimit=2.0, tileGrid=(8, 8))
• License: MIT License










Comments