MarineSparkLM-4Bv1
LoRA for marine and offshore engineering
2 min readupdated
This model is a LoRA (a lightweight add-on for a base model) adapter built on top of OceanGPT-basic-4B-Instruct, a 4.06B-parameter language model based on the Qwen3 architecture. Trained on 2,295 question-answer pairs, it spans ten domains in marine and offshore engineering—from hydrodynamics and naval architecture to offshore energy and autonomous control systems. It improves response relevance over the base model when generating text and answering domain-specific engineering queries, making it useful for engineers, researchers, and maritime students.
Key features
- Domain-specific training on 2,295 Q&A pairs covering 10 marine engineering topics.
- Stronger performance on benchmark prompts compared to the base model.
- Parameter-efficient fine-tuning of just 33M parameters (0.81% of the base model) via PEFT.
- 4-bit quantization support via bitsandbytes to reduce VRAM usage.
What you can do
- Run a chatbot to answer questions on naval architecture and offshore structures.
- Draft technical documents for maritime research projects.
- Plug the adapter into local text-processing pipelines for marine engineering workflows.
Example
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_id = "zjunlp/OceanGPT-basic-4B-Instruct"
adapter_id = "mcschin75/MarineSparkLM-4Bv1"
tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base_model, adapter_id)
messages = [{"role": "user", "content": "Explain the key considerations in ballast water treatment systems for compliance with maritime regulations."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
output = model.generate(inputs, max_new_tokens=200, temperature=0.8, top_k=50, repetition_penalty=1.2, no_repeat_ngram_size=3)
print(tokenizer.decode(output[0], skip_special_tokens=True))Read more
DetailsBenchmarks: Scored 84/120 on manual evaluation (up from 42 on the base model).
Limitations: Overall accuracy remains modest, so outputs should be independently verified.
Training: 33,030,144 trainable parameters using response-masked instruction tuning.










Comments