Domain-specific legal model that stopped hallucinating citations
1. Problem. A major international law firm was running general-purpose models that hallucinated citations and produced inconsistent clause analysis across 11 jurisdictions. Every wrong answer was a liability risk in client-facing work.
2. Solution. We fine-tuned Gemma 3 27B on 400,000 of the firm's own legal documents using QLoRA on GCP Vertex AI. DPO alignment sharpened precision for time-pressured lawyers. A custom evaluation pipeline covering ROUGE-L, BERTScore and citation accuracy ran before any model reached production.
3. Impact
- 91%
- Citation accuracy on legal queries
- 0
- Hallucinated citations after alignment
- 400K
- Proprietary documents in the training set
- 60%
- Lower inference cost vs proprietary APIs