CASE STUDY · CLASSIFICATION

A document classifier that reads like a person: words, pictures, and layout

Production classification of real-world documents by fusing three signals — visual features, text, and page structure — reaching 95% accuracy.

Three translucent layers of one document page — visual regions, text blocks, and layout wireframe — converging into a single sharp page

At a glance

role
R&D ML Engineer, Ninestars (led)
result
95% classification accuracy
approach
hybrid CV + NLP + layout analysis
stage
full-scale production

Problem

Document pipelines break when they lean on one signal. Text-only classifiers can’t tell apart documents that share vocabulary but differ in structure; vision-only models see the structure but not the meaning. Real document streams — scans of varying quality, mixed templates, dense pages — punish both.

At Ninestars I led the adoption of AI and computer vision for document analysis and data extraction, and classification was the front door: every downstream extraction step depended on routing each document correctly first.

Approach & architecture

The classifier is deliberately hybrid, combining three families of features:

  • Vision — what the page looks like: visual structure and image-level features.
  • Language — what the page says: NLP features over the extracted text.
  • Layout — how the page is organized: layout analysis of the document’s spatial structure.

I owned the full lifecycle: data exploration and quality assessment, feature engineering, model prototyping, parameter fine-tuning against the business problem, and deployment to full-scale production — including SQL work over large datasets to build and evaluate training data.

The hard part

Getting three signal families to agree. Documents that were trivial for one modality were adversarial for another, and the failure modes only surfaced through systematic data-quality assessment and error analysis — the unglamorous loop of exploring the data, finding where the model was wrong, and engineering features or fine-tuning parameters until it wasn’t.

Result

95% accuracy in production. The classifier became the routing layer for the document-extraction workflows that cut manual intervention by 30% across the pipeline.

Stack

  • Python
  • PyTorch
  • OpenCV
  • Transformers
  • SQL
  • Docker

stack mapping from skills inventory