melwyn.ai
← Lab / Selected work
Data engineering

Document Intelligence

Pull the answer out of tens of thousands of scanned reports where it only exists as an image.

Built in-house for a market research team
The problem

The fact the team needed, which firm audited each company, existed only inside scanned PDF reports. The structured field on the website was empty, the documents were image-only, and there were tens of thousands of them, none of it text-searchable. No query could touch it.

The approach

A standalone pipeline scrapes the report index into a candidate list, renders each report's first pages, runs OCR on the scanned text, and matches the firm names. It is kept off the main automation platform because OCR over thousands of files needs its own process, and it is resume-friendly and threaded, so a full run finishes in hours and a clean workbook drops out the other end.

How it runs

Walk the pipeline

Click through each stage to see what the system does and what the team gets back.

Stage 1 of 4

Catalogue the whole archive first

It walks the report index and builds a table of every record, flagging the larger organizations as the candidates worth reading.

~37krecords catalogued
What changed

~29k

scanned reports read

Image-only

data made searchable

~6 hrs

for the full run

The answer was always there, just trapped in pictures of pages that no search could reach. Reading it by hand would have taken weeks. Now OCR does the reading, the pipeline can stop and resume without losing its place, and a question that lived inside tens of thousands of scans comes out as one spreadsheet you can sort.

Index scraperPDF renderingOCRSpreadsheet export

Want something like this on your team?