Document Data Extraction: Formats, Libraries, Approaches

Document data extraction is the practice of pulling structured data out of files in many different formats and turning it into something you can work with downstream: load into a database, push into a spreadsheet, feed to analytics, or hand to another system. In practice the job almost never involves a single format: a price list arrives in Excel, a CRM export in CSV, a contract in PDF, a third-party feed in JSON or XML, and the sitemap you crawl before collecting anything is its own XML dialect.

This article is an overview. We break down the logic of working with each type of file, show minimal examples in a few languages, and link to focused guides where each format is covered in depth. If you need a solution for a specific job rather than a DIY project, we also do done-for-you data extraction.

What every document parse is made of

Regardless of format, the process almost always comes down to three steps. First is reading — getting the file's bytes from a local folder, a URL, or a stream. Then comes parsing the structure — turning the raw content into your language's objects: a tree of nodes, a list of rows, a table. And finally extraction and normalization — selecting the fields you want, cleaning out the junk, coercing types (dates, numbers, currencies), and exporting the result.

The difficulty shifts between these steps depending on the format. For strictly structured data like JSON or XML, the heavy lifting is done for you — there's a formal grammar, and the parser is guaranteed to build a tree. For loosely structured formats — PDF, logs, arbitrary text files — most of the effort goes into that third step: the data has to be teased out with heuristics and regular expressions.

Structured data formats

JSON

JSON is the most common data-interchange format between services and the primary response of most APIs. It maps directly onto language structures: an object becomes a dictionary (or associative array), an array becomes a list. So parsing collapses into a single standard-library call, and the real work starts afterward — walking the nested structures. A detailed treatment, with nesting, streaming large files, and common pitfalls, is in the guide to parsing JSON.

import json

with open("data.json", encoding="utf-8") as f:
    data = json.load(f)

print(data["items"][0]["name"])

XML

XML is a tree of nodes with tags, attributes, and text. It's stricter and more verbose than JSON, but a great many industry standards are built on it: marketplace exports, banking formats, industrial data exchange. To parse it, you either use a streaming approach (for large files) or build a tree in memory and navigate with XPath. Details, libraries, and examples are in the guide to parsing XML.

RSS and Atom

RSS is a special case of XML with a fixed schema for news feeds, blogs, and podcasts. Technically you can parse it with any XML parser, but in practice a specialized library is more convenient — it hands you a ready list of entries with titles, dates, and links. More in the guide to parsing RSS feeds.

Sitemap

A sitemap is another XML format, describing a site's list of URLs for search engines and crawlers. Before collecting data from a large site you almost always crawl the sitemap first: it gives you a ready inventory of pages, sometimes with update dates and priorities, and saves you from blind link discovery. How to read plain and nested (sitemap index) files, honor lastmod, and handle gzipped versions is covered in the guide to parsing sitemaps.

Tabular formats

CSV

CSV is a simple text format — "rows and columns separated by a comma (or some other character)." Despite the apparent simplicity, this is where the most practical gotchas live: different delimiters, quotes inside values, line breaks within cells, encodings, and BOMs. That's why rolling your own "split on comma" parser is almost always a mistake — you need a proper CSV reader. The pitfalls are broken down in the guide to parsing CSV.

import csv

with open("prices.csv", encoding="utf-8", newline="") as f:
    for row in csv.DictReader(f):
        print(row["sku"], row["price"])

Excel (XLSX, XLS)

Excel isn't text — it's a binary (or, for XLSX, ZIP-packed) format with sheets, formulas, cell formats, and merged regions. You can't just read the file as lines: you need libraries that understand the workbook's internal structure. Often it's easier to convert the source price list or report to CSV first, then process it uniformly. How to read sheets, merged cells, and formula values is in the guide to parsing Excel.

Loosely structured formats

PDF

PDF is a format for printing and viewing, not for storing data, which makes it the hardest to extract from reliably. Text in a PDF can be "real" (selectable) or an image that requires OCR; tables aren't stored as tables but as a set of lines and text blocks with coordinates. Which libraries extract text, which extract tables, and when you can't avoid image recognition is covered in the guide to parsing PDF.

HTML

HTML is structurally close to XML, but in practice it's almost always "dirty": unclosed tags, extra markup, dynamic content. It's the primary format when collecting data from websites — extracting text, links, prices, and contacts. Parsers, CSS selectors, and link handling are covered in the guide to parsing HTML.

TXT and logs

Arbitrary text files (.txt) and server logs have no shared schema at all — the format is set by whoever created the file. Here the main tools are line-by-line reading and regular expressions. Logs are such a common and important case that they get their own article; the general approach to text parsing is in the guide to parsing TXT.

Scanned documents and OCR

A large share of real-world documents — invoices, receipts, IDs, signed contracts — arrive as scans or photos: images wrapped in a PDF, with no selectable text at all. Here parsing starts with optical character recognition (OCR).

• Tesseract (via pytesseract) is the open-source workhorse: free, scriptable, good enough for clean scans in many languages.
• OCRmyPDF wraps Tesseract to add a searchable text layer to an existing PDF without changing its appearance.
• For messy, rotated, or low-quality scans, preprocessing with OpenCV (deskew, denoise, threshold) makes a bigger difference than the OCR engine itself.

import pytesseract
from PIL import Image

text = pytesseract.image_to_string(Image.open("invoice_scan.png"), lang="eng")

OCR gives you raw text; you still need the third step — extraction and normalization — to turn "a wall of characters" into fields like invoice number, date, and total.

AI-based and API document parsing

For high-variation documents — thousands of invoice layouts, resumes, or forms where no two vendors format things the same way — hand-written rules and regexes stop scaling. This is where AI document parsing and hosted document parsing APIs have taken over as of 2026.

• Cloud document AI: AWS Textract, Google Document AI, and Azure AI Document Intelligence combine OCR with layout-aware models that return key-value pairs, tables, and form fields directly. You pay per page and skip the model training.
• LLM-based extraction: modern large language models accept a document (or its OCR text) and a target schema, and return clean JSON. This handles wording and layout variation that rule-based systems can't, and it's the fastest way to prototype extraction from a new document type.
• Structured-output prompting: pin the model to a JSON schema so the output is machine-parseable every time, then validate it before it enters your pipeline.

The trade-off: AI approaches are flexible and fast to build, but cost more per document and require validation, since a model can hallucinate a plausible-looking value. A common production pattern is a hybrid — deterministic parsing for the structured 80%, an LLM or document-AI API for the messy long tail, and a validation layer over both.

Choosing an approach

The rule of thumb: use the cheapest deterministic tool a format allows, and escalate to AI only where structure breaks down.

Which language and stack to choose

There's no single "right" choice, but there's an established practice. Python is the de facto standard for document parsing: rich libraries for every format (pandas, lxml, openpyxl, pdfplumber) and minimal code. JavaScript/Node.js is handy when parsing is embedded in a web project or you need browser rendering for dynamic content. PHP is often chosen when the data lands directly in a site on that stack. Go gets picked for speed and parallel processing of large volumes. In each focused guide we give examples in several languages, so you can grab the one closest to your project.

One thing unites all these stacks: the format dictates the library, not the other way around. So start by pinning down the exact format of your source files — then move to the matching guide: JSON, XML, RSS, sitemap, CSV, Excel, PDF, HTML, TXT, or logs.

FAQ

What's the difference between document parsing and web scraping? Web scraping fetches and extracts data from live web pages (HTML); document parsing works on files you already have — PDF, Excel, CSV, scans. They share the same third step: cleaning and normalizing the extracted data.

How do I extract data from scanned PDFs? Run OCR first (Tesseract, OCRmyPDF, or a cloud document-AI API) to recover text, then apply field extraction. For high-variation scans like invoices, a document-parsing API or LLM handles layout differences better than fixed rules.

When should I use an AI document parsing API instead of a library? When documents vary widely in layout and wording, and maintaining rules becomes costly. For consistent, structured files, a standard library is cheaper, faster, and fully deterministic.

If you have a pile of mixed-format documents — or a live feed of them — and want clean, structured output without building and maintaining the pipeline, scraping.pro provides data as a service and custom data extraction tuned to your formats and schema.