What to read to collect data from the web — and where to put it afterward.
Web scraping is a discipline that's deceptively simple to start and genuinely broad underneath. To write a working scraper you'll end up touching HTTP and HTML, regular expressions, getting past anti-bot protection, cleaning up "dirty" data, and finally storing it. That's why a smart reading list isn't a single book — it's a small stack: a core text on scraping itself plus a few titles on the adjacent topics. Below is a roundup of the best web scraping books worth your time, broken out by skill level, with separate sections on regular expressions and databases.
Let's settle the headline up front: the definitive book in the niche is Web Scraping with Python by Ryan Mitchell. The current third edition is substantially reworked and the one to get. Everything else lines up around it.
Core scraping books
If you're going to learn web scraping from books, start here — this is the material that teaches the actual mechanics rather than a single library's API.
Ryan Mitchell — Web Scraping with Python, 3rd ed. (O'Reilly, 2024)
The foundational textbook and the Python web scraping book to reach for first. Its strength is honesty: the author openly admits that scraping sits at the intersection of many fields — databases, web servers, HTTP, HTML, internet security, image processing, and data science — and gives an introductory tour of each. The structure is classic: the first part covers the mechanics of scraping (making requests to a server, handling responses, automation), and the second covers advanced tools and scenarios (Scrapy, storage, documents, natural language, forms and logins, JavaScript, APIs, image recognition, and getting past blocks).
A useful signal from reader feedback: the book lands best for people who already have six months to a year of Python experience but no scraping experience specifically. If your Python is still shaky, start with Sweigart (below) first.
Seppe vanden Broucke & Bart Baesens — Practical Web Scraping for Data Science (Apress)
The data-science angle. The authors are data scientists, and the book is a concise, modern guide to scraping that doesn't hide the important details and best practices. A good pick if scraping is, for you, the first step of a data-science project — building a dataset, cleaning it, and preparing it for analysis.
Python Web Scraping Cookbook (Packt)
A "recipes" format organized around concrete tasks: proxies, CAPTCHAs, Scrapy, dockerization, deployment. Useful as a desk reference rather than a from-scratch textbook.
Hands-On Web Scraping with Python (Packt)
Practice-oriented, with an emphasis on extracting quality data and on current techniques. A good applied complement to Mitchell.
Al Sweigart — Automate the Boring Stuff with Python
Not about scraping in full, but it has an excellent introductory chapter and is available free online. The ideal starting point if your Python is still wobbly.
A caveat. Books from Packt and similar publishers "rot" faster than Mitchell: libraries and anti-bot protection change fast. Always check code samples against the current library documentation.
Regular expressions
Regex is a workhorse of scraping. One important caveat: for parsing HTML you should use dedicated parsers (BeautifulSoup, lxml) and keep regex for "dirty" text, logs, validation, and post-processing of data you've already extracted. The canon here is stable and barely dates.
Jeffrey Friedl — Mastering Regular Expressions, 3rd ed. (O'Reilly, 2006)
The "bible" on the subject. Despite its age, it hasn't aged conceptually: it explains how a regex engine works internally — NFA/DFA, backtracking, pattern optimization. It's a heavy read, but after it, regex stops being magic.
Jan Goyvaerts & Steven Levithan — Regular Expressions Cookbook, 2nd ed. (O'Reilly, 2012)
A practical collection of ready-made solutions across languages, including Python. Handy as a reference to keep next to Friedl.
Michael Fitzgerald — Introducing Regular Expressions (O'Reilly)
A gentle introduction if Friedl is intimidating from the outset.
How to read these. For most scraping tasks, the regex chapter in Mitchell's own book plus the Cookbook within reach is enough. Turn to Friedl when you hit a wall on performance or complex patterns.
Data storage: "Seven Databases in Seven Weeks" and its context
Scraped results need to go somewhere, and the choice of store depends on the data type:
- flat tables → a relational DB (PostgreSQL, MySQL);
- nested JSON → a document DB (MongoDB, CouchDB);
- relationships between entities → a graph DB (Neo4J);
- cache, queues, temporary data → key-value (Redis).
There are dedicated books for exactly this.
Luc Perkins, Eric Redmond, Jim Wilson — Seven Databases in Seven Weeks, 2nd ed. (Pragmatic Bookshelf, 2018)
The famous "7 databases in 7 weeks." The second edition is a thorough tour of the NoSQL world across seven technologies — Redis, Neo4J, CouchDB, MongoDB, HBase, Postgres, and DynamoDB. Its main value for a scraping specialist is that it teaches you to think in data models: along the way it works through five models — relational, key-value, columnar, document, and graph — from the standpoint of real application needs. That's exactly the answer to "which data type goes into which database."
On editions: the first edition had a chapter on Riak; the second replaced it with DynamoDB and refreshed the examples. Get the second edition. Caveat: it's a 2018 book, so some CLI commands may have drifted — but as a map of database "genres" it's still the best.
Martin Kleppmann — Designing Data-Intensive Applications (O'Reilly)
If "7 databases" answers "what kinds of databases exist," DDIA answers "how to choose deliberately and not shoot yourself in the foot": replication, consistency, encoding formats, fault tolerance. The gold standard once your scraping volumes grow.
Bruce Tate — Seven Languages in Seven Weeks
The book that started the series — the "seven in seven" format came from here. It has no direct bearing on scraping and is mentioned only for completeness.
Narrow books for a specific database
Paired with scraping, targeted books on PostgreSQL (often a single relational store is enough to start) and MongoDB (if you're dumping raw JSON) are useful. For the SQL fundamentals themselves, Alan Beaulieu's Learning SQL (O'Reilly) is a solid, approachable baseline.
How to assemble your own set
A minimal, practical set looks like this:
- A scraping core — Mitchell, latest edition.
- Regular expressions — the chapter in Mitchell plus Regular Expressions Cookbook as a reference; Friedl as you go deeper.
- Storage — Seven Databases in Seven Weeks (2nd ed.) to choose a data model; DDIA once you scale; a book on the specific database you settle on.
Where you narrow from there depends on your stack and the type of target data: product tables, article text, relationship graphs, and streaming data all call for different storage tools and, therefore, a different reading order.
One last thing worth saying: books teach you the fundamentals, but they can't keep pace with the anti-bot arms race, and a full production pipeline is a lot to build from a standing start. If you need the data more than the learning journey, we run scraping as a done-for-you data-as-a-service offering — you skip the stack and get clean, structured output on a schedule. And when you do want to build it yourself, this reading list is where to begin.