Parsing Text Files: Line-by-Line Reading, Regex, and Examples

Text parsing is the process of reading raw, unstructured text and pulling the meaningful values out of it. A plain text file (.txt) is the most "free-form" format there is: it has no schema, no grammar, and no standard. What's inside is entirely up to whoever created the file. It might be a simple list, a fixed-width column export, a dump with an unusual delimiter, or just a wall of prose.

That's why parsing text isn't about a library built for the format — it's about being able to break down an arbitrary structure by hand. This article is part of our overview of parsing documents.

What Is Text Parsing, and Where to Start

At its core, a text parser reads a file and converts loosely structured content into clean, usable data — rows, fields, records. Before you write a line of code, you need to understand the character of the file.

• If it's lines of one kind (one record per line), line-by-line reading plus per-line breakdown is enough.
• If values are separated by a consistent character — a tab, a semicolon, a pipe — then what you really have is a CSV with a non-standard delimiter, and it's better to use a CSV reader that correctly handles quotes.
• If it's a wall of unstructured text, regular expressions are the tool: you extract the fragments you need by pattern.

Two things are worth checking up front: the encoding (text from older systems is often in a legacy encoding such as Windows-1252 rather than UTF-8) and the line-ending character — Windows files use \r\n, Unix uses \n. Most languages read lines correctly either way, but a trailing \r sometimes has to be stripped by hand.

Line-by-Line Reading

Python

A file is an iterator over its lines, so reading it line by line is natural and memory-efficient even for gigabyte-sized files. This is the everyday pattern for parsing text files in Python:

with open("data.txt", encoding="utf-8") as f:
    for line in f:
        line = line.strip()          # strip whitespace and the newline
        if not line:                 # skip empty lines
            continue
        print(line)

JavaScript / Node.js

To avoid loading a large file all at once, Node reads it as a stream, line by line, via the readline module.

const fs = require("fs");
const readline = require("readline");

const rl = readline.createInterface({
  input: fs.createReadStream("data.txt", "utf-8"),
});

rl.on("line", (line) => {
  const clean = line.trim();
  if (clean) console.log(clean);
});

PHP

In PHP, fgets() reads one line at a time from an open handle, keeping memory use flat regardless of file size.

<?php
$handle = fopen("data.txt", "r");
if ($handle) {
    while (($line = fgets($handle)) !== false) {
        $line = trim($line);          // strip whitespace and the newline
        if ($line === "") continue;   // skip empty lines
        echo $line, PHP_EOL;
    }
    fclose($handle);
}

Go

package main

import (
    "bufio"
    "fmt"
    "os"
    "strings"
)

func main() {
    f, _ := os.Open("data.txt")
    defer f.Close()

    sc := bufio.NewScanner(f)
    for sc.Scan() {
        line := strings.TrimSpace(sc.Text())
        if line != "" {
            fmt.Println(line)
        }
    }
}

Extracting Data with Regular Expressions

When the values you want are "baked into" the text, you pull them out by pattern. Say the file has lines like [2026-06-01] Order #1042: $3490 and you need the date, order number, and amount.

import re

pattern = re.compile(r"\[(\d{4}-\d{2}-\d{2})\] Order #(\d+): \$(\d+)")

with open("orders.txt", encoding="utf-8") as f:
    for line in f:
        m = pattern.search(line)
        if m:
            date, order_id, amount = m.groups()
            print(date, order_id, amount)

The same approach in JavaScript:

const re = /\[(\d{4}-\d{2}-\d{2})\] Order #(\d+): \$(\d+)/;
const m = line.match(re);
if (m) {
  const [, date, orderId, amount] = m;
  console.log(date, orderId, amount);
}

And in PHP:

<?php
$re = '/\[(\d{4}-\d{2}-\d{2})\] Order #(\d+): \$(\d+)/';
if (preg_match($re, $line, $m)) {
    [, $date, $orderId, $amount] = $m;
    echo "$date $orderId $amount", PHP_EOL;
}

Regular expressions are the main tool for breaking down unstructured text, and it's worth keeping an interactive tester handy (regex101, for example) to debug your pattern against real lines. Remember their limits, though: don't use regex to parse nested formats like XML or HTML — those need proper parsers. For flat, line-oriented text, however, regex is the ideal choice.

Fixed-Width Column Data

Legacy systems and mainframe exports often produce text where columns are defined not by a delimiter but by character position: the first 10 characters are the SKU, the next 30 the name, and so on. You parse this format with string slices at fixed indices.

with open("fixed.txt", encoding="utf-8") as f:
    for line in f:
        sku   = line[0:10].strip()
        name  = line[10:40].strip()
        price = line[40:50].strip()
        print(sku, name, price)

A Special but Very Important Case: Logs

The most common kind of TXT file you'll parse in practice is server and application logs. Logs have no universal standard, but within a single source the line format is stable — which lets you apply the same techniques (line-by-line reading plus regular expressions), with a few wrinkles of their own: multi-line entries, rotation, and volumes measured in gigabytes. There's a dedicated article on parsing log files for that.

Get It Done for You

If you regularly receive text exports with a non-standard or "shifting" structure and need to turn them into clean tables, scraping.pro can set up parsing tailored to your format and deliver the result as CSV, Excel, or straight into a database — as a one-off cleanup or an ongoing data extraction service.