Anyone who has ever scraped a website knows the raw output is rarely clean. The same company shows up as "Apple", "Apple Inc.", "Apple, Inc." and "APPLE INC"; prices arrive as strings with currency symbols; dates come in five different formats. Before any analysis is possible, that mess has to be tamed — and OpenRefine is still one of the best free tools for the job.
If you remember it as Google Refine, you are not out of date by much: it is the same tool. This hands-on OpenRefine tutorial walks through a lightweight data mining workflow — importing a scraped dataset, faceting it, clustering inconsistent values, transforming columns with expressions, and enriching records against an external knowledge base.
What happened to Google Refine?
The tool has had three names. It started as Freebase Gridworks (2010), became Google Refine when Google acquired Metaweb, and in 2012 Google handed it to the community as the open-source OpenRefine. Google Refine and OpenRefine are the same project — if a search still points you at "Google Refine", you want OpenRefine.
It remains free, open source, and actively maintained (the 3.x line). One thing has genuinely changed: the old Freebase reconciliation service that the Google Refine tutorials referenced was retired by Google in 2016. Reconciliation still exists and is arguably better than ever — it now points at Wikidata and other standardized services instead. More on that below.
Installing and launching OpenRefine
OpenRefine is a desktop application with an unusual twist: it runs a small web server on your own machine, and you drive it through your browser. Nothing is uploaded anywhere — your data stays local, which matters for confidential datasets.
- Download the release for your OS from the official OpenRefine site.
- It runs on Java. The Windows and macOS builds can bundle a runtime; on Linux you typically need a recent Java (JDK/JRE 11 or newer) installed.
- Launch it, and it opens
http://127.0.0.1:3333in your browser automatically.
That local address — not a cloud URL — is your workspace.
Loading a scraped dataset
OpenRefine reads almost anything tabular: CSV, TSV and other delimited text, Excel (.xls/.xlsx), ODS, JSON, XML, and Google Sheets. Choose Create Project, point it at your scraped file, and OpenRefine shows a preview where you confirm the delimiter, header row, and encoding (set this to UTF-8 to avoid mangled characters — a common headache with international data).
Give the project a name and click Create Project. You now have an editable grid. Nothing you do here touches the original file; OpenRefine records every step separately, which is the foundation of everything that follows.
Faceting: finding the mess
A facet is OpenRefine's way of summarizing a column so you can see and isolate its distinct values. On a text column, click the dropdown → Facet → Text facet. A panel appears listing every unique value and how many rows have it.
This is where dirty data reveals itself. A "Country" column might show USA, U.S.A., United States, and US as four separate entries. Click any value to filter the grid to just those rows, then edit or transform them as a group. Facets come in several flavors:
- Text facet — unique values in a column (the workhorse).
- Numeric facet — a histogram with a range slider to catch outliers.
- Timeline facet — for date columns.
- Custom facet — built from an expression, e.g. facet by string length to find truncated records.
Filters (the search box above the grid) narrow rows the same way. Together, facets and filters let you act on a targeted slice instead of the whole sheet.
Clustering: OpenRefine's killer feature
Manually merging Apple, Apple Inc. and APPLE INC is tedious across thousands of rows. Clustering does it algorithmically. Open a text facet and click the Cluster button.
OpenRefine offers two families of methods:
- Key collision (fast, conservative). Each value is reduced to a "fingerprint" — lowercased, punctuation and whitespace normalized, tokens sorted — and values that share a fingerprint are grouped. Variants include n-gram fingerprint (catches word-order and spacing differences) and phonetic methods like Metaphone3 and Cologne (catches "Catherine" vs "Katharine").
- Nearest neighbor (slower, broader). Methods like Levenshtein distance (counts single-character edits) and PPM catch typos and near-duplicates that fingerprints miss. You tune a radius/threshold to control how aggressive the matching is.
For each cluster, OpenRefine proposes a single canonical value; you tick the ones you trust and click Merge Selected & Re-Cluster. A column that had 400 spelling variants of 30 real companies collapses to 30 in minutes. Start with fingerprint, then try n-gram and Levenshtein for the stragglers.
Transforming data with GREL
For anything clustering can't do, OpenRefine has GREL — the General Refine Expression Language (still called Google Refine Expression Language in old docs). Choose a column → Edit cells → Transform, and write an expression that runs on every cell, where value is the current cell.
value.trim().toTitlecase() // normalize casing
value.replace(/[$£€,]/, "").toNumber() // "$1,299" -> 1299
value.split(" ")[0] // keep the first token
if(value.startsWith("**"), "", value) // blank out flagged rows
value.match(/(\d{4})-(\d{2})-(\d{2})/)[0] // pull a date out of text
Common structural operations live in the column menu too: splitting one column into several, transposing cells across columns into rows (and back), collapsing internal whitespace, and running regular-expression replacements. If GREL feels limiting, OpenRefine also lets you write transforms in Python (via Jython) or Clojure.
Reconciliation: turning names into entities
This is the "data mining" step that goes beyond cleaning. Reconciliation matches your free-text values against a structured knowledge base, so ambiguous text becomes a precisely identified entity. "Apple" resolves to Apple Inc. (the company) rather than the fruit or a nickname for New York City — and once matched, you can pull in extra columns like industry, headquarters, or a stable ID.
Where the old Google Refine tutorials used Freebase, modern OpenRefine ships with Wikidata reconciliation built in, and the reconciliation protocol is now a documented standard — so you can point it at other services (VIAF for names, GeoNames for places, or an in-house endpoint). Select a column → Reconcile → Start reconciling, pick a service and entity type, and OpenRefine scores candidate matches for you to approve. For developers, these services speak a standard HTTP JSON API, so you can build your own reconciliation endpoint against a private database.
Reusing your work
Every operation you perform is recorded in the Undo/Redo tab, and — crucially — you can Extract that history as JSON and Apply it to any other dataset with the same shape. Clean one month's scrape, save the recipe, and next month's file cleans itself. That portability is the big advantage over doing the same work by hand in Excel: the process, not just the result, is reusable.
Limits: when OpenRefine isn't the answer
OpenRefine is desktop-scale. It comfortably handles up to a few hundred thousand rows if your machine has the memory (you can raise its RAM allocation), but it is not a big-data engine — for millions of rows, reach for pandas, DuckDB, or a proper database. It is also interactive by design; for a fully automated pipeline you would script the same transforms in Python. Think of OpenRefine as the ideal tool for exploratory cleaning and one-off-to-recurring jobs, not for streaming ingestion.
If cleaning and enriching scraped data at scale is a recurring need rather than a weekend project, scraping.pro delivers it end to end as a data-as-a-service offering — you receive analysis-ready data instead of the raw mess.
FAQ
Is OpenRefine the same as Google Refine? Yes. It was renamed when Google open-sourced it in 2012. Any "Google Refine" guide applies, except the Freebase reconciliation parts.
Does OpenRefine send my data to the cloud? No. It runs entirely on your machine; the browser interface just talks to a local server on port 3333.
Is it free? Completely — it is open source under a BSD license.
What replaced Freebase for reconciliation? Wikidata is the built-in default, and OpenRefine supports any service that implements the standard reconciliation API.
Can I automate OpenRefine? You can export an operation's JSON history and reapply it to new files, and there are third-party clients for scripting it, but for full automation most teams re-implement the cleaning logic in code.