The task
The agency needed an objective picture of IT salaries — by specialty, seniority, stack and region — to back up salary ranges in front of clients and candidates. The company hires across different markets, and at that volume it is easier to run your own consolidated analytics tool than to rely on scattered third-party reports.
What we did and what data we collected
We set up regular job posting collection from several boards: title, salary range, seniority, stack, city, work format, publication date. The data was mapped to unified dictionaries of roles and skills and aggregated into metrics (medians and trends). The finished data layer fed a dashboard for the agency's analysts.
Challenges and how we solved them
First, the salary is often missing or given as a range. We worked with ranges and filled the gaps with estimates based on similar postings, honestly flagging computed values.
Second, duplicates: the same vacancy is posted on several boards and in several cities. We deduplicated postings to avoid inflating volumes and distorting medians.
Third, inconsistent role names and stacks. We mapped everything to unified dictionaries — otherwise "PHP programmer" and "Backend developer (PHP)" would count as different roles.
The result
The agency got a continuously updated dashboard of IT salaries with breakdowns by role, stack and city, updated on schedule — instead of one-off reports that age quickly.
Services in this case study: Job Postings Scraping · AI Data Processing · Market Research on Real-World Data · DaaS and data via API
Useful reading: Data normalization · Scraping dynamic sites