legal

Linkedin lost in court to data analytic company that scrapes Linkedin’s public profiles info

On September 9th, 2019 the UNITED STATES COURT OF APPEALS 1 has affirmed the former district court’s determination that a certain [data] analytic company is lawful to scrape [perform automated gathering] LinkedIn’s public profiles info. Now the historical event has happened in which a court is protecting a data extractor’s right for mass gathering openly presented business directory information. more…

New European e-communication regulations and web scraping

GDPR-eu-rulesGeneral Data Protection Regulation or GDPR: enforcement date –  25 May 2018. The GDPR covers the matter of online user data privacy rules for electronic communication and data protection. The regulation includes modern communication messengers and services, eg. Skype, Viber, Gmail, etc., that have not been previously mentioned in the former EU e-communication directives.   more…

Is this a legal method of acquiring insurance leads?

Recently I received a question on insurance leads:

Is this a legal method of acquiring insurance leads [from the web]? Are there any agent testimonials on the efficiency of this type of service?

Legality issue in web scraping

With the matter of legality in web scraping, there should be a clear approach –  it depends on the website and its privacy policy. There could be at least 2 cases:

  1. Public info (prices, inventory info, public offers), i.e. everything that is not protected by copyright and available for scraping.
  2. The copyright protected info –  website Terms of Use or Terms of Service restrictions make copying and therefore web scraping illegal.
The US court of appeals has affirmed that a certain [data] analytic company is lawful to scrape data aggregator’s (LinkedIn’s) public profiles info.

So far I have no insurance agent testimonies on the efficiency of any insurance lead scrape service. The web sites I searched [on the insurance leads] have given me the impression that the customer info they gather is highly secured (not viewable). I doubt that any sites are going to expose insurance leads. In most of them the leads are available by paid subscription plans.

If there are any such websites like insurance leads directories (public insurance quotes), we might develop a scraper that consistently grabs fresh or new info for further analysis. It does save the agent’s time for re-searching, re-visiting and so on. One scraper might work with multiple directory pages for scrape.

The US district court has concluded that moderate scraping, even when against ToS, is legal.

You might find it interesting to read about web page change tracking if you only need to see updates (no data storing applied).

How to alarm of your site being illegally scraped

Have you encountered the issue of your site being scraped and your online content being infringed? Yes, you’ve warned your content abuser with no response or you have received just some excuses. But, after Google indexing, your content does not stick out of the similar content heap of stolen material in search results? What can one do to set an alarm and enforce some consequences or even punishment?  more…

Back to top