I have already written several articles on how to use WebDriver for web scraping, but I have never touched on the topic of changing WebDriver’s IP address. Nevertheless, this topic is quite crucial when you come to web scraping, and here I’d like to show you an example of using proxies with WebDriver in Python (and you can easily convert it into your language API). more…
Search Results for: web driver
6 Tips for Using WebDriver with Java
In this article I’ll share with you 6 usefult tips that may help you when you work with Selenium WebDriver in Java. more…
How to scrape Amazon with WebDriver in Java
Here is a real-world example of using Selenium WebDriver for scraping.
This short program is written in Java and scrapes book title and author from the Amazon webstore. more…
How to use Selenium WebDriver with Java
As we already showed you the example of using WebDriver with C#, in this post we will see how to extract web data using Selenium WebDriver with Java, the native language of Selenium WebDriver. more…
Example of Scraping with Selenium WebDriver in C#
In this article I will show you how it is easy to scrape a web site using Selenium WebDriver. I will guide you through a sample project which is written in C# and uses WebDriver in conjunction with the Chrome browser to login on the testing page and scrape the text from the private area of the website. more…
Pros and Cons of using Selenium WebDriver for Website Scraping
Since Selenium WebDriver is created for browser automation, it can be easily used for scraping data from the web. In this post we will consider some advantages and drawbacks of using WebDriver for web scraping. more…
What is Selenium WebDriver?
If you are interested in browser automation or web application testing you may have already heard of Selenium. Since there is a lot of terminology related to this framework, it is easy for you to get lost, especially if you come to Selenium for the first time. In this article I want to save your day by providing a short and clear explanation of what is what in the Selenium project. more…
7 Ways to Protect Website from Scraping and How to Bypass this Protection
In this article I’d love to revise few well-known methods of protecting website content from automatic scraping. Each one has its advantages and disadvantages, so you need to make your choice basing on the particular situation. None of these methods is ultimate and each one has its own ways around I will mention further.
more…
Selenium IDE and Web Scraping
Selenium is a web application testing framework that supports for a wide variety of browsers and platforms including Java, .Net, Ruby, Python and other. In this post we touch on the basic structure of the framework and how it can be applied in Web Scraping. more…
A Simple Code that Extracts a Hotel List from Booking.com
In this post I will show you how easy it is to write a Python code that extracts hotel list from booking.com. The simplicity of this code is achieved with the help of Selenium Web Driver which acts as the main data extraction means here.
more…
