This article explains what IP rotation is, how to achieve it using rotating proxies, and what are the available options.
What You’ll Learn
What is IP rotation?
When the static IP addresses of internet daemons, servers or services are changed at random or predetermined intervals, this is called IP rotation. Put differently, IP Rotation is allocating IPs to resources in a random or configurable way specified by the administrator.
When DSL users connect to an ISP, an IP address from a pool of available IPs is assigned to the user and their internet address becomes whichever IP was allocated. If the link is disconnected and then reconnected, the ISP will allocate the next available IP, thus implementing an IP address rotation that is transparent to the user.
Internet-facing daemons often implement automatic IP address rotation for incoming traffic. DNS servers may for example regularly change the IP address of a web server to enable equal distribution of resources among role-based access control lists or load-balancing of incoming traffic.
IP rotation can also be used for outgoing internet traffic. Destination firewalls in all types of internet communications use source IPs for access control. Restrictions put in place can often be evaded by rotating the IP address.
What is the use of IP rotation?
Regular internet users may be interested in changing their IP automatically every several minutes for two main reasons: to improve the level of anonymity, or to automate tasks like data scraping and web crawling.
Automatic IP rotation regularly changes your online identity and hides your public IP address, thereby increasing your online anonymity. Tracking any downloading or browsing activities becomes much more difficult, although not impossible.
For web crawling and data scraping rotating IP addresses is the way to go. Although this can be done manually, it is not very efficient, and auto IP rotation is a better solution.
A list of common uses for IP rotation is given below:
- Anti-piracy and anti-counterfeiting and agencies use IP rotation to harvest data or to research questionable content.
- Big IT companies like Google, Yahoo, Bing, and Amazon all rotate IP addresses for their outgoing bots to distribute the loads on their networks.
- Qualitative and quantitative research organizations use IP rotation to observe variations.
- Business intelligence companies automatically change IP addresses to scrape websites or mine, retrieve and harvest data for data analytics and performance metrics.
- Data triangulation companies rotate IPs to verify the validity of the content.
- Data warehouses change IP addresses automatically to access a wide selection from their destinations.
- SEO companies use SEO proxies to auto change IP addresses and check keyword rankings from different locations or to promote their content on various social media accounts.
- Corporate firms rotate proxy IP addresses to eliminate geo-targeted presentation and price gouging.
IP rotation methods
There are four ways in which IP rotation is implemented.
- Random IPs: Each connection is assigned to a random IP from a pool of IP addresses.
- Pre-configured IPs: IP rotation is done at minute intervals with a new IP assigned every minute or specific interval.
- Specific IPs: An origin source chooses which IP address is to be used for outgoing connections.
- Burst IPs: IP addresses are rotated every specified number of hits, e.g., once 10 connections have been made with an IP, the 11th will be done with a different IP.
What is a rotating proxy?
A rotating proxy server does IP address rotation to enable users to hide their IP addresses similar to TOR. Each request made will go through a different, random IP address.
IP rotation can also be achieved by changing (rotating) proxies at regular or random intervals. The advantage of this approach is that it can be implemented using cheaper proxies, datacenter proxies, even free ones.
Rotating proxies are implemented as anonymous proxy servers that completely hide the real IP address of the proxy user.
Rotating proxies options
Various rotating proxy options are available, and several are described below. If you know others, feel free to present them in the comments below the article.
Rotating proxy software
EliteProxySwitcher is a complete Windows application that can be used for proxy testing, downloading proxy lists, easily maintaining personal proxy lists, and quickly change the proxy settings of a web browser. The last feature is doubled by the possibility of changing the proxy settings at regular intervals.
Here is how to achieve IP rotation with EliteProxySwitcher while using a web browser:
- Download and install EliteProxySwitcher.
- Open Elite Proxy Switcher and add your proxies. You may add them one-by-one, copy-paste them from a proxy list, or import a proxy list. For this tutorial, I used proxies provided by InstantProxies that I simply copied from the Control Panel and paste them into the EliteProxySwitcher interface.
- After you have the proxy network loaded, select the ones you want to automatically rotate. Next, click on Switch > Use automatic switching and the IPs will start rotating.
- The default interval for the automatic IP change is set to 15 seconds. You may change it from Switch > Proxy switching settings it to any desired value.
- The proxy rotation is set up! Every 15 seconds your browser proxy settings will be changed to a proxy from the list.
If you are interested in the EliteProxySwitcher features, here is a video presentation.
Browser extensions
Proxy rotation can also be achieved using proxy browser extensions. One such extension for Chrome is the BP Proxy Switcher and here is how to use it:
- Install the BP Proxy Switcher extension.
- Add your proxies (for this test I used proxies provided by SSLPrivateProxy). You may either use proxies with username&password authentication or with IP authentication. For the SSLPrivateProxy proxies, go to the Client Area and download the file containing the anonymous proxy data (IP:port). Edit the file in order to add the username and password required and the copy-paste the proxies in the BP Proxy Switcher extension.
- From the main screen select Autorotate every …
- Select the rotation interval and the list behavior (shuffle, restart from the top).
- You are all set up!
The BP Proxy Switcher also offers advanced features like User-agent edit, automatic cookie & javascript removal, WebRTC leak protection.
Backconnect rotating proxies
When it comes to rotating proxies, reverse backconnect proxies are always a good idea. A backconnect rotating proxy forwards the web requests to a random proxy from a proxy pool. The important feature: the proxy client is assigned a different IP address every time a proxy connection is established.
Depending on the types of IPs included in the pool of proxies, rotating backconnect proxies can be residential proxies or data center IP proxies.
The main advantage of a backconnect proxy is that there is no need for additional proxy rotation software as the IPs are rotated behind the scene. The proxy user gets access to one or several proxy gateways (the IP or IPs of the backconnect proxy) that can be used directly in the web scraping application or another type of proxy bot.
The disadvantage is related to cost as backconnect proxies are more expensive than regular private dedicated proxies. Also, as they involve important infrastructure costs, only a few proxy providers offer them (e.g., StormProxies, Oxylabs, Luminati, Smartproxy). You may buy proxies from the list of recommended services here.
Rotating proxy API
Building web scraping tools that need a rotating proxy network can be quite challenging. However, proxy APIs simplify the equation as they reduce the implementation of the proxy rotation to simple API calls.
GimmeProxy API
GimmeProxy exposes a JSON REST API that returns a working proxy from a list of free, open proxies. The service can be used for free with several usage limitations, and it returns a random proxy. The paid service starts at €79.99/month and allows:
- API parametrization, including country, anonymity level, speed, proxy protocol;
- Up to 5 concurrent requests per second;
- Unlimited number of monthly requests;
- Unlimited bandwidth.
Private proxies are also available €399.99/month.
The GimmeProxy API can be used from various programming languages. A Node.js wrapper is available on the GimmeProxy site and here is an example of how to use the GimmeProxy API in Python.
SSLPrivateProxy API
SSLPrivateProxy, a premium proxy service that I recently reviewed, offers a similar API. It is available upon request. The getrandomproxy API function returns a random proxy from your proxy list for every API call.
Web scraping services
Web scraping is a term used for collecting information from websites on the Internet. Scraping services are specialized in simplifying the process of scraping data. As web scraping involves proxies to avoid IP bans, these services also handle the proxy rotation.
Scraper API
The ScraperAPI gracefully solves the proxy rotation problem. It uses a proxy pool behind the scene, and all you have to do is to make the call to the desired URL. The Scraper API automatically routes the requests through proxies, solves CAPTCHAs, and returns the page’s HTML.
Crawlera
A similar service is provided by Crawlera. It manages thousands of proxies internally and bypasses IP bans while scraping the web. Residential IPs in 50+ countries with city granularity are available.
Summary
Despite having a high potential for abuse, legal uses of IP rotation far outweighs its negative sides. Scraping and crawling will not be possible without using IP rotation. Plenty of options are available for achieving IP rotation with proxies. Depending on the specific needs rotating proxy applications, browser extensions, backconnect proxies, APIs, or web scrapers may be what you are looking for.
