Case Study:

RankTank: Crawling SERP Real-Time With 99.8% Success Rate

ranktank logo

Smart proxy solution to crawl Search Engine Result Pages (SERP) real-time

Many online businesses’ success depends on getting traffic from search engines. Being on the first page and at the top of the search results is key to get relevant visitors to your website and grow your business online. However, nowadays it’s not easy to rank first for most of the keywords because the competition is huge. One of our Crawlera customers, RankTank, helps marketers to better optimize their website with free and paid SEO tools.
Try Crawlera Free!
what out customers say
kinzen goals when web scraping and data extraction

RankTank’s goals

RankTank’s goal is to provide reliable SEO tools for individuals and businesses to help their SEO efforts. RankTank’s ability to serve its customers is highly dependent on crawling and getting real-time data from the web. RankTank collects data from search engine result pages (SERP) real-time and then shows the information that the user has requested. Their most popular tools are the Keyword Rank Checker, the RankTank Google Sheet Add-on and Unlinked Brand Mention Finder. These tools are based on crawling SERPs in real-time.
RankTank also tried to maintain an in-house proxy solution. But this wasn’t effective as they spent a ton of time just managing proxies, updating user-agents and dealing with other technical issues. With Crawlera, they spend no time managing proxies anymore.

Challenges

In the initial years, RankTank was able to crawl the web without using any proxies. But then, things changed and they became unable to get the data they needed to serve users. That’s when RankTank developed a need for a reliable proxy solution that enables them to crawl at scale.

They tried many proxy providers in the market. But RankTank found that none of them satisfied their needs. Before starting to use Crawlera, RankTank was struggling with slow response times and failing/banned proxies.
Crawlera saved a ton of hours in terms of development time as well. When RankTank was managing their own proxies in-house they spent a staggering 240 developer hours per month just on proxies. With Crawlera, they spend no time managing proxies anymore.
“We tried every single proxy provider out there, including running our own proxies at a very large scale, but Crawlera gives us the best results by far.”
Jaco van Wyk, Founder @ RankTank

Solution

When RankTank tried Crawlera, they immediately felt a positive difference in terms of speed and success rate. Based on RankTank’s experience, Crawlera is the proxy solution that provides the fastest response times in the market. Before using Crawlera, RankTank’s average response time was 30-60 seconds. With Crawlera it’s 1-5 seconds. Crawlera is not aimed at real-time usage, but for RankTank it still works well and outperforms other proxy services. RankTank also experienced many banned and failed requests. With previous providers, they had a 5-10% failure rate. With Crawlera their success rate is 99.8%.

For RankTank, ensuring a high success rate was also important because they make the requests using a serverless environment. Meaning, the more requests they needed to retry, the more hardware resources they consumed. With Crawlera, their success rate is super high so they don’t have to make unnecessary requests and waste CPU/memory usage.

Business Results

Without Crawlera, RankTank’s service would be a lot slower. Crawlera has been able to minimize response times and enable RankTank to crawl the web at scale, without dealing with proxy issues.
200k proxy requests
200K requests/month
(real-time requests only)




Near perfect proxy success
99.8% success rate


hours saved
240 hours/month saved in development



fast rotating proxy service
1-5 sec average response time


To sum it all up...

With the help of Crawlera, RankTank is able to tackle web scraping and proxy challenges to crawl search engine result pages and provide top-notch SEO tools.

Challenge:
Being able to crawl search engine result pages (SERP) at scale, real-time.

Solution:
Crawlera, the smart proxy network, made it possible to crawl the required website. Crawlera helps RankTank make real-time requests with 99.8% success rate and 1-5 sec average response time, saving 240 development hours per month.
SEO rank monitoring
Crawling SERP
Real-time crawling
Building web scraping based tools
“Now we can actually focus on improving our own service instead of worrying about getting the data.”
Jaco van Wyk, Founder @ RankTank

About RankTank

RankTank provides free and paid SEO tools for marketers, freelancers, small and large companies. They have a free keyword tool, rank checker and brand mention finder, among many others. RankTank serves more than 50000 users on its platform.

Learn more about RankTank >

Learn more about Crawlera and how it can help you to drive innovative products by signing up for a 14-day free trial today:

Try Crawlera free!

We enable over 2,000 companies extract
the data they need every day

web scraping experts

Need data you can rely on?

Tell us about your project or start using our scraping tools today.

© 2010 - 2020 Scrapinghub

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram