Use a proxy with Puppeteer

princepeterhansen

Peter Hansen

Posted on February 8, 2020

Use a proxy with Puppeteer

In this article, I would like to show 2 ways how to use proxy in Puppeteer.



1) Configuring puppeteer LaunchOptions
2) Using proxy API service


For people who prefer watching videos, there is a quick video demonstrating how to use the proxy API service.

For the demo purposes, I will use the
https://whatismyipaddress.com website to see the IP address in the incoming requests.


The website shows information about an incoming request: IP address, country, region, etc..



The “whatismyipaddress.com” website showing information about the incoming request.

We will open this page with puppeteer using different proxies and see how country information is changed depending on proxy server we use.

Below you can see our basic puppeteer script opening ‘whatismyipaddress.com’ pape.

In the below sections we will modify this code snippet to use a proxy server.


const puppeteer = require('puppeteer');

async function run() {
  const browser = await puppeteer.launch({
    headless: false
  });

  const page = await browser.newPage();

  const pageUrl = 'https://whatismyipaddress.com/';

  await page.goto(pageUrl);
}

run();
Enter fullscreen mode Exit fullscreen mode

1) Configuring puppeteer LaunchOptions

In order to use a proxy, we need to modify LaunchOptions object and pass additional property: args specifying IP and port of a proxy server we would like to use.


args: [ '--proxy-server=IP_HERE:PORT_HERE' ]
Enter fullscreen mode Exit fullscreen mode



The modified script will look like this.

const puppeteer = require('puppeteer');

async function run() {
  const browser = await puppeteer.launch({
    headless: false,
    args: [ '--proxy-server=200.73.128.156:3128' ]
  });
  const page = await browser.newPage();

  const pageUrl = 'https://whatismyipaddress.com/';

  await page.goto(pageUrl);
}

run();
Enter fullscreen mode Exit fullscreen mode



Now the ‘whatismyipaddress.com’ website will be opened through the specified proxy ( which is located in Argentina) and we will see different IP and information about the country.


Image showing the ‘whatismyipaddress.com’ opened with puppeteer using a proxy server located in Argentina.


Please note that the life of public proxy servers is short and at the moment of reading this article the server used in the above example might be already dead.

You can find plenty of online resources providing lists with public proxy servers you can use for FREE. However, free doesn’t always mean good and reliable.

Very often public proxies are slow, not reliable and expire pretty quickly. My advice would be to invest in access to premium proxies you can rely on.

2) Using proxy API service

For this example, we don’t need to find proxy servers because API service we going to is doing it internally.

The only thing we need to change is to modify the URL we trying to open.

There are several API proxies available, but for this demo, I’ve decided to use proxybot service because, in my opinion, it is one of the easiest to use.

We simply need to prepend our url with API service url:


https://proxybot.io/api/v1/API_KEY?url=www.your-target-website.com
Enter fullscreen mode Exit fullscreen mode



The modified puppeteer script will look like this:

const puppeteer = require('puppeteer');

async function run() {
  const browser = await puppeteer.launch({ headless: false });
  const page = await browser.newPage();

  const proxy = 'https://proxybot.io/api/v1/API_KEY?url=';
  const url = 'https://whatismyipaddress.com/';
  const pageUrl = proxy + url;

  await page.goto(pageUrl);
}

run();
Enter fullscreen mode Exit fullscreen mode



This approach removes the need to search and maintain proxy servers because the Proxybot will take care of it.

As you can it is very easy to use proxy with the puppeteer. Simply choose an option that suits your needs the best.

I really hope it was useful and helpful for you guys and girls.

PS: Proxybot it just one of the services allowing you to proxy your requests. If you are looking for proxy providers here you can find a list with top proxy providers in 2021.

Happy coding 👨‍💻👩‍💻

💖 💪 🙅 🚩
princepeterhansen
Peter Hansen

Posted on February 8, 2020

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Using Puppeteer with a proxy
puppeteer Using Puppeteer with a proxy

January 29, 2021

Jordan Uses Luminati
luminati Jordan Uses Luminati

October 22, 2020

Use a proxy with Puppeteer
proxy Use a proxy with Puppeteer

February 8, 2020