Web Scraping

Taking a screenshot of your website is very straightforward using ScrapingBee. You can either take a screenshot of the visible portion of the page, the whole page, or an element of the page.
That can be done by specifying one of these parameters with your request:
screenshot to true or false. screenshot_full_page to true or false. screenshot_selector to the CSS selector of the element. In this tutorial, we will see how to take a screenshot of ScrapingBee’s blog using the three methods.

Taking a screenshot of your website is very straightforward using ScrapingBee. You can either take a screenshot of the visible portion of the page, the whole page, or an element of the page.
That can be done by specifying one of these parameters with your request:
screenshot to true or false. screenshot_full_page to true or false. screenshot_selector to the CSS selector of the element. In this tutorial, we will see how to take a screenshot of ScrapingBee’s blog using the three methods.

In this tutorial, we will see how you can log in to any website, using three different methods:
A JavaScript scenario. A POST request. Cookies As an example, we’re going to log into this demo website and take a screenshot of the account page. So make sure to create an account there before you start!
1. Login using a js_scenario: This is the easiest solution among the three, as it mimics the behavior of a normal user.

In this tutorial, we will see how you can log in to any website, using three different methods:
A JavaScript scenario. A POST request. Cookies As an example, we’re going to log into this demo website and take a screenshot of the account page. So make sure to create an account there before you start!
1. Login using a js_scenario: This is the easiest solution among the three, as it mimics the behavior of a normal user.

Nowadays, most websites use different methods and techniques to decrease the load and data served to their clients’ devices. One of these techniques is the infinite scroll.
In this tutorial, we will see how we can scrape infinite scroll web pages using a js_scenario, specifically the scroll_y and scroll_x features. And we will use this page as a demo. Only 9 boxes are loaded when we first open the page, but as soon as we scroll to the end of it, we will load 9 more, and that will keep happening each time we scroll to the bottom of the page.

Nowadays, most websites use different methods and techniques to decrease the load and data served to their clients’ devices. One of these techniques is the infinite scroll.
In this tutorial, we will see how we can scrape infinite scroll web pages using a js_scenario, specifically the& scroll_y and scroll_x features. And we will use this page as a demo. Only 9 boxes are loaded when we first open the page, but as soon as we scroll to the end of it, we will load 9 more, and that will keep happening each time we scroll to the bottom of the page.

Open the Network tab in the Developer Tools Right click (or Ctrl-click or two-finger click) a request Click "Copy as cURL" in the dropdown menu You can now paste it in the relevant curl converter to translate it in the language you want

Open the Network Monitor tab in the Developer Tools Right click (or Ctrl-click) a request Click "Copy" → "Copy as cURL" You can now paste it in the relevant curl converter to translate it in the language you want

Open the Network tab in the DevTools Right click (or Ctrl-click) a request Click "Copy" → "Copy as cURL" You can now paste it in the relevant curl converter to translate it in the language you want

Finding the CSS selector of an element you want to scrape can be tricky at times. This is why we can use the Inspect Element feature in most modern browsers to extract the selector with ease.
The process is very simple, first we find the element, right click on it, we then click on Inspect Element. The developer tools window will show up with the element highlighted. We then right click on the selected HTML code, go to Copy, and click on Copy selector.

In this tutorial, we will see how you can integrate ScrapingBee’s API with NodeJS using our Software Development Kit (SDK), and use it to scrape web pages. As such, we will cover these topics:
Install ScrapingBee’s NodeJS SDK Create your first API request. Let’s get started! 1. Install the SDK Before using an SDK, we will have to install the SDK. And we can do that using this command: npm install scrapingbee.

In this tutorial, we will see how you can use ScrapingBee’s API with PHP, and use it to scrape web pages. As such, we will cover these topics:
General structure of an API request Create your first API request. Let’s get started!
1. General structure of an API request The general structure of an API request made in PHP will always look like this:
<?php // Get cURL resource $ch = curl_init(); // Set base url & API key $BASE_URL = "https://app.