• Open file “index.js”
• Import the Puppeteer module within the “index.js” file

const puppeteer = require(‘puppeteer’);

• Import the Express framework within the “index.js” file

const express = require(‘express’);

• Instantiate the Express app

const app = express();

• Set our port:

const port = 3000;

The port will be used a bit later when we tell the app to listen to requests.

• Finalized selectors
  

Web Scraper uses CSS selectors to find HTML elements in web pages and extract data from them. When selecting an element the Web Scraper will try to make its best guess of what the CSS selector might be for the selected elements. But you can also write it yourself and test it by clicking “Element preview”.

Selectors = {

    name:’.prod-subtitle’,

    price:’span.push-right:nth-child(1) > strong:nth-child(1)’

}

• Empty JSON object to send to bubble later, when data has been scrapped and stored into this JSON object

let productDetail = {

    name:”,

    price:”

}

We need to keep in mind that Puppeteer is a promise-based library: It performs asynchronous calls to the headless Chrome instance under the hood. Let’s keep the code clean by using async/await. For that, we need to define an async function and put all the Puppeteer code in there.

• Define HTTP Get endpoint to accept requests from bubble server

When a user hits the endpoint with a GET request, the  JSON object, from express” will be returned to the bubble application. We’d like to set it to be on the product page, so the URL for the endpoint is /product:

app.get(‘/product’, async (req, res) => {

• Launch the browser

const browser = await puppeteer.launch()

• Open a new tab

const page = await browser.newPage()

Puppeteer has a newPage() method that creates a new page instance in the browser, and these page instances can do quite a few things. In our scraper() method, you created a page instance and then used the page.goto() method to navigate to the target site

• Pass URL of target site

await page.goto(req.query.url)

• Save scraped data from HTML element’s selector ( name of product ) into JSON object

productDetail.name = await page.$eval(Selectors.name, el=>el.textContent)

• Save scraped data from HTML element’s selector ( price of product ) into JSON object

price_ = await page.$eval(Selectors.price, el=>el.textContent)

“price_” is a temporary variable declared anywhere, above this line of code.

Here, we need to clean data ( price ), as price contains ‘,’ and ‘.’ swapped. 

This is not a mandatory case, but necessary here. 

lett = 0;

price_  = price_.replace(/,/g, match => ++t === 2 ?          ‘.’ : match)
productDetail.price = price_;

• Close browser

await browser.close()

• Send JSON object to bubble as a response

res.json(productDetail);

• let’s start with our clients

app.listen(port, () => {

  console.log(`Example app listening on port ${port}`)

})

• To run the application, open terminal and write command “node index.js” and press enter