Convert live HTML into structured JSON.

Step 1: Obtain Website URL

You need to know the URL of the website from which you want to extract data.

For example, to extract prices for iPhones from the Apple website, the URL would be:
https://www.apple.com/iphone/

Step 2: Create or Generate JavaScript Code

Provide the JavaScript code to run on the website to convert HTML to JSON. The output can be tested using the steps below:

1. Visit the website (https://www.apple.com/iphone/) in your browser such as Chrome or Edge.
2. Use the DevTools feature in Chrome or Edge (e.g., F12 shortcut key) to view the Console.
3. Write the JavaScript code to extract the desired data (e.g., prices for iPhones) and test the results.
4. Once you have the JavaScript code that successfully extracts the data, you can automate this process with the API.

Here is the sample JavaScript code, written by ChatGPT to extract the prices of iPhone:

// Find the product tile elements
const productTiles = document.querySelectorAll('.product-tile');

// Initialize an array to store the product data
const products = [];

// Iterate over each product tile
productTiles.forEach(tile => {
    // Extract product name
    const productName = tile.querySelector('.product-tile-headline').textContent.trim();

    // Extract product price
    const priceElement = tile.querySelector('.product-tile-price');
    const priceText = priceElement.textContent.trim().replace(/\s+/g, ' ');
    const priceRegex = /From\s*(\$\d+(\.\d+)?)(?:\s*or\s*)?(\$\d+(\.\d+)?)?/i;
    const match = priceText.match(priceRegex);

    let price;
    if (match) {
        const fromPrice = match[1];
        const monthlyPrice = match[3] ? match[3] : null;
        price = {
            from: fromPrice,
            monthly: monthlyPrice
        };
    } else {
        price = null;
    }

    // Add product data to the array
    products.push({
        name: productName,
        price: price
    });
});

// Display the results
console.log(JSON.stringify(products, null, 2));

// Return the results that will be included in the API
return products;

The JSON output is shown in the Console window:

Step 3: Create the Add Screenshots API Request

Add Screenshots automates the HTML to JSON conversion process by:

1. Loading a fully functional browser and navigating to the website
2. Running the provided JavaScript code
3. Returning the converted data through the Add Screenshots API

Sample Conversion Request

Here's a sample request that will navigate to the https://www.apple.com/iphone/ website, execute the JavaScript code that has been base64 encoded, and then return JSON data as the response.

POST https://api.addscreenshots.com/screenshots/?apikey=YOUR_API_KEY

{
  "url": "https://www.apple.com/iphone/",
  "json": true,
  "data": {
    "script": "Y29uc3QgcHJvZHVjdFRpbGVzID0gZG9jdW1lbnQucXVlcnlTZWxlY3RvckFsbCgnLnByb2R1Y3QtdGlsZScpOwpjb25zdCBwcm9kdWN0cyA9IFtdOwoKcHJvZHVjdFRpbGVzLmZvckVhY2godGlsZSA9PiB7CiAgICBjb25zdCBwcm9kdWN0TmFtZSA9IHRpbGUucXVlcnlTZWxlY3RvcignLnByb2R1Y3QtdGlsZS1oZWFkbGluZScpLnRleHRDb250ZW50LnRyaW0oKTsKCiAgICBjb25zdCBwcmljZUVsZW1lbnQgPSB0aWxlLnF1ZXJ5U2VsZWN0b3IoJy5wcm9kdWN0LXRpbGUtcHJpY2UnKTsKICAgIGNvbnN0IHByaWNlVGV4dCA9IHByaWNlRWxlbWVudC50ZXh0Q29udGVudC50cmltKCkucmVwbGFjZSgvXHMrL2csICcgJyk7CiAgICBjb25zdCBwcmljZVJlZ2V4ID0gL0Zyb21ccyooXCRcZCsoXC5cZCspPykoPzpccypvclxzKik/KFwkXGQrKFwuXGQrKT8pPy9pOwogICAgY29uc3QgbWF0Y2ggPSBwcmljZVRleHQubWF0Y2gocHJpY2VSZWdleCk7CgogICAgbGV0IHByaWNlOwogICAgaWYgKG1hdGNoKSB7CiAgICAgICAgY29uc3QgZnJvbVByaWNlID0gbWF0Y2hbMV07CiAgICAgICAgY29uc3QgbW9udGhseVByaWNlID0gbWF0Y2hbM10gPyBtYXRjaFszXSA6IG51bGw7CiAgICAgICAgcHJpY2UgPSB7CiAgICAgICAgICAgIGZyb206IGZyb21QcmljZSwKICAgICAgICAgICAgbW9udGhseTogbW9udGhseVByaWNlCiAgICAgICAgfTsKICAgIH0gZWxzZSB7CiAgICAgICAgcHJpY2UgPSBudWxsOwogICAgfQoKICAgIC8vIEFkZCBwcm9kdWN0IGRhdGEgdG8gdGhlIGFycmF5CiAgICBwcm9kdWN0cy5wdXNoKHsKICAgICAgICBuYW1lOiBwcm9kdWN0TmFtZSwKICAgICAgICBwcmljZTogcHJpY2UKICAgIH0pOwp9KTsKCnJldHVybiBKU09OLnN0cmluZ2lmeShwcm9kdWN0cywgbnVsbCwgMik7"
  }
}

Your API key can be found on the API Keys page. Need an API Key? Sign up to get started.
For internal applications, replace YOUR_API_KEY with your own unique API Key.
For public facing websites or hotlinks, generate a signed URL.

Conversion Response

The conversion response is shown below:

{
  "response": {
    "success": true,
    "processed": "2024-05-09T00:18:01.431Z",
    "data": [
      {
        "name": "iPhone 15 Pro",
        "price": {
          "from": "$999",
          "monthly": "$41.62"
        }
      },
      {
        "name": "iPhone 15",
        "price": {
          "from": "$799",
          "monthly": "$33.29"
        }
      },
      {
        "name": "iPhone 14",
        "price": {
          "from": "$699",
          "monthly": "$29.12"
        }
      },
      {
        "name": "iPhone 13",
        "price": {
          "from": "$599",
          "monthly": "$24.95"
        }
      },
      {
        "name": "iPhone SE",
        "price": {
          "from": "$429",
          "monthly": "$17.87"
        }
      }
    ]
  }
}

• data: The JSON returned by the executed JavaScript code.
• processed: The date and time when the request was processed or executed.
• success: A boolean value (True/False) indicating whether the script ran successfully or encountered an error during execution.
• error: Present when an error occurred during the execution of the script. It provides details about the error.
• conversionError: Present when there was an error parsing the result as JSON. It provides details about the parsing error.