Website Data Monitoring

Applies to:   Pro Plan  Enterprise Plan

Add Screenshots provides businesses with the ability to monitor and track website changes over time, allowing them to stay ahead of competitors and ensure that their website is functioning properly. By converting HTML to JSON and storing and monitoring the data for changes, businesses can easily track changes, perform audits, and conduct automated testing, leading to improved decision-making and increased customer satisfaction.

Use Cases for Monitoring and Tracking Website Changes

There are many use cases for tracking website changes using HTML to JSON conversion and monitoring, such as:

  • Keep track and watch out for price changes or discounts on a product you're interested in buying
  • Stay on top of your competitors by monitoring their website and stay informed when they make a change
  • Stay in touch with your customers by getting notified whenever new reviews are posted about your business or service online
  • Monitor product pages for items that are frequently out of stock and get notified when they become available
  • Monitor websites of favorite writers, filmmakers, or musicians and be notified when new content is released
  • Book appointments by monitoring popular websites and being the first one to know whenever a slot opens up

How to Monitor a Website for Changes

There are two ways to evaluate a webpage for changes:

  • Scheduled: Define an interval to regularly monitor a website (e.g., hourly, daily, weekly).
  • On-demand: Send a web request to check for changes immediately (e.g., convert HTML to JSON and monitor data).

Comparing Scheduled with Track Changes Data

Scheduled data monitoring is ideal when you'd like to capture and store data at regular intervals. On the other hand, track changes data monitoring is suitable when you'd like to capture and store changed data at regular intervals and receive notifications when a change is detected.

Using the approach above to track website changes, it is possible to achieve:

  • Receive notifications with the current and previous data when a change is detected.
  • Determine when and how often a competitor updates their data.
  • Track what has changed at a particular time.
  • Show the latest data using a fixed file path.

For example, capture website data of your competitor's pricing page may produce the following files:

  • pricing 2020-08-01.json
  • pricing 2020-08-02.json
  • ...
  • pricing 2020-08-31.json

Scheduled data changes are great when you'd like to capture and store changed data at a regular interval and receive an email notification when a change was detected. For example, capturing your competitor's pricing on a daily basis may produce the following files:

  • pricing.json (last version)
  • pricing 2020-08-07.json (send email notification)
  • pricing 2020-08-25.png (send email notification)

Using the approach above to track slow changing website data, it is possible to achieve the following:

  1. Receive an email notification with the current and previous data when a change was detected
  2. Determine when and how often a competitor updates their prices (e.g. twice this month)
  3. Save on storage by only keeping JSON data files when a change was detected
  4. Show the data of the latest prices using a fixed file path (pricing.json)

A comparison between scheduled and track data changes:

DescriptionScheduledTrack Changes
Run at scheduled intervals
Only store changed data
Track when data was changed
Track what data changed
Send an email notification when a change was detected

Track website changes on demand

The following request properties are available on the data object in the payload:

ParameterRequiredDescription
fileRequiredThe name or pattern (without the extension) of the JSON file.
Daily Example: use {filepath}_{date} to save the date when the change was detected.
Time Example: use {filepath}_{datetime} to save the date and time when the change was detected.
folderOptionalThe folder name or pattern to store JSON files to
Example: use track-changes to save the JSON file in the "track-changes" folder.
prettyOptionalSet to true for pretty print formatting of JSON data
imageOptionalSet to true to add a screenshot image
track_changesOptionalEnable track changes by setting it to true
old_fileRequiredThe name or pattern (without the extension) to rename the old JSON file to.
Daily Example: use {filepath}_{date} to save the date when the change was detected.
Time Example: use {filepath}_{datetime} to save the date and time when the change was detected.
old_folderOptionalThe folder name or pattern to store old JSON files to
Same Folder Example: use {domain} to save the old JSON file in the "example.com" folder.
Archive Folder Example: use {domain}_{date} to save the old JSON file in the "example.com_2020-08-25" folder based on when the change was detected.
change_emailOptionalThe email address to send change notification and the new and old JSON files to.
Use a semicolon ; separated list for multiple recipients such as:
[email protected]; [email protected]; [email protected]
change_email_subjectOptionalThe subject of the email. Defaults to: "Data change detected"

On-Demand Track Changes POST example

The example below is using a comprehensive POST request with:

  • Azure storage account (azure)
  • Storage file name patterns (old_folder, old_file)
  • Change tracking enabled
  • Email notification enabled
POST https://api.addscreenshots.com/screenshots/?apikey=YOUR_API_KEY

{
  "url": "https://example.com/",
  "azure": "https://screenshots.blob.core.windows.net/screenshotdemo?st=2020-08-10T07%...fEWvNz6fbdeUenY%3D",
  "data": {
    "script": "bGV0..Base64-Encoded-JavaScript...",
    "image": false,
    "pretty": true,
    "track_changes": true,
    "change_email": "[email protected];[email protected]",
    "change_email_subject: "The Data Changed",
    "file": "current",
    "folder": "json-files",
    "old_file": "{date}",
    "old_folder" "history"
  }
}

Your API key can be found on the API Keys page. Need an API Key? Sign up to get started.
For internal applications, replace YOUR_API_KEY with your own unique API Key.
For public facing websites or hotlinks, generate a signed URL.