Website Screenshot Design Patterns
The follow page describes common website screenshot design patterns and a high-level architecture overview of each core component.
- Pattern 1 - Single page, real-time web screenshot
- Pattern 2 - Cached website screenshot
- Pattern 3 - Single page screenshot with website interaction
- Pattern 4 - Multi page website screenshots
- Pattern 5 - Sitemaps and URL list screenshots
- Pattern 6 - Catpure screenshots of an entire domain
- Pattern 7 - Take website screenshots at scheduled intervals
- Pattern 8 - Track website changes with email notification
What are the real-time website screenshots patterns? (Request-Response)
Single Page Website Screenshots
Given a website URL and an API key, a single page request will create a website screenshot in real-time. There are no queues or polling required to wait for a screenshot to be completed in the background. Screen capturing starts immediately and a response containing the screenshot will be returned while you wait using a sync or async connection.
Add Screenshots does not throttle or limit the number of concurrent requests a client can make. Add Screenshots scales across multiple cloud providers to provide a fast, scalable and reliable solution.
Hotlinking using fresh, real-time website screenshots are also supported.
For additional performance gains, consider using screenshot caching.
Pattern 1 - Single page, real-time web screenshot
Pattern 1 involves taking a single real-time screenshot of a webpage in response to a single-page request using an API key, without the need for background processing or queuing, and with support for synchronous or asynchronous connections.
The high-level screenshot architecture is shown below.
- Make a HTTP GET or POST request that contains the URL, your API key and optional configuration options such as storage or request templates
- The Screenshot API will visit the website URL and capture a screenshot
- The screenshot image is returned in a format such as a png, jpeg or JSON (base64 encoded)
Here is an example request on how to take a screenshot with the Screenshot API using the default options:
// Line breaks added for readability
https://api.addscreenshots.com/screenshots
?apikey=YOUR_API_KEY
&url=https://www.youtube.com
Your API key can be found on the API Keys page.
Need an API Key? Sign up to get started.
For internal applications, replace YOUR_API_KEY with your own unique API Key.
For public facing websites or hotlinks, generate a signed URL.
Pattern 2 - Cached website screenshots
The cached website screenshot pattern 2 is similar to pattern 1, but the screenshot is cached based on a specified time-to-live duration. This pattern is ideal for public websites with multiple users who require the same screenshot without the need for re-capturing every time.
Refer to the screenshot caching page for more information.
- Make a HTTP GET request that contains the URL, your API key and the cache duration in seconds
- The Screenshot API look for a cached version of the screenshot in Cloudflare's Content Delivery Network (CDN)
- A screenshot will be taken of the website if the image does not exist in the cache or if the cache expired; otherwise the cached image would be returned to the user
- If a screenshot was created then it would be added to the cache based on the duration specified in seconds
- The screenshot image is returned to the user
Pattern 3 - Single page screenshot with website interaction
As shown in the high-level diagram below, a screenshot request can also include information such as automation steps to interact with the website (e.g. login or search for a flight) before capturing a screenshot. See the website screenshot automation page for more information and request examples.
- Make a HTTP POST request that contains the URL, your API key and automation steps to interact with the website
- The Screenshot API will interact with the website (e.g. filling in forms and clicking on buttons) and capture a screenshot
- The screenshot image is returned in a format such as a png, jpeg or JSON (base64 encoded)
What are the bulk background screenshots with cloud storage patterns?
Background screenshots provides the ability to take large volumes of screenshots in bulk using a single request that can contain multiple web pages, a sitemap or a domain name. The screenshot images will be uploaded to a cloud storage drive that you manage such as an AWS S3 Bucket, Azure Blob Storage Account and more.
A background request must:
- Use the POST method
- Supply storage account details or a storage account template to store screenshots
Unlike real-time screenshots that requires the user to wait for the screenshot to complete, background requests work in a "fire-and-forget" manner, where a success response will be returned immediately while the bulk website screenshots are processed in the background.
The client can monitor their cloud storage account for screenshot images to show up.
The background worker pool is responsible for processing screenshots. The worker pool is highly scalable and allows you to capture more than a thousand websites per minutes (depending on how fast your web pages load, the interaction required and if your website can handle the concurrent load).
- Make a HTTP POST request that contains the URL (multi pages, sitemap, domain), your API key and optional browser, image and automation options
- The Screenshot API will add the screenshot request to a queue; allowing the screenshot to be processing in the background without the client waiting for the work to complete
- A JSON confirmation will be returned that includes the number of screenshots that will be taken
- An auto-scalable pool of workers will process the screenshot request(s)
- The screenshot worker will visit the website and capture the screenshot (persisted in memory, not disk)
- The screenshot worker will write (or email) the screenshot to your cloud storage (e.g. AWS S3, Azure Blobs etc)
Pattern 4 - Multi page website screenshots
Specify many URLs in a single POST request.
The example below shows how to take a multi page screenshot using a single POST request. The screenshot images will be uploaded to an Azure Blob store using a Shared Access Token. The screenshots will be stored in the folder "{domain}" (pattern) with the filename "{filepath}" (pattern).
POST https://api.addscreenshots.com/screenshots/bulk/multi?apikey=YOUR_API_KEY
{
"urls": [
"google.com",
"youtube.com",
"apple.com"
],
"azure": "https://screenshots.blob.core.windows.net/screenshotdemo?st=2020-08-10T07%...fEWvNz6fbdeUenY%3D",
"file": "{filepath}",
"folder": "{domain}"
}
The API response will include the number of screenshots as shown below:
{
"message": "The screenshots are being processed.",
"jobId": "536b69dbefa240d493330ae5de1c0d46",
"screenshots": 3
}
Take multiple screenshots by overriding various properties per screenshot, such as the file name as shown below.
POST https://api.addscreenshots.com/screenshots/bulk/multi?apikey=YOUR_API_KEY
{
"urls": [
{
"url": "google.com",
"file": "hey-google"
},
{
"url": "apple.com",
"file": "hello-apple",
"fullpage": true
},
{
"url": "youtube.com"
}
],
"azure": "https://screenshots.blob.core.windows.net/screenshotdemo?st=2020-08-10T07%...fEWvNz6fbdeUenY%3D",
"file": "{filepath}",
"folder": "{domain}",
"fullpage": false
}
Note that a full page screenshot will be taken of apple.com since the default full page property was overridden for that URL.
Pattern 5 - Sitemaps and URL list screenshots
Specify a sitemap.xml file to capture all of the URLs contained in the sitemap. For example https://www.addscreenshots.com/sitemap.xml
It is also possible to specify a text file (e.g. websites.txt) that contains a list of website URLs to take screenshots of. Note that each website URL must start on a new line.
The response will include the number of URLs read from the sitemap and website list. If the request times-out, verify that the sitemap or list is hosted on a fast server or that the list is not too large (e.g. > 5000 websites) which may cause a read-timeout.
The example below shows how to take screenshots using multiple sitemaps in a single POST request. The screenshot images will be uploaded to an Azure Blob store using a Shared Access Token. The screenshots will be stored in the folder "{domain}" (pattern) with the filename "{filepath}" (pattern).
POST https://api.addscreenshots.com/screenshots/bulk/sitemap?apikey=YOUR_API_KEY
{
"urls": [
"https://www.apple.com/sitemap.xml",
"https://www.cloudflare.com/sitemap.xml"
],
"azure": "https://screenshots.blob.core.windows.net/screenshotdemo?st=2020-08-10T07%...fEWvNz6fbdeUenY%3D",
"file": "{filepath}",
"folder": "{domain}"
}
The API response will include the number of screenshots as shown below:
{
"message": "The sitemap screenshots are being processed.",
"jobId": "7b07194f706e47a7af868bb9b2ff12d7",
"screenshots": 1366
}
Pattern 6 - Capture screenshots of an entire domain
Scrape and crawl and entire domain to take screenshots of. This approach works well for small, fast websites that can be processed within a couple of minutes.
Use a sitemap or list for websites with many pages.
The example below shows how to take screenshots of an entire domain using a single POST request. The screenshot images will be uploaded to an Azure Blob store using a Shared Access Token. The screenshots will be stored in the folder "apple" using the filename "{filepath}" (pattern).
POST https://api.addscreenshots.com/screenshots/bulk/domain?apikey=YOUR_API_KEY
{
"url": "https://www.apple.com",
"azure": "https://screenshots.blob.core.windows.net/screenshotdemo?st=2020-08-10T07%...fEWvNz6fbdeUenY%3D",
"file": "{filepath}",
"folder": "apple"
}
The API response will include the number of screenshots as shown below:
{
"message": "The domain screenshots are being processed.",
"jobId": "233b69dbefa240d493330ae5de1c0d38"
}
The domain request above took less than a minute to process, take screenshots and upload over a 1,000 screenshots.
How can I schedule repeated website screenshot?
Pattern 7 - Take website screenshots at scheduled intervals
Use scheduling to capture screenshots at regular intervals such as weekly, daily, hourly or based on a custom interval (using a CRON notation).
Scheduling is shown below:
- Setup a schedule that will enqueue the request
- The worker pool will process the scheduled request
- Capture the website screenshot
- Upload the screenshot to a cloud storage account based on a customizable file and folder naming conversion
Only WRITE access to the cloud storage account is required for this pattern.
Add Screenshots is using Cloudy Scheduler to schedule the automation of website screenshots.
Pattern 8 - Track website changes with email notification
To extend the scheduling pattern, it is also possible to compare the current screenshot with the previous screenshot to determine if a notable change (based on a configurable % difference) was detected.
Track changes can save a copy of the old screenshot, save the new screenshot and save a diff image to highlight what has changed.
It is also possible to configure an email notification to alert clients when a change was detected. For example, notify me when my competitor updates the pricing page on their website.
READ and WRITE access to the cloud storage account is required for this pattern.
