Log inSign up
vipshek avatar
Public
Cron
Fork
1 Like
Readme

Detect New Website Contents

This val fetches a given publicly-accessible URL and detects whether its contents have changed. If they have, it sends an email to notify about the change.

Changes are detected by computing a quick hash of the website's contents, storing the hash, and comparing against the previously stored hash on each request. Trivial changes to the website's contents will count as a change, which may not be desired for some use cases.

Usage

  1. Fork this val and update the URL to point to your website.
  2. Optionally, update the storage key or email subject.
  3. Set a schedule for how often you want to detect changes (default 1h).
  4. Receive updates when the given website changes.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
import { blob } from "https://esm.town/v/std/blob";
import { email } from "https://esm.town/v/std/email?v=9";
import shortHash from "npm:short-hash";
const URL = "https://www.vipshek.com/blog/product-engineer";
const KEY = "website-contents-hash";
const SUBJECT = "New contents detected";
export const detectNewWebsiteContents = async () => {
// Fetch contents
let contents;
try {
const res = await fetch(URL, { redirect: "follow" });
if (res.status !== 200) {
console.log(`bad status code: ${res.status}`);
}
contents = await res.text();
} catch (e) {
console.log(`couldn't fetch: ${e}`);
}
// Derive hashes and compare
const hash = shortHash(contents);
const previousHash = await blob.getJSON(KEY);
if (!previousHash) {
console.log(`no previous hash found. storing hash ${hash}`);
await blob.setJSON(KEY, hash);
} else if (hash !== previousHash) {
console.log(`new hash found (prev: ${previousHash}, new: ${hash}). updating storage and emailing.`);
await blob.setJSON(KEY, hash);
await email({
subject: SUBJECT,
text: `New website contents were detected at ${URL}`,
});
} else {
console.log(`found matching hash ${hash}`);
}
};
Val Town is a social website to write and deploy JavaScript.
Build APIs and schedule functions from your browser.
Sign up with GitHub
Comments
Nobody has commented on this val yet: be the first!
July 5, 2024