postpostscript
he/him
Public vals
54
postpostscript
expiringBlob
Script
expiringBlob: create-and-forget blobs using UUIDv7 How it Works This tool uses UUIDv7s in blob keys with UNIX timestamps included to keep track of when they expire.
Since UUIDv7s are sortable and the blob list endpoint returns results alphabetically, cleanup (checking the blob list and deleting any expired keys) is efficient since any expired keys will be at the front of the results! Each ID is prefixed with a prefix (default: expireBlob: ) and can be suffixed with a label (parameter order depends on the method).
Example blob keys: expiringBlob:01944432-4837-7a99-94e6-e0e162554e21
^Prefix ^UUIDv7
expiringBlob:01944432-4837-7a99-94e6-e0e162554e21:aaaa
^Prefix ^UUIDv7 ^Label Methods createKey : generate a key you can use with @std/blob methods set or setJSON : generate a blob key and save content to it in one step cleanup : delete expired blobs. You can run this manually or periodically with a cron val . If using a custom prefix, make sure to pass it to this method so that the correct blob list is searched. By default, createKey , set , and setJSON will create keys that expire in an hour, but you can override this by passing the expireAt: Date | number parameter. For convenience, expireIn is provided with methods to create future dates for each of the following: expireIn.hours(2) expireIn.minutes(5) expireIn.seconds(45) expireIn.ms(500) To request data once you've saved it, use @std/blob's get and getJSON as usual with the key provided by set or setJSON ! Full Example import { cleanup, createKey, expireIn, set, setJSON } from "https://esm.town/v/postpostscript/expiringBlob";
import { blob } from "https://esm.town/v/std/blob";
const aaaa = createKey("aaaa"); // equivalent of createKey("aaaa", expireIn.hours(1))
await blob.setJSON(aaaa, {
x: 1,
});
console.log(aaaa, await blob.getJSON(aaaa));
// expiringBlob:01944432-4837-7a99-94e6-e0e162554e21:aaaa { x: 1 }
console.log(await set("bbbb", "test", expireIn.seconds(1)));
// expiringBlob:019443fb-6106-77f2-ab0c-5142b099c81a:bbbb
console.log(await setJSON("cccc", { x: 1 }, expireIn.seconds(1)));
// expiringBlob:019443fb-6327-733c-9a0b-c8b9ee9faef0:cccc
await cleanup();
// [no logs]
await new Promise((resolve) => setTimeout(resolve, 1000));
await cleanup();
// deleting expired blob: expiringBlob:019443fb-6106-77f2-ab0c-5142b099c81a:bbbb
// deleting expired blob: expiringBlob:019443fb-6327-733c-9a0b-c8b9ee9faef0:cccc Forkable Cron Example Create a cron val with the following code to periodically delete expired blobs: import { cleanup } from "https://esm.town/v/postpostscript/expiringBlob";
export default () => cleanup();
0
postpostscript
interruptibleChain
Script
interruptibleChain: simple interface for pausing and resuming execution chains ⚠️ Moved to GitHub ⚠️ import { $chain } from "https://esm.town/v/postpostscript/interruptibleChain";
const multiply = $chain()
.then(() => 1)
.then((x) => x + 3)
.then(async (x) => {
await new Promise((resolve) => setTimeout(resolve, 1000));
return x * 2;
});
const handle = multiply.init();
console.log(await handle.untilDone());
// { done: true, value: 8, handle: InterruptibleChainHandle { ... } }
const handle2 = multiply.init({
index: 2,
value: 4,
});
await handle2.advance();
console.log(handle2.done, handle2.state);
// true { index: 3, value: 8 }
/**** Interrupts ****/
const handle3 = $chain().then(() => {
throw new Error("failed");
}).init();
await handle3.advance();
console.log(handle3.done, handle3.interrupt);
// false Interrupt { reason: Error: failed, chain: InterruptibleChain { index: 0 } }
console.log(await handle3.untilDone());
// { done: false, interrupt: Interrupt { ... }, handle: InterruptibleChainHandle { ... } }
const handle4 = $chain().then((_, interrupt) => {
return interrupt("call me later");
}).init();
await handle4.advance()
console.log(handle4.done, handle4.interrupt);
// false Interrupt { reason: "call me later", chain: InterruptibleChain { index: 0 } }
0
postpostscript
fetchWorker
Script
fetchWorker: communicate with a worker over a fetch -like interface! Example: import { fetchWorker } from "https://esm.town/v/postpostscript/fetchWorker";
using worker = await fetchWorker({
url: "https://esm.town/v/postpostscript/fetchWorkerExample",
handler: "handler",
});
const res = await worker.fetch("/", {
method: "POST",
body: JSON.stringify({
test: 1,
}),
headers: {
"Content-Type": "application/json",
},
});
console.log(await res.json());
// {
// method: "POST",
// url: "https://7ae81ab0-04cf-485a-ae09-054c4d3be6b3.val.town/",
// text: { test: 1 },
// headers: { "content-type": "application/json" }
// } Full Example Options: url ( string , required): URL that the worker will fetch the handler from handler ( string , defaults to default ): name of export that will execute the request
0
postpostscript
disposable
Script
disposable: wrapper to make an object disposable with the using keyword using feature explanation in TypeScript 5.2 docs Example: import { disposable } from "https://esm.town/v/postpostscript/disposable";
using state = disposable(
{
x: 1,
y: 2,
},
(value) => console.log("state disposed", value),
);
// state disposed { x: 1, y: 2, [Symbol(Symbol.dispose)]: [Function (anonymous)] } or, to use a proxy instead of modifying the original object: using state = disposable(
{
x: 1,
y: 2,
},
(value) => console.log("proxyState disposed", value),
true,
);
// proxyState disposed { x: 1, y: 2 } Full Example
0
postpostscript
workerify
Script
workerify: proxy a module through a worker (with types!) Example: import { workerify } from "https://esm.town/v/postpostscript/workerify";
using mod = await workerify(() => import("https://esm.town/v/postpostscript/workerifyExampleUnsafeLibrary"));
console.log(await mod.x.y()); Full Example: @postpostscript/workerifyExample @postpostscript/workerifyExampleUnsafeLibrary It's limited by what can be sent over [worker].postMessage() , i.e. inputs and outputs will need to be JSONifiable
0
postpostscript
reactiveStateBlob
Script
reactiveStateBlob: wrap blob state in a proxy to autosave it on changes Examples (full example at @postpostscript/reactiveStateBlobExample ) import { reactiveStateBlob } from "https://esm.town/v/postpostscript/reactiveStateBlob"
using state = await reactiveStateBlob({
viewCount: 0,
rows: [] as {
x: number;
y: number;
}[],
});
state.viewCount += 1;
state.rows.push({
x: Math.random(),
y: Math.random(),
}); This infers the key from the name of the val that uses it. To specify it, pass the key option: using state = await reactiveStateBlob({
viewCount: 0,
rows: [] as {
x: number;
y: number;
}[],
}, {
key: 'reactiveStateBlobExample.state',
}); Updating Schema If you want to update the schema, or always verify the state that is pulled from the job, pass a function as the first argument: using state = await reactiveStateBlob((existingState) => {
return {
viewCount: (existingState.viewCount ?? 0) as number,
rows: (existingState.rows ?? []) as {
x: number;
y: number;
}[],
someNewField: (existingState.someNewField ?? "") as string,
}
}) Options using state = await reactiveStateBlob<{
value: number;
}>({
value: 0,
}, {
log: true, // log when saving
key: "blobKey", // blob key to fetch/save to
timeout: 100, // ms, defaults to 10
lock: true, // or LockOptions (see https://www.val.town/v/postpostscript/lock#options)
}) See Also @postpostscript/counter (example at @postpostscript/counterExample )
0
postpostscript
blogSqliteUniversePart2
HTTP
sqliteUniverse: Make SQLite Queries Against Multiple Endpoints in Deno (Val Town) (Part 2) If you haven't already, check out Part 1 for an overview of this system! Examples Example: find my vals that are related to my public tables import { Statement } from "https://esm.town/v/postpostscript/sqliteBuilder";
import { sqliteUniverse } from "https://esm.town/v/postpostscript/sqliteUniverse";
console.log(await Statement`
SELECT v.name AS val, t.name AS "table"
FROM "https://postpostscript-sqlitevals.web.val.run/vals" v
INNER JOIN "https://postpostscript-sqlitepublic.web.val.run/sqlite_schema" t
ON t.name LIKE ('https://postpostscript-sqlitepublic.web.val.run/' || v.name || '%')
AND t.type = 'table'
WHERE v.author_username = 'postpostscript'
`.execute({
sqlite: sqliteUniverse,
})) Since table names matching /^@/ will let you pass val names in the place of their HTTP endpoints, we can write the query as: SELECT v.name AS val, t.name AS "table"
FROM "@postpostscript/sqliteVals/vals" v
INNER JOIN "@postpostscript/sqlitePublic/sqlite_schema" t
ON t.name LIKE ('@postpostscript/sqlitePublic/' || v.name || '%')
AND t.type = 'table'
WHERE v.author_username = 'postpostscript' The result for the latter: [
{
val: "authId",
table: "@postpostscript/sqlitePublic/authIdExampleComments_comment"
},
{
val: "authIdExampleComments",
table: "@postpostscript/sqlitePublic/authIdExampleComments_comment"
}
] Example: query against a backup import { sqliteFromBlob } from "https://esm.town/v/postpostscript/sqliteBackup";
import { Statement } from "https://esm.town/v/postpostscript/sqliteBuilder";
import { defaultPatterns, patterns, sqliteUniverseWithOptions } from "https://esm.town/v/postpostscript/sqliteUniverse";
const sqlite = sqliteUniverseWithOptions({
interfaces: {
patterns: [
...defaultPatterns,
patterns.blob,
],
},
});
console.log(await Statement`
SELECT *
FROM "blob://backup:sqlite:1709960402936/someTable"
`.execute({ sqlite })); Example: query from @std/sqlite and public data simultaneously import { sqliteFromBlob } from "https://esm.town/v/postpostscript/sqliteBackup";
import { Statement } from "https://esm.town/v/postpostscript/sqliteBuilder";
import { defaultPatterns, sqliteUniverseWithOptions } from "https://esm.town/v/postpostscript/sqliteUniverse";
import { sqlite as sqlitePrivate } from "https://esm.town/v/std/sqlite?v=4";
const sqlite = sqliteUniverseWithOptions({
interfaces: {
patterns: defaultPatterns,
fallback({ endpoint, tables }) {
return sqlitePrivate
},
},
});
console.log(await Statement`
SELECT t.*, p.*
FROM privateTable t
JOIN "@example/sqlitePublic/publicTable" p
`.execute({ sqlite })); You could also do it like this to make it more explicit: const sqlite = sqliteUniverseWithOptions({
interfaces: {
exact: {
"@std/sqlite": sqlitePrivate,
},
patterns: defaultPatterns,
},
});
console.log(await Statement`
SELECT t.*, p.*
FROM "@std/sqlite/privateTable" t
JOIN "@example/sqlitePublic/publicTable" p
`.execute({ sqlite })); Next Steps I'd like to make patterns to allow queries against JSON and Val/ESM exports e.g. SELECT * FROM "json://example.com/example.json" or SELECT * FROM "export://@postpostscript/someVal/someExport" but those will have to come later! Another necessary feature for querying against larger databases will be to use the WHERE or JOIN conditions when dumping from them, but this will be more complicated P.S. This has been a super fun project to work on and I hope you find it interesting or inspiring too! Let me know if you find any errors in this post or if you'd like me to expand more on a specific feature, and let me know if you make anything cool with this and I'll make a note of it! P.P.S (😏) this interface is not 100% stable yet, so I would recommend pinning to specific versions if it not breaking is important to you P.P.P.S. Want to serve your own public SQLite endpoints? This is currently the simplest example I have of how to do that: @postpostscript/sqlitePublic
1
postpostscript
blogSqliteUniverse
HTTP
sqliteUniverse: Make SQLite Queries Against Multiple Endpoints in Deno (Val Town) (Part 1) Prerequisite Knowledge Val Town-Hosted SQLite Val Town hosts SQLite as part of its standard library ( @std/sqlite ). This makes a fetch request against their closed-source API (using your API token) which returns results in a consistent format. This is great because you can host your own endpoints that work similarly, and reuse code that was only designed in mind for that original hosted interface The standard format (abridged to important fields): POST /execute {
"statement": {
"sql": "SELECT * FROM some_table",
"args": []
}
} Output: {
"rows": [[1, "first", 1709942400], [2, "second", 1709942401]],
"columns": ["id", "name", "lastModified"]
} POST /batch {
"statements": [
{
"sql": "INSERT INTO some_table VALUES (?, ?, ?)",
"args": [3, "third", 1709942402]
},
{
"sql": "SELECT * FROM some_table",
"args": []
}
]
} Output: [
{
"rows": [],
"columns": ["id", "name", "lastModified"]
},
{
"rows": [[1, "first", 1709942400], [2, "second", 1709942401], [3, "third", 1709942402]],
"columns": ["id", "name", "lastModified"]
},
] SQLite in Wasm There is a deno package (sqlite) which lets you (among other things) create SQLite databases in-memory using WebAssembly. I've created a Val which wraps this to enable it to be a drop-in replacement for @std/sqlite: @postpostscript/sqliteWasm Example import { createSqlite } from "https://esm.town/v/postpostscript/sqliteWasm";
import { Statement } from "https://esm.town/v/postpostscript/sqliteBuilder";
const sqlite = createSqlite();
console.log(sqlite.batch([
Statement`
CREATE TABLE test (
id TEXT PRIMARY KEY,
value TEXT
)
`,
Statement`
INSERT INTO test VALUES (
${"some-id"},
${"some-value"}
)
`,
Statement`
SELECT * FROM test
`,
])) Result: [
{
rows: [],
rowsAffected: 0,
columns: []
},
{
rows: [],
rowsAffected: 1,
columns: []
},
{
rows: [ [ "some-id", "some-value" ] ],
rowsAffected: 0,
columns: [ "id", "value" ]
}
] Dump Tool I have modified @nbbaier's great work at @postpostscript/sqliteDump to support dumping from any sqlite interface, whether the standard library's version, over HTTP, or through the above Wasm implementation Putting it All Together All of the above enables: Serving a subset of your private data publicly for others to query (Example: @postpostscript/sqlitePublic ) Backing up your database and querying against that backup (via @postpostscript/sqliteBackup's sqliteFromBlob and sqliteToBlob ) But we can do more..! What if we could query from multiple of these data sources.. at the same time! 😱 sqliteUniverse sqliteUniverse is an @std/sqlite compatible interface that determines where a table should route to based on different patterns Table Name Patterns The actual table name will always come after a "/", with the exception of tables without any endpoint, for example users . Everything before the last "/" is the endpoint name. Endpoint interfaces will be chosen in the following order: Exact match in options.interfaces.exact e.g. @std/sqlite/someTable would match options.interfaces.exact["@std/sqlite"] Each pattern in options.interfaces.patterns options.interfaces.fallback will be called An error is thrown if none of the above matches AND returns an sqlite interface. If there is a match but the handler returns nothing, it will continue down the list Default options.interfaces.patterns : patterns.https - /^https:\/\// ( https://example.com/somePath/tableName ): fetch from https://example.com/somePath/batch patterns.val - /^@/ ( @author/name/somePath/tableName ): fetch from the val's endpoint, https://author-name.web.val.run/somePath/batch Other Available Patterns: The following patterns are accessible through import { patterns } from "https://esm.town/v/postpostscript/sqliteUniverse" : patterns.blob - /^blob:\/\// ( blob://backup:sqlite:1709960402936 ) - import the database from private blob backup:sqlite:1709960402936 Overriding Default Options The sqliteUniverse export contains defaults insuring no private data will be leaked. If you want to reduce or extend these options, use the sqliteUniverseWithOptions export and pass a modified interfaces option in the first argument: Examples of how to set options.interfaces.exact , options.interfaces.patterns , and options.interfaces.fallback : import { sqliteUniverseWithOptions, patterns, defaultPatterns } from "https://esm.town/v/postpostscript/sqliteUniverse";
import { createSqlite } from "https://esm.town/v/postpostscript/sqliteWasm";
import { sqliteFromAPI } from "https://esm.town/v/postpostscript/sqliteFromAPI";
import { Statement } from "https://esm.town/v/postpostscript/sqliteBuilder";
const sqlite = sqliteUniverseWithOptions({
interfaces: {
exact: {
// `SELECT * FROM "some-endpoint/someTable"` will match
// `SELECT * FROM "some-endpoint/somePath/someTable"` will NOT match
"some-endpoint": ({ endpoint, tables }) => {
const sqlite = createSqlite()
sqlite.batch([
Statement`CREATE TABLE someTable (someField TEXT PRIMARY KEY)`,
Statement`INSERT INTO someTable VALUES (${"some-field"})`
])
return sqlite
},
},
patterns: [
...defaultPatterns,
[
// shorthand e.g. ~/sqlitePublic -> @postpostscript/sqlitePublic
/^~\/(\w+)/,
({ endpoint, tables, match }) => {
return sqliteFromAPI(`@postpostscript/${match[1]}`)
},
]
],
fallback({ endpoint, tables }) {
// if an endpoint is not found, this will be called
return sqliteFromAPI(`@postpostscript/sqlitePublic`)
},
},
})
console.log(await Statement`
SELECT *
FROM "some-endpoint/someTable"
JOIN "~/sqliteVals/vals"
JOIN authIdExampleComments_comment
LIMIT 1
`.execute({ sqlite })) Output: [
{
someField: "some-field",
id: "aeb70bbb-05fc-403b-8d6a-130c423ecb53",
name: "discordWelcomedMembers",
code: "// set at Sun Mar 10 2024 00:32:48 GMT+0000 (Coordinated Universal Time)\n" +
"export let discordWelcomedM"... 8289 more characters,
version: 234771,
privacy: "public",
public: 1,
run_start_at: "2024-03-10T00:32:48.978Z",
run_end_at: "2024-03-10T00:32:48.978Z",
created_at: "2024-03-10T00:32:48.978Z",
author_id: "a0bf3b31-15a5-4d5c-880e-4b1e22c9bc18",
author_username: "stevekrouse",
username: "postpostscript",
comment: "test",
date_added: 1709776325.857
}
] Part 2 Since Val Town currently has a character limit for val readmes, this will have to continue in Part 2 !
1