Log inSign Up
Avatar

mattx

17 public vals
Joined January 11, 2023

gsheet_call

Wrapper around Google Sheets API v4.

Parameters

  • service_account: JSON string containing Google Service Account key
  • sheet_id: Google Sheet ID
  • method: HTTP method to use
  • action: Full URL with https://sheets.googleapis.com/v4/spreadsheets/{spreadsheetId}/ removed
  • data: HTTP request body

Requirements

  • a Google Cloud service account
  • the Google Sheets API v4 enabled in your Google Cloud project
  • the spreadsheet ID (provide it in the sheet_id parameter)

Instructions

  1. Share the spreadsheet with the service account
  2. Make a JSON key for the service account, then set it as a secret. Use the secret for the service_account parameter.
  3. Figure out the action you want to perform. You will need to provide everything that comes after {spreadsheetId}/ as the action parameter. For example: values/A1:C1:append?valueInputOption=RAW
  4. Figure out the request body. For example: {values: [["foo", "bar", "baz"]]}
Fork
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import { fetchJSON } from "https://esm.town/v/stevekrouse/fetchJSON?v=41";
export const gsheet_call = async (service_account, sheet_id, method, action, data) => {
const { getToken } = await import(
"https://deno.land/x/google_jwt_sa@v0.2.3/mod.ts"
);
const googleAuthOptions = {
scope: ["https://www.googleapis.com/auth/spreadsheets"],
};
const token = await getToken(service_account, googleAuthOptions);
const result = fetchJSON(
`https://sheets.googleapis.com/v4/spreadsheets/${sheet_id}/${action}`,
{
method,
headers: {
Authorization: `Bearer ${token.access_token}`,
},
body: (method == "GET" || method == "HEAD") ? undefined : JSON.stringify(data),
},
);
return result;
};
235
4

verify_discord_signature

Verify HTTP signatures coming from Discord.

  • public_key should be the Ed25519 public key from Discord, as a hex string
  • body should be the request body as a string. If you have a JSON object as the request body, use JSON.stringify.
  • signature should be the X-Signature-Ed25519 header
  • timestamp should be the X-Signature-Timestamp header You must return a 401 error and return early if this function returns false, otherwise you will pretty quickly get a big scary warning from Discord that your endpoint has been removed. Note that you'll only be able to add one once you've integrated this correctly.

As this function only deals with strings, it doesn't matter whether you use an Express or web endpoint.

Fork
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
export let verify_discord_signature = async (
public_key: String,
body: any,
signature: String,
timestamp: any, // ugly hack to allow concatenation
) => {
const encoder = new TextEncoder();
const fromHexString = (hexString) => Uint8Array.from(hexString.match(/.{1,2}/g).map((byte) => parseInt(byte, 16)));
const nacl = await import("https://cdn.skypack.dev/tweetnacl@v1.0.3?dts");
// This used to use node.js buffers...
const result = nacl.sign.detached.verify(
new Uint8Array(encoder.encode(timestamp + body)),
fromHexString(signature),
fromHexString(public_key),
);
return result;
};
70
1
Fork
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import { gsheet_call } from "https://esm.town/v/mattx/gsheet_call";
// Appending to a sheet
await gsheet_call(
Deno.env.get("google_sa"),
"1LDgOhO6Fxg2wt5rGFH29t6XYbmKe6fXI7fLSFaqZkDA",
"POST",
"values/A1:C1:append?valueInputOption=RAW",
{ values: [[Date(), Math.random(), 1]] },
);
// Reading from the top of the sheet
const data = await gsheet_call(
Deno.env.get("google_sa"),
"1LDgOhO6Fxg2wt5rGFH29t6XYbmKe6fXI7fLSFaqZkDA",
"GET",
"values/A1:C1?majorDimension=ROWS",
{},
);
export const gsheet_append_example = data["values"][0];
0

parse_cookies

Parses Cookie headers into objects.

  • jar: Value of the Cookie header
  • decoder: Function to run on all cookie names and values. This is to get around character limitations (see RFC 6265). There is no formal standard, but as most sites prefer URL encoding, it is the default. x => x can be used as a way to disable decoding.
Fork
1
2
3
4
5
6
export function parse_cookies(
jar: String,
decoder: (data: string) => string = decodeURIComponent,
): { [key: string]: string } {
return Object.fromEntries(jar.split("; ").map(c => c.split("=").map(x => decoder(x.trim()))));
}
1
0

remind_new_open_llm

This val sends you an email whenever a new LLM appears in the top 10 LLMs on the Open LLM Leaderboard. Use an interval of at least 12 hours as the top 10 doesn't change that frequently. You can adjust the array slice to change how many places on the leaderboard to monitor, or remove it entirely to fill up your mailbox.

Bugs

If a model unfairly gets to the top 10 and then gets flagged for it, you'll get a reminder for an LLM that's in most cases not new at all. Hopefully this doesn't happen too often.

Fork
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import { email } from "https://esm.town/v/std/email?v=9";
import { set } from "https://esm.town/v/std/set?v=11";
import { last_open_llm_top_10 } from "https://esm.town/v/mattx/last_open_llm_top_10";
import { scrape_open_llm_leaderboard } from "https://esm.town/v/mattx/scrape_open_llm_leaderboard";
export const remind_new_open_llm = async () => {
const data = (await scrape_open_llm_leaderboard()).slice(
0,
10,
);
if (!last_open_llm_top_10) {
console.log("First run, saving data");
await set("last_open_llm_top_10", data);
return;
}
if (
JSON.stringify(last_open_llm_top_10) != JSON.stringify(data)
) {
await email({
html: `<ul>${
data.filter((x) => !last_open_llm_top_10.includes(x)).map(
(x) =>
"<li>" + x + "</li>"
).join("")
}</ul>`,
subject: "New open LLMs",
});
await set("last_open_llm_top_10", data);
}
};
0

scrape_open_llm_leaderboard

This uses scrape_gradio_data to scrape the Open LLM Leaderboard and grab the names of all models that are not flagged. Array is ordered by average benchmark accuracy.

Fork
1
2
3
4
5
6
7
8
9
10
import { scrape_gradio_data } from "https://esm.town/v/mattx/scrape_gradio_data";
export let scrape_open_llm_leaderboard = async () => {
const result = await scrape_gradio_data(
"https://huggingfaceh4-open-llm-leaderboard.hf.space/",
);
return result["components"][21]["props"]["value"]["data"].filter((x) =>
!x[1].includes("has been flagged!")
).map((x) => x[7]);
};
2
0

scrape_gradio_data

Scrapes the gradio_config object from Gradio sites/spaces. Most useful when pointed at a gradio site with lots of static data like the Open LLM Leaderboard

Fork
1
2
3
4
5
6
7
8
9
10
import { fetchText } from "https://esm.town/v/stevekrouse/fetchText?v=5";
export const scrape_gradio_data = async (url) => {
const cheerio = await import("npm:cheerio@1.0.0-rc.12");
const html = await fetchText(url);
const $ = cheerio.load(html);
// This is the part most likely to break:
const blob = $("script:nth-of-type(2)").first().text().slice(23, -1);
return JSON.parse(blob);
};
3
0
Fork
1
2
// set by mattx.write_example1 at 2023-06-07T19:10:23.195Z
export let example1 = 42;
0
Fork
1
2
3
4
5
let { example1 } = await import("https://esm.town/v/mattx/example1");
export const write_example1 = async () => {
example1 = 42;
};
1
0
Fork
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import process from "node:process";
export const airtable_deno_sample = (async () => {
const { Airtable } = await import(
"https://deno.land/x/airtable@v1.1.1/mod.ts"
);
const airtable = new Airtable({
apiKey: process.env.airtable_pat,
baseId: "appXSrKDlwbAijRmD",
tableName: "All content",
});
// Sample data from: https://blog.airtable.com/database-vs-spreadsheet/
const results = await airtable.select();
return results;
})();
0