Log inSign up
mattx avatar
Public
Script
Fork
0 Likes
Readme

remind_new_open_llm

This val sends you an email whenever a new LLM appears in the top 10 LLMs on the Open LLM Leaderboard. Use an interval of at least 12 hours as the top 10 doesn't change that frequently. You can adjust the array slice to change how many places on the leaderboard to monitor, or remove it entirely to fill up your mailbox.

Bugs

If a model unfairly gets to the top 10 and then gets flagged for it, you'll get a reminder for an LLM that's in most cases not new at all. Hopefully this doesn't happen too often.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import { email } from "https://esm.town/v/std/email?v=9";
import { set } from "https://esm.town/v/std/set?v=11";
import { last_open_llm_top_10 } from "https://esm.town/v/mattx/last_open_llm_top_10";
import { scrape_open_llm_leaderboard } from "https://esm.town/v/mattx/scrape_open_llm_leaderboard";
export const remind_new_open_llm = async () => {
const data = (await scrape_open_llm_leaderboard()).slice(
0,
10,
);
if (!last_open_llm_top_10) {
console.log("First run, saving data");
await set("last_open_llm_top_10", data);
return;
}
if (
JSON.stringify(last_open_llm_top_10) != JSON.stringify(data)
) {
await email({
html: `<ul>${
data.filter((x) => !last_open_llm_top_10.includes(x)).map(
(x) =>
"<li>" + x + "</li>"
).join("")
}</ul>`,
subject: "New open LLMs",
});
await set("last_open_llm_top_10", data);
}
};
Val Town is a social website to write and deploy JavaScript.
Build APIs and schedule functions from your browser.
Sign up with GitHub
Comments
Nobody has commented on this val yet: be the first!
October 23, 2023