Search

Uses vector embeddings to get "vibes" search on emojis


import _ from "npm:lodash";

import { searchEmojis } from "https://esm.town/v/maxm/emojiVectorEmbeddings";

import { extractValInfo } from "https://esm.town/v/pomdtr/extractValInfo?v=29";

const app = new Hono();

Uses Val Town's [blob storage](https://docs.val.town/std/blob/) to search embeddings of all vals, by downloading them all and iterating through all of them to compute distance. Slow and terrible, but it works!

- Get metadata from blob storage: `allValsBlob${dimensions}EmbeddingsMeta` (currently `allValsBlob1536EmbeddingsMeta`), which has a list of all indexed vals and where their embedding is stored (`batchDataIndex` points to the blob, and `valIndex` represents the offset within the blob).

- Get all blobs with embeddings pointed to by the metadata, e.g. `allValsBlob1536EmbeddingsData_0` for `batchDataIndex` 0.

const allValsBlobEmbeddingsMeta = (await blob.getJSON(`allValsBlob${dimensions}EmbeddingsMeta`)) ?? {};

const allBatchDataIndexes = _.uniq(Object.values(allValsBlobEmbeddingsMeta).map((item: any) => item.batchDataIndex));

const embeddingsBatches = [];

const embeddingsBatchBlobName = `allValsBlob${dimensions}EmbeddingsData_${batchDataIndex}`;

const promise = blob.get(embeddingsBatchBlobName).then((response) => response.arrayBuffer());

embeddingsBatches[batchDataIndex as any] = data;

console.log(`Loaded ${embeddingsBatchBlobName} (${data.byteLength} bytes)`);

const queryEmbedding = (await openai.embeddings.create({

for (const id in allValsBlobEmbeddingsMeta) {

const meta = allValsBlobEmbeddingsMeta[id];

"https://esm.sh/langchain/chat_models/openai"

const { OpenAIEmbeddings } = await import(

"https://esm.sh/langchain/embeddings/openai"

const { ConversationSummaryMemory } = await import(

[{ id: 2 }, { id: 1 }, { id: 3 }],

new OpenAIEmbeddings({

openAIApiKey: process.env.OPENAI_API_KEY,

import blogPostEmbeddingsDimensionalityReduction from "https://esm.town/v/janpaul123/blogPostEmbeddingsDimensionalityReduction";

export async function semanticSearchBlogPostPlot() {

const Plot = await import("https://cdn.jsdelivr.net/npm/@observablehq/plot@0.6.14/+esm");

{ parseHTML: p },

) => p(`<a>`));

const points = await blogPostEmbeddingsDimensionalityReduction();

const chart = Plot.plot({

document,

Use embeddings / Lunr search on Airtable. Embeddings need to have been generated / stored on Airtable, or this gets very slow / costly.

- Simple usage: https://yawnxyz-buildclubprojectsearch.web.val.run/search?query=cars

<th>Content</th>

<th>embeddingsContent</th>

</tr>

<td>{result.content}</td>

<td>{result.embeddingsContent}</td>

</tr>

const defaultContentColumn = "Content";

const defaultEmbeddingColumn = "Embeddings";

async function fetchAirtableData(baseId, tableName, nameColumn, contentColumn, embeddingColumn) {

embedding: record.get(embeddingColumn).split(",").map(parseFloat),

embeddingsContent: record.get('EmbeddingsContent'),

return documents;

console.log('documents:', documents)

await semanticSearch.addDocuments({documents, fields: 'embeddingsContent'});

const results = await semanticSearch.search({query, similarityThreshold});

Uses [Turso](https://turso.tech/) to search embeddings of all vals, using the [sqlite-vss](https://github.com/asg017/sqlite-vss) extension.

- Call OpenAI to generate an embedding for the search query.

- Query the `vss_vals_embeddings` table in Turso using `vss_search`.

- The `vss_vals_embeddings` table has been generated by [janpaul123/indexValsTurso](https://www.val.town/v/janpaul123/indexValsTurso). It is not run automatically.

- This table is incomplete due to a [bug in Turso](https://discord.com/channels/933071162680958986/1245378515679973420/1245378515679973420).

const sqlite = createClient({

url: "libsql://valsembeddings-jpvaltown.turso.io",

authToken: Deno.env.get("TURSO_AUTH_TOKEN_VALSEMBEDDINGS"),

const openai = new OpenAI();

const embedding = await openai.embeddings.create({

model: "text-embedding-3-small",

sql:

"WITH matches AS (SELECT rowid, distance FROM vss_vals_embeddings WHERE vss_search(embedding, :embeddingBinary) LIMIT 50) SELECT id, distance FROM matches JOIN vals_embeddings ON matches.rowid = vals_embeddings.rowid",

args: { embeddingBinary },

const { url, opts = 'content', crawlMode = 'jina', useCache, getTags, getTagsPrompt, getSummary, getSummaryPrompt, getEmbeddings } = body;

const [summary, tags, embeddings] = await Promise.all([

getEmbeddings ? getEmbeddingsFn(content) : null

return c.json({ summary: summary?.text, tags: tags?.text, content, embeddings });

getSummary = true, getSummaryPrompt, getTags = true, getTagsPrompt, getEmbeddings,

let summary, tags, embeddings

if (getEmbeddings) {

embeddings = await getEmbeddingsFn(content + " " + summary + " " + tags);

embeddings

export const getEmbeddingsFn = async (content) => {

- Mistral: `https://yawnxyz-ai.web.val.run/generate?prompt="tell me a joke?"&provider=mistral&model="mistral-small-latest"`

async function calculateEmbeddings(text) {

"https://esm.sh/langchain/chat_models/openai"

const { OpenAIEmbeddings } = await import(

"https://esm.sh/langchain/embeddings/openai"

const { BufferMemory } = await import("https://esm.sh/langchain/memory");

[{ id: 2 }, { id: 1 }, { id: 3 }],

new OpenAIEmbeddings({

openAIApiKey: process.env.OPENAI_API_KEY,

description: "The length of the movie in minutes",

type: "number",

* Next, we instantiate a vector store. This is where we store the embeddings of the documents.

* We also need to provide an embeddings object. This is used to embed the documents.

const modelBuilder = await getModelBuilder();

const llm = await modelBuilder();

Todo

- Support lunr / semantic search, and embeddings

- Collections that support pointing to multiple blobs, like {description, embeddings, fileblob, ...} with a shared index / lookup

const { SupabaseVectorStore } = await import("npm:langchain/vectorstores");

const { ChatOpenAI } = await import("npm:langchain/chat_models");

const { OpenAIEmbeddings } = await import("npm:langchain/embeddings");

const { createClient } = await import(

"https://esm.sh/@supabase/supabase-js@2"

handleLLMNewToken(token) {

const vectorStore = await SupabaseVectorStore.fromExistingIndex(

new OpenAIEmbeddings({

openAIApiKey: process.env.OPEN_API_KEY,

client,

Uses Val Town's [blob storage](https://docs.val.town/std/blob/) to search embeddings of all vals, by downloading them all and iterating through all of them to compute distance. Slow and terrible, but it works!

This val hosts an [HTTP server](https://janpaul123-valtownsemanticsearch.web.val.run/) that lets you search all vals based on vibes. If you search for "discord bot" it shows all vals that have "discord bot" vibes.

It does this by comparing [embeddings from OpenAI](https://platform.openai.com/docs/guides/embeddings) generated for the code of all public vals, to an embedding of your search query.

This is an experiment to see if and how we want to incorporate semantic search in the actual Val Town search page.

I implemented three backends, which you can switch between in the search UI. Check out these vals for details on their implementation.

- **Neon:** storing and searching embeddings using the [pg_vector](https://neon.tech/docs/extensions/pgvector) extension in Neon's Postgres database.

- Searching: [janpaul123/semanticSearchNeon](https://www.val.town/v/janpaul123/semanticSearchNeon)

- Indexing: [janpaul123/indexValsNeon](https://www.val.town/v/janpaul123/indexValsNeon)

- **Blobs:** storing embeddings in Val Town's [standard blob storage](https://docs.val.town/std/blob/), and iterating through all of them to compute distance. Slow and terrible, but it works!

- Searching: [janpaul123/semanticSearchBlobs](https://www.val.town/v/janpaul123/semanticSearchBlobs)

"https://esm.sh/langchain/vectorstores/hnswlib"

const { OpenAIEmbeddings } = await import(

"https://esm.sh/langchain/embeddings/openai"

const { ConversationalRetrievalQAChain } = await import(

[{ id: 2 }, { id: 1 }, { id: 3 }, { id: 4 }, { id: 5 }],

new OpenAIEmbeddings({

openAIApiKey: process.env.OPENAI_API_KEY,