/** * This microservice implements a high-performance dimensionality reduction API using UMAP. * It uses the umap-js library for efficient UMAP computation and implements caching for improved performance. * The service accepts POST requests with JSON payloads containing high-dimensional embeddings and configuration parameters. * It returns 2D coordinates as the result of dimensionality reduction. * * Key features: * - UMAP implementation for dimensionality reduction * - Caching of results for identical inputs * - Error handling for oversized inputs and timeout scenarios * - Configurable parameters for UMAP algorithm * * This service is particularly useful for: * 1. Visualizing high-dimensional data in 2D space * 2. Reducing dimensionality of large datasets for machine learning tasks * 3. Exploring relationships and clusters in complex data * * Common use cases include: * - Visualizing word embeddings or document vectors in NLP tasks * - Analyzing gene expression data in bioinformatics * - Exploring customer segmentation in marketing analytics * - Visualizing image embeddings in computer vision tasks */ import { crypto } from "https://deno.land/std@0.190.0/crypto/mod.ts"; import { UMAP } from "https://esm.sh/umap-js@1.3.3"; // Constants const MAX_POINTS = 10000; const MAX_DIMENSIONS = 1000; const TIMEOUT_MS = 30000; // 30 seconds // Cache for storing results const resultCache = new Map(); export default async function server(request: Request): Promise { if (request.method === "GET") { return new Response(html, { headers: { "Content-Type": "text/html" }, }); } if (request.method !== "POST") { return new Response("Method Not Allowed", { status: 405 }); } try { const { embeddings, config } = await request.json(); // Input validation if (!Array.isArray(embeddings) || embeddings.length === 0) { return new Response("Invalid input: embeddings must be a non-empty array", { status: 400 }); } if (embeddings.length > MAX_POINTS) { return new Response(`Input too large: maximum ${MAX_POINTS} points allowed`, { status: 413 }); } if (embeddings[0].length > MAX_DIMENSIONS) { return new Response(`Input too high-dimensional: maximum ${MAX_DIMENSIONS} dimensions allowed`, { status: 413 }); } // Generate cache key const encoder = new TextEncoder(); const data = encoder.encode(JSON.stringify({ embeddings, config })); const hashBuffer = await crypto.subtle.digest("MD5", data); const hashArray = Array.from(new Uint8Array(hashBuffer)); const cacheKey = hashArray.map(b => b.toString(16).padStart(2, "0")).join(""); // Check cache if (resultCache.has(cacheKey)) { return new Response(JSON.stringify(resultCache.get(cacheKey)), { headers: { "Content-Type": "application/json" }, }); } // UMAP configuration const umap = new UMAP({ nComponents: 2, nEpochs: 400, nNeighbors: config.nNeighbors || 15, minDist: config.minDist || 0.1, spread: config.spread || 1.0, }); // Perform UMAP with timeout const result = await Promise.race([ umap.fit(embeddings), new Promise((_, reject) => setTimeout(() => reject(new Error("Computation timed out")), TIMEOUT_MS)), ]); // Cache the result resultCache.set(cacheKey, result); return new Response(JSON.stringify(result), { headers: { "Content-Type": "application/json" }, }); } catch (error) { console.error("Error:", error); return new Response(`Error: ${error.message}`, { status: 500 }); } } // Expanded HTML interface with more explanations and examples const html = ` UMAP Dimensionality Reduction API

UMAP Dimensionality Reduction API

This is a high-performance dimensionality reduction microservice using UMAP (Uniform Manifold Approximation and Projection). It provides an efficient way to reduce high-dimensional data to 2D or 3D representations, making it easier to visualize and analyze complex datasets.

When to Use This Service

Visualizing high-dimensional data in 2D or 3D space
Reducing dimensionality of large datasets for machine learning tasks
Exploring relationships and clusters in complex data
Preprocessing step for other machine learning algorithms

Common Use Cases

Visualizing word embeddings or document vectors in NLP tasks
Analyzing gene expression data in bioinformatics
Exploring customer segmentation in marketing analytics
Visualizing image embeddings in computer vision tasks

How to Use

Send a POST request to this endpoint with your data to use the API. The request should include an array of embeddings and optional configuration parameters.

Basic Example:

fetch("https://ejfox-umap.web.val.run/", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
        embeddings: [[1,2,3], [4,5,6], [7,8,9]],
        config: { nNeighbors: 15, minDist: 0.1, spread: 1.0 }
    })
})
.then(response => response.json())
.then(data => console.log(data))
.catch(error => console.error('Error:', error));

Curl Example:

        curl -X POST -H "Content-Type: application/json" -d '{"embeddings": [[1,2,3], [4,5,6], [7,8,9]], "config": { "nNeighbors": 15, "minDist": 0.1, "spread": 1.0 }}' https://ejfox-umap.web.val.run/

Example with OpenAI Embeddings:

This example shows how to use the UMAP service with OpenAI embeddings:

// First, generate embeddings using OpenAI API
import { OpenAI } from "https://esm.town/v/std/openai";
const openai = new OpenAI();

async function getEmbeddings(texts) {
    const response = await openai.embeddings.create({
        model: "text-embedding-ada-002",
        input: texts,
    });
    return response.data.map(item => item.embedding);
}

// Then, use these embeddings with the UMAP service
const texts = ["Hello world", "OpenAI is amazing", "UMAP reduces dimensions"];
const embeddings = await getEmbeddings(texts);

fetch("/", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
        embeddings: embeddings,
        config: { nNeighbors: 15, minDist: 0.1, spread: 1.0 }
    })
})
.then(response => response.json())
.then(data => console.log(data))
.catch(error => console.error('Error:', error));

Live Demo

Click the button below to see a live demo with random data:

View source: Val Town