Generate CuePoints and transcripts for your Mux video
Adapted from the blog post, Build an AI-powered interactive video transcript with Mux Player CuePoints
This Val exposes an HTTP endpoint that takes a Mux Asset ID and a list of speakers and
- Uses Mux's auto-generated captions to generate a CuePoints object for Mux Player
- Uses AssemblyAI for speaker labeling (diarization)
- Uses GPT-4o to format text
Fork it and use it as a foundation for your own interactive video transcript project.
Required environment variables:
- Mux Access token details (
MUX_TOKEN_ID, MUX_TOKEN_SECRET) This endpoint requires an existing Mux asset that's ready with an audio-only static rendition associated with it. You can run this val to create a new one for testing.
- AssemblyAI API key (
ASSEMBLYAI_API_KEY). Get it from their dashboard here
- OpenAI API key (
OPENAI_API_KEY). Get it from their dashboard here
Make a POST request to the Val's endpoint with the following body, replacing the values with your own asset ID and the list of speakers. Speakers are listed in order of appearance.
{
"asset_id": "00OZ8VnQ01wDNQDdI8Qw3kf01FkGTtkMq2CW901ltq64Jyc",
"speakers": ["Matt", "Nick"]
}
This is just a demo, so it's obviously not battle hardened. The biggest issue is that it does this whole process synchronously, so if the any step takes longer than the Val's timeout, you're hosed.
