Mux + LLMs = @mux/ai
Video infrastructure for AI workflows
Mux works seamlessly as the video foundation for your AI tools through to final end user video delivery.
Start buildingRead our docs
Turn your video into AI-ready primitives to build workflows at scale. Mux bridges raw video and AI workflows with developer-first infrastructure that handles video complexity while exposing AI-ready video primitives.
import { translateAudio } from "@mux/ai/workflows";
// Create dubbed audio and upload to Mux
// Uses default audio track, language auto-detected
const result = await translateAudio(
"your-mux-asset-id",
"es", // target language
{
provider: "elevenlabs",
numSpeakers: 0 // Auto-detect speakers
}
);
console.log(result.dubbingId); // ElevenLabs dubbing job ID
console.log(result.uploadedTrackId); // New Mux audio track ID
console.log(result.presignedUrl); // S3 audio file URLimport { generateChapters } from "@mux/ai/workflows";
const result = await generateChapters("your-mux-asset-id", "en", {
provider: "openai"
});
console.log(result.chapters); // Array of {startTime: number, title: string}
// Use with Mux Player
const player = document.querySelector("mux-player");
player.addChapters(result.chapters);import { getSummaryAndTags } from "@mux/ai/workflows";
const result = await getSummaryAndTags("sMP4ES801fUTrIwceiqvbZxrzuzOfI00P3", {
tone: "normal" // or "sassy" or "professional"
});
console.log(result.title); // "Mux thumbnails demo"
console.log(result.description); // "An individual humorously struggles with traditional thumbnail methods..."
console.log(result.tags); // ["video streaming", "thumbnails", "tutorial", "images", "mux"]import { getModerationScores } from "@mux/ai/workflows";
const assetId = "sMP4ES801fUTrIwceiqvbZxrzuzOfI00P3";
const result = await getModerationScores(assetId, {
provider: "openai", // or "hive"
thresholds: {
sexual: 0.7, // Flag content with 70%+ confidence
violence: 0.8 // Flag content with 80%+ confidence
}
});
console.log(result.exceedsThreshold); // true if content flagged
console.log(result.maxScores.sexual); // Highest sexual content score
console.log(result.maxScores.violence); // Highest violence scoreimport { generateVideoEmbeddings } from "@mux/ai/workflows";
// Token-based chunking
const result = await generateVideoEmbeddings("your-mux-asset-id", {
provider: "openai",
chunkingStrategy: {
type: "token",
maxTokens: 500,
overlap: 100
}
});
console.log(result.chunks); // Array of chunk embeddings with timestamps
console.log(result.averagedEmbedding); // Single embedding for entire video
// Store chunks in vector database for timestamp-accurate search
for (const chunk of result.chunks) {
await vectorDB.insert({
id: `${result.assetId}:${chunk.chunkId}`,
embedding: chunk.embedding,
startTime: chunk.metadata.startTime,
endTime: chunk.metadata.endTime
});
}
Translation & dubbing
Generate new language tracks of your video’s audio using AI. Then, create translated versions of your video with multi-track playback that automatically switches languages based on viewer preferences.
Chaptering
Turn your long-form videos into navigable content using AI. Automatically analyze your video content to identify natural breakpoints, create descriptive chapter titles, and add timestamps to help your viewers find what they need.
Summarization & tagging
Intelligently analyze and summarize the content of your video through AI processing of your storyboards and transcripts. Automatically generate summaries, contextual tags, and metadata to ensure content organization and discoverability.
Content moderation
Safeguard your platform using AI to detect policy violations. Video frames pass through your moderation flow to be scanned for violence, nudity, or prohibited content. Flagged videos are held, edited, or rejected before reaching your audience.
Recommendation engine
Build content-based recommendations by converting video transcripts into AI embeddings. Perform a nearest neighbor search to surface relevant content. Enhance discovery by matching videos based on content similarity, not just tags or categories.
Building beyond: your AI workflows, your way
Access your video's AI-ready primitives to build your next workflow. Mux’s video infrastructure powering today’s AI workflows of translation, moderation, and clipping, is the same foundation developers will use to create tomorrow’s innovations.
Start using your video’s AI-ready primitives. Build something new.
Create a Mux accountDeveloper friendly video API
Upload and retrieve videos, create clips or monitor video delivery all from an API call. DevEx is core to our product philosophy. We take pride in great docs, tooling, and best-in-class developer support.
Video experts
We serve billions of streams for thousands of customers. Mux is battle-hardened having handled everything from volume spikes to Internet outages, so your mission-critical streams are safe with us.
Affordable, flexible pricing
Start for free with no credit card required. Pay only for what you use with transparent usage-based pricing. Receive automatic discounts as usage grows and see your costs decrease as you scale.
Trusted by top brands
No credit card required to start using Mux.