EveryDev.ai
Subscribe
Home
Tools

5,901+ AI tools

  • Featured
  • Gems
  • Trending
  • New
  • Rate tools
  • Compare
  • Arena
Categories
  • Agents2972
  • Coding2248
  • Infrastructure1067
  • Projects818
  • MCP794
  • Research744
  • Analytics565
  • Marketing527
  • Design488
  • Testing476
  • Data442
  • Security412
  • Integration288
  • Communication284
  • Prompts283
  • Extensions238
  • Voice213
  • DevOps190
  • Commerce183
  • Web128
  • Learning123
  • Finance44
AI Tools by Topic
  • AI Coding Assistants
  • Agent Frameworks
  • MCP Servers
  • AI Prompt Tools
  • Vibe Coding Tools
  • AI Design Tools
  • AI Database Tools
  • AI Website Builders
  • AI Testing Tools
  • LLM Evaluations
Follow Us
  • X / Twitter
  • LinkedIn
  • Reddit
  • Discord
  • Threads
  • Bluesky
  • Mastodon
  • YouTube
  • GitHub
  • Instagram
Get Started
  • Users
  • Rate Tools
  • About
  • Editorial Standards
  • Corrections & Disclosures
  • Community Guidelines
  • Advertise
  • Contact Us
  • Newsletter
  • Submit a Tool
  • Start a Discussion
  • Write A Blog
  • Share A Build
  • Terms of Service
  • Privacy Policy
Explore with AI
  • ChatGPT
  • Gemini
  • Claude
  • Grok
  • Perplexity
Agent Experience
  • llms.txt
Theme
With AI, Everyone is a Dev. EveryDev.ai ©
    1. Home
    2. Tools
    3. LatentSync
    LatentSync icon

    LatentSync

    3D and Animation
    Featured

    Open-source lip-sync framework using audio-conditioned latent diffusion models with SyncNet supervision.

    Visit Website

    At a Glance

    Pricing
    Open Source

    Self-hosted LatentSync lip-sync code, inference, data processing and training code released under the Apache License 2.0.

    Engagement

    Available On

    Linux
    API
    CLI

    Resources

    WebsiteDocsGitHubllms.txt

    Topics

    3D and AnimationAcademic ResearchVideo

    Alternatives

    LanceDuix AvatarMoviePy
    Developer
    ByteDanceBeijing, ChinaEst. 2012$9.8B raised

    Listed Nov 2026

    About LatentSync

    LatentSync is an open-source lip-sync method from ByteDance that generates lip-synced video from audio using audio-conditioned latent diffusion, built on Stable Diffusion. The repository provides inference code, checkpoints, a data processing pipeline, and training code. Version 1.6, released 2025/06/11, is trained on 512x512 videos to reduce blurriness.

    What It Is

    LatentSync is an end-to-end lip-sync approach that models audio-visual correlations directly in latent space, without any intermediate motion representation. The README contrasts this with earlier diffusion-based lip-sync methods that use pixel-space diffusion or two-stage generation. Whisper converts the mel-spectrogram into audio embeddings, which are fed into a U-Net through cross-attention layers. The accompanying paper introduces StableSyncNet and a Temporal Representation Alignment (TREPA) mechanism to improve lip-sync accuracy and temporal consistency.

    Running Inference

    Users set up the environment and download checkpoints with a setup script, then generate lip-synced videos through either a Gradio app or a command-line script. Adjustable parameters include inference steps (20-50), where higher values improve visual quality but slow generation, and guidance scale (1.0-3.0), where higher values improve lip-sync accuracy but may cause distortion or jitter. Minimum VRAM for inference is documented as 8 GB for version 1.5 and 18 GB for version 1.6.

    Training and Evaluation Tooling

    The repository includes a data processing pipeline that removes broken videos, resamples video to 25 FPS and audio to 16000 Hz, detects scenes, splits clips, aligns faces, and filters by sync confidence and hyperIQA scores. U-Net training uses several config files with documented VRAM needs from 20 GB to 55 GB, and SyncNet can be trained on custom datasets. Evaluation scripts compute sync confidence scores and SyncNet accuracy.

    Release Updates

    Version 1.5 (2025/03/14) added a temporal layer for better temporal consistency, improved performance on Chinese videos, and lowered stage 2 training VRAM to 20 GB.

    LatentSync - 1

    Community Discussions

    Be the first to start a conversation about LatentSync

    Share your experience with LatentSync, ask questions, or help others learn from your insights.

    Pricing

    OPEN SOURCE

    Open Source (Apache-2.0)

    Self-hosted LatentSync lip-sync code, inference, data processing and training code released under the Apache License 2.0.

    • Apache License 2.0
    • Inference code and checkpoints
    • Data processing pipeline
    • Training code
    • Gradio app and CLI inference

    Capabilities

    Key Features

    • Audio-conditioned latent diffusion lip-sync
    • Whisper audio embeddings via cross-attention
    • StableSyncNet supervision
    • TREPA temporal consistency
    • Gradio app and CLI inference
    • Adjustable inference steps and guidance scale
    • Data processing pipeline
    • U-Net and SyncNet training scripts
    • Sync confidence and SyncNet accuracy evaluation
    • 512x512 resolution model (v1.6)

    Integrations

    Hugging Face
    Replicate
    Whisper
    Stable Diffusion
    Gradio
    API Available
    View Docs

    Ratings & Reviews

    No ratings yet

    Be the first to rate LatentSync and help others make informed decisions.

    Rate other tools you’ve used

    Developer

    ByteDance

    ByteDance is the developer of the TARS framework and UI-TARS Desktop, exploring multi-agent orchestration systems for building applications with natural language.

    Founded 2012
    Beijing, China
    $9.8B raised
    110,000 employees

    Used by

    JD.com
    Vivo
    Geely
    Read more about ByteDance
    WebsiteGitHubX / Twitter
    5 tools in directory

    Similar Tools

    Lance icon

    Lance

    A 3B-parameter open-source unified multimodal model from ByteDance supporting image and video understanding, generation, and editing within a single framework.

    Duix Avatar icon

    Duix Avatar

    Offline AI avatar toolkit for cloning a person's appearance and voice and generating lip-synced videos from text or audio.

    MoviePy icon

    MoviePy

    Open-source Python library for scripted video editing, compositing, and custom effects.

    Browse all tools

    Related Topics

    3D and Animation

    AI-powered tools for creating, manipulating, and animating 3D models, characters, and environments with intelligent rigging, texturing, and motion synthesis.

    99 tools

    Academic Research

    AI tools designed specifically for academic and scientific research.

    111 tools

    Video

    AI tools that generate or edit video — from text-to-video and animation to avatars, dubbing, and short-form clips.

    141 tools
    Browse all topics
    Back to all toolsSuggest an edit
    ratings
    discussions