Video Translation Dubbing

Word count

645 words

Reading time

5 minutes

Simply provide the materials, and it will automatically complete speech recognition, translation, synthesis, and draft export

Preview

Example

Translation

Original Chinese	English

Dubbing

🎯 Features

• 🎤 Subtitle Recognition

  Accurately transcribe voices from videos and audio, with flexible configuration to adapt to different devices and scenarios, ensuring high-quality text generation

• 🌐 Language Translation

  Translate between multiple languages, switch translation providers, and adjust advanced parameters to optimize translation results and overcome language barriers

• 🎧 Speech Synthesis

  A rich library of voices and customization options for personalized dubbing experiences, meeting creative needs with real-time previews for precise creation

• ✂️ Draft Editing

  Export materials to editing tools with multi-dimensional control over visuals, audio, and subtitles, aiding post-production adjustments and customization

• 🧩 Application Components

  Built-in application components for efficient collaboration and flexible usage, catering to diverse user needs

• 🔧 Preview and Debugging

  Flexible and efficient configuration adjustments to ensure perfect presentation at every stage, enhancing creation efficiency and quality

📅 Planned Support

Subtitles

• Multi-Provider Support: Switch between Srt, Capcut, FunAsr, and FasterWhisper for processing.
• Video Download: Automatically download videos from YouTube and TikTok.
• Video-to-Text Transcription: Transcribe audio from videos into text and extract subtitles.
• Multi-Track Separation: Separate multiple audio tracks including vocals, accompaniment, drums, bass, etc.
• Speaker Recognition & Alignment: Extract speaker voiceprints and align audio precisely with subtitle text.
• Multi-Speaker Detection: Automatically identify multiple speakers in a video.
• Emotion Recognition: Recognize emotions such as Angry, Disgusted, Fearful, Happy, Neutral, Other, Sad, and Surprised.

Translation

• Multiple Providers: Support switching between OpenAi, Gemini and DashScope
• Custom Models and Prompt Commands
• Batch Processing for Long Texts

Speech

• Multi-Provider Support: Switch between AzureTTS、EdgeTTS、ElevenLabs、VoxCPM1.5、IndexTTS2、CosyVoice3、GPT-SoVITS4 for voice synthesis.
• Real-Time Voice Preview: Supports real-time voice synthesis and preview for quick evaluation.
• Popular Voices: Provides 559 built-in voices based on AzureTTS and EdgeTTS, with over 895 sample recordings. Common voices include but are not limited to Xiaomei, Xiaoshuai, Dashan, Dazhuang, General Arthur, etc.
• Voice Cloning: Perform voice cloning using VoxCPM1.5、IndexTTS2、CosyVoice3 and GPT-SoVITS4, supporting voice command control, cross-language cloning, and emotion-aware synthesis.
• Fine-Tuning Training: Supports emotion fine-tuning and inference on GPT-SoVITS4 baseline models (v1, v2, v2Pro, v2ProPlus, v3, v4) to create personalized voice effects.
• Extended Capabilities: Access the ElevenLabs voice library and support voice cloning to easily expand voice expressiveness.
• Custom Voice Library: Built-in three types of voices: Built-in, Video, and Custom, suitable for various scenarios.

Draft

• Track Control: Supports up to 6 tracks for Visuals, Audio, and Subtitles
• Subtitle Generation: Customize Size, Position, Color, and Outline settings
• Volume Adjustment: Control Original Sound, Speech, and Background Music volumes

Others

• Dubbing Modes: Choose between Video, Audio, and Adaptive modes
• Translation Modes: Translate videos from the original language to another
• Narration Mode: Planned...
• Automated Posting

🎨 What Can It Do?

• 🎥 Content Creators: Optimize video dubbing, translation, and editing workflows, enhancing creation efficiency and unleashing creative potential.

• 🌍 Multilingual Translation/Dubbing: Operate localized content creation and cross-language publishing.

• ⚙️ Independent Deployment: Deploy locally with flexible configurations to ensure privacy.