The University of Jordan :: Research Groups :: Dublejli: AI-Based Adaptive and Multi-Lingual...
Featured Publications

Dublejli: AI-Based Adaptive and Multi-Lingual Video Dubbing System

​Video dubbing is essential for breaking language barriers, but traditional methods are costly, time-consuming, and often suffer from poor synchronization. AI-based dubbing systems using Automatic Speech Recognition (ASR), Neural Machine Translation (NMT), and Text-To-Speech (TTS) models have improved efficiency; yet, ensuring speech duration alignment remains a challenge, especially in multilingual settings. This work introduces an Artificial Intelligence-based approach to enhance dubbing synchronization by leveraging silent moments, deploying multilingual TTS models, predicting speech duration, utilizing speaker-gender detection, and integrating Large Language Models for text summarization. Our method significantly improves synchronization accuracy and audio-visual coherence, resulting in a more natural and immersive dubbed experience. The modified Gender Identification model is able to achieve 93.92% accuracy. Also, the standard deviation of the audio speed-up variation in the target videos has been reduced significantly from 0.3 to 0.05.​