{"observation":{"id":"bd9dc91f-e926-41d1-9fe8-1b99b27501e7","tool":"veed","tool_name":"VEED","criterion":"lip-sync-accuracy","criterion_name":"Lip Sync Accuracy","criterion_definition":"Does the dubbed audio visually match lip movements in the original video?","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"The ranking is specifically about lip sync, so visual alignment of speech to mouth movement is a core success criterion. (3 of 3 judges)","scenario":"hindi-vlog-style-talking-head-hindi-english","scenario_name":"Hindi vlog-style talking head (Hindi → English)","group_tag":"video-translation-voice-clone-lip-sync","scenario_description":"A casual Hindi talking-head/vlog-style video used to test English dubbing on informal speech, Hinglish-like phrasing, speaker personality preservation, and lip sync under natural creator-style delivery.","modality":"video","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://cdn.futuresmart.ai/public/aidemos/c414bbbfcd2a49faa9b162d10cd64c2f.mp4?v=1","role":"input","filename":"Free Copyright Stock Videos Images And Music.publer.com (online-video-cutter.com).mp4"}],"stresses":["Informal Hindi speech transcription","Hindi-to-English translation of casual phrasing","Voice cloning for conversational creator tone","Handling of slang/Hinglish-style content","Lip-sync stability during expressive speech"],"verdict":"worked","score":null,"score_total":null,"note":"Test 3 does regenerate lip movement, with reported face-region diffs of 5-60/255 and visibly different mouth shapes at matching timestamps.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/330f9a7be00845109be703bedbb9595c.mp4?v=1","role":"input","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/12e906e3ed784b13b4427138f11e3f72.mp4?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/620b33bc36114eb4a2a536924ad7d614.mp4?v=1","role":"context","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/3649d5c66c594e57b10a78d2b40006fe.mp4?v=1","role":"context","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/fb81f934cd7644e8a82507dc7c486a8c.mp4?v=1","role":"context","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/c1796a852a1744868f7d099f7b202092.mp4?v=1","role":"context","alt":null}],"run_id":"ec1dd100-6af8-4f49-8f19-b78492702b14","study_title":"Translate Videos with Voice Cloning and Lip Sync Using AI","study_kind":"generation","research_task":"86b96dfpf","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://cdn.futuresmart.ai/public/aidemos/c414bbbfcd2a49faa9b162d10cd64c2f.mp4?v=1","filename":"Free Copyright Stock Videos Images And Music.publer.com (online-video-cutter.com).mp4","alt":"Hindi vlog-style talking head (Hindi → English)","role":"input"}],"modality":"video","stresses":["Informal Hindi speech transcription","Hindi-to-English translation of casual phrasing","Voice cloning for conversational creator tone","Handling of slang/Hinglish-style content","Lip-sync stability during expressive speech"]},"tool_page_slug":"veed-io","tool_url":"https://aidemos.com/tools/veed-io","permalink":"https://aidemos.com/evidence/bd9dc91f-e926-41d1-9fe8-1b99b27501e7","api_url":"https://ai.aidemos.com/v1/observations/bd9dc91f-e926-41d1-9fe8-1b99b27501e7"},"peers":[{"id":"13c1dd85-d4a7-43f0-9dd0-cfbbef06d06c","tool":"akool","tool_name":"Akool","verdict":"mixed","score":null,"score_total":null,"note":"Mouth movement stayed active and varying across the clip, but precise frame-matched verification was not possible because timing drifted partway through.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/211f00ca9ecd40e2a76e2b44dbdfd903.mp4?v=1","evidence_url":"https://aidemos.com/evidence/13c1dd85-d4a7-43f0-9dd0-cfbbef06d06c"},{"id":"6b7e4114-5ac3-4fdf-8efa-b18734408788","tool":"camb-ai","tool_name":"Camb AI","verdict":"failed","score":null,"score_total":null,"note":"On the Hindi→English vlog clip, the output again left mouths unchanged, with only ~1.7-2.7/255 pixel difference across the 10-second clip and frame-identical mouths at t=0.0s.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/b8c161fc9536403398e9fae5b54df14a.png?v=1","evidence_url":"https://aidemos.com/evidence/6b7e4114-5ac3-4fdf-8efa-b18734408788"},{"id":"c0d050dc-4fd5-4a4e-b353-9c03b150659f","tool":"d-id","tool_name":"D-ID","verdict":"failed","score":null,"score_total":null,"note":"Lip sync is stable on the avatar, but it does not reflect the original speaker's expressions.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/c0d050dc-4fd5-4a4e-b353-9c03b150659f"},{"id":"e7c14804-2418-4ae7-a8a2-9c7788b1ffb5","tool":"dubverse","tool_name":"Dubverse","verdict":"failed","score":null,"score_total":null,"note":"Lip sync is the weakest of the three tests, with sync inconsistencies and slightly off audio-video alignment in fast speech.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/e7c14804-2418-4ae7-a8a2-9c7788b1ffb5"},{"id":"0cfbd684-e250-4511-8895-843a8cb236cc","tool":"elevenlabs","tool_name":"ElevenLabs","verdict":"failed","score":null,"score_total":null,"note":"The tool provides no built-in lip sync, so the vlog output does not visually match the speaker’s mouth movements without external re-editing.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/241e94e71c8f4de396a3918a43f5f833.mp4?v=1","evidence_url":"https://aidemos.com/evidence/0cfbd684-e250-4511-8895-843a8cb236cc"},{"id":"a5037630-c7f4-42f6-b401-6afb6e9d608c","tool":"heygen","tool_name":"HeyGen","verdict":"failed","score":null,"score_total":null,"note":"At the matched \"AAP\" caption, the output kept an identical open mouth and identical raised finger, so no visible lip-sync regeneration was observed.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/51014ac6fa024de384455a5e38c9d103.png?v=1","evidence_url":"https://aidemos.com/evidence/a5037630-c7f4-42f6-b401-6afb6e9d608c"},{"id":"d28ff670-9b13-4d19-a21b-4c2f102240c9","tool":"rask-ai","tool_name":"Rask AI","verdict":"failed","score":null,"score_total":null,"note":"The Hindi→English talking-head export is reported as frame-identical to the source at the same timestamp, with no mouth or face modification visible.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/a914df6570ba4668987382c6e3c24d67.png?v=1","evidence_url":"https://aidemos.com/evidence/d28ff670-9b13-4d19-a21b-4c2f102240c9"},{"id":"181ee2a3-9dda-4c96-b2e3-36f09eb80b52","tool":"sync-labs","tool_name":"Sync Labs","verdict":"mixed","score":null,"score_total":null,"note":"Lip sync works, but it struggles during expressive facial movements and fast speech.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/e464c888af70482f8ea63c2791b4cfc0.mp4?v=1","evidence_url":"https://aidemos.com/evidence/181ee2a3-9dda-4c96-b2e3-36f09eb80b52"},{"id":"01ff1594-c458-4ddb-8631-bfaf5e957268","tool":"synthesia","tool_name":"Synthesia","verdict":"worked","score":null,"score_total":null,"note":"The report says the vlog dub's timing held up consistently with the source, so the mouth timing looks visually stable in this run.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/f7c9aa4fb1e04fe0920f9ff5c7adc5dc.mp4?v=1","evidence_url":"https://aidemos.com/evidence/01ff1594-c458-4ddb-8631-bfaf5e957268"}],"other_criteria":[{"id":"1e1f8faf-3b07-4aa3-ab63-88da22e098c5","criterion":"translation-accuracy","criterion_name":"Translation Accuracy","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"The English dub is polluted by untranslated Romanized-Hindi caption text; 'KAR RAHE HAIN' and 'MAIN' appear in the output, and 'MAIN' is described as a fabricated caption that was not present in the source frame.","artifact_count":4,"evidence_url":"https://aidemos.com/evidence/1e1f8faf-3b07-4aa3-ab63-88da22e098c5"}],"appears_in":[{"page_type":"ranking","slug":"video-translation-tools","title":"Best AI Tools for Video Translation with Voice Cloning and Lip Sync","url":"https://aidemos.com/best/video-translation-tools","binding":"run"},{"page_type":"ranking","slug":"ai-video-dubbing-tools","title":"Best AI Tools to Translate Videos with Voice Cloning and Lip Sync","url":"https://aidemos.com/best/ai-video-dubbing-tools","binding":"study"}],"same_scenario":[{"id":"13c1dd85-d4a7-43f0-9dd0-cfbbef06d06c","tool":"akool","tool_name":"Akool","verdict":"mixed","score":null,"score_total":null,"note":"Mouth movement stayed active and varying across the clip, but precise frame-matched verification was not possible because timing drifted partway through."},{"id":"6b7e4114-5ac3-4fdf-8efa-b18734408788","tool":"camb-ai","tool_name":"Camb AI","verdict":"failed","score":null,"score_total":null,"note":"On the Hindi→English vlog clip, the output again left mouths unchanged, with only ~1.7-2.7/255 pixel difference across the 10-second clip and frame-identical mouths at t=0.0s."},{"id":"c0d050dc-4fd5-4a4e-b353-9c03b150659f","tool":"d-id","tool_name":"D-ID","verdict":"failed","score":null,"score_total":null,"note":"Lip sync is stable on the avatar, but it does not reflect the original speaker's expressions."},{"id":"e7c14804-2418-4ae7-a8a2-9c7788b1ffb5","tool":"dubverse","tool_name":"Dubverse","verdict":"failed","score":null,"score_total":null,"note":"Lip sync is the weakest of the three tests, with sync inconsistencies and slightly off audio-video alignment in fast speech."},{"id":"0cfbd684-e250-4511-8895-843a8cb236cc","tool":"elevenlabs","tool_name":"ElevenLabs","verdict":"failed","score":null,"score_total":null,"note":"The tool provides no built-in lip sync, so the vlog output does not visually match the speaker’s mouth movements without external re-editing."},{"id":"a5037630-c7f4-42f6-b401-6afb6e9d608c","tool":"heygen","tool_name":"HeyGen","verdict":"failed","score":null,"score_total":null,"note":"At the matched \"AAP\" caption, the output kept an identical open mouth and identical raised finger, so no visible lip-sync regeneration was observed."},{"id":"d28ff670-9b13-4d19-a21b-4c2f102240c9","tool":"rask-ai","tool_name":"Rask AI","verdict":"failed","score":null,"score_total":null,"note":"The Hindi→English talking-head export is reported as frame-identical to the source at the same timestamp, with no mouth or face modification visible."},{"id":"181ee2a3-9dda-4c96-b2e3-36f09eb80b52","tool":"sync-labs","tool_name":"Sync Labs","verdict":"mixed","score":null,"score_total":null,"note":"Lip sync works, but it struggles during expressive facial movements and fast speech."},{"id":"01ff1594-c458-4ddb-8631-bfaf5e957268","tool":"synthesia","tool_name":"Synthesia","verdict":"worked","score":null,"score_total":null,"note":"The report says the vlog dub's timing held up consistently with the source, so the mouth timing looks visually stable in this run."}]}