{"observation":{"id":"613b4473-fc70-4624-8f51-cd1f957349de","tool":"veed","tool_name":"VEED","criterion":"lip-sync-accuracy","criterion_name":"Lip Sync Accuracy","criterion_definition":"Does the dubbed audio visually match lip movements in the original video?","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"The ranking is specifically about lip sync, so visual alignment of speech to mouth movement is a core success criterion. (3 of 3 judges)","scenario":"fitness-instructor-short-english-hindi","scenario_name":"Fitness instructor short (English → Hindi)","group_tag":"video-translation-voice-clone-lip-sync","scenario_description":"A short fitness/coaching video with energetic single-speaker English speech, used to test whether tools can translate into Hindi while preserving fast delivery, motivational tone, and original-face lip sync.","modality":"video","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://cdn.futuresmart.ai/public/aidemos/e0437017b0894e3486f5a2b2015a4da6.mp4?v=1","role":"input","filename":"Input 1 Fitness Video (online-video-cutter.com).mp4"}],"stresses":["Fast, energetic speech transcription","Tone preservation for motivational fitness content","English-to-Hindi translation quality","Lip-sync accuracy on a moving face","End-to-end dubbing workflow automation"],"verdict":"failed","score":null,"score_total":null,"note":"No lip-sync regeneration happened in Test 1; the report says the mouth region was essentially identical at about 1-2/255 pixel difference, so VEED swapped the audio but left the original mouth movement unchanged.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/a3510e801c9a46c1bbf774e6fa392f8d.mp4?v=1","role":"input","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/4802f98e7e3d4497a8312c8238c50d9d.mp4?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/244a8c101e25412ca396dc22c9ee7e94.png?v=1","role":"context","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/00bc7cbfbd3049439100fde403107be3.mp4?v=1","role":"context","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/f7c91821f2dc45e592dce9051e519167.mp4?v=1","role":"context","alt":null}],"run_id":"ec1dd100-6af8-4f49-8f19-b78492702b14","study_title":"Translate Videos with Voice Cloning and Lip Sync Using AI","study_kind":"generation","research_task":"86b96dfpf","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://cdn.futuresmart.ai/public/aidemos/e0437017b0894e3486f5a2b2015a4da6.mp4?v=1","filename":"Input 1 Fitness Video (online-video-cutter.com).mp4","alt":"Fitness instructor short (English → Hindi)","role":"input"}],"modality":"video","stresses":["Fast, energetic speech transcription","Tone preservation for motivational fitness content","English-to-Hindi translation quality","Lip-sync accuracy on a moving face","End-to-end dubbing workflow automation"]},"tool_page_slug":"veed-io","tool_url":"https://aidemos.com/tools/veed-io","permalink":"https://aidemos.com/evidence/613b4473-fc70-4624-8f51-cd1f957349de","api_url":"https://ai.aidemos.com/v1/observations/613b4473-fc70-4624-8f51-cd1f957349de"},"peers":[{"id":"f31a5d5d-a136-4ec4-b614-bcb9ae854d7b","tool":"camb-ai","tool_name":"Camb AI","verdict":"failed","score":null,"score_total":null,"note":"On the fitness clip, the dubbed video shows no mouth regeneration; whole-clip pixel-diff stayed around 1.7-2.3/255 across 5-frame intervals, and the split-screen mouth crops are frame-identical.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/030c2c642aaf4cbcaf831203037b0307.png?v=1","evidence_url":"https://aidemos.com/evidence/f31a5d5d-a136-4ec4-b614-bcb9ae854d7b"},{"id":"f20300e9-63fd-4fd5-bbc7-8a8b015f31a0","tool":"d-id","tool_name":"D-ID","verdict":"failed","score":null,"score_total":null,"note":"Lip sync is only aligned to the avatar output and not to the original live-action footage.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/f20300e9-63fd-4fd5-bbc7-8a8b015f31a0"},{"id":"9e1843d1-e7fa-4f78-aa4f-90c943c9b518","tool":"dubverse","tool_name":"Dubverse","verdict":"mixed","score":null,"score_total":null,"note":"Lip sync is good for slow to medium speech, but the report says it shows a slight delay in fast instruction segments.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/e120792604b24c0ab8fe30690529029b.mp4?v=1","evidence_url":"https://aidemos.com/evidence/9e1843d1-e7fa-4f78-aa4f-90c943c9b518"},{"id":"6dcb7f04-3845-4070-b29b-c68ef1ab96e5","tool":"elevenlabs","tool_name":"ElevenLabs","verdict":"failed","score":null,"score_total":null,"note":"The tool provides no built-in lip sync, so the fitness output does not visually track mouth movements and would require external editing for sync.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/f06ae0bd183d48fe8368c15db83f1951.mp4?v=1","evidence_url":"https://aidemos.com/evidence/6dcb7f04-3845-4070-b29b-c68ef1ab96e5"},{"id":"5b693a71-46c3-41fc-8236-36a2b350f31a","tool":"heygen","tool_name":"HeyGen","verdict":"failed","score":null,"score_total":null,"note":"At a matched content point at t=12.0s, the output face was frame-for-frame identical to the input, so no visible lip-sync or face regeneration occurred.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/0ed75a751b3e429c821a9ae212a635b5.png?v=1","evidence_url":"https://aidemos.com/evidence/5b693a71-46c3-41fc-8236-36a2b350f31a"},{"id":"eecee290-853e-46ba-a43d-eaefdd0ca6b4","tool":"rask-ai","tool_name":"Rask AI","verdict":"failed","score":null,"score_total":null,"note":"At matched timestamps the output stays frame-identical to the source, with no visible mouth or face regeneration, so the free-tier export does not perform lip sync.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/d25fee8e34384325bb199ac609c44259.png?v=1","evidence_url":"https://aidemos.com/evidence/eecee290-853e-46ba-a43d-eaefdd0ca6b4"},{"id":"15a284a2-8d1e-4429-a8d5-15d6fb4aa0f1","tool":"sync-labs","tool_name":"Sync Labs","verdict":"mixed","score":null,"score_total":null,"note":"Lip sync works on the original face, but slight delay and mismatch are visible during fast movements.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/855a96949c7f487784b76fd99a758dcb.mp4?v=1","evidence_url":"https://aidemos.com/evidence/15a284a2-8d1e-4429-a8d5-15d6fb4aa0f1"},{"id":"60b462ba-09d7-4e57-bd54-8e31408f9133","tool":"synthesia","tool_name":"Synthesia","verdict":"mixed","score":null,"score_total":null,"note":"This fitness run did not receive an independent lip-sync judgment in the report, so there is no direct evidence here for how closely the dubbed mouth movement matched the source.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/60b462ba-09d7-4e57-bd54-8e31408f9133"}],"other_criteria":[{"id":"4230fcc0-8792-4215-a8fd-e31fb11b156f","criterion":"translation-accuracy","criterion_name":"Translation Accuracy","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Spoken dialogue captions translate correctly into Hindi with proper Devanagari, but code-switched terms like 'bcaa', 'fitness', and 'plate' stay in English inside the Hindi caption lines.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/4230fcc0-8792-4215-a8fd-e31fb11b156f"},{"id":"ca2fc90c-1529-4271-8964-af0b63321d18","criterion":"translation-accuracy","criterion_name":"Translation Accuracy","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"The tool does not localize baked-in quiz overlays: the shoulder question and BCAA headline remain 100% English while Hindi dialogue captions appear below them.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/ca2fc90c-1529-4271-8964-af0b63321d18"}],"appears_in":[{"page_type":"ranking","slug":"video-translation-tools","title":"Best AI Tools for Video Translation with Voice Cloning and Lip Sync","url":"https://aidemos.com/best/video-translation-tools","binding":"run"},{"page_type":"ranking","slug":"ai-video-dubbing-tools","title":"Best AI Tools to Translate Videos with Voice Cloning and Lip Sync","url":"https://aidemos.com/best/ai-video-dubbing-tools","binding":"study"}],"same_scenario":[{"id":"f31a5d5d-a136-4ec4-b614-bcb9ae854d7b","tool":"camb-ai","tool_name":"Camb AI","verdict":"failed","score":null,"score_total":null,"note":"On the fitness clip, the dubbed video shows no mouth regeneration; whole-clip pixel-diff stayed around 1.7-2.3/255 across 5-frame intervals, and the split-screen mouth crops are frame-identical."},{"id":"f20300e9-63fd-4fd5-bbc7-8a8b015f31a0","tool":"d-id","tool_name":"D-ID","verdict":"failed","score":null,"score_total":null,"note":"Lip sync is only aligned to the avatar output and not to the original live-action footage."},{"id":"9e1843d1-e7fa-4f78-aa4f-90c943c9b518","tool":"dubverse","tool_name":"Dubverse","verdict":"mixed","score":null,"score_total":null,"note":"Lip sync is good for slow to medium speech, but the report says it shows a slight delay in fast instruction segments."},{"id":"6dcb7f04-3845-4070-b29b-c68ef1ab96e5","tool":"elevenlabs","tool_name":"ElevenLabs","verdict":"failed","score":null,"score_total":null,"note":"The tool provides no built-in lip sync, so the fitness output does not visually track mouth movements and would require external editing for sync."},{"id":"5b693a71-46c3-41fc-8236-36a2b350f31a","tool":"heygen","tool_name":"HeyGen","verdict":"failed","score":null,"score_total":null,"note":"At a matched content point at t=12.0s, the output face was frame-for-frame identical to the input, so no visible lip-sync or face regeneration occurred."},{"id":"eecee290-853e-46ba-a43d-eaefdd0ca6b4","tool":"rask-ai","tool_name":"Rask AI","verdict":"failed","score":null,"score_total":null,"note":"At matched timestamps the output stays frame-identical to the source, with no visible mouth or face regeneration, so the free-tier export does not perform lip sync."},{"id":"15a284a2-8d1e-4429-a8d5-15d6fb4aa0f1","tool":"sync-labs","tool_name":"Sync Labs","verdict":"mixed","score":null,"score_total":null,"note":"Lip sync works on the original face, but slight delay and mismatch are visible during fast movements."},{"id":"60b462ba-09d7-4e57-bd54-8e31408f9133","tool":"synthesia","tool_name":"Synthesia","verdict":"mixed","score":null,"score_total":null,"note":"This fitness run did not receive an independent lip-sync judgment in the report, so there is no direct evidence here for how closely the dubbed mouth movement matched the source."}]}