{"observation":{"id":"7b2219e2-7642-4ec4-88cd-d0d4ad20c414","tool":"heygen","tool_name":"HeyGen","criterion":"translation-accuracy","criterion_name":"Translation Accuracy","criterion_definition":"Is the translated output semantically correct, and does it preserve technical terms, tone, and meaning?","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"If the translation changes meaning, tone, or technical terms, the tool fails the core job of translating the video. (3 of 3 judges)","scenario":"fitness-instructor-short-english-hindi","scenario_name":"Fitness instructor short (English → Hindi)","group_tag":"video-translation-voice-clone-lip-sync","scenario_description":"A short fitness/coaching video with energetic single-speaker English speech, used to test whether tools can translate into Hindi while preserving fast delivery, motivational tone, and original-face lip sync.","modality":"video","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://cdn.futuresmart.ai/public/aidemos/e0437017b0894e3486f5a2b2015a4da6.mp4?v=1","role":"input","filename":"Input 1 Fitness Video (online-video-cutter.com).mp4"}],"stresses":["Fast, energetic speech transcription","Tone preservation for motivational fitness content","English-to-Hindi translation quality","Lip-sync accuracy on a moving face","End-to-end dubbing workflow automation"],"verdict":"failed","score":null,"score_total":null,"note":"The tool left the burned-in fitness captions in English instead of localizing them; the review cites unchanged lines such as \"This is a famous,\" \"For every question,\" and \"I don't know.\"","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/b1a4b4b2cfb445c0a5e3dbd6c25891ec.mp4?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/bdcfa7afb45e40fcb89c48fe5606b00c.png?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/ec11926e7c8144e6918801e11d2e2768.mp4?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/7acb1804066a4d799ee20b1232b1d3d2.mp4?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/4dd1312d1e9c48a68fc29df2956b9b9c.mp4?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/363ea28256d84cfba44d5cd86e64559f.mp4?v=1","role":"output","alt":null}],"run_id":"ec1dd100-6af8-4f49-8f19-b78492702b14","study_title":"Translate Videos with Voice Cloning and Lip Sync Using AI","study_kind":"generation","research_task":"86b96dfpf","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://cdn.futuresmart.ai/public/aidemos/e0437017b0894e3486f5a2b2015a4da6.mp4?v=1","filename":"Input 1 Fitness Video (online-video-cutter.com).mp4","alt":"Fitness instructor short (English → Hindi)","role":"input"}],"modality":"video","stresses":["Fast, energetic speech transcription","Tone preservation for motivational fitness content","English-to-Hindi translation quality","Lip-sync accuracy on a moving face","End-to-end dubbing workflow automation"]},"tool_page_slug":"heygen","tool_url":"https://aidemos.com/tools/heygen","permalink":"https://aidemos.com/evidence/7b2219e2-7642-4ec4-88cd-d0d4ad20c414","api_url":"https://ai.aidemos.com/v1/observations/7b2219e2-7642-4ec4-88cd-d0d4ad20c414"},"peers":[{"id":"25e9f1ec-67de-4574-ad9b-32e89d1be6d3","tool":"akool","tool_name":"Akool","verdict":"failed","score":null,"score_total":null,"note":"Burned-in English captions such as \"This is a famous\" were not localized in the Hindi output, so the tool left source captions untranslated.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/c4eada53777a4279ab3bb2d86380c38b.mp4?v=1","evidence_url":"https://aidemos.com/evidence/25e9f1ec-67de-4574-ad9b-32e89d1be6d3"},{"id":"e430bfdd-fc84-4724-90c2-4d974532f707","tool":"camb-ai","tool_name":"Camb AI","verdict":"failed","score":null,"score_total":null,"note":"On the fitness clip, burned-in English captions were not translated; the output keeps captions like 'This is a famous' in English instead of localizing them.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/5832fb62b8ef4f6da1be22f4c42a1818.png?v=1","evidence_url":"https://aidemos.com/evidence/e430bfdd-fc84-4724-90c2-4d974532f707"},{"id":"649df64b-e358-4034-a64e-9fcff92c161f","tool":"d-id","tool_name":"D-ID","verdict":"worked","score":null,"score_total":null,"note":"It produces an acceptable Hindi translation of the source speech.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/649df64b-e358-4034-a64e-9fcff92c161f"},{"id":"53a8ffe8-3839-49b6-94fa-a0923ba19cdf","tool":"dubverse","tool_name":"Dubverse","verdict":"worked","score":null,"score_total":null,"note":"It translates common fitness terminology from English to Hindi accurately, though the report notes some fitness-specific words were not localized naturally.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/e120792604b24c0ab8fe30690529029b.mp4?v=1","evidence_url":"https://aidemos.com/evidence/53a8ffe8-3839-49b6-94fa-a0923ba19cdf"},{"id":"f0a7ce18-ae25-48ca-a3b9-2bdce46580e0","tool":"elevenlabs","tool_name":"ElevenLabs","verdict":"worked","score":null,"score_total":null,"note":"The Hindi translation was reported as accurate for the fitness video.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/f06ae0bd183d48fe8368c15db83f1951.mp4?v=1","evidence_url":"https://aidemos.com/evidence/f0a7ce18-ae25-48ca-a3b9-2bdce46580e0"},{"id":"cd7cc16b-6988-4c13-86c5-ac91c5abbf1a","tool":"rask-ai","tool_name":"Rask AI","verdict":"failed","score":null,"score_total":null,"note":"The English→Hindi fitness export does not localize the visible on-screen text: the translated clip still shows English questions such as 'Which head of the shoulder is most active on this exercise?' and 'Can you name the three BCAAs?'.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/411705d5ac5849f4b0a20a3d8b7618a6.mp4?v=1","evidence_url":"https://aidemos.com/evidence/cd7cc16b-6988-4c13-86c5-ac91c5abbf1a"},{"id":"f020bdf2-a641-4b7a-ba0b-7eb10485830f","tool":"sync-labs","tool_name":"Sync Labs","verdict":"mixed","score":null,"score_total":null,"note":"The Hindi translation is moderately accurate, so the meaning is mostly preserved but not at a strong or flawless level.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/c8f9e4ff7d404701a7598bf22576639c.mp4?v=1","evidence_url":"https://aidemos.com/evidence/f020bdf2-a641-4b7a-ba0b-7eb10485830f"},{"id":"ee26b601-e997-43e0-aead-de5cf0738015","tool":"synthesia","tool_name":"Synthesia","verdict":"failed","score":null,"score_total":null,"note":"Caption translation failed outright: in the Hindi output run, every caption stayed in English from start to finish, including the final \"I don't know.\" line.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/295c3edffa3a4be6a5845da61d71732b.mp4?v=1","evidence_url":"https://aidemos.com/evidence/ee26b601-e997-43e0-aead-de5cf0738015"},{"id":"ca2fc90c-1529-4271-8964-af0b63321d18","tool":"veed","tool_name":"VEED","verdict":"struggled","score":null,"score_total":null,"note":"The tool does not localize baked-in quiz overlays: the shoulder question and BCAA headline remain 100% English while Hindi dialogue captions appear below them.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/a3510e801c9a46c1bbf774e6fa392f8d.mp4?v=1","evidence_url":"https://aidemos.com/evidence/ca2fc90c-1529-4271-8964-af0b63321d18"}],"other_criteria":[{"id":"5b693a71-46c3-41fc-8236-36a2b350f31a","criterion":"lip-sync-accuracy","criterion_name":"Lip Sync Accuracy","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"At a matched content point at t=12.0s, the output face was frame-for-frame identical to the input, so no visible lip-sync or face regeneration occurred.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/5b693a71-46c3-41fc-8236-36a2b350f31a"}],"appears_in":[{"page_type":"ranking","slug":"video-translation-tools","title":"Best AI Tools for Video Translation with Voice Cloning and Lip Sync","url":"https://aidemos.com/best/video-translation-tools","binding":"run"},{"page_type":"ranking","slug":"ai-video-dubbing-tools","title":"Best AI Tools to Translate Videos with Voice Cloning and Lip Sync","url":"https://aidemos.com/best/ai-video-dubbing-tools","binding":"study"}],"same_scenario":[{"id":"25e9f1ec-67de-4574-ad9b-32e89d1be6d3","tool":"akool","tool_name":"Akool","verdict":"failed","score":null,"score_total":null,"note":"Burned-in English captions such as \"This is a famous\" were not localized in the Hindi output, so the tool left source captions untranslated."},{"id":"e430bfdd-fc84-4724-90c2-4d974532f707","tool":"camb-ai","tool_name":"Camb AI","verdict":"failed","score":null,"score_total":null,"note":"On the fitness clip, burned-in English captions were not translated; the output keeps captions like 'This is a famous' in English instead of localizing them."},{"id":"649df64b-e358-4034-a64e-9fcff92c161f","tool":"d-id","tool_name":"D-ID","verdict":"worked","score":null,"score_total":null,"note":"It produces an acceptable Hindi translation of the source speech."},{"id":"53a8ffe8-3839-49b6-94fa-a0923ba19cdf","tool":"dubverse","tool_name":"Dubverse","verdict":"worked","score":null,"score_total":null,"note":"It translates common fitness terminology from English to Hindi accurately, though the report notes some fitness-specific words were not localized naturally."},{"id":"f0a7ce18-ae25-48ca-a3b9-2bdce46580e0","tool":"elevenlabs","tool_name":"ElevenLabs","verdict":"worked","score":null,"score_total":null,"note":"The Hindi translation was reported as accurate for the fitness video."},{"id":"cd7cc16b-6988-4c13-86c5-ac91c5abbf1a","tool":"rask-ai","tool_name":"Rask AI","verdict":"failed","score":null,"score_total":null,"note":"The English→Hindi fitness export does not localize the visible on-screen text: the translated clip still shows English questions such as 'Which head of the shoulder is most active on this exercise?' and 'Can you name the three BCAAs?'."},{"id":"f020bdf2-a641-4b7a-ba0b-7eb10485830f","tool":"sync-labs","tool_name":"Sync Labs","verdict":"mixed","score":null,"score_total":null,"note":"The Hindi translation is moderately accurate, so the meaning is mostly preserved but not at a strong or flawless level."},{"id":"ee26b601-e997-43e0-aead-de5cf0738015","tool":"synthesia","tool_name":"Synthesia","verdict":"failed","score":null,"score_total":null,"note":"Caption translation failed outright: in the Hindi output run, every caption stayed in English from start to finish, including the final \"I don't know.\" line."},{"id":"ca2fc90c-1529-4271-8964-af0b63321d18","tool":"veed","tool_name":"VEED","verdict":"struggled","score":null,"score_total":null,"note":"The tool does not localize baked-in quiz overlays: the shoulder question and BCAA headline remain 100% English while Hindi dialogue captions appear below them."}]}