{"observation":{"id":"adbf7b48-4745-4883-a18a-94517e2a766a","tool":"synthesia","tool_name":"Synthesia","criterion":"lip-sync-accuracy","criterion_name":"Lip Sync Accuracy","criterion_definition":"Does the dubbed audio visually match lip movements in the original video?","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"The ranking is specifically about lip sync, so visual alignment of speech to mouth movement is a core success criterion. (3 of 3 judges)","scenario":"educational-airport-conversation-short-english-spanish","scenario_name":"Educational airport conversation short (English → Spanish)","group_tag":"video-translation-voice-clone-lip-sync","scenario_description":"A structured educational/conversation-style English short video used to test translation into Spanish with clear narration, steadier pacing, and easier lip-sync alignment than the other scenarios.","modality":"video","input_text":null,"input_artifact_refs":[{"alt":null,"url":null,"role":null,"filename":"Airport English Conversation #english #learnenglish.publer.com.mp4"}],"stresses":["Structured speech translation accuracy","English-to-Spanish narration quality","Longer-phrase lip-sync consistency","Clear voice rendering for informational content","Export/download reliability in a simple speaking scenario"],"verdict":"mixed","score":null,"score_total":null,"note":"The dubbed mouth movement roughly tracks the source at the compared timestamps in the English→Spanish test, so lip sync appears visually usable but not perfect.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/845ffbfad23742d3ace0ceb2a7c0c467.mp4?v=1","role":"input","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/fd180cd564e04798a17b9b9ec063c5a3.mp4?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/a75bf5fb00374cd190d88661cbe051e2.png?v=1","role":"context","alt":null}],"run_id":"ec1dd100-6af8-4f49-8f19-b78492702b14","study_title":"Translate Videos with Voice Cloning and Lip Sync Using AI","study_kind":"generation","research_task":"86b96dfpf","tested_at":null,"completeness":"input-and-output","input":{"state":"not-captured","text":null,"files":[],"modality":"video","stresses":["Structured speech translation accuracy","English-to-Spanish narration quality","Longer-phrase lip-sync consistency","Clear voice rendering for informational content","Export/download reliability in a simple speaking scenario"]},"tool_page_slug":"synthesia","tool_url":"https://aidemos.com/tools/synthesia","permalink":"https://aidemos.com/evidence/adbf7b48-4745-4883-a18a-94517e2a766a","api_url":"https://ai.aidemos.com/v1/observations/adbf7b48-4745-4883-a18a-94517e2a766a"},"peers":[{"id":"f87cb4ef-144c-47d0-b532-276414c110ed","tool":"akool","tool_name":"Akool","verdict":"worked","score":null,"score_total":null,"note":"A content-matched frame comparison confirmed genuine mouth regeneration, with the output mouth shape differing from the source.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/9927e089ead84837aa1be3792400a94f.mp4?v=1","evidence_url":"https://aidemos.com/evidence/f87cb4ef-144c-47d0-b532-276414c110ed"},{"id":"c5ea41b6-dfb4-47bc-891d-b2edfd343d96","tool":"camb-ai","tool_name":"Camb AI","verdict":"failed","score":null,"score_total":null,"note":"On the educational clip, lip sync was absent as well; the report measured 2.5-4/255 pixel differences over the 9-second clip, including a wide-open 'oo' frame where the English and Spanish mouths were frame-identical.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/afb7cecb26a949c4b305f41067fdf3ab.png?v=1","evidence_url":"https://aidemos.com/evidence/c5ea41b6-dfb4-47bc-891d-b2edfd343d96"},{"id":"de7b314f-15f6-4fff-8140-c031d9641bc5","tool":"dubverse","tool_name":"Dubverse","verdict":"worked","score":null,"score_total":null,"note":"Lip sync is described as more precise than on the other inputs.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/de7b314f-15f6-4fff-8140-c031d9641bc5"},{"id":"ab435d70-42dd-430e-8a86-a6527474db0e","tool":"elevenlabs","tool_name":"ElevenLabs","verdict":"failed","score":null,"score_total":null,"note":"The tool provides no built-in lip sync, so the Spanish educational output would need external video editing to align audio with the mouth movements.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/e2db6468583448f28495cee5b785431e.mp4?v=1","evidence_url":"https://aidemos.com/evidence/ab435d70-42dd-430e-8a86-a6527474db0e"},{"id":"e725e7b7-c76f-4254-8b93-085446418a34","tool":"heygen","tool_name":"HeyGen","verdict":"failed","score":null,"score_total":null,"note":"At the matched \"Unripe\" moment, the mouth shape, eyebrows, and expression were essentially identical between input and output, showing no visible lip-sync change.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/4914c9061ad84dd29bfd86839345e336.png?v=1","evidence_url":"https://aidemos.com/evidence/e725e7b7-c76f-4254-8b93-085446418a34"},{"id":"1b9e40e3-5b72-4de4-a90f-11500f6ab564","tool":"rask-ai","tool_name":"Rask AI","verdict":"failed","score":null,"score_total":null,"note":"At t=4.0s the input and output are frame-for-frame identical, with the same mouth shape, so no lip sync or face regeneration is visible on the free tier.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/bd7626b397ce472bad101cc68868a129.png?v=1","evidence_url":"https://aidemos.com/evidence/1b9e40e3-5b72-4de4-a90f-11500f6ab564"},{"id":"83888617-eedb-4955-a52f-443fa404ecd7","tool":"sync-labs","tool_name":"Sync Labs","verdict":"mixed","score":null,"score_total":null,"note":"Lip sync is better on the slower educational clip, although slight lag in lip movement remains.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/2e19d15a954b426fba4f1126d3c78c89.mp4?v=1","evidence_url":"https://aidemos.com/evidence/83888617-eedb-4955-a52f-443fa404ecd7"},{"id":"9c39fd28-8fd8-44e5-b325-95e74b63f8a1","tool":"veed","tool_name":"VEED","verdict":"worked","score":null,"score_total":null,"note":"Test 2 does regenerate mouth motion: the report measures 6-38/255 pixel differences in the face region, and at one matched timestamp the English source has her mouth open while the Spanish output has it closed.","artifact_count":4,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/e2db6468583448f28495cee5b785431e.mp4?v=1","evidence_url":"https://aidemos.com/evidence/9c39fd28-8fd8-44e5-b325-95e74b63f8a1"}],"other_criteria":[{"id":"72dc0f6f-7ba1-49b3-bbed-d75987dd86fe","criterion":"translation-accuracy","criterion_name":"Translation Accuracy","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"On-screen graphic translation is skipped: the banana-stage labels \"Unripe\", \"Ripe\", \"Overripe\", and \"Rotten\" stay in English throughout the Spanish output.","artifact_count":5,"evidence_url":"https://aidemos.com/evidence/72dc0f6f-7ba1-49b3-bbed-d75987dd86fe"}],"appears_in":[{"page_type":"ranking","slug":"video-translation-tools","title":"Best AI Tools for Video Translation with Voice Cloning and Lip Sync","url":"https://aidemos.com/best/video-translation-tools","binding":"run"},{"page_type":"ranking","slug":"ai-video-dubbing-tools","title":"Best AI Tools to Translate Videos with Voice Cloning and Lip Sync","url":"https://aidemos.com/best/ai-video-dubbing-tools","binding":"study"}],"same_scenario":[{"id":"f87cb4ef-144c-47d0-b532-276414c110ed","tool":"akool","tool_name":"Akool","verdict":"worked","score":null,"score_total":null,"note":"A content-matched frame comparison confirmed genuine mouth regeneration, with the output mouth shape differing from the source."},{"id":"c5ea41b6-dfb4-47bc-891d-b2edfd343d96","tool":"camb-ai","tool_name":"Camb AI","verdict":"failed","score":null,"score_total":null,"note":"On the educational clip, lip sync was absent as well; the report measured 2.5-4/255 pixel differences over the 9-second clip, including a wide-open 'oo' frame where the English and Spanish mouths were frame-identical."},{"id":"de7b314f-15f6-4fff-8140-c031d9641bc5","tool":"dubverse","tool_name":"Dubverse","verdict":"worked","score":null,"score_total":null,"note":"Lip sync is described as more precise than on the other inputs."},{"id":"ab435d70-42dd-430e-8a86-a6527474db0e","tool":"elevenlabs","tool_name":"ElevenLabs","verdict":"failed","score":null,"score_total":null,"note":"The tool provides no built-in lip sync, so the Spanish educational output would need external video editing to align audio with the mouth movements."},{"id":"e725e7b7-c76f-4254-8b93-085446418a34","tool":"heygen","tool_name":"HeyGen","verdict":"failed","score":null,"score_total":null,"note":"At the matched \"Unripe\" moment, the mouth shape, eyebrows, and expression were essentially identical between input and output, showing no visible lip-sync change."},{"id":"1b9e40e3-5b72-4de4-a90f-11500f6ab564","tool":"rask-ai","tool_name":"Rask AI","verdict":"failed","score":null,"score_total":null,"note":"At t=4.0s the input and output are frame-for-frame identical, with the same mouth shape, so no lip sync or face regeneration is visible on the free tier."},{"id":"83888617-eedb-4955-a52f-443fa404ecd7","tool":"sync-labs","tool_name":"Sync Labs","verdict":"mixed","score":null,"score_total":null,"note":"Lip sync is better on the slower educational clip, although slight lag in lip movement remains."},{"id":"9c39fd28-8fd8-44e5-b325-95e74b63f8a1","tool":"veed","tool_name":"VEED","verdict":"worked","score":null,"score_total":null,"note":"Test 2 does regenerate mouth motion: the report measures 6-38/255 pixel differences in the face region, and at one matched timestamp the English source has her mouth open while the Spanish output has it closed."}]}