{"observation":{"id":"d85a401c-453e-41da-8da1-d013b88303b0","tool":"heygen","tool_name":"HeyGen","criterion":"platform-formats","criterion_name":"Platform formats","criterion_definition":"Correct aspect ratios and lengths for each channel or short-form platform.","criterion_evidence_type":"transformation","criterion_rank_role":"context","criterion_rank_role_reason":"Aspect ratios and length presets are delivery convenience; they matter, but they do not by themselves make the ad better or worse. (3 of 3 judges)","scenario":"futuresmart-ai-saas","scenario_name":"FutureSmart AI – SaaS","group_tag":"ugc-style-video-ads","scenario_description":"A short-form UGC ad prompt for a SaaS product discovery/comparison platform, designed to test whether the tool understands a software product, writes a believable testimonial-style script, and produces a realistic AI presenter video with captions.","modality":"mixed","input_text":"Product URL: https://aidemos.com/\n\nExact script:\nHook: Finding the right AI tools shouldn't take hours.\nScript: I've been using FutureSmart AI to discover and compare AI tools in one place. It helps me find the right tool faster with real use cases, rankings, and detailed comparisons. If you regularly use AI tools for work, it's definitely worth checking out.\n\nOutput requirements:\n- Vertical (9:16)\n- 20–30 seconds\n- UGC-style AI presenter\n- Natural AI voice\n- Captions enabled","input_artifact_refs":[],"stresses":["SaaS product understanding","Script quality","Avatar realism","Voice quality","Caption quality","Overall ad readiness"],"verdict":"failed","score":null,"score_total":null,"note":"The output only fills about 55% of the 9:16 canvas, leaving large dead white bars top and bottom, so the clip is not a native full-bleed vertical export.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/6855710cb28e4a2ebbe73c25eda6508a.png?v=1","role":"context","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/04a030ae7bbc48a28ae9395f52551b8d.mp4?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/b6364cd4380f42aab006131b103e4334.mp4?v=1","role":"output","alt":null}],"run_id":"9fdbb84b-4aa0-484d-8f89-5c46608019a5","study_title":"Generate UGC-Style Video Ads With AI Avatars","study_kind":"generation","research_task":"86baxegpg","tested_at":null,"completeness":"input-and-output","input":{"state":"text","text":"Product URL: https://aidemos.com/\n\nExact script:\nHook: Finding the right AI tools shouldn't take hours.\nScript: I've been using FutureSmart AI to discover and compare AI tools in one place. It helps me find the right tool faster with real use cases, rankings, and detailed comparisons. If you regularly use AI tools for work, it's definitely worth checking out.\n\nOutput requirements:\n- Vertical (9:16)\n- 20–30 seconds\n- UGC-style AI presenter\n- Natural AI voice\n- Captions enabled","files":[],"modality":"mixed","stresses":["SaaS product understanding","Script quality","Avatar realism","Voice quality","Caption quality","Overall ad readiness"]},"tool_page_slug":"heygen","tool_url":"https://aidemos.com/tools/heygen","permalink":"https://aidemos.com/evidence/d85a401c-453e-41da-8da1-d013b88303b0","api_url":"https://ai.aidemos.com/v1/observations/d85a401c-453e-41da-8da1-d013b88303b0"},"peers":[{"id":"69db4d2f-5b9b-4753-8f89-173a86326bf0","tool":"joggai","tool_name":"JoggAI","verdict":"mixed","score":null,"score_total":null,"note":"The export format is correct at full-bleed vertical 9:16 (1080×1920), but the clip is only 18.4 seconds long, below the requested 20–30 second range.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/e2479f4eefeb463ba33c50f244fd2be0.png?v=1","evidence_url":"https://aidemos.com/evidence/69db4d2f-5b9b-4753-8f89-173a86326bf0"},{"id":"d06a1b86-8996-4392-a912-35b98f2a9700","tool":"synthesia","tool_name":"Synthesia","verdict":"failed","score":null,"score_total":null,"note":"The FutureSmart output was generated in landscape rather than the requested vertical 9:16 short-form format.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/6217146be0434680a0d9e8add56290da.mp4?v=1","evidence_url":"https://aidemos.com/evidence/d06a1b86-8996-4392-a912-35b98f2a9700"}],"other_criteria":[{"id":"2399684a-6762-4a39-a524-676aeddfa25c","criterion":"avatar-realism","criterion_name":"Avatar realism","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"The presenter stays visually believable overall, but the output has a fused/warped right hand at 1s and 14.9s, which breaks the realism enough to keep it from being cleanly convincing.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/2399684a-6762-4a39-a524-676aeddfa25c"},{"id":"fc836d16-388e-4ee4-bec0-cb53fc29b6f0","criterion":"caption-quality","criterion_name":"Caption quality","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"Captioning is absent in the FutureSmart run: the researcher checked 8 timestamps from 0s through 14.9s and found zero burned-in text anywhere in the clip.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/fc836d16-388e-4ee4-bec0-cb53fc29b6f0"}],"appears_in":[{"page_type":"ranking","slug":"ugc-video-ad-generators","title":"Best AI Tools for Generating UGC-Style Video Ads With AI Avatars","url":"https://aidemos.com/best/ugc-video-ad-generators","binding":"run"}],"same_scenario":[{"id":"69db4d2f-5b9b-4753-8f89-173a86326bf0","tool":"joggai","tool_name":"JoggAI","verdict":"mixed","score":null,"score_total":null,"note":"The export format is correct at full-bleed vertical 9:16 (1080×1920), but the clip is only 18.4 seconds long, below the requested 20–30 second range."},{"id":"d06a1b86-8996-4392-a912-35b98f2a9700","tool":"synthesia","tool_name":"Synthesia","verdict":"failed","score":null,"score_total":null,"note":"The FutureSmart output was generated in landscape rather than the requested vertical 9:16 short-form format."}]}