{"observation":{"id":"2d00ddc3-5c2f-4e48-ab54-e9b6cdfb2ac0","tool":"claude-artifacts","tool_name":"Claude Artifacts","criterion":"output-quality","criterion_name":"Output Quality","criterion_definition":"Conceptually matches the prompt and is visually usable.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"The tool is being chosen to produce usable code-based animations, so conceptual match to the prompt and visual usability are core success criteria. (3 of 3 judges)","scenario":"search-engines-basics-explainer","scenario_name":"Search engines basics explainer","group_tag":null,"scenario_description":"A vague, linear educational animation prompt asking the tool to explain how search engines work with minimal guidance. Designed to stress inference, scene structuring, pacing, and coherent visual flow from an underspecified request.","modality":"text","input_text":"Create an animation video explaining how search engines work.","input_artifact_refs":[],"stresses":[],"verdict":"worked","score":null,"score_total":null,"note":"Delivered a polished animation with smooth motion, clean hierarchy, and a recognizable search-engine workflow.","evidence_state":"verified","source":"first-party","artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/claude-artifacts-screen-recording-2026-04-30-141425.mp4","role":"output","alt":"Claude Artifacts — Screen Recording 2026-04-30 141425.mp4"}],"run_id":"c751a76b-a105-4b3e-9ff3-6c4d6adc9f04","study_title":"Generate Code-based Animations from text inputs","study_kind":"backfill","research_task":"86b9nnzmy","tested_at":"2026-06-24T08:34:34.142000+00:00","completeness":"input-and-output","input":{"state":"text","text":"Create an animation video explaining how search engines work.","files":[],"modality":"text","stresses":[]},"tool_page_slug":null,"tool_url":null,"permalink":"https://aidemos.com/evidence/2d00ddc3-5c2f-4e48-ab54-e9b6cdfb2ac0","api_url":"https://ai.aidemos.com/v1/observations/2d00ddc3-5c2f-4e48-ab54-e9b6cdfb2ac0"},"peers":[{"id":"5a63f48b-4d4b-4175-9fa4-c907315be462","tool":"animg","tool_name":"AnimG","verdict":"mixed","score":null,"score_total":null,"note":"Matches the approved search-engine workflow concept, but the final animation stays visually basic and shows minor clutter even on the low-complexity scene.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/animg-20260419-090730-searchengineworkflow.mp4","evidence_url":"https://aidemos.com/evidence/5a63f48b-4d4b-4175-9fa4-c907315be462"},{"id":"bae09520-6cfa-40cc-a95e-776d599142a6","tool":"antigravity","tool_name":"Antigravity","verdict":"worked","score":null,"score_total":null,"note":"Produces a visually readable 15-second explainer that presents crawling, indexing, ranking, and serving in sequence.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/antigravity-output.mp4","evidence_url":"https://aidemos.com/evidence/bae09520-6cfa-40cc-a95e-776d599142a6"},{"id":"df5fad76-48f1-4a26-8164-a82ffcbc7976","tool":"chatgpt-canvas","tool_name":"ChatGPT Canvas","verdict":"struggled","score":null,"score_total":null,"note":"The first-pass visual output was only a plain horizontal flowchart: text overflowed node boxes, there were no icons or visual hierarchy, and component differentiation was minimal.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/chatgpt-canvas-screen-recording-2026-04-30-170046.mp4","evidence_url":"https://aidemos.com/evidence/df5fad76-48f1-4a26-8164-a82ffcbc7976"},{"id":"12a9b2c1-b3be-4562-8a14-87ccf9f4540c","tool":"gemini-canvas","tool_name":"Gemini Canvas","verdict":"worked","score":null,"score_total":null,"note":"It can cover the full search-engine workflow—crawling, indexing, and ranking—without overlap or visual clutter.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/gemini-canvas-gemini-search-engine-animation.mp4","evidence_url":"https://aidemos.com/evidence/12a9b2c1-b3be-4562-8a14-87ccf9f4540c"},{"id":"e8505de0-5cac-48fb-811d-29a4753d36b6","tool":"grok","tool_name":"Grok","verdict":"failed","score":null,"score_total":null,"note":"Produces output that reads more like an interactive webpage than a video-like animation, with cramped spacing, overlapping elements, weak visual hierarchy, and visible controls.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/grok-screen-recording-2026-05-04-135441.mp4","evidence_url":"https://aidemos.com/evidence/e8505de0-5cac-48fb-811d-29a4753d36b6"},{"id":"0da2a252-41c7-4636-bdfb-32bc3df40e96","tool":"remotionvideo","tool_name":"RemotionVideo","verdict":"worked","score":null,"score_total":null,"note":"Produces a clean, polished explainer that visually communicates a 3-step crawl → index → rank flow with no overlap or layout clutter.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/remotionvideo-589f02b7-33c6-43cd-9d90-262a33680d94.mp4","evidence_url":"https://aidemos.com/evidence/0da2a252-41c7-4636-bdfb-32bc3df40e96"},{"id":"31d4ba1b-655a-4062-ae44-4c5622c0e687","tool":"replit","tool_name":"Replit","verdict":"worked","score":null,"score_total":null,"note":"Delivers clean motion graphics with clear sequencing and visual hierarchy, making the explanation immediately understandable.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/replit-search-engine-explainer-apr-18-15-53-35.mp4","evidence_url":"https://aidemos.com/evidence/31d4ba1b-655a-4062-ae44-4c5622c0e687"}],"other_criteria":[{"id":"66a96576-7f89-45d6-9960-686650ac8834","criterion":"code-generation-quality","criterion_name":"Code Generation Quality","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"It generated a runnable webpage on the first attempt, with autoplay working on load.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/66a96576-7f89-45d6-9960-686650ac8834"},{"id":"26cde3a7-20de-455b-924c-158433afa4eb","criterion":"export-experience","criterion_name":"Export Experience","rank_role":"context","verdict":"struggled","score":null,"score_total":null,"note":"Getting an MP4 was not native; the report says a browser-capture/render workflow was still required.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/26cde3a7-20de-455b-924c-158433afa4eb"},{"id":"2844fa71-9f82-4175-b908-5a20fb79ea92","criterion":"in-platform-preview","criterion_name":"In-Platform Preview","rank_role":"context","verdict":"worked","score":null,"score_total":null,"note":"The preview was available immediately in-platform, so the animation could be inspected before any external capture step.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/2844fa71-9f82-4175-b908-5a20fb79ea92"},{"id":"0675da71-4772-4c4e-a2a8-bb2b660f1cb2","criterion":"reliability","criterion_name":"Reliability","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"The animation works, but timing and pacing still often need manual refinement.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/0675da71-4772-4c4e-a2a8-bb2b660f1cb2"}],"appears_in":[{"page_type":"ranking","slug":"code-animation-generators","title":"Best AI Tools for Generating Code-Based Animations from Text Prompts","url":"https://aidemos.com/best/code-animation-generators","binding":"run"},{"page_type":"use-case","slug":"generate-code-animation-videos","title":"Generate Code-Based Animation Videos from Text Prompts Using AI","url":"https://aidemos.com/use-cases/generate-code-animation-videos","binding":"run"}],"same_scenario":[{"id":"5a63f48b-4d4b-4175-9fa4-c907315be462","tool":"animg","tool_name":"AnimG","verdict":"mixed","score":null,"score_total":null,"note":"Matches the approved search-engine workflow concept, but the final animation stays visually basic and shows minor clutter even on the low-complexity scene."},{"id":"bae09520-6cfa-40cc-a95e-776d599142a6","tool":"antigravity","tool_name":"Antigravity","verdict":"worked","score":null,"score_total":null,"note":"Produces a visually readable 15-second explainer that presents crawling, indexing, ranking, and serving in sequence."},{"id":"df5fad76-48f1-4a26-8164-a82ffcbc7976","tool":"chatgpt-canvas","tool_name":"ChatGPT Canvas","verdict":"struggled","score":null,"score_total":null,"note":"The first-pass visual output was only a plain horizontal flowchart: text overflowed node boxes, there were no icons or visual hierarchy, and component differentiation was minimal."},{"id":"12a9b2c1-b3be-4562-8a14-87ccf9f4540c","tool":"gemini-canvas","tool_name":"Gemini Canvas","verdict":"worked","score":null,"score_total":null,"note":"It can cover the full search-engine workflow—crawling, indexing, and ranking—without overlap or visual clutter."},{"id":"e8505de0-5cac-48fb-811d-29a4753d36b6","tool":"grok","tool_name":"Grok","verdict":"failed","score":null,"score_total":null,"note":"Produces output that reads more like an interactive webpage than a video-like animation, with cramped spacing, overlapping elements, weak visual hierarchy, and visible controls."},{"id":"0da2a252-41c7-4636-bdfb-32bc3df40e96","tool":"remotionvideo","tool_name":"RemotionVideo","verdict":"worked","score":null,"score_total":null,"note":"Produces a clean, polished explainer that visually communicates a 3-step crawl → index → rank flow with no overlap or layout clutter."},{"id":"31d4ba1b-655a-4062-ae44-4c5622c0e687","tool":"replit","tool_name":"Replit","verdict":"worked","score":null,"score_total":null,"note":"Delivers clean motion graphics with clear sequencing and visual hierarchy, making the explanation immediately understandable."}]}