{"observation":{"id":"91b7cde8-f3b8-42ec-9ec1-cd3d79e9aec5","tool":"animg","tool_name":"AnimG","criterion":"animation-smoothness","criterion_name":"Animation smoothness","criterion_definition":"No overlapping elements or broken transitions","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Broken transitions or overlapping elements directly undermine the quality of the generated animation. (3 of 3 judges)","scenario":"rag-ingestion-pipeline","scenario_name":"RAG Ingestion Pipeline","group_tag":null,"scenario_description":"A text prompt asking a tool to create an animated flowchart of a RAG ingestion pipeline: document upload, text extraction, chunking, embeddings, and parallel storage into a vector database and a metadata store. It stresses sequential pipeline animation plus a branch into two output paths.","modality":"text","input_text":"Create an animated flowchart titled \"RAG Ingestion Pipeline\". A Document is uploaded and its text is extracted. The extracted text is split into smaller chunks. Each chunk is converted into embeddings. The generated embeddings are stored in a Vector Database, and the metadata is stored in a Metadata Store.","input_artifact_refs":[],"stresses":["Sequential process visualization","Parallel branching into two outputs","Label accuracy for pipeline stages","Animated flowchart generation from plain text"],"verdict":"struggled","score":null,"score_total":null,"note":"The animation can remove intermediate nodes while leaving connectors behind; in the tested RAG flowchart, the Chunk 1/2/3 boxes vanish and the arrows remain pointing at empty space.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/research-media-image-0376253f6906.png","role":"output","alt":null}],"run_id":"e7fbb451-ba1e-4de0-ad94-41cde2700aef","study_title":"Generate Diagram Animations from Text Descriptions","study_kind":"generation","research_task":"86b8vp172","tested_at":null,"completeness":"input-and-output","input":{"state":"text","text":"Create an animated flowchart titled \"RAG Ingestion Pipeline\". A Document is uploaded and its text is extracted. The extracted text is split into smaller chunks. Each chunk is converted into embeddings. The generated embeddings are stored in a Vector Database, and the metadata is stored in a Metadata Store.","files":[],"modality":"text","stresses":["Sequential process visualization","Parallel branching into two outputs","Label accuracy for pipeline stages","Animated flowchart generation from plain text"]},"tool_page_slug":"animg","tool_url":"https://aidemos.com/tools/animg","permalink":"https://aidemos.com/evidence/91b7cde8-f3b8-42ec-9ec1-cd3d79e9aec5","api_url":"https://ai.aidemos.com/v1/observations/91b7cde8-f3b8-42ec-9ec1-cd3d79e9aec5"},"peers":[{"id":"81abe789-fc06-4451-8a2c-ad5a638860be","tool":"claude-ai","tool_name":"Claude AI","verdict":"worked","score":null,"score_total":null,"note":"Uses genuine connector-following motion with a continuous loop, rather than a static reveal, and the tested animation shows no overlap or broken transitions.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-claude-20output-201-51d86eb21e0d.mp4","evidence_url":"https://aidemos.com/evidence/81abe789-fc06-4451-8a2c-ad5a638860be"},{"id":"0f559835-5a1d-41a2-a4f2-f9356906310d","tool":"framia","tool_name":"Framia","verdict":"worked","score":null,"score_total":null,"note":"The motion is described as smooth and visually polished, with consistent camera framing and no broken transitions reported for the RAG run.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-framia-converge-ai-20output-201-d90de8918e39.mp4","evidence_url":"https://aidemos.com/evidence/0f559835-5a1d-41a2-a4f2-f9356906310d"},{"id":"22d6b6c8-a44c-45f4-b37b-c3bdcf1b8fd0","tool":"kodisc","tool_name":"Kodisc","verdict":"mixed","score":null,"score_total":null,"note":"Does not stage the first four pipeline blocks one by one; they appear together in a single burst around 6 seconds into the render.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-kodisc-20output-201-6ca24ebf47c5.mp4","evidence_url":"https://aidemos.com/evidence/22d6b6c8-a44c-45f4-b37b-c3bdcf1b8fd0"},{"id":"93bb8f46-e890-4890-85c6-963df5189b10","tool":"replit","tool_name":"Replit","verdict":"worked","score":null,"score_total":null,"note":"The reveal animation was smooth, with the report explicitly stating no overlaps, no ghost artifacts, and no rendering bugs in frame-by-frame review.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-replit-20output-201-8ed41f8146df.mp4","evidence_url":"https://aidemos.com/evidence/93bb8f46-e890-4890-85c6-963df5189b10"}],"other_criteria":[{"id":"cf48ca25-620f-4f5b-8a19-6cbc68e76557","criterion":"text-to-animation-accuracy","criterion_name":"Text-to-animation accuracy","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"The tool can fail to keep parallel storage targets visually distinct; in the tested RAG flowchart, Vector DB and Metadata Store end up stacked on top of each other instead of separating into two nodes.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/cf48ca25-620f-4f5b-8a19-6cbc68e76557"},{"id":"ddb9a0c3-5b49-446a-b6dd-bec8a27b61dc","criterion":"text-to-animation-accuracy","criterion_name":"Text-to-animation accuracy","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"The tool can draw a dead-end branch with no destination node; in the tested RAG flowchart, the Retrieval arrow points into empty space.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/ddb9a0c3-5b49-446a-b6dd-bec8a27b61dc"},{"id":"af6a9ce1-b173-48c9-80ef-563f24321274","criterion":"text-to-animation-accuracy","criterion_name":"Text-to-animation accuracy","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"The tool can omit a distinct incoming connector for the metadata branch; in the tested RAG flowchart, Metadata Store has no arrow of its own and only piggybacks on the Vector DB connection.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/af6a9ce1-b173-48c9-80ef-563f24321274"},{"id":"9e6fb9c6-28d0-4fde-b6b4-8457509022e1","criterion":"visual-clarity","criterion_name":"Visual clarity","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"Connector lines can partially obscure node labels; in the tested RAG flowchart, the line passes straight through the Upload text instead of staying clear of the label.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/9e6fb9c6-28d0-4fde-b6b4-8457509022e1"}],"appears_in":[{"page_type":"ranking","slug":"diagram-animation-generators","title":"Best AI Tools to Generate Diagram Animations from Text Descriptions","url":"https://aidemos.com/best/diagram-animation-generators","binding":"run"}],"same_scenario":[{"id":"81abe789-fc06-4451-8a2c-ad5a638860be","tool":"claude-ai","tool_name":"Claude AI","verdict":"worked","score":null,"score_total":null,"note":"Uses genuine connector-following motion with a continuous loop, rather than a static reveal, and the tested animation shows no overlap or broken transitions."},{"id":"0f559835-5a1d-41a2-a4f2-f9356906310d","tool":"framia","tool_name":"Framia","verdict":"worked","score":null,"score_total":null,"note":"The motion is described as smooth and visually polished, with consistent camera framing and no broken transitions reported for the RAG run."},{"id":"22d6b6c8-a44c-45f4-b37b-c3bdcf1b8fd0","tool":"kodisc","tool_name":"Kodisc","verdict":"mixed","score":null,"score_total":null,"note":"Does not stage the first four pipeline blocks one by one; they appear together in a single burst around 6 seconds into the render."},{"id":"93bb8f46-e890-4890-85c6-963df5189b10","tool":"replit","tool_name":"Replit","verdict":"worked","score":null,"score_total":null,"note":"The reveal animation was smooth, with the report explicitly stating no overlaps, no ghost artifacts, and no rendering bugs in frame-by-frame review."}]}