{"observation":{"id":"f8bccaa7-10f8-477a-a445-4e6ed5f0049e","tool":"kodisc","tool_name":"Kodisc","criterion":"animation-smoothness","criterion_name":"Animation smoothness","criterion_definition":"No overlapping elements or broken transitions","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Broken transitions or overlapping elements directly undermine the quality of the generated animation. (3 of 3 judges)","scenario":"rag-ingestion-pipeline","scenario_name":"RAG Ingestion Pipeline","group_tag":null,"scenario_description":"A text prompt asking a tool to create an animated flowchart of a RAG ingestion pipeline: document upload, text extraction, chunking, embeddings, and parallel storage into a vector database and a metadata store. It stresses sequential pipeline animation plus a branch into two output paths.","modality":"text","input_text":"Create an animated flowchart titled \"RAG Ingestion Pipeline\". A Document is uploaded and its text is extracted. The extracted text is split into smaller chunks. Each chunk is converted into embeddings. The generated embeddings are stored in a Vector Database, and the metadata is stored in a Metadata Store.","input_artifact_refs":[],"stresses":["Sequential process visualization","Parallel branching into two outputs","Label accuracy for pipeline stages","Animated flowchart generation from plain text"],"verdict":"failed","score":null,"score_total":null,"note":"Can freeze mid-animation: every frame from about 15 seconds to the final frame is pixel-identical, leaving the output stuck on an unresolved glitch.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/research-media-kodisc-20output-201-6ca24ebf47c5.mp4","role":"output","alt":null}],"run_id":"e7fbb451-ba1e-4de0-ad94-41cde2700aef","study_title":"Generate Diagram Animations from Text Descriptions","study_kind":"generation","research_task":"86b8vp172","tested_at":null,"completeness":"input-and-output","input":{"state":"text","text":"Create an animated flowchart titled \"RAG Ingestion Pipeline\". A Document is uploaded and its text is extracted. The extracted text is split into smaller chunks. Each chunk is converted into embeddings. The generated embeddings are stored in a Vector Database, and the metadata is stored in a Metadata Store.","files":[],"modality":"text","stresses":["Sequential process visualization","Parallel branching into two outputs","Label accuracy for pipeline stages","Animated flowchart generation from plain text"]},"tool_page_slug":"kodisc","tool_url":"https://aidemos.com/tools/kodisc","permalink":"https://aidemos.com/evidence/f8bccaa7-10f8-477a-a445-4e6ed5f0049e","api_url":"https://ai.aidemos.com/v1/observations/f8bccaa7-10f8-477a-a445-4e6ed5f0049e"},"peers":[{"id":"bf7ba507-9118-486c-a911-42d56aef152e","tool":"animg","tool_name":"AnimG","verdict":"struggled","score":null,"score_total":null,"note":"The animation can fade out supporting icons before the transition is resolved; in the tested RAG flowchart, the waveform icons disappear and the embedding label and arrow are left floating without a visible base.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-image-092669e29551.png","evidence_url":"https://aidemos.com/evidence/bf7ba507-9118-486c-a911-42d56aef152e"},{"id":"81abe789-fc06-4451-8a2c-ad5a638860be","tool":"claude-ai","tool_name":"Claude AI","verdict":"worked","score":null,"score_total":null,"note":"Uses genuine connector-following motion with a continuous loop, rather than a static reveal, and the tested animation shows no overlap or broken transitions.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-claude-20output-201-51d86eb21e0d.mp4","evidence_url":"https://aidemos.com/evidence/81abe789-fc06-4451-8a2c-ad5a638860be"},{"id":"0f559835-5a1d-41a2-a4f2-f9356906310d","tool":"framia","tool_name":"Framia","verdict":"worked","score":null,"score_total":null,"note":"The motion is described as smooth and visually polished, with consistent camera framing and no broken transitions reported for the RAG run.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-framia-converge-ai-20output-201-d90de8918e39.mp4","evidence_url":"https://aidemos.com/evidence/0f559835-5a1d-41a2-a4f2-f9356906310d"},{"id":"93bb8f46-e890-4890-85c6-963df5189b10","tool":"replit","tool_name":"Replit","verdict":"worked","score":null,"score_total":null,"note":"The reveal animation was smooth, with the report explicitly stating no overlaps, no ghost artifacts, and no rendering bugs in frame-by-frame review.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-replit-20output-201-8ed41f8146df.mp4","evidence_url":"https://aidemos.com/evidence/93bb8f46-e890-4890-85c6-963df5189b10"}],"other_criteria":[{"id":"9a01d3b2-b773-4fa6-9778-529d4d3c2b56","criterion":"text-to-animation-accuracy","criterion_name":"Text-to-animation accuracy","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Renders the full RAG ingestion sequence correctly, including document upload, text extraction, chunking, embedding generation, and fan-out to both a vector database and a metadata store.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/9a01d3b2-b773-4fa6-9778-529d4d3c2b56"},{"id":"a162e389-b71c-4998-9b26-3df71022071e","criterion":"visual-clarity","criterion_name":"Visual clarity","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"Leaves overprinted labels that reduce readability: the chunking stage shows garbled duplicate text, and the embedding stage is obscured by stacked icons.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/a162e389-b71c-4998-9b26-3df71022071e"}],"appears_in":[{"page_type":"ranking","slug":"diagram-animation-generators","title":"Best AI Tools to Generate Diagram Animations from Text Descriptions","url":"https://aidemos.com/best/diagram-animation-generators","binding":"run"}],"same_scenario":[{"id":"bf7ba507-9118-486c-a911-42d56aef152e","tool":"animg","tool_name":"AnimG","verdict":"struggled","score":null,"score_total":null,"note":"The animation can fade out supporting icons before the transition is resolved; in the tested RAG flowchart, the waveform icons disappear and the embedding label and arrow are left floating without a visible base."},{"id":"81abe789-fc06-4451-8a2c-ad5a638860be","tool":"claude-ai","tool_name":"Claude AI","verdict":"worked","score":null,"score_total":null,"note":"Uses genuine connector-following motion with a continuous loop, rather than a static reveal, and the tested animation shows no overlap or broken transitions."},{"id":"0f559835-5a1d-41a2-a4f2-f9356906310d","tool":"framia","tool_name":"Framia","verdict":"worked","score":null,"score_total":null,"note":"The motion is described as smooth and visually polished, with consistent camera framing and no broken transitions reported for the RAG run."},{"id":"93bb8f46-e890-4890-85c6-963df5189b10","tool":"replit","tool_name":"Replit","verdict":"worked","score":null,"score_total":null,"note":"The reveal animation was smooth, with the report explicitly stating no overlaps, no ghost artifacts, and no rendering bugs in frame-by-frame review."}]}