{"observation":{"id":"74670d9d-491c-4219-98d9-1053b07f50f3","tool":"quizizz","tool_name":"Quizizz","criterion":"are-the-questions-actually-good","criterion_name":"Are the Questions Actually Good?","criterion_definition":"Whether the questions test understanding at the requested difficulty level rather than only surface-level recall or low-quality near-duplicates.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Question quality is the heart of the job here; weak or shallow questions mean the tool is failing at generating useful quizzes. (3 of 3 judges)","scenario":"ncert-class-12-biology-chapter-pdf-on-sexual-reproduction-in-flowering-plants","scenario_name":"NCERT Class 12 Biology chapter PDF on sexual reproduction in flowering plants","group_tag":"sexual-reproduction-in-flowering-plants","scenario_description":"A dense typed NCERT Class 12 Biology chapter on sexual reproduction in flowering plants, including technical terminology and diagrams. It was used to test how well tools extract content and generate high-quality questions from a long, structured study PDF.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://d3epheqghktydj.cloudfront.net/generate-quizzes-and-practice-tests-from-input1-primary-366aec5861ae.pdf","role":"input","filename":"input1_primary.pdf"}],"stresses":["long-form document extraction","technical biology terminology","diagram-heavy content","question generation quality from clean typed text"],"verdict":"mixed","score":null,"score_total":null,"note":"Even with DOK Level 3 selected, the PDF run still included surface-level items; the report says roughly 5–6 of 20 questions were shallow, such as flower symbolism and the three parts of a pistil.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/8607e53301e148cda9b89d0503eeff6f.pdf?v=1","role":"input","alt":"8607e53301e148cda9b89d0503eeff6f.pdf"},{"url":"https://cdn.futuresmart.ai/public/aidemos/8581a072c84147709b9619fa548fe5d6.png?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/70c3abb8687048b680b1cfdd382dde91.mp4?v=1","role":"context","alt":null}],"run_id":"de7b348e-4326-4fbf-87c1-b104750bc459","study_title":"Generate Quizzes and Practice Tests from Study Material Using AI","study_kind":"generation","research_task":"86ba51dce","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://d3epheqghktydj.cloudfront.net/generate-quizzes-and-practice-tests-from-input1-primary-366aec5861ae.pdf","filename":"input1_primary.pdf","alt":"NCERT Class 12 Biology chapter PDF on sexual reproduction in flowering plants","role":"input"}],"modality":"pdf","stresses":["long-form document extraction","technical biology terminology","diagram-heavy content","question generation quality from clean typed text"]},"tool_page_slug":"quizizz","tool_url":"https://aidemos.com/tools/quizizz","permalink":"https://aidemos.com/evidence/74670d9d-491c-4219-98d9-1053b07f50f3","api_url":"https://ai.aidemos.com/v1/observations/74670d9d-491c-4219-98d9-1053b07f50f3"},"peers":[{"id":"4b226047-ef59-4f5c-adef-8be459e380d1","tool":"doctoquiz","tool_name":"DoctoQuiz","verdict":"mixed","score":null,"score_total":null,"note":"Produces cleanly formatted multiple-choice questions, but the set stays mostly at recall level and includes at least one ambiguous item where two options are defensibly true.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/77c844c5170a48c2bb161af89d8c6faa.png?v=1","evidence_url":"https://aidemos.com/evidence/4b226047-ef59-4f5c-adef-8be459e380d1"},{"id":"b98f0ab1-d272-45af-b348-3bdae0087ea5","tool":"edcafe-ai","tool_name":"Edcafe AI","verdict":"struggled","score":null,"score_total":null,"note":"When it falls back to a typed topic label, the quiz generator stays at surface-level recall, producing MCQ-only items rather than deeper comprehension questions.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/609815a8cbe74f14a3227160ac87398d.png?v=1","evidence_url":"https://aidemos.com/evidence/b98f0ab1-d272-45af-b348-3bdae0087ea5"},{"id":"eade66c7-0510-40e6-aa86-b554aee80d21","tool":"knowt","tool_name":"Knowt","verdict":"mixed","score":null,"score_total":null,"note":"On the typed PDF, the questions are clearly on-topic and exam-style, but they are mostly direct recall rather than deeply reframed or high-twist questions.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/f80c400d05e34fe8a232cbdfaa1bb2bf.png?v=1","evidence_url":"https://aidemos.com/evidence/eade66c7-0510-40e6-aa86-b554aee80d21"},{"id":"ee64a3f8-5dc8-4b2a-b9ed-ba292fe9442a","tool":"pdf-to-quiz","tool_name":"PDF to Quiz","verdict":"worked","score":null,"score_total":null,"note":"On the typed PDF it generated inferential MCQs that went beyond recall, including questions about why a pollen grain is the male gametophyte and which cell supplies resources for pollen tube growth.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/8607e53301e148cda9b89d0503eeff6f.pdf?v=1","evidence_url":"https://aidemos.com/evidence/ee64a3f8-5dc8-4b2a-b9ed-ba292fe9442a"},{"id":"100e011a-110e-4f7a-b15a-496026b99533","tool":"questionwell","tool_name":"QuestionWell","verdict":"mixed","score":null,"score_total":null,"note":"Produces genuinely higher-order Discussion questions that ask learners to compare mechanisms and reason about real-world effects, but the same run also includes basic recall-style items such as fill-in-the-blank and true/false questions.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/109f29a0710b40f0b7447b2e331ba8ec.png?v=1","evidence_url":"https://aidemos.com/evidence/100e011a-110e-4f7a-b15a-496026b99533"},{"id":"c3a50a22-8c1e-489b-a9e9-512e3f77a09c","tool":"quizgecko","tool_name":"Quizgecko","verdict":"mixed","score":null,"score_total":null,"note":"Some questions use specific botanical framing and check understanding, but the set still caps out at recall-level and lacks inferential or application-based questions.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/83b9dfde13d546abadb4063d7103a072.png?v=1","evidence_url":"https://aidemos.com/evidence/c3a50a22-8c1e-489b-a9e9-512e3f77a09c"},{"id":"579c71e1-6e46-466e-86f6-7b2a59085f64","tool":"quizrise","tool_name":"QuizRise","verdict":"struggled","score":null,"score_total":null,"note":"On the NCERT PDF run, the requested Hard level still came out as easy-to-medium recall-style items, and the explanations shown in the quiz view did not include source citations back to the material. That makes the set usable, but not convincingly hard or especially well-grounded.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/b5d00401d520425783583aa75eb13d35.png?v=1","evidence_url":"https://aidemos.com/evidence/579c71e1-6e46-466e-86f6-7b2a59085f64"},{"id":"e95572bf-3dd1-4c65-a4bb-99927d86e621","tool":"studyglen","tool_name":"Studyglen","verdict":"failed","score":null,"score_total":null,"note":"One generated short-answer question leaked the internal label 'Visual: STRUCTURAL (v2)' directly into the visible stem, which is a visible output-quality failure.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/52130cc805544cd8b07c21b4f2c08dfa.png?v=1","evidence_url":"https://aidemos.com/evidence/e95572bf-3dd1-4c65-a4bb-99927d86e621"},{"id":"2c71d274-9808-4780-9616-334c59a8aa18","tool":"studyx-ai","tool_name":"StudyX AI","verdict":"worked","score":null,"score_total":null,"note":"The chat-based reframe feature upgrades a plain MCQ into a more applied, scenario-based question while keeping the same biology concept and answer choices aligned to the chapter.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/30d7e5b5130c4e4fa9abdb88afa3927c.png?v=1","evidence_url":"https://aidemos.com/evidence/2c71d274-9808-4780-9616-334c59a8aa18"}],"other_criteria":[{"id":"45db708e-69b1-441e-969b-6b1a2f9d211d","criterion":"can-it-read-your-material","criterion_name":"Can It Read Your Material?","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The typed NCERT PDF was ingested successfully and turned into a 20-question biology quiz that reached chapter-specific details such as tapetum, pollen exine, pollination, and hybrid seeds.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/45db708e-69b1-441e-969b-6b1a2f9d211d"}],"appears_in":[{"page_type":"ranking","slug":"quiz-generators","title":"Best AI Tools for Generating Quizzes and Practice Tests from Study Material","url":"https://aidemos.com/best/quiz-generators","binding":"run"},{"page_type":"tool","slug":"doctoquiz","title":null,"url":"https://aidemos.com/tools/doctoquiz","binding":"run"}],"same_scenario":[{"id":"4b226047-ef59-4f5c-adef-8be459e380d1","tool":"doctoquiz","tool_name":"DoctoQuiz","verdict":"mixed","score":null,"score_total":null,"note":"Produces cleanly formatted multiple-choice questions, but the set stays mostly at recall level and includes at least one ambiguous item where two options are defensibly true."},{"id":"b98f0ab1-d272-45af-b348-3bdae0087ea5","tool":"edcafe-ai","tool_name":"Edcafe AI","verdict":"struggled","score":null,"score_total":null,"note":"When it falls back to a typed topic label, the quiz generator stays at surface-level recall, producing MCQ-only items rather than deeper comprehension questions."},{"id":"eade66c7-0510-40e6-aa86-b554aee80d21","tool":"knowt","tool_name":"Knowt","verdict":"mixed","score":null,"score_total":null,"note":"On the typed PDF, the questions are clearly on-topic and exam-style, but they are mostly direct recall rather than deeply reframed or high-twist questions."},{"id":"ee64a3f8-5dc8-4b2a-b9ed-ba292fe9442a","tool":"pdf-to-quiz","tool_name":"PDF to Quiz","verdict":"worked","score":null,"score_total":null,"note":"On the typed PDF it generated inferential MCQs that went beyond recall, including questions about why a pollen grain is the male gametophyte and which cell supplies resources for pollen tube growth."},{"id":"100e011a-110e-4f7a-b15a-496026b99533","tool":"questionwell","tool_name":"QuestionWell","verdict":"mixed","score":null,"score_total":null,"note":"Produces genuinely higher-order Discussion questions that ask learners to compare mechanisms and reason about real-world effects, but the same run also includes basic recall-style items such as fill-in-the-blank and true/false questions."},{"id":"c3a50a22-8c1e-489b-a9e9-512e3f77a09c","tool":"quizgecko","tool_name":"Quizgecko","verdict":"mixed","score":null,"score_total":null,"note":"Some questions use specific botanical framing and check understanding, but the set still caps out at recall-level and lacks inferential or application-based questions."},{"id":"579c71e1-6e46-466e-86f6-7b2a59085f64","tool":"quizrise","tool_name":"QuizRise","verdict":"struggled","score":null,"score_total":null,"note":"On the NCERT PDF run, the requested Hard level still came out as easy-to-medium recall-style items, and the explanations shown in the quiz view did not include source citations back to the material. That makes the set usable, but not convincingly hard or especially well-grounded."},{"id":"e95572bf-3dd1-4c65-a4bb-99927d86e621","tool":"studyglen","tool_name":"Studyglen","verdict":"failed","score":null,"score_total":null,"note":"One generated short-answer question leaked the internal label 'Visual: STRUCTURAL (v2)' directly into the visible stem, which is a visible output-quality failure."},{"id":"2c71d274-9808-4780-9616-334c59a8aa18","tool":"studyx-ai","tool_name":"StudyX AI","verdict":"worked","score":null,"score_total":null,"note":"The chat-based reframe feature upgrades a plain MCQ into a more applied, scenario-based question while keeping the same biology concept and answer choices aligned to the chapter."}]}