{"observation":{"id":"c3a50a22-8c1e-489b-a9e9-512e3f77a09c","tool":"quizgecko","tool_name":"Quizgecko","criterion":"are-the-questions-actually-good","criterion_name":"Are the Questions Actually Good?","criterion_definition":"Whether the questions test understanding at the requested difficulty level rather than only surface-level recall or low-quality near-duplicates.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Question quality is the heart of the job here; weak or shallow questions mean the tool is failing at generating useful quizzes. (3 of 3 judges)","scenario":"ncert-class-12-biology-chapter-pdf-on-sexual-reproduction-in-flowering-plants","scenario_name":"NCERT Class 12 Biology chapter PDF on sexual reproduction in flowering plants","group_tag":"sexual-reproduction-in-flowering-plants","scenario_description":"A dense typed NCERT Class 12 Biology chapter on sexual reproduction in flowering plants, including technical terminology and diagrams. It was used to test how well tools extract content and generate high-quality questions from a long, structured study PDF.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://d3epheqghktydj.cloudfront.net/generate-quizzes-and-practice-tests-from-input1-primary-366aec5861ae.pdf","role":"input","filename":"input1_primary.pdf"}],"stresses":["long-form document extraction","technical biology terminology","diagram-heavy content","question generation quality from clean typed text"],"verdict":"mixed","score":null,"score_total":null,"note":"Some questions use specific botanical framing and check understanding, but the set still caps out at recall-level and lacks inferential or application-based questions.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/83b9dfde13d546abadb4063d7103a072.png?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/2b210e553c2346ab92a903ca799905a4.png?v=1","role":"output","alt":null}],"run_id":"de7b348e-4326-4fbf-87c1-b104750bc459","study_title":"Generate Quizzes and Practice Tests from Study Material Using AI","study_kind":"generation","research_task":"86ba51dce","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://d3epheqghktydj.cloudfront.net/generate-quizzes-and-practice-tests-from-input1-primary-366aec5861ae.pdf","filename":"input1_primary.pdf","alt":"NCERT Class 12 Biology chapter PDF on sexual reproduction in flowering plants","role":"input"}],"modality":"pdf","stresses":["long-form document extraction","technical biology terminology","diagram-heavy content","question generation quality from clean typed text"]},"tool_page_slug":"quizgecko","tool_url":"https://aidemos.com/tools/quizgecko","permalink":"https://aidemos.com/evidence/c3a50a22-8c1e-489b-a9e9-512e3f77a09c","api_url":"https://ai.aidemos.com/v1/observations/c3a50a22-8c1e-489b-a9e9-512e3f77a09c"},"peers":[{"id":"4b226047-ef59-4f5c-adef-8be459e380d1","tool":"doctoquiz","tool_name":"DoctoQuiz","verdict":"mixed","score":null,"score_total":null,"note":"Produces cleanly formatted multiple-choice questions, but the set stays mostly at recall level and includes at least one ambiguous item where two options are defensibly true.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/77c844c5170a48c2bb161af89d8c6faa.png?v=1","evidence_url":"https://aidemos.com/evidence/4b226047-ef59-4f5c-adef-8be459e380d1"},{"id":"b98f0ab1-d272-45af-b348-3bdae0087ea5","tool":"edcafe-ai","tool_name":"Edcafe AI","verdict":"struggled","score":null,"score_total":null,"note":"When it falls back to a typed topic label, the quiz generator stays at surface-level recall, producing MCQ-only items rather than deeper comprehension questions.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/609815a8cbe74f14a3227160ac87398d.png?v=1","evidence_url":"https://aidemos.com/evidence/b98f0ab1-d272-45af-b348-3bdae0087ea5"},{"id":"eade66c7-0510-40e6-aa86-b554aee80d21","tool":"knowt","tool_name":"Knowt","verdict":"mixed","score":null,"score_total":null,"note":"On the typed PDF, the questions are clearly on-topic and exam-style, but they are mostly direct recall rather than deeply reframed or high-twist questions.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/f80c400d05e34fe8a232cbdfaa1bb2bf.png?v=1","evidence_url":"https://aidemos.com/evidence/eade66c7-0510-40e6-aa86-b554aee80d21"},{"id":"ee64a3f8-5dc8-4b2a-b9ed-ba292fe9442a","tool":"pdf-to-quiz","tool_name":"PDF to Quiz","verdict":"worked","score":null,"score_total":null,"note":"On the typed PDF it generated inferential MCQs that went beyond recall, including questions about why a pollen grain is the male gametophyte and which cell supplies resources for pollen tube growth.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/8607e53301e148cda9b89d0503eeff6f.pdf?v=1","evidence_url":"https://aidemos.com/evidence/ee64a3f8-5dc8-4b2a-b9ed-ba292fe9442a"},{"id":"100e011a-110e-4f7a-b15a-496026b99533","tool":"questionwell","tool_name":"QuestionWell","verdict":"mixed","score":null,"score_total":null,"note":"Produces genuinely higher-order Discussion questions that ask learners to compare mechanisms and reason about real-world effects, but the same run also includes basic recall-style items such as fill-in-the-blank and true/false questions.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/109f29a0710b40f0b7447b2e331ba8ec.png?v=1","evidence_url":"https://aidemos.com/evidence/100e011a-110e-4f7a-b15a-496026b99533"},{"id":"74670d9d-491c-4219-98d9-1053b07f50f3","tool":"quizizz","tool_name":"Quizizz","verdict":"mixed","score":null,"score_total":null,"note":"Even with DOK Level 3 selected, the PDF run still included surface-level items; the report says roughly 5–6 of 20 questions were shallow, such as flower symbolism and the three parts of a pistil.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/8607e53301e148cda9b89d0503eeff6f.pdf?v=1","evidence_url":"https://aidemos.com/evidence/74670d9d-491c-4219-98d9-1053b07f50f3"},{"id":"579c71e1-6e46-466e-86f6-7b2a59085f64","tool":"quizrise","tool_name":"QuizRise","verdict":"struggled","score":null,"score_total":null,"note":"On the NCERT PDF run, the requested Hard level still came out as easy-to-medium recall-style items, and the explanations shown in the quiz view did not include source citations back to the material. That makes the set usable, but not convincingly hard or especially well-grounded.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/b5d00401d520425783583aa75eb13d35.png?v=1","evidence_url":"https://aidemos.com/evidence/579c71e1-6e46-466e-86f6-7b2a59085f64"},{"id":"e95572bf-3dd1-4c65-a4bb-99927d86e621","tool":"studyglen","tool_name":"Studyglen","verdict":"failed","score":null,"score_total":null,"note":"One generated short-answer question leaked the internal label 'Visual: STRUCTURAL (v2)' directly into the visible stem, which is a visible output-quality failure.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/52130cc805544cd8b07c21b4f2c08dfa.png?v=1","evidence_url":"https://aidemos.com/evidence/e95572bf-3dd1-4c65-a4bb-99927d86e621"},{"id":"c09ec537-a8cf-45d7-addc-b7ff9a78d718","tool":"studyx-ai","tool_name":"StudyX AI","verdict":"worked","score":null,"score_total":null,"note":"Generates strong 4-option MCQs with closely related distractors that are hard to eliminate by pattern matching; in the PDF run, the double-fertilization question shows concept-level options rather than obvious throwaways.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/4981100116614889a30aaee74e0d647b.png?v=1","evidence_url":"https://aidemos.com/evidence/c09ec537-a8cf-45d7-addc-b7ff9a78d718"}],"other_criteria":[{"id":"6975a414-8ae0-4e75-8210-354fafd19afc","criterion":"can-it-read-your-material","criterion_name":"Can It Read Your Material?","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The tool ingests a dense typed chapter PDF well enough to organize the chapter into 4 topic modules and generate a 39-question quiz from it.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/6975a414-8ae0-4e75-8210-354fafd19afc"},{"id":"e5eff9e6-2881-4835-b420-245c4f36f532","criterion":"can-it-read-your-material","criterion_name":"Can It Read Your Material?","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"The PDF run handled the chapter text, but it produced zero diagram-based questions, leaving a diagram-heavy biology source only partially covered.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/e5eff9e6-2881-4835-b420-245c4f36f532"}],"appears_in":[{"page_type":"ranking","slug":"quiz-generators","title":"Best AI Tools for Generating Quizzes and Practice Tests from Study Material","url":"https://aidemos.com/best/quiz-generators","binding":"run"},{"page_type":"tool","slug":"doctoquiz","title":null,"url":"https://aidemos.com/tools/doctoquiz","binding":"run"}],"same_scenario":[{"id":"4b226047-ef59-4f5c-adef-8be459e380d1","tool":"doctoquiz","tool_name":"DoctoQuiz","verdict":"mixed","score":null,"score_total":null,"note":"Produces cleanly formatted multiple-choice questions, but the set stays mostly at recall level and includes at least one ambiguous item where two options are defensibly true."},{"id":"b98f0ab1-d272-45af-b348-3bdae0087ea5","tool":"edcafe-ai","tool_name":"Edcafe AI","verdict":"struggled","score":null,"score_total":null,"note":"When it falls back to a typed topic label, the quiz generator stays at surface-level recall, producing MCQ-only items rather than deeper comprehension questions."},{"id":"eade66c7-0510-40e6-aa86-b554aee80d21","tool":"knowt","tool_name":"Knowt","verdict":"mixed","score":null,"score_total":null,"note":"On the typed PDF, the questions are clearly on-topic and exam-style, but they are mostly direct recall rather than deeply reframed or high-twist questions."},{"id":"ee64a3f8-5dc8-4b2a-b9ed-ba292fe9442a","tool":"pdf-to-quiz","tool_name":"PDF to Quiz","verdict":"worked","score":null,"score_total":null,"note":"On the typed PDF it generated inferential MCQs that went beyond recall, including questions about why a pollen grain is the male gametophyte and which cell supplies resources for pollen tube growth."},{"id":"100e011a-110e-4f7a-b15a-496026b99533","tool":"questionwell","tool_name":"QuestionWell","verdict":"mixed","score":null,"score_total":null,"note":"Produces genuinely higher-order Discussion questions that ask learners to compare mechanisms and reason about real-world effects, but the same run also includes basic recall-style items such as fill-in-the-blank and true/false questions."},{"id":"74670d9d-491c-4219-98d9-1053b07f50f3","tool":"quizizz","tool_name":"Quizizz","verdict":"mixed","score":null,"score_total":null,"note":"Even with DOK Level 3 selected, the PDF run still included surface-level items; the report says roughly 5–6 of 20 questions were shallow, such as flower symbolism and the three parts of a pistil."},{"id":"579c71e1-6e46-466e-86f6-7b2a59085f64","tool":"quizrise","tool_name":"QuizRise","verdict":"struggled","score":null,"score_total":null,"note":"On the NCERT PDF run, the requested Hard level still came out as easy-to-medium recall-style items, and the explanations shown in the quiz view did not include source citations back to the material. That makes the set usable, but not convincingly hard or especially well-grounded."},{"id":"e95572bf-3dd1-4c65-a4bb-99927d86e621","tool":"studyglen","tool_name":"Studyglen","verdict":"failed","score":null,"score_total":null,"note":"One generated short-answer question leaked the internal label 'Visual: STRUCTURAL (v2)' directly into the visible stem, which is a visible output-quality failure."},{"id":"c09ec537-a8cf-45d7-addc-b7ff9a78d718","tool":"studyx-ai","tool_name":"StudyX AI","verdict":"worked","score":null,"score_total":null,"note":"Generates strong 4-option MCQs with closely related distractors that are hard to eliminate by pattern matching; in the PDF run, the double-fertilization question shows concept-level options rather than obvious throwaways."}]}