{"observation":{"id":"100e011a-110e-4f7a-b15a-496026b99533","tool":"questionwell","tool_name":"QuestionWell","criterion":"are-the-questions-actually-good","criterion_name":"Are the Questions Actually Good?","criterion_definition":"Whether the questions test understanding at the requested difficulty level rather than only surface-level recall or low-quality near-duplicates.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Question quality is the heart of the job here; weak or shallow questions mean the tool is failing at generating useful quizzes. (3 of 3 judges)","scenario":"ncert-class-12-biology-chapter-pdf-on-sexual-reproduction-in-flowering-plants","scenario_name":"NCERT Class 12 Biology chapter PDF on sexual reproduction in flowering plants","group_tag":"sexual-reproduction-in-flowering-plants","scenario_description":"A dense typed NCERT Class 12 Biology chapter on sexual reproduction in flowering plants, including technical terminology and diagrams. It was used to test how well tools extract content and generate high-quality questions from a long, structured study PDF.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://d3epheqghktydj.cloudfront.net/generate-quizzes-and-practice-tests-from-input1-primary-366aec5861ae.pdf","role":"input","filename":"input1_primary.pdf"}],"stresses":["long-form document extraction","technical biology terminology","diagram-heavy content","question generation quality from clean typed text"],"verdict":"mixed","score":null,"score_total":null,"note":"Produces genuinely higher-order Discussion questions that ask learners to compare mechanisms and reason about real-world effects, but the same run also includes basic recall-style items such as fill-in-the-blank and true/false questions.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/109f29a0710b40f0b7447b2e331ba8ec.png?v=1","role":null,"alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/fc36c2da28384d14acc25a1cbc29e5d5.png?v=1","role":null,"alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/3a7a1ab0254e4f2f8eafe684e10821aa.png?v=1","role":null,"alt":null}],"run_id":"de7b348e-4326-4fbf-87c1-b104750bc459","study_title":"Generate Quizzes and Practice Tests from Study Material Using AI","study_kind":"generation","research_task":"86ba51dce","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://d3epheqghktydj.cloudfront.net/generate-quizzes-and-practice-tests-from-input1-primary-366aec5861ae.pdf","filename":"input1_primary.pdf","alt":"NCERT Class 12 Biology chapter PDF on sexual reproduction in flowering plants","role":"input"}],"modality":"pdf","stresses":["long-form document extraction","technical biology terminology","diagram-heavy content","question generation quality from clean typed text"]},"tool_page_slug":"questionwell","tool_url":"https://aidemos.com/tools/questionwell","permalink":"https://aidemos.com/evidence/100e011a-110e-4f7a-b15a-496026b99533","api_url":"https://ai.aidemos.com/v1/observations/100e011a-110e-4f7a-b15a-496026b99533"},"peers":[{"id":"4b226047-ef59-4f5c-adef-8be459e380d1","tool":"doctoquiz","tool_name":"DoctoQuiz","verdict":"mixed","score":null,"score_total":null,"note":"Produces cleanly formatted multiple-choice questions, but the set stays mostly at recall level and includes at least one ambiguous item where two options are defensibly true.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/77c844c5170a48c2bb161af89d8c6faa.png?v=1","evidence_url":"https://aidemos.com/evidence/4b226047-ef59-4f5c-adef-8be459e380d1"},{"id":"b98f0ab1-d272-45af-b348-3bdae0087ea5","tool":"edcafe-ai","tool_name":"Edcafe AI","verdict":"struggled","score":null,"score_total":null,"note":"When it falls back to a typed topic label, the quiz generator stays at surface-level recall, producing MCQ-only items rather than deeper comprehension questions.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/609815a8cbe74f14a3227160ac87398d.png?v=1","evidence_url":"https://aidemos.com/evidence/b98f0ab1-d272-45af-b348-3bdae0087ea5"},{"id":"eade66c7-0510-40e6-aa86-b554aee80d21","tool":"knowt","tool_name":"Knowt","verdict":"mixed","score":null,"score_total":null,"note":"On the typed PDF, the questions are clearly on-topic and exam-style, but they are mostly direct recall rather than deeply reframed or high-twist questions.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/f80c400d05e34fe8a232cbdfaa1bb2bf.png?v=1","evidence_url":"https://aidemos.com/evidence/eade66c7-0510-40e6-aa86-b554aee80d21"},{"id":"ee64a3f8-5dc8-4b2a-b9ed-ba292fe9442a","tool":"pdf-to-quiz","tool_name":"PDF to Quiz","verdict":"worked","score":null,"score_total":null,"note":"On the typed PDF it generated inferential MCQs that went beyond recall, including questions about why a pollen grain is the male gametophyte and which cell supplies resources for pollen tube growth.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/8607e53301e148cda9b89d0503eeff6f.pdf?v=1","evidence_url":"https://aidemos.com/evidence/ee64a3f8-5dc8-4b2a-b9ed-ba292fe9442a"},{"id":"c3a50a22-8c1e-489b-a9e9-512e3f77a09c","tool":"quizgecko","tool_name":"Quizgecko","verdict":"mixed","score":null,"score_total":null,"note":"Some questions use specific botanical framing and check understanding, but the set still caps out at recall-level and lacks inferential or application-based questions.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/83b9dfde13d546abadb4063d7103a072.png?v=1","evidence_url":"https://aidemos.com/evidence/c3a50a22-8c1e-489b-a9e9-512e3f77a09c"},{"id":"74670d9d-491c-4219-98d9-1053b07f50f3","tool":"quizizz","tool_name":"Quizizz","verdict":"mixed","score":null,"score_total":null,"note":"Even with DOK Level 3 selected, the PDF run still included surface-level items; the report says roughly 5–6 of 20 questions were shallow, such as flower symbolism and the three parts of a pistil.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/8607e53301e148cda9b89d0503eeff6f.pdf?v=1","evidence_url":"https://aidemos.com/evidence/74670d9d-491c-4219-98d9-1053b07f50f3"},{"id":"579c71e1-6e46-466e-86f6-7b2a59085f64","tool":"quizrise","tool_name":"QuizRise","verdict":"struggled","score":null,"score_total":null,"note":"On the NCERT PDF run, the requested Hard level still came out as easy-to-medium recall-style items, and the explanations shown in the quiz view did not include source citations back to the material. That makes the set usable, but not convincingly hard or especially well-grounded.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/b5d00401d520425783583aa75eb13d35.png?v=1","evidence_url":"https://aidemos.com/evidence/579c71e1-6e46-466e-86f6-7b2a59085f64"},{"id":"b592f4a3-f553-497d-b831-c68d17da508a","tool":"studyglen","tool_name":"Studyglen","verdict":"worked","score":null,"score_total":null,"note":"The generator can produce understanding-oriented prompts rather than only recall: one question asks learners to interpret a diagrammed Vallisneria pollination sequence, and another asks for a short explanation of why a megaspore mother cell yields four megaspores but only one persists functionally.","artifact_count":7,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/755a379869a24e28a2deb0e16c37b57d.png?v=1","evidence_url":"https://aidemos.com/evidence/b592f4a3-f553-497d-b831-c68d17da508a"},{"id":"f98d3c36-275e-4e98-91a9-56f5a685bcfc","tool":"studyx-ai","tool_name":"StudyX AI","verdict":"mixed","score":null,"score_total":null,"note":"Can generate 15 non-MCQ practice items only through a manual chat prompt: 5 true/false, 5 fill-in-the-blanks, and 5 short-answer questions, with an answer key requested in the same prompt. That expands format variety, but it is not native quiz generation and costs extra credits.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/b94752601d1140cfb7036bc483beceef.mp4?v=1","evidence_url":"https://aidemos.com/evidence/f98d3c36-275e-4e98-91a9-56f5a685bcfc"}],"other_criteria":[{"id":"6ec2ec39-9858-4b25-b855-efd94746dcb2","criterion":"can-it-read-your-material","criterion_name":"Can It Read Your Material?","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"Reads dense typed chapter PDFs well enough to generate a 30-question quiz plus a concise chapter summary, with sections organized into Comprehension, Discussion, and Vocabulary, but it does not turn diagrams into diagram-based questions.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/6ec2ec39-9858-4b25-b855-efd94746dcb2"}],"appears_in":[{"page_type":"ranking","slug":"quiz-generators","title":"Best AI Tools for Generating Quizzes and Practice Tests from Study Material","url":"https://aidemos.com/best/quiz-generators","binding":"run"},{"page_type":"tool","slug":"doctoquiz","title":null,"url":"https://aidemos.com/tools/doctoquiz","binding":"run"}],"same_scenario":[{"id":"4b226047-ef59-4f5c-adef-8be459e380d1","tool":"doctoquiz","tool_name":"DoctoQuiz","verdict":"mixed","score":null,"score_total":null,"note":"Produces cleanly formatted multiple-choice questions, but the set stays mostly at recall level and includes at least one ambiguous item where two options are defensibly true."},{"id":"b98f0ab1-d272-45af-b348-3bdae0087ea5","tool":"edcafe-ai","tool_name":"Edcafe AI","verdict":"struggled","score":null,"score_total":null,"note":"When it falls back to a typed topic label, the quiz generator stays at surface-level recall, producing MCQ-only items rather than deeper comprehension questions."},{"id":"eade66c7-0510-40e6-aa86-b554aee80d21","tool":"knowt","tool_name":"Knowt","verdict":"mixed","score":null,"score_total":null,"note":"On the typed PDF, the questions are clearly on-topic and exam-style, but they are mostly direct recall rather than deeply reframed or high-twist questions."},{"id":"ee64a3f8-5dc8-4b2a-b9ed-ba292fe9442a","tool":"pdf-to-quiz","tool_name":"PDF to Quiz","verdict":"worked","score":null,"score_total":null,"note":"On the typed PDF it generated inferential MCQs that went beyond recall, including questions about why a pollen grain is the male gametophyte and which cell supplies resources for pollen tube growth."},{"id":"c3a50a22-8c1e-489b-a9e9-512e3f77a09c","tool":"quizgecko","tool_name":"Quizgecko","verdict":"mixed","score":null,"score_total":null,"note":"Some questions use specific botanical framing and check understanding, but the set still caps out at recall-level and lacks inferential or application-based questions."},{"id":"74670d9d-491c-4219-98d9-1053b07f50f3","tool":"quizizz","tool_name":"Quizizz","verdict":"mixed","score":null,"score_total":null,"note":"Even with DOK Level 3 selected, the PDF run still included surface-level items; the report says roughly 5–6 of 20 questions were shallow, such as flower symbolism and the three parts of a pistil."},{"id":"579c71e1-6e46-466e-86f6-7b2a59085f64","tool":"quizrise","tool_name":"QuizRise","verdict":"struggled","score":null,"score_total":null,"note":"On the NCERT PDF run, the requested Hard level still came out as easy-to-medium recall-style items, and the explanations shown in the quiz view did not include source citations back to the material. That makes the set usable, but not convincingly hard or especially well-grounded."},{"id":"b592f4a3-f553-497d-b831-c68d17da508a","tool":"studyglen","tool_name":"Studyglen","verdict":"worked","score":null,"score_total":null,"note":"The generator can produce understanding-oriented prompts rather than only recall: one question asks learners to interpret a diagrammed Vallisneria pollination sequence, and another asks for a short explanation of why a megaspore mother cell yields four megaspores but only one persists functionally."},{"id":"f98d3c36-275e-4e98-91a9-56f5a685bcfc","tool":"studyx-ai","tool_name":"StudyX AI","verdict":"mixed","score":null,"score_total":null,"note":"Can generate 15 non-MCQ practice items only through a manual chat prompt: 5 true/false, 5 fill-in-the-blanks, and 5 short-answer questions, with an answer key requested in the same prompt. That expands format variety, but it is not native quiz generation and costs extra credits."}]}