{"observation":{"id":"f98d3c36-275e-4e98-91a9-56f5a685bcfc","tool":"studyx-ai","tool_name":"StudyX AI","criterion":"are-the-questions-actually-good","criterion_name":"Are the Questions Actually Good?","criterion_definition":"Whether the questions test understanding at the requested difficulty level rather than only surface-level recall or low-quality near-duplicates.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Question quality is the heart of the job here; weak or shallow questions mean the tool is failing at generating useful quizzes. (3 of 3 judges)","scenario":"ncert-class-12-biology-chapter-pdf-on-sexual-reproduction-in-flowering-plants","scenario_name":"NCERT Class 12 Biology chapter PDF on sexual reproduction in flowering plants","group_tag":"sexual-reproduction-in-flowering-plants","scenario_description":"A dense typed NCERT Class 12 Biology chapter on sexual reproduction in flowering plants, including technical terminology and diagrams. It was used to test how well tools extract content and generate high-quality questions from a long, structured study PDF.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://d3epheqghktydj.cloudfront.net/generate-quizzes-and-practice-tests-from-input1-primary-366aec5861ae.pdf","role":"input","filename":"input1_primary.pdf"}],"stresses":["long-form document extraction","technical biology terminology","diagram-heavy content","question generation quality from clean typed text"],"verdict":"mixed","score":null,"score_total":null,"note":"Can generate 15 non-MCQ practice items only through a manual chat prompt: 5 true/false, 5 fill-in-the-blanks, and 5 short-answer questions, with an answer key requested in the same prompt. That expands format variety, but it is not native quiz generation and costs extra credits.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/b94752601d1140cfb7036bc483beceef.mp4?v=1","role":"input","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/c722a1cbed1d46d4ac13b960353ab10f.mp4?v=1","role":"output","alt":null}],"run_id":"de7b348e-4326-4fbf-87c1-b104750bc459","study_title":"Generate Quizzes and Practice Tests from Study Material Using AI","study_kind":"generation","research_task":"86ba51dce","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://d3epheqghktydj.cloudfront.net/generate-quizzes-and-practice-tests-from-input1-primary-366aec5861ae.pdf","filename":"input1_primary.pdf","alt":"NCERT Class 12 Biology chapter PDF on sexual reproduction in flowering plants","role":"input"}],"modality":"pdf","stresses":["long-form document extraction","technical biology terminology","diagram-heavy content","question generation quality from clean typed text"]},"tool_page_slug":"studyx-ai","tool_url":"https://aidemos.com/tools/studyx-ai","permalink":"https://aidemos.com/evidence/f98d3c36-275e-4e98-91a9-56f5a685bcfc","api_url":"https://ai.aidemos.com/v1/observations/f98d3c36-275e-4e98-91a9-56f5a685bcfc"},"peers":[{"id":"4b226047-ef59-4f5c-adef-8be459e380d1","tool":"doctoquiz","tool_name":"DoctoQuiz","verdict":"mixed","score":null,"score_total":null,"note":"Produces cleanly formatted multiple-choice questions, but the set stays mostly at recall level and includes at least one ambiguous item where two options are defensibly true.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/77c844c5170a48c2bb161af89d8c6faa.png?v=1","evidence_url":"https://aidemos.com/evidence/4b226047-ef59-4f5c-adef-8be459e380d1"},{"id":"b98f0ab1-d272-45af-b348-3bdae0087ea5","tool":"edcafe-ai","tool_name":"Edcafe AI","verdict":"struggled","score":null,"score_total":null,"note":"When it falls back to a typed topic label, the quiz generator stays at surface-level recall, producing MCQ-only items rather than deeper comprehension questions.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/609815a8cbe74f14a3227160ac87398d.png?v=1","evidence_url":"https://aidemos.com/evidence/b98f0ab1-d272-45af-b348-3bdae0087ea5"},{"id":"eade66c7-0510-40e6-aa86-b554aee80d21","tool":"knowt","tool_name":"Knowt","verdict":"mixed","score":null,"score_total":null,"note":"On the typed PDF, the questions are clearly on-topic and exam-style, but they are mostly direct recall rather than deeply reframed or high-twist questions.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/f80c400d05e34fe8a232cbdfaa1bb2bf.png?v=1","evidence_url":"https://aidemos.com/evidence/eade66c7-0510-40e6-aa86-b554aee80d21"},{"id":"ee64a3f8-5dc8-4b2a-b9ed-ba292fe9442a","tool":"pdf-to-quiz","tool_name":"PDF to Quiz","verdict":"worked","score":null,"score_total":null,"note":"On the typed PDF it generated inferential MCQs that went beyond recall, including questions about why a pollen grain is the male gametophyte and which cell supplies resources for pollen tube growth.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/8607e53301e148cda9b89d0503eeff6f.pdf?v=1","evidence_url":"https://aidemos.com/evidence/ee64a3f8-5dc8-4b2a-b9ed-ba292fe9442a"},{"id":"100e011a-110e-4f7a-b15a-496026b99533","tool":"questionwell","tool_name":"QuestionWell","verdict":"mixed","score":null,"score_total":null,"note":"Produces genuinely higher-order Discussion questions that ask learners to compare mechanisms and reason about real-world effects, but the same run also includes basic recall-style items such as fill-in-the-blank and true/false questions.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/109f29a0710b40f0b7447b2e331ba8ec.png?v=1","evidence_url":"https://aidemos.com/evidence/100e011a-110e-4f7a-b15a-496026b99533"},{"id":"c3a50a22-8c1e-489b-a9e9-512e3f77a09c","tool":"quizgecko","tool_name":"Quizgecko","verdict":"mixed","score":null,"score_total":null,"note":"Some questions use specific botanical framing and check understanding, but the set still caps out at recall-level and lacks inferential or application-based questions.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/83b9dfde13d546abadb4063d7103a072.png?v=1","evidence_url":"https://aidemos.com/evidence/c3a50a22-8c1e-489b-a9e9-512e3f77a09c"},{"id":"74670d9d-491c-4219-98d9-1053b07f50f3","tool":"quizizz","tool_name":"Quizizz","verdict":"mixed","score":null,"score_total":null,"note":"Even with DOK Level 3 selected, the PDF run still included surface-level items; the report says roughly 5–6 of 20 questions were shallow, such as flower symbolism and the three parts of a pistil.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/8607e53301e148cda9b89d0503eeff6f.pdf?v=1","evidence_url":"https://aidemos.com/evidence/74670d9d-491c-4219-98d9-1053b07f50f3"},{"id":"579c71e1-6e46-466e-86f6-7b2a59085f64","tool":"quizrise","tool_name":"QuizRise","verdict":"struggled","score":null,"score_total":null,"note":"On the NCERT PDF run, the requested Hard level still came out as easy-to-medium recall-style items, and the explanations shown in the quiz view did not include source citations back to the material. That makes the set usable, but not convincingly hard or especially well-grounded.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/b5d00401d520425783583aa75eb13d35.png?v=1","evidence_url":"https://aidemos.com/evidence/579c71e1-6e46-466e-86f6-7b2a59085f64"},{"id":"b592f4a3-f553-497d-b831-c68d17da508a","tool":"studyglen","tool_name":"Studyglen","verdict":"worked","score":null,"score_total":null,"note":"The generator can produce understanding-oriented prompts rather than only recall: one question asks learners to interpret a diagrammed Vallisneria pollination sequence, and another asks for a short explanation of why a megaspore mother cell yields four megaspores but only one persists functionally.","artifact_count":7,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/755a379869a24e28a2deb0e16c37b57d.png?v=1","evidence_url":"https://aidemos.com/evidence/b592f4a3-f553-497d-b831-c68d17da508a"}],"other_criteria":[{"id":"314342e1-9466-4e08-a1bd-18f21c16a336","criterion":"can-it-read-your-material","criterion_name":"Can It Read Your Material?","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Successfully ingests a dense typed chapter PDF and turns it into a full 20-question botany quiz; the output shows later items from Question 2 through Question 19 with chapter-specific content such as geitonogamy, androecium, embryo sac structures, and scutellum.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/314342e1-9466-4e08-a1bd-18f21c16a336"},{"id":"a934b0f5-a5d5-4010-ba03-f6729c0c3df0","criterion":"does-it-feel-like-a-real-quiz","criterion_name":"Does It Feel Like a Real Quiz?","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"Provides 2 key quiz cues — answers are hidden by default and a score tracker updates — but 0 enforced gates stop the user from revealing answers before attempting them, so the quiz feels interactive without being locked down.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/a934b0f5-a5d5-4010-ba03-f6729c0c3df0"}],"appears_in":[{"page_type":"ranking","slug":"quiz-generators","title":"Best AI Tools for Generating Quizzes and Practice Tests from Study Material","url":"https://aidemos.com/best/quiz-generators","binding":"run"},{"page_type":"tool","slug":"doctoquiz","title":null,"url":"https://aidemos.com/tools/doctoquiz","binding":"run"}],"same_scenario":[{"id":"4b226047-ef59-4f5c-adef-8be459e380d1","tool":"doctoquiz","tool_name":"DoctoQuiz","verdict":"mixed","score":null,"score_total":null,"note":"Produces cleanly formatted multiple-choice questions, but the set stays mostly at recall level and includes at least one ambiguous item where two options are defensibly true."},{"id":"b98f0ab1-d272-45af-b348-3bdae0087ea5","tool":"edcafe-ai","tool_name":"Edcafe AI","verdict":"struggled","score":null,"score_total":null,"note":"When it falls back to a typed topic label, the quiz generator stays at surface-level recall, producing MCQ-only items rather than deeper comprehension questions."},{"id":"eade66c7-0510-40e6-aa86-b554aee80d21","tool":"knowt","tool_name":"Knowt","verdict":"mixed","score":null,"score_total":null,"note":"On the typed PDF, the questions are clearly on-topic and exam-style, but they are mostly direct recall rather than deeply reframed or high-twist questions."},{"id":"ee64a3f8-5dc8-4b2a-b9ed-ba292fe9442a","tool":"pdf-to-quiz","tool_name":"PDF to Quiz","verdict":"worked","score":null,"score_total":null,"note":"On the typed PDF it generated inferential MCQs that went beyond recall, including questions about why a pollen grain is the male gametophyte and which cell supplies resources for pollen tube growth."},{"id":"100e011a-110e-4f7a-b15a-496026b99533","tool":"questionwell","tool_name":"QuestionWell","verdict":"mixed","score":null,"score_total":null,"note":"Produces genuinely higher-order Discussion questions that ask learners to compare mechanisms and reason about real-world effects, but the same run also includes basic recall-style items such as fill-in-the-blank and true/false questions."},{"id":"c3a50a22-8c1e-489b-a9e9-512e3f77a09c","tool":"quizgecko","tool_name":"Quizgecko","verdict":"mixed","score":null,"score_total":null,"note":"Some questions use specific botanical framing and check understanding, but the set still caps out at recall-level and lacks inferential or application-based questions."},{"id":"74670d9d-491c-4219-98d9-1053b07f50f3","tool":"quizizz","tool_name":"Quizizz","verdict":"mixed","score":null,"score_total":null,"note":"Even with DOK Level 3 selected, the PDF run still included surface-level items; the report says roughly 5–6 of 20 questions were shallow, such as flower symbolism and the three parts of a pistil."},{"id":"579c71e1-6e46-466e-86f6-7b2a59085f64","tool":"quizrise","tool_name":"QuizRise","verdict":"struggled","score":null,"score_total":null,"note":"On the NCERT PDF run, the requested Hard level still came out as easy-to-medium recall-style items, and the explanations shown in the quiz view did not include source citations back to the material. That makes the set usable, but not convincingly hard or especially well-grounded."},{"id":"b592f4a3-f553-497d-b831-c68d17da508a","tool":"studyglen","tool_name":"Studyglen","verdict":"worked","score":null,"score_total":null,"note":"The generator can produce understanding-oriented prompts rather than only recall: one question asks learners to interpret a diagrammed Vallisneria pollination sequence, and another asks for a short explanation of why a megaspore mother cell yields four megaspores but only one persists functionally."}]}