{"observation":{"id":"649f0906-a783-463d-9404-08ad20a01dbf","tool":"pdf-to-quiz","tool_name":"PDF to Quiz","criterion":"are-the-questions-actually-good","criterion_name":"Are the Questions Actually Good?","criterion_definition":"Whether the questions test understanding at the requested difficulty level rather than only surface-level recall or low-quality near-duplicates.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Question quality is the heart of the job here; weak or shallow questions mean the tool is failing at generating useful quizzes. (3 of 3 judges)","scenario":"handwritten-student-notes-on-sexual-reproduction-in-flowering-plants","scenario_name":"Handwritten student notes on sexual reproduction in flowering plants","group_tag":"sexual-reproduction-in-flowering-plants","scenario_description":"Scanned handwritten student notes covering the same biology chapter. It was used to test OCR accuracy and the usefulness of quiz generation when the source is informal handwriting rather than typed text.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://d3epheqghktydj.cloudfront.net/generate-quizzes-and-practice-tests-from-input2-secondary-98543a951470.pdf","role":"input","filename":"input2_secondary.pdf"}],"stresses":["handwriting OCR","noisy scan interpretation","informal note structure","question generation quality from OCR-dependent input"],"verdict":"worked","score":null,"score_total":null,"note":"On the handwritten notes it also produced higher-order questions, including a sporopollenin item about exine sculpture preservation and a full reviewed set of biology MCQs.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/3fad40db698c4cc18f6181c5612b67dc.png?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/5e389fb1081241d78b70f9335e10be5b.mp4?v=1","role":"context","alt":null}],"run_id":"de7b348e-4326-4fbf-87c1-b104750bc459","study_title":"Generate Quizzes and Practice Tests from Study Material Using AI","study_kind":"generation","research_task":"86ba51dce","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://d3epheqghktydj.cloudfront.net/generate-quizzes-and-practice-tests-from-input2-secondary-98543a951470.pdf","filename":"input2_secondary.pdf","alt":"Handwritten student notes on sexual reproduction in flowering plants","role":"input"}],"modality":"pdf","stresses":["handwriting OCR","noisy scan interpretation","informal note structure","question generation quality from OCR-dependent input"]},"tool_page_slug":"pdf-to-quiz","tool_url":"https://aidemos.com/tools/pdf-to-quiz","permalink":"https://aidemos.com/evidence/649f0906-a783-463d-9404-08ad20a01dbf","api_url":"https://ai.aidemos.com/v1/observations/649f0906-a783-463d-9404-08ad20a01dbf"},"peers":[{"id":"daa5fdfa-36bf-4213-bc13-41df436b3823","tool":"doctoquiz","tool_name":"DoctoQuiz","verdict":"mixed","score":null,"score_total":null,"note":"The handwritten-notes run is slightly richer and uses fuller distractors, but it is still only marginally harder than the typed-PDF run and remains recall-heavy.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/67a8bf5adb784d2b873f4deb23c78e0a.png?v=1","evidence_url":"https://aidemos.com/evidence/daa5fdfa-36bf-4213-bc13-41df436b3823"},{"id":"76f084e6-3f1c-4741-b8eb-2748494dbce5","tool":"knowt","tool_name":"Knowt","verdict":"worked","score":null,"score_total":null,"note":"From handwritten notes, the tool can generate stronger, more exam-relevant questions; the report describes this output as critical-level, and the visible item requires defining both syngamy and triple fusion plus their products.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/f43f812eb1a44467bbbebf666d4b33db.png?v=1","evidence_url":"https://aidemos.com/evidence/76f084e6-3f1c-4741-b8eb-2748494dbce5"},{"id":"d3e4c2f2-44c9-4e90-abca-8158bbdf7340","tool":"questionwell","tool_name":"QuestionWell","verdict":"worked","score":null,"score_total":null,"note":"Carries the same higher-order question depth into handwritten-note runs, with Bloom's/DOK-tagged MCQs and application-level Discussion prompts rather than only surface recall.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/327c5bc97e2b4cd9833bec34dd564eb6.jpeg?v=1","evidence_url":"https://aidemos.com/evidence/d3e4c2f2-44c9-4e90-abca-8158bbdf7340"},{"id":"abc30775-baec-4f20-a712-7863273c4f27","tool":"quizgecko","tool_name":"Quizgecko","verdict":"worked","score":null,"score_total":null,"note":"From handwritten notes, the tool can produce a good-to-high quality question set with a genuine mix of difficulty levels instead of only low-value recall prompts.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/abc30775-baec-4f20-a712-7863273c4f27"},{"id":"040cce73-1d64-4f50-9167-3adb51131079","tool":"quizrise","tool_name":"QuizRise","verdict":"failed","score":null,"score_total":null,"note":"The handwritten-notes run produced confusable sibling distractors such as microsporogenesis/megasporogenesis/microgametogenesis and geitonogamy/autogamy, which weakens discrimination and does not feel like a genuinely strong Hard-level quiz.","artifact_count":7,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/9a133305a3ea4098a77df6a2c5e03738.png?v=1","evidence_url":"https://aidemos.com/evidence/040cce73-1d64-4f50-9167-3adb51131079"}],"other_criteria":[{"id":"a708aa45-a485-40c3-b002-014741d9e9f9","criterion":"can-it-read-your-material","criterion_name":"Can It Read Your Material?","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"It OCR-read the scanned handwritten notes well enough to produce a full quiz from them, and the review footage shows a 10-question set generated from that input.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/a708aa45-a485-40c3-b002-014741d9e9f9"}],"appears_in":[{"page_type":"ranking","slug":"quiz-generators","title":"Best AI Tools for Generating Quizzes and Practice Tests from Study Material","url":"https://aidemos.com/best/quiz-generators","binding":"run"},{"page_type":"tool","slug":"doctoquiz","title":null,"url":"https://aidemos.com/tools/doctoquiz","binding":"run"}],"same_scenario":[{"id":"daa5fdfa-36bf-4213-bc13-41df436b3823","tool":"doctoquiz","tool_name":"DoctoQuiz","verdict":"mixed","score":null,"score_total":null,"note":"The handwritten-notes run is slightly richer and uses fuller distractors, but it is still only marginally harder than the typed-PDF run and remains recall-heavy."},{"id":"76f084e6-3f1c-4741-b8eb-2748494dbce5","tool":"knowt","tool_name":"Knowt","verdict":"worked","score":null,"score_total":null,"note":"From handwritten notes, the tool can generate stronger, more exam-relevant questions; the report describes this output as critical-level, and the visible item requires defining both syngamy and triple fusion plus their products."},{"id":"d3e4c2f2-44c9-4e90-abca-8158bbdf7340","tool":"questionwell","tool_name":"QuestionWell","verdict":"worked","score":null,"score_total":null,"note":"Carries the same higher-order question depth into handwritten-note runs, with Bloom's/DOK-tagged MCQs and application-level Discussion prompts rather than only surface recall."},{"id":"abc30775-baec-4f20-a712-7863273c4f27","tool":"quizgecko","tool_name":"Quizgecko","verdict":"worked","score":null,"score_total":null,"note":"From handwritten notes, the tool can produce a good-to-high quality question set with a genuine mix of difficulty levels instead of only low-value recall prompts."},{"id":"040cce73-1d64-4f50-9167-3adb51131079","tool":"quizrise","tool_name":"QuizRise","verdict":"failed","score":null,"score_total":null,"note":"The handwritten-notes run produced confusable sibling distractors such as microsporogenesis/megasporogenesis/microgametogenesis and geitonogamy/autogamy, which weakens discrimination and does not feel like a genuinely strong Hard-level quiz."}]}