{"observation":{"id":"485c214e-dd91-49d7-903c-529e96c365a1","tool":"notebooklm","tool_name":"NotebookLM","criterion":"summary-output-quality","criterion_name":"Summary Output Quality","criterion_definition":"How well the tool turns source material into structured, exam-ready summaries instead of plain paragraph compression.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"The main job here is producing structured, exam-ready notes rather than generic compression, so summary quality directly determines whether the tool succeeds. (3 of 3 judges)","scenario":"long-academic-chapter-pdf-for-exam-ready-notes","scenario_name":"Long academic chapter PDF for exam-ready notes","group_tag":null,"scenario_description":"A long academic chapter PDF used to test whether tools can turn dense study material into structured, exam-ready revision notes with source traceability and active recall support.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://d3epheqghktydj.cloudfront.net/turn-academic-pdfs-into-exam-ready-revis-biology-input-chapter-bcce2d971009.pdf","role":"input","filename":"biology-input-chapter.pdf"}],"stresses":["Long-document PDF processing","Structured summarization into bullets/tables/sections","Preserving academic clarity and key concepts","Generating flashcards and quiz questions","Source verification and jump-to-source traceability","Revision-friendly formatting and workflow support"],"verdict":"worked","score":null,"score_total":null,"note":"Transforms source material into a structured revision table with explicit fields for gland, secretion, site of action, enzymes, food acted upon, end product, and source, rather than only compressing into paragraphs.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/notegpt-notegpt-biology-chapter-input-4a0d2c7b839f.pdf","role":"input","alt":"Notegpt biology chapter input.pdf"},{"url":"https://d3epheqghktydj.cloudfront.net/notebooklm-image-048ccca5ed8d.png","role":"output","alt":null},{"url":"https://d3epheqghktydj.cloudfront.net/notebooklm-notebooklm-written-summary-output-fbde374f3bce.png","role":"output","alt":null}],"run_id":"17ca9a9c-03d4-402e-b34d-2ea8078316be","study_title":"Turn Academic PDFs Into Exam-Ready Revision Notes Using AI","study_kind":"generation","research_task":"86b8anp1v","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://d3epheqghktydj.cloudfront.net/turn-academic-pdfs-into-exam-ready-revis-biology-input-chapter-bcce2d971009.pdf","filename":"biology-input-chapter.pdf","alt":"Long academic chapter PDF for exam-ready notes","role":"input"}],"modality":"pdf","stresses":["Long-document PDF processing","Structured summarization into bullets/tables/sections","Preserving academic clarity and key concepts","Generating flashcards and quiz questions","Source verification and jump-to-source traceability","Revision-friendly formatting and workflow support"]},"tool_page_slug":"notebooklm","tool_url":"https://aidemos.com/tools/notebooklm","permalink":"https://aidemos.com/evidence/485c214e-dd91-49d7-903c-529e96c365a1","api_url":"https://ai.aidemos.com/v1/observations/485c214e-dd91-49d7-903c-529e96c365a1"},"peers":[{"id":"293ad598-b680-42fb-b8f9-3c17df93de22","tool":"gpai","tool_name":"GPAI","verdict":"failed","score":null,"score_total":null,"note":"Does not recreate diagrams or tables in the summary output, so the result stays text-only even when the source material has visual structure.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/293ad598-b680-42fb-b8f9-3c17df93de22"},{"id":"64c4053b-8e3a-455b-996b-067ea0ac0540","tool":"notegpt","tool_name":"NoteGPT","verdict":"worked","score":null,"score_total":null,"note":"Converts long source text into exam-ready structured notes instead of plain paragraph compression, using concise bullet points, bold headings, and table-like organization.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/notegpt-notegpt-smart-summary-output-34d645486197.mp4","evidence_url":"https://aidemos.com/evidence/64c4053b-8e3a-455b-996b-067ea0ac0540"},{"id":"6929cce5-1941-462a-954b-ab85bd040a4d","tool":"studyfetch","tool_name":"StudyFetch","verdict":"mixed","score":null,"score_total":null,"note":"The report says Detailed mode has some redundancy, so the intermediate summary depth is not fully clean or minimal.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/6929cce5-1941-462a-954b-ab85bd040a4d"},{"id":"39d621a8-e831-45dc-9b68-09b8cc37494e","tool":"youlearn-ai","tool_name":"YouLearn AI","verdict":"mixed","score":null,"score_total":null,"note":"The summaries do not reproduce the source PDF’s embedded diagrams, so diagram-heavy material loses visual context even when the text summary is well structured.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/39d621a8-e831-45dc-9b68-09b8cc37494e"}],"other_criteria":[{"id":"03ca0aec-7627-481b-aeca-f011cccac369","criterion":"active-recall-support","criterion_name":"Active Recall Support","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"The flashcard set can repeat items occasionally, which the report flags as a quality limitation.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/03ca0aec-7627-481b-aeca-f011cccac369"},{"id":"e3545172-7bf0-4e26-a81f-f88750409be7","criterion":"active-recall-support","criterion_name":"Active Recall Support","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"Quiz volume is low for a full chapter: the report says around 10–12 MCQs were generated and that this is insufficient for exam prep at chapter scale.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/e3545172-7bf0-4e26-a81f-f88750409be7"},{"id":"b14804a0-9380-4ad6-82e2-a373baca578a","criterion":"active-recall-support","criterion_name":"Active Recall Support","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Produces a 12-question multiple-choice quiz with immediate correctness feedback and explanations; the UI shows 1/12 and the correct option is marked in green.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/b14804a0-9380-4ad6-82e2-a373baca578a"},{"id":"0535ee6e-6a7d-44a6-bde4-e11d7a1a1bf2","criterion":"active-recall-support","criterion_name":"Active Recall Support","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Generates a large flashcard deck for active recall; the UI shows 1/70 and the report says the chapter produced 65+ flashcards, with flip navigation, answer reveal, and Good/Bad content feedback.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/0535ee6e-6a7d-44a6-bde4-e11d7a1a1bf2"},{"id":"19c621f4-f999-489c-93a4-96f9a32c5193","criterion":"study-material-handling","criterion_name":"Study Material Handling","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Processed a 36-page academic PDF end-to-end and began generating multiple output formats simultaneously, showing it can ingest a long study chapter without manual splitting.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/19c621f4-f999-489c-93a4-96f9a32c5193"},{"id":"9e801f4f-5fa3-410c-a047-148484fbd826","criterion":"traceability-transparency","criterion_name":"Traceability & Transparency","rank_role":"context","verdict":"worked","score":null,"score_total":null,"note":"Links generated notes back to the source with numbered citations like [1], and the report says those references jump to the exact page and paragraph in the original PDF.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/9e801f4f-5fa3-410c-a047-148484fbd826"}],"appears_in":[{"page_type":"ranking","slug":"pdf-to-study-notes","title":"Best AI Tools to Turn Academic PDFs Into Exam-Ready Revision Notes","url":"https://aidemos.com/best/pdf-to-study-notes","binding":"run"},{"page_type":"tool","slug":"notebooklm","title":null,"url":"https://aidemos.com/tools/notebooklm","binding":"run"},{"page_type":"use-case","slug":"turn-academic-pdfs-into-revision-notes","title":"Turn Academic PDFs Into Exam-Ready Revision Notes Using AI","url":"https://aidemos.com/use-cases/turn-academic-pdfs-into-revision-notes","binding":"study"}],"same_scenario":[{"id":"293ad598-b680-42fb-b8f9-3c17df93de22","tool":"gpai","tool_name":"GPAI","verdict":"failed","score":null,"score_total":null,"note":"Does not recreate diagrams or tables in the summary output, so the result stays text-only even when the source material has visual structure."},{"id":"64c4053b-8e3a-455b-996b-067ea0ac0540","tool":"notegpt","tool_name":"NoteGPT","verdict":"worked","score":null,"score_total":null,"note":"Converts long source text into exam-ready structured notes instead of plain paragraph compression, using concise bullet points, bold headings, and table-like organization."},{"id":"6929cce5-1941-462a-954b-ab85bd040a4d","tool":"studyfetch","tool_name":"StudyFetch","verdict":"mixed","score":null,"score_total":null,"note":"The report says Detailed mode has some redundancy, so the intermediate summary depth is not fully clean or minimal."},{"id":"39d621a8-e831-45dc-9b68-09b8cc37494e","tool":"youlearn-ai","tool_name":"YouLearn AI","verdict":"mixed","score":null,"score_total":null,"note":"The summaries do not reproduce the source PDF’s embedded diagrams, so diagram-heavy material loses visual context even when the text summary is well structured."}]}