{"observation":{"id":"0a57537c-4ad4-4b2f-8115-4281f671bd30","tool":"meetgeek","tool_name":"MeetGeek","criterion":"editability","criterion_name":"Editability","criterion_definition":"Can you fix a wrong summary or action item before sharing?","criterion_evidence_type":"capability","criterion_rank_role":"context","criterion_rank_role_reason":"Being able to fix mistakes before sharing is valuable, but it does not by itself measure how well the tool captures the meeting in the first place. (3 of 3 judges)","scenario":"ai-demos-daily-standup-31-july-2026","scenario_name":"AI Demos Daily Standup — 31 July 2026","group_tag":"ai-meeting-notetaker","scenario_description":"A real 25-minute technical engineering daily standup with 14 attendees and about 10 active speakers, used as the single parallel-capture meeting for evaluating AI meeting notetakers on transcription, diarization, summaries, action items, search/chat, and collaboration features.","modality":"image","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://cdn.futuresmart.ai/public/aidemos/547dd6f13e4a420fa8ad7bf2c88c7612.png?v=1","role":"input","filename":"31-july-meeting-screenshot.png"}],"stresses":["Transcription accuracy for real names, tool names, numbers, and technical jargon","Speaker diarization across multiple active speakers","Robustness to overlapping speech, crosstalk, and rapid turn-taking","Join reliability for bot-based and botless capture","Summary quality on identical source material","Action-item extraction with correct owners and commitments","Topic segmentation of standup updates","Search and chat grounded in the meeting content","Sharing, API, MCP, integrations, plan limits, languages, and privacy feature coverage"],"verdict":"worked","score":null,"score_total":null,"note":"Users can edit generated outputs inline before sharing; the report says summary, action items, and the full transcript are all editable, and the UI shows editable summary text.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/c7ceaf56115b45bb86e3c89f78424ea4.png?v=1","role":"output","alt":null}],"run_id":"ace58582-3d1e-48ee-996c-9b3cd03f27a2","study_title":"AI Meeting Notetakers — Capture Accurate Transcripts, Summaries & Action Items From Live Calls","study_kind":"generation","research_task":"86baxegnv","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://cdn.futuresmart.ai/public/aidemos/547dd6f13e4a420fa8ad7bf2c88c7612.png?v=1","filename":"31-july-meeting-screenshot.png","alt":"AI Demos Daily Standup — 31 July 2026","role":"input"}],"modality":"image","stresses":["Transcription accuracy for real names, tool names, numbers, and technical jargon","Speaker diarization across multiple active speakers","Robustness to overlapping speech, crosstalk, and rapid turn-taking","Join reliability for bot-based and botless capture","Summary quality on identical source material","Action-item extraction with correct owners and commitments","Topic segmentation of standup updates","Search and chat grounded in the meeting content","Sharing, API, MCP, integrations, plan limits, languages, and privacy feature coverage"]},"tool_page_slug":"meetgeek","tool_url":"https://aidemos.com/tools/meetgeek","permalink":"https://aidemos.com/evidence/0a57537c-4ad4-4b2f-8115-4281f671bd30","api_url":"https://ai.aidemos.com/v1/observations/0a57537c-4ad4-4b2f-8115-4281f671bd30"},"peers":[{"id":"a47ae17f-dfbd-439c-9ba9-e3153156cca8","tool":"fellow","tool_name":"Fellow","verdict":"mixed","score":null,"score_total":null,"note":"Summary and action items are editable inline before sharing, but the transcript itself is locked for audit-trail purposes, so editing is only partial.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/14b659d136f643cca123a6031ecfdee2.png?v=1","evidence_url":"https://aidemos.com/evidence/a47ae17f-dfbd-439c-9ba9-e3153156cca8"},{"id":"e181aa66-75ab-4b14-a804-0bb3f6481466","tool":"granola","tool_name":"Granola","verdict":"mixed","score":null,"score_total":null,"note":"Granola supports editing the summary layer, but transcript text is not editable in-app; the report describes transcript correction as locked by design, so users can fix notes and action items but not the underlying transcript.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/d2e0703f86a040cea768260e1ddae9f4.png?v=1","evidence_url":"https://aidemos.com/evidence/e181aa66-75ab-4b14-a804-0bb3f6481466"},{"id":"0364fbbc-9644-47cc-97a6-60174baab699","tool":"otter-ai","tool_name":"Otter.ai","verdict":"worked","score":null,"score_total":null,"note":"Otter exposes inline editing controls for transcript and summary outputs before sharing, so wrong content can be corrected in-product rather than only exported as-is.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/a0307003e47741e68372b1f2d17c91ec.png?v=1","evidence_url":"https://aidemos.com/evidence/0364fbbc-9644-47cc-97a6-60174baab699"}],"other_criteria":[{"id":"dfeacb85-baa0-4c8b-a44e-e4a7b005d54f","criterion":"action-item-extraction","criterion_name":"Action-Item Extraction","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"It extracts real commitments as action items rather than noise; the report says all extracted items had correct ownership and timing, and the visible note includes an owned action item with timestamp 19:51.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/dfeacb85-baa0-4c8b-a44e-e4a7b005d54f"},{"id":"75aaf0b0-c9d5-4376-8b81-5b6f2574c0fb","criterion":"chat-with-notes-ask-questions","criterion_name":"Chat with Notes / Ask Questions","rank_role":"context","verdict":"mixed","score":null,"score_total":null,"note":"It answers direct grounded questions correctly, but the report records an incorrect answer on a speaker-dependent scheduling question, so chat is reliable for simple queries but weaker when attribution/context matters.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/75aaf0b0-c9d5-4376-8b81-5b6f2574c0fb"},{"id":"9ebe6fbe-6f09-440e-afde-66593a710483","criterion":"join-method-reliability","criterion_name":"Join Method & Reliability","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The bot successfully joined a Google Meet call and the report says it captured the full ~30-minute meeting with zero disconnections or data loss.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/9ebe6fbe-6f09-440e-afde-66593a710483"},{"id":"e8929c90-b25d-47cf-ab27-f340ed881ef4","criterion":"search-across-notes","criterion_name":"Search Across Notes","rank_role":"context","verdict":"worked","score":null,"score_total":null,"note":"It supports transcript search with precise retrieval: searching for \"api\" surfaces the matching text in context and the report says timestamps are returned to within a few seconds.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/e8929c90-b25d-47cf-ab27-f340ed881ef4"},{"id":"4f7e5b21-ff4f-4846-9a07-3219c0659681","criterion":"speaker-diarization","criterion_name":"Speaker Diarization","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"It identifies most speakers in a multi-speaker standup, but leaves at least one utterance as \"Unknown speaker\" and misattributes some lines to the wrong speaker, so attribution is not fully reliable.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/4f7e5b21-ff4f-4846-9a07-3219c0659681"},{"id":"b482399a-b91b-4aaa-b6f1-a56363792548","criterion":"summary-quality","criterion_name":"Summary Quality","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"It produces a clear, skimmable meeting summary with topic organization and a Next Steps section, and the report says it preserved the major decisions and discussion points.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/b482399a-b91b-4aaa-b6f1-a56363792548"},{"id":"b45a857b-9306-4a29-a568-dcdf9f79fdf2","criterion":"topic-segmentation","criterion_name":"Topic Segmentation","rank_role":"context","verdict":"worked","score":null,"score_total":null,"note":"It breaks the meeting into useful numbered topic sections instead of one blob, with a visible hierarchy under \"Topics & Highlights\" and the report also noting an Insights tab alongside the segmentation.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/b45a857b-9306-4a29-a568-dcdf9f79fdf2"},{"id":"5a191ed7-02a9-4979-931d-9219e0b75fce","criterion":"transcription-accuracy","criterion_name":"Transcription Accuracy","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"It transcribes a normal ~25-minute, ~10-active-speaker engineering standup mostly accurately, with only minor proper-noun/term drift noted in the report; one example given is \"Madin\" being misheard for \"Mahreen\".","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/5a191ed7-02a9-4979-931d-9219e0b75fce"}],"appears_in":[{"page_type":"ranking","slug":"ai-meeting-notetakers","title":"Best AI Meeting Notetakers for Accurate Transcripts, Summaries, and Action Items","url":"https://aidemos.com/best/ai-meeting-notetakers","binding":"run"}],"same_scenario":[{"id":"a47ae17f-dfbd-439c-9ba9-e3153156cca8","tool":"fellow","tool_name":"Fellow","verdict":"mixed","score":null,"score_total":null,"note":"Summary and action items are editable inline before sharing, but the transcript itself is locked for audit-trail purposes, so editing is only partial."},{"id":"e181aa66-75ab-4b14-a804-0bb3f6481466","tool":"granola","tool_name":"Granola","verdict":"mixed","score":null,"score_total":null,"note":"Granola supports editing the summary layer, but transcript text is not editable in-app; the report describes transcript correction as locked by design, so users can fix notes and action items but not the underlying transcript."},{"id":"0364fbbc-9644-47cc-97a6-60174baab699","tool":"otter-ai","tool_name":"Otter.ai","verdict":"worked","score":null,"score_total":null,"note":"Otter exposes inline editing controls for transcript and summary outputs before sharing, so wrong content can be corrected in-product rather than only exported as-is."}]}