{"observation":{"id":"64cce11b-7973-439a-bcc1-68219903d5eb","tool":"fathom","tool_name":"Fathom","criterion":"join-method-reliability","criterion_name":"Join Method & Reliability","criterion_definition":"Bot-in-call vs. botless/local capture, and reliability of joining Zoom, Meet, or Teams.","criterion_evidence_type":"capability","criterion_rank_role":"decisive","criterion_rank_role_reason":"A meeting notetaker must reliably get into the call or capture it locally; if it cannot join or record consistently, it cannot do the job at all. (2 of 3 judges)","scenario":"ai-demos-daily-standup-31-july-2026","scenario_name":"AI Demos Daily Standup — 31 July 2026","group_tag":"ai-meeting-notetaker","scenario_description":"A real 25-minute technical engineering daily standup with 14 attendees and about 10 active speakers, used as the single parallel-capture meeting for evaluating AI meeting notetakers on transcription, diarization, summaries, action items, search/chat, and collaboration features.","modality":"image","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://cdn.futuresmart.ai/public/aidemos/547dd6f13e4a420fa8ad7bf2c88c7612.png?v=1","role":"input","filename":"31-july-meeting-screenshot.png"}],"stresses":["Transcription accuracy for real names, tool names, numbers, and technical jargon","Speaker diarization across multiple active speakers","Robustness to overlapping speech, crosstalk, and rapid turn-taking","Join reliability for bot-based and botless capture","Summary quality on identical source material","Action-item extraction with correct owners and commitments","Topic segmentation of standup updates","Search and chat grounded in the meeting content","Sharing, API, MCP, integrations, plan limits, languages, and privacy feature coverage"],"verdict":"worked","score":null,"score_total":null,"note":"The bot joined Google Meet successfully and stayed connected for the full ~25-minute call, with no mid-call disconnections or plan-limit cutoffs.","evidence_state":"observed","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/ca085ba7495e48de8bcb08c818d56a36.png?v=1","role":"context","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/05c7321001c745038bc5bb302cdc1682.png?v=1","role":"context","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/c10acfeb2e934d12b74eb27b15b19fc7.png?v=1","role":"context","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/d422a99b490745e99b9568b976a8f000.png?v=1","role":"context","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/aed8fbd4b84a4577b65ce55bf1fd1669.png?v=1","role":"context","alt":null}],"run_id":"ace58582-3d1e-48ee-996c-9b3cd03f27a2","study_title":"AI Meeting Notetakers — Capture Accurate Transcripts, Summaries & Action Items From Live Calls","study_kind":"generation","research_task":"86baxegnv","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://cdn.futuresmart.ai/public/aidemos/547dd6f13e4a420fa8ad7bf2c88c7612.png?v=1","filename":"31-july-meeting-screenshot.png","alt":"AI Demos Daily Standup — 31 July 2026","role":"input"}],"modality":"image","stresses":["Transcription accuracy for real names, tool names, numbers, and technical jargon","Speaker diarization across multiple active speakers","Robustness to overlapping speech, crosstalk, and rapid turn-taking","Join reliability for bot-based and botless capture","Summary quality on identical source material","Action-item extraction with correct owners and commitments","Topic segmentation of standup updates","Search and chat grounded in the meeting content","Sharing, API, MCP, integrations, plan limits, languages, and privacy feature coverage"]},"tool_page_slug":"fathom","tool_url":"https://aidemos.com/tools/fathom","permalink":"https://aidemos.com/evidence/64cce11b-7973-439a-bcc1-68219903d5eb","api_url":"https://ai.aidemos.com/v1/observations/64cce11b-7973-439a-bcc1-68219903d5eb"},"peers":[{"id":"5d9e1f80-f657-4216-8e73-e81047b806aa","tool":"fellow","tool_name":"Fellow","verdict":"worked","score":null,"score_total":null,"note":"The bot auto-joined via Google Calendar integration and stayed connected for the full meeting capture without drops or disconnections, including the late closing portion of the call.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/3a01097650fc49a2827e4f746d150bec.png?v=1","evidence_url":"https://aidemos.com/evidence/5d9e1f80-f657-4216-8e73-e81047b806aa"},{"id":"f9249df8-2091-492c-b5a9-4f692c3110a9","tool":"fireflies-ai","tool_name":"Fireflies.ai","verdict":"worked","score":null,"score_total":null,"note":"Used a bot-based Google Meet join: the Fireflies notetaker appeared in the People panel, and the recording player showed it still present in a 23:40 capture, matching the report’s claim of uninterrupted full-call capture.","artifact_count":5,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/81054d3b33a94e5782fee8e6f877e7af.png?v=1","evidence_url":"https://aidemos.com/evidence/f9249df8-2091-492c-b5a9-4f692c3110a9"},{"id":"c83c6126-1c92-4f50-a102-cbdfe3713f0f","tool":"granola","tool_name":"Granola","verdict":"worked","score":null,"score_total":null,"note":"The botless desktop capture recorded the full ~25-minute meeting end-to-end without visible dropouts; the transcript reaches the call’s closing lines, indicating uninterrupted capture rather than a mid-call failure.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/7fcbe8cb7009429e98b95bdcabbaddcd.png?v=1","evidence_url":"https://aidemos.com/evidence/c83c6126-1c92-4f50-a102-cbdfe3713f0f"},{"id":"45f5aaac-5c02-4796-8e1c-058a8fce59c8","tool":"happyscribe","tool_name":"HappyScribe","verdict":"worked","score":null,"score_total":null,"note":"The bot joined the Google Meet call and stayed connected through the full meeting capture with no visible dropout or mid-call disconnection.","artifact_count":4,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/e8b523853472416086473489ced66b2b.png?v=1","evidence_url":"https://aidemos.com/evidence/45f5aaac-5c02-4796-8e1c-058a8fce59c8"},{"id":"9ebe6fbe-6f09-440e-afde-66593a710483","tool":"meetgeek","tool_name":"MeetGeek","verdict":"worked","score":null,"score_total":null,"note":"The bot successfully joined a Google Meet call and the report says it captured the full ~30-minute meeting with zero disconnections or data loss.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/75a3b9a7a5a744e48ae416a4c23cd654.png?v=1","evidence_url":"https://aidemos.com/evidence/9ebe6fbe-6f09-440e-afde-66593a710483"},{"id":"d31ec8ee-e1f2-4d37-8cd5-9eeaf5e99255","tool":"notta","tool_name":"Notta","verdict":"worked","score":null,"score_total":null,"note":"The bot-based Google Meet join was reliable in the tested call: Notta Bot appeared in the meeting list, admitted/managed normally, and the capture ran through the end of the session without disconnects or plan-limit cutoffs.","artifact_count":4,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/00e1f74ee5b94aa7ac2fcb3fa7e3cf49.png?v=1","evidence_url":"https://aidemos.com/evidence/d31ec8ee-e1f2-4d37-8cd5-9eeaf5e99255"},{"id":"1ec8c4c4-913b-4e17-8150-83589c5c2f7b","tool":"otter-ai","tool_name":"Otter.ai","verdict":"worked","score":null,"score_total":null,"note":"Otter’s bot joined Google Meet successfully and stayed connected through the full ~25-minute call with no mid-call dropout or ejection, so the meeting was captured end to end.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/0db8542fb3c24e2e936f4fb01433dcac.png?v=1","evidence_url":"https://aidemos.com/evidence/1ec8c4c4-913b-4e17-8150-83589c5c2f7b"}],"other_criteria":[{"id":"8340c79d-88de-4721-9c3e-c58e057071fb","criterion":"action-item-extraction","criterion_name":"Action-Item Extraction","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Fathom extracts real commitments into an ACTION ITEMS section with owner attribution; the published output shows timestamped tasks and a named owner on the item.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/8340c79d-88de-4721-9c3e-c58e057071fb"},{"id":"070f7f5f-046c-4139-a132-6e00a3b57acc","criterion":"chat-with-notes-ask-questions","criterion_name":"Chat with Notes / Ask Questions","rank_role":"context","verdict":"worked","score":null,"score_total":null,"note":"Ask Fathom answers direct factual questions from the meeting notes with grounded references; for one query it answered that a call was scheduled for 6th August and linked the supporting transcript mention.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/070f7f5f-046c-4139-a132-6e00a3b57acc"},{"id":"79d2b9bb-26b6-4b8e-858a-25c5849095c9","criterion":"search-across-notes","criterion_name":"Search Across Notes","rank_role":"context","verdict":"worked","score":null,"score_total":null,"note":"Transcript search supports keyword lookup and returns a matched snippet for \"API,\" surfacing the relevant moment and the linked action item \"Create subtasks for API benchmarking; tag Divya on completion.\"","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/79d2b9bb-26b6-4b8e-858a-25c5849095c9"},{"id":"e9695d25-df6e-448b-b185-4ad01e885bdf","criterion":"speaker-diarization","criterion_name":"Speaker Diarization","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"Fathom separates most speakers correctly in a busy multi-speaker standup, but the report observed one rapid-transition segment where two speakers' lines were merged into a single speaker block.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/e9695d25-df6e-448b-b185-4ad01e885bdf"},{"id":"53731c87-42c6-49f3-b19d-6a3ab5fe169e","criterion":"summary-quality","criterion_name":"Summary Quality","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Fathom produces a skimmable written recap with named sections such as Meeting Purpose, Key Takeaways, and Topics; the report describes the summary as structured and concise.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/53731c87-42c6-49f3-b19d-6a3ab5fe169e"},{"id":"46b5c695-1c28-461a-ad98-573516adf29f","criterion":"topic-segmentation","criterion_name":"Topic Segmentation","rank_role":"context","verdict":"worked","score":null,"score_total":null,"note":"Fathom breaks the standup into named topical sections instead of one blob, including headers like \"Process & System Blockers\" and \"Content Quality & Review Process.\"","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/46b5c695-1c28-461a-ad98-573516adf29f"},{"id":"1c4f4fe3-3ed8-490d-992b-a25ddd604669","criterion":"transcription-accuracy","criterion_name":"Transcription Accuracy","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"Fathom's transcript mostly preserves the meeting's names and technical content, but the report records one confirmed name-level error: \"Mahreen\" was rendered as \"Meryl.\"","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/1c4f4fe3-3ed8-490d-992b-a25ddd604669"}],"appears_in":[{"page_type":"ranking","slug":"ai-meeting-notetakers","title":"Best AI Meeting Notetakers for Accurate Transcripts, Summaries, and Action Items","url":"https://aidemos.com/best/ai-meeting-notetakers","binding":"run"}],"same_scenario":[{"id":"5d9e1f80-f657-4216-8e73-e81047b806aa","tool":"fellow","tool_name":"Fellow","verdict":"worked","score":null,"score_total":null,"note":"The bot auto-joined via Google Calendar integration and stayed connected for the full meeting capture without drops or disconnections, including the late closing portion of the call."},{"id":"f9249df8-2091-492c-b5a9-4f692c3110a9","tool":"fireflies-ai","tool_name":"Fireflies.ai","verdict":"worked","score":null,"score_total":null,"note":"Used a bot-based Google Meet join: the Fireflies notetaker appeared in the People panel, and the recording player showed it still present in a 23:40 capture, matching the report’s claim of uninterrupted full-call capture."},{"id":"c83c6126-1c92-4f50-a102-cbdfe3713f0f","tool":"granola","tool_name":"Granola","verdict":"worked","score":null,"score_total":null,"note":"The botless desktop capture recorded the full ~25-minute meeting end-to-end without visible dropouts; the transcript reaches the call’s closing lines, indicating uninterrupted capture rather than a mid-call failure."},{"id":"45f5aaac-5c02-4796-8e1c-058a8fce59c8","tool":"happyscribe","tool_name":"HappyScribe","verdict":"worked","score":null,"score_total":null,"note":"The bot joined the Google Meet call and stayed connected through the full meeting capture with no visible dropout or mid-call disconnection."},{"id":"9ebe6fbe-6f09-440e-afde-66593a710483","tool":"meetgeek","tool_name":"MeetGeek","verdict":"worked","score":null,"score_total":null,"note":"The bot successfully joined a Google Meet call and the report says it captured the full ~30-minute meeting with zero disconnections or data loss."},{"id":"d31ec8ee-e1f2-4d37-8cd5-9eeaf5e99255","tool":"notta","tool_name":"Notta","verdict":"worked","score":null,"score_total":null,"note":"The bot-based Google Meet join was reliable in the tested call: Notta Bot appeared in the meeting list, admitted/managed normally, and the capture ran through the end of the session without disconnects or plan-limit cutoffs."},{"id":"1ec8c4c4-913b-4e17-8150-83589c5c2f7b","tool":"otter-ai","tool_name":"Otter.ai","verdict":"worked","score":null,"score_total":null,"note":"Otter’s bot joined Google Meet successfully and stayed connected through the full ~25-minute call with no mid-call dropout or ejection, so the meeting was captured end to end."}]}