{"observation":{"id":"30c8040c-4369-44fa-a1f9-2c9315d276f5","tool":"google-cloud-speech-to-text","tool_name":"Google Cloud Speech-to-Text","criterion":"automation-level","criterion_name":"Automation level","criterion_definition":"How many API steps or calls the workflow requires, and whether it completes without operator input.","criterion_evidence_type":"capability","criterion_rank_role":"context","criterion_rank_role_reason":"How many API steps or whether operator input is needed affects convenience and workflow, but not whether the engine transcribes accurately once run. (3 of 3 judges)","scenario":"overlapping-meeting-speech-with-cross-talk","scenario_name":"Overlapping meeting speech with cross-talk","group_tag":"speech-to-text-benchmark","scenario_description":"A long AMI meeting audio file with multiple speakers talking over one another, background room noise, and crosstalk. It was used to test how well an STT system handles noisy multi-speaker conversational audio and speaker separation.","modality":"audio","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://cdn.futuresmart.ai/public/aidemos/12990982a1614cc5b18a3a450e0e657f.wav?v=1","role":"input","filename":"crosstalk.wav"}],"stresses":["overlapping speech","background noise robustness","multi-speaker separation","speaker diarization accuracy","long-form audio handling"],"verdict":"mixed","score":null,"score_total":null,"note":"The recorded workflow submits the audio as a POST to the v2 endpoint and reaches a scored result without operator input, but the trace explicitly says per-call timings were not instrumented, so no measured call count is claimed.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/91b7c793294a479abdfe4c0813886c0c.png?v=1","role":"input","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/ef993bd0e519426dadfbd740e61797cb.png?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/0e057a02d9d64d09b7aa20a2d5e4da1e.png?v=1","role":"output","alt":null}],"run_id":"469de0c2-d727-4f8f-a60e-e3a5bf8e8588","study_title":"Transcribe Audio Accurately — Speech-to-Text Engine Benchmark","study_kind":"generation","research_task":"86baxegpu","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://cdn.futuresmart.ai/public/aidemos/12990982a1614cc5b18a3a450e0e657f.wav?v=1","filename":"crosstalk.wav","alt":"Overlapping meeting speech with cross-talk","role":"input"}],"modality":"audio","stresses":["overlapping speech","background noise robustness","multi-speaker separation","speaker diarization accuracy","long-form audio handling"]},"tool_page_slug":"google-cloud-speech-to-text","tool_url":"https://aidemos.com/tools/google-cloud-speech-to-text","permalink":"https://aidemos.com/evidence/30c8040c-4369-44fa-a1f9-2c9315d276f5","api_url":"https://ai.aidemos.com/v1/observations/30c8040c-4369-44fa-a1f9-2c9315d276f5"},"peers":[{"id":"e9a2ac7f-48cc-4493-a912-7cc95aadb130","tool":"groqcloud","tool_name":"GroqCloud","verdict":"mixed","score":null,"score_total":null,"note":"Uses a single multipart POST workflow, but this run stops at API rejection and never completes transcription end to end, so the automation is one-step but unsuccessful on this input.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/fe907e170abe427bb355b1fef95ebc59.png?v=1","evidence_url":"https://aidemos.com/evidence/e9a2ac7f-48cc-4493-a912-7cc95aadb130"}],"other_criteria":[{"id":"818d5ab9-cd17-4b1b-b244-887a4102298d","criterion":"export","criterion_name":"Export","rank_role":"context","verdict":"mixed","score":null,"score_total":null,"note":"Returns a mid-depth transcript payload: payload depth 2/3 with 5009 word-level timed tokens, confidence present, and no speaker labels.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/818d5ab9-cd17-4b1b-b244-887a4102298d"},{"id":"a04e7fb4-6535-45db-83e4-d36ad4ec0258","criterion":"output-quality","criterion_name":"Output quality","rank_role":"decisive","verdict":"struggled","score":43.5,"score_total":null,"note":"On overlapping crosstalk, the transcript quality is poor at 43.50% WER, with 633 substitutions, 2614 deletions, and 50 insertions against 7579 reference words, yielding 5015 hypothesis words.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/a04e7fb4-6535-45db-83e4-d36ad4ec0258"}],"appears_in":[{"page_type":"ranking","slug":"speech-to-text-apis","title":"Best AI Tools for Accurate Speech-to-Text on Hard Audio","url":"https://aidemos.com/best/speech-to-text-apis","binding":"run"}],"same_scenario":[{"id":"e9a2ac7f-48cc-4493-a912-7cc95aadb130","tool":"groqcloud","tool_name":"GroqCloud","verdict":"mixed","score":null,"score_total":null,"note":"Uses a single multipart POST workflow, but this run stops at API rejection and never completes transcription end to end, so the automation is one-step but unsuccessful on this input."}]}