{"observation":{"id":"462a2150-a995-4fcf-9d2e-b66545b9de36","tool":"skyvern","tool_name":"Skyvern","criterion":"output-quality","criterion_name":"Output Quality","criterion_definition":"Conceptually matches the prompt and is visually usable.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Whether the result matches the page and is actually usable is the main outcome the ranking is trying to measure. (3 of 3 judges)","scenario":"glassdoor-software-engineer-jobs-behind-sign-in-modal","scenario_name":"Glassdoor software engineer jobs behind sign-in modal","group_tag":"web-scraping-benchmark","scenario_description":"A Glassdoor jobs listing page protected by Cloudflare and a sign-in/interstitial overlay, used to test proxy evasion, anti-bot handling, and the ability to dismiss blocking modal UI before extracting listings.","modality":"mixed","input_text":"https://www.glassdoor.com/Job/software-engineer-jobs-SRCH_KO0,17.htm — Dismiss any immediate sign-in or signup modal overlays that block the view. Once cleared, extract the top 5 job listings, including job title, company name, location, and the short summary snippet.","input_artifact_refs":[],"stresses":[],"verdict":"worked","score":null,"score_total":null,"note":"Produces a clean job-listing extract with deterministic fields; the visible markdown output contains 3 numbered entries and includes company, location, and summary information instead of boilerplate.","evidence_state":"verified","source":"first-party","artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/skyvern-skyvern-job-listings-output-markdown.png","role":"output","alt":"Markdown output listing extracted job postings"}],"run_id":"06e1dbd6-5518-4af8-aa1a-735259a75b4f","study_title":"Scrape Web Pages Into Clean Markdown or Structured Data Using AI","study_kind":"backfill","research_task":"86b9jm3a3","tested_at":"2026-06-23T04:32:57.304000+00:00","completeness":"input-and-output","input":{"state":"text","text":"https://www.glassdoor.com/Job/software-engineer-jobs-SRCH_KO0,17.htm — Dismiss any immediate sign-in or signup modal overlays that block the view. Once cleared, extract the top 5 job listings, including job title, company name, location, and the short summary snippet.","files":[],"modality":"mixed","stresses":[]},"tool_page_slug":"skyvern","tool_url":"https://aidemos.com/tools/skyvern","permalink":"https://aidemos.com/evidence/462a2150-a995-4fcf-9d2e-b66545b9de36","api_url":"https://ai.aidemos.com/v1/observations/462a2150-a995-4fcf-9d2e-b66545b9de36"},"peers":[{"id":"578ce21d-ead2-4717-a60a-d00722e7a805","tool":"firecrawl","tool_name":"Firecrawl","verdict":"worked","score":null,"score_total":null,"note":"Preserves Markdown structure and core content cleanly, including headings, the ingredients table, and the step-by-step workflow, with excellent textual fidelity.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/578ce21d-ead2-4717-a60a-d00722e7a805"},{"id":"8bb24646-3803-4641-86bc-6cad9801e3e7","tool":"jina-ai-reader","tool_name":"Jina AI Reader","verdict":"failed","score":null,"score_total":null,"note":"When blocked, it emits an anti-bot 'Humans only' text wall in multiple languages rather than job listings, so the returned text is not usable as a listing extract.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/jina-ai-reader-jina-reader-glassdoor-humans-only.png","evidence_url":"https://aidemos.com/evidence/8bb24646-3803-4641-86bc-6cad9801e3e7"},{"id":"2dddc43a-aa71-4fd9-89a6-c195bdbcd6e6","tool":"spider","tool_name":"Spider","verdict":"mixed","score":null,"score_total":null,"note":"The extractor preserves the main recipe content accurately, including the ingredients block and directions layout, but the returned markdown is highly unrefined and bloated with boilerplate text.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/2dddc43a-aa71-4fd9-89a6-c195bdbcd6e6"}],"other_criteria":[{"id":"f5b362d1-37e4-4dde-ab5f-84a4b216911b","criterion":"automation-level","criterion_name":"Automation Level","rank_role":"context","verdict":"worked","score":null,"score_total":null,"note":"Runs fully autonomously and handles modal elements dynamically without hardcoded user interaction scripts or selector mapping.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/f5b362d1-37e4-4dde-ab5f-84a4b216911b"},{"id":"540d0e24-1d3a-41a0-9061-fb9f2791210a","criterion":"automation-level","criterion_name":"Automation Level","rank_role":"context","verdict":"mixed","score":null,"score_total":null,"note":"Can execute the extraction autonomously, but the interface recorder can drift out of sync during the run.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/540d0e24-1d3a-41a0-9061-fb9f2791210a"},{"id":"5446bb3c-1af4-47a1-b859-1fd1d7431f49","criterion":"interaction-stability","criterion_name":"Interaction Stability","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"Adds significant visual-validation overhead, making runs much slower than raw text parsing even when the extraction completes.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/5446bb3c-1af4-47a1-b859-1fd1d7431f49"},{"id":"43515b7c-a158-4738-8839-4c0647bf3511","criterion":"proxy-evasion","criterion_name":"Proxy Evasion","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Gets through a blocking interstitial/sign-in layer and reaches job listings content instead of stopping at the gate.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/43515b7c-a158-4738-8839-4c0647bf3511"},{"id":"3ffd638c-cf33-4a5f-8246-bdce17a568cf","criterion":"proxy-evasion","criterion_name":"Proxy Evasion","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The tool can bypass a standard interstitial sign-in/modal barrier and still recover the target content, outputting structured job listings with deterministic fields such as title, company, location, and summary.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/3ffd638c-cf33-4a5f-8246-bdce17a568cf"},{"id":"4c1cca8d-a9e3-49f9-bd51-3e8ca0ccdb98","criterion":"proxy-evasion","criterion_name":"Proxy Evasion","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Can get past a dynamic sign-in modal overlay and continue extraction on a blocked or gated page sequence.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/4c1cca8d-a9e3-49f9-bd51-3e8ca0ccdb98"},{"id":"b4680024-2a4a-45f1-a433-a3934b998a46","criterion":"schema-extraction-integrity","criterion_name":"Schema Extraction Integrity","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Produces deterministic structured job records with the requested fields; the visible output shows multiple listings with consistent company, location, and summary keys, including the first three entries.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/b4680024-2a4a-45f1-a433-a3934b998a46"},{"id":"39a695ad-5fa0-4806-afc6-c46feb530807","criterion":"visual-spatial-awareness","criterion_name":"Visual Spatial Awareness","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Localizes the meaningful job-listing region on a page with a sign-in overlay and filters the surrounding noise, extracting the top 3 listings rather than the modal chrome.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/39a695ad-5fa0-4806-afc6-c46feb530807"}],"appears_in":[{"page_type":"ranking","slug":"ai-web-scraping-tools","title":"Best AI Tools to Scrape Web Pages Into Clean Markdown or Structured Data","url":"https://aidemos.com/best/ai-web-scraping-tools","binding":"run"}],"same_scenario":[{"id":"578ce21d-ead2-4717-a60a-d00722e7a805","tool":"firecrawl","tool_name":"Firecrawl","verdict":"worked","score":null,"score_total":null,"note":"Preserves Markdown structure and core content cleanly, including headings, the ingredients table, and the step-by-step workflow, with excellent textual fidelity."},{"id":"8bb24646-3803-4641-86bc-6cad9801e3e7","tool":"jina-ai-reader","tool_name":"Jina AI Reader","verdict":"failed","score":null,"score_total":null,"note":"When blocked, it emits an anti-bot 'Humans only' text wall in multiple languages rather than job listings, so the returned text is not usable as a listing extract."},{"id":"2dddc43a-aa71-4fd9-89a6-c195bdbcd6e6","tool":"spider","tool_name":"Spider","verdict":"mixed","score":null,"score_total":null,"note":"The extractor preserves the main recipe content accurately, including the ingredients block and directions layout, but the returned markdown is highly unrefined and bloated with boilerplate text."}]}