{"observation":{"id":"43515b7c-a158-4738-8839-4c0647bf3511","tool":"skyvern","tool_name":"Skyvern","criterion":"proxy-evasion","criterion_name":"Proxy Evasion","criterion_definition":"How well the tool gets past anti-bot protections and modal or firewall barriers to reach the page content.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Bypassing anti-bot and modal barriers is often required to reach the page content at all, so it directly affects scraping success. (3 of 3 judges)","scenario":"glassdoor-software-engineer-jobs-behind-sign-in-modal","scenario_name":"Glassdoor software engineer jobs behind sign-in modal","group_tag":"web-scraping-benchmark","scenario_description":"A Glassdoor jobs listing page protected by Cloudflare and a sign-in/interstitial overlay, used to test proxy evasion, anti-bot handling, and the ability to dismiss blocking modal UI before extracting listings.","modality":"mixed","input_text":"https://www.glassdoor.com/Job/software-engineer-jobs-SRCH_KO0,17.htm — Dismiss any immediate sign-in or signup modal overlays that block the view. Once cleared, extract the top 5 job listings, including job title, company name, location, and the short summary snippet.","input_artifact_refs":[],"stresses":[],"verdict":"worked","score":null,"score_total":null,"note":"Gets through a blocking interstitial/sign-in layer and reaches job listings content instead of stopping at the gate.","evidence_state":"verified","source":"first-party","artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/skyvern-skyvern-job-listings-output-markdown.png","role":"output","alt":"Markdown output listing extracted job postings"}],"run_id":"06e1dbd6-5518-4af8-aa1a-735259a75b4f","study_title":"Scrape Web Pages Into Clean Markdown or Structured Data Using AI","study_kind":"backfill","research_task":"86b9jm3a3","tested_at":"2026-06-23T04:32:57.304000+00:00","completeness":"input-and-output","input":{"state":"text","text":"https://www.glassdoor.com/Job/software-engineer-jobs-SRCH_KO0,17.htm — Dismiss any immediate sign-in or signup modal overlays that block the view. Once cleared, extract the top 5 job listings, including job title, company name, location, and the short summary snippet.","files":[],"modality":"mixed","stresses":[]},"tool_page_slug":"skyvern","tool_url":"https://aidemos.com/tools/skyvern","permalink":"https://aidemos.com/evidence/43515b7c-a158-4738-8839-4c0647bf3511","api_url":"https://ai.aidemos.com/v1/observations/43515b7c-a158-4738-8839-4c0647bf3511"},"peers":[{"id":"3d6a4fa2-e8e0-4a86-b2f5-866746cfeb98","tool":"firecrawl","tool_name":"Firecrawl","verdict":"mixed","score":null,"score_total":null,"note":"Returns the protected job page in a noisy flattened form, interleaving the target content with navigation buttons, search filter blocks, and internal page links instead of a cleanly isolated listing block.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/firecrawl-firecrawl-glassdoor-scrape-markdown-output.png","evidence_url":"https://aidemos.com/evidence/3d6a4fa2-e8e0-4a86-b2f5-866746cfeb98"},{"id":"cc9693d1-db28-4dc3-9db7-5bcf20dd9ebd","tool":"jina-ai-reader","tool_name":"Jina AI Reader","verdict":"struggled","score":null,"score_total":null,"note":"Although the anti-bot wall is bypassed, the recovered Glassdoor output is still a raw DOM dump with sign-in notices, framework noise, and header redirects interleaved, so the target listings require heavy downstream cleanup.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/cc9693d1-db28-4dc3-9db7-5bcf20dd9ebd"},{"id":"e69ce784-8987-4af8-bba7-1a1e1fe4ee59","tool":"spider","tool_name":"Spider","verdict":"failed","score":null,"score_total":null,"note":"It was stopped by the site's security interstitial and returned only anti-bot warning text instead of the target listings, showing no recovered job content.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/spider-spider-playground-glassdoor-humans-only.png","evidence_url":"https://aidemos.com/evidence/e69ce784-8987-4af8-bba7-1a1e1fe4ee59"}],"other_criteria":[{"id":"540d0e24-1d3a-41a0-9061-fb9f2791210a","criterion":"automation-level","criterion_name":"Automation Level","rank_role":"context","verdict":"mixed","score":null,"score_total":null,"note":"Can execute the extraction autonomously, but the interface recorder can drift out of sync during the run.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/540d0e24-1d3a-41a0-9061-fb9f2791210a"},{"id":"f5b362d1-37e4-4dde-ab5f-84a4b216911b","criterion":"automation-level","criterion_name":"Automation Level","rank_role":"context","verdict":"worked","score":null,"score_total":null,"note":"Runs fully autonomously and handles modal elements dynamically without hardcoded user interaction scripts or selector mapping.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/f5b362d1-37e4-4dde-ab5f-84a4b216911b"},{"id":"5446bb3c-1af4-47a1-b859-1fd1d7431f49","criterion":"interaction-stability","criterion_name":"Interaction Stability","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"Adds significant visual-validation overhead, making runs much slower than raw text parsing even when the extraction completes.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/5446bb3c-1af4-47a1-b859-1fd1d7431f49"},{"id":"6883c340-cb0d-4463-abd0-44546868d454","criterion":"output-quality","criterion_name":"Output Quality","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Produces a clean markdown_content payload with readable headings and bold labels for the extracted job listings.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/6883c340-cb0d-4463-abd0-44546868d454"},{"id":"462a2150-a995-4fcf-9d2e-b66545b9de36","criterion":"output-quality","criterion_name":"Output Quality","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Produces a clean job-listing extract with deterministic fields; the visible markdown output contains 3 numbered entries and includes company, location, and summary information instead of boilerplate.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/462a2150-a995-4fcf-9d2e-b66545b9de36"},{"id":"0cfacd04-ec5e-46d4-94da-beb0d54ab9c9","criterion":"output-quality","criterion_name":"Output Quality","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Produces a highly accurate structured schema for dynamic page data, with the extracted sizes captured cleanly.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/0cfacd04-ec5e-46d4-94da-beb0d54ab9c9"},{"id":"86f5dc67-b3bf-4ee8-afe5-02e3f720c517","criterion":"output-quality","criterion_name":"Output Quality","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Produces a perfectly formatted JSON schema with deterministic keys for titles, locations, and company names.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/86f5dc67-b3bf-4ee8-afe5-02e3f720c517"},{"id":"b4680024-2a4a-45f1-a433-a3934b998a46","criterion":"schema-extraction-integrity","criterion_name":"Schema Extraction Integrity","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Produces deterministic structured job records with the requested fields; the visible output shows multiple listings with consistent company, location, and summary keys, including the first three entries.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/b4680024-2a4a-45f1-a433-a3934b998a46"},{"id":"39a695ad-5fa0-4806-afc6-c46feb530807","criterion":"visual-spatial-awareness","criterion_name":"Visual Spatial Awareness","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Localizes the meaningful job-listing region on a page with a sign-in overlay and filters the surrounding noise, extracting the top 3 listings rather than the modal chrome.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/39a695ad-5fa0-4806-afc6-c46feb530807"}],"appears_in":[{"page_type":"ranking","slug":"ai-web-scraping-tools","title":"Best AI Tools to Scrape Web Pages Into Clean Markdown or Structured Data","url":"https://aidemos.com/best/ai-web-scraping-tools","binding":"run"}],"same_scenario":[{"id":"3d6a4fa2-e8e0-4a86-b2f5-866746cfeb98","tool":"firecrawl","tool_name":"Firecrawl","verdict":"mixed","score":null,"score_total":null,"note":"Returns the protected job page in a noisy flattened form, interleaving the target content with navigation buttons, search filter blocks, and internal page links instead of a cleanly isolated listing block."},{"id":"cc9693d1-db28-4dc3-9db7-5bcf20dd9ebd","tool":"jina-ai-reader","tool_name":"Jina AI Reader","verdict":"struggled","score":null,"score_total":null,"note":"Although the anti-bot wall is bypassed, the recovered Glassdoor output is still a raw DOM dump with sign-in notices, framework noise, and header redirects interleaved, so the target listings require heavy downstream cleanup."},{"id":"e69ce784-8987-4af8-bba7-1a1e1fe4ee59","tool":"spider","tool_name":"Spider","verdict":"failed","score":null,"score_total":null,"note":"It was stopped by the site's security interstitial and returned only anti-bot warning text instead of the target listings, showing no recovered job content."}]}