{"observation":{"id":"cd6fdc46-4e9e-4109-be28-5233a244d2ee","tool":"jina-ai-reader","tool_name":"Jina AI Reader","criterion":"schema-extraction-integrity","criterion_name":"Schema Extraction Integrity","criterion_definition":"How accurately the tool outputs the requested structured data with the correct fields and valid formatting.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Accurate fields and valid formatting are central to structured-data extraction quality. (3 of 3 judges)","scenario":"glassdoor-software-engineer-jobs-behind-sign-in-modal","scenario_name":"Glassdoor software engineer jobs behind sign-in modal","group_tag":"web-scraping-benchmark","scenario_description":"A Glassdoor jobs listing page protected by Cloudflare and a sign-in/interstitial overlay, used to test proxy evasion, anti-bot handling, and the ability to dismiss blocking modal UI before extracting listings.","modality":"mixed","input_text":"https://www.glassdoor.com/Job/software-engineer-jobs-SRCH_KO0,17.htm — Dismiss any immediate sign-in or signup modal overlays that block the view. Once cleared, extract the top 5 job listings, including job title, company name, location, and the short summary snippet.","input_artifact_refs":[],"stresses":[],"verdict":"failed","score":null,"score_total":null,"note":"Can break entirely when URL handling loops or duplicates the target path: the report says the primary output became completely useless for data compilation after an address-resolution bug.","evidence_state":"observed","source":"first-party","artifacts":[],"run_id":"06e1dbd6-5518-4af8-aa1a-735259a75b4f","study_title":"Scrape Web Pages Into Clean Markdown or Structured Data Using AI","study_kind":"backfill","research_task":"86b9jm3a3","tested_at":"2026-06-23T04:32:57.304000+00:00","completeness":"input-only","input":{"state":"text","text":"https://www.glassdoor.com/Job/software-engineer-jobs-SRCH_KO0,17.htm — Dismiss any immediate sign-in or signup modal overlays that block the view. Once cleared, extract the top 5 job listings, including job title, company name, location, and the short summary snippet.","files":[],"modality":"mixed","stresses":[]},"tool_page_slug":"jina-ai-reader","tool_url":"https://aidemos.com/tools/jina-ai-reader","permalink":"https://aidemos.com/evidence/cd6fdc46-4e9e-4109-be28-5233a244d2ee","api_url":"https://ai.aidemos.com/v1/observations/cd6fdc46-4e9e-4109-be28-5233a244d2ee"},"peers":[{"id":"4a5a4574-2020-475c-9c89-24c4e72996c2","tool":"firecrawl","tool_name":"Firecrawl","verdict":"worked","score":null,"score_total":null,"note":"The extractor returned the core job-listing payload accurately, including active software engineering listings, corporate profile names, salary estimates, and required technical skill arrays.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/4a5a4574-2020-475c-9c89-24c4e72996c2"},{"id":"b4680024-2a4a-45f1-a433-a3934b998a46","tool":"skyvern","tool_name":"Skyvern","verdict":"worked","score":null,"score_total":null,"note":"Produces deterministic structured job records with the requested fields; the visible output shows multiple listings with consistent company, location, and summary keys, including the first three entries.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/skyvern-skyvern-job-listings-output-markdown.png","evidence_url":"https://aidemos.com/evidence/b4680024-2a4a-45f1-a433-a3934b998a46"},{"id":"5d31e225-3519-47b9-97f4-420a31f39baa","tool":"spider","tool_name":"Spider","verdict":"worked","score":null,"score_total":null,"note":"It can preserve the main recipe content accurately, keeping the central ingredients block and recipe directions layout intact.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/5d31e225-3519-47b9-97f4-420a31f39baa"}],"other_criteria":[{"id":"cdad132e-c3cd-48e9-a6d7-fc50fa82d10b","criterion":"interaction-stability","criterion_name":"Interaction Stability","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Can process a protected page through edge security without being dropped, successfully bypassing the firewall and recovering plain-text markers from the page flow.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/cdad132e-c3cd-48e9-a6d7-fc50fa82d10b"},{"id":"9ea0b03c-f458-4cb0-827b-c31ddd8edde7","criterion":"js-dom-hydration","criterion_name":"JS DOM Hydration","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"It fails to wait for client-side hydration on a dynamic product page: the size selector grid comes back as empty layout nodes while the extractor instead pulls the site's international menu and regional index.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/9ea0b03c-f458-4cb0-827b-c31ddd8edde7"},{"id":"1c2ec4aa-ed8e-4021-8ef3-56d35ce08689","criterion":"noise-filtering","criterion_name":"Noise Filtering","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"It can recover the page text layer, but the extraction still leaves job data interleaved with French and German translation strings and header redirect text, so heavy post-processing cleanup is still required.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/1c2ec4aa-ed8e-4021-8ef3-56d35ce08689"},{"id":"8bb24646-3803-4641-86bc-6cad9801e3e7","criterion":"output-quality","criterion_name":"Output Quality","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"When blocked, it emits an anti-bot 'Humans only' text wall in multiple languages rather than job listings, so the returned text is not usable as a listing extract.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/8bb24646-3803-4641-86bc-6cad9801e3e7"},{"id":"fc132656-20f1-48c6-9daa-c78d98a6dc3e","criterion":"output-quality","criterion_name":"Output Quality","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"Its output quality is partial on the hydrated ecommerce page: it preserves the SEO header and static price markers, but replaces the transactional product data with a giant global link directory.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/fc132656-20f1-48c6-9daa-c78d98a6dc3e"},{"id":"b374e445-5d10-4511-9fda-d190ada897a4","criterion":"output-quality","criterion_name":"Output Quality","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"When this run collapses, the extractor returns only a standard HTTP 404 text string wrapped inside a global website layout frame, leaving the primary output unusable for data compilation.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/b374e445-5d10-4511-9fda-d190ada897a4"},{"id":"cc9693d1-db28-4dc3-9db7-5bcf20dd9ebd","criterion":"proxy-evasion","criterion_name":"Proxy Evasion","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"Although the anti-bot wall is bypassed, the recovered Glassdoor output is still a raw DOM dump with sign-in notices, framework noise, and header redirects interleaved, so the target listings require heavy downstream cleanup.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/cc9693d1-db28-4dc3-9db7-5bcf20dd9ebd"},{"id":"9ba9480d-5de1-471e-8f98-e75d0c31664e","criterion":"proxy-evasion","criterion_name":"Proxy Evasion","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The tool can bypass standard edge security filters on the Glassdoor target and is not dropped by the firewall checks in this run sequence.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/9ba9480d-5de1-471e-8f98-e75d0c31664e"},{"id":"2822453f-1b39-4d2c-89f4-0cdbb838e1f2","criterion":"proxy-evasion","criterion_name":"Proxy Evasion","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The backend can bypass a standard Glassdoor \"Humans only\" interstitial and return page text in about 3.6 seconds, indicating that basic anti-bot and proxy barriers were cleared in this run.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/2822453f-1b39-4d2c-89f4-0cdbb838e1f2"},{"id":"a4d67da1-472a-4def-8bba-880a1779ac0f","criterion":"proxy-evasion","criterion_name":"Proxy Evasion","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"It failed to get past the anti-bot barrier; the 3.6 s response is the \"Humans only\" block page instead of the target job listings.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/a4d67da1-472a-4def-8bba-880a1779ac0f"},{"id":"5d43e73a-3990-4b49-9e1c-c2260bee64dd","criterion":"visual-spatial-awareness","criterion_name":"Visual Spatial Awareness","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Can isolate basic boilerplate elements from a broken page render: it still extracted header navigation and privacy-disclosure text into markdown even when the target resolved into a 404 layout.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/5d43e73a-3990-4b49-9e1c-c2260bee64dd"},{"id":"1a549a35-c517-452c-ae84-1e94c110de1b","criterion":"visual-spatial-awareness","criterion_name":"Visual Spatial Awareness","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"Fails to structurally filter interstitial noise, leaving raw framework and security-page text instead of isolating meaningful job content.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/1a549a35-c517-452c-ae84-1e94c110de1b"},{"id":"b100940e-da7e-4098-8065-1e0b0972fe58","criterion":"visual-spatial-awareness","criterion_name":"Visual Spatial Awareness","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"Produces poor structural filtering on guarded pages: the recovered job text is heavily interleaved with framework noise, sign-in alerts, and header redirects, requiring substantial post-processing to clean.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/b100940e-da7e-4098-8065-1e0b0972fe58"},{"id":"d7fabde1-b4d9-4b9a-a64f-652a6da9ea7d","criterion":"visual-spatial-awareness","criterion_name":"Visual Spatial Awareness","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"It fails to isolate the content region and instead emits the sign-in overlay text in at least 5 languages (English, French, German, Dutch, Spanish).","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/d7fabde1-b4d9-4b9a-a64f-652a6da9ea7d"}],"appears_in":[{"page_type":"ranking","slug":"ai-web-scraping-tools","title":"Best AI Tools to Scrape Web Pages Into Clean Markdown or Structured Data","url":"https://aidemos.com/best/ai-web-scraping-tools","binding":"run"}],"same_scenario":[{"id":"4a5a4574-2020-475c-9c89-24c4e72996c2","tool":"firecrawl","tool_name":"Firecrawl","verdict":"worked","score":null,"score_total":null,"note":"The extractor returned the core job-listing payload accurately, including active software engineering listings, corporate profile names, salary estimates, and required technical skill arrays."},{"id":"b4680024-2a4a-45f1-a433-a3934b998a46","tool":"skyvern","tool_name":"Skyvern","verdict":"worked","score":null,"score_total":null,"note":"Produces deterministic structured job records with the requested fields; the visible output shows multiple listings with consistent company, location, and summary keys, including the first three entries."},{"id":"5d31e225-3519-47b9-97f4-420a31f39baa","tool":"spider","tool_name":"Spider","verdict":"worked","score":null,"score_total":null,"note":"It can preserve the main recipe content accurately, keeping the central ingredients block and recipe directions layout intact."}]}