{"observation":{"id":"813f348c-4de5-462c-b815-5c06a67402cc","tool":"jina-ai-reader","tool_name":"Jina AI Reader","criterion":"interaction-stability","criterion_name":"Interaction Stability","criterion_definition":"How reliably the tool executes through dynamic loading and interaction steps without losing sync or failing at runtime.","criterion_evidence_type":"capability","criterion_rank_role":"decisive","criterion_rank_role_reason":"Reliably completing dynamic loading and interaction steps is part of successfully scraping pages, not just convenience. (3 of 3 judges)","scenario":"nike-air-force-1-07-size-options-extraction","scenario_name":"Nike Air Force 1 '07 size options extraction","group_tag":null,"scenario_description":"A Nike product page with client-side JavaScript hydration used to test whether a headless scraper waits for dynamic DOM content before extracting product details and all available shoe sizes.","modality":"mixed","input_text":"https://www.nike.com/t/air-force-1-07-mens-shoes-jBrhbr/CW2288-111 — Wait for the size selection options to fully render. Extract the product name, price, and a list of all available shoe sizes.","input_artifact_refs":[],"stresses":[],"verdict":"failed","score":null,"score_total":null,"note":"It did not wait for the client-side size selector to hydrate; the response shows the product title and $115 price but no concrete size inventory values.","evidence_state":"verified","source":"first-party","artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/jina-ai-reader-jina-reader-nike-product-extraction.png","role":"output","alt":"Jina Reader extracting a Nike product page"}],"run_id":"06e1dbd6-5518-4af8-aa1a-735259a75b4f","study_title":"Scrape Web Pages Into Clean Markdown or Structured Data Using AI","study_kind":"backfill","research_task":"86b9jm3a3","tested_at":"2026-06-23T04:32:57.304000+00:00","completeness":"input-and-output","input":{"state":"text","text":"https://www.nike.com/t/air-force-1-07-mens-shoes-jBrhbr/CW2288-111 — Wait for the size selection options to fully render. Extract the product name, price, and a list of all available shoe sizes.","files":[],"modality":"mixed","stresses":[]},"tool_page_slug":"jina-ai-reader","tool_url":"https://aidemos.com/tools/jina-ai-reader","permalink":"https://aidemos.com/evidence/813f348c-4de5-462c-b815-5c06a67402cc","api_url":"https://ai.aidemos.com/v1/observations/813f348c-4de5-462c-b815-5c06a67402cc"},"peers":[{"id":"e41889fa-4519-4c96-9183-067dad4dbeb5","tool":"firecrawl","tool_name":"Firecrawl","verdict":"worked","score":null,"score_total":null,"note":"Reliably waited for client-side hydration and captured the complete size set, from M 5 / W 6.5 through M 18 / W 19.5.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/firecrawl-firecrawl-nike-scrape-markdown-output.png","evidence_url":"https://aidemos.com/evidence/e41889fa-4519-4c96-9183-067dad4dbeb5"},{"id":"94f367b2-9659-4081-bbe0-e887a50559e8","tool":"skyvern","tool_name":"Skyvern","verdict":"struggled","score":null,"score_total":null,"note":"Handles the page extraction itself, but the run’s live recording pipeline can fall out of sync during hydration: the report states the screen capture froze on the initial page view, making the recording unwatchable and hard to debug.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/skyvern-skyvern-extract-nike-af1-product-data.png","evidence_url":"https://aidemos.com/evidence/94f367b2-9659-4081-bbe0-e887a50559e8"}],"other_criteria":[{"id":"ee913a05-fce4-4910-b38e-711b6e632b06","criterion":"js-dom-hydration","criterion_name":"JS DOM Hydration","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"It fails to wait for client-side hydration on the size-selector grid: the response leaves the selector as empty layout nodes and instead spills the site-wide international menu and regional index into the output.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/ee913a05-fce4-4910-b38e-711b6e632b06"},{"id":"4c7cd93d-06c7-4f02-86b8-14711013fddf","criterion":"js-dom-hydration","criterion_name":"JS DOM Hydration","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"The engine can recover static product metadata from a JavaScript-heavy product page, including the title \"Nike Air Force 1 '07 Men's Shoes\" and the $115 price, but it does not needlessly enrich the dynamic state beyond those static markers.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/4c7cd93d-06c7-4f02-86b8-14711013fddf"},{"id":"d6a37898-46aa-424c-8ed2-cacb300ba94b","criterion":"js-dom-hydration","criterion_name":"JS DOM Hydration","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"It did not capture the client-rendered size selector; after 7.0 s the extract still lacked the live size grid and showed only static page content.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/d6a37898-46aa-424c-8ed2-cacb300ba94b"},{"id":"92e7286e-bc9b-43d1-ac1a-fb62c00e6053","criterion":"output-quality","criterion_name":"Output Quality","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"It recovered the product title and the $115 price, but the result was padded with image/link markup and navigation artifacts rather than a clean product-only extract.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/92e7286e-bc9b-43d1-ac1a-fb62c00e6053"},{"id":"81fd1d9a-a98b-4d03-8089-b844f39cd611","criterion":"schema-extraction-integrity","criterion_name":"Schema Extraction Integrity","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"Can still extract some static fields correctly on a dynamic product page, including the SEO header and price markers, but it corrupts the requested product-specific output by substituting broad site-directory text for the size data.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/81fd1d9a-a98b-4d03-8089-b844f39cd611"}],"appears_in":[{"page_type":"ranking","slug":"ai-web-scraping-tools","title":"Best AI Tools to Scrape Web Pages Into Clean Markdown or Structured Data","url":"https://aidemos.com/best/ai-web-scraping-tools","binding":"run"}],"same_scenario":[{"id":"e41889fa-4519-4c96-9183-067dad4dbeb5","tool":"firecrawl","tool_name":"Firecrawl","verdict":"worked","score":null,"score_total":null,"note":"Reliably waited for client-side hydration and captured the complete size set, from M 5 / W 6.5 through M 18 / W 19.5."},{"id":"94f367b2-9659-4081-bbe0-e887a50559e8","tool":"skyvern","tool_name":"Skyvern","verdict":"struggled","score":null,"score_total":null,"note":"Handles the page extraction itself, but the run’s live recording pipeline can fall out of sync during hydration: the report states the screen capture froze on the initial page view, making the recording unwatchable and hard to debug."}]}