{"observation":{"id":"aa8c092a-63a7-467f-ae2c-636e6034ac55","tool":"skyvern","tool_name":"Skyvern","criterion":"js-dom-hydration","criterion_name":"JS DOM Hydration","criterion_definition":"How well the tool waits for and captures content rendered by client-side JavaScript on dynamic pages.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Capturing JavaScript-rendered content is a core scraping capability for modern web pages. (3 of 3 judges)","scenario":"nike-air-force-1-07-size-options-extraction","scenario_name":"Nike Air Force 1 '07 size options extraction","group_tag":null,"scenario_description":"A Nike product page with client-side JavaScript hydration used to test whether a headless scraper waits for dynamic DOM content before extracting product details and all available shoe sizes.","modality":"mixed","input_text":"https://www.nike.com/t/air-force-1-07-mens-shoes-jBrhbr/CW2288-111 — Wait for the size selection options to fully render. Extract the product name, price, and a list of all available shoe sizes.","input_artifact_refs":[],"stresses":[],"verdict":"mixed","score":null,"score_total":null,"note":"The extraction pipeline recovered the hydrated Nike size data, but the screen-recording/visual trace subsystem was out of sync and froze on an initial page view, so the captured recording did not reflect the final dynamic state.","evidence_state":"observed","source":"first-party","artifacts":[],"run_id":"06e1dbd6-5518-4af8-aa1a-735259a75b4f","study_title":"Scrape Web Pages Into Clean Markdown or Structured Data Using AI","study_kind":"backfill","research_task":"86b9jm3a3","tested_at":"2026-06-23T04:32:57.304000+00:00","completeness":"input-only","input":{"state":"text","text":"https://www.nike.com/t/air-force-1-07-mens-shoes-jBrhbr/CW2288-111 — Wait for the size selection options to fully render. Extract the product name, price, and a list of all available shoe sizes.","files":[],"modality":"mixed","stresses":[]},"tool_page_slug":"skyvern","tool_url":"https://aidemos.com/tools/skyvern","permalink":"https://aidemos.com/evidence/aa8c092a-63a7-467f-ae2c-636e6034ac55","api_url":"https://ai.aidemos.com/v1/observations/aa8c092a-63a7-467f-ae2c-636e6034ac55"},"peers":[{"id":"539003d1-a873-4167-8a3e-5427f4a36e74","tool":"firecrawl","tool_name":"Firecrawl","verdict":"mixed","score":null,"score_total":null,"note":"Captures hydrated product content but leaves substantial non-content noise in the output, including raw backend code artifacts and raw media-attachment matrices.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/firecrawl-firecrawl-nike-scrape-markdown-output.png","evidence_url":"https://aidemos.com/evidence/539003d1-a873-4167-8a3e-5427f4a36e74"},{"id":"d6a37898-46aa-424c-8ed2-cacb300ba94b","tool":"jina-ai-reader","tool_name":"Jina AI Reader","verdict":"failed","score":null,"score_total":null,"note":"It did not capture the client-rendered size selector; after 7.0 s the extract still lacked the live size grid and showed only static page content.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/jina-ai-reader-jina-reader-nike-product-extraction.png","evidence_url":"https://aidemos.com/evidence/d6a37898-46aa-424c-8ed2-cacb300ba94b"},{"id":"9158d75f-e4b0-45ac-8a1f-7bfc1e64e2d2","tool":"spider","tool_name":"Spider","verdict":"worked","score":null,"score_total":null,"note":"It captured the rendered product headline and basic metadata, including the Nike Air Force 1 '07 title, the Men's Shoes label, and the $115 price.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/spider-spider-playground-nike-air-force-scrape.png","evidence_url":"https://aidemos.com/evidence/9158d75f-e4b0-45ac-8a1f-7bfc1e64e2d2"}],"other_criteria":[{"id":"94f367b2-9659-4081-bbe0-e887a50559e8","criterion":"interaction-stability","criterion_name":"Interaction Stability","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"Handles the page extraction itself, but the run’s live recording pipeline can fall out of sync during hydration: the report states the screen capture froze on the initial page view, making the recording unwatchable and hard to debug.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/94f367b2-9659-4081-bbe0-e887a50559e8"},{"id":"55dd439d-f92d-466b-8207-4755b499332d","criterion":"interaction-stability","criterion_name":"Interaction Stability","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"The extraction completed, but the screen-capture recorder fell out of sync and froze on an early page state, so runtime observability degraded.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/55dd439d-f92d-466b-8207-4755b499332d"},{"id":"0aa94cc5-85f5-48c0-b3c8-b2d350d5f1fb","criterion":"interaction-stability","criterion_name":"Interaction Stability","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"The live recording pipeline went out of sync during hydration and froze on the initial page view even though backend extraction still completed, showing unreliable handling of dynamic UI state changes.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/0aa94cc5-85f5-48c0-b3c8-b2d350d5f1fb"},{"id":"bb5b1f48-5766-4e41-b40f-989875cac6cc","criterion":"output-quality","criterion_name":"Output Quality","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Returns accurate product data for the hydrated page, including the correct product name Nike Air Force 1 '07 and a long size list; the visible output shows 14 fully readable size pairs from M 7 / W 8.5 through M 14 / W 15.5.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/bb5b1f48-5766-4e41-b40f-989875cac6cc"},{"id":"ce4fbd1b-c3b9-4ce0-a6a0-fe0bf600238f","criterion":"schema-extraction-integrity","criterion_name":"Schema Extraction Integrity","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Can accurately preserve a hydrated page’s structured output, extracting a complete schema of all 22 shoe sizes without corrupting the requested JSON structure.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/ce4fbd1b-c3b9-4ce0-a6a0-fe0bf600238f"},{"id":"baefb4cb-3fde-43e0-8f39-e0556ffa208d","criterion":"schema-extraction-integrity","criterion_name":"Schema Extraction Integrity","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Accurately extracts hydrated product-size data into structured output; the visible result enumerates multiple size pairs, including M 7 / W 8.5, M 7.5 / W 9, M 8 / W 9.5, and through M 15 / W 16.5 in the shown excerpt.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/baefb4cb-3fde-43e0-8f39-e0556ffa208d"},{"id":"b903a087-1027-4526-92f2-daf9f93cd3b0","criterion":"schema-extraction-integrity","criterion_name":"Schema Extraction Integrity","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Outputs a structured size list with many men/women pairs, with the visible payload spanning at least M 6 / W 7.5 through M 14 / W 15.5.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/b903a087-1027-4526-92f2-daf9f93cd3b0"}],"appears_in":[{"page_type":"ranking","slug":"ai-web-scraping-tools","title":"Best AI Tools to Scrape Web Pages Into Clean Markdown or Structured Data","url":"https://aidemos.com/best/ai-web-scraping-tools","binding":"run"}],"same_scenario":[{"id":"539003d1-a873-4167-8a3e-5427f4a36e74","tool":"firecrawl","tool_name":"Firecrawl","verdict":"mixed","score":null,"score_total":null,"note":"Captures hydrated product content but leaves substantial non-content noise in the output, including raw backend code artifacts and raw media-attachment matrices."},{"id":"d6a37898-46aa-424c-8ed2-cacb300ba94b","tool":"jina-ai-reader","tool_name":"Jina AI Reader","verdict":"failed","score":null,"score_total":null,"note":"It did not capture the client-rendered size selector; after 7.0 s the extract still lacked the live size grid and showed only static page content."},{"id":"9158d75f-e4b0-45ac-8a1f-7bfc1e64e2d2","tool":"spider","tool_name":"Spider","verdict":"worked","score":null,"score_total":null,"note":"It captured the rendered product headline and basic metadata, including the Nike Air Force 1 '07 title, the Men's Shoes label, and the $115 price."}]}