{"observation":{"id":"55dd439d-f92d-466b-8207-4755b499332d","tool":"skyvern","tool_name":"Skyvern","criterion":"interaction-stability","criterion_name":"Interaction Stability","criterion_definition":"How reliably the tool executes through dynamic loading and interaction steps without losing sync or failing at runtime.","criterion_evidence_type":"capability","criterion_rank_role":"decisive","criterion_rank_role_reason":"Reliably completing dynamic loading and interaction steps is part of successfully scraping pages, not just convenience. (3 of 3 judges)","scenario":"nike-air-force-1-07-size-options-extraction","scenario_name":"Nike Air Force 1 '07 size options extraction","group_tag":null,"scenario_description":"A Nike product page with client-side JavaScript hydration used to test whether a headless scraper waits for dynamic DOM content before extracting product details and all available shoe sizes.","modality":"mixed","input_text":"https://www.nike.com/t/air-force-1-07-mens-shoes-jBrhbr/CW2288-111 — Wait for the size selection options to fully render. Extract the product name, price, and a list of all available shoe sizes.","input_artifact_refs":[],"stresses":[],"verdict":"mixed","score":null,"score_total":null,"note":"The extraction completed, but the screen-capture recorder fell out of sync and froze on an early page state, so runtime observability degraded.","evidence_state":"observed","source":"first-party","artifacts":[],"run_id":"06e1dbd6-5518-4af8-aa1a-735259a75b4f","study_title":"Scrape Web Pages Into Clean Markdown or Structured Data Using AI","study_kind":"backfill","research_task":"86b9jm3a3","tested_at":"2026-06-23T04:32:57.304000+00:00","completeness":"input-only","input":{"state":"text","text":"https://www.nike.com/t/air-force-1-07-mens-shoes-jBrhbr/CW2288-111 — Wait for the size selection options to fully render. Extract the product name, price, and a list of all available shoe sizes.","files":[],"modality":"mixed","stresses":[]},"tool_page_slug":"skyvern","tool_url":"https://aidemos.com/tools/skyvern","permalink":"https://aidemos.com/evidence/55dd439d-f92d-466b-8207-4755b499332d","api_url":"https://ai.aidemos.com/v1/observations/55dd439d-f92d-466b-8207-4755b499332d"},"peers":[{"id":"e41889fa-4519-4c96-9183-067dad4dbeb5","tool":"firecrawl","tool_name":"Firecrawl","verdict":"worked","score":null,"score_total":null,"note":"Reliably waited for client-side hydration and captured the complete size set, from M 5 / W 6.5 through M 18 / W 19.5.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/firecrawl-firecrawl-nike-scrape-markdown-output.png","evidence_url":"https://aidemos.com/evidence/e41889fa-4519-4c96-9183-067dad4dbeb5"},{"id":"813f348c-4de5-462c-b815-5c06a67402cc","tool":"jina-ai-reader","tool_name":"Jina AI Reader","verdict":"failed","score":null,"score_total":null,"note":"It did not wait for the client-side size selector to hydrate; the response shows the product title and $115 price but no concrete size inventory values.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/jina-ai-reader-jina-reader-nike-product-extraction.png","evidence_url":"https://aidemos.com/evidence/813f348c-4de5-462c-b815-5c06a67402cc"}],"other_criteria":[{"id":"d49869b0-9f36-4e26-b289-a3f2ab804939","criterion":"js-dom-hydration","criterion_name":"JS DOM Hydration","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The tool can wait for client-side hydration and capture the rendered product state, including a populated size-selection grid; in this run it extracted a structured schema of the Nike Air Force 1 '07 size options and showed multiple size variants rather than an empty shell.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/d49869b0-9f36-4e26-b289-a3f2ab804939"},{"id":"aa8c092a-63a7-467f-ae2c-636e6034ac55","criterion":"js-dom-hydration","criterion_name":"JS DOM Hydration","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"The extraction pipeline recovered the hydrated Nike size data, but the screen-recording/visual trace subsystem was out of sync and froze on an initial page view, so the captured recording did not reflect the final dynamic state.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/aa8c092a-63a7-467f-ae2c-636e6034ac55"},{"id":"ffd61b21-8c43-4795-bacb-ffa1d87744aa","criterion":"js-dom-hydration","criterion_name":"JS DOM Hydration","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Waits for the client-rendered product page to hydrate and extracts the size grid instead of stopping at the initial shell.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/ffd61b21-8c43-4795-bacb-ffa1d87744aa"},{"id":"bb5b1f48-5766-4e41-b40f-989875cac6cc","criterion":"output-quality","criterion_name":"Output Quality","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Returns accurate product data for the hydrated page, including the correct product name Nike Air Force 1 '07 and a long size list; the visible output shows 14 fully readable size pairs from M 7 / W 8.5 through M 14 / W 15.5.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/bb5b1f48-5766-4e41-b40f-989875cac6cc"},{"id":"ce4fbd1b-c3b9-4ce0-a6a0-fe0bf600238f","criterion":"schema-extraction-integrity","criterion_name":"Schema Extraction Integrity","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Can accurately preserve a hydrated page’s structured output, extracting a complete schema of all 22 shoe sizes without corrupting the requested JSON structure.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/ce4fbd1b-c3b9-4ce0-a6a0-fe0bf600238f"},{"id":"baefb4cb-3fde-43e0-8f39-e0556ffa208d","criterion":"schema-extraction-integrity","criterion_name":"Schema Extraction Integrity","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Accurately extracts hydrated product-size data into structured output; the visible result enumerates multiple size pairs, including M 7 / W 8.5, M 7.5 / W 9, M 8 / W 9.5, and through M 15 / W 16.5 in the shown excerpt.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/baefb4cb-3fde-43e0-8f39-e0556ffa208d"},{"id":"b903a087-1027-4526-92f2-daf9f93cd3b0","criterion":"schema-extraction-integrity","criterion_name":"Schema Extraction Integrity","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Outputs a structured size list with many men/women pairs, with the visible payload spanning at least M 6 / W 7.5 through M 14 / W 15.5.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/b903a087-1027-4526-92f2-daf9f93cd3b0"}],"appears_in":[{"page_type":"ranking","slug":"ai-web-scraping-tools","title":"Best AI Tools to Scrape Web Pages Into Clean Markdown or Structured Data","url":"https://aidemos.com/best/ai-web-scraping-tools","binding":"run"}],"same_scenario":[{"id":"e41889fa-4519-4c96-9183-067dad4dbeb5","tool":"firecrawl","tool_name":"Firecrawl","verdict":"worked","score":null,"score_total":null,"note":"Reliably waited for client-side hydration and captured the complete size set, from M 5 / W 6.5 through M 18 / W 19.5."},{"id":"813f348c-4de5-462c-b815-5c06a67402cc","tool":"jina-ai-reader","tool_name":"Jina AI Reader","verdict":"failed","score":null,"score_total":null,"note":"It did not wait for the client-side size selector to hydrate; the response shows the product title and $115 price but no concrete size inventory values."}]}