{"observation":{"id":"cb41b326-14c7-4854-ae36-24b1588cda74","tool":"reducto","tool_name":"Reducto","criterion":"reading-order-structure","criterion_name":"Reading Order & Structure","criterion_definition":"Maintains headings, sections, column order, document hierarchy, and overall flow.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Clean Markdown requires the original reading order, headings, sections, and hierarchy to be maintained correctly. (3 of 3 judges)","scenario":"financial-report-table-heavy","scenario_name":"Financial Report - Table Heavy","group_tag":"financial-report-table-heavy","scenario_description":"A table-heavy corporate financial report used to test extraction of dense, hierarchical financial statements with grouped columns, multi-row headers, segment-reporting tables, and narrative disclosures.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://cdn.futuresmart.ai/public/aidemos/fba42ade5adb4814a3381dd4487a3910.pdf?v=1","role":"input","filename":"doc.pdf"}],"stresses":["Multi-page financial statement extraction","Hierarchical table reconstruction","Grouped columns and multi-row headers","Reading order in a report with mixed narrative and tables","Document structure retention","Markdown usability"],"verdict":"failed","score":null,"score_total":null,"note":"Provides zero markdown heading markup across all 18 pages; the Roman-numeral sections and numbered subsections are flattened into plain text despite the source’s visible section hierarchy.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/1b4327d7397e412589707a3e40e2ad25.png?v=1","role":"input","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/47b96197bd55404fbc3d1ca7280c2fc0.png?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/26a6176821d24574bf506ad8e85586e7.png?v=1","role":"input","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/e6e50071c0774c1fb1d9adef3e589dc0.png?v=1","role":"output","alt":null}],"run_id":"6e3160de-fe46-4b45-b071-72560b5c5d0e","study_title":"Convert a Complex PDF into Clean Markdown with an API","study_kind":"generation","research_task":"86b9h7t37","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://cdn.futuresmart.ai/public/aidemos/fba42ade5adb4814a3381dd4487a3910.pdf?v=1","filename":"doc.pdf","alt":"Financial Report - Table Heavy","role":"input"}],"modality":"pdf","stresses":["Multi-page financial statement extraction","Hierarchical table reconstruction","Grouped columns and multi-row headers","Reading order in a report with mixed narrative and tables","Document structure retention","Markdown usability"]},"tool_page_slug":"reducto","tool_url":"https://aidemos.com/tools/reducto","permalink":"https://aidemos.com/evidence/cb41b326-14c7-4854-ae36-24b1588cda74","api_url":"https://ai.aidemos.com/v1/observations/cb41b326-14c7-4854-ae36-24b1588cda74"},"peers":[{"id":"1d364c7d-b237-4aab-83e8-71f460eb2f3f","tool":"adobe-api","tool_name":"Adobe API","verdict":"failed","score":null,"score_total":null,"note":"Collapses a nested table-of-contents page into a flat list, so the indentation-based section hierarchy is no longer represented.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/b12d5cd772a647818ad1833789b090d6.pdf?v=1","evidence_url":"https://aidemos.com/evidence/1d364c7d-b237-4aab-83e8-71f460eb2f3f"},{"id":"bc4b6065-ad76-4715-a558-d76e3cfb1103","tool":"extend-ai","tool_name":"Extend AI","verdict":"worked","score":null,"score_total":null,"note":"Keeps section hierarchy and narrative flow intact in the table-heavy quarterly financial report, preserving headings and explanatory notes instead of collapsing them into one text block.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/1967e1c5f9944428a4dbc41122a50e81.png?v=1","evidence_url":"https://aidemos.com/evidence/bc4b6065-ad76-4715-a558-d76e3cfb1103"},{"id":"b9c70af6-5ade-4248-96e3-f6a02e23da6f","tool":"landing-ai","tool_name":"Landing AI","verdict":"failed","score":null,"score_total":null,"note":"Flattens major headings on the report title/abstract page, reducing structural clarity in the opening pages.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/1036a7a2e37f4e41a92554b56a4d59ee.png?v=1","evidence_url":"https://aidemos.com/evidence/b9c70af6-5ade-4248-96e3-f6a02e23da6f"},{"id":"06f3e900-c9d1-4767-bae7-efa3ac0e9779","tool":"llamaparse","tool_name":"LlamaParse","verdict":"worked","score":null,"score_total":null,"note":"Preserves the document title, disclaimer, and section headings in a readable hierarchy for the Sumitomo Heavy Industries financial report.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/3d0164220e144971802b39f1400c7d9c.png?v=1","evidence_url":"https://aidemos.com/evidence/06f3e900-c9d1-4767-bae7-efa3ac0e9779"},{"id":"16adbd4c-8e66-485d-9c8a-4c463d4aa966","tool":"mistral-ai","tool_name":"Mistral AI","verdict":"failed","score":null,"score_total":null,"note":"The table of contents is flattened into raw text, so the original navigational hierarchy is not preserved.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/b12d5cd772a647818ad1833789b090d6.pdf?v=1","evidence_url":"https://aidemos.com/evidence/16adbd4c-8e66-485d-9c8a-4c463d4aa966"},{"id":"f0d4e470-bcfd-41d7-858c-0f81bd6a8e82","tool":"nutrient-io","tool_name":"Nutrient.io","verdict":"struggled","score":null,"score_total":null,"note":"Does not consistently preserve paragraph boundaries, so narrative flow fragments across sections instead of staying in the source order.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/77abf45ba2654077814c47d1db1561d2.png?v=1","evidence_url":"https://aidemos.com/evidence/f0d4e470-bcfd-41d7-858c-0f81bd6a8e82"},{"id":"9bb6f16e-5cab-4c4b-86e7-adcc09d97654","tool":"tensorlake","tool_name":"Tensorlake","verdict":"worked","score":null,"score_total":null,"note":"Keeps section ordering and narrative flow in an 18-page table-heavy financial report, preserving the operating-performance section before the tabular disclosures.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/4a1b21a5855f4064b049eaf39823a417.png?v=1","evidence_url":"https://aidemos.com/evidence/9bb6f16e-5cab-4c4b-86e7-adcc09d97654"},{"id":"a3d4e38d-a868-41ef-97ba-4e95ab890a76","tool":"upstage-ai","tool_name":"Upstage AI","verdict":"failed","score":null,"score_total":null,"note":"Flattens the Summary of Operating Performance page so the section heading is no longer visually or structurally distinguished from the running text.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/91e20565ff53454384aa492ae0c1eaee.png?v=1","evidence_url":"https://aidemos.com/evidence/a3d4e38d-a868-41ef-97ba-4e95ab890a76"}],"other_criteria":[{"id":"2ca4b8f8-5317-4d78-bf58-1e1f88758a01","criterion":"advanced-features","criterion_name":"Advanced Features","rank_role":"context","verdict":"failed","score":null,"score_total":null,"note":"The confidence field is nearly useless here: the only low-confidence flag is a false alarm on the correct page-18 section header, while the genuinely broken Business Results table stays high-confidence throughout.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/2ca4b8f8-5317-4d78-bf58-1e1f88758a01"},{"id":"991345dd-5c2a-4f09-8dc3-76918b2843d9","criterion":"advanced-features","criterion_name":"Advanced Features","rank_role":"context","verdict":"worked","score":null,"score_total":null,"note":"Schema-driven extract.run fully recovers the broken Business Results table, separating the 2024 value and percent-change fields that parse.run had merged together.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/991345dd-5c2a-4f09-8dc3-76918b2843d9"},{"id":"bfda059c-a8f9-41ce-8796-bcc01ea2026d","criterion":"complex-document-handling","criterion_name":"Complex Document Handling","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"Processes the 18-page report in 9.7 seconds with no truncation; the 9-column segment table at page 17 is flawless even though the first table on page 2 is broken.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/bfda059c-a8f9-41ce-8796-bcc01ea2026d"},{"id":"aa04c457-96be-40c1-87a5-5813f3f86c31","criterion":"markdown-quality","criterion_name":"Markdown Quality","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Keeps the syntax clean: no <signature>, <empty>, <b>, <i>, or <u> tags appear anywhere in the output, and the table-of-contents extract renders as a valid pipe table.","artifact_count":4,"evidence_url":"https://aidemos.com/evidence/aa04c457-96be-40c1-87a5-5813f3f86c31"},{"id":"f597cbbc-cbe9-4694-8b5e-078b1489986b","criterion":"table-preservation","criterion_name":"Table Preservation","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"Breaks a 5-column business-results table by duplicating the 2025 header and merging the 2024 value and percent change into one cell; the Net sales row ends up with 254,811 and 2.6 smashed together.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/f597cbbc-cbe9-4694-8b5e-078b1489986b"},{"id":"8c8cffd6-9942-4cc3-95c3-b837dfd045e3","criterion":"table-preservation","criterion_name":"Table Preservation","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Preserves the balance-sheet hierarchy and values, including Assets > Current assets > Cash and deposits, and also reconstructs the later 9-column segment table with all values correct.","artifact_count":4,"evidence_url":"https://aidemos.com/evidence/8c8cffd6-9942-4cc3-95c3-b837dfd045e3"},{"id":"2d1ae485-b6cc-4e88-813c-fce6d08fdd87","criterion":"text-ocr-completeness","criterion_name":"Text & OCR Completeness","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Converts all 18 pages with no omitted text; a dense financial-condition paragraph preserves multiple JPY-billion values exactly, including JPY1,252.7 billion, JPY623.6 billion, JPY27.2 billion, JPY10.8 billion, JPY5.0 billion, and JPY9.7 billion.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/2d1ae485-b6cc-4e88-813c-fce6d08fdd87"},{"id":"cd90d364-bb6e-4ca0-9248-fe516d1a8765","criterion":"visual-content-retention","criterion_name":"Visual Content Retention","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Retains the recurring page-header logo consistently across all 17 pages that carry it; the returned crops on pages 2 and 18 are the same mark and wordmark, with only a 1px width difference (453x36 vs 452x36) consistent with rounding.","artifact_count":4,"evidence_url":"https://aidemos.com/evidence/cd90d364-bb6e-4ca0-9248-fe516d1a8765"}],"appears_in":[{"page_type":"ranking","slug":"pdf-to-markdown-apis","title":"Best AI Tools to Convert Complex PDFs into Clean Markdown with an API","url":"https://aidemos.com/best/pdf-to-markdown-apis","binding":"run"}],"same_scenario":[{"id":"1d364c7d-b237-4aab-83e8-71f460eb2f3f","tool":"adobe-api","tool_name":"Adobe API","verdict":"failed","score":null,"score_total":null,"note":"Collapses a nested table-of-contents page into a flat list, so the indentation-based section hierarchy is no longer represented."},{"id":"bc4b6065-ad76-4715-a558-d76e3cfb1103","tool":"extend-ai","tool_name":"Extend AI","verdict":"worked","score":null,"score_total":null,"note":"Keeps section hierarchy and narrative flow intact in the table-heavy quarterly financial report, preserving headings and explanatory notes instead of collapsing them into one text block."},{"id":"b9c70af6-5ade-4248-96e3-f6a02e23da6f","tool":"landing-ai","tool_name":"Landing AI","verdict":"failed","score":null,"score_total":null,"note":"Flattens major headings on the report title/abstract page, reducing structural clarity in the opening pages."},{"id":"06f3e900-c9d1-4767-bae7-efa3ac0e9779","tool":"llamaparse","tool_name":"LlamaParse","verdict":"worked","score":null,"score_total":null,"note":"Preserves the document title, disclaimer, and section headings in a readable hierarchy for the Sumitomo Heavy Industries financial report."},{"id":"16adbd4c-8e66-485d-9c8a-4c463d4aa966","tool":"mistral-ai","tool_name":"Mistral AI","verdict":"failed","score":null,"score_total":null,"note":"The table of contents is flattened into raw text, so the original navigational hierarchy is not preserved."},{"id":"f0d4e470-bcfd-41d7-858c-0f81bd6a8e82","tool":"nutrient-io","tool_name":"Nutrient.io","verdict":"struggled","score":null,"score_total":null,"note":"Does not consistently preserve paragraph boundaries, so narrative flow fragments across sections instead of staying in the source order."},{"id":"9bb6f16e-5cab-4c4b-86e7-adcc09d97654","tool":"tensorlake","tool_name":"Tensorlake","verdict":"worked","score":null,"score_total":null,"note":"Keeps section ordering and narrative flow in an 18-page table-heavy financial report, preserving the operating-performance section before the tabular disclosures."},{"id":"a3d4e38d-a868-41ef-97ba-4e95ab890a76","tool":"upstage-ai","tool_name":"Upstage AI","verdict":"failed","score":null,"score_total":null,"note":"Flattens the Summary of Operating Performance page so the section heading is no longer visually or structurally distinguished from the running text."}]}