{"observation":{"id":"06f3e900-c9d1-4767-bae7-efa3ac0e9779","tool":"llamaparse","tool_name":"LlamaParse","criterion":"reading-order-structure","criterion_name":"Reading Order & Structure","criterion_definition":"Maintains headings, sections, column order, document hierarchy, and overall flow.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Clean Markdown requires the original reading order, headings, sections, and hierarchy to be maintained correctly. (3 of 3 judges)","scenario":"financial-report-table-heavy","scenario_name":"Financial Report - Table Heavy","group_tag":"financial-report-table-heavy","scenario_description":"A table-heavy corporate financial report used to test extraction of dense, hierarchical financial statements with grouped columns, multi-row headers, segment-reporting tables, and narrative disclosures.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://cdn.futuresmart.ai/public/aidemos/fba42ade5adb4814a3381dd4487a3910.pdf?v=1","role":"input","filename":"doc.pdf"}],"stresses":["Multi-page financial statement extraction","Hierarchical table reconstruction","Grouped columns and multi-row headers","Reading order in a report with mixed narrative and tables","Document structure retention","Markdown usability"],"verdict":"worked","score":null,"score_total":null,"note":"Preserves the document title, disclaimer, and section headings in a readable hierarchy for the Sumitomo Heavy Industries financial report.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/3d0164220e144971802b39f1400c7d9c.png?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/24646a35481f4bc9906fafbfc62a27e6.png?v=1","role":"input","alt":null}],"run_id":"6e3160de-fe46-4b45-b071-72560b5c5d0e","study_title":"Convert a Complex PDF into Clean Markdown with an API","study_kind":"generation","research_task":"86b9h7t37","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://cdn.futuresmart.ai/public/aidemos/fba42ade5adb4814a3381dd4487a3910.pdf?v=1","filename":"doc.pdf","alt":"Financial Report - Table Heavy","role":"input"}],"modality":"pdf","stresses":["Multi-page financial statement extraction","Hierarchical table reconstruction","Grouped columns and multi-row headers","Reading order in a report with mixed narrative and tables","Document structure retention","Markdown usability"]},"tool_page_slug":"llamaparse","tool_url":"https://aidemos.com/tools/llamaparse","permalink":"https://aidemos.com/evidence/06f3e900-c9d1-4767-bae7-efa3ac0e9779","api_url":"https://ai.aidemos.com/v1/observations/06f3e900-c9d1-4767-bae7-efa3ac0e9779"},"peers":[{"id":"1d364c7d-b237-4aab-83e8-71f460eb2f3f","tool":"adobe-api","tool_name":"Adobe API","verdict":"failed","score":null,"score_total":null,"note":"Collapses a nested table-of-contents page into a flat list, so the indentation-based section hierarchy is no longer represented.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/b12d5cd772a647818ad1833789b090d6.pdf?v=1","evidence_url":"https://aidemos.com/evidence/1d364c7d-b237-4aab-83e8-71f460eb2f3f"},{"id":"bc4b6065-ad76-4715-a558-d76e3cfb1103","tool":"extend-ai","tool_name":"Extend AI","verdict":"worked","score":null,"score_total":null,"note":"Keeps section hierarchy and narrative flow intact in the table-heavy quarterly financial report, preserving headings and explanatory notes instead of collapsing them into one text block.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/1967e1c5f9944428a4dbc41122a50e81.png?v=1","evidence_url":"https://aidemos.com/evidence/bc4b6065-ad76-4715-a558-d76e3cfb1103"},{"id":"4ef15ab4-2775-4a1c-8619-d2b570d9cbd5","tool":"landing-ai","tool_name":"Landing AI","verdict":"worked","score":null,"score_total":null,"note":"Keeps the operating-performance section hierarchy intact, with the main section heading, subsection heading, and narrative paragraphs remaining in the correct flow.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/0cbc700309f74252b0e25174522df747.png?v=1","evidence_url":"https://aidemos.com/evidence/4ef15ab4-2775-4a1c-8619-d2b570d9cbd5"},{"id":"1b28b2b5-cfd9-48db-82e9-11ad35d536f4","tool":"mistral-ai","tool_name":"Mistral AI","verdict":"worked","score":null,"score_total":null,"note":"Document hierarchy and reading flow are preserved across the financial report, with headings staying connected to their explanatory text in the reconstructed output.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/38267592d83e49d4bbc2cee8d74263e5.png?v=1","evidence_url":"https://aidemos.com/evidence/1b28b2b5-cfd9-48db-82e9-11ad35d536f4"},{"id":"6d402ad0-3be9-4daf-ba1b-50420c603ead","tool":"nutrient-io","tool_name":"Nutrient.io","verdict":"worked","score":null,"score_total":null,"note":"The tool preserves a long narrative operating-performance section with headings and paragraph flow in a quarterly financial report page.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/b6aeec75b8d843cd92c330c8ed7e32f7.png?v=1","evidence_url":"https://aidemos.com/evidence/6d402ad0-3be9-4daf-ba1b-50420c603ead"},{"id":"cb41b326-14c7-4854-ae36-24b1588cda74","tool":"reducto","tool_name":"Reducto","verdict":"failed","score":null,"score_total":null,"note":"Provides zero markdown heading markup across all 18 pages; the Roman-numeral sections and numbered subsections are flattened into plain text despite the source’s visible section hierarchy.","artifact_count":4,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/1b4327d7397e412589707a3e40e2ad25.png?v=1","evidence_url":"https://aidemos.com/evidence/cb41b326-14c7-4854-ae36-24b1588cda74"},{"id":"9bb6f16e-5cab-4c4b-86e7-adcc09d97654","tool":"tensorlake","tool_name":"Tensorlake","verdict":"worked","score":null,"score_total":null,"note":"Keeps section ordering and narrative flow in an 18-page table-heavy financial report, preserving the operating-performance section before the tabular disclosures.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/4a1b21a5855f4064b049eaf39823a417.png?v=1","evidence_url":"https://aidemos.com/evidence/9bb6f16e-5cab-4c4b-86e7-adcc09d97654"},{"id":"e412a7d6-0046-498a-89c9-37dec3a9b8fa","tool":"upstage-ai","tool_name":"Upstage AI","verdict":"worked","score":null,"score_total":null,"note":"Preserves section hierarchy in the Mechatronics/Industrial Machinery prose, keeping subsection headings and their paragraphs in the same structural order as the source.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/89eb31208784451686036318d2662105.png?v=1","evidence_url":"https://aidemos.com/evidence/e412a7d6-0046-498a-89c9-37dec3a9b8fa"}],"other_criteria":[{"id":"c2b3fb59-2c32-4edb-a21b-125cb364f2ea","criterion":"complex-document-handling","criterion_name":"Complex Document Handling","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Processes an 18-page table-heavy report end-to-end and reaches SUCCESS in the Results view without manual correction.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/c2b3fb59-2c32-4edb-a21b-125cb364f2ea"},{"id":"91c495ea-e42f-469b-8dbf-70720d32cb86","criterion":"markdown-quality","criterion_name":"Markdown Quality","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"Renders the table of contents as sequential text with page numbers instead of a nested TOC structure, so the markdown is usable but flattened.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/91c495ea-e42f-469b-8dbf-70720d32cb86"},{"id":"b83ea780-c133-4101-8a90-10799b20ab97","criterion":"table-preservation","criterion_name":"Table Preservation","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Preserves a multi-level segment-results table with grouped headers and year-over-year columns, keeping the hierarchy legible in the extracted output.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/b83ea780-c133-4101-8a90-10799b20ab97"},{"id":"f41bfe47-b4b5-4125-aabb-b2f1fe86c857","criterion":"table-preservation","criterion_name":"Table Preservation","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"Keeps the numeric values in a dense hierarchical table but makes grouped-header semantics less explicit, so parent-child column roles become less clear.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/f41bfe47-b4b5-4125-aabb-b2f1fe86c857"}],"appears_in":[{"page_type":"ranking","slug":"pdf-to-markdown-apis","title":"Best AI Tools to Convert Complex PDFs into Clean Markdown with an API","url":"https://aidemos.com/best/pdf-to-markdown-apis","binding":"run"}],"same_scenario":[{"id":"1d364c7d-b237-4aab-83e8-71f460eb2f3f","tool":"adobe-api","tool_name":"Adobe API","verdict":"failed","score":null,"score_total":null,"note":"Collapses a nested table-of-contents page into a flat list, so the indentation-based section hierarchy is no longer represented."},{"id":"bc4b6065-ad76-4715-a558-d76e3cfb1103","tool":"extend-ai","tool_name":"Extend AI","verdict":"worked","score":null,"score_total":null,"note":"Keeps section hierarchy and narrative flow intact in the table-heavy quarterly financial report, preserving headings and explanatory notes instead of collapsing them into one text block."},{"id":"4ef15ab4-2775-4a1c-8619-d2b570d9cbd5","tool":"landing-ai","tool_name":"Landing AI","verdict":"worked","score":null,"score_total":null,"note":"Keeps the operating-performance section hierarchy intact, with the main section heading, subsection heading, and narrative paragraphs remaining in the correct flow."},{"id":"1b28b2b5-cfd9-48db-82e9-11ad35d536f4","tool":"mistral-ai","tool_name":"Mistral AI","verdict":"worked","score":null,"score_total":null,"note":"Document hierarchy and reading flow are preserved across the financial report, with headings staying connected to their explanatory text in the reconstructed output."},{"id":"6d402ad0-3be9-4daf-ba1b-50420c603ead","tool":"nutrient-io","tool_name":"Nutrient.io","verdict":"worked","score":null,"score_total":null,"note":"The tool preserves a long narrative operating-performance section with headings and paragraph flow in a quarterly financial report page."},{"id":"cb41b326-14c7-4854-ae36-24b1588cda74","tool":"reducto","tool_name":"Reducto","verdict":"failed","score":null,"score_total":null,"note":"Provides zero markdown heading markup across all 18 pages; the Roman-numeral sections and numbered subsections are flattened into plain text despite the source’s visible section hierarchy."},{"id":"9bb6f16e-5cab-4c4b-86e7-adcc09d97654","tool":"tensorlake","tool_name":"Tensorlake","verdict":"worked","score":null,"score_total":null,"note":"Keeps section ordering and narrative flow in an 18-page table-heavy financial report, preserving the operating-performance section before the tabular disclosures."},{"id":"e412a7d6-0046-498a-89c9-37dec3a9b8fa","tool":"upstage-ai","tool_name":"Upstage AI","verdict":"worked","score":null,"score_total":null,"note":"Preserves section hierarchy in the Mechatronics/Industrial Machinery prose, keeping subsection headings and their paragraphs in the same structural order as the source."}]}