{"observation":{"id":"a86ffd95-175a-40ec-8670-6791368379df","tool":"nutrient-io","tool_name":"Nutrient.io","criterion":"table-preservation","criterion_name":"Table Preservation","criterion_definition":"Preserves complex table structures, including rows, columns, multi-row headers, and merged-cell relationships in markdown.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Accurate Markdown conversion of complex PDFs depends on keeping table structure intact, not flattening it into plain text. (3 of 3 judges)","scenario":"financial-report-table-heavy","scenario_name":"Financial Report - Table Heavy","group_tag":"financial-report-table-heavy","scenario_description":"A table-heavy corporate financial report used to test extraction of dense, hierarchical financial statements with grouped columns, multi-row headers, segment-reporting tables, and narrative disclosures.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://cdn.futuresmart.ai/public/aidemos/fba42ade5adb4814a3381dd4487a3910.pdf?v=1","role":"input","filename":"doc.pdf"}],"stresses":["Multi-page financial statement extraction","Hierarchical table reconstruction","Grouped columns and multi-row headers","Reading order in a report with mixed narrative and tables","Document structure retention","Markdown usability"],"verdict":"failed","score":null,"score_total":null,"note":"Loses multi-level header organization in hierarchical tables, so parent-child column relationships are not reconstructed in the markdown output.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/33be110e75ed4e7ca9c753431c9ecf0c.png?v=1","role":"input","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/76638d7f318d4adf91ca43b9ffbd56c1.png?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/700be13d678e4a0a8e9cfb35615ef7ee.png?v=1","role":"input","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/5d5fd8b9be734967aaf4a59901473fc5.png?v=1","role":"output","alt":null}],"run_id":"6e3160de-fe46-4b45-b071-72560b5c5d0e","study_title":"Convert a Complex PDF into Clean Markdown with an API","study_kind":"generation","research_task":"86b9h7t37","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://cdn.futuresmart.ai/public/aidemos/fba42ade5adb4814a3381dd4487a3910.pdf?v=1","filename":"doc.pdf","alt":"Financial Report - Table Heavy","role":"input"}],"modality":"pdf","stresses":["Multi-page financial statement extraction","Hierarchical table reconstruction","Grouped columns and multi-row headers","Reading order in a report with mixed narrative and tables","Document structure retention","Markdown usability"]},"tool_page_slug":"nutrient-io","tool_url":"https://aidemos.com/tools/nutrient-io","permalink":"https://aidemos.com/evidence/a86ffd95-175a-40ec-8670-6791368379df","api_url":"https://ai.aidemos.com/v1/observations/a86ffd95-175a-40ec-8670-6791368379df"},"peers":[{"id":"0ca22480-01a9-4e56-9b68-305585f6dd8c","tool":"adobe-api","tool_name":"Adobe API","verdict":"worked","score":null,"score_total":null,"note":"Preserves a grouped multi-level segment table with previous/current quarter amounts and year-over-year change columns, keeping the segment rows and totals aligned.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/7e4a0f994215484c920ba868f35fcef4.png?v=1","evidence_url":"https://aidemos.com/evidence/0ca22480-01a9-4e56-9b68-305585f6dd8c"},{"id":"97367dd9-249f-4997-95e4-14cc616e9713","tool":"extend-ai","tool_name":"Extend AI","verdict":"worked","score":null,"score_total":null,"note":"Preserves grouped-column structure in the financial tables, including the multilevel segment matrix and the balance-sheet comparison layout.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/7e4a0f994215484c920ba868f35fcef4.png?v=1","evidence_url":"https://aidemos.com/evidence/97367dd9-249f-4997-95e4-14cc616e9713"},{"id":"3e92e668-2248-45fe-bbe3-3bbe48017ea9","tool":"landing-ai","tool_name":"Landing AI","verdict":"worked","score":null,"score_total":null,"note":"Reconstructs a multi-column segment comparison table with previous-year, present-year, and year-over-year change columns aligned across the rows.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/7e4a0f994215484c920ba868f35fcef4.png?v=1","evidence_url":"https://aidemos.com/evidence/3e92e668-2248-45fe-bbe3-3bbe48017ea9"},{"id":"f41bfe47-b4b5-4125-aabb-b2f1fe86c857","tool":"llamaparse","tool_name":"LlamaParse","verdict":"mixed","score":null,"score_total":null,"note":"Keeps the numeric values in a dense hierarchical table but makes grouped-header semantics less explicit, so parent-child column roles become less clear.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/6d49f71dfdfa4f49bac8b828bbd2ef93.png?v=1","evidence_url":"https://aidemos.com/evidence/f41bfe47-b4b5-4125-aabb-b2f1fe86c857"},{"id":"5fe802fc-62a5-4a6a-aa9c-9ae77852cde1","tool":"mistral-ai","tool_name":"Mistral AI","verdict":"failed","score":null,"score_total":null,"note":"Distinct header levels are collapsed into a single cell in the multilevel table, weakening the parent-child relationships that define the column semantics.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/79983e2875fa4de3adacd9d60f5e9dd0.png?v=1","evidence_url":"https://aidemos.com/evidence/5fe802fc-62a5-4a6a-aa9c-9ae77852cde1"},{"id":"f597cbbc-cbe9-4694-8b5e-078b1489986b","tool":"reducto","tool_name":"Reducto","verdict":"failed","score":null,"score_total":null,"note":"Breaks a 5-column business-results table by duplicating the 2025 header and merging the 2024 value and percent change into one cell; the Net sales row ends up with 254,811 and 2.6 smashed together.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/af3dd85d1d40412d8e591a94c1db85f0.png?v=1","evidence_url":"https://aidemos.com/evidence/f597cbbc-cbe9-4694-8b5e-078b1489986b"},{"id":"63381a32-714a-48c6-b8dc-3ce4fc67279d","tool":"tensorlake","tool_name":"Tensorlake","verdict":"failed","score":null,"score_total":null,"note":"Breaks on a more complex multi-header table by losing header hierarchy and omitting at least one header label, producing an incomplete and structurally incorrect reconstruction.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/bf75efe477c745c68266f27b04959671.png?v=1","evidence_url":"https://aidemos.com/evidence/63381a32-714a-48c6-b8dc-3ce4fc67279d"},{"id":"c66e2b14-a549-4ce7-bbd9-4f0fb50a2057","tool":"upstage-ai","tool_name":"Upstage AI","verdict":"failed","score":null,"score_total":null,"note":"Misaligns column headers with data regions in dense financial tables, producing a structurally inconsistent balance-sheet extraction rather than a clean header-to-row mapping.","artifact_count":4,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/06a0e21888cc4bb187dec34ceb255186.png?v=1","evidence_url":"https://aidemos.com/evidence/c66e2b14-a549-4ce7-bbd9-4f0fb50a2057"}],"other_criteria":[{"id":"e6305cde-5b3c-4eda-bdc9-83b05fbcfb79","criterion":"complex-document-handling","criterion_name":"Complex Document Handling","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Handles an 18-page table-heavy financial report and returns a parsed markdown output plus preview, showing end-to-end extraction on a dense corporate document.","artifact_count":6,"evidence_url":"https://aidemos.com/evidence/e6305cde-5b3c-4eda-bdc9-83b05fbcfb79"},{"id":"6d402ad0-3be9-4daf-ba1b-50420c603ead","criterion":"reading-order-structure","criterion_name":"Reading Order & Structure","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The tool preserves a long narrative operating-performance section with headings and paragraph flow in a quarterly financial report page.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/6d402ad0-3be9-4daf-ba1b-50420c603ead"},{"id":"f0d4e470-bcfd-41d7-858c-0f81bd6a8e82","criterion":"reading-order-structure","criterion_name":"Reading Order & Structure","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"Does not consistently preserve paragraph boundaries, so narrative flow fragments across sections instead of staying in the source order.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/f0d4e470-bcfd-41d7-858c-0f81bd6a8e82"},{"id":"df79f428-906a-474d-a081-7c35b9d43549","criterion":"text-ocr-completeness","criterion_name":"Text & OCR Completeness","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"Partially truncates OCR on the scanned disclaimer page, dropping the right edge of several sentences in the GAAP, translation, forward-looking statements, and rounding notice.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/df79f428-906a-474d-a081-7c35b9d43549"}],"appears_in":[{"page_type":"ranking","slug":"pdf-to-markdown-apis","title":"Best AI Tools to Convert Complex PDFs into Clean Markdown with an API","url":"https://aidemos.com/best/pdf-to-markdown-apis","binding":"run"}],"same_scenario":[{"id":"0ca22480-01a9-4e56-9b68-305585f6dd8c","tool":"adobe-api","tool_name":"Adobe API","verdict":"worked","score":null,"score_total":null,"note":"Preserves a grouped multi-level segment table with previous/current quarter amounts and year-over-year change columns, keeping the segment rows and totals aligned."},{"id":"97367dd9-249f-4997-95e4-14cc616e9713","tool":"extend-ai","tool_name":"Extend AI","verdict":"worked","score":null,"score_total":null,"note":"Preserves grouped-column structure in the financial tables, including the multilevel segment matrix and the balance-sheet comparison layout."},{"id":"3e92e668-2248-45fe-bbe3-3bbe48017ea9","tool":"landing-ai","tool_name":"Landing AI","verdict":"worked","score":null,"score_total":null,"note":"Reconstructs a multi-column segment comparison table with previous-year, present-year, and year-over-year change columns aligned across the rows."},{"id":"f41bfe47-b4b5-4125-aabb-b2f1fe86c857","tool":"llamaparse","tool_name":"LlamaParse","verdict":"mixed","score":null,"score_total":null,"note":"Keeps the numeric values in a dense hierarchical table but makes grouped-header semantics less explicit, so parent-child column roles become less clear."},{"id":"5fe802fc-62a5-4a6a-aa9c-9ae77852cde1","tool":"mistral-ai","tool_name":"Mistral AI","verdict":"failed","score":null,"score_total":null,"note":"Distinct header levels are collapsed into a single cell in the multilevel table, weakening the parent-child relationships that define the column semantics."},{"id":"f597cbbc-cbe9-4694-8b5e-078b1489986b","tool":"reducto","tool_name":"Reducto","verdict":"failed","score":null,"score_total":null,"note":"Breaks a 5-column business-results table by duplicating the 2025 header and merging the 2024 value and percent change into one cell; the Net sales row ends up with 254,811 and 2.6 smashed together."},{"id":"63381a32-714a-48c6-b8dc-3ce4fc67279d","tool":"tensorlake","tool_name":"Tensorlake","verdict":"failed","score":null,"score_total":null,"note":"Breaks on a more complex multi-header table by losing header hierarchy and omitting at least one header label, producing an incomplete and structurally incorrect reconstruction."},{"id":"c66e2b14-a549-4ce7-bbd9-4f0fb50a2057","tool":"upstage-ai","tool_name":"Upstage AI","verdict":"failed","score":null,"score_total":null,"note":"Misaligns column headers with data regions in dense financial tables, producing a structurally inconsistent balance-sheet extraction rather than a clean header-to-row mapping."}]}