{"observation":{"id":"457e867a-acde-40fc-ad65-8f9b5f51378d","tool":"mistral-ai","tool_name":"Mistral AI","criterion":"visual-content-retention","criterion_name":"Visual Content Retention","criterion_definition":"Retains charts, figures, diagrams, and images in the output and places them in the correct reading position.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"For complex PDFs, figures, charts, and images are part of the document content, so preserving them in the right place is part of the core job. (3 of 3 judges)","scenario":"hybrid-earnings-report","scenario_name":"Hybrid Earnings Report","group_tag":"hybrid-earnings-report","scenario_description":"A long, real-world hybrid annual report used to benchmark end-to-end PDF-to-markdown conversion. It combines native text, financial tables, charts/graphics, and scanned signature/stamp regions, stressing preservation of document structure, reading order, and embedded visual content across a multi-section report.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://cdn.futuresmart.ai/public/aidemos/f63a44b441704753b65d1b89dc9b4bbb.pdf?v=1","role":"input","filename":"Target-2015-Annual-Report.pdf"}],"stresses":["Native digital text extraction","Complex financial table preservation","Chart and graphic retention","Image-based signatures and stamps","Reading order across a long multi-section report","Markdown quality and consistency"],"verdict":"worked","score":null,"score_total":null,"note":"Visual assets are extracted into page-specific folders, keeping charts, signatures, and other document elements tied to their source pages instead of collapsing them into one consolidated output.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/07829e6dcb414445886e618cd047efb2.pdf?v=1","role":"input","alt":"07829e6dcb414445886e618cd047efb2.pdf"},{"url":"https://cdn.futuresmart.ai/public/aidemos/ce5837f984214699955ea8ad4875492f.png?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/1522d37359bf4c14abc97f0f1deb9d1b.png?v=1","role":"output","alt":null}],"run_id":"6e3160de-fe46-4b45-b071-72560b5c5d0e","study_title":"Convert a Complex PDF into Clean Markdown with an API","study_kind":"generation","research_task":"86b9h7t37","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://cdn.futuresmart.ai/public/aidemos/f63a44b441704753b65d1b89dc9b4bbb.pdf?v=1","filename":"Target-2015-Annual-Report.pdf","alt":"Hybrid Earnings Report","role":"input"}],"modality":"pdf","stresses":["Native digital text extraction","Complex financial table preservation","Chart and graphic retention","Image-based signatures and stamps","Reading order across a long multi-section report","Markdown quality and consistency"]},"tool_page_slug":"mistral-ai","tool_url":"https://aidemos.com/tools/mistral-ai","permalink":"https://aidemos.com/evidence/457e867a-acde-40fc-ad65-8f9b5f51378d","api_url":"https://ai.aidemos.com/v1/observations/457e867a-acde-40fc-ad65-8f9b5f51378d"},"peers":[{"id":"5a2c32cc-14a5-43f9-b805-dc8dd67cd337","tool":"adobe-api","tool_name":"Adobe API","verdict":"failed","score":null,"score_total":null,"note":"Drops handwritten signature images from a scanned signatures page while retaining the surrounding printed legal text, so the signature content is missing from the extracted output.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/5dd6e0b5b03c475ca73ddc8672f3af69.png?v=1","evidence_url":"https://aidemos.com/evidence/5a2c32cc-14a5-43f9-b805-dc8dd67cd337"},{"id":"3c7b525c-4c85-4bab-9ff6-9033a037e77c","tool":"extend-ai","tool_name":"Extend AI","verdict":"mixed","score":null,"score_total":null,"note":"Retains charts and logos as standalone figure blocks with captions, but they are extracted out of page position rather than embedded inline.","artifact_count":5,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/ab0bf9cefafd4414a8fbc135950b69e8.png?v=1","evidence_url":"https://aidemos.com/evidence/3c7b525c-4c85-4bab-9ff6-9033a037e77c"},{"id":"97bf4c42-ab87-4b0a-913c-2823e833dd10","tool":"landing-ai","tool_name":"Landing AI","verdict":"failed","score":null,"score_total":null,"note":"Replaces a waterfall chart with text, preserving the SG&A rate values and year-to-year change labels but not the chart as a visual asset in the output.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/d6cecafea14044a1800a5809e34963a2.png?v=1","evidence_url":"https://aidemos.com/evidence/97bf4c42-ab87-4b0a-913c-2823e833dd10"},{"id":"4c0c1596-7314-44a9-9fa4-2f4b240bef65","tool":"llamaparse","tool_name":"LlamaParse","verdict":"mixed","score":null,"score_total":null,"note":"Converts a waterfall chart into a text/table representation, preserving the SG&A rate sequence and deltas but not the chart as a visual object.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/bb341a5637d24cd7b0d5b4c5bb9effb5.png?v=1","evidence_url":"https://aidemos.com/evidence/4c0c1596-7314-44a9-9fa4-2f4b240bef65"},{"id":"c0ba2799-f41e-4682-8f37-943670bb0f38","tool":"nutrient-io","tool_name":"Nutrient.io","verdict":"failed","score":null,"score_total":null,"note":"Loses the visual semantics of the SG&A waterfall chart; the 2013–2015 rate values are present, but the chart type, axes, and legend relationships are not preserved.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/bb341a5637d24cd7b0d5b4c5bb9effb5.png?v=1","evidence_url":"https://aidemos.com/evidence/c0ba2799-f41e-4682-8f37-943670bb0f38"},{"id":"d44d1388-5a3e-4021-94ac-d7281d773533","tool":"reducto","tool_name":"Reducto","verdict":"worked","score":null,"score_total":null,"note":"Retains visual content as real image outputs when requested: the portrait, donut chart, and bar chart are all returned as preserved figure/image crops rather than being dropped.","artifact_count":10,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/8bd3462da00d457ab2c649926bfe8653.png?v=1","evidence_url":"https://aidemos.com/evidence/d44d1388-5a3e-4021-94ac-d7281d773533"},{"id":"e528b05b-ad28-4fda-a633-eda580b60c6f","tool":"tensorlake","tool_name":"Tensorlake","verdict":"worked","score":null,"score_total":null,"note":"Retains scanned signature-page visuals as figure blocks plus signer text, including named executives and dates, rather than dropping the signature regions from the output.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/1f45fe222ffc4719ad527014bd6fabbf.png?v=1","evidence_url":"https://aidemos.com/evidence/e528b05b-ad28-4fda-a633-eda580b60c6f"},{"id":"417a2325-3f7b-4e6f-a06f-e7653967e43e","tool":"upstage-ai","tool_name":"Upstage AI","verdict":"failed","score":null,"score_total":null,"note":"Does not retain handwritten signature content as visual output on the signatures page, leaving only partial surrounding legal text and no clearly identifiable signatures.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/1f45fe222ffc4719ad527014bd6fabbf.png?v=1","evidence_url":"https://aidemos.com/evidence/417a2325-3f7b-4e6f-a06f-e7653967e43e"}],"other_criteria":[{"id":"8217f0a1-861e-4d16-9d7a-f644f646e9c2","criterion":"complex-document-handling","criterion_name":"Complex Document Handling","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The tool handles an 84-page mixed-content annual report end-to-end, including tables, charts, and scanned signature/stamp regions, and returns usable markdown output without manual correction.","artifact_count":5,"evidence_url":"https://aidemos.com/evidence/8217f0a1-861e-4d16-9d7a-f644f646e9c2"},{"id":"ec898578-9641-4f6a-b228-bebde59abc51","criterion":"markdown-quality","criterion_name":"Markdown Quality","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The export is usable markdown rather than a flat text dump: it includes an overall markdown file plus page-wise markdown files delivered in a downloadable ZIP.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/ec898578-9641-4f6a-b228-bebde59abc51"},{"id":"d89c69f3-025a-4dd1-9d85-6524482d5b29","criterion":"reading-order-structure","criterion_name":"Reading Order & Structure","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The parser preserves document hierarchy across most sections of the long report, keeping headings and supporting text structurally aligned through the majority of the output.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/d89c69f3-025a-4dd1-9d85-6524482d5b29"},{"id":"ca7155cc-ae34-4e12-aa47-954393f05576","criterion":"reading-order-structure","criterion_name":"Reading Order & Structure","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"Heading hierarchy is reconstructed inconsistently, so some parts of the document are flattened even though the underlying content is still recovered.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/ca7155cc-ae34-4e12-aa47-954393f05576"},{"id":"e8c8bbc3-204e-4a77-b1ed-b025c3f77ac0","criterion":"table-preservation","criterion_name":"Table Preservation","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The financial table is reconstructed into a usable structured layout without losing the relationships between headers, rows, and corresponding values.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/e8c8bbc3-204e-4a77-b1ed-b025c3f77ac0"}],"appears_in":[{"page_type":"ranking","slug":"pdf-to-markdown-apis","title":"Best AI Tools to Convert Complex PDFs into Clean Markdown with an API","url":"https://aidemos.com/best/pdf-to-markdown-apis","binding":"run"}],"same_scenario":[{"id":"5a2c32cc-14a5-43f9-b805-dc8dd67cd337","tool":"adobe-api","tool_name":"Adobe API","verdict":"failed","score":null,"score_total":null,"note":"Drops handwritten signature images from a scanned signatures page while retaining the surrounding printed legal text, so the signature content is missing from the extracted output."},{"id":"3c7b525c-4c85-4bab-9ff6-9033a037e77c","tool":"extend-ai","tool_name":"Extend AI","verdict":"mixed","score":null,"score_total":null,"note":"Retains charts and logos as standalone figure blocks with captions, but they are extracted out of page position rather than embedded inline."},{"id":"97bf4c42-ab87-4b0a-913c-2823e833dd10","tool":"landing-ai","tool_name":"Landing AI","verdict":"failed","score":null,"score_total":null,"note":"Replaces a waterfall chart with text, preserving the SG&A rate values and year-to-year change labels but not the chart as a visual asset in the output."},{"id":"4c0c1596-7314-44a9-9fa4-2f4b240bef65","tool":"llamaparse","tool_name":"LlamaParse","verdict":"mixed","score":null,"score_total":null,"note":"Converts a waterfall chart into a text/table representation, preserving the SG&A rate sequence and deltas but not the chart as a visual object."},{"id":"c0ba2799-f41e-4682-8f37-943670bb0f38","tool":"nutrient-io","tool_name":"Nutrient.io","verdict":"failed","score":null,"score_total":null,"note":"Loses the visual semantics of the SG&A waterfall chart; the 2013–2015 rate values are present, but the chart type, axes, and legend relationships are not preserved."},{"id":"d44d1388-5a3e-4021-94ac-d7281d773533","tool":"reducto","tool_name":"Reducto","verdict":"worked","score":null,"score_total":null,"note":"Retains visual content as real image outputs when requested: the portrait, donut chart, and bar chart are all returned as preserved figure/image crops rather than being dropped."},{"id":"e528b05b-ad28-4fda-a633-eda580b60c6f","tool":"tensorlake","tool_name":"Tensorlake","verdict":"worked","score":null,"score_total":null,"note":"Retains scanned signature-page visuals as figure blocks plus signer text, including named executives and dates, rather than dropping the signature regions from the output."},{"id":"417a2325-3f7b-4e6f-a06f-e7653967e43e","tool":"upstage-ai","tool_name":"Upstage AI","verdict":"failed","score":null,"score_total":null,"note":"Does not retain handwritten signature content as visual output on the signatures page, leaving only partial surrounding legal text and no clearly identifiable signatures."}]}