{"observation":{"id":"8c64ae90-42e2-4970-a51d-5abbc49c12c4","tool":"reducto","tool_name":"Reducto","criterion":"schema-adherence","criterion_name":"Schema Adherence","criterion_definition":"Does the output follow the supplied JSON schema hierarchy exactly, with correct nesting, field names, and data types?","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"If the tool does not follow the supplied JSON schema exactly, the extracted data cannot be reliably used for structured querying or downstream automation. (3 of 3 judges)","scenario":"bank-statement-pdf","scenario_name":"Bank Statement PDF","group_tag":"business-document-extraction","scenario_description":"A four-page bank statement PDF with dense transaction tables, balance-forward bridges, account metadata, rewards data, and disclaimer text. It was used to stress schema-driven extraction, multi-page continuity, row completeness, and financial numerical accuracy.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://cdn.futuresmart.ai/public/aidemos/a459a9de9c8a466d8f69b112e04ddad2.pdf?v=1","role":"input","filename":"Bank Statement PDF.pdf"}],"stresses":["Table extraction across 50+ transaction rows","Multi-page continuity with balance-forward bridges","Parsing structured account metadata alongside unstructured transaction descriptions","Numerical accuracy for deposits, withdrawals, running balances, and summaries","Extraction of nested rewards and disclaimer sections"],"verdict":"worked","score":null,"score_total":null,"note":"Preserves a nested, schema-shaped JSON structure for the bank statement instead of flattening the document into raw OCR, including top-level objects like metadata, account_holder, account, branch, statement_period, transactions, and summary.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","role":"input","alt":"Research media bank statement 2 jul.png"},{"url":"https://d3epheqghktydj.cloudfront.net/research-media-reducto-bank-statement-output-320fdfd45c76.png","role":"output","alt":null},{"url":"https://d3epheqghktydj.cloudfront.net/research-media-reducto-bank-statement-output-c7b305d1c61b.json","role":"output","alt":null}],"run_id":"a061b9e7-a9c5-443d-a171-b296aaf51b8c","study_title":"Extract and query structured data from documents using natural language","study_kind":"generation","research_task":"86b9y25e5","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://cdn.futuresmart.ai/public/aidemos/a459a9de9c8a466d8f69b112e04ddad2.pdf?v=1","filename":"Bank Statement PDF.pdf","alt":"Bank Statement PDF","role":"input"}],"modality":"pdf","stresses":["Table extraction across 50+ transaction rows","Multi-page continuity with balance-forward bridges","Parsing structured account metadata alongside unstructured transaction descriptions","Numerical accuracy for deposits, withdrawals, running balances, and summaries","Extraction of nested rewards and disclaimer sections"]},"tool_page_slug":"reducto","tool_url":"https://aidemos.com/tools/reducto","permalink":"https://aidemos.com/evidence/8c64ae90-42e2-4970-a51d-5abbc49c12c4","api_url":"https://ai.aidemos.com/v1/observations/8c64ae90-42e2-4970-a51d-5abbc49c12c4"},"peers":[{"id":"62e6efd4-2d0b-4870-8159-8b09877da655","tool":"datalab","tool_name":"Datalab","verdict":"worked","score":null,"score_total":null,"note":"Can emit the bank-statement extraction as nested schema-shaped JSON, with separate metadata, account_holder/account, branch, transactions, summary, rewards, and disclaimers objects instead of flat OCR text.","artifact_count":7,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","evidence_url":"https://aidemos.com/evidence/62e6efd4-2d0b-4870-8159-8b09877da655"},{"id":"b1ccd11a-19d7-4af9-93c2-1ba1086296e5","tool":"extend-ai","tool_name":"Extend AI","verdict":"worked","score":null,"score_total":null,"note":"The tool reconstructs the bank-statement hierarchy into nested JSON with branch, account, rewards, metadata, balances, summary, and transaction-related objects present in the requested layout.","artifact_count":5,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","evidence_url":"https://aidemos.com/evidence/b1ccd11a-19d7-4af9-93c2-1ba1086296e5"},{"id":"253299f2-f4f9-4a12-ac12-278c46cc1d98","tool":"landing-ai","tool_name":"Landing AI","verdict":"worked","score":null,"score_total":null,"note":"Reconstructs the supplied bank-statement hierarchy instead of flat OCR, with nested statement.metadata, account_holder.address, account, branch, statement_period, balances, transactions, summary, rewards, and disclaimers objects present in the extracted JSON flow.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/48962ee031ec4496b0c08a3aacebe0d8.mp4?v=1","evidence_url":"https://aidemos.com/evidence/253299f2-f4f9-4a12-ac12-278c46cc1d98"},{"id":"8fb0c5fe-7b66-4d4c-bd2e-f63382a6c431","tool":"llamaparse","tool_name":"LlamaParse","verdict":"worked","score":null,"score_total":null,"note":"The bank-statement output follows the requested nested schema closely, reconstructing metadata, account_holder, account, branch, statement_period, balances, transactions, summary, rewards, and disclaimers as structured objects rather than flat OCR text.","artifact_count":5,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","evidence_url":"https://aidemos.com/evidence/8fb0c5fe-7b66-4d4c-bd2e-f63382a6c431"},{"id":"cb734437-1435-4fa1-a680-6250f75e51e4","tool":"nanonets","tool_name":"Nanonets","verdict":"worked","score":null,"score_total":null,"note":"Preserves the supplied bank-statement hierarchy in structured JSON, with nested statement.metadata, account_holder, account, balances, transactions, summary, rewards, and disclaimers objects instead of flattening the document into OCR text.","artifact_count":3,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","evidence_url":"https://aidemos.com/evidence/cb734437-1435-4fa1-a680-6250f75e51e4"},{"id":"0a4f8a1a-6dec-4424-b76b-d0568f8dcd32","tool":"retab","tool_name":"Retab","verdict":"worked","score":null,"score_total":null,"note":"Reconstructs the supplied bank-statement schema into a nested JSON object with separate statement.metadata, account_holder, account, branch, balances, transactions, summary, rewards, and disclaimers sections instead of flattening the document into OCR text.","artifact_count":3,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","evidence_url":"https://aidemos.com/evidence/0a4f8a1a-6dec-4424-b76b-d0568f8dcd32"},{"id":"7723ce9b-b9fc-4a93-ba5e-6a1782b61721","tool":"unstract","tool_name":"Unstract","verdict":"worked","score":null,"score_total":null,"note":"Follows the requested nested bank-statement schema instead of flattening the document, populating structured objects such as metadata, account_holder, account, branch, balances, transactions, and summary.","artifact_count":4,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-unstract-bank-statement-output-f0bd422ffa9e.json","evidence_url":"https://aidemos.com/evidence/7723ce9b-b9fc-4a93-ba5e-6a1782b61721"}],"other_criteria":[{"id":"f4299c5d-6cfa-4e80-8298-ffb066723331","criterion":"extraction-accuracy","criterion_name":"Extraction Accuracy","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Correctly extracts the bank statement currency as INR and attaches high-confidence citation metadata, with extract confidence shown as 0.984.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/f4299c5d-6cfa-4e80-8298-ffb066723331"},{"id":"07a518db-9ca5-4d73-b004-716f8b27d2c3","criterion":"extraction-accuracy","criterion_name":"Extraction Accuracy","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Correctly extracts the account-holder/customer name from the bank statement and attaches a citation bounding box to the source text.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/07a518db-9ca5-4d73-b004-716f8b27d2c3"},{"id":"0c08d768-cdc2-4ae7-93b1-cc5a1014cf8d","criterion":"extraction-accuracy","criterion_name":"Extraction Accuracy","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"Miscomputes the bank summary aggregation by reporting total_transactions as 70 even though the report says the correct count is 40 after exclusions, so the financial summary is not reliable for reconciliation.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/0c08d768-cdc2-4ae7-93b1-cc5a1014cf8d"},{"id":"b786a10c-15a5-4f31-9a78-3508cf20cdf5","criterion":"semantic-field-enrichment","criterion_name":"Semantic Field Enrichment","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"Does not consistently enrich transaction rows with schema-level fields such as transaction_type and transaction_id; the visible extracted rows remain partially populated and omit those fields.","artifact_count":4,"evidence_url":"https://aidemos.com/evidence/b786a10c-15a5-4f31-9a78-3508cf20cdf5"},{"id":"ff5e7c6c-df96-4b6c-aaf7-ec4140ca5783","criterion":"table-record-completeness","criterion_name":"Table & Record Completeness","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"Over-produces the repeating transaction table, returning 53 transaction records when the bank statement contains 51, indicating row-count drift in line-item extraction.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/ff5e7c6c-df96-4b6c-aaf7-ec4140ca5783"}],"appears_in":[{"page_type":"ranking","slug":"document-extraction","title":"Best AI Tools for Extracting Structured Data from Business Documents","url":"https://aidemos.com/best/document-extraction","binding":"run"},{"page_type":"tool","slug":"docsumo","title":null,"url":"https://aidemos.com/tools/docsumo","binding":"run"}],"same_scenario":[{"id":"62e6efd4-2d0b-4870-8159-8b09877da655","tool":"datalab","tool_name":"Datalab","verdict":"worked","score":null,"score_total":null,"note":"Can emit the bank-statement extraction as nested schema-shaped JSON, with separate metadata, account_holder/account, branch, transactions, summary, rewards, and disclaimers objects instead of flat OCR text."},{"id":"b1ccd11a-19d7-4af9-93c2-1ba1086296e5","tool":"extend-ai","tool_name":"Extend AI","verdict":"worked","score":null,"score_total":null,"note":"The tool reconstructs the bank-statement hierarchy into nested JSON with branch, account, rewards, metadata, balances, summary, and transaction-related objects present in the requested layout."},{"id":"253299f2-f4f9-4a12-ac12-278c46cc1d98","tool":"landing-ai","tool_name":"Landing AI","verdict":"worked","score":null,"score_total":null,"note":"Reconstructs the supplied bank-statement hierarchy instead of flat OCR, with nested statement.metadata, account_holder.address, account, branch, statement_period, balances, transactions, summary, rewards, and disclaimers objects present in the extracted JSON flow."},{"id":"8fb0c5fe-7b66-4d4c-bd2e-f63382a6c431","tool":"llamaparse","tool_name":"LlamaParse","verdict":"worked","score":null,"score_total":null,"note":"The bank-statement output follows the requested nested schema closely, reconstructing metadata, account_holder, account, branch, statement_period, balances, transactions, summary, rewards, and disclaimers as structured objects rather than flat OCR text."},{"id":"cb734437-1435-4fa1-a680-6250f75e51e4","tool":"nanonets","tool_name":"Nanonets","verdict":"worked","score":null,"score_total":null,"note":"Preserves the supplied bank-statement hierarchy in structured JSON, with nested statement.metadata, account_holder, account, balances, transactions, summary, rewards, and disclaimers objects instead of flattening the document into OCR text."},{"id":"0a4f8a1a-6dec-4424-b76b-d0568f8dcd32","tool":"retab","tool_name":"Retab","verdict":"worked","score":null,"score_total":null,"note":"Reconstructs the supplied bank-statement schema into a nested JSON object with separate statement.metadata, account_holder, account, branch, balances, transactions, summary, rewards, and disclaimers sections instead of flattening the document into OCR text."},{"id":"7723ce9b-b9fc-4a93-ba5e-6a1782b61721","tool":"unstract","tool_name":"Unstract","verdict":"worked","score":null,"score_total":null,"note":"Follows the requested nested bank-statement schema instead of flattening the document, populating structured objects such as metadata, account_holder, account, branch, balances, transactions, and summary."}]}