{"observation":{"id":"ad17e2c7-e810-4619-96c5-09931380ff90","tool":"landing-ai","tool_name":"Landing AI","criterion":"extraction-accuracy","criterion_name":"Extraction Accuracy","criterion_definition":"Are field values correct, complete, and free of OCR or parsing errors? Includes numerical precision on financial fields.","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"The whole point is to pull the right values from documents; wrong or incomplete field values mean the tool failed at the job. (3 of 3 judges)","scenario":"bank-statement-pdf","scenario_name":"Bank Statement PDF","group_tag":"financial-document-extraction","scenario_description":"A 4-page bank statement PDF with 51 transactions, balances, rewards, and disclaimer text, used to test schema-driven extraction of dense financial tables and multi-page continuity.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://d3epheqghktydj.cloudfront.net/extract-and-query-structured-data-from-d-bank-statement-pdf-25ec532be6e1.pdf","role":"input","filename":"Bank Statement PDF.pdf"}],"stresses":["Table extraction across 50+ transaction rows","Multi-page continuity with BALANCE FORWARD bridges","Structured metadata vs. free-text transaction descriptions","Numerical accuracy for balances, deposits, withdrawals, and summaries","Nested schema population for account, branch, balances, rewards, and disclaimers"],"verdict":"worked","score":null,"score_total":null,"note":"Extracts the statement’s top-level values with the correct content and precision, including bank_name \"Standard Chartered\", statement_date \"16 Jul 2019\", currency \"INR\", account number \"42710540422\", opening balance 114453.65, and closing balance 116149.46.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","role":"input","alt":"Research media bank statement 2 jul.png"},{"url":"https://d3epheqghktydj.cloudfront.net/research-media-landing-ai-extracted-bank-statement-c4910d90b9f3.png","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/3229d3afff884329964ab8a73bf2ceb1.png?v=1","role":"output","alt":null}],"run_id":"ec4d736d-95f9-4c88-884c-e280435f7b7b","study_title":"Extract and query structured data from documents using natural language","study_kind":"generation","research_task":"86b9y25e5","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://d3epheqghktydj.cloudfront.net/extract-and-query-structured-data-from-d-bank-statement-pdf-25ec532be6e1.pdf","filename":"Bank Statement PDF.pdf","alt":"Bank Statement PDF","role":"input"}],"modality":"pdf","stresses":["Table extraction across 50+ transaction rows","Multi-page continuity with BALANCE FORWARD bridges","Structured metadata vs. free-text transaction descriptions","Numerical accuracy for balances, deposits, withdrawals, and summaries","Nested schema population for account, branch, balances, rewards, and disclaimers"]},"tool_page_slug":"landing-ai","tool_url":"https://aidemos.com/tools/landing-ai","permalink":"https://aidemos.com/evidence/ad17e2c7-e810-4619-96c5-09931380ff90","api_url":"https://ai.aidemos.com/v1/observations/ad17e2c7-e810-4619-96c5-09931380ff90"},"peers":[{"id":"52cee941-b823-4752-81ff-8803d734fa1a","tool":"datalab","tool_name":"Datalab","verdict":"worked","score":null,"score_total":null,"note":"Extracts the visible document-level values correctly, including State Bank of India, 16 Jul 2019, INR, account number 42710540422, account type SMART BANKING SAVINGS ACCOUNT, branch Rajaji Salai, MICR 600036005, IFSC SCBL0036078, and phone 25349005.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/71f9443de6a54eb6b5500ddc4edef9c5.png?v=1","evidence_url":"https://aidemos.com/evidence/52cee941-b823-4752-81ff-8803d734fa1a"},{"id":"13420641-f30e-4375-8289-ef9240cc49f9","tool":"extend-ai","tool_name":"Extend AI","verdict":"struggled","score":null,"score_total":null,"note":"Its bank summary aggregation is off: the report says `summary.total_transactions` is 49, while the expected count is 40 after excluding Balance Forward, tax, and charge entries.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","evidence_url":"https://aidemos.com/evidence/13420641-f30e-4375-8289-ef9240cc49f9"},{"id":"81a8a18e-5656-4db6-8d73-4dd8203a9209","tool":"llamaparse","tool_name":"LlamaParse","verdict":"struggled","score":null,"score_total":null,"note":"Leaves some transaction value_date fields blank even where the statement shows dates in that column, so transaction metadata is incomplete.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-llamaparse-bank-statement-16-jul-value-d-4906fee56b7b.png","evidence_url":"https://aidemos.com/evidence/81a8a18e-5656-4db6-8d73-4dd8203a9209"},{"id":"4b039231-2d8c-4c73-8ccb-1d789b555f37","tool":"nanonets","tool_name":"Nanonets","verdict":"failed","score":null,"score_total":null,"note":"It leaves summary.total_transactions null even though the statement has 51 transactions, so the summary count is not extracted as a usable value.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-docstrange-nanonets-extracted-total-tran-166ed6f76bd4.png","evidence_url":"https://aidemos.com/evidence/4b039231-2d8c-4c73-8ccb-1d789b555f37"},{"id":"956bec9e-b310-4628-a049-b04b3e645b5e","tool":"reducto","tool_name":"Reducto","verdict":"worked","score":null,"score_total":null,"note":"The tool correctly extracts the bank statement currency as INR and includes citation metadata for that field.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/8bb08fbfe42e417db6ccbeb6262e87f7.png?v=1","evidence_url":"https://aidemos.com/evidence/956bec9e-b310-4628-a049-b04b3e645b5e"},{"id":"bb804c81-06cb-4d05-82a9-804c000f4321","tool":"retab","tool_name":"Retab","verdict":"worked","score":null,"score_total":null,"note":"Extracts key statement values as typed fields, including account holder MR SEENIVASAN, account number 42710540422, and opening and closing balances 114453.65 and 116149.46.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/84e1a6224746446ab5abbd3712c635b8.mp4?v=1","evidence_url":"https://aidemos.com/evidence/bb804c81-06cb-4d05-82a9-804c000f4321"},{"id":"9256b7b3-d9fb-4b69-ba61-8ffde0fa3f6d","tool":"unstract","tool_name":"Unstract","verdict":"failed","score":null,"score_total":null,"note":"Derives an incorrect summary transaction count, reporting 43 when the statement actually contains 51 rows.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/6495c5a868854823bcf3cd94c90bb6aa.pdf?v=1","evidence_url":"https://aidemos.com/evidence/9256b7b3-d9fb-4b69-ba61-8ffde0fa3f6d"}],"other_criteria":[{"id":"91f210ef-e7ef-4d23-a3c8-badb7719e663","criterion":"schema-adherence","criterion_name":"Schema Adherence","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Reconstructs the bank statement into the requested nested JSON hierarchy, with distinct statement.metadata, account_holder.address, account, branch, statement_period, and balances objects rather than flat OCR text.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/91f210ef-e7ef-4d23-a3c8-badb7719e663"},{"id":"9f99b1ec-e13d-49eb-a3c3-0eb907f19ffb","criterion":"semantic-field-enrichment","criterion_name":"Semantic Field Enrichment","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"Does not preserve source-derived transaction identifiers; it normalizes them into sequential transaction_id values such as \"2\", \"3\", and \"4\", so the embedded IDs in the descriptions cannot be traced back directly.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/9f99b1ec-e13d-49eb-a3c3-0eb907f19ffb"},{"id":"a2a1f6f5-e278-415c-afcd-a43a2ec3e91b","criterion":"semantic-field-enrichment","criterion_name":"Semantic Field Enrichment","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Adds meaningful transaction_type labels to extracted rows, classifying the sample 18 Jun records as Withdrawal, Withdrawal, and Deposit instead of leaving the field as raw OCR text.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/a2a1f6f5-e278-415c-afcd-a43a2ec3e91b"},{"id":"76a18022-79e8-45a6-b3c9-bacd1c8b7330","criterion":"table-and-record-completeness","criterion_name":"Table & Record Completeness","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Preserves the full transaction table as 51 separate statement records, without merging rows or dropping entries.","artifact_count":5,"evidence_url":"https://aidemos.com/evidence/76a18022-79e8-45a6-b3c9-bacd1c8b7330"}],"appears_in":[{"page_type":"ranking","slug":"document-extraction","title":"Best AI Tools for Extracting Structured Data from PDFs and Business Documents","url":"https://aidemos.com/best/document-extraction","binding":"run"},{"page_type":"tool","slug":"unstract","title":null,"url":"https://aidemos.com/tools/unstract","binding":"run"}],"same_scenario":[{"id":"52cee941-b823-4752-81ff-8803d734fa1a","tool":"datalab","tool_name":"Datalab","verdict":"worked","score":null,"score_total":null,"note":"Extracts the visible document-level values correctly, including State Bank of India, 16 Jul 2019, INR, account number 42710540422, account type SMART BANKING SAVINGS ACCOUNT, branch Rajaji Salai, MICR 600036005, IFSC SCBL0036078, and phone 25349005."},{"id":"13420641-f30e-4375-8289-ef9240cc49f9","tool":"extend-ai","tool_name":"Extend AI","verdict":"struggled","score":null,"score_total":null,"note":"Its bank summary aggregation is off: the report says `summary.total_transactions` is 49, while the expected count is 40 after excluding Balance Forward, tax, and charge entries."},{"id":"81a8a18e-5656-4db6-8d73-4dd8203a9209","tool":"llamaparse","tool_name":"LlamaParse","verdict":"struggled","score":null,"score_total":null,"note":"Leaves some transaction value_date fields blank even where the statement shows dates in that column, so transaction metadata is incomplete."},{"id":"4b039231-2d8c-4c73-8ccb-1d789b555f37","tool":"nanonets","tool_name":"Nanonets","verdict":"failed","score":null,"score_total":null,"note":"It leaves summary.total_transactions null even though the statement has 51 transactions, so the summary count is not extracted as a usable value."},{"id":"956bec9e-b310-4628-a049-b04b3e645b5e","tool":"reducto","tool_name":"Reducto","verdict":"worked","score":null,"score_total":null,"note":"The tool correctly extracts the bank statement currency as INR and includes citation metadata for that field."},{"id":"bb804c81-06cb-4d05-82a9-804c000f4321","tool":"retab","tool_name":"Retab","verdict":"worked","score":null,"score_total":null,"note":"Extracts key statement values as typed fields, including account holder MR SEENIVASAN, account number 42710540422, and opening and closing balances 114453.65 and 116149.46."},{"id":"9256b7b3-d9fb-4b69-ba61-8ffde0fa3f6d","tool":"unstract","tool_name":"Unstract","verdict":"failed","score":null,"score_total":null,"note":"Derives an incorrect summary transaction count, reporting 43 when the statement actually contains 51 rows."}]}