{"observation":{"id":"f7dda2d7-aecd-4aa9-b323-a15764652e44","tool":"datalab","tool_name":"Datalab","criterion":"semantic-field-enrichment","criterion_name":"Semantic Field Enrichment","criterion_definition":"Are derived fields such as transaction_type, transaction_id, cheque_number, day patterns, and ad codes correctly classified or extracted beyond raw OCR?","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"For this ranking, correctly identifying derived business fields is part of producing usable structured data, not just a nice extra. (3 of 3 judges)","scenario":"bank-statement-pdf","scenario_name":"Bank Statement PDF","group_tag":"business-document-extraction","scenario_description":"A four-page bank statement PDF with dense transaction tables, balance-forward bridges, account metadata, rewards data, and disclaimer text. It was used to stress schema-driven extraction, multi-page continuity, row completeness, and financial numerical accuracy.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://cdn.futuresmart.ai/public/aidemos/a459a9de9c8a466d8f69b112e04ddad2.pdf?v=1","role":"input","filename":"Bank Statement PDF.pdf"}],"stresses":["Table extraction across 50+ transaction rows","Multi-page continuity with balance-forward bridges","Parsing structured account metadata alongside unstructured transaction descriptions","Numerical accuracy for deposits, withdrawals, running balances, and summaries","Extraction of nested rewards and disclaimer sections"],"verdict":"failed","score":null,"score_total":null,"note":"Leaves the derived bank-statement transaction_id field null, including on the 18 Jun ATM withdrawal that visibly contains the embedded ID 916910098754.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/80135a758c5a4d2aa2187e8af3f050e8.png?v=1","role":"input","alt":null},{"url":"https://d3epheqghktydj.cloudfront.net/research-media-datalab-bank-statement-extracted-18-jun--b2e835587aaa.png","role":"output","alt":null},{"url":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-18-jun-transaction-id-c53a58985a4d.png","role":"input","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/ae80ff9ae7db40e4b27291b0b308083b.png?v=1","role":"input","alt":null}],"run_id":"a061b9e7-a9c5-443d-a171-b296aaf51b8c","study_title":"Extract and query structured data from documents using natural language","study_kind":"generation","research_task":"86b9y25e5","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://cdn.futuresmart.ai/public/aidemos/a459a9de9c8a466d8f69b112e04ddad2.pdf?v=1","filename":"Bank Statement PDF.pdf","alt":"Bank Statement PDF","role":"input"}],"modality":"pdf","stresses":["Table extraction across 50+ transaction rows","Multi-page continuity with balance-forward bridges","Parsing structured account metadata alongside unstructured transaction descriptions","Numerical accuracy for deposits, withdrawals, running balances, and summaries","Extraction of nested rewards and disclaimer sections"]},"tool_page_slug":"datalab","tool_url":"https://aidemos.com/tools/datalab","permalink":"https://aidemos.com/evidence/f7dda2d7-aecd-4aa9-b323-a15764652e44","api_url":"https://ai.aidemos.com/v1/observations/f7dda2d7-aecd-4aa9-b323-a15764652e44"},"peers":[{"id":"ca67652c-8a96-456d-92ef-c2817c381589","tool":"extend-ai","tool_name":"Extend AI","verdict":"worked","score":null,"score_total":null,"note":"The tool infers transaction direction from free text, classifying an ATM entry as `Withdrawal` and populating `withdrawal_amount: 1000`.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-extend-ai-transaction-type-5ef541ac7fec.png","evidence_url":"https://aidemos.com/evidence/ca67652c-8a96-456d-92ef-c2817c381589"},{"id":"e93da7d1-3512-4bea-a7ef-f17bb7e9574c","tool":"landing-ai","tool_name":"Landing AI","verdict":"worked","score":null,"score_total":null,"note":"Classifies extracted bank rows with semantic transaction_type labels such as Withdrawal and Deposit, showing derived type tagging beyond raw transaction text.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-landing-ai-bank-statement-18-jun-extract-321aa2936cc6.png","evidence_url":"https://aidemos.com/evidence/e93da7d1-3512-4bea-a7ef-f17bb7e9574c"},{"id":"ef93fe5b-83f1-421a-87bf-2923b9c82aae","tool":"llamaparse","tool_name":"LlamaParse","verdict":"failed","score":null,"score_total":null,"note":"The tool does not reliably enrich bank transactions with derived identifiers or types: transaction_id and transaction_type are left empty across extracted records even when the descriptions contain transaction clues.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-llamaparse-bank-statement-transaction-id-c01a0d30b3be.png","evidence_url":"https://aidemos.com/evidence/ef93fe5b-83f1-421a-87bf-2923b9c82aae"},{"id":"b9c9cb8d-7aa6-4c2a-b477-8b9c1626da05","tool":"nanonets","tool_name":"Nanonets","verdict":"failed","score":null,"score_total":null,"note":"Does not derive transaction_type on the bank statement at all; the report states that transaction_type is null for every extracted transaction.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","evidence_url":"https://aidemos.com/evidence/b9c9cb8d-7aa6-4c2a-b477-8b9c1626da05"},{"id":"b786a10c-15a5-4f31-9a78-3508cf20cdf5","tool":"reducto","tool_name":"Reducto","verdict":"failed","score":null,"score_total":null,"note":"Does not consistently enrich transaction rows with schema-level fields such as transaction_type and transaction_id; the visible extracted rows remain partially populated and omit those fields.","artifact_count":4,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-schema-transactions-00d0c871c403.png","evidence_url":"https://aidemos.com/evidence/b786a10c-15a5-4f31-9a78-3508cf20cdf5"},{"id":"65edc0fa-f547-4d58-bc57-0b7a22dfae4b","tool":"retab","tool_name":"Retab","verdict":"worked","score":null,"score_total":null,"note":"Derives transaction_type and transaction_id from the transaction narration, labeling the sample record as UPI with transaction_id 917615251879 instead of leaving only raw description text.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","evidence_url":"https://aidemos.com/evidence/65edc0fa-f547-4d58-bc57-0b7a22dfae4b"},{"id":"5cf41e1e-0a8d-4000-a164-198ad5e3b579","tool":"unstract","tool_name":"Unstract","verdict":"worked","score":null,"score_total":null,"note":"Correctly enriches bank rows with transaction_type classification, with the report stating Withdrawal/Deposit was assigned correctly on all 51 transaction rows.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/dca3068e880f473f843a77f119b862b7.png?v=1","evidence_url":"https://aidemos.com/evidence/5cf41e1e-0a8d-4000-a164-198ad5e3b579"}],"other_criteria":[{"id":"3f57949b-b930-488c-9c76-b9ebc3b73d51","criterion":"extraction-accuracy","criterion_name":"Extraction Accuracy","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"Misplaces a bank-statement transaction across dates: a row that belongs to 18 Jun is attached to the 19 Jun record after merging, so date association is unreliable.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/3f57949b-b930-488c-9c76-b9ebc3b73d51"},{"id":"62e6efd4-2d0b-4870-8159-8b09877da655","criterion":"schema-adherence","criterion_name":"Schema Adherence","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Can emit the bank-statement extraction as nested schema-shaped JSON, with separate metadata, account_holder/account, branch, transactions, summary, rewards, and disclaimers objects instead of flat OCR text.","artifact_count":7,"evidence_url":"https://aidemos.com/evidence/62e6efd4-2d0b-4870-8159-8b09877da655"},{"id":"9ff398df-39f0-4ff1-b7cd-d485a9d07b0e","criterion":"table-record-completeness","criterion_name":"Table & Record Completeness","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"Merges adjacent bank-statement rows into one record; the 21 Jun crop shows two separate transactions, but the extracted description collapses them into one long UPI/merchant string.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/9ff398df-39f0-4ff1-b7cd-d485a9d07b0e"},{"id":"36c397f5-e058-43d6-96e8-d14d1b64ef2f","criterion":"table-record-completeness","criterion_name":"Table & Record Completeness","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"Over-segments the bank-statement table: the report says 54 transaction records were extracted where 51 were expected, a +3 overcount caused by continuation rows and split records.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/36c397f5-e058-43d6-96e8-d14d1b64ef2f"}],"appears_in":[{"page_type":"ranking","slug":"document-extraction","title":"Best AI Tools for Extracting Structured Data from Business Documents","url":"https://aidemos.com/best/document-extraction","binding":"run"},{"page_type":"tool","slug":"docsumo","title":null,"url":"https://aidemos.com/tools/docsumo","binding":"run"}],"same_scenario":[{"id":"ca67652c-8a96-456d-92ef-c2817c381589","tool":"extend-ai","tool_name":"Extend AI","verdict":"worked","score":null,"score_total":null,"note":"The tool infers transaction direction from free text, classifying an ATM entry as `Withdrawal` and populating `withdrawal_amount: 1000`."},{"id":"e93da7d1-3512-4bea-a7ef-f17bb7e9574c","tool":"landing-ai","tool_name":"Landing AI","verdict":"worked","score":null,"score_total":null,"note":"Classifies extracted bank rows with semantic transaction_type labels such as Withdrawal and Deposit, showing derived type tagging beyond raw transaction text."},{"id":"ef93fe5b-83f1-421a-87bf-2923b9c82aae","tool":"llamaparse","tool_name":"LlamaParse","verdict":"failed","score":null,"score_total":null,"note":"The tool does not reliably enrich bank transactions with derived identifiers or types: transaction_id and transaction_type are left empty across extracted records even when the descriptions contain transaction clues."},{"id":"b9c9cb8d-7aa6-4c2a-b477-8b9c1626da05","tool":"nanonets","tool_name":"Nanonets","verdict":"failed","score":null,"score_total":null,"note":"Does not derive transaction_type on the bank statement at all; the report states that transaction_type is null for every extracted transaction."},{"id":"b786a10c-15a5-4f31-9a78-3508cf20cdf5","tool":"reducto","tool_name":"Reducto","verdict":"failed","score":null,"score_total":null,"note":"Does not consistently enrich transaction rows with schema-level fields such as transaction_type and transaction_id; the visible extracted rows remain partially populated and omit those fields."},{"id":"65edc0fa-f547-4d58-bc57-0b7a22dfae4b","tool":"retab","tool_name":"Retab","verdict":"worked","score":null,"score_total":null,"note":"Derives transaction_type and transaction_id from the transaction narration, labeling the sample record as UPI with transaction_id 917615251879 instead of leaving only raw description text."},{"id":"5cf41e1e-0a8d-4000-a164-198ad5e3b579","tool":"unstract","tool_name":"Unstract","verdict":"worked","score":null,"score_total":null,"note":"Correctly enriches bank rows with transaction_type classification, with the report stating Withdrawal/Deposit was assigned correctly on all 51 transaction rows."}]}