{"observation":{"id":"a72ab809-ef71-47eb-8461-d9518c96d28b","tool":"extend-ai","tool_name":"Extend AI","criterion":"schema-adherence","criterion_name":"Schema Adherence","criterion_definition":"Does the output follow the supplied JSON schema hierarchy exactly, with correct nesting, field names, and data types?","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"If the output does not match the requested JSON schema exactly, the extracted data cannot be reliably consumed or queried, so this is core to the task. (3 of 3 judges)","scenario":"bank-statement-pdf","scenario_name":"Bank Statement PDF","group_tag":"financial-document-extraction","scenario_description":"A 4-page bank statement PDF with 51 transactions, balances, rewards, and disclaimer text, used to test schema-driven extraction of dense financial tables and multi-page continuity.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://d3epheqghktydj.cloudfront.net/extract-and-query-structured-data-from-d-bank-statement-pdf-25ec532be6e1.pdf","role":"input","filename":"Bank Statement PDF.pdf"}],"stresses":["Table extraction across 50+ transaction rows","Multi-page continuity with BALANCE FORWARD bridges","Structured metadata vs. free-text transaction descriptions","Numerical accuracy for balances, deposits, withdrawals, and summaries","Nested schema population for account, branch, balances, rewards, and disclaimers"],"verdict":"worked","score":null,"score_total":null,"note":"The bank output is rebuilt as nested JSON rather than raw OCR, with branch, account, rewards, balances, summary, and metadata objects populated under the requested statement root.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","role":"input","alt":"Research media bank statement 2 jul.png"},{"url":"https://d3epheqghktydj.cloudfront.net/research-media-extend-ai-extracted-output-1-61394080a2b2.json","role":"output","alt":null},{"url":"https://d3epheqghktydj.cloudfront.net/research-media-extend-ai-document-level-bank-statement-6a3ae27f2afa.png","role":"output","alt":null},{"url":"https://d3epheqghktydj.cloudfront.net/research-media-extend-ai-bank-statement-extracted-detai-ee627956b4b7.png","role":"output","alt":null}],"run_id":"ec4d736d-95f9-4c88-884c-e280435f7b7b","study_title":"Extract and query structured data from documents using natural language","study_kind":"generation","research_task":"86b9y25e5","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://d3epheqghktydj.cloudfront.net/extract-and-query-structured-data-from-d-bank-statement-pdf-25ec532be6e1.pdf","filename":"Bank Statement PDF.pdf","alt":"Bank Statement PDF","role":"input"}],"modality":"pdf","stresses":["Table extraction across 50+ transaction rows","Multi-page continuity with BALANCE FORWARD bridges","Structured metadata vs. free-text transaction descriptions","Numerical accuracy for balances, deposits, withdrawals, and summaries","Nested schema population for account, branch, balances, rewards, and disclaimers"]},"tool_page_slug":"extend-ai","tool_url":"https://aidemos.com/tools/extend-ai","permalink":"https://aidemos.com/evidence/a72ab809-ef71-47eb-8461-d9518c96d28b","api_url":"https://ai.aidemos.com/v1/observations/a72ab809-ef71-47eb-8461-d9518c96d28b"},"peers":[{"id":"3741073d-f776-4beb-b189-4f22c0b5d11e","tool":"datalab","tool_name":"Datalab","verdict":"worked","score":null,"score_total":null,"note":"Maps the bank statement into the requested nested JSON hierarchy instead of flattening it into OCR text, and preserves field-level citation metadata on the extracted objects.","artifact_count":6,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","evidence_url":"https://aidemos.com/evidence/3741073d-f776-4beb-b189-4f22c0b5d11e"},{"id":"91f210ef-e7ef-4d23-a3c8-badb7719e663","tool":"landing-ai","tool_name":"Landing AI","verdict":"worked","score":null,"score_total":null,"note":"Reconstructs the bank statement into the requested nested JSON hierarchy, with distinct statement.metadata, account_holder.address, account, branch, statement_period, and balances objects rather than flat OCR text.","artifact_count":3,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","evidence_url":"https://aidemos.com/evidence/91f210ef-e7ef-4d23-a3c8-badb7719e663"},{"id":"c7ef413a-1654-4252-a5fe-451fa0f92388","tool":"llamaparse","tool_name":"LlamaParse","verdict":"worked","score":null,"score_total":null,"note":"Keeps a nested statement schema intact, emitting separate metadata, account_holder, account, branch, statement_period, balances, transactions, summary, rewards, and disclaimers objects instead of flattening the document.","artifact_count":3,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","evidence_url":"https://aidemos.com/evidence/c7ef413a-1654-4252-a5fe-451fa0f92388"},{"id":"86ec5538-27d3-490e-88ab-a30bbd4479e8","tool":"nanonets","tool_name":"Nanonets","verdict":"worked","score":null,"score_total":null,"note":"It preserves the requested nested schema directly in the output, populating structured objects such as statement, account, balances, transactions, summary, rewards, and disclaimers instead of flattening the document into OCR text.","artifact_count":3,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","evidence_url":"https://aidemos.com/evidence/86ec5538-27d3-490e-88ab-a30bbd4479e8"},{"id":"95071c07-2f7a-463b-b7c7-3738ddbd3651","tool":"reducto","tool_name":"Reducto","verdict":"worked","score":null,"score_total":null,"note":"Reconstructs the bank statement into a nested JSON structure aligned to the requested schema, with document metadata, account, branch, statement period, transactions, summary, and rewards-style sections instead of flat OCR text.","artifact_count":6,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","evidence_url":"https://aidemos.com/evidence/95071c07-2f7a-463b-b7c7-3738ddbd3651"},{"id":"18014a53-cc1f-4a5d-8552-fc026341e820","tool":"retab","tool_name":"Retab","verdict":"worked","score":null,"score_total":null,"note":"Reconstructs dense statement OCR into the requested nested JSON hierarchy, populating separate statement, account_holder, account, branch, balances, transactions, rewards, and disclaimers objects instead of flattening everything into text.","artifact_count":4,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-bank-statement-2-jul-a511cf7bd0d6.png","evidence_url":"https://aidemos.com/evidence/18014a53-cc1f-4a5d-8552-fc026341e820"},{"id":"db903e38-5c91-4688-ac04-2930dc9e72ab","tool":"unstract","tool_name":"Unstract","verdict":"worked","score":null,"score_total":null,"note":"Preserves the requested nested statement hierarchy instead of flattening it, with separate metadata, account_holder, account, branch, balances, transactions, summary, and other top-level sections.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/6495c5a868854823bcf3cd94c90bb6aa.pdf?v=1","evidence_url":"https://aidemos.com/evidence/db903e38-5c91-4688-ac04-2930dc9e72ab"}],"other_criteria":[{"id":"13420641-f30e-4375-8289-ef9240cc49f9","criterion":"extraction-accuracy","criterion_name":"Extraction Accuracy","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"Its bank summary aggregation is off: the report says `summary.total_transactions` is 49, while the expected count is 40 after excluding Balance Forward, tax, and charge entries.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/13420641-f30e-4375-8289-ef9240cc49f9"},{"id":"414af8f0-b7ac-40ed-914e-051d339dfa2b","criterion":"semantic-field-enrichment","criterion_name":"Semantic Field Enrichment","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"It leaves derived `transaction_id` values as `null` even when reference identifiers are present in the description, so identifier extraction does not generalize.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/414af8f0-b7ac-40ed-914e-051d339dfa2b"},{"id":"2c4cdb0a-b344-4604-b614-c97af6d74ea2","criterion":"semantic-field-enrichment","criterion_name":"Semantic Field Enrichment","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"It classifies bank transactions into derived `transaction_type` values such as Withdrawal or Deposit from the description text.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/2c4cdb0a-b344-4604-b614-c97af6d74ea2"},{"id":"4d3907e7-7988-4063-b5ba-89a42de6eab6","criterion":"structural-clean-output","criterion_name":"Structural Clean Output","rank_role":"context","verdict":"failed","score":null,"score_total":null,"note":"It does not preserve the authored top-level schema sequence, so consumers expecting the original field order need a transformation step.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/4d3907e7-7988-4063-b5ba-89a42de6eab6"},{"id":"3b66a1fa-5d29-4130-8e37-0c4cdc033160","criterion":"table-and-record-completeness","criterion_name":"Table & Record Completeness","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The extractor keeps transaction rows as separate records, and the report says it captured all 51 transactions without merging adjacent rows.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/3b66a1fa-5d29-4130-8e37-0c4cdc033160"}],"appears_in":[{"page_type":"ranking","slug":"document-extraction","title":"Best AI Tools for Extracting Structured Data from PDFs and Business Documents","url":"https://aidemos.com/best/document-extraction","binding":"run"},{"page_type":"tool","slug":"unstract","title":null,"url":"https://aidemos.com/tools/unstract","binding":"run"}],"same_scenario":[{"id":"3741073d-f776-4beb-b189-4f22c0b5d11e","tool":"datalab","tool_name":"Datalab","verdict":"worked","score":null,"score_total":null,"note":"Maps the bank statement into the requested nested JSON hierarchy instead of flattening it into OCR text, and preserves field-level citation metadata on the extracted objects."},{"id":"91f210ef-e7ef-4d23-a3c8-badb7719e663","tool":"landing-ai","tool_name":"Landing AI","verdict":"worked","score":null,"score_total":null,"note":"Reconstructs the bank statement into the requested nested JSON hierarchy, with distinct statement.metadata, account_holder.address, account, branch, statement_period, and balances objects rather than flat OCR text."},{"id":"c7ef413a-1654-4252-a5fe-451fa0f92388","tool":"llamaparse","tool_name":"LlamaParse","verdict":"worked","score":null,"score_total":null,"note":"Keeps a nested statement schema intact, emitting separate metadata, account_holder, account, branch, statement_period, balances, transactions, summary, rewards, and disclaimers objects instead of flattening the document."},{"id":"86ec5538-27d3-490e-88ab-a30bbd4479e8","tool":"nanonets","tool_name":"Nanonets","verdict":"worked","score":null,"score_total":null,"note":"It preserves the requested nested schema directly in the output, populating structured objects such as statement, account, balances, transactions, summary, rewards, and disclaimers instead of flattening the document into OCR text."},{"id":"95071c07-2f7a-463b-b7c7-3738ddbd3651","tool":"reducto","tool_name":"Reducto","verdict":"worked","score":null,"score_total":null,"note":"Reconstructs the bank statement into a nested JSON structure aligned to the requested schema, with document metadata, account, branch, statement period, transactions, summary, and rewards-style sections instead of flat OCR text."},{"id":"18014a53-cc1f-4a5d-8552-fc026341e820","tool":"retab","tool_name":"Retab","verdict":"worked","score":null,"score_total":null,"note":"Reconstructs dense statement OCR into the requested nested JSON hierarchy, populating separate statement, account_holder, account, branch, balances, transactions, rewards, and disclaimers objects instead of flattening everything into text."},{"id":"db903e38-5c91-4688-ac04-2930dc9e72ab","tool":"unstract","tool_name":"Unstract","verdict":"worked","score":null,"score_total":null,"note":"Preserves the requested nested statement hierarchy instead of flattening it, with separate metadata, account_holder, account, branch, balances, transactions, summary, and other top-level sections."}]}