{"observation":{"id":"3638d045-d475-41e7-8277-b423ad99a540","tool":"landing-ai","tool_name":"Landing AI","criterion":"table-record-completeness","criterion_name":"Table & Record Completeness","criterion_definition":"Are all tabular rows and line items extracted without merging, duplication, omission, or phantom records?","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Missing, duplicated, or merged rows and line items directly corrupt the structured dataset and break query results. (3 of 3 judges)","scenario":"invoice-pdf","scenario_name":"Invoice PDF","group_tag":"business-document-extraction","scenario_description":"A two-page broadcast advertising invoice PDF with nested header metadata, billing and remit addresses, and eight line items spanning a page break. It was used to stress hierarchical line-item extraction, amount precision, time/day parsing, code extraction, and summary validation.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://cdn.futuresmart.ai/public/aidemos/e0e9f678bcde4e29a492a11927fc38f7.pdf?v=1","role":"input","filename":"Invoice PDF.pdf"}],"stresses":["Nested line-item hierarchy extraction","Multi-page continuity across a page break","Precision on large dollar amounts and totals","Parsing complex time slots and day patterns","Extraction of Ad IDs and reconciliation codes","Mapping structured metadata sections correctly","Financial summary validation","Handling political advertising compliance text"],"verdict":"worked","score":null,"score_total":null,"note":"Extracts all 8 advertising line items as separate records, preserving the page-break continuation instead of merging or omitting rows.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/research-media-landing-ai-invoice-extracted-line-items-5af44b5e570e.png","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/90a9cfea70bb42c1bc3838858f1543dd.png?v=1","role":"output","alt":null}],"run_id":"a061b9e7-a9c5-443d-a171-b296aaf51b8c","study_title":"Extract and query structured data from documents using natural language","study_kind":"generation","research_task":"86b9y25e5","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://cdn.futuresmart.ai/public/aidemos/e0e9f678bcde4e29a492a11927fc38f7.pdf?v=1","filename":"Invoice PDF.pdf","alt":"Invoice PDF","role":"input"}],"modality":"pdf","stresses":["Nested line-item hierarchy extraction","Multi-page continuity across a page break","Precision on large dollar amounts and totals","Parsing complex time slots and day patterns","Extraction of Ad IDs and reconciliation codes","Mapping structured metadata sections correctly","Financial summary validation","Handling political advertising compliance text"]},"tool_page_slug":"landing-ai","tool_url":"https://aidemos.com/tools/landing-ai","permalink":"https://aidemos.com/evidence/3638d045-d475-41e7-8277-b423ad99a540","api_url":"https://ai.aidemos.com/v1/observations/3638d045-d475-41e7-8277-b423ad99a540"},"peers":[{"id":"13edb4ed-0752-4b00-afa4-f5afd50a9f3b","tool":"datalab","tool_name":"Datalab","verdict":"worked","score":null,"score_total":null,"note":"Extracts all eight invoice line items as separate records without merging, including rows that span the page break.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-datalab-invoice-line-items-d9e385e05d92.png","evidence_url":"https://aidemos.com/evidence/13edb4ed-0752-4b00-afa4-f5afd50a9f3b"},{"id":"e758fe02-058e-420a-a354-2e050be0d1a1","tool":"docsumo","tool_name":"Docsumo","verdict":"worked","score":null,"score_total":null,"note":"The line-item table keeps all 8 rows, including the row that spans the page break, without introducing a duplicate line item.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/9beeb894cf5d41ce8390db3f758c1a98.png?v=1","evidence_url":"https://aidemos.com/evidence/e758fe02-058e-420a-a354-2e050be0d1a1"},{"id":"18ef9ee1-fba7-4665-a17e-e1951b8b0758","tool":"extend-ai","tool_name":"Extend AI","verdict":"worked","score":null,"score_total":null,"note":"It extracts all 8 advertising line items as separate records, preserving item boundaries across the page break without merging adjacent rows.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-extendai-extracted-invoice-line-items-a11681b31146.png","evidence_url":"https://aidemos.com/evidence/18ef9ee1-fba7-4665-a17e-e1951b8b0758"},{"id":"5c4af38f-4e13-4a9f-a53f-7c44e253ff56","tool":"llamaparse","tool_name":"LlamaParse","verdict":"failed","score":null,"score_total":null,"note":"The tool extracts invoice line items as structured rows, but it introduces an incorrect 8th line item with line_number 9, so the table record set is not fully faithful to the source invoice’s eight advertising records.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-llamaparse-invoice-extracted-line-item-8-ddd5553310ea.png","evidence_url":"https://aidemos.com/evidence/5c4af38f-4e13-4a9f-a53f-7c44e253ff56"},{"id":"9501f1c6-ecdd-4252-bf26-6fa252a44f29","tool":"nanonets","tool_name":"Nanonets","verdict":"worked","score":null,"score_total":null,"note":"Keeps all 8 advertising line items as separate records with no row merging or duplication.","artifact_count":3,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-docstrange-nanonets-extracted-line-items-5ec513388423.png","evidence_url":"https://aidemos.com/evidence/9501f1c6-ecdd-4252-bf26-6fa252a44f29"},{"id":"95c46d00-e947-4dd4-86ed-954cc46fac3c","tool":"reducto","tool_name":"Reducto","verdict":"worked","score":null,"score_total":null,"note":"Extracts all eight advertising line items as separate records and preserves the page-spanning table without merging adjacent rows.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-reducto-invoice-extracted-line0items-e23fe4fdf946.png","evidence_url":"https://aidemos.com/evidence/95c46d00-e947-4dd4-86ed-954cc46fac3c"},{"id":"dcbeb5fd-b152-4212-82e3-048c6a20b6d6","tool":"retab","tool_name":"Retab","verdict":"worked","score":null,"score_total":null,"note":"Extracts all 8 advertising line items as separate structured records without duplication or omission.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-retab-invoice-extracted-line-items-b376a1b8acd0.png","evidence_url":"https://aidemos.com/evidence/dcbeb5fd-b152-4212-82e3-048c6a20b6d6"},{"id":"d025ae3c-59d0-423e-b723-7bfdbdfd3f34","tool":"unstract","tool_name":"Unstract","verdict":"worked","score":null,"score_total":null,"note":"Extracts the line-item table without losing the row that crosses the page break; line 8 is returned in full with no duplication or data loss.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/1f84ee4b4a7148df9a8abec29b0bcd88.png?v=1","evidence_url":"https://aidemos.com/evidence/d025ae3c-59d0-423e-b723-7bfdbdfd3f34"}],"other_criteria":[{"id":"b808d6bb-8133-43cb-8a88-7d5e3160cf25","criterion":"extraction-accuracy","criterion_name":"Extraction Accuracy","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"Misreads at least one alphanumeric identifier: line 1's source Ad-ID NRCCWI071005 is extracted as NRCCW1071005.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/b808d6bb-8133-43cb-8a88-7d5e3160cf25"},{"id":"1d681a8e-4d71-4675-8ec6-e582d1a6cad5","criterion":"extraction-accuracy","criterion_name":"Extraction Accuracy","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Extracts invoice metadata and totals consistently, including invoice_number \"4064621-1\", invoice_date \"10/28/12\", aired_spots 8, gross_total 29750, and net_amount_due 25287.5.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/1d681a8e-4d71-4675-8ec6-e582d1a6cad5"},{"id":"50b53c14-a574-47ae-acad-7924694462b2","criterion":"extraction-accuracy","criterion_name":"Extraction Accuracy","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"Leaves several station-level fields null—address, city, state, postal_code, main_phone, and billing_phone—even though the station block is present in the schema and related info exists elsewhere in the invoice.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/50b53c14-a574-47ae-acad-7924694462b2"},{"id":"a1af2980-0fa9-4aa3-910d-7dcdb0e75178","criterion":"schema-adherence","criterion_name":"Schema Adherence","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Reconstructs the invoice hierarchy into invoice_metadata, advertiser, station, billing_address, remit_address, flight_dates, line_items, and summary objects rather than returning flat OCR text.","artifact_count":4,"evidence_url":"https://aidemos.com/evidence/a1af2980-0fa9-4aa3-910d-7dcdb0e75178"},{"id":"cff21b29-9003-4011-b627-44d5e3ba01dd","criterion":"semantic-field-enrichment","criterion_name":"Semantic Field Enrichment","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"Populates derived scheduling fields such as day_of_week \"M\" and days_pattern \"MTWT\" on invoice line items.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/cff21b29-9003-4011-b627-44d5e3ba01dd"},{"id":"dbf84eab-a8e4-49e1-b7a5-ddf3319de6bc","criterion":"structural-clean-output","criterion_name":"Structural Clean Output","rank_role":"context","verdict":"worked","score":null,"score_total":null,"note":"Returns directly consumable JSON for the invoice workflow, with the demo reaching extracted results and the report stating the JSON is immediately usable for integration.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/dbf84eab-a8e4-49e1-b7a5-ddf3319de6bc"}],"appears_in":[{"page_type":"ranking","slug":"document-extraction","title":"Best AI Tools for Extracting Structured Data from Business Documents","url":"https://aidemos.com/best/document-extraction","binding":"run"},{"page_type":"tool","slug":"docsumo","title":null,"url":"https://aidemos.com/tools/docsumo","binding":"run"}],"same_scenario":[{"id":"13edb4ed-0752-4b00-afa4-f5afd50a9f3b","tool":"datalab","tool_name":"Datalab","verdict":"worked","score":null,"score_total":null,"note":"Extracts all eight invoice line items as separate records without merging, including rows that span the page break."},{"id":"e758fe02-058e-420a-a354-2e050be0d1a1","tool":"docsumo","tool_name":"Docsumo","verdict":"worked","score":null,"score_total":null,"note":"The line-item table keeps all 8 rows, including the row that spans the page break, without introducing a duplicate line item."},{"id":"18ef9ee1-fba7-4665-a17e-e1951b8b0758","tool":"extend-ai","tool_name":"Extend AI","verdict":"worked","score":null,"score_total":null,"note":"It extracts all 8 advertising line items as separate records, preserving item boundaries across the page break without merging adjacent rows."},{"id":"5c4af38f-4e13-4a9f-a53f-7c44e253ff56","tool":"llamaparse","tool_name":"LlamaParse","verdict":"failed","score":null,"score_total":null,"note":"The tool extracts invoice line items as structured rows, but it introduces an incorrect 8th line item with line_number 9, so the table record set is not fully faithful to the source invoice’s eight advertising records."},{"id":"9501f1c6-ecdd-4252-bf26-6fa252a44f29","tool":"nanonets","tool_name":"Nanonets","verdict":"worked","score":null,"score_total":null,"note":"Keeps all 8 advertising line items as separate records with no row merging or duplication."},{"id":"95c46d00-e947-4dd4-86ed-954cc46fac3c","tool":"reducto","tool_name":"Reducto","verdict":"worked","score":null,"score_total":null,"note":"Extracts all eight advertising line items as separate records and preserves the page-spanning table without merging adjacent rows."},{"id":"dcbeb5fd-b152-4212-82e3-048c6a20b6d6","tool":"retab","tool_name":"Retab","verdict":"worked","score":null,"score_total":null,"note":"Extracts all 8 advertising line items as separate structured records without duplication or omission."},{"id":"d025ae3c-59d0-423e-b723-7bfdbdfd3f34","tool":"unstract","tool_name":"Unstract","verdict":"worked","score":null,"score_total":null,"note":"Extracts the line-item table without losing the row that crosses the page break; line 8 is returned in full with no duplication or data loss."}]}