{"observation":{"id":"7b3d81c6-ee20-4e87-9fb9-921655183bd5","tool":"definite","tool_name":"Definite","criterion":"business-insight","criterion_name":"Business Insight","criterion_definition":"Does it explain what the result means?","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Explaining what the result means is part of making the database answer actually useful to a business user. (2 of 3 judges)","scenario":"best-customers-with-unpaid-order-and-payment-method-follow-ups","scenario_name":"Best customers with unpaid-order and payment-method follow-ups","group_tag":"live-database-plain-english-queries","scenario_description":"A conversational multi-table customer analysis with follow-up questions. It asks for the best customers by both order volume and spend, then drills into unpaid orders for the top 3 and their usual payment methods. Designed to test ranking logic, join-heavy analysis, and follow-up context retention.","modality":"text","input_text":"Who are my best customers — the ones who order the most and spend the most?\n\nFollow-up 1: For the top 3 from that list — do any of them have unpaid orders?\n\nFollow-up 2: What payment methods do these top 3 usually use?","input_artifact_refs":[],"stresses":["ambiguous business term interpretation","multi-table joins","aggregation and ranking","follow-up context retention","scoping to a selected subset","payment behavior analysis"],"verdict":"worked","score":null,"score_total":null,"note":"It adds useful interpretation by flagging Rahul Sharma's unpaid $2,199 order, noting it had been confirmed but unpaid since April 2025, and linking his payment methods to the issue as likely an oversight rather than a pattern.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/4ecf960c56f643f48609e0a54c971906.png?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/e589576ff3454ec4917a7588a94ad54b.png?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/0ff08babcc2b49b2b7110bd20fcc131d.mp4?v=1","role":"output","alt":null}],"run_id":"af2abc96-3311-484b-a19d-854a2fdd2bf3","study_title":"Query Live Databases Using Plain English with AI","study_kind":"generation","research_task":"86b9y6c99","tested_at":null,"completeness":"input-and-output","input":{"state":"text","text":"Who are my best customers — the ones who order the most and spend the most?\n\nFollow-up 1: For the top 3 from that list — do any of them have unpaid orders?\n\nFollow-up 2: What payment methods do these top 3 usually use?","files":[],"modality":"text","stresses":["ambiguous business term interpretation","multi-table joins","aggregation and ranking","follow-up context retention","scoping to a selected subset","payment behavior analysis"]},"tool_page_slug":"definite","tool_url":"https://aidemos.com/tools/definite","permalink":"https://aidemos.com/evidence/7b3d81c6-ee20-4e87-9fb9-921655183bd5","api_url":"https://ai.aidemos.com/v1/observations/7b3d81c6-ee20-4e87-9fb9-921655183bd5"},"peers":[{"id":"239f04d5-19bb-454b-bebc-39db63df6c63","tool":"askyourdatabase","tool_name":"AskYourDatabase","verdict":"worked","score":null,"score_total":null,"note":"It adds interpretation such as Rahul Sharma being the all-rounder and Mohan Vishe being a payment-risk red flag, rather than just listing rows.","artifact_count":4,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/ec0ed41fe3dd45b5b13b082d174be6b7.png?v=1","evidence_url":"https://aidemos.com/evidence/239f04d5-19bb-454b-bebc-39db63df6c63"},{"id":"12691fa1-bfda-47ed-8398-2622c0de0b66","tool":"basedash","tool_name":"Basedash","verdict":"worked","score":null,"score_total":null,"note":"It explains what the numbers mean by naming Rahul Sharma as the best overall customer, Mohan Vishe as the most frequent, and Deepak Kulkarni as the biggest spender.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/b32adc19ccfc434bb534d9c3684fee55.png?v=1","evidence_url":"https://aidemos.com/evidence/12691fa1-bfda-47ed-8398-2622c0de0b66"},{"id":"bc2aee6c-97d9-4eb4-845a-d4e6cb8db1df","tool":"blazesql","tool_name":"BlazeSQL","verdict":"worked","score":null,"score_total":null,"note":"It explains the follow-up data in business terms, including that Rahul Sharma has 1 unpaid order worth $2,199 in CONFIRMED status while Deepak Kulkarni and Karan Joshi are fully paid up.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/8659db3b02aa49d68b26dd7e6267c998.png?v=1","evidence_url":"https://aidemos.com/evidence/bc2aee6c-97d9-4eb4-845a-d4e6cb8db1df"},{"id":"28716eba-141e-4cde-8b6c-528960426776","tool":"camelai","tool_name":"camelAI","verdict":"mixed","score":null,"score_total":null,"note":"It gives a one-line business takeaway by naming Rahul Sharma as the strongest frequency-and-value combination, but it stops there without any deeper interpretation or recommendation.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/1c1df761c0854b5da1ff79f846fd34b9.png?v=1","evidence_url":"https://aidemos.com/evidence/28716eba-141e-4cde-8b6c-528960426776"},{"id":"86d2a8b3-d9d0-4b8e-8e46-8bc17c6b5c6d","tool":"dot","tool_name":"Dot","verdict":"worked","score":null,"score_total":null,"note":"It adds interpretation instead of only listing rows: Rahul is framed as the strongest all-around customer, and the tool warns that Deepak and Karan's 'usual' payment method is not yet a stable preference because each has only one order.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/c126c58ac82e4d899b11a7006a73d94d.png?v=1","evidence_url":"https://aidemos.com/evidence/86d2a8b3-d9d0-4b8e-8e46-8bc17c6b5c6d"},{"id":"d2139d74-3e9f-4b36-a3f8-519598bf53d9","tool":"draxlr","tool_name":"Draxlr","verdict":"struggled","score":null,"score_total":null,"note":"It produced query descriptions, but no after-the-table narrative that distilled the result into a takeaway like which top customer had unpaid orders.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/565a041698d44f05ad75ba27fe1fd48d.png?v=1","evidence_url":"https://aidemos.com/evidence/d2139d74-3e9f-4b36-a3f8-519598bf53d9"},{"id":"e91c9860-3478-4496-b5ac-5bbd4f47b93b","tool":"futuresmart-nl2sql-agent","tool_name":"FutureSmart NL2SQL Agent","verdict":"mixed","score":null,"score_total":null,"note":"The payment-method follow-up gives a concise dominant-method insight for the top customers, but the result is incomplete as a business answer because it does not surface the full payment-method picture implied by the customer totals.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/a7df6cf0481642a4b37828421e617f13.png?v=1","evidence_url":"https://aidemos.com/evidence/e91c9860-3478-4496-b5ac-5bbd4f47b93b"}],"other_criteria":[{"id":"0c9c2717-901f-4b0f-a114-233eda2981c1","criterion":"ambiguity-handling","criterion_name":"Ambiguity Handling","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"It did not clarify the ambiguous 'best customers' wording; it silently chose spend-based ranking and ignored the order-frequency part of the question.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/0c9c2717-901f-4b0f-a114-233eda2981c1"},{"id":"84d433cb-4227-4de7-ba67-e1a80fda66e4","criterion":"follow-up-context","criterion_name":"Follow-Up Context","rank_role":"context","verdict":"worked","score":null,"score_total":null,"note":"It retained the top-3 customer context across both follow-ups, answering unpaid-order and payment-method questions for the same three customers from the initial list.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/84d433cb-4227-4de7-ba67-e1a80fda66e4"},{"id":"736c4cc8-4b83-445e-b321-4bf1c78cd426","criterion":"plain-english-query-handling","criterion_name":"Plain English Query Handling","rank_role":"decisive","verdict":"mixed","score":null,"score_total":null,"note":"It accepted the natural-language request but only ranked customers by total spend, so it did not fully understand the combined 'order the most and spend the most' intent.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/736c4cc8-4b83-445e-b321-4bf1c78cd426"},{"id":"cfe90f8a-a4cf-4c44-bafb-2e6f73ca7a40","criterion":"result-readability","criterion_name":"Result Readability","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"It returns a ranked table with order counts, total spend, and average order value that is straightforward to read and scan.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/cfe90f8a-a4cf-4c44-bafb-2e6f73ca7a40"}],"appears_in":[{"page_type":"ranking","slug":"ai-database-query-tools","title":"Best AI Tools to Query Live Databases Using Plain English","url":"https://aidemos.com/best/ai-database-query-tools","binding":"run"},{"page_type":"ranking","slug":"text-to-sql-tools","title":"Best AI Tools to Query Live Databases Using Plain English","url":"https://aidemos.com/best/text-to-sql-tools","binding":"study"},{"page_type":"use-case","slug":"query-live-databases","title":"Query Live Databases Using Plain English with AI","url":"https://aidemos.com/use-cases/query-live-databases","binding":"study"}],"same_scenario":[{"id":"239f04d5-19bb-454b-bebc-39db63df6c63","tool":"askyourdatabase","tool_name":"AskYourDatabase","verdict":"worked","score":null,"score_total":null,"note":"It adds interpretation such as Rahul Sharma being the all-rounder and Mohan Vishe being a payment-risk red flag, rather than just listing rows."},{"id":"12691fa1-bfda-47ed-8398-2622c0de0b66","tool":"basedash","tool_name":"Basedash","verdict":"worked","score":null,"score_total":null,"note":"It explains what the numbers mean by naming Rahul Sharma as the best overall customer, Mohan Vishe as the most frequent, and Deepak Kulkarni as the biggest spender."},{"id":"bc2aee6c-97d9-4eb4-845a-d4e6cb8db1df","tool":"blazesql","tool_name":"BlazeSQL","verdict":"worked","score":null,"score_total":null,"note":"It explains the follow-up data in business terms, including that Rahul Sharma has 1 unpaid order worth $2,199 in CONFIRMED status while Deepak Kulkarni and Karan Joshi are fully paid up."},{"id":"28716eba-141e-4cde-8b6c-528960426776","tool":"camelai","tool_name":"camelAI","verdict":"mixed","score":null,"score_total":null,"note":"It gives a one-line business takeaway by naming Rahul Sharma as the strongest frequency-and-value combination, but it stops there without any deeper interpretation or recommendation."},{"id":"86d2a8b3-d9d0-4b8e-8e46-8bc17c6b5c6d","tool":"dot","tool_name":"Dot","verdict":"worked","score":null,"score_total":null,"note":"It adds interpretation instead of only listing rows: Rahul is framed as the strongest all-around customer, and the tool warns that Deepak and Karan's 'usual' payment method is not yet a stable preference because each has only one order."},{"id":"d2139d74-3e9f-4b36-a3f8-519598bf53d9","tool":"draxlr","tool_name":"Draxlr","verdict":"struggled","score":null,"score_total":null,"note":"It produced query descriptions, but no after-the-table narrative that distilled the result into a takeaway like which top customer had unpaid orders."},{"id":"e91c9860-3478-4496-b5ac-5bbd4f47b93b","tool":"futuresmart-nl2sql-agent","tool_name":"FutureSmart NL2SQL Agent","verdict":"mixed","score":null,"score_total":null,"note":"The payment-method follow-up gives a concise dominant-method insight for the top customers, but the result is incomplete as a business answer because it does not surface the full payment-method picture implied by the customer totals."}]}