{"observation":{"id":"8631a190-2b78-4695-a457-2b0ac633578c","tool":"basedash","tool_name":"Basedash","criterion":"result-readability","criterion_name":"Result Readability","criterion_definition":"Is the answer easy for a non-technical user to understand?","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"The point is to get usable answers from non-technical users, so the result has to be understandable. (3 of 3 judges)","scenario":"best-customers-with-unpaid-order-and-payment-method-follow-ups","scenario_name":"Best customers with unpaid-order and payment-method follow-ups","group_tag":"live-database-plain-english-queries","scenario_description":"A conversational multi-table customer analysis with follow-up questions. It asks for the best customers by both order volume and spend, then drills into unpaid orders for the top 3 and their usual payment methods. Designed to test ranking logic, join-heavy analysis, and follow-up context retention.","modality":"text","input_text":"Who are my best customers — the ones who order the most and spend the most?\n\nFollow-up 1: For the top 3 from that list — do any of them have unpaid orders?\n\nFollow-up 2: What payment methods do these top 3 usually use?","input_artifact_refs":[],"stresses":["ambiguous business term interpretation","multi-table joins","aggregation and ranking","follow-up context retention","scoping to a selected subset","payment behavior analysis"],"verdict":"worked","score":null,"score_total":null,"note":"The output stays readable for non-technical users, using compact tables and short summaries with explicit customer names and amounts.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://cdn.futuresmart.ai/public/aidemos/b32adc19ccfc434bb534d9c3684fee55.png?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/1d5ee66912954067955529095f49f9ef.png?v=1","role":"output","alt":null},{"url":"https://cdn.futuresmart.ai/public/aidemos/e446d0843d2b4891a9889ffb47c32d96.png?v=1","role":"output","alt":null}],"run_id":"af2abc96-3311-484b-a19d-854a2fdd2bf3","study_title":"Query Live Databases Using Plain English with AI","study_kind":"generation","research_task":"86b9y6c99","tested_at":null,"completeness":"input-and-output","input":{"state":"text","text":"Who are my best customers — the ones who order the most and spend the most?\n\nFollow-up 1: For the top 3 from that list — do any of them have unpaid orders?\n\nFollow-up 2: What payment methods do these top 3 usually use?","files":[],"modality":"text","stresses":["ambiguous business term interpretation","multi-table joins","aggregation and ranking","follow-up context retention","scoping to a selected subset","payment behavior analysis"]},"tool_page_slug":"basedash","tool_url":"https://aidemos.com/tools/basedash","permalink":"https://aidemos.com/evidence/8631a190-2b78-4695-a457-2b0ac633578c","api_url":"https://ai.aidemos.com/v1/observations/8631a190-2b78-4695-a457-2b0ac633578c"},"peers":[{"id":"6a130b11-3ffe-4dd1-b405-30c887af3d65","tool":"askyourdatabase","tool_name":"AskYourDatabase","verdict":"worked","score":null,"score_total":null,"note":"It presents the answer as clearly labeled ranking tables and customer-level payment summaries, with visual risk cues for unpaid or shipped orders.","artifact_count":4,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/ec0ed41fe3dd45b5b13b082d174be6b7.png?v=1","evidence_url":"https://aidemos.com/evidence/6a130b11-3ffe-4dd1-b405-30c887af3d65"},{"id":"cfe90f8a-a4cf-4c44-bafb-2e6f73ca7a40","tool":"definite","tool_name":"Definite","verdict":"worked","score":null,"score_total":null,"note":"It returns a ranked table with order counts, total spend, and average order value that is straightforward to read and scan.","artifact_count":1,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/852010033067408ba7a475364358d97f.png?v=1","evidence_url":"https://aidemos.com/evidence/cfe90f8a-a4cf-4c44-bafb-2e6f73ca7a40"},{"id":"ee22d062-e8e0-4ac9-8870-8cca0db9d8ec","tool":"dot","tool_name":"Dot","verdict":"mixed","score":null,"score_total":null,"note":"The follow-up answers are readable but terse: each is a one-line response, while the supporting table and SQL are pushed out of the main surface into Full logs.","artifact_count":2,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/e94fab739e414924b69bd67beb632d2f.png?v=1","evidence_url":"https://aidemos.com/evidence/ee22d062-e8e0-4ac9-8870-8cca0db9d8ec"},{"id":"492b3c4c-2b5a-42db-80b7-7b8f2464e506","tool":"draxlr","tool_name":"Draxlr","verdict":"struggled","score":null,"score_total":null,"note":"All three best-customer result tables keep customer_id as the lead column, so the output stays UUID-heavy and hard to scan.","artifact_count":3,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/ab41b8340d0a45188a25417725b1cede.png?v=1","evidence_url":"https://aidemos.com/evidence/492b3c4c-2b5a-42db-80b7-7b8f2464e506"},{"id":"ac7d4ce7-d376-4989-bb44-e8abdd293fbf","tool":"querio","tool_name":"Querio","verdict":"mixed","score":null,"score_total":null,"note":"The answer is usable, but it is less readable than it could be because the output centers customer UUIDs instead of human names.","artifact_count":4,"thumbnail":"https://cdn.futuresmart.ai/public/aidemos/a2212df102a643b4bd41951fd064776d.png?v=1","evidence_url":"https://aidemos.com/evidence/ac7d4ce7-d376-4989-bb44-e8abdd293fbf"}],"other_criteria":[{"id":"099c140b-33a3-43e3-a1b4-33e60dba5b32","criterion":"ambiguity-handling","criterion_name":"Ambiguity Handling","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"It did not ask what 'top 3 from that list' meant; instead it silently chose the spend ranking first and then added the order-count check.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/099c140b-33a3-43e3-a1b4-33e60dba5b32"},{"id":"12691fa1-bfda-47ed-8398-2622c0de0b66","criterion":"business-insight","criterion_name":"Business Insight","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"It explains what the numbers mean by naming Rahul Sharma as the best overall customer, Mohan Vishe as the most frequent, and Deepak Kulkarni as the biggest spender.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/12691fa1-bfda-47ed-8398-2622c0de0b66"},{"id":"56f455ce-aca2-4d40-bfe0-67dd5955602e","criterion":"follow-up-context","criterion_name":"Follow-Up Context","rank_role":"context","verdict":"mixed","score":null,"score_total":null,"note":"It remembered enough of the prior answer to check both ranking lists, but it still narrowed the follow-up instead of preserving the user's intended scope cleanly.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/56f455ce-aca2-4d40-bfe0-67dd5955602e"},{"id":"2bd93ade-7def-4250-ad0a-dd4f54aafd4a","criterion":"plain-english-query-handling","criterion_name":"Plain English Query Handling","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"It correctly handled a conversational, multi-part customer question without requiring SQL, producing two rankings plus follow-up answers.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/2bd93ade-7def-4250-ad0a-dd4f54aafd4a"},{"id":"bff55c46-ffad-430e-a91c-0a9b7651909a","criterion":"sql-generation","criterion_name":"SQL Generation","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"It successfully generated the ranking queries and follow-up lookups, including the top spenders, top order-count customers, unpaid-order status, and usual payment methods.","artifact_count":3,"evidence_url":"https://aidemos.com/evidence/bff55c46-ffad-430e-a91c-0a9b7651909a"}],"appears_in":[{"page_type":"ranking","slug":"ai-database-query-tools","title":"Best AI Tools to Query Live Databases Using Plain English","url":"https://aidemos.com/best/ai-database-query-tools","binding":"run"},{"page_type":"ranking","slug":"text-to-sql-tools","title":"Best AI Tools to Query Live Databases Using Plain English","url":"https://aidemos.com/best/text-to-sql-tools","binding":"study"},{"page_type":"use-case","slug":"query-live-databases","title":"Query Live Databases Using Plain English with AI","url":"https://aidemos.com/use-cases/query-live-databases","binding":"study"}],"same_scenario":[{"id":"6a130b11-3ffe-4dd1-b405-30c887af3d65","tool":"askyourdatabase","tool_name":"AskYourDatabase","verdict":"worked","score":null,"score_total":null,"note":"It presents the answer as clearly labeled ranking tables and customer-level payment summaries, with visual risk cues for unpaid or shipped orders."},{"id":"cfe90f8a-a4cf-4c44-bafb-2e6f73ca7a40","tool":"definite","tool_name":"Definite","verdict":"worked","score":null,"score_total":null,"note":"It returns a ranked table with order counts, total spend, and average order value that is straightforward to read and scan."},{"id":"ee22d062-e8e0-4ac9-8870-8cca0db9d8ec","tool":"dot","tool_name":"Dot","verdict":"mixed","score":null,"score_total":null,"note":"The follow-up answers are readable but terse: each is a one-line response, while the supporting table and SQL are pushed out of the main surface into Full logs."},{"id":"492b3c4c-2b5a-42db-80b7-7b8f2464e506","tool":"draxlr","tool_name":"Draxlr","verdict":"struggled","score":null,"score_total":null,"note":"All three best-customer result tables keep customer_id as the lead column, so the output stays UUID-heavy and hard to scan."},{"id":"ac7d4ce7-d376-4989-bb44-e8abdd293fbf","tool":"querio","tool_name":"Querio","verdict":"mixed","score":null,"score_total":null,"note":"The answer is usable, but it is less readable than it could be because the output centers customer UUIDs instead of human names."}]}