{"observation":{"id":"a76bbf6a-d3bf-4614-88df-e466d93830b7","tool":"askyourdatabase","tool_name":"AskYourDatabase","criterion":"plain-english-query-handling","criterion_name":"Plain English Query Handling","criterion_definition":"Can the tool understand business questions without SQL?","criterion_evidence_type":"transformation","criterion_rank_role":null,"criterion_rank_role_reason":null,"scenario":"best-customers-with-unpaid-order-and-payment-method-follow-ups","scenario_name":"Best customers with unpaid-order and payment-method follow-ups","group_tag":"ecommerce-nl2sql-benchmark","scenario_description":"A conversational multi-table customer analysis that identifies best customers by both order volume and spend, then drills into unpaid orders and payment methods for the top 3.","modality":"text","input_text":"Who are my best customers — the ones who order the most and spend the most?\n\nFollow-up 1: For the top 3 from that list — do any of them have unpaid orders?\n\nFollow-up 2: What payment methods do these top 3 usually use?","input_artifact_refs":["AskYourDatabase_Best_Customers_SQL_Visible-2.png","AskYourDatabase_Top_Customers_Unpaid_Orders-2.png","Draxlr_Top_Customers_Payment_Methods.png"],"stresses":["Ambiguous business-term interpretation","Multi-table joins across customers orders and payments","Aggregation and ranking","Follow-up context retention","Scoped drill-down to the top 3 customers","Readable customer-level output"],"verdict":"worked","score":null,"score_total":null,"note":"Accepted an informal customer-analytics question and split it into two dimensions without requiring SQL: who orders most and who spends most.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/askyourdatabase-image-3-901a942fcd93.png","role":null,"alt":null}],"run_id":"db2bb5d5-0e0e-4cb3-8d76-3555c45c23cd","study_title":"Query Live Databases Using Plain English with AI","study_kind":"generation","research_task":"86b9y6c99","tested_at":null,"completeness":"input-and-output","input":{"state":"text","text":"Who are my best customers — the ones who order the most and spend the most?\n\nFollow-up 1: For the top 3 from that list — do any of them have unpaid orders?\n\nFollow-up 2: What payment methods do these top 3 usually use?","files":[],"modality":"text","stresses":["Ambiguous business-term interpretation","Multi-table joins across customers orders and payments","Aggregation and ranking","Follow-up context retention","Scoped drill-down to the top 3 customers","Readable customer-level output"]},"tool_page_slug":"askyourdatabase","tool_url":"https://aidemos.com/tools/askyourdatabase","permalink":"https://aidemos.com/evidence/a76bbf6a-d3bf-4614-88df-e466d93830b7","api_url":"https://ai.aidemos.com/v1/observations/a76bbf6a-d3bf-4614-88df-e466d93830b7"},"peers":[{"id":"e2c2aa98-4929-4513-a2fb-47aeb04b5b48","tool":"querio","tool_name":"Querio","verdict":"worked","score":null,"score_total":null,"note":"Handles an informal conversational request about top customers without needing the user to translate it into SQL.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/e2c2aa98-4929-4513-a2fb-47aeb04b5b48"}],"other_criteria":[{"id":"87bac5d0-cd93-46de-a5fa-5daec5d3d86d","criterion":"ambiguity-handling","criterion_name":"Ambiguity Handling","rank_role":null,"verdict":"worked","score":null,"score_total":null,"note":"Handled the ambiguous phrase by not collapsing it into one ranking; it produced separate order-count and total-spend rankings instead of guessing silently.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/87bac5d0-cd93-46de-a5fa-5daec5d3d86d"},{"id":"820cfe6d-d216-49fe-bb1f-78694880d4d4","criterion":"business-insight","criterion_name":"Business Insight","rank_role":null,"verdict":"worked","score":null,"score_total":null,"note":"The follow-up reasoning went beyond listing payments by flagging Mohan Vishe's unpaid shipped orders as a process risk and noting that Rahul Sharma's pattern was largely UPI-driven.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/820cfe6d-d216-49fe-bb1f-78694880d4d4"},{"id":"16753552-a29f-4dca-bb73-44bd7c26065f","criterion":"business-insight","criterion_name":"Business Insight","rank_role":null,"verdict":"worked","score":null,"score_total":null,"note":"Generated useful takeaways automatically, including Rahul Sharma as the standout all-rounder, Mohan Vishe as the most loyal frequent buyer, and Vikram Singh as the largest spender.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/16753552-a29f-4dca-bb73-44bd7c26065f"},{"id":"88894e72-df01-4f3d-bf3f-a8bcefcc91b1","criterion":"chart-visualization-support","criterion_name":"Chart / Visualization Support","rank_role":null,"verdict":"failed","score":null,"score_total":null,"note":"Visualization did not auto-generate on this customer analysis flow; the report says it required an additional prompt every time.","artifact_count":0,"evidence_url":"https://aidemos.com/evidence/88894e72-df01-4f3d-bf3f-a8bcefcc91b1"},{"id":"0f379e1b-c980-49c7-9d60-df438430e631","criterion":"follow-up-context","criterion_name":"Follow-Up Context","rank_role":null,"verdict":"worked","score":null,"score_total":null,"note":"Kept the top-3 customer context across the follow-up chain by hardcoding the same three customer IDs into the unpaid-order check.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/0f379e1b-c980-49c7-9d60-df438430e631"},{"id":"4a24d35a-e481-4ca2-bbed-3a5c9c86e0ea","criterion":"follow-up-context","criterion_name":"Follow-Up Context","rank_role":null,"verdict":"worked","score":null,"score_total":null,"note":"The tool preserved conversational context across turns by reusing the previously identified top three customers in the unpaid-order follow-up.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/4a24d35a-e481-4ca2-bbed-3a5c9c86e0ea"},{"id":"e62d9777-e371-46ea-939d-39f2db2b1909","criterion":"result-readability","criterion_name":"Result Readability","rank_role":null,"verdict":"worked","score":null,"score_total":null,"note":"Presented the follow-up results as clearly labeled customer status blocks such as 'All Clear', '1 Unpaid', and 'All 4 Unpaid!', which makes the risk scan easy.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/e62d9777-e371-46ea-939d-39f2db2b1909"},{"id":"05dbced4-de87-4e7a-a1bf-84da4f7a6be8","criterion":"result-readability","criterion_name":"Result Readability","rank_role":null,"verdict":"worked","score":null,"score_total":null,"note":"The customer ranking output was easy to scan because it separated frequent shoppers from highest spenders into clearly named tables with totals and ranks.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/05dbced4-de87-4e7a-a1bf-84da4f7a6be8"},{"id":"d379e546-f71f-42c9-9b76-510e9a9b61bc","criterion":"sql-generation","criterion_name":"SQL Generation","rank_role":null,"verdict":"worked","score":null,"score_total":null,"note":"Ran 2 SQL queries simultaneously for the main request, and then generated a filtered follow-up query for the exact top 3 customers.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/d379e546-f71f-42c9-9b76-510e9a9b61bc"}],"appears_in":[{"page_type":"use-case","slug":"query-live-databases","title":"Query Live Databases Using Plain English with AI","url":"https://aidemos.com/use-cases/query-live-databases","binding":"run"},{"page_type":"ranking","slug":"text-to-sql-tools","title":"Best AI Tools to Query Live Databases Using Plain English","url":"https://aidemos.com/best/text-to-sql-tools","binding":"study"}],"same_scenario":[{"id":"e2c2aa98-4929-4513-a2fb-47aeb04b5b48","tool":"querio","tool_name":"Querio","verdict":"worked","score":null,"score_total":null,"note":"Handles an informal conversational request about top customers without needing the user to translate it into SQL."}]}