{"observation":{"id":"fcafc622-fe49-4c1a-88e4-ecbbfb5ef7d5","tool":"parseur","tool_name":"Parseur","criterion":"output-quality","criterion_name":"Output quality","criterion_definition":"Is the parsed result clean, complete, and usable overall?","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"The whole point is usable structured extraction; clean, complete, usable output is the core measure of success. (3 of 3 judges)","scenario":"messy-real-world-resume-john-kumar","scenario_name":"Messy real-world resume — John Kumar","group_tag":"resume-parsing","scenario_description":"A poorly formatted, inconsistent real-world resume for John Kumar, used to test robustness against noisy structure, inconsistent dates, and mixed-content sections.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://d3epheqghktydj.cloudfront.net/parseur-input3-john-kumar-messyresume-75275a848042.pdf","role":"input"}],"stresses":["messy formatting robustness","section detection fallback","inconsistent date parsing","soft-skills extraction","noise and hallucination control"],"verdict":"struggled","score":null,"score_total":null,"note":"Skills were returned as one unstructured, space-separated string with no array structure or delimiters, making them hard to split programmatically.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/parseur-parseur-task3-skills-flat-mock-bae2975459e2.png","role":"output","alt":null},{"url":"https://d3epheqghktydj.cloudfront.net/parseur-pa9-skills-flat-mock-32be32047a81.png","role":"output","alt":null}],"run_id":"cbbef4db-964c-49fa-a57f-a2977822bdfc","study_title":"Parse resumes into structured data using an API","study_kind":"generation","research_task":"86b9jm30n","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://d3epheqghktydj.cloudfront.net/parseur-input3-john-kumar-messyresume-75275a848042.pdf","filename":"parseur-input3-john-kumar-messyresume-75275a848042.pdf","alt":"Messy real-world resume — John Kumar","role":"input"}],"modality":"pdf","stresses":["messy formatting robustness","section detection fallback","inconsistent date parsing","soft-skills extraction","noise and hallucination control"]},"tool_page_slug":"parseur","tool_url":"https://aidemos.com/tools/parseur","permalink":"https://aidemos.com/evidence/fcafc622-fe49-4c1a-88e4-ecbbfb5ef7d5","api_url":"https://ai.aidemos.com/v1/observations/fcafc622-fe49-4c1a-88e4-ecbbfb5ef7d5"},"peers":[{"id":"865b57b7-f18b-4795-be43-bb0b47c559e2","tool":"airparser","tool_name":"Airparser","verdict":"mixed","score":null,"score_total":null,"note":"Produces usable JSON from noisy text, but the skills section collapses into one long string and one education record keeps an unnormalized marks value.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-image-43885cfb8772.png","evidence_url":"https://aidemos.com/evidence/865b57b7-f18b-4795-be43-bb0b47c559e2"},{"id":"5ecc3f0c-d625-475d-8251-0b44a2f88196","tool":"extracta-ai","tool_name":"Extracta.ai","verdict":"worked","score":null,"score_total":null,"note":"Produces a usable JSON result even on the messy resume, with the important sections still readable and structured.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-screenshot-202026-05-05-20124544-174ecd83f60c.png","evidence_url":"https://aidemos.com/evidence/5ecc3f0c-d625-475d-8251-0b44a2f88196"},{"id":"2cb7ceb9-2755-46c7-8fe5-ca539f649643","tool":"hrflow","tool_name":"HrFlow","verdict":"mixed","score":null,"score_total":null,"note":"The messy-resume output is moderate overall: contact and basic structure survive, but the result is incomplete for soft skills, certifications, and task detail.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/2cb7ceb9-2755-46c7-8fe5-ca539f649643"},{"id":"c2f3f2c8-8c18-40fc-8445-b9e12f157210","tool":"llamaparse","tool_name":"LlamaParse","verdict":"worked","score":null,"score_total":null,"note":"The messy-resume output was the richest and cleanest among the tested tools, with all 3 education entries, all 14 skills, both certifications, hobbies, and a boolean references field.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-image-180f1deae78b.png","evidence_url":"https://aidemos.com/evidence/c2f3f2c8-8c18-40fc-8445-b9e12f157210"},{"id":"f3277ca4-0119-47a4-a0b2-fb948ba0dbd9","tool":"skima-ai","tool_name":"Skima AI","verdict":"struggled","score":null,"score_total":null,"note":"The messy-resume output degraded sharply: responsibilities became a run-on string, skills collapsed into one concatenated line, and references/certifications/hobbies were missing.","artifact_count":3,"thumbnail":"https://d3epheqghktydj.cloudfront.net/skima-ai-05-runon-input3-2809394923e0.png","evidence_url":"https://aidemos.com/evidence/f3277ca4-0119-47a4-a0b2-fb948ba0dbd9"}],"other_criteria":[{"id":"cf3ad31b-ebf3-4e51-b58c-8bac258004f9","criterion":"accuracy","criterion_name":"Accuracy","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"The education CGPA value was wrong: the tool returned 67, which the report says is the percentage score rather than the actual CGPA.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/cf3ad31b-ebf3-4e51-b58c-8bac258004f9"},{"id":"3b62bba2-461d-4397-b0de-bd07c47ce05d","criterion":"field-coverage","criterion_name":"Field coverage","rank_role":"decisive","verdict":"struggled","score":null,"score_total":null,"note":"Only one education record was extracted, and the 12th Science and 10th std entries were omitted from the output.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/3b62bba2-461d-4397-b0de-bd07c47ce05d"},{"id":"9abe5851-aa36-491a-9070-15fc53cc58da","criterion":"messy-resume-handling","criterion_name":"Messy resume handling","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The parser still accepted a highly inconsistent resume with missing headers and mixed date formats, and it continued parsing through the messy structure.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/9abe5851-aa36-491a-9070-15fc53cc58da"}],"appears_in":[{"page_type":"ranking","slug":"resume-parsing-api","title":"Best AI Tools for Parsing Resumes via API (2026)","url":"https://aidemos.com/best/resume-parsing-api","binding":"run"}],"same_scenario":[{"id":"865b57b7-f18b-4795-be43-bb0b47c559e2","tool":"airparser","tool_name":"Airparser","verdict":"mixed","score":null,"score_total":null,"note":"Produces usable JSON from noisy text, but the skills section collapses into one long string and one education record keeps an unnormalized marks value."},{"id":"5ecc3f0c-d625-475d-8251-0b44a2f88196","tool":"extracta-ai","tool_name":"Extracta.ai","verdict":"worked","score":null,"score_total":null,"note":"Produces a usable JSON result even on the messy resume, with the important sections still readable and structured."},{"id":"2cb7ceb9-2755-46c7-8fe5-ca539f649643","tool":"hrflow","tool_name":"HrFlow","verdict":"mixed","score":null,"score_total":null,"note":"The messy-resume output is moderate overall: contact and basic structure survive, but the result is incomplete for soft skills, certifications, and task detail."},{"id":"c2f3f2c8-8c18-40fc-8445-b9e12f157210","tool":"llamaparse","tool_name":"LlamaParse","verdict":"worked","score":null,"score_total":null,"note":"The messy-resume output was the richest and cleanest among the tested tools, with all 3 education entries, all 14 skills, both certifications, hobbies, and a boolean references field."},{"id":"f3277ca4-0119-47a4-a0b2-fb948ba0dbd9","tool":"skima-ai","tool_name":"Skima AI","verdict":"struggled","score":null,"score_total":null,"note":"The messy-resume output degraded sharply: responsibilities became a run-on string, skills collapsed into one concatenated line, and references/certifications/hobbies were missing."}]}