{"observation":{"id":"c90b4e68-d0af-47f9-a85c-43c3ec8a9fac","tool":"llamaparse","tool_name":"LlamaParse","criterion":"accuracy","criterion_name":"Accuracy","criterion_definition":"Are extracted values correct and complete?","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Correct and complete values are the essence of resume parsing, so this directly determines whether the tool succeeds. (3 of 3 judges)","scenario":"messy-real-world-resume-john-kumar","scenario_name":"Messy real-world resume — John Kumar","group_tag":"resume-parsing","scenario_description":"A poorly formatted, inconsistent real-world resume for John Kumar, used to test robustness against noisy structure, inconsistent dates, and mixed-content sections.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://d3epheqghktydj.cloudfront.net/parseur-input3-john-kumar-messyresume-75275a848042.pdf","role":"input"}],"stresses":["messy formatting robustness","section detection fallback","inconsistent date parsing","soft-skills extraction","noise and hallucination control"],"verdict":"worked","score":null,"score_total":null,"note":"On the messy resume, the name, email, phone, location, and objective statement were extracted correctly.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/research-media-image-180f1deae78b.png","role":"input","alt":null},{"url":"https://d3epheqghktydj.cloudfront.net/research-media-llama-20output-203-211c2f59deaf.txt","role":"output","alt":null}],"run_id":"cbbef4db-964c-49fa-a57f-a2977822bdfc","study_title":"Parse resumes into structured data using an API","study_kind":"generation","research_task":"86b9jm30n","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://d3epheqghktydj.cloudfront.net/parseur-input3-john-kumar-messyresume-75275a848042.pdf","filename":"parseur-input3-john-kumar-messyresume-75275a848042.pdf","alt":"Messy real-world resume — John Kumar","role":"input"}],"modality":"pdf","stresses":["messy formatting robustness","section detection fallback","inconsistent date parsing","soft-skills extraction","noise and hallucination control"]},"tool_page_slug":"llamaparse","tool_url":"https://aidemos.com/tools/llamaparse","permalink":"https://aidemos.com/evidence/c90b4e68-d0af-47f9-a85c-43c3ec8a9fac","api_url":"https://ai.aidemos.com/v1/observations/c90b4e68-d0af-47f9-a85c-43c3ec8a9fac"},"peers":[{"id":"6b3eb118-2ba6-4652-b602-ce3fd5db643a","tool":"affinda","tool_name":"Affinda","verdict":"failed","score":null,"score_total":null,"note":"Its total experience calculation overstates tenure: it returned 7.3 years even though the resume explicitly says 3 years of experience.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-image-d6a62fc24445.png","evidence_url":"https://aidemos.com/evidence/6b3eb118-2ba6-4652-b602-ce3fd5db643a"},{"id":"993bb825-5a88-466b-9fff-dccd2dff8b1b","tool":"airparser","tool_name":"Airparser","verdict":"mixed","score":null,"score_total":null,"note":"Leaves one education marks value as raw text (\"72 percent marks\") instead of normalizing it to the percentage form used by the other entries.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-image-0e184cd31272.png","evidence_url":"https://aidemos.com/evidence/993bb825-5a88-466b-9fff-dccd2dff8b1b"},{"id":"24d4e0ed-62c4-4c4c-a760-b27edcbfbf8b","tool":"cvparserpro","tool_name":"CVParserPro","verdict":"failed","score":null,"score_total":null,"note":"The education end date was wrong: a completed program was rendered as ending in Present.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/cvparserpro-p8-edu-present-wrong-b5d75dabbce2.png","evidence_url":"https://aidemos.com/evidence/24d4e0ed-62c4-4c4c-a760-b27edcbfbf8b"},{"id":"95b1a04c-460c-4ce9-92a3-12b5c95a28df","tool":"extracta-ai","tool_name":"Extracta.ai","verdict":"mixed","score":null,"score_total":null,"note":"Returns both certification names in lowercase exactly as written, with no capitalization normalisation applied.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-screenshot-202026-05-05-20124544-174ecd83f60c.png","evidence_url":"https://aidemos.com/evidence/95b1a04c-460c-4ce9-92a3-12b5c95a28df"},{"id":"ef02c50a-97cd-4bd6-a0bd-0d2828e199cc","tool":"hireability","tool_name":"Hireability","verdict":"failed","score":null,"score_total":null,"note":"The second employer name was merged with the role title, returning 'Junior Developer XYZ InfoTech' instead of just 'XYZ InfoTech'.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/hireability-hireability-task3-employer-name-merged-m-85d6c25840b0.png","evidence_url":"https://aidemos.com/evidence/ef02c50a-97cd-4bd6-a0bd-0d2828e199cc"},{"id":"979b5f53-2a09-4a4a-9bc1-e6cf8836f843","tool":"hrflow","tool_name":"HrFlow","verdict":"mixed","score":null,"score_total":null,"note":"On the messy resume, it keeps the main contact/work/education structure but misses the soft-skill section and certification entries.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/979b5f53-2a09-4a4a-9bc1-e6cf8836f843"},{"id":"519f0262-59a4-4339-bf98-d72364340046","tool":"openresume","tool_name":"OpenResume","verdict":"failed","score":null,"score_total":null,"note":"Education parsing collapses: School and Degree are empty, GPA is misread as 12, and the full education line is pushed into Date.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/openresume-o7-education-failed-9f9f7dee6cc7.png","evidence_url":"https://aidemos.com/evidence/519f0262-59a4-4339-bf98-d72364340046"},{"id":"cf3ad31b-ebf3-4e51-b58c-8bac258004f9","tool":"parseur","tool_name":"Parseur","verdict":"failed","score":null,"score_total":null,"note":"The education CGPA value was wrong: the tool returned 67, which the report says is the percentage score rather than the actual CGPA.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/parseur-parseur-task3-cgpa-67-wrong-value-mock-fa517c22b6e0.png","evidence_url":"https://aidemos.com/evidence/cf3ad31b-ebf3-4e51-b58c-8bac258004f9"}],"other_criteria":[{"id":"98df030a-f07f-4fd8-9b36-3f5e603318b0","criterion":"field-coverage","criterion_name":"Field coverage","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The messy-resume output included the core fields and additional sections: name, contact information, objective, work experience, education, skills, certifications, hobbies, and references.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/98df030a-f07f-4fd8-9b36-3f5e603318b0"},{"id":"409473d2-3b39-4ba9-b7b1-ce7f11b69f27","criterion":"messy-resume-handling","criterion_name":"Messy resume handling","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The parser degraded gracefully on the messy resume, handling missing section headers and mixed date formats while still producing structured output.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/409473d2-3b39-4ba9-b7b1-ce7f11b69f27"},{"id":"c2f3f2c8-8c18-40fc-8445-b9e12f157210","criterion":"output-quality","criterion_name":"Output quality","rank_role":"decisive","verdict":"worked","score":null,"score_total":null,"note":"The messy-resume output was the richest and cleanest among the tested tools, with all 3 education entries, all 14 skills, both certifications, hobbies, and a boolean references field.","artifact_count":2,"evidence_url":"https://aidemos.com/evidence/c2f3f2c8-8c18-40fc-8445-b9e12f157210"}],"appears_in":[{"page_type":"ranking","slug":"resume-parsing-api","title":"Best AI Tools for Parsing Resumes via API (2026)","url":"https://aidemos.com/best/resume-parsing-api","binding":"run"}],"same_scenario":[{"id":"6b3eb118-2ba6-4652-b602-ce3fd5db643a","tool":"affinda","tool_name":"Affinda","verdict":"failed","score":null,"score_total":null,"note":"Its total experience calculation overstates tenure: it returned 7.3 years even though the resume explicitly says 3 years of experience."},{"id":"993bb825-5a88-466b-9fff-dccd2dff8b1b","tool":"airparser","tool_name":"Airparser","verdict":"mixed","score":null,"score_total":null,"note":"Leaves one education marks value as raw text (\"72 percent marks\") instead of normalizing it to the percentage form used by the other entries."},{"id":"24d4e0ed-62c4-4c4c-a760-b27edcbfbf8b","tool":"cvparserpro","tool_name":"CVParserPro","verdict":"failed","score":null,"score_total":null,"note":"The education end date was wrong: a completed program was rendered as ending in Present."},{"id":"95b1a04c-460c-4ce9-92a3-12b5c95a28df","tool":"extracta-ai","tool_name":"Extracta.ai","verdict":"mixed","score":null,"score_total":null,"note":"Returns both certification names in lowercase exactly as written, with no capitalization normalisation applied."},{"id":"ef02c50a-97cd-4bd6-a0bd-0d2828e199cc","tool":"hireability","tool_name":"Hireability","verdict":"failed","score":null,"score_total":null,"note":"The second employer name was merged with the role title, returning 'Junior Developer XYZ InfoTech' instead of just 'XYZ InfoTech'."},{"id":"979b5f53-2a09-4a4a-9bc1-e6cf8836f843","tool":"hrflow","tool_name":"HrFlow","verdict":"mixed","score":null,"score_total":null,"note":"On the messy resume, it keeps the main contact/work/education structure but misses the soft-skill section and certification entries."},{"id":"519f0262-59a4-4339-bf98-d72364340046","tool":"openresume","tool_name":"OpenResume","verdict":"failed","score":null,"score_total":null,"note":"Education parsing collapses: School and Degree are empty, GPA is misread as 12, and the full education line is pushed into Date."},{"id":"cf3ad31b-ebf3-4e51-b58c-8bac258004f9","tool":"parseur","tool_name":"Parseur","verdict":"failed","score":null,"score_total":null,"note":"The education CGPA value was wrong: the tool returned 67, which the report says is the percentage score rather than the actual CGPA."}]}