{"observation":{"id":"3e5537d4-ce91-4ba3-972c-64457e97deb6","tool":"cvparserpro","tool_name":"CVParserPro","criterion":"messy-resume-handling","criterion_name":"Messy resume handling","criterion_definition":"Does it degrade gracefully on poorly formatted input?","criterion_evidence_type":"transformation","criterion_rank_role":"decisive","criterion_rank_role_reason":"Resumes are often inconsistent or poorly formatted, so graceful handling of messy input is a core capability. (3 of 3 judges)","scenario":"messy-real-world-resume-john-kumar","scenario_name":"Messy real-world resume — John Kumar","group_tag":"resume-parsing","scenario_description":"A poorly formatted, inconsistent real-world resume for John Kumar, used to test robustness against noisy structure, inconsistent dates, and mixed-content sections.","modality":"pdf","input_text":null,"input_artifact_refs":[{"alt":null,"url":"https://d3epheqghktydj.cloudfront.net/parseur-input3-john-kumar-messyresume-75275a848042.pdf","role":"input"}],"stresses":["messy formatting robustness","section detection fallback","inconsistent date parsing","soft-skills extraction","noise and hallucination control"],"verdict":"struggled","score":null,"score_total":null,"note":"On the messy resume, the parser collapsed the education section to 1 extracted entry and dropped the 10th- and 12th-standard records.","evidence_state":"verified","source":null,"artifacts":[{"url":"https://d3epheqghktydj.cloudfront.net/cvparserpro-p9-only-one-education-8657fa0d4757.png","role":"output","alt":null}],"run_id":"cbbef4db-964c-49fa-a57f-a2977822bdfc","study_title":"Parse resumes into structured data using an API","study_kind":"generation","research_task":"86b9jm30n","tested_at":null,"completeness":"input-and-output","input":{"state":"files","text":null,"files":[{"url":"https://d3epheqghktydj.cloudfront.net/parseur-input3-john-kumar-messyresume-75275a848042.pdf","filename":"parseur-input3-john-kumar-messyresume-75275a848042.pdf","alt":"Messy real-world resume — John Kumar","role":"input"}],"modality":"pdf","stresses":["messy formatting robustness","section detection fallback","inconsistent date parsing","soft-skills extraction","noise and hallucination control"]},"tool_page_slug":"cvparserpro","tool_url":"https://aidemos.com/tools/cvparserpro","permalink":"https://aidemos.com/evidence/3e5537d4-ce91-4ba3-972c-64457e97deb6","api_url":"https://ai.aidemos.com/v1/observations/3e5537d4-ce91-4ba3-972c-64457e97deb6"},"peers":[{"id":"adc87f30-2a28-4fd8-a203-a64e455e74e9","tool":"affinda","tool_name":"Affinda","verdict":"worked","score":null,"score_total":null,"note":"The parser handled the messy resume gracefully, still extracting the main identity, work history, education entries, skills, objective, and hobbies despite non-standard dates and raw dash-style bullets.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-image-d6a62fc24445.png","evidence_url":"https://aidemos.com/evidence/adc87f30-2a28-4fd8-a203-a64e455e74e9"},{"id":"0feb19f4-5bd4-4455-889b-647775b6ab42","tool":"airparser","tool_name":"Airparser","verdict":"mixed","score":null,"score_total":null,"note":"Parses a poorly formatted resume, but degrades into non-machine-readable skill output rather than a structured array.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-image-43885cfb8772.png","evidence_url":"https://aidemos.com/evidence/0feb19f4-5bd4-4455-889b-647775b6ab42"},{"id":"87069ea1-3f6c-451a-93a0-147d3a3cc323","tool":"extracta-ai","tool_name":"Extracta.ai","verdict":"worked","score":null,"score_total":null,"note":"Handles the noisy, inconsistent layout without crashing and still extracts the main sections.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-screenshot-202026-05-05-20124544-174ecd83f60c.png","evidence_url":"https://aidemos.com/evidence/87069ea1-3f6c-451a-93a0-147d3a3cc323"},{"id":"e8f31662-b9b9-4b8e-8b89-b56d3b4f574d","tool":"hireability","tool_name":"Hireability","verdict":"worked","score":null,"score_total":null,"note":"On the poorly formatted resume, the parser still recovered the main structure instead of failing outright, including contact data, experience, education, skills, and hobbies.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/hireability-hireability-task3-education-entries-miss-26097fa84061.png","evidence_url":"https://aidemos.com/evidence/e8f31662-b9b9-4b8e-8b89-b56d3b4f574d"},{"id":"38efacdb-841e-4b48-ab20-6ccd7916d0b8","tool":"hrflow","tool_name":"HrFlow","verdict":"mixed","score":null,"score_total":null,"note":"It parses a poorly formatted resume without crashing, but degrades by dropping soft skills, omitting certifications, and losing most task bullets.","artifact_count":0,"thumbnail":null,"evidence_url":"https://aidemos.com/evidence/38efacdb-841e-4b48-ab20-6ccd7916d0b8"},{"id":"409473d2-3b39-4ba9-b7b1-ce7f11b69f27","tool":"llamaparse","tool_name":"LlamaParse","verdict":"worked","score":null,"score_total":null,"note":"The parser degraded gracefully on the messy resume, handling missing section headers and mixed date formats while still producing structured output.","artifact_count":2,"thumbnail":"https://d3epheqghktydj.cloudfront.net/research-media-image-180f1deae78b.png","evidence_url":"https://aidemos.com/evidence/409473d2-3b39-4ba9-b7b1-ce7f11b69f27"},{"id":"eb6362c1-a60b-45e7-81bb-344e5f2e3ef0","tool":"openresume","tool_name":"OpenResume","verdict":"failed","score":null,"score_total":null,"note":"On a poorly formatted resume without bold uppercase section headers, the parser degrades badly: core sections are misassigned or empty instead of being recovered gracefully.","artifact_count":3,"thumbnail":"https://d3epheqghktydj.cloudfront.net/openresume-o7-education-failed-9f9f7dee6cc7.png","evidence_url":"https://aidemos.com/evidence/eb6362c1-a60b-45e7-81bb-344e5f2e3ef0"},{"id":"9abe5851-aa36-491a-9070-15fc53cc58da","tool":"parseur","tool_name":"Parseur","verdict":"worked","score":null,"score_total":null,"note":"The parser still accepted a highly inconsistent resume with missing headers and mixed date formats, and it continued parsing through the messy structure.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/parseur-image-4-a76ecb3d1154.png","evidence_url":"https://aidemos.com/evidence/9abe5851-aa36-491a-9070-15fc53cc58da"},{"id":"862b02fa-8023-4785-8d7b-d98fd681eeed","tool":"skima-ai","tool_name":"Skima AI","verdict":"struggled","score":null,"score_total":null,"note":"The tool accepted the messy file, but it degraded badly in structure by merging bullets into a run-on responsibility line instead of preserving clean formatting.","artifact_count":1,"thumbnail":"https://d3epheqghktydj.cloudfront.net/skima-ai-05-runon-input3-2809394923e0.png","evidence_url":"https://aidemos.com/evidence/862b02fa-8023-4785-8d7b-d98fd681eeed"}],"other_criteria":[{"id":"24d4e0ed-62c4-4c4c-a760-b27edcbfbf8b","criterion":"accuracy","criterion_name":"Accuracy","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"The education end date was wrong: a completed program was rendered as ending in Present.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/24d4e0ed-62c4-4c4c-a760-b27edcbfbf8b"},{"id":"f4a9dbe8-9c63-425b-a0b9-39c490e6965a","criterion":"noise-in-output","criterion_name":"Noise in output","rank_role":"decisive","verdict":"failed","score":null,"score_total":null,"note":"The parser hallucinated unsupported month values in the work date range, expanding 2019–2021 into January 2019 to December 2021.","artifact_count":1,"evidence_url":"https://aidemos.com/evidence/f4a9dbe8-9c63-425b-a0b9-39c490e6965a"}],"appears_in":[{"page_type":"ranking","slug":"resume-parsing-api","title":"Best AI Tools for Parsing Resumes via API (2026)","url":"https://aidemos.com/best/resume-parsing-api","binding":"run"}],"same_scenario":[{"id":"adc87f30-2a28-4fd8-a203-a64e455e74e9","tool":"affinda","tool_name":"Affinda","verdict":"worked","score":null,"score_total":null,"note":"The parser handled the messy resume gracefully, still extracting the main identity, work history, education entries, skills, objective, and hobbies despite non-standard dates and raw dash-style bullets."},{"id":"0feb19f4-5bd4-4455-889b-647775b6ab42","tool":"airparser","tool_name":"Airparser","verdict":"mixed","score":null,"score_total":null,"note":"Parses a poorly formatted resume, but degrades into non-machine-readable skill output rather than a structured array."},{"id":"87069ea1-3f6c-451a-93a0-147d3a3cc323","tool":"extracta-ai","tool_name":"Extracta.ai","verdict":"worked","score":null,"score_total":null,"note":"Handles the noisy, inconsistent layout without crashing and still extracts the main sections."},{"id":"e8f31662-b9b9-4b8e-8b89-b56d3b4f574d","tool":"hireability","tool_name":"Hireability","verdict":"worked","score":null,"score_total":null,"note":"On the poorly formatted resume, the parser still recovered the main structure instead of failing outright, including contact data, experience, education, skills, and hobbies."},{"id":"38efacdb-841e-4b48-ab20-6ccd7916d0b8","tool":"hrflow","tool_name":"HrFlow","verdict":"mixed","score":null,"score_total":null,"note":"It parses a poorly formatted resume without crashing, but degrades by dropping soft skills, omitting certifications, and losing most task bullets."},{"id":"409473d2-3b39-4ba9-b7b1-ce7f11b69f27","tool":"llamaparse","tool_name":"LlamaParse","verdict":"worked","score":null,"score_total":null,"note":"The parser degraded gracefully on the messy resume, handling missing section headers and mixed date formats while still producing structured output."},{"id":"eb6362c1-a60b-45e7-81bb-344e5f2e3ef0","tool":"openresume","tool_name":"OpenResume","verdict":"failed","score":null,"score_total":null,"note":"On a poorly formatted resume without bold uppercase section headers, the parser degrades badly: core sections are misassigned or empty instead of being recovered gracefully."},{"id":"9abe5851-aa36-491a-9070-15fc53cc58da","tool":"parseur","tool_name":"Parseur","verdict":"worked","score":null,"score_total":null,"note":"The parser still accepted a highly inconsistent resume with missing headers and mixed date formats, and it continued parsing through the messy structure."},{"id":"862b02fa-8023-4785-8d7b-d98fd681eeed","tool":"skima-ai","tool_name":"Skima AI","verdict":"struggled","score":null,"score_total":null,"note":"The tool accepted the messy file, but it degraded badly in structure by merging bullets into a run-on responsibility line instead of preserving clean formatting."}]}