[Evaluation] Normalize evaluator validation errors to EvaluationException with USER_ERROR blame

[m7md7sien]

Summary

Normalizes evaluation/validation error handling in azure-ai-evaluation so that user input and configuration errors are consistently raised as EvaluationException with blame=ErrorBlame.USER_ERROR (plus an appropriate category and target).

Previously several evaluators raised bare ValueError/TypeError for input/threshold validation, and a few existing EvaluationException raises did not set blame, so they defaulted to Unknown/InternalError even though they were caused by user input.

Changes

Raw ValueError/TypeError → EvaluationException(USER_ERROR)

• ContentSafetyEvaluator — threshold type check
• QAEvaluator — threshold type check
• RougeScoreEvaluator — threshold type check
• DocumentRetrievalEvaluator — ground-truth label and input-record validation
• Task navigation efficiency evaluator — matching_mode and ground_truth validation

Existing EvaluationException missing USER_ERROR

• Evaluator base (_base_eval.py) — conversation message mismatch, malformed tool-call parsing, and threshold-not-a-number checks now set blame=USER_ERROR (one category=UNKNOWN corrected to INVALID_VALUE)

Supporting

• Added QA_EVALUATOR, ROUGE_EVALUATOR, and DOCUMENT_RETRIEVAL_EVALUATOR members to ErrorTarget
• Updated the task navigation test to expect EvaluationException for an invalid matching_mode
• CHANGELOG entry under 1.17.1 (Unreleased)

Intentionally left unchanged (not user errors)

• "Evaluator returned invalid output" / "Invalid score value" across the prompty and tool evaluators remain SYSTEM_ERROR (malformed LLM output, not user input).
• Internal/defensive checks (_conversation_aggregators.py UNKNOWN, _base_rai_svc_eval.py "Not implemented") are unchanged.

Validation

• All affected unit tests pass (document retrieval, task navigation, threshold behavior, common validators, built-in & agent evaluators).
• black (pinned 24.4.0, repo config) passes on all modified files.

[Copilot]

Pull request overview

This PR normalizes input/configuration validation error handling in the azure-ai-evaluation package so that user-caused validation failures are consistently raised as EvaluationException with blame=ErrorBlame.USER_ERROR (plus an appropriate category and target). Previously several evaluators raised bare ValueError/TypeError, and a few existing EvaluationException raises omitted blame, causing them to display as (InternalError) even when caused by user input (since EvaluationException.__str__ maps any non-USER_ERROR blame to "InternalError"). This change improves error classification surfaced to users and aligns with the package-wide validation convention already used in the common validators.

Changes:

• Converted raw ValueError/TypeError validation raises to EvaluationException(USER_ERROR) in ContentSafetyEvaluator, QAEvaluator, RougeScoreEvaluator, DocumentRetrievalEvaluator, and the task navigation efficiency evaluator.
• Added explicit blame=USER_ERROR (and corrected one category=UNKNOWN → INVALID_VALUE) to existing EvaluationException raises in the evaluator base (_base_eval.py), and added three new ErrorTarget members (QA_EVALUATOR, ROUGE_EVALUATOR, DOCUMENT_RETRIEVAL_EVALUATOR).
• Updated the task navigation test to expect EvaluationException, plus a CHANGELOG entry under 1.17.1 (Unreleased).

I verified the new ErrorTarget values match their class names, no duplicate imports were introduced, ErrorBlame was already imported where used in _base_eval.py, the document-retrieval reordering (type checks now precede the >= comparison) is a correctness improvement and keeps existing test messages intact, and no existing tests still expect ValueError/TypeError for the changed evaluators. One note: changing these raises from ValueError/TypeError to EvaluationException (which is not a subclass of either) is a behavioral change for any downstream code catching those specific exception types — this is intentional and documented in the CHANGELOG.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
_exceptions.py	Adds QA_EVALUATOR, ROUGE_EVALUATOR, DOCUMENT_RETRIEVAL_EVALUATOR to ErrorTarget.
_evaluators/_common/_base_eval.py	Adds USER_ERROR blame to conversation/tool-call/threshold validation raises; one category corrected to INVALID_VALUE.
_evaluators/_content_safety/_content_safety.py	Threshold type check now raises EvaluationException(USER_ERROR) instead of TypeError.
_evaluators/_qa/_qa.py	Threshold type check now raises EvaluationException(USER_ERROR).
_evaluators/_rouge/_rouge.py	Threshold type check now raises EvaluationException(USER_ERROR).
_evaluators/_document_retrieval/_document_retrieval.py	Label-bound/input-record validation normalized to EvaluationException; type checks reordered before the bound comparison; internal missing-threshold kept as SYSTEM_ERROR.
_evaluators/_task_navigation_efficiency/_task_navigation_efficiency.py	matching_mode and ground_truth validation normalized to EvaluationException(USER_ERROR).
tests/unittests/test_task_navigation_efficiency_evaluators.py	Updated to expect EvaluationException for invalid matching_mode.
CHANGELOG.md	Adds a Bugs Fixed entry under 1.17.1 (Unreleased).