agents.common.models.grade.rubric¶

Rubric grading model for multi-criteria evaluations.

This module implements a rubric-based grading system that evaluates multiple criteria with individual scores and weights.

Classes¶

`RubricCriterion`	Individual criterion within a rubric.
`RubricGrade`	Rubric grading model for multi-criteria evaluations.

Module Contents¶

class agents.common.models.grade.rubric.RubricCriterion(/, **data)¶

Bases: pydantic.BaseModel

Individual criterion within a rubric.

Represents a single evaluation criterion with its own score, weight, and justification.

Parameters:: data (Any)

name¶: Name of the criterion

score¶: Score for this criterion

max_score¶: Maximum possible score for this criterion

weight¶: Relative weight of this criterion (default 1.0)

justification¶: Explanation for the score given

Example

criterion = RubricCriterion(
name="Content Quality",
score=8.5,
max_score=10,
weight=0.4,
justification="Strong content with minor gaps in coverage"
)

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

get_normalized_score()¶

Get the criterion score as a normalized value (0.0 to 1.0).

Returns:: Normalized score (score / max_score)
Return type:: float

get_percentage_score()¶

Get the criterion score as a percentage.

Returns:: Percentage score (normalized score * 100)
Return type:: float

get_weighted_max_score()¶

Get the weighted maximum score for this criterion.

Returns:: Max score multiplied by weight
Return type:: float

get_weighted_score()¶

Get the weighted score for this criterion.

Returns:: Score multiplied by weight
Return type:: float

classmethod validate_score_range(v, info)¶

Validate that score is within valid range.

Parameters:

v (int | float) – The score value
info – Validation context containing other field values

Returns:

Validated score

Raises:

ValueError – If score is outside valid range

Return type:

int | float

validate_score_within_max()¶

Validate that score does not exceed max_score.

Returns:: Self if validation passes
Raises:: ValueError – If score exceeds max_score
Return type:: RubricCriterion

class agents.common.models.grade.rubric.RubricGrade(/, **data)¶

Bases: haive.agents.common.models.grade.base.Grade

Rubric grading model for multi-criteria evaluations.

This grade model evaluates multiple criteria, each with their own scores, weights, and justifications, then combines them into an overall grade.

Parameters:: data (Any)

criteria¶: List of individual rubric criteria

grade_type¶: Always GradeType.RUBRIC

overall_justification¶: Overall summary justification

Example

criteria = [
RubricCriterion(
name="Content Quality",
score=8.5,
max_score=10,
weight=0.4,
justification="Strong content with comprehensive coverage"
),
RubricCriterion(
name="Organization",
score=7.0,
max_score=10,
weight=0.3,
justification="Good structure but some transitions unclear"
),
RubricCriterion(
name="Grammar",
score=9.0,
max_score=10,
weight=0.3,
justification="Excellent grammar with only minor errors"
)
]

grade = RubricGrade(
criteria=criteria,
justification="Overall strong performance with room for improvement in organization"
)

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

classmethod create_simple_rubric(criteria_scores, justification, max_score=10.0, **kwargs)¶

Create a simple rubric with equal weights.

Parameters:

criteria_scores (dict[str, float | dict[str, Any]]) – Dict mapping criterion names to scores or score dicts
justification (str) – Overall justification
max_score (float) – Maximum score for all criteria (default 10.0)
**kwargs – Additional parameters for the grade

Returns:

RubricGrade instance with equal-weighted criteria

Return type:

RubricGrade

Example

grade = RubricGrade.create_simple_rubric(
criteria_scores={
"Content": 8.5,
"Style": {"score": 7.0, "justification": "Good style overall"},
"Accuracy": 9.0
},
justification="Strong overall performance",
max_score=10
)

get_criteria_summary()¶

Get a summary of all criteria performance.

Returns:: Dictionary mapping criterion names to their summary info
Return type:: dict[str, dict[str, Any]]

get_criterion_by_name(name)¶

Get a specific criterion by name.

Parameters:: name (str) – Name of the criterion to find
Returns:: RubricCriterion if found, None otherwise
Return type:: RubricCriterion | None

get_improvement_suggestions()¶

Generate improvement suggestions based on weakest criteria.

Returns:: List of suggestions for improvement
Return type:: list[str]

get_max_weighted_score()¶

Get the maximum possible weighted score.

Returns:: Sum of all weighted maximum scores
Return type:: float

get_normalized_score()¶

Get the overall grade as a normalized score between 0.0 and 1.0.

Calculates weighted average of all criteria scores.

Returns:: Weighted normalized score across all criteria
Return type:: float

get_raw_weighted_score()¶

Get the total raw weighted score.

Returns:: Sum of all weighted scores
Return type:: float

get_strongest_criteria(count=3)¶

Get the criteria with the highest normalized scores.

Parameters:: count (int) – Number of strongest criteria to return
Returns:: List of criteria sorted by normalized score (highest first)
Return type:: list[RubricCriterion]

get_weakest_criteria(count=3)¶

Get the criteria with the lowest normalized scores.

Parameters:: count (int) – Number of weakest criteria to return
Returns:: List of criteria sorted by normalized score (lowest first)
Return type:: list[RubricCriterion]

is_passing(threshold=None)¶

Determine if the rubric grade represents a passing score.

Parameters:: threshold (float | None) – Custom threshold for passing (0.0 to 1.0). If None, uses 0.7 (70%) as default
Returns:: True if the overall score meets or exceeds the threshold
Return type:: bool

to_display_string()¶

Convert grade to a human-readable display string.

Returns:: Formatted string representation of the rubric grade
Return type:: str

classmethod validate_criteria_names_unique(v)¶

Validate that all criterion names are unique.

Parameters:: v (list[RubricCriterion]) – List of criteria to validate
Returns:: Validated criteria list
Raises:: ValueError – If criterion names are not unique
Return type:: list[RubricCriterion]

validate_grade_value(value)¶

Validate that a value represents valid rubric criteria.

Parameters:: value (Any) – The value to validate (should be list of criteria)
Returns:: True if the value can be converted to valid criteria, False otherwise
Return type:: bool