Crystal RLM Eval Analysis

Reward Trend

Pass@k Trend

Rubric Components

Operational Errors

Tool Calls per Rollout

Harness Exploration Calls per Rollout

First Candidate Turn

First Checker Turn

Second Checker Turn

Candidate + Checker Same Call by Turn

Primary Labels

Top Secondary Tags

Reasoning Context Tags

Error Subtypes

Exception Details

Silent / Non-Exception Bugs

Error Cause Tags

Run Summary

Per-Rollout Compact Audit

Examples