# Coding-agent evaluation worksheet

Published by Heksync, the maker of LoopCode. Updated 20 October 2026.
Use this template with any coding agent. It contains no measured results or rankings.

Guide: https://loopcode.heksync.in/guides/choosing-an-ai-coding-agent

## Shared baseline

- Repository and commit:
- Language / runtime / dependency versions:
- Dependency installation command:
- Baseline check command and result:
- Fixture or sample data:
- Context that may be shared with the model service:
- Trial task: explanation / bug fix / behavior-preserving refactor
- Expected behavior:
- Acceptance criteria (set these BEFORE the trial):
- Exact prompt:
- Time and usage budget:

## Per-run record (duplicate for every attempt)

- Trial ID and date:
- Agent, version, and interface:
- Model and model settings (or "not exposed"):
- Plan, billing rules, and usage limits:
- Branch or worktree at the baseline commit:
- Permissions and enabled tools:
- Context supplied:
- Start and finish time:
- Input tokens (if exposed):
- Cached input tokens (if exposed):
- Output tokens (if exposed):
- Usage not exposed by the service:
- Additional retries, failed calls, or abandoned runs:
- Attributable cost, currency, and evidence (if available):
- Final diff:
- Verification commands, exit statuses, and output:
- Regression fails before patch and passes after: yes / no / unverified
- Acceptance criteria passed: yes / no / partial
- Unrelated changes or weakened assertions:
- Manual review and correction time:
- Remaining checks and limitations:
- Accepted change: yes / no

## Summarize repeated trials

- Number of attempts (include failures):
- Number of accepted changes:
- Total attributable cost:
- Cost per accepted change = total attributable cost / accepted changes:
- Total review and correction time:
- Results by task type:
- Conditions under which results may differ:

If no changes were accepted, report that outcome instead of a cost-per-change
number. If token or cost data is unavailable, label it unavailable. Different
models can tokenize and price the same context differently. Do not present a
percentage saving without a stated baseline and equivalent acceptance checks.