Choose by workflow before choosing by model
A strong model is only part of a coding agent. The tools available to it, the project context it finds, and the way it handles failed checks affect the work you receive. A polished demo is useful for understanding the interface; it is not evidence that the agent will finish your task.
This is a first-party guide from Heksync, the maker of LoopCode. It is an evaluation method, not an independent ranking or a claim that LoopCode beats every alternative. We have not published a controlled cross-agent accuracy or token-savings benchmark.
| Your working environment | What to look for | What to verify in a trial |
|---|---|---|
| Desktop and local folders | A clear project boundary, visible commands, and readable diffs | The agent changes the intended project and lets you review the result |
| Terminal and scripts | Resumable sessions, process exit status, and structured output | A bounded task returns usable output and reports failures |
| VS Code | Selected-code context and a review flow beside the source | You can move from the answer to the affected file and inspect the edit |
| Hosted or background execution | A documented execution environment and repository-access controls | Dependencies, secrets, network access, and handoff behavior fit your policies |
Run the same three tasks on the same repository
- Choose a small repository you may share with the selected model provider. Record its commit, dependencies, and baseline test result. Use a fresh branch or worktree for each trial.
- Ask the agent to explain one request path and identify the implementation, its callers, and an existing test. Check the file references yourself.
- Give it a reproducible bug with one failing input. Require a focused fix and a regression test that fails before the patch and passes after it.
- Ask for a small behavior-preserving refactor. Define the public contract and run the original tests without weakening their assertions.
- Keep the prompt, model, permission settings, run outcome, usage, and final diff. Repeat the trials before drawing a general conclusion.
Project commit: <commit>
Bug: <observable behavior and failing input>
Expected: <acceptance criteria>
Baseline check: <existing test command>
Read the implementation and its callers first.
Make the smallest fix and add a regression test.
Do not remove assertions or unrelated features.
Report changed files, commands, results, and unverified checks.Record evidence instead of assigning a vague score
| Criterion | Evidence to keep | Reason to investigate |
|---|---|---|
| Correctness | Regression test, existing suite result, and manual acceptance check | The answer says done but checks were skipped or assertions removed |
| Codebase understanding | Correct entry points, callers, and project conventions | A plausible explanation cites files or APIs that do not exist |
| Review effort | Final diff, unrelated changes, and time spent correcting the patch | The patch is broad enough to hide the intended fix |
| Tool control | Permission settings, requested actions, and command output | The agent acts outside the task or cannot explain a failed command |
| Usage and cost | Model, input/output and cached usage where exposed, retries, plan rules | A short final answer hides repeated model calls or incomplete work |
| Recovery | What happens after a test failure, interruption, or missing dependency | The run repeats the same unsuccessful action without new evidence |
Compare cost per accepted change, not just token counts
Count all attempts, including failed and abandoned runs. Record uncached input, cached input, and output separately when the service exposes them. Different models and plans can price these differently; a subscription allowance is not the same as a per-token API bill.
For a small trial, divide the total attributable cost by the number of changes that pass your acceptance criteria. Record review time separately. A run that consumes fewer tokens but produces an incorrect patch is not a better result. When a plan does not expose comparable token data, mark it unavailable instead of guessing.
Read the current product documentation
For tools such as Claude Code, Codex, Cursor, GitHub Copilot, Windsurf, and Cline, check the vendor’s current documentation for supported interfaces, execution environment, permissions, and model access. Product names, capabilities, and plans change. These links are starting points for your research, not test results or endorsements.
When to include LoopCode in your shortlist
Try LoopCode if you want a coding agent for a local workspace, with desktop, CLI, and VS Code interfaces sharing one account and plan usage. The desktop browser can inspect an accessible running app; configured language servers can provide definitions, references, and diagnostics. Use the same task and acceptance criteria to evaluate those capabilities.
Free provides starter usage; paid plans provide more capacity. Model availability and usage limits depend on the current plan. Relevant prompts and code context go to hosted models, so local-first does not mean offline inference. Choose a different deployment arrangement if your requirement is that code never leaves your machine.
KEEP GOING
A useful next step.
Put it to work in your project.
Desktop, CLI, and VS Code. One LoopCode account. Free to start.