Code

History

Commits on example2 to crates/eval/src/example.rs32
  1. eval: Fine-grained assertions (#29246)Agus Zubiaga committed 1 year agoce1a674
  2. Rework eval to support interpretable scores and efficient repetitions (#29197)Max Brunsfeld committed 1 year ago36d02de
  3. Add new action to run agent eval (#29158)Nathan Sobo committed 1 year ago458ffaa
  4. Write `{result_count}.diff` and `last.diff` eval run outputs (#29181)Michael Sloan committed 1 year ago9249919
  5. Streaming tool calls (#29179)Richard Feldman committed 1 year ago4f2f9ff
  6. Agent eval: Copy `.rules` file into eval worktree for examples based on Zed (#29116)Michael Sloan committed 1 year ago0f3ac38
  7. debugger: More tidy up for SSH (#28993)Conrad Irwin committed 1 year ago9d35f03
  8. Start tracking tool failure rates in eval (#29122)Antonio Scandurra committed 1 year ago97ab098
  9. eval: Improve lang server idle detection (#29135)Agus Zubiaga committed 1 year agoceeae79
  10. Rename regex search tool to grep and accept an include glob pattern (#29100)Nathan Sobo committed 1 year ago107d8ca
  11. Default to fast model for thread summaries and titles + don't include system prompt / context / thinking segments (#29102)Michael Sloan committed 1 year agofbf7caf
  12. agent: Preserve thinking blocks between requests (#29055)Bennet Bo Fenner committed 1 year agobafc086
  13. Pretty tool inputs in eval output markdown + numbered assistant messages (#29082)Michael Sloan committed 1 year ago98ceffe
  14. Systematically optimize agentic editing performance (#28961)Nathan Sobo committed 1 year agobab2856
  15. agent: Attach thread ID and prompt ID to telemetry events (#29069)Marshall Bowers committed 1 year ago7abe2c9
  16. agent: Use default prompts from prompt library in system prompt (#28915)Michael Sloan committed 1 year ago502a0f6
  17. agent: Show request usage in the panel (#29006)Marshall Bowers committed 1 year agoc2cd4fd
  18. eval: Fix git revision existence check (#28959)Oleksiy Syvokon committed 1 year ago6dd622d
  19. Agent Eval: Check if SHA already fetched (#28846)Michael Sloan committed 1 year ago320abe9
  20. Agent Eval: Initial support for running examples repeatedly (#28844)Michael Sloan committed 1 year ago9a9f2e7
  21. Agent Eval: Distinguish tool successes and failures in log (#28839)Michael Sloan committed 1 year agoda2d8bd
  22. Add support for judge repetitions in eval (#28811)Michael Sloan committed 1 year ago102ea6a
  23. eval: Do not create run files for skipped examples (#28800)Agus Zubiaga committed 1 year ago0182e09
  24. eval: Fix stalling on tool confirmation (#28786)Agus Zubiaga committed 1 year agoff4334e
  25. eval: Improve readability with colors and alignment (#28761)Agus Zubiaga committed 1 year agoe4cf7fe
  26. agent: Make `ToolWorkingSet` an `Entity` (#28757)Bennet Bo Fenner committed 1 year agoe26f0a3
  27. Agent eval: output paths to log files at the end (#28724)Michael Sloan committed 1 year ago0d6e455
  28. Agent Eval: Fail example when there are no events in 2 minutes (#28725)Michael Sloan committed 1 year ago5f897b0
  29. Add more eval examples + filtering examples by language + fix git concurrent usage (#28719)Thomas Mickley-Doyle committed 1 year agod74f073
  30. Track tool use counts (#28722)Michael Sloan committed 1 year agoc8ccc47
  31. Add judge to new eval + provide LSP diagnostics (#28713)Michael Sloan committed 1 year ago6b80eb5
  32. Actually run the eval and fix a hang when retrieving outline (#28547)Antonio Scandurra committed 1 year ago2440faf