Code

History

Commits on example2 to crates/eval35+
  1. Start on a new approach to assertions in evalAntonio Scandurra committed 1 year ago4c4519c
  2. Do not log unfinished tools use that are in the middle of streaming (#29275)Oleksiy Syvokon committed 1 year agof69aeb6
  3. eval: Write JSON-serialized thread (#29271)Oleksiy Syvokon committed 1 year ago76a78b5
  4. eval: Fine-grained assertions (#29246)Agus Zubiaga committed 1 year agoce1a674
  5. Rework eval to support interpretable scores and efficient repetitions (#29197)Max Brunsfeld committed 1 year ago36d02de
  6. Add new action to run agent eval (#29158)Nathan Sobo committed 1 year ago458ffaa
  7. agent eval: Default to also running typescript examples (#29185)Michael Sloan committed 1 year ago70c51b5
  8. Write `{result_count}.diff` and `last.diff` eval run outputs (#29181)Michael Sloan committed 1 year ago9249919
  9. Streaming tool calls (#29179)Richard Feldman committed 1 year ago4f2f9ff
  10. agent: Remove non-rust examples from evals (#29139)Thomas Mickley-Doyle committed 1 year ago733cd6b
  11. Agent eval: Copy `.rules` file into eval worktree for examples based on Zed (#29116)Michael Sloan committed 1 year ago0f3ac38
  12. debugger: More tidy up for SSH (#28993)Conrad Irwin committed 1 year ago9d35f03
  13. Start tracking tool failure rates in eval (#29122)Antonio Scandurra committed 1 year ago97ab098
  14. eval: Improve lang server idle detection (#29135)Agus Zubiaga committed 1 year agoceeae79
  15. Rename regex search tool to grep and accept an include glob pattern (#29100)Nathan Sobo committed 1 year ago107d8ca
  16. agent: Reorder some linux keybindings to match mac keybindings (#29107)Michael Sloan committed 1 year agoa91948a
  17. Default to fast model for thread summaries and titles + don't include system prompt / context / thinking segments (#29102)Michael Sloan committed 1 year agofbf7caf
  18. agent: Preserve thinking blocks between requests (#29055)Bennet Bo Fenner committed 1 year agobafc086
  19. Simplify language model registry + only emit change events on change (#29086)Michael Sloan committed 1 year agod88b06a
  20. Pretty tool inputs in eval output markdown + numbered assistant messages (#29082)Michael Sloan committed 1 year ago98ceffe
  21. Systematically optimize agentic editing performance (#28961)Nathan Sobo committed 1 year agobab2856
  22. agent: Attach thread ID and prompt ID to telemetry events (#29069)Marshall Bowers committed 1 year ago7abe2c9
  23. Init prompt store in agent eval (#29068)Michael Sloan committed 1 year ago327fee4
  24. agent: Use default prompts from prompt library in system prompt (#28915)Michael Sloan committed 1 year ago502a0f6
  25. agent: Show request usage in the panel (#29006)Marshall Bowers committed 1 year agoc2cd4fd
  26. eval: Fix git revision existence check (#28959)Oleksiy Syvokon committed 1 year ago6dd622d
  27. agent: Add git commit ID to the eval telemetry data (#28895)Thomas Mickley-Doyle committed 1 year ago8de53bd
  28. Agent Eval: Check if SHA already fetched (#28846)Michael Sloan committed 1 year ago320abe9
  29. Agent Eval: Initial support for running examples repeatedly (#28844)Michael Sloan committed 1 year ago9a9f2e7
  30. Agent Eval: bounded concurrency (#28843)Michael Sloan committed 1 year ago609895d
  31. Agent Eval: Distinguish tool successes and failures in log (#28839)Michael Sloan committed 1 year agoda2d8bd
  32. agent: Add telemetry for eval runs (#28816)Thomas Mickley-Doyle committed 1 year ago222d4a2
  33. Add support for judge repetitions in eval (#28811)Michael Sloan committed 1 year ago102ea6a
  34. eval: Do not create run files for skipped examples (#28800)Agus Zubiaga committed 1 year ago0182e09
  35. eval: Fix stalling on tool confirmation (#28786)Agus Zubiaga committed 1 year agoff4334e