Code

History

Commits on assertion-groups to crates/eval/src/example.rs35+
  1. Fix warningsNathan Sobo committed 1 year ago878312a
  2. Send serialized EvaluatedExample structs to SnowflakeNathan Sobo committed 1 year ago6dc61ce
  3. Register assertion groups in programmatic testsNathan Sobo committed 1 year ago88008c9
  4. eval: New `add_arg_to_trait_method` example (#29297)Agus Zubiaga committed 1 year ago45d3f51
  5. agent: Render diffs for the edit file tool (#29234)Danilo Leal committed 1 year ago8366cd0
  6. eval: Fine-grained assertions (#29246)Agus Zubiaga committed 1 year agoce1a674
  7. Rework eval to support interpretable scores and efficient repetitions (#29197)Max Brunsfeld committed 1 year ago36d02de
  8. Add new action to run agent eval (#29158)Nathan Sobo committed 1 year ago458ffaa
  9. Write `{result_count}.diff` and `last.diff` eval run outputs (#29181)Michael Sloan committed 1 year ago9249919
  10. Streaming tool calls (#29179)Richard Feldman committed 1 year ago4f2f9ff
  11. Agent eval: Copy `.rules` file into eval worktree for examples based on Zed (#29116)Michael Sloan committed 1 year ago0f3ac38
  12. debugger: More tidy up for SSH (#28993)Conrad Irwin committed 1 year ago9d35f03
  13. Start tracking tool failure rates in eval (#29122)Antonio Scandurra committed 1 year ago97ab098
  14. eval: Improve lang server idle detection (#29135)Agus Zubiaga committed 1 year agoceeae79
  15. Rename regex search tool to grep and accept an include glob pattern (#29100)Nathan Sobo committed 1 year ago107d8ca
  16. Default to fast model for thread summaries and titles + don't include system prompt / context / thinking segments (#29102)Michael Sloan committed 1 year agofbf7caf
  17. agent: Preserve thinking blocks between requests (#29055)Bennet Bo Fenner committed 1 year agobafc086
  18. Pretty tool inputs in eval output markdown + numbered assistant messages (#29082)Michael Sloan committed 1 year ago98ceffe
  19. Systematically optimize agentic editing performance (#28961)Nathan Sobo committed 1 year agobab2856
  20. agent: Attach thread ID and prompt ID to telemetry events (#29069)Marshall Bowers committed 1 year ago7abe2c9
  21. agent: Use default prompts from prompt library in system prompt (#28915)Michael Sloan committed 1 year ago502a0f6
  22. agent: Show request usage in the panel (#29006)Marshall Bowers committed 1 year agoc2cd4fd
  23. eval: Fix git revision existence check (#28959)Oleksiy Syvokon committed 1 year ago6dd622d
  24. Agent Eval: Check if SHA already fetched (#28846)Michael Sloan committed 1 year ago320abe9
  25. Agent Eval: Initial support for running examples repeatedly (#28844)Michael Sloan committed 1 year ago9a9f2e7
  26. Agent Eval: Distinguish tool successes and failures in log (#28839)Michael Sloan committed 1 year agoda2d8bd
  27. Add support for judge repetitions in eval (#28811)Michael Sloan committed 1 year ago102ea6a
  28. eval: Do not create run files for skipped examples (#28800)Agus Zubiaga committed 1 year ago0182e09
  29. eval: Fix stalling on tool confirmation (#28786)Agus Zubiaga committed 1 year agoff4334e
  30. eval: Improve readability with colors and alignment (#28761)Agus Zubiaga committed 1 year agoe4cf7fe
  31. agent: Make `ToolWorkingSet` an `Entity` (#28757)Bennet Bo Fenner committed 1 year agoe26f0a3
  32. Agent eval: output paths to log files at the end (#28724)Michael Sloan committed 1 year ago0d6e455
  33. Agent Eval: Fail example when there are no events in 2 minutes (#28725)Michael Sloan committed 1 year ago5f897b0
  34. Add more eval examples + filtering examples by language + fix git concurrent usage (#28719)Thomas Mickley-Doyle committed 1 year agod74f073
  35. Track tool use counts (#28722)Michael Sloan committed 1 year agoc8ccc47