Code

History

Commits on feature/write_checkpoints35+
  1. fix bug check nullAndrej Karpathy committed 2 years agoa13b2bd
  2. attempt to fix windows episode 2Andrej Karpathy committed 2 years ago2a0f78d
  3. attempt to fix windows?Andrej Karpathy committed 2 years ago5b96d91
  4. resolve merge conflictAndrej Karpathy committed 2 years agof2ee356
  5. Merge pull request #450 from rosslwheeler/dataloader_win_fixesAndrej committed 2 years ago4ff0412
  6. Merge pull request #439 from lancerts/matmul-fixAndrej committed 2 years ago6a7fd56
  7. Merge pull request #458 from karpathy/feature/weight_decay_2d_onlyAndrej committed 2 years ago5ef2c04
  8. add weight decay, but only for 2D tensors, as done in GPT series and in general too. this forces us to break up our adamw kernel again into one call per tensor, so there is a small throughput hit, of about 0.5% for me. but we have to break up this kernel in near future anywayAndrej Karpathy committed 2 years ago9f08882
  9. fixed mkdir changeRoss Wheeler committed 2 years ago79738d2
  10. Moved windows glob() over to dev/unistd.hRoss Wheeler committed 2 years agoe5083be
  11. Merge branch 'karpathy:master' into dataloader_win_fixesrosslwheeler committed 2 years ago1ec081e
  12. Merge branch 'ngc92-cpu-matmul'Andrej Karpathy committed 2 years agoe60c484
  13. small cosmetic changesAndrej Karpathy committed 2 years ago3221e4b
  14. Merge branch 'cpu-matmul' of https://github.com/ngc92/llm.c into ngc92-cpu-matmulAndrej Karpathy committed 2 years ago6e4296f
  15. Merge branch 'karpathy:master' into dataloader_win_fixesrosslwheeler committed 2 years ago5e0fa45
  16. Merge branch 'ademeure-deterministic_encoder_v2'Andrej Karpathy committed 2 years agob8674ce
  17. resolve merge conflictAndrej Karpathy committed 2 years ago8b95105
  18. Merge branch 'ademeure-deterministic_layernorm'Andrej Karpathy committed 2 years ago5ddb061
  19. small formatting fix before mergeAndrej Karpathy committed 2 years ago25f17e6
  20. Merge branch 'deterministic_layernorm' of https://github.com/ademeure/llm.c into ademeure-deterministic_layernormAndrej Karpathy committed 2 years agodbacaf8
  21. Merge pull request #456 from ngc92/large-batch-fixAndrej committed 2 years agobf03e7f
  22. change -o to be not file but directory, and create a main.log inside it for the log. this is to prepare for writing of checkpoints, where we want to re-use this directory for those as wellAndrej Karpathy committed 2 years ago16b364d
  23. int -> int64_tErik Schultheis committed 2 years ago1b98637
  24. int64_tErik Schultheis committed 2 years agodf2e0da
  25. fix bug pretty sure this should just be a void* pointerAndrej Karpathy committed 2 years ago1d10e78
  26. add checkpoint function write to fileAndrej Karpathy committed 2 years ago613a125
  27. fix for large batch sizesErik Schultheis committed 2 years ago2a736cb
  28. change default shard size for fineweb10BAndrej Karpathy committed 2 years ago08fe3ee
  29. Merge pull request #455 from ngc92/finewebAndrej committed 2 years agob0f065f
  30. update fine-web preprocessing: faster, and much less RAM consumptionErik Schultheis committed 2 years agob66eb66
  31. Merge branch 'karpathy:master' into dataloader_win_fixesrosslwheeler committed 2 years agoa241a00
  32. add option to not run hellaswag, interferes with a bunch of testing, e.g. if T is lowAndrej Karpathy committed 2 years agodee4e42
  33. start putting llm.c and pytorch right next to each other, identical training runs with identical results and prints. almostAndrej Karpathy committed 2 years ago032e76c
  34. Merge pull request #453 from otabuzzman/masterAndrej committed 2 years ago4b88d2a
  35. add learning rate decay schedule, now we have the full scheduler implementedAndrej Karpathy committed 2 years ago64b6a14