Code

History

Commits on feature/write_checkpoints to dev35+
  1. attempt to fix windows episode 2Andrej Karpathy committed 2 years ago2a0f78d
  2. attempt to fix windows?Andrej Karpathy committed 2 years ago5b96d91
  3. Merge pull request #450 from rosslwheeler/dataloader_win_fixesAndrej committed 2 years ago4ff0412
  4. Merge pull request #439 from lancerts/matmul-fixAndrej committed 2 years ago6a7fd56
  5. fixed mkdir changeRoss Wheeler committed 2 years ago79738d2
  6. Moved windows glob() over to dev/unistd.hRoss Wheeler committed 2 years agoe5083be
  7. Merge branch 'deterministic_layernorm' of https://github.com/ademeure/llm.c into ademeure-deterministic_layernormAndrej Karpathy committed 2 years agodbacaf8
  8. Merge pull request #456 from ngc92/large-batch-fixAndrej committed 2 years agobf03e7f
  9. int -> int64_tErik Schultheis committed 2 years ago1b98637
  10. change default shard size for fineweb10BAndrej Karpathy committed 2 years ago08fe3ee
  11. update fine-web preprocessing: faster, and much less RAM consumptionErik Schultheis committed 2 years agob66eb66
  12. make hellaswag optional eval yayAndrej Karpathy committed 2 years agoda59861
  13. fixed a bunch of small issues and we now utilize the full batch dimension, this is very close to ready to merge, might even be readyAndrej Karpathy committed 2 years ago67239d9
  14. first draft, apparently this works. needs cleanups, and also we are not yet utilizing the full batch dimension. we actually have to load in multiple examples and fully utilize batchAndrej Karpathy committed 2 years ago051f3ca
  15. continued changes for sharded dataloaderAndrej Karpathy committed 2 years agoedb0df9
  16. added new layernorm backward to /dev/cuda/ademeure committed 2 years ago7cbeefc
  17. extend dataloader to be shardedAndrej Karpathy committed 2 years ago3131028
  18. the write_datafile function accepts python list, which i think is faster but i didn't checkAndrej Karpathy committed 2 years agod536088
  19. adjust the dataloader to load the new .bin data files, and both prod datasets to use it insteadAndrej Karpathy committed 2 years ago1defbd4
  20. add fineweb, and add the first version of a new write_shard function that contains a header properlyAndrej Karpathy committed 2 years ago4bbd01f
  21. i also forgot to include the readme file for the new dev/data dirAndrej Karpathy committed 2 years ago7d11b79
  22. oops forgot to include data_common.pyAndrej Karpathy committed 2 years ago71774b3
  23. refactor how we treat datasets, because we're about to have more of them and we don't want them to clutter up root dir etc. this is only step 1, i'm about to refactor a bunch of the dataloading, how the .bin files work and are loaded, how the DataLoader works, etc. This is all needed to support good evals and training at scaleAndrej Karpathy committed 2 years ago722e5b2
  24. update the utils function and assertlancer committed 2 years ago2b0667a
  25. fix the unsupported block_sizelancer committed 2 years ago6348d41
  26. small touchups to grad clipAndrej Karpathy committed 2 years agoc2d12f7
  27. Merge branch 'gradient-clipping' of https://github.com/ngc92/llm.c into ngc92-gradient-clippingAndrej Karpathy committed 2 years agoda9f4e2
  28. fix a bug, we have to be careful to make sure we evaluate loss at the token just before the first completion token, because that is the prediction for the first token, and its accuracyAndrej Karpathy committed 2 years ago8d55c4a
  29. report both acc and acc_normAndrej Karpathy committed 2 years ago4e14b52
  30. add mmlu as well and refine both a bitAndrej Karpathy committed 2 years ago7bd2389
  31. move hellaswag file to devAndrej Karpathy committed 2 years agoaec9ce5
  32. fixed up dev/cudaErik Schultheis committed 2 years ago66ce576
  33. added a useful mixed precision utility for dev/cudaErik Schultheis committed 2 years agod7a81ef
  34. Merge pull request #383 from KarhouTam/feature/online-softmax-forward-without-cgsAndrej committed 2 years ago109f516
  35. Merge pull request #416 from azret/feature/cpu_devAndrej committed 2 years ago9e04dea