Code

History

Commits on move/encoder_kernel to dev105+
  1. automatically switch to buffer-less version if that can fill up the GPUErik Schultheis committed 2 years ago081d224
  2. new kernel version with fewer atomicsErik Schultheis committed 2 years agoc0329eb
  3. enable bf16Erik Schultheis committed 2 years ago2287da0
  4. added current backward bias kernel to dev cudaErik Schultheis committed 2 years agoec92368
  5. update the -lcublas -lcublasLt flag in the commentlancer committed 2 years ago5c90845
  6. Merge branch 'fuse-ln-res-2' of https://github.com/ngc92/llm.c into ngc92-fuse-ln-res-2Andrej Karpathy committed 2 years ago43c4870
  7. remove duplicate functionngc92 committed 2 years ago75ec629
  8. remove warning noisengc92 committed 2 years ago5920143
  9. fused layernorm+residualngc92 committed 2 years ago691c1df
  10. moved bf16 boilerplate to common.hngc92 committed 2 years ago99e765d
  11. Adds cuDNN install instructions for a smaller but sufficient packageJoe Halabi committed 2 years ago69aa64c
  12. Adjust commentsKarhouTam committed 2 years ago6d7a99c
  13. Optimize codes and commentsKarhouTam committed 2 years ago6a52d86
  14. Implementation of online softmax forward kernel without cgs.KarhouTam committed 2 years agob3e8a9f
  15. remove BF16 default from classified_fused before PRademeure committed 2 years agoe1f89b3
  16. tiny irrelevant optimisation to final unaligned fused_classifier loop + add missing common.h changesademeure committed 2 years ago1ea7f9b
  17. More crazy optimisations to layernorm_backward, fused_classifier, and matmul_backward_bias.ademeure committed 2 years ago9b55ea8
  18. Minor update on the codelancer committed 2 years agobbfe8c9
  19. move gelu_backward to backward blocklancer committed 2 years agoc81adeb
  20. small doc fixesAndrej Karpathy committed 2 years ago937fbd8
  21. bias backward kernel that will use all available threadsErik Schultheis committed 2 years ago6e48501
  22. pragma unroll quick fixAndrej Karpathy committed 2 years ago50714d2
  23. update encoder_forward with the latest and greatest Packed128 and incorporate into train_gpt2cuAndrej Karpathy committed 2 years ago9da2729
  24. Merge branch 'patch-1' of https://github.com/JaneIllario/llm.c into JaneIllario-patch-1Andrej Karpathy committed 2 years ago3d1761b
  25. Merge branch 'gelu_backwards' of https://github.com/JaneIllario/llm.c into JaneIllario-gelu_backwards-3Andrej Karpathy committed 2 years ago99f51ba
  26. Merge branch 'removing_cg_example' of https://github.com/ChrisDryden/llm.c into ChrisDryden-removing_cg_exampleAndrej Karpathy committed 2 years agoec73086
  27. add kernel 4 to docs. have to improve these docs more and document them betterAndrej Karpathy committed 2 years ago2202c9a
  28. Removed cooperative groups and added packed128 to fused classifier and prepare softmax blockwide“Christopher committed 2 years ago398cdaf
  29. Added example of removing cooperative groups“Christopher committed 2 years ago665d0a4
  30. fix typoJane Illarionova committed 2 years agod6d0d50
  31. update residual_forward to use util functionsJane Illarionova committed 2 years agod0e5fea
  32. update residual forward to allow kernels to use both typesJane Illarionova committed 2 years ago52b4dd0
  33. Update residual_forward.cuJane Illarionova committed 2 years ago4ffcf5b
  34. update kernel with util functionsJane Illarionova committed 2 years ago3de3c53
  35. update gelu backward rto allow all kernels to use both typesJane Illarionova committed 2 years ago7746217