Code

History

Commits on move/encoder_kernel to dev140+
  1. Create gelu_backward.cuJane Illarionova committed 2 years ago8007280
  2. this kernel will be released later, sorry for the spoiler :)Erik Schultheis committed 2 years agob42db70
  3. fixupErik Schultheis committed 2 years ago91bc72d
  4. mixed precision utilities for dev/cudaErik Schultheis committed 2 years ago06980b6
  5. Add missing cudnnDestroy() callsademeure committed 2 years ago1147983
  6. attention_forward.cu ~matches train_gpt2cu & allocate less memory for stats in "att"ademeure committed 2 years ago2b540b6
  7. Rename lowp_float to floatX in /dev/cuda/attention_forward.cuademeure committed 2 years agobe54842
  8. Merge remote-tracking branch 'karpathy/master' into cudnn_try2ademeure committed 2 years agoc4ecc04
  9. small tweaksAndrej Karpathy committed 2 years ago769c911
  10. fix up bandwidth calculationErik Schultheis committed 2 years ago603d862
  11. fixed potential error and generalized gelu forwardErik Schultheis committed 2 years agoc3515cf
  12. delete packed128 .fp32, use (float) okAndrej Karpathy committed 2 years ago7ba1ed8
  13. fixes to the PR, careful with float/floatX etcAndrej Karpathy committed 2 years ago9141e04
  14. Merge branch 'gelu_forwards_packing' of https://github.com/ChrisDryden/llm.c into ChrisDryden-gelu_forwards_packingAndrej Karpathy committed 2 years ago242981e
  15. add modal scriptFranz Louis Cesista committed 2 years agob239b67
  16. re-commit of my old cudnn forward attention changesademeure committed 2 years ago1c516c7
  17. clear the L2 cache between consecutive invokations of our microbenchmarks to get reliable resultsErik Schultheis committed 2 years agod7813d2
  18. Added packing for gelu forwards kernelChristopher Dryden committed 2 years ago568615f
  19. remove forceinline. blame @arund42 if this ever does not get inlined by the compilerAndrej Karpathy committed 2 years agoaf2bc47
  20. move packed128 to commonAndrej Karpathy committed 2 years ago5e12c66
  21. reset back to precious hyperparametersAndrej Karpathy committed 2 years agof435929
  22. make it imo clearer that the grid size is one quarter as large as it would otherwise be. because each thread now does 4X workAndrej Karpathy committed 2 years agoee125cf
  23. small changesAndrej Karpathy committed 2 years ago083b210
  24. minor changes to packed128Andrej Karpathy committed 2 years agod9506a7
  25. Merge branch 'yet-another-gelu' of https://github.com/ngc92/llm.c into ngc92-yet-another-geluAndrej Karpathy committed 2 years ago7eefefb
  26. commentsErik Schultheis committed 2 years ago4872c57
  27. simplify to only support csErik Schultheis committed 2 years ago9f07a17
  28. slight simplificationErik Schultheis committed 2 years ago896b644
  29. yet another geluErik Schultheis committed 2 years ago45252d4
  30. Merge remote-tracking branch 'karpathy/master' into layernormbf16ademeure committed 2 years agod23bb1f
  31. Fix sizeof(T) -> sizeof(D) (was overallocating memory)ademeure committed 2 years ago49ffd68
  32. Only enable kernel4 with BF16ademeure committed 2 years ago1e50f3b
  33. Use kernel6 for train_gpt2, and add all other kernels to /dev/cuda (+bugfixes)ademeure committed 2 years ago82fef03
  34. incorporate faster encoder_forward kernel to fp32 CUDA versionAndrej Karpathy committed 2 years ago327eef3
  35. amend the float4 kernellancer committed 2 years ago1c7d23a