Code

History

Commits on move/encoder_kernel to dev175+
  1. amend the float4 kernellancer committed 2 years ago61c5c05
  2. Include the float4 kernellancer committed 2 years agocfccd82
  3. slight docs tweaks to dev/cudaAndrej Karpathy committed 2 years ago12da2c1
  4. Use a make pattern rule to simplify Makefile, group kernels into forward/backward/updates/nccls, add some comments and update the README.mdaustinvhuang committed 2 years ago25d703f
  5. Add a local convenience Makefile for dev/cuda/, fix 2 minor warnings observed when building all targets.austinvhuang committed 2 years agod937f93
  6. fix bug with all_hostsname_hashes missing, re-arrange code a bit, make sure we only print on rank0Andrej Karpathy committed 2 years agoa00a3a8
  7. Merge branch 'multi_gpu_support' of https://github.com/PeterZhizhin/llm.c into PeterZhizhin-multi_gpu_supportAndrej Karpathy committed 2 years ago8389fba
  8. [Multi-GPU] llm.c now runs on multiple GPUs with NCCLPetr Zhizhin committed 2 years agod49e830
  9. encoder backward add, we want to get rid of atomics, hmmAndrej Karpathy committed 2 years ago250159e
  10. rename positional_forward to encoder_forward as it should have beenAndrej Karpathy committed 2 years ago2491402
  11. incorporate layernorm backward kernel 2, we are now down to 73.1ms/iter on averageAndrej Karpathy committed 2 years agoe3bcae6
  12. Merge branch 'sharedmem_layernormback' of https://github.com/ChrisDryden/llm.c into ChrisDryden-sharedmem_layernormbackAndrej Karpathy committed 2 years ago86b3401
  13. nice new kernel for bias reduce, we are down by ~1ms/iter, to 76.13ms/iter on averageAndrej Karpathy committed 2 years ago9884895
  14. Merge branch 'matmul-backward-bias' of https://github.com/al0vya/llm.c into al0vya-matmul-backward-biasAndrej Karpathy committed 2 years agoa42f739
  15. rename to _forward for the kernel, and small fixes to docs, and avoid autoAndrej Karpathy committed 2 years agob1e5595
  16. add more comments to explain the philosophy behind the kernelcip19aac committed 2 years ago35393b4
  17. add commentcip19aac committed 2 years agob82ec20
  18. add a faster matmul backward bias kernel that uses coalesced reads and shared memorycip19aac committed 2 years ago55fbd7e
  19. added NaN-based makes for reference checksErik Schultheis committed 2 years ago732a8b4
  20. added triangular matrix multiplication kernelErik Schultheis committed 2 years agocaa69a5
  21. Added shared memory for the atomic additions for the layernorm_backward.cu weights and biasesChris Dryden committed 2 years ago2752212
  22. Merge branch 'karpathy:master' into masterpatricxu committed 2 years ago41c3cc1
  23. change baseline kernel to float, for now, for fair comparison tooAndrej Karpathy committed 2 years ago9fb9c91
  24. speed up the backward bias kernel by 45% and speed up the full running time by 1%Andrej Karpathy committed 2 years ago8488669
  25. a comment and some breathing roomAndrej Karpathy committed 2 years ago19e337b
  26. Merge branch 'unistd_h_in_dev' of https://github.com/rosslwheeler/llm.c into rosslwheeler-unistd_h_in_devAndrej Karpathy committed 2 years ago7e42c8b
  27. slow attention kernel that demonstrates a direct translation of the CPU codeErik Schultheis committed 2 years ago8ffc3e3
  28. Adding unistd.h for Windows in Dev directoryRoss Wheeler committed 2 years agod245629
  29. fix indexing in matmul_forward_kernel1patricxu committed 2 years agod48b0d2
  30. Merge pull request #181 from lancerts/const-fix-adamwAndrej committed 2 years ago2d3a0fb
  31. fix kernel 5 of layernorm, bug from previous copy paste with loop boundsAndrej Karpathy committed 2 years ago91aab62
  32. add one more kernel, allocating a block per row. bad idea if C is too low, as we have it right nowAndrej Karpathy committed 2 years ago49d41ae
  33. new kernel that does a single pass over x on load, using a more clever variance formula. only very slightly faster on my A100 sadlyAndrej Karpathy committed 2 years agocb791c4
  34. one more version of attention backward for another few percent perf gainsErik Schultheis committed 2 years ago44b5bb0
  35. some low-level optimizationsErik Schultheis committed 2 years agoa360f10