Code

History

Commits on separate_tokenizer to dev35+
  1. Merge remote-tracking branch 'karpathy/master' into layernormbf16ademeure committed 2 years agod23bb1f
  2. Fix sizeof(T) -> sizeof(D) (was overallocating memory)ademeure committed 2 years ago49ffd68
  3. Only enable kernel4 with BF16ademeure committed 2 years ago1e50f3b
  4. Use kernel6 for train_gpt2, and add all other kernels to /dev/cuda (+bugfixes)ademeure committed 2 years ago82fef03
  5. incorporate faster encoder_forward kernel to fp32 CUDA versionAndrej Karpathy committed 2 years ago327eef3
  6. amend the float4 kernellancer committed 2 years ago1c7d23a
  7. amend the float4 kernellancer committed 2 years ago61c5c05
  8. Include the float4 kernellancer committed 2 years agocfccd82
  9. slight docs tweaks to dev/cudaAndrej Karpathy committed 2 years ago12da2c1
  10. Use a make pattern rule to simplify Makefile, group kernels into forward/backward/updates/nccls, add some comments and update the README.mdaustinvhuang committed 2 years ago25d703f
  11. Add a local convenience Makefile for dev/cuda/, fix 2 minor warnings observed when building all targets.austinvhuang committed 2 years agod937f93
  12. fix bug with all_hostsname_hashes missing, re-arrange code a bit, make sure we only print on rank0Andrej Karpathy committed 2 years agoa00a3a8
  13. Merge branch 'multi_gpu_support' of https://github.com/PeterZhizhin/llm.c into PeterZhizhin-multi_gpu_supportAndrej Karpathy committed 2 years ago8389fba
  14. [Multi-GPU] llm.c now runs on multiple GPUs with NCCLPetr Zhizhin committed 2 years agod49e830
  15. encoder backward add, we want to get rid of atomics, hmmAndrej Karpathy committed 2 years ago250159e
  16. rename positional_forward to encoder_forward as it should have beenAndrej Karpathy committed 2 years ago2491402
  17. incorporate layernorm backward kernel 2, we are now down to 73.1ms/iter on averageAndrej Karpathy committed 2 years agoe3bcae6
  18. Merge branch 'sharedmem_layernormback' of https://github.com/ChrisDryden/llm.c into ChrisDryden-sharedmem_layernormbackAndrej Karpathy committed 2 years ago86b3401
  19. nice new kernel for bias reduce, we are down by ~1ms/iter, to 76.13ms/iter on averageAndrej Karpathy committed 2 years ago9884895
  20. Merge branch 'matmul-backward-bias' of https://github.com/al0vya/llm.c into al0vya-matmul-backward-biasAndrej Karpathy committed 2 years agoa42f739
  21. rename to _forward for the kernel, and small fixes to docs, and avoid autoAndrej Karpathy committed 2 years agob1e5595
  22. add more comments to explain the philosophy behind the kernelcip19aac committed 2 years ago35393b4
  23. add commentcip19aac committed 2 years agob82ec20
  24. add a faster matmul backward bias kernel that uses coalesced reads and shared memorycip19aac committed 2 years ago55fbd7e
  25. added NaN-based makes for reference checksErik Schultheis committed 2 years ago732a8b4
  26. added triangular matrix multiplication kernelErik Schultheis committed 2 years agocaa69a5
  27. Added shared memory for the atomic additions for the layernorm_backward.cu weights and biasesChris Dryden committed 2 years ago2752212
  28. Merge branch 'karpathy:master' into masterpatricxu committed 2 years ago41c3cc1
  29. change baseline kernel to float, for now, for fair comparison tooAndrej Karpathy committed 2 years ago9fb9c91
  30. speed up the backward bias kernel by 45% and speed up the full running time by 1%Andrej Karpathy committed 2 years ago8488669
  31. a comment and some breathing roomAndrej Karpathy committed 2 years ago19e337b
  32. Merge branch 'unistd_h_in_dev' of https://github.com/rosslwheeler/llm.c into rosslwheeler-unistd_h_in_devAndrej Karpathy committed 2 years ago7e42c8b
  33. slow attention kernel that demonstrates a direct translation of the CPU codeErik Schultheis committed 2 years ago8ffc3e3
  34. Adding unistd.h for Windows in Dev directoryRoss Wheeler committed 2 years agod245629
  35. fix indexing in matmul_forward_kernel1patricxu committed 2 years agod48b0d2