lael-labs/llm.c
CodeIssuesPull RequestsAntiqueWikiInsights
/
Files
ademeure-layernormbf16-3attempt_fix_zero1feature/bias_backward2feature/cublasltfeature/cudnnfeature/delete_backward_actfeature/finetune_llama31pyfeature/fix_int_overflowfeature/fp32_weight_master_copyfeature/gpt3v1feature/grad_accumfeature/grad_accum3feature/hellaswagfeature/hellaswagcfeature/hf_evalfeature/libdirfeature/load_bf16feature/lr_schedulersfeature/managed2feature/nccl-onlyfeature/openmpi_flexible_pathfeature/organize_datafeature/outliersfeature/overlap-backward-reducefeature/packed128feature/permuted_dataloaderfeature/pytorch_parityfeature/random_initfeature/recomputefeature/refactor_part2feature/restore_from_masterfeature/resumefeature/rolling_checkpointsfeature/sharded_datafeature/streamsfeature/weight_decay_2d_onlyfeature/wikitext103feature/write_checkpointsfix/illegalaccessfix/reshuffle_zero_gradgordicaleksa-fix_dataloader2lael-labsllama3masterdefaultmove/attentionmove/encoder_kernelmove/fused_classifiermove/gelumove/matmuloverfit-batch-actually
No branch matches.
View all branches
  • .github
  • dev
    • cpu
    • cuda
    • hellaswag.py
    • mmlu.py
    • unistd.h
  • doc
  • .gitignore
  • cudnn_att.cpp
  • LICENSE
  • Makefile
  • prepro_tinyshakespeare.py
  • prepro_tinystories.py
  • profile_gpt2.cu
  • profile_gpt2cu.py
  • README.md
  • requirements.txt
  • test_gpt2.c
  • test_gpt2.cu
  • test_gpt2_fp32.cu
  • tokenizer.h
  • train_gpt2.c
  • train_gpt2.cu
  • train_gpt2.py
  • train_gpt2_fp32.cu
  • utils.h
llm.c/dev
AKAndrej Karpathyfix a bug, we have to be careful to make sure we evaluate loss at the token just before the first completion token, because that is the prediction for the first token, and its accuracy8d55c4a2 years agoHistory
NameLast commit messageLast commit date
..
cpu
cuda
hellaswag.py
mmlu.py
unistd.h