check_atomic_rmw() open codes the mapping from a BPF_ATOMIC to the register it reads the old value into, the BPF_STX case of insn_def_regno() open codes the very same mapping a second time, the const folding and the liveness transfer functions a third and a fourth time, and BPF JITs need it as well to know which register a faulting BPF_PROBE_ATOMIC has to clear. Add a small helper so that all of them can share it. No functional change. The BPF_LOAD_ACQ case is there for the JITs, which do walk all instruction classes. const_reg_xfer() loses its explicit BPF_ATOMIC mode test since the helper checks class and mode itself; the BPF_PROBE_ATOMIC it additionally accepts cannot be seen there as it is only set from bpf_do_misc_fixups(), that is, after const folding has run. arg_track_xfer() keeps its mode test since that also guards the stack clearing next to it.
Showing the changes to include (1 of 5 files). Show all
Unified · Split
| @@ -414,6 +414,30 @@ static inline bool bpf_atomic_is_load_acq(const struct bpf_insn *insn) · 413 unmodified lines | |||
| 414 | insn->imm == BPF_LOAD_ACQ; | 414 | insn->imm == BPF_LOAD_ACQ; |
| 415 | } | 415 | } |
| 416 | 416 | ||
| 417 | /* | ||
| 418 | * Given an instruction @insn, return the number of the BPF register that a | ||
| 419 | * BPF_ATOMIC reads the value at its memory operand into, or -1 if there is | ||
| 420 | * no such register. That is the register a BPF_PROBE_ATOMIC has to clear when | ||
| 421 | * the access faults. Like bpf_atomic_is_load_acq(), @insn is not assumed to | ||
| 422 | * be a BPF_ATOMIC here. | ||
| 423 | */ | ||
| 424 | static inline int bpf_atomic_load_reg(const struct bpf_insn *insn) | ||
| 425 | { | ||
| 426 | if (BPF_CLASS(insn->code) != BPF_STX || | ||
| 427 | (BPF_MODE(insn->code) != BPF_ATOMIC && | ||
| 428 | BPF_MODE(insn->code) != BPF_PROBE_ATOMIC)) | ||
| 429 | return -1; | ||
| 430 | |||
| 431 | switch (insn->imm) { | ||
| 432 | case BPF_LOAD_ACQ: | ||
| 433 | return insn->dst_reg; | ||
| 434 | case BPF_CMPXCHG: | ||
| 435 | return BPF_REG_0; | ||
| 436 | default: | ||
| 437 | return (insn->imm & BPF_FETCH) ? insn->src_reg : -1; | ||
| 438 | } | ||
| 439 | } | ||
| 440 | |||
| 417 | /* Memory store, *(uint *) (dst_reg + off16) = imm32 */ | 441 | /* Memory store, *(uint *) (dst_reg + off16) = imm32 */ |
| 418 | 442 | ||
| 419 | #define BPF_ST_MEM(SIZE, DST, OFF, IMM) \ | 443 | #define BPF_ST_MEM(SIZE, DST, OFF, IMM) \ |