check_atomic_rmw() open codes the mapping from a BPF_ATOMIC to the register it reads the old value into, the BPF_STX case of insn_def_regno() open codes the very same mapping a second time, the const folding and the liveness transfer functions a third and a fourth time, and BPF JITs need it as well to know which register a faulting BPF_PROBE_ATOMIC has to clear. Add a small helper so that all of them can share it. No functional change. The BPF_LOAD_ACQ case is there for the JITs, which do walk all instruction classes. const_reg_xfer() loses its explicit BPF_ATOMIC mode test since the helper checks class and mode itself; the BPF_PROBE_ATOMIC it additionally accepts cannot be seen there as it is only set from bpf_do_misc_fixups(), that is, after const folding has run. arg_track_xfer() keeps its mode test since that also guards the stack clearing next to it.
Showing the changes to include (1 of 5 files). Show all
Unified · Split
| @@ -414,6 +414,30 @@ static inline bool bpf_atomic_is_load_acq(const struct bpf_insn *insn) · 413 unmodified lines | ||
| 414 | 414 | insn->imm == BPF_LOAD_ACQ; |
| 415 | 415 | } |
| 416 | 416 | |
| 417 | +/* | |
| 418 | + * Given an instruction @insn, return the number of the BPF register that a | |
| 419 | + * BPF_ATOMIC reads the value at its memory operand into, or -1 if there is | |
| 420 | + * no such register. That is the register a BPF_PROBE_ATOMIC has to clear when | |
| 421 | + * the access faults. Like bpf_atomic_is_load_acq(), @insn is not assumed to | |
| 422 | + * be a BPF_ATOMIC here. | |
| 423 | + */ | |
| 424 | +static inline int bpf_atomic_load_reg(const struct bpf_insn *insn) | |
| 425 | +{ | |
| 426 | + if (BPF_CLASS(insn->code) != BPF_STX || | |
| 427 | + (BPF_MODE(insn->code) != BPF_ATOMIC && | |
| 428 | + BPF_MODE(insn->code) != BPF_PROBE_ATOMIC)) | |
| 429 | + return -1; | |
| 430 | + | |
| 431 | + switch (insn->imm) { | |
| 432 | + case BPF_LOAD_ACQ: | |
| 433 | + return insn->dst_reg; | |
| 434 | + case BPF_CMPXCHG: | |
| 435 | + return BPF_REG_0; | |
| 436 | + default: | |
| 437 | + return (insn->imm & BPF_FETCH) ? insn->src_reg : -1; | |
| 438 | + } | |
| 439 | +} | |
| 440 | + | |
| 417 | 441 | /* Memory store, *(uint *) (dst_reg + off16) = imm32 */ |
| 418 | 442 | |
| 419 | 443 | #define BPF_ST_MEM(SIZE, DST, OFF, IMM) \ |