bpf: Fix WARNING in bpf_tracing_link_release

The trampoline could be corrupted by the blindly 'tr->flags = BPF_TRAMP_F_TAIL_CALL_CTX' in verifier. 1. A fexit attached to a tail_call_reachable prog. 'tr->flags' became 'BPF_TRAMP_F_CALL_ORIG | BPF_TRAMP_F_TAIL_CALL_CTX'. And, the trampoline would poke the target prog's nop insn using jmp insn instead of call insn. 2. Another fexit loaded with the same tail_call_reachable prog target. 'tr->flags' became 'BPF_TRAMP_F_TAIL_CALL_CTX'. 3. Close the first fexit link. Due to no BPF_TRAMP_F_CALL_ORIG in 'tr->flags', the trampoline will fail to restore the prog's nop insn using call insn. [ 3.410719] WARNING: kernel/bpf/syscall.c:3551 at bpf_tracing_link_release+0x53/0x60, CPU#1: test_progs/98 ... [ 3.428793] bpf_link_free+0x58/0x130 [ 3.429293] bpf_link_release+0x23/0x30 Fix the warning by updating 'tr->flags' with '|=' and lock.

Leon Hwang committed 3 months ago

Commit 61aaa871 parent 2805abd3 files · +10 −1Browse files

Showing the changes to kernel (2 of 3 files). Show all

Unified · Split

2 files

kernel/bpf/trampoline.c +7 −0
kernel/bpf/verifier.c +1 −1

kernel/bpf/trampoline.c+7−0

@@ -670,6 +670,13 @@ out: · 669 unmodified lines
670	670	return ERR_PTR(err);
671	671	}
672	672
	673	+void bpf_trampoline_set_flags(struct bpf_trampoline *tr, u32 flags)
	674	+{
	675	+ trampoline_lock(tr);
	676	+ tr->flags \|= flags;
	677	+ trampoline_unlock(tr);
	678	+}
	679	+
673	680	static int bpf_trampoline_update(struct bpf_trampoline *tr, bool lock_direct_mutex,
674	681	const struct bpf_trampoline_ops ops, void data)
675	682	{

kernel/bpf/verifier.c+1−1

@@ -19523,7 +19523,7 @@ static int check_attach_btf_id(struct bpf_verifier_env *env) · 19522 unmodified lines
19523	19523	return -ENOMEM;
19524	19524
19525	19525	if (tgt_prog && tgt_prog->aux->tail_call_reachable)
19526		− tr->flags = BPF_TRAMP_F_TAIL_CALL_CTX;
	19526	+ bpf_trampoline_set_flags(tr, BPF_TRAMP_F_TAIL_CALL_CTX);
19527	19527
19528	19528	prog->aux->dst_trampoline = tr;
19529	19529	return 0;