Merge tag 'vfs-6.17-rc1.pidfs' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs

Pull pidfs updates from Christian Brauner: - persistent info Persist exit and coredump information independent of whether anyone currently holds a pidfd for the struct pid. The current scheme allocated pidfs dentries on-demand repeatedly. This scheme is reaching it's limits as it makes it impossible to pin information that needs to be available after the task has exited or coredumped and that should not be lost simply because the pidfd got closed temporarily. The next opener should still see the stashed information. This is also a prerequisite for supporting extended attributes on pidfds to allow attaching meta information to them. If someone opens a pidfd for a struct pid a pidfs dentry is allocated and stashed in pid->stashed. Once the last pidfd for the struct pid is closed the pidfs dentry is released and removed from pid->stashed. So if 10 callers create a pidfs dentry for the same struct pid sequentially, i.e., each closing the pidfd before the other creates a new one then a new pidfs dentry is allocated every time. Because multiple tasks acquiring and releasing a pidfd for the same struct pid can race with each another a task may still find a valid pidfs entry from the previous task in pid->stashed and reuse it. Or it might find a dead dentry in there and fail to reuse it and so stashes a new pidfs dentry. Multiple tasks may race to stash a new pidfs dentry but only one will succeed, the other ones will put their dentry. The current scheme aims to ensure that a pidfs dentry for a struct pid can only be created if the task is still alive or if a pidfs dentry already existed before the task was reaped and so exit information has been was stashed in the pidfs inode. That's great except that it's buggy. If a pidfs dentry is stashed in pid->stashed after pidfs_exit() but before __unhash_process() is called we will return a pidfd for a reaped task without exit information being available. The pidfds_pid_valid() check does not guard against this race as it doens't sync at all with pidfs_exit(). The pid_has_task() check might be successful simply because we're before __unhash_process() but after pidfs_exit(). Introduce a new scheme where the lifetime of information associated with a pidfs entry (coredump and exit information) isn't bound to the lifetime of the pidfs inode but the struct pid itself. The first time a pidfs dentry is allocated for a struct pid a struct pidfs_attr will be allocated which will be used to store exit and coredump information. If all pidfs for the pidfs dentry are closed the dentry and inode can be cleaned up but the struct pidfs_attr will stick until the struct pid itself is freed. This will ensure minimal memory usage while persisting relevant information. The new scheme has various advantages. First, it allows to close the race where we end up handing out a pidfd for a reaped task for which no exit information is available. Second, it minimizes memory usage. Third, it allows to remove complex lifetime tracking via dentries when registering a struct pid with pidfs. There's no need to get or put a reference. Instead, the lifetime of exit and coredump information associated with a struct pid is bound to the lifetime of struct pid itself. - extended attributes Now that we have a way to persist information for pidfs dentries we can start supporting extended attributes on pidfds. This will allow userspace to attach meta information to tasks. One natural extension would be to introduce a custom pidfs.* extended attribute space and allow for the inheritance of extended attributes across fork() and exec(). The first simple scheme will allow privileged userspace to set trusted extended attributes on pidfs inodes. - Allow autonomous pidfs file handles Various filesystems such as pidfs and drm support opening file handles without having to require a file descriptor to identify the filesystem. The filesystem are global single instances and can be trivially identified solely on the information encoded in the file handle. This makes it possible to not have to keep or acquire a sentinal file descriptor just to pass it to open_by_handle_at() to identify the filesystem. That's especially useful when such sentinel file descriptor cannot or should not be acquired. For pidfs this means a file handle can function as full replacement for storing a pid in a file. Instead a file handle can be stored and reopened purely based on the file handle. Such autonomous file handles can be opened with or without specifying a a file descriptor. If no proper file descriptor is used the FD_PIDFS_ROOT sentinel must be passed. This allows us to define further special negative fd sentinels in the future. Userspace can trivially test for support by trying to open the file handle with an invalid file descriptor. - Allow pidfds for reaped tasks with SCM_PIDFD messages This is a logical continuation of the earlier work to create pidfds for reaped tasks through the SO_PEERPIDFD socket option merged in 923ea4d4482b ("Merge patch series "net, pidfs: enable handing out pidfds for reaped sk->sk_peer_pid""). - Two minor fixes: * Fold fs_struct->{lock,seq} into a seqlock * Don't bother with path_{get,put}() in unix_open_file() * tag 'vfs-6.17-rc1.pidfs' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (37 commits) don't bother with path_get()/path_put() in unix_open_file() fold fs_struct->{lock,seq} into a seqlock selftests: net: extend SCM_PIDFD test to cover stale pidfds af_unix: enable handing out pidfds for reaped tasks in SCM_PIDFD af_unix: stash pidfs dentry when needed af_unix/scm: fix whitespace errors af_unix: introduce and use scm_replace_pid() helper af_unix: introduce unix_skb_to_scm helper af_unix: rework unix_maybe_add_creds() to allow sleep selftests/pidfd: decode pidfd file handles withou having to specify an fd fhandle, pidfs: support open_by_handle_at() purely based on file handle uapi/fcntl: add FD_PIDFS_ROOT uapi/fcntl: add FD_INVALID fcntl/pidfd: redefine PIDFD_SELF_THREAD_GROUP uapi/fcntl: mark range as reserved fhandle: reflow get_path_anchor() pidfs: add pidfs_root_path() helper fhandle: rename to get_path_anchor() fhandle: hoist copy_from_user() above get_path_from_fd() fhandle: raise FILEID_IS_DIR in handle_type ...

Linus Torvalds committed 1 year ago

Commit 672dcda2 parents 7031769 1f531e326 files · +894 −381Browse files

Unified · Split

26 files

fs/coredump.c +0 −5
fs/d_path.c +4 −4
fs/exec.c +2 −2
fs/fhandle.c +30 −32
fs/fs_struct.c +14 −22
fs/internal.h +4 −0
fs/libfs.c +22 −12
fs/namei.c +4 −4
fs/pidfs.c +253 −183
include/linux/fs_struct.h +5 −6
include/linux/pid.h +9 −5
include/linux/pidfs.h +1 −2
include/net/scm.h +2 −2
include/uapi/linux/fcntl.h +18 −0
include/uapi/linux/pidfd.h +0 −15
kernel/fork.c +5 −5
kernel/pid.c +1 −1
net/core/scm.c +28 −4
net/unix/af_unix.c +48 −30
tools/testing/selftests/net/af_unix/scm_pidfd.c +173 −44
tools/testing/selftests/pidfd/.gitignore +2 −0
tools/testing/selftests/pidfd/Makefile +3 −2
tools/testing/selftests/pidfd/pidfd.h +5 −1
tools/testing/selftests/pidfd/pidfd_file_handle_test.c +60 −0
tools/testing/selftests/pidfd/pidfd_setattr_test.c +69 −0
tools/testing/selftests/pidfd/pidfd_xattr_test.c +132 −0

fs/coredump.c+0−5

 	retval = kernel_connect(socket, (struct sockaddr *)(&addr), addr_len,
 				O_NONBLOCK | SOCK_COREDUMP);
-	/*
-	 * ... Make sure to only put our reference after connect() took
-	 * its own reference keeping the pidfs entry alive ...
-	 */
-	pidfs_put_pid(cprm->pid);
 	if (retval) {
 		if (retval == -EAGAIN)

fs/d_path.c+4−4

@@ -241,9 +241,9 @@ static void get_fs_root_rcu(struct fs_struct fs, struct path root) · 240 unmodified lines
241	unsigned seq;	241	unsigned seq;
242		242
243	do {	243	do {
244	seq = read_seqcount_begin(&fs->seq);	244	seq = read_seqbegin(&fs->seq);
245	*root = fs->root;	245	*root = fs->root;
246	} while (read_seqcount_retry(&fs->seq, seq));	246	} while (read_seqretry(&fs->seq, seq));
247	}	247	}
248		248
249	/**	249	/**
@@ -385,10 +385,10 @@ static void get_fs_root_and_pwd_rcu(struct fs_struct fs, struct path root, · 135 unmodified lines
385	unsigned seq;	385	unsigned seq;
386		386
387	do {	387	do {
388	seq = read_seqcount_begin(&fs->seq);	388	seq = read_seqbegin(&fs->seq);
389	*root = fs->root;	389	*root = fs->root;
390	*pwd = fs->pwd;	390	*pwd = fs->pwd;
391	} while (read_seqcount_retry(&fs->seq, seq));	391	} while (read_seqretry(&fs->seq, seq));
392	}	392	}
393		393
394	/*	394	/*

fs/exec.c+2−2

@@ -1515,7 +1515,7 @@ static void check_unsafe_exec(struct linux_binprm *bprm) · 1514 unmodified lines
1515	* state is protected by cred_guard_mutex we hold.	1515	* state is protected by cred_guard_mutex we hold.
1516	*/	1516	*/
1517	n_fs = 1;	1517	n_fs = 1;
1518	spin_lock(&p->fs->lock);	1518	read_seqlock_excl(&p->fs->seq);
1519	rcu_read_lock();	1519	rcu_read_lock();
1520	for_other_threads(p, t) {	1520	for_other_threads(p, t) {
1521	if (t->fs == p->fs)	1521	if (t->fs == p->fs)
@@ -1528,7 +1528,7 @@ static void check_unsafe_exec(struct linux_binprm *bprm) · 6 unmodified lines
1528	bprm->unsafe \|= LSM_UNSAFE_SHARE;	1528	bprm->unsafe \|= LSM_UNSAFE_SHARE;
1529	else	1529	else
1530	p->fs->in_exec = 1;	1530	p->fs->in_exec = 1;
1531	spin_unlock(&p->fs->lock);	1531	read_sequnlock_excl(&p->fs->seq);
1532	}	1532	}
1533		1533
1534	static void bprm_fill_uid(struct linux_binprm bprm, struct file file)	1534	static void bprm_fill_uid(struct linux_binprm bprm, struct file file)

fs/fhandle.c+30−32

 		if (fh_flags & EXPORT_FH_CONNECTABLE) {
 			handle->handle_type |= FILEID_IS_CONNECTABLE;
 			if (d_is_dir(path->dentry))
-				fh_flags |= FILEID_IS_DIR;
+				handle->handle_type |= FILEID_IS_DIR;
 		}
 		retval = 0;
 	}
 	return err;
 }
-static int get_path_from_fd(int fd, struct path *root)
+static int get_path_anchor(int fd, struct path *root)
 {
-	if (fd == AT_FDCWD) {
+	if (fd >= 0) {
-		struct fs_struct *fs = current->fs;
-		spin_lock(&fs->lock);
-		*root = fs->pwd;
-		path_get(root);
-		spin_unlock(&fs->lock);
-	} else {
 		CLASS(fd, f)(fd);
 		if (fd_empty(f))
 			return -EBADF;
 		*root = fd_file(f)->f_path;
 		path_get(root);
+		return 0;
 	}
-	return 0;
+	if (fd == AT_FDCWD) {
+		get_fs_pwd(current->fs, root);
+		return 0;
+	}
+	if (fd == FD_PIDFS_ROOT) {
+		pidfs_get_root(root);
+		return 0;
+	}
+	return -EBADF;
 }
 static int vfs_dentry_acceptable(void *context, struct dentry *dentry)
 {
 	int retval = 0;
 	struct file_handle f_handle;
-	struct file_handle *handle = NULL;
+	struct file_handle *handle __free(kfree) = NULL;
 	struct handle_to_path_ctx ctx = {};
 	const struct export_operations *eops;
-	retval = get_path_from_fd(mountdirfd, &ctx.root);
+	if (copy_from_user(&f_handle, ufh, sizeof(struct file_handle)))
+		return -EFAULT;
+	if ((f_handle.handle_bytes > MAX_HANDLE_SZ) ||
+	    (f_handle.handle_bytes == 0))
+		return -EINVAL;
+	if (f_handle.handle_type < 0 ||
+	    FILEID_USER_FLAGS(f_handle.handle_type) & ~FILEID_VALID_USER_FLAGS)
+		return -EINVAL;
+	retval = get_path_anchor(mountdirfd, &ctx.root);
 	if (retval)
-		goto out_err;
+		return retval;
 	eops = ctx.root.mnt->mnt_sb->s_export_op;
 	if (eops && eops->permission)
 	if (retval)
 		goto out_path;
-	if (copy_from_user(&f_handle, ufh, sizeof(struct file_handle))) {
-		retval = -EFAULT;
-		goto out_path;
-	}
-	if ((f_handle.handle_bytes > MAX_HANDLE_SZ) ||
-	    (f_handle.handle_bytes == 0)) {
-		retval = -EINVAL;
-		goto out_path;
-	}
-	if (f_handle.handle_type < 0 ||
-	    FILEID_USER_FLAGS(f_handle.handle_type) & ~FILEID_VALID_USER_FLAGS) {
-		retval = -EINVAL;
-		goto out_path;
-	}
 	handle = kmalloc(struct_size(handle, f_handle, f_handle.handle_bytes),
 			 GFP_KERNEL);
 	if (!handle) {
 			   &ufh->f_handle,
 			   f_handle.handle_bytes)) {
 		retval = -EFAULT;
-		goto out_handle;
+		goto out_path;
 	}
 	/*
 	handle->handle_type &= ~FILEID_USER_FLAGS_MASK;
 	retval = do_handle_to_path(handle, path, &ctx);
-out_handle:
-	kfree(handle);
 out_path:
 	path_put(&ctx.root);
-out_err:
 	return retval;
 }

fs/fs_struct.c+14−22

 	struct path old_root;
 	path_get(path);
-	spin_lock(&fs->lock);
+	write_seqlock(&fs->seq);
-	write_seqcount_begin(&fs->seq);
 	old_root = fs->root;
 	fs->root = *path;
-	write_seqcount_end(&fs->seq);
+	write_sequnlock(&fs->seq);
-	spin_unlock(&fs->lock);
 	if (old_root.dentry)
 		path_put(&old_root);
 }
 	struct path old_pwd;
 	path_get(path);
-	spin_lock(&fs->lock);
+	write_seqlock(&fs->seq);
-	write_seqcount_begin(&fs->seq);
 	old_pwd = fs->pwd;
 	fs->pwd = *path;
-	write_seqcount_end(&fs->seq);
+	write_sequnlock(&fs->seq);
-	spin_unlock(&fs->lock);
 	if (old_pwd.dentry)
 		path_put(&old_pwd);
 		fs = p->fs;
 		if (fs) {
 			int hits = 0;
-			spin_lock(&fs->lock);
+			write_seqlock(&fs->seq);
-			write_seqcount_begin(&fs->seq);
 			hits += replace_path(&fs->root, old_root, new_root);
 			hits += replace_path(&fs->pwd, old_root, new_root);
-			write_seqcount_end(&fs->seq);
 			while (hits--) {
 				count++;
 				path_get(new_root);
 			}
-			spin_unlock(&fs->lock);
+			write_sequnlock(&fs->seq);
 		}
 		task_unlock(p);
 	}
 	if (fs) {
 		int kill;
 		task_lock(tsk);
-		spin_lock(&fs->lock);
+		read_seqlock_excl(&fs->seq);
 		tsk->fs = NULL;
 		kill = !--fs->users;
-		spin_unlock(&fs->lock);
+		read_sequnlock_excl(&fs->seq);
 		task_unlock(tsk);
 		if (kill)
 			free_fs_struct(fs);
 	if (fs) {
 		fs->users = 1;
 		fs->in_exec = 0;
-		spin_lock_init(&fs->lock);
+		seqlock_init(&fs->seq);
-		seqcount_spinlock_init(&fs->seq, &fs->lock);
 		fs->umask = old->umask;
-		spin_lock(&old->lock);
+		read_seqlock_excl(&old->seq);
 		fs->root = old->root;
 		path_get(&fs->root);
 		fs->pwd = old->pwd;
 		path_get(&fs->pwd);
-		spin_unlock(&old->lock);
+		read_sequnlock_excl(&old->seq);
 	}
 	return fs;
 }
 		return -ENOMEM;
 	task_lock(current);
-	spin_lock(&fs->lock);
+	read_seqlock_excl(&fs->seq);
 	kill = !--fs->users;
 	current->fs = new_fs;
-	spin_unlock(&fs->lock);
+	read_sequnlock_excl(&fs->seq);
 	task_unlock(current);
 	if (kill)
 /* to be mentioned only in INIT_TASK */
 struct fs_struct init_fs = {
 	.users		= 1,
-	.lock		= __SPIN_LOCK_UNLOCKED(init_fs.lock),
+	.seq		= __SEQLOCK_UNLOCKED(init_fs.seq),
-	.seq		= SEQCNT_SPINLOCK_ZERO(init_fs.seq, &init_fs.lock),
 	.umask		= 0022,
 };

fs/internal.h+4−0

 struct mnt_idmap *mnt_idmap_get(struct mnt_idmap *idmap);
 void mnt_idmap_put(struct mnt_idmap *idmap);
 struct stashed_operations {
+	struct dentry *(*stash_dentry)(struct dentry **stashed,
+				       struct dentry *dentry);
 	void (*put_data)(void *data);
 	int (*init_inode)(struct inode *inode, void *data);
 };
 int path_from_stashed(struct dentry **stashed, struct vfsmount *mnt, void *data,
 		      struct path *path);
 void stashed_dentry_prune(struct dentry *dentry);
+struct dentry *stash_dentry(struct dentry **stashed, struct dentry *dentry);
 struct dentry *stashed_dentry_get(struct dentry **stashed);
 /**
  * path_mounted - check whether path is mounted
 		       unsigned int query_flags);
 int anon_inode_setattr(struct mnt_idmap *idmap, struct dentry *dentry,
 		       struct iattr *attr);
+void pidfs_get_root(struct path *path);

fs/libfs.c+22−12

 	dentry = rcu_dereference(*stashed);
 	if (!dentry)
 		return NULL;
+	if (IS_ERR(dentry))
+		return dentry;
 	if (!lockref_get_not_dead(&dentry->d_lockref))
 		return NULL;
 	return dentry;
 	/* Notice when this is changed. */
 	WARN_ON_ONCE(!S_ISREG(inode->i_mode));
-	WARN_ON_ONCE(!IS_IMMUTABLE(inode));
 	dentry = d_alloc_anon(sb);
 	if (!dentry) {
 	return dentry;
 }
-static struct dentry *stash_dentry(struct dentry **stashed,
+struct dentry *stash_dentry(struct dentry **stashed, struct dentry *dentry)
-				   struct dentry *dentry)
 {
 	guard(rcu)();
 	for (;;) {
 int path_from_stashed(struct dentry **stashed, struct vfsmount *mnt, void *data,
 		      struct path *path)
 {
-	struct dentry *dentry;
+	struct dentry *dentry, *res;
 	const struct stashed_operations *sops = mnt->mnt_sb->s_fs_info;
 	/* See if dentry can be reused. */
-	path->dentry = stashed_dentry_get(stashed);
+	res = stashed_dentry_get(stashed);
-	if (path->dentry) {
+	if (IS_ERR(res))
+		return PTR_ERR(res);
+	if (res) {
 		sops->put_data(data);
-		goto out_path;
+		goto make_path;
 	}
 	/* Allocate a new dentry. */
 		return PTR_ERR(dentry);
 	/* Added a new dentry. @data is now owned by the filesystem. */
-	path->dentry = stash_dentry(stashed, dentry);
+	if (sops->stash_dentry)
-	if (path->dentry != dentry)
+		res = sops->stash_dentry(stashed, dentry);
+	else
+		res = stash_dentry(stashed, dentry);
+	if (IS_ERR(res)) {
+		dput(dentry);
+		return PTR_ERR(res);
+	}
+	if (res != dentry)
 		dput(dentry);
-out_path:
+make_path:
-	WARN_ON_ONCE(path->dentry->d_fsdata != stashed);
+	path->dentry = res;
-	WARN_ON_ONCE(d_inode(path->dentry)->i_private != data);
 	path->mnt = mntget(mnt);
+	VFS_WARN_ON_ONCE(path->dentry->d_fsdata != stashed);
+	VFS_WARN_ON_ONCE(d_inode(path->dentry)->i_private != data);
 	return 0;
 }

fs/namei.c+4−4

@@ -1012,10 +1012,10 @@ static int set_root(struct nameidata *nd) · 1011 unmodified lines
1012	unsigned seq;	1012	unsigned seq;
1013		1013
1014	do {	1014	do {
1015	seq = read_seqcount_begin(&fs->seq);	1015	seq = read_seqbegin(&fs->seq);
1016	nd->root = fs->root;	1016	nd->root = fs->root;
1017	nd->root_seq = __read_seqcount_begin(&nd->root.dentry->d_seq);	1017	nd->root_seq = __read_seqcount_begin(&nd->root.dentry->d_seq);
1018	} while (read_seqcount_retry(&fs->seq, seq));	1018	} while (read_seqretry(&fs->seq, seq));
1019	} else {	1019	} else {
1020	get_fs_root(fs, &nd->root);	1020	get_fs_root(fs, &nd->root);
1021	nd->state \|= ND_ROOT_GRABBED;	1021	nd->state \|= ND_ROOT_GRABBED;
@@ -2571,11 +2571,11 @@ static const char path_init(struct nameidata nd, unsigned flags) · 1549 unmodified lines
2571	unsigned seq;	2571	unsigned seq;
2572		2572
2573	do {	2573	do {
2574	seq = read_seqcount_begin(&fs->seq);	2574	seq = read_seqbegin(&fs->seq);
2575	nd->path = fs->pwd;	2575	nd->path = fs->pwd;
2576	nd->inode = nd->path.dentry->d_inode;	2576	nd->inode = nd->path.dentry->d_inode;
2577	nd->seq = __read_seqcount_begin(&nd->path.dentry->d_seq);	2577	nd->seq = __read_seqcount_begin(&nd->path.dentry->d_seq);
2578	} while (read_seqcount_retry(&fs->seq, seq));	2578	} while (read_seqretry(&fs->seq, seq));
2579	} else {	2579	} else {
2580	get_fs_pwd(current->fs, &nd->path);	2580	get_fs_pwd(current->fs, &nd->path);
2581	nd->inode = nd->path.dentry->d_inode;	2581	nd->inode = nd->path.dentry->d_inode;

fs/pidfs.c+253−183

@@ -21,11 +21,23 @@ · 20 unmodified lines
21	#include <linux/utsname.h>	21	#include <linux/utsname.h>
22	#include <net/net_namespace.h>	22	#include <net/net_namespace.h>
23	#include <linux/coredump.h>	23	#include <linux/coredump.h>
		24	#include <linux/xattr.h>
24		25
25	#include "internal.h"	26	#include "internal.h"
26	#include "mount.h"	27	#include "mount.h"
27		28
28	static struct kmem_cache *pidfs_cachep __ro_after_init;	29	#define PIDFS_PID_DEAD ERR_PTR(-ESRCH)
		30
		31	static struct kmem_cache *pidfs_attr_cachep __ro_after_init;
		32	static struct kmem_cache *pidfs_xattr_cachep __ro_after_init;
		33
		34	static struct path pidfs_root_path = {};
		35
		36	void pidfs_get_root(struct path *path)
		37	{
		38	*path = pidfs_root_path;
		39	path_get(path);
		40	}
29		41
30	/*	42	/*
31	* Stashes information that userspace needs to access even after the	43	* Stashes information that userspace needs to access even after the
@@ -37,17 +49,12 @@ struct pidfs_exit_info { · 5 unmodified lines
37	__u32 coredump_mask;	49	__u32 coredump_mask;
38	};	50	};
39		51
40	struct pidfs_inode {	52	struct pidfs_attr {
		53	struct simple_xattrs *xattrs;
41	struct pidfs_exit_info __pei;	54	struct pidfs_exit_info __pei;
42	struct pidfs_exit_info *exit_info;	55	struct pidfs_exit_info *exit_info;
43	struct inode vfs_inode;
44	};	56	};
45		57
46	static inline struct pidfs_inode pidfs_i(struct inode inode)
47	{
48	return container_of(inode, struct pidfs_inode, vfs_inode);
49	}
50
51	static struct rb_root pidfs_ino_tree = RB_ROOT;	58	static struct rb_root pidfs_ino_tree = RB_ROOT;
52		59
53	#if BITS_PER_LONG == 32	60	#if BITS_PER_LONG == 32
@@ -125,6 +132,7 @@ void pidfs_add_pid(struct pid *pid) · 71 unmodified lines
125		132
126	pid->ino = pidfs_ino_nr;	133	pid->ino = pidfs_ino_nr;
127	pid->stashed = NULL;	134	pid->stashed = NULL;
		135	pid->attr = NULL;
128	pidfs_ino_nr++;	136	pidfs_ino_nr++;
129		137
130	write_seqcount_begin(&pidmap_lock_seq);	138	write_seqcount_begin(&pidmap_lock_seq);
@@ -139,6 +147,33 @@ void pidfs_remove_pid(struct pid *pid) · 8 unmodified lines
139	write_seqcount_end(&pidmap_lock_seq);	147	write_seqcount_end(&pidmap_lock_seq);
140	}	148	}
141		149
		150	void pidfs_free_pid(struct pid *pid)
		151	{
		152	struct pidfs_attr *attr __free(kfree) = no_free_ptr(pid->attr);
		153	struct simple_xattrs *xattrs __free(kfree) = NULL;
		154
		155	/*
		156	* Any dentry must've been wiped from the pid by now.
		157	* Otherwise there's a reference count bug.
		158	*/
		159	VFS_WARN_ON_ONCE(pid->stashed);
		160
		161	/*
		162	* This if an error occurred during e.g., task creation that
		163	* causes us to never go through the exit path.
		164	*/
		165	if (unlikely(!attr))
		166	return;
		167
		168	/* This never had a pidfd created. */
		169	if (IS_ERR(attr))
		170	return;
		171
		172	xattrs = no_free_ptr(attr->xattrs);
		173	if (xattrs)
		174	simple_xattrs_free(xattrs, NULL);
		175	}
		176
142	#ifdef CONFIG_PROC_FS	177	#ifdef CONFIG_PROC_FS
143	/**	178	/**
144	* pidfd_show_fdinfo - print information about a pidfd	179	* pidfd_show_fdinfo - print information about a pidfd
@@ -261,13 +296,13 @@ static __u32 pidfs_coredump_mask(unsigned long mm_flags) · 116 unmodified lines
261	static long pidfd_info(struct file *file, unsigned int cmd, unsigned long arg)	296	static long pidfd_info(struct file *file, unsigned int cmd, unsigned long arg)
262	{	297	{
263	struct pidfd_info __user uinfo = (struct pidfd_info __user )arg;	298	struct pidfd_info __user uinfo = (struct pidfd_info __user )arg;
264	struct inode *inode = file_inode(file);
265	struct pid *pid = pidfd_pid(file);	299	struct pid *pid = pidfd_pid(file);
266	size_t usize = _IOC_SIZE(cmd);	300	size_t usize = _IOC_SIZE(cmd);
267	struct pidfd_info kinfo = {};	301	struct pidfd_info kinfo = {};
268	struct pidfs_exit_info *exit_info;	302	struct pidfs_exit_info *exit_info;
269	struct user_namespace *user_ns;	303	struct user_namespace *user_ns;
270	struct task_struct *task;	304	struct task_struct *task;
		305	struct pidfs_attr *attr;
271	const struct cred *c;	306	const struct cred *c;
272	__u64 mask;	307	__u64 mask;
273		308
@@ -286,8 +321,9 @@ static long pidfd_info(struct file *file, unsigned int cmd, unsigned long arg) · 12 unmodified lines
286	if (!pid_in_current_pidns(pid))	321	if (!pid_in_current_pidns(pid))
287	return -ESRCH;	322	return -ESRCH;
288		323
		324	attr = READ_ONCE(pid->attr);
289	if (mask & PIDFD_INFO_EXIT) {	325	if (mask & PIDFD_INFO_EXIT) {
290	exit_info = READ_ONCE(pidfs_i(inode)->exit_info);	326	exit_info = READ_ONCE(attr->exit_info);
291	if (exit_info) {	327	if (exit_info) {
292	kinfo.mask \|= PIDFD_INFO_EXIT;	328	kinfo.mask \|= PIDFD_INFO_EXIT;
293	#ifdef CONFIG_CGROUPS	329	#ifdef CONFIG_CGROUPS
@@ -300,7 +336,7 @@ static long pidfd_info(struct file *file, unsigned int cmd, unsigned long arg) · 6 unmodified lines
300		336
301	if (mask & PIDFD_INFO_COREDUMP) {	337	if (mask & PIDFD_INFO_COREDUMP) {
302	kinfo.mask \|= PIDFD_INFO_COREDUMP;	338	kinfo.mask \|= PIDFD_INFO_COREDUMP;
303	kinfo.coredump_mask = READ_ONCE(pidfs_i(inode)->__pei.coredump_mask);	339	kinfo.coredump_mask = READ_ONCE(attr->__pei.coredump_mask);
304	}	340	}
305		341
306	task = get_pid_task(pid, PIDTYPE_PID);	342	task = get_pid_task(pid, PIDTYPE_PID);
@@ -552,41 +588,61 @@ struct pid pidfd_pid(const struct file file) · 245 unmodified lines
552	* task has been reaped which cannot happen until we're out of	588	* task has been reaped which cannot happen until we're out of
553	* release_task().	589	* release_task().
554	*	590	*
555	* If this struct pid is referred to by a pidfd then	591	* If this struct pid has at least once been referred to by a pidfd then
556	* stashed_dentry_get() will return the dentry and inode for that struct	592	* pid->attr will be allocated. If not we mark the struct pid as dead so
557	* pid. Since we've taken a reference on it there's now an additional	593	* anyone who is trying to register it with pidfs will fail to do so.
558	* reference from the exit path on it. Which is fine. We're going to put	594	* Otherwise we would hand out pidfs for reaped tasks without having
559	* it again in a second and we know that the pid is kept alive anyway.	595	* exit information available.
560	*	596	*
561	* Worst case is that we've filled in the info and immediately free the	597	* Worst case is that we've filled in the info and the pid gets freed
562	* dentry and inode afterwards since the pidfd has been closed. Since	598	* right away in free_pid() when no one holds a pidfd anymore. Since
563	* pidfs_exit() currently is placed after exit_task_work() we know that	599	* pidfs_exit() currently is placed after exit_task_work() we know that
564	* it cannot be us aka the exiting task holding a pidfd to ourselves.	600	* it cannot be us aka the exiting task holding a pidfd to itself.
565	*/	601	*/
566	void pidfs_exit(struct task_struct *tsk)	602	void pidfs_exit(struct task_struct *tsk)
567	{	603	{
568	struct dentry *dentry;	604	struct pid *pid = task_pid(tsk);
		605	struct pidfs_attr *attr;
		606	struct pidfs_exit_info *exit_info;
		607	#ifdef CONFIG_CGROUPS
		608	struct cgroup *cgrp;
		609	#endif
569		610
570	might_sleep();	611	might_sleep();
571		612
572	dentry = stashed_dentry_get(&task_pid(tsk)->stashed);	613	guard(spinlock_irq)(&pid->wait_pidfd.lock);
573	if (dentry) {	614	attr = pid->attr;
574	struct inode *inode = d_inode(dentry);	615	if (!attr) {
575	struct pidfs_exit_info *exit_info = &pidfs_i(inode)->__pei;	616	/*
576	#ifdef CONFIG_CGROUPS	617	* No one ever held a pidfd for this struct pid.
577	struct cgroup *cgrp;	618	* Mark it as dead so no one can add a pidfs
		619	* entry anymore. We're about to be reaped and
		620	* so no exit information would be available.
		621	*/
		622	pid->attr = PIDFS_PID_DEAD;
		623	return;
		624	}
578		625
579	rcu_read_lock();	626	/*
580	cgrp = task_dfl_cgroup(tsk);	627	* If @pid->attr is set someone might still legitimately hold a
581	exit_info->cgroupid = cgroup_id(cgrp);	628	* pidfd to @pid or someone might concurrently still be getting
582	rcu_read_unlock();	629	* a reference to an already stashed dentry from @pid->stashed.
		630	* So defer cleaning @pid->attr until the last reference to @pid
		631	* is put
		632	*/
		633
		634	exit_info = &attr->__pei;
		635
		636	#ifdef CONFIG_CGROUPS
		637	rcu_read_lock();
		638	cgrp = task_dfl_cgroup(tsk);
		639	exit_info->cgroupid = cgroup_id(cgrp);
		640	rcu_read_unlock();
583	#endif	641	#endif
584	exit_info->exit_code = tsk->exit_code;	642	exit_info->exit_code = tsk->exit_code;
585		643
586	/* Ensure that PIDFD_GET_INFO sees either all or nothing. */	644	/* Ensure that PIDFD_GET_INFO sees either all or nothing. */
587	smp_store_release(&pidfs_i(inode)->exit_info, &pidfs_i(inode)->__pei);	645	smp_store_release(&attr->exit_info, &attr->__pei);
588	dput(dentry);
589	}
590	}	646	}
591		647
592	#ifdef CONFIG_COREDUMP	648	#ifdef CONFIG_COREDUMP
@@ -594,16 +650,15 @@ void pidfs_coredump(const struct coredump_params *cprm) · 1 unmodified line
594	{	650	{
595	struct pid *pid = cprm->pid;	651	struct pid *pid = cprm->pid;
596	struct pidfs_exit_info *exit_info;	652	struct pidfs_exit_info *exit_info;
597	struct dentry *dentry;	653	struct pidfs_attr *attr;
598	struct inode *inode;
599	__u32 coredump_mask = 0;	654	__u32 coredump_mask = 0;
600		655
601	dentry = pid->stashed;	656	attr = READ_ONCE(pid->attr);
602	if (WARN_ON_ONCE(!dentry))
603	return;
604		657
605	inode = d_inode(dentry);	658	VFS_WARN_ON_ONCE(!attr);
606	exit_info = &pidfs_i(inode)->__pei;	659	VFS_WARN_ON_ONCE(attr == PIDFS_PID_DEAD);
		660
		661	exit_info = &attr->__pei;
607	/* Note how we were coredumped. */	662	/* Note how we were coredumped. */
608	coredump_mask = pidfs_coredump_mask(cprm->mm_flags);	663	coredump_mask = pidfs_coredump_mask(cprm->mm_flags);
609	/* Note that we actually did coredump. */	664	/* Note that we actually did coredump. */
@@ -634,9 +689,24 @@ static int pidfs_getattr(struct mnt_idmap idmap, const struct path path, · 24 unmodified lines
634	return anon_inode_getattr(idmap, path, stat, request_mask, query_flags);	689	return anon_inode_getattr(idmap, path, stat, request_mask, query_flags);
635	}	690	}
636		691
		692	static ssize_t pidfs_listxattr(struct dentry dentry, char buf, size_t size)
		693	{
		694	struct inode *inode = d_inode(dentry);
		695	struct pid *pid = inode->i_private;
		696	struct pidfs_attr *attr = pid->attr;
		697	struct simple_xattrs *xattrs;
		698
		699	xattrs = READ_ONCE(attr->xattrs);
		700	if (!xattrs)
		701	return 0;
		702
		703	return simple_xattr_list(inode, xattrs, buf, size);
		704	}
		705
637	static const struct inode_operations pidfs_inode_operations = {	706	static const struct inode_operations pidfs_inode_operations = {
638	.getattr = pidfs_getattr,	707	.getattr = pidfs_getattr,
639	.setattr = pidfs_setattr,	708	.setattr = pidfs_setattr,
		709	.listxattr = pidfs_listxattr,
640	};	710	};
641		711
642	static void pidfs_evict_inode(struct inode *inode)	712	static void pidfs_evict_inode(struct inode *inode)
@@ -647,30 +717,9 @@ static void pidfs_evict_inode(struct inode *inode) · 4 unmodified lines
647	put_pid(pid);	717	put_pid(pid);
648	}	718	}
649		719
650	static struct inode pidfs_alloc_inode(struct super_block sb)
651	{
652	struct pidfs_inode *pi;
653
654	pi = alloc_inode_sb(sb, pidfs_cachep, GFP_KERNEL);
655	if (!pi)
656	return NULL;
657
658	memset(&pi->__pei, 0, sizeof(pi->__pei));
659	pi->exit_info = NULL;
660
661	return &pi->vfs_inode;
662	}
663
664	static void pidfs_free_inode(struct inode *inode)
665	{
666	kmem_cache_free(pidfs_cachep, pidfs_i(inode));
667	}
668
669	static const struct super_operations pidfs_sops = {	720	static const struct super_operations pidfs_sops = {
670	.alloc_inode = pidfs_alloc_inode,
671	.drop_inode = generic_delete_inode,	721	.drop_inode = generic_delete_inode,
672	.evict_inode = pidfs_evict_inode,	722	.evict_inode = pidfs_evict_inode,
673	.free_inode = pidfs_free_inode,
674	.statfs = simple_statfs,	723	.statfs = simple_statfs,
675	};	724	};
676		725
@@ -770,6 +819,8 @@ static struct dentry pidfs_fh_to_dentry(struct super_block sb, · 93 unmodified lines
770	if (ret < 0)	819	if (ret < 0)
771	return ERR_PTR(ret);	820	return ERR_PTR(ret);
772		821
		822	VFS_WARN_ON_ONCE(!pid->attr);
		823
773	mntput(path.mnt);	824	mntput(path.mnt);
774	return path.dentry;	825	return path.dentry;
775	}	826	}
@@ -796,53 +847,8 @@ static int pidfs_export_permission(struct handle_to_path_ctx *ctx, · 20 unmodified lines
796	return 0;	847	return 0;
797	}	848	}
798		849
799	static inline bool pidfs_pid_valid(struct pid pid, const struct path path,
800	unsigned int flags)
801	{
802	enum pid_type type;
803
804	if (flags & PIDFD_STALE)
805	return true;
806
807	/*
808	* Make sure that if a pidfd is created PIDFD_INFO_EXIT
809	* information will be available. So after an inode for the
810	* pidfd has been allocated perform another check that the pid
811	* is still alive. If it is exit information is available even
812	* if the task gets reaped before the pidfd is returned to
813	* userspace. The only exception are indicated by PIDFD_STALE:
814	*
815	* (1) The kernel is in the middle of task creation and thus no
816	* task linkage has been established yet.
817	* (2) The caller knows @pid has been registered in pidfs at a
818	* time when the task was still alive.
819	*
820	* In both cases exit information will have been reported.
821	*/
822	if (flags & PIDFD_THREAD)
823	type = PIDTYPE_PID;
824	else
825	type = PIDTYPE_TGID;
826
827	/*
828	* Since pidfs_exit() is called before struct pid's task linkage
829	* is removed the case where the task got reaped but a dentry
830	* was already attached to struct pid and exit information was
831	* recorded and published can be handled correctly.
832	*/
833	if (unlikely(!pid_has_task(pid, type))) {
834	struct inode *inode = d_inode(path->dentry);
835	return !!READ_ONCE(pidfs_i(inode)->exit_info);
836	}
837
838	return true;
839	}
840
841	static struct file pidfs_export_open(struct path path, unsigned int oflags)	850	static struct file pidfs_export_open(struct path path, unsigned int oflags)
842	{	851	{
843	if (!pidfs_pid_valid(d_inode(path->dentry)->i_private, path, oflags))
844	return ERR_PTR(-ESRCH);
845
846	/*	852	/*
847	* Clear O_LARGEFILE as open_by_handle_at() forces it and raise	853	* Clear O_LARGEFILE as open_by_handle_at() forces it and raise
848	* O_RDWR as pidfds always are.	854	* O_RDWR as pidfds always are.
@@ -864,6 +870,8 @@ static int pidfs_init_inode(struct inode inode, void data) · 15 unmodified lines
864		870
865	inode->i_private = data;	871	inode->i_private = data;
866	inode->i_flags \|= S_PRIVATE \| S_ANON_INODE;	872	inode->i_flags \|= S_PRIVATE \| S_ANON_INODE;
		873	/* We allow to set xattrs. */
		874	inode->i_flags &= ~S_IMMUTABLE;
867	inode->i_mode \|= S_IRWXU;	875	inode->i_mode \|= S_IRWXU;
868	inode->i_op = &pidfs_inode_operations;	876	inode->i_op = &pidfs_inode_operations;
869	inode->i_fop = &pidfs_file_operations;	877	inode->i_fop = &pidfs_file_operations;
@@ -878,9 +886,127 @@ static void pidfs_put_data(void *data) · 8 unmodified lines
878	put_pid(pid);	886	put_pid(pid);
879	}	887	}
880		888
		889	/**
		890	* pidfs_register_pid - register a struct pid in pidfs
		891	* @pid: pid to pin
		892	*
		893	* Register a struct pid in pidfs.
		894	*
		895	* Return: On success zero, on error a negative error code is returned.
		896	*/
		897	int pidfs_register_pid(struct pid *pid)
		898	{
		899	struct pidfs_attr *new_attr __free(kfree) = NULL;
		900	struct pidfs_attr *attr;
		901
		902	might_sleep();
		903
		904	if (!pid)
		905	return 0;
		906
		907	attr = READ_ONCE(pid->attr);
		908	if (unlikely(attr == PIDFS_PID_DEAD))
		909	return PTR_ERR(PIDFS_PID_DEAD);
		910	if (attr)
		911	return 0;
		912
		913	new_attr = kmem_cache_zalloc(pidfs_attr_cachep, GFP_KERNEL);
		914	if (!new_attr)
		915	return -ENOMEM;
		916
		917	/* Synchronize with pidfs_exit(). */
		918	guard(spinlock_irq)(&pid->wait_pidfd.lock);
		919
		920	attr = pid->attr;
		921	if (unlikely(attr == PIDFS_PID_DEAD))
		922	return PTR_ERR(PIDFS_PID_DEAD);
		923	if (unlikely(attr))
		924	return 0;
		925
		926	pid->attr = no_free_ptr(new_attr);
		927	return 0;
		928	}
		929
		930	static struct dentry pidfs_stash_dentry(struct dentry *stashed,
		931	struct dentry *dentry)
		932	{
		933	int ret;
		934	struct pid *pid = d_inode(dentry)->i_private;
		935
		936	VFS_WARN_ON_ONCE(stashed != &pid->stashed);
		937
		938	ret = pidfs_register_pid(pid);
		939	if (ret)
		940	return ERR_PTR(ret);
		941
		942	return stash_dentry(stashed, dentry);
		943	}
		944
881	static const struct stashed_operations pidfs_stashed_ops = {	945	static const struct stashed_operations pidfs_stashed_ops = {
882	.init_inode = pidfs_init_inode,	946	.stash_dentry = pidfs_stash_dentry,
883	.put_data = pidfs_put_data,	947	.init_inode = pidfs_init_inode,
		948	.put_data = pidfs_put_data,
		949	};
		950
		951	static int pidfs_xattr_get(const struct xattr_handler *handler,
		952	struct dentry unused, struct inode inode,
		953	const char suffix, void value, size_t size)
		954	{
		955	struct pid *pid = inode->i_private;
		956	struct pidfs_attr *attr = pid->attr;
		957	const char *name;
		958	struct simple_xattrs *xattrs;
		959
		960	xattrs = READ_ONCE(attr->xattrs);
		961	if (!xattrs)
		962	return 0;
		963
		964	name = xattr_full_name(handler, suffix);
		965	return simple_xattr_get(xattrs, name, value, size);
		966	}
		967
		968	static int pidfs_xattr_set(const struct xattr_handler *handler,
		969	struct mnt_idmap idmap, struct dentry unused,
		970	struct inode inode, const char suffix,
		971	const void *value, size_t size, int flags)
		972	{
		973	struct pid *pid = inode->i_private;
		974	struct pidfs_attr *attr = pid->attr;
		975	const char *name;
		976	struct simple_xattrs *xattrs;
		977	struct simple_xattr *old_xattr;
		978
		979	/* Ensure we're the only one to set @attr->xattrs. */
		980	WARN_ON_ONCE(!inode_is_locked(inode));
		981
		982	xattrs = READ_ONCE(attr->xattrs);
		983	if (!xattrs) {
		984	xattrs = kmem_cache_zalloc(pidfs_xattr_cachep, GFP_KERNEL);
		985	if (!xattrs)
		986	return -ENOMEM;
		987
		988	simple_xattrs_init(xattrs);
		989	smp_store_release(&pid->attr->xattrs, xattrs);
		990	}
		991
		992	name = xattr_full_name(handler, suffix);
		993	old_xattr = simple_xattr_set(xattrs, name, value, size, flags);
		994	if (IS_ERR(old_xattr))
		995	return PTR_ERR(old_xattr);
		996
		997	simple_xattr_free(old_xattr);
		998	return 0;
		999	}
		1000
		1001	static const struct xattr_handler pidfs_trusted_xattr_handler = {
		1002	.prefix = XATTR_TRUSTED_PREFIX,
		1003	.get = pidfs_xattr_get,
		1004	.set = pidfs_xattr_set,
		1005	};
		1006
		1007	static const struct xattr_handler *const pidfs_xattr_handlers[] = {
		1008	&pidfs_trusted_xattr_handler,
		1009	NULL
884	};	1010	};
885		1011
886	static int pidfs_init_fs_context(struct fs_context *fc)	1012	static int pidfs_init_fs_context(struct fs_context *fc)
@@ -891,9 +1017,12 @@ static int pidfs_init_fs_context(struct fs_context *fc) · 4 unmodified lines
891	if (!ctx)	1017	if (!ctx)
892	return -ENOMEM;	1018	return -ENOMEM;
893		1019
		1020	fc->s_iflags \|= SB_I_NOEXEC;
		1021	fc->s_iflags \|= SB_I_NODEV;
894	ctx->ops = &pidfs_sops;	1022	ctx->ops = &pidfs_sops;
895	ctx->eops = &pidfs_export_operations;	1023	ctx->eops = &pidfs_export_operations;
896	ctx->dops = &pidfs_dentry_operations;	1024	ctx->dops = &pidfs_dentry_operations;
		1025	ctx->xattr = pidfs_xattr_handlers;
897	fc->s_fs_info = (void *)&pidfs_stashed_ops;	1026	fc->s_fs_info = (void *)&pidfs_stashed_ops;
898	return 0;	1027	return 0;
899	}	1028	}
@@ -921,8 +1050,7 @@ struct file pidfs_alloc_file(struct pid pid, unsigned int flags) · 21 unmodified lines
921	if (ret < 0)	1050	if (ret < 0)
922	return ERR_PTR(ret);	1051	return ERR_PTR(ret);
923		1052
924	if (!pidfs_pid_valid(pid, &path, flags))	1053	VFS_WARN_ON_ONCE(!pid->attr);
925	return ERR_PTR(-ESRCH);
926		1054
927	flags &= ~PIDFD_STALE;	1055	flags &= ~PIDFD_STALE;
928	flags \|= O_RDWR;	1056	flags \|= O_RDWR;
@@ -934,79 +1062,21 @@ struct file pidfs_alloc_file(struct pid pid, unsigned int flags) · 5 unmodified lines
934	return pidfd_file;	1062	return pidfd_file;
935	}	1063	}
936		1064
937	/**
938	* pidfs_register_pid - register a struct pid in pidfs
939	* @pid: pid to pin
940	*
941	* Register a struct pid in pidfs. Needs to be paired with
942	* pidfs_put_pid() to not risk leaking the pidfs dentry and inode.
943	*
944	* Return: On success zero, on error a negative error code is returned.
945	*/
946	int pidfs_register_pid(struct pid *pid)
947	{
948	struct path path __free(path_put) = {};
949	int ret;
950
951	might_sleep();
952
953	if (!pid)
954	return 0;
955
956	ret = path_from_stashed(&pid->stashed, pidfs_mnt, get_pid(pid), &path);
957	if (unlikely(ret))
958	return ret;
959	/* Keep the dentry and only put the reference to the mount. */
960	path.dentry = NULL;
961	return 0;
962	}
963
964	/**
965	* pidfs_get_pid - pin a struct pid through pidfs
966	* @pid: pid to pin
967	*
968	* Similar to pidfs_register_pid() but only valid if the caller knows
969	* there's a reference to the @pid through a dentry already that can't
970	* go away.
971	*/
972	void pidfs_get_pid(struct pid *pid)
973	{
974	if (!pid)
975	return;
976	WARN_ON_ONCE(!stashed_dentry_get(&pid->stashed));
977	}
978
979	/**
980	* pidfs_put_pid - drop a pidfs reference
981	* @pid: pid to drop
982	*
983	* Drop a reference to @pid via pidfs. This is only safe if the
984	* reference has been taken via pidfs_get_pid().
985	*/
986	void pidfs_put_pid(struct pid *pid)
987	{
988	might_sleep();
989
990	if (!pid)
991	return;
992	VFS_WARN_ON_ONCE(!pid->stashed);
993	dput(pid->stashed);
994	}
995
996	static void pidfs_inode_init_once(void *data)
997	{
998	struct pidfs_inode *pi = data;
999
1000	inode_init_once(&pi->vfs_inode);
1001	}
1002
1003	void __init pidfs_init(void)	1065	void __init pidfs_init(void)
1004	{	1066	{
1005	pidfs_cachep = kmem_cache_create("pidfs_cache", sizeof(struct pidfs_inode), 0,	1067	pidfs_attr_cachep = kmem_cache_create("pidfs_attr_cache", sizeof(struct pidfs_attr), 0,
1006	(SLAB_HWCACHE_ALIGN \| SLAB_RECLAIM_ACCOUNT \|	1068	(SLAB_HWCACHE_ALIGN \| SLAB_RECLAIM_ACCOUNT \|
1007	SLAB_ACCOUNT \| SLAB_PANIC),	1069	SLAB_ACCOUNT \| SLAB_PANIC), NULL);
1008	pidfs_inode_init_once);	1070
		1071	pidfs_xattr_cachep = kmem_cache_create("pidfs_xattr_cache",
		1072	sizeof(struct simple_xattrs), 0,
		1073	(SLAB_HWCACHE_ALIGN \| SLAB_RECLAIM_ACCOUNT \|
		1074	SLAB_ACCOUNT \| SLAB_PANIC), NULL);
		1075
1009	pidfs_mnt = kern_mount(&pidfs_type);	1076	pidfs_mnt = kern_mount(&pidfs_type);
1010	if (IS_ERR(pidfs_mnt))	1077	if (IS_ERR(pidfs_mnt))
1011	panic("Failed to mount pidfs pseudo filesystem");	1078	panic("Failed to mount pidfs pseudo filesystem");
		1079
		1080	pidfs_root_path.mnt = pidfs_mnt;
		1081	pidfs_root_path.dentry = pidfs_mnt->mnt_root;
1012	}	1082	}

include/linux/fs_struct.h+5−6

 struct fs_struct {
 	int users;
-	spinlock_t lock;
+	seqlock_t seq;
-	seqcount_spinlock_t seq;
 	int umask;
 	int in_exec;
 	struct path root, pwd;
 static inline void get_fs_root(struct fs_struct *fs, struct path *root)
 {
-	spin_lock(&fs->lock);
+	read_seqlock_excl(&fs->seq);
 	*root = fs->root;
 	path_get(root);
-	spin_unlock(&fs->lock);
+	read_sequnlock_excl(&fs->seq);
 }
 static inline void get_fs_pwd(struct fs_struct *fs, struct path *pwd)
 {
-	spin_lock(&fs->lock);
+	read_seqlock_excl(&fs->seq);
 	*pwd = fs->pwd;
 	path_get(pwd);
-	spin_unlock(&fs->lock);
+	read_sequnlock_excl(&fs->seq);
 }
 extern bool current_chrooted(void);

include/linux/pid.h+9−5

 #define RESERVED_PIDS 300
+struct pidfs_attr;
 struct upid {
 	int nr;
 	struct pid_namespace *ns;
 };
-struct pid
+struct pid {
-{
 	refcount_t count;
 	unsigned int level;
 	spinlock_t lock;
-	struct dentry *stashed;
+	struct {
-	u64 ino;
+		u64 ino;
-	struct rb_node pidfs_node;
+		struct rb_node pidfs_node;
+		struct dentry *stashed;
+		struct pidfs_attr *attr;
+	};
 	/* lists of tasks that use this pid */
 	struct hlist_head tasks[PIDTYPE_MAX];
 	struct hlist_head inodes;

include/linux/pidfs.h+1−2

@@ -14,7 +14,6 @@ void pidfs_coredump(const struct coredump_params *cprm); · 13 unmodified lines
14	#endif	14	#endif
15	extern const struct dentry_operations pidfs_dentry_operations;	15	extern const struct dentry_operations pidfs_dentry_operations;
16	int pidfs_register_pid(struct pid *pid);	16	int pidfs_register_pid(struct pid *pid);
17	void pidfs_get_pid(struct pid *pid);	17	void pidfs_free_pid(struct pid *pid);
18	void pidfs_put_pid(struct pid *pid);
19		18
20	#endif /* _LINUX_PID_FS_H */	19	#endif /* _LINUX_PID_FS_H */

include/net/scm.h+2−2

@@ -69,7 +69,7 @@ static __inline__ void unix_get_peersec_dgram(struct socket *sock, struct scm_co · 68 unmodified lines
69	static __inline__ void scm_set_cred(struct scm_cookie *scm,	69	static __inline__ void scm_set_cred(struct scm_cookie *scm,
70	struct pid *pid, kuid_t uid, kgid_t gid)	70	struct pid *pid, kuid_t uid, kgid_t gid)
71	{	71	{
72	scm->pid = get_pid(pid);	72	scm->pid = get_pid(pid);
73	scm->creds.pid = pid_vnr(pid);	73	scm->creds.pid = pid_vnr(pid);
74	scm->creds.uid = uid;	74	scm->creds.uid = uid;
75	scm->creds.gid = gid;	75	scm->creds.gid = gid;
@@ -78,7 +78,7 @@ static __inline__ void scm_set_cred(struct scm_cookie *scm, · 2 unmodified lines
78	static __inline__ void scm_destroy_cred(struct scm_cookie *scm)	78	static __inline__ void scm_destroy_cred(struct scm_cookie *scm)
79	{	79	{
80	put_pid(scm->pid);	80	put_pid(scm->pid);
81	scm->pid = NULL;	81	scm->pid = NULL;
82	}	82	}
83		83
84	static __inline__ void scm_destroy(struct scm_cookie *scm)	84	static __inline__ void scm_destroy(struct scm_cookie *scm)

include/uapi/linux/fcntl.h+18−0

@@ -90,10 +90,28 @@ · 89 unmodified lines
90	#define DN_ATTRIB 0x00000020 /* File changed attibutes */	90	#define DN_ATTRIB 0x00000020 /* File changed attibutes */
91	#define DN_MULTISHOT 0x80000000 /* Don't remove notifier */	91	#define DN_MULTISHOT 0x80000000 /* Don't remove notifier */
92		92
		93	/* Reserved kernel ranges [-100], [-10000, -40000]. */
93	#define AT_FDCWD -100 /* Special value for dirfd used to	94	#define AT_FDCWD -100 /* Special value for dirfd used to
94	indicate openat should use the	95	indicate openat should use the
95	current working directory. */	96	current working directory. */
96		97
		98	/*
		99	* The concept of process and threads in userland and the kernel is a confusing
		100	* one - within the kernel every thread is a 'task' with its own individual PID,
		101	* however from userland's point of view threads are grouped by a single PID,
		102	* which is that of the 'thread group leader', typically the first thread
		103	* spawned.
		104	*
		105	* To cut the Gideon knot, for internal kernel usage, we refer to
		106	* PIDFD_SELF_THREAD to refer to the current thread (or task from a kernel
		107	* perspective), and PIDFD_SELF_THREAD_GROUP to refer to the current thread
		108	* group leader...
		109	*/
		110	#define PIDFD_SELF_THREAD -10000 /* Current thread. */
		111	#define PIDFD_SELF_THREAD_GROUP -10001 /* Current thread group leader. */
		112
		113	#define FD_PIDFS_ROOT -10002 /* Root of the pidfs filesystem */
		114	#define FD_INVALID -10009 /* Invalid file descriptor: -10000 - EBADF = -10009 */
97		115
98	/* Generic flags for the at(2) family of syscalls. /	116	/* Generic flags for the at(2) family of syscalls. /
99		117

include/uapi/linux/pidfd.h+0−15

@@ -42,21 +42,6 @@ · 41 unmodified lines
42	#define PIDFD_COREDUMP_USER (1U << 2) /* coredump was done as the user. */	42	#define PIDFD_COREDUMP_USER (1U << 2) /* coredump was done as the user. */
43	#define PIDFD_COREDUMP_ROOT (1U << 3) /* coredump was done as root. */	43	#define PIDFD_COREDUMP_ROOT (1U << 3) /* coredump was done as root. */
44		44
45	/*
46	* The concept of process and threads in userland and the kernel is a confusing
47	* one - within the kernel every thread is a 'task' with its own individual PID,
48	* however from userland's point of view threads are grouped by a single PID,
49	* which is that of the 'thread group leader', typically the first thread
50	* spawned.
51	*
52	* To cut the Gideon knot, for internal kernel usage, we refer to
53	* PIDFD_SELF_THREAD to refer to the current thread (or task from a kernel
54	* perspective), and PIDFD_SELF_THREAD_GROUP to refer to the current thread
55	* group leader...
56	*/
57	#define PIDFD_SELF_THREAD -10000 /* Current thread. */
58	#define PIDFD_SELF_THREAD_GROUP -20000 /* Current thread group leader. */
59
60	/*	45	/*
61	* ...and for userland we make life simpler - PIDFD_SELF refers to the current	46	* ...and for userland we make life simpler - PIDFD_SELF refers to the current
62	* thread, PIDFD_SELF_PROCESS refers to the process thread group leader.	47	* thread, PIDFD_SELF_PROCESS refers to the process thread group leader.

kernel/fork.c+5−5

@@ -1542,14 +1542,14 @@ static int copy_fs(unsigned long clone_flags, struct task_struct *tsk) · 1541 unmodified lines
1542	struct fs_struct *fs = current->fs;	1542	struct fs_struct *fs = current->fs;
1543	if (clone_flags & CLONE_FS) {	1543	if (clone_flags & CLONE_FS) {
1544	/* tsk->fs is already what we want */	1544	/* tsk->fs is already what we want */
1545	spin_lock(&fs->lock);	1545	read_seqlock_excl(&fs->seq);
1546	/* "users" and "in_exec" locked for check_unsafe_exec() */	1546	/* "users" and "in_exec" locked for check_unsafe_exec() */
1547	if (fs->in_exec) {	1547	if (fs->in_exec) {
1548	spin_unlock(&fs->lock);	1548	read_sequnlock_excl(&fs->seq);
1549	return -EAGAIN;	1549	return -EAGAIN;
1550	}	1550	}
1551	fs->users++;	1551	fs->users++;
1552	spin_unlock(&fs->lock);	1552	read_sequnlock_excl(&fs->seq);
1553	return 0;	1553	return 0;
1554	}	1554	}
1555	tsk->fs = copy_fs_struct(fs);	1555	tsk->fs = copy_fs_struct(fs);
@@ -3149,13 +3149,13 @@ int ksys_unshare(unsigned long unshare_flags) · 1593 unmodified lines
3149		3149
3150	if (new_fs) {	3150	if (new_fs) {
3151	fs = current->fs;	3151	fs = current->fs;
3152	spin_lock(&fs->lock);	3152	read_seqlock_excl(&fs->seq);
3153	current->fs = new_fs;	3153	current->fs = new_fs;
3154	if (--fs->users)	3154	if (--fs->users)
3155	new_fs = NULL;	3155	new_fs = NULL;
3156	else	3156	else
3157	new_fs = fs;	3157	new_fs = fs;
3158	spin_unlock(&fs->lock);	3158	read_sequnlock_excl(&fs->seq);
3159	}	3159	}
3160		3160
3161	if (new_fd)	3161	if (new_fd)

kernel/pid.c+1−1

@@ -100,7 +100,7 @@ void put_pid(struct pid *pid) · 99 unmodified lines
100		100
101	ns = pid->numbers[pid->level].ns;	101	ns = pid->numbers[pid->level].ns;
102	if (refcount_dec_and_test(&pid->count)) {	102	if (refcount_dec_and_test(&pid->count)) {
103	WARN_ON_ONCE(pid->stashed);	103	pidfs_free_pid(pid);
104	kmem_cache_free(ns->pid_cachep, pid);	104	kmem_cache_free(ns->pid_cachep, pid);
105	put_pid_ns(ns);	105	put_pid_ns(ns);
106	}	106	}

net/core/scm.c+28−4

@@ -23,6 +23,8 @@ · 22 unmodified lines
23	#include <linux/security.h>	23	#include <linux/security.h>
24	#include <linux/pid_namespace.h>	24	#include <linux/pid_namespace.h>
25	#include <linux/pid.h>	25	#include <linux/pid.h>
		26	#include <uapi/linux/pidfd.h>
		27	#include <linux/pidfs.h>
26	#include <linux/nsproxy.h>	28	#include <linux/nsproxy.h>
27	#include <linux/slab.h>	29	#include <linux/slab.h>
28	#include <linux/errqueue.h>	30	#include <linux/errqueue.h>
@@ -145,6 +147,22 @@ void __scm_destroy(struct scm_cookie *scm) · 116 unmodified lines
145	}	147	}
146	EXPORT_SYMBOL(__scm_destroy);	148	EXPORT_SYMBOL(__scm_destroy);
147		149
		150	static inline int scm_replace_pid(struct scm_cookie scm, struct pid pid)
		151	{
		152	int err;
		153
		154	/* drop all previous references */
		155	scm_destroy_cred(scm);
		156
		157	err = pidfs_register_pid(pid);
		158	if (unlikely(err))
		159	return err;
		160
		161	scm->pid = pid;
		162	scm->creds.pid = pid_vnr(pid);
		163	return 0;
		164	}
		165
148	int __scm_send(struct socket sock, struct msghdr msg, struct scm_cookie *p)	166	int __scm_send(struct socket sock, struct msghdr msg, struct scm_cookie *p)
149	{	167	{
150	const struct proto_ops *ops = READ_ONCE(sock->ops);	168	const struct proto_ops *ops = READ_ONCE(sock->ops);
@@ -189,15 +207,21 @@ int __scm_send(struct socket sock, struct msghdr msg, struct scm_cookie *p) · 38 unmodified lines
189	if (err)	207	if (err)
190	goto error;	208	goto error;
191		209
192	p->creds.pid = creds.pid;
193	if (!p->pid \|\| pid_vnr(p->pid) != creds.pid) {	210	if (!p->pid \|\| pid_vnr(p->pid) != creds.pid) {
194	struct pid *pid;	211	struct pid *pid;
195	err = -ESRCH;	212	err = -ESRCH;
196	pid = find_get_pid(creds.pid);	213	pid = find_get_pid(creds.pid);
197	if (!pid)	214	if (!pid)
198	goto error;	215	goto error;
199	put_pid(p->pid);	216
200	p->pid = pid;	217	/* pass a struct pid reference from
		218	* find_get_pid() to scm_replace_pid().
		219	*/
		220	err = scm_replace_pid(p, pid);
		221	if (err) {
		222	put_pid(pid);
		223	goto error;
		224	}
201	}	225	}
202		226
203	err = -EINVAL;	227	err = -EINVAL;
@@ -459,7 +483,7 @@ static void scm_pidfd_recv(struct msghdr msg, struct scm_cookie scm) · 255 unmodified lines
459	if (!scm->pid)	483	if (!scm->pid)
460	return;	484	return;
461		485
462	pidfd = pidfd_prepare(scm->pid, 0, &pidfd_file);	486	pidfd = pidfd_prepare(scm->pid, PIDFD_STALE, &pidfd_file);
463		487
464	if (put_cmsg(msg, SOL_SOCKET, SCM_PIDFD, sizeof(int), &pidfd)) {	488	if (put_cmsg(msg, SOL_SOCKET, SCM_PIDFD, sizeof(int), &pidfd)) {
465	if (pidfd_file) {	489	if (pidfd_file) {

net/unix/af_unix.c+48−30

 		return;
 	}
-	if (sk->sk_peer_pid)
-		pidfs_put_pid(sk->sk_peer_pid);
 	if (u->addr)
 		unix_release_addr(u->addr);
 	swap(peercred->peer_pid, pid);
 	swap(peercred->peer_cred, cred);
-	pidfs_put_pid(pid);
 	put_pid(pid);
 	put_cred(cred);
 }
 	spin_lock(&sk->sk_peer_lock);
 	sk->sk_peer_pid = get_pid(peersk->sk_peer_pid);
-	pidfs_get_pid(sk->sk_peer_pid);
 	sk->sk_peer_cred = get_cred(peersk->sk_peer_cred);
 	spin_unlock(&sk->sk_peer_lock);
 }
 	struct scm_cookie scm;
 	memset(&scm, 0, sizeof(scm));
-	scm.pid  = UNIXCB(skb).pid;
+	scm.pid = UNIXCB(skb).pid;
 	if (UNIXCB(skb).fp)
 		unix_detach_fds(&scm, skb);
 	return err;
 }
-/*
+static void unix_skb_to_scm(struct sk_buff *skb, struct scm_cookie *scm)
+{
+	scm_set_cred(scm, UNIXCB(skb).pid, UNIXCB(skb).uid, UNIXCB(skb).gid);
+	unix_set_secdata(scm, skb);
+}
+/**
+ * unix_maybe_add_creds() - Adds current task uid/gid and struct pid to skb if needed.
+ * @skb: skb to attach creds to.
+ * @sk: Sender sock.
+ * @other: Receiver sock.
+ *
  * Some apps rely on write() giving SCM_CREDENTIALS
  * We include credentials if source or destination socket
  * asserted SOCK_PASSCRED.
+ *
+ * Context: May sleep.
+ * Return: On success zero, on error a negative error code is returned.
  */
-static void unix_maybe_add_creds(struct sk_buff *skb, const struct sock *sk,
+static int unix_maybe_add_creds(struct sk_buff *skb, const struct sock *sk,
-				 const struct sock *other)
+				const struct sock *other)
 {
 	if (UNIXCB(skb).pid)
-		return;
+		return 0;
 	if (unix_may_passcred(sk) || unix_may_passcred(other) ||
 	    !other->sk_socket) {
-		UNIXCB(skb).pid = get_pid(task_tgid(current));
+		struct pid *pid;
+		int err;
+		pid = task_tgid(current);
+		err = pidfs_register_pid(pid);
+		if (unlikely(err))
+			return err;
+		UNIXCB(skb).pid = get_pid(pid);
 		current_uid_gid(&UNIXCB(skb).uid, &UNIXCB(skb).gid);
 	}
+	return 0;
 }
 static bool unix_skb_scm_eq(struct sk_buff *skb,
 		goto out_sock_put;
 	}
+	err = unix_maybe_add_creds(skb, sk, other);
+	if (err)
+		goto out_sock_put;
 restart:
 	sk_locked = 0;
 	unix_state_lock(other);
 	if (sock_flag(other, SOCK_RCVTSTAMP))
 		__net_timestamp(skb);
-	unix_maybe_add_creds(skb, sk, other);
 	scm_stat_add(other, skb);
 	skb_queue_tail(&other->sk_receive_queue, skb);
 	unix_state_unlock(other);
 	if (err < 0)
 		goto out;
+	err = unix_maybe_add_creds(skb, sk, other);
+	if (err)
+		goto out;
 	skb_put(skb, 1);
 	err = skb_copy_datagram_from_iter(skb, 0, &msg->msg_iter, 1);
 		goto out_unlock;
 	}
-	unix_maybe_add_creds(skb, sk, other);
 	scm_stat_add(other, skb);
 	spin_lock(&other->sk_receive_queue.lock);
 		fds_sent = true;
+		err = unix_maybe_add_creds(skb, sk, other);
+		if (err)
+			goto out_free;
 		if (unlikely(msg->msg_flags & MSG_SPLICE_PAGES)) {
 			skb->ip_summed = CHECKSUM_UNNECESSARY;
 			err = skb_splice_from_iter(skb, &msg->msg_iter, size,
 			goto out_free;
 		}
-		unix_maybe_add_creds(skb, sk, other);
 		scm_stat_add(other, skb);
 		skb_queue_tail(&other->sk_receive_queue, skb);
 		unix_state_unlock(other);
 	memset(&scm, 0, sizeof(scm));
-	scm_set_cred(&scm, UNIXCB(skb).pid, UNIXCB(skb).uid, UNIXCB(skb).gid);
+	unix_skb_to_scm(skb, &scm);
-	unix_set_secdata(&scm, skb);
 	if (!(flags & MSG_PEEK)) {
 		if (UNIXCB(skb).fp)
 				break;
 		} else if (unix_may_passcred(sk)) {
 			/* Copy credentials */
-			scm_set_cred(&scm, UNIXCB(skb).pid, UNIXCB(skb).uid, UNIXCB(skb).gid);
+			unix_skb_to_scm(skb, &scm);
-			unix_set_secdata(&scm, skb);
 			check_creds = true;
 		}
 static int unix_open_file(struct sock *sk)
 {
-	struct path path;
 	struct file *f;
 	int fd;
 	if (!smp_load_acquire(&unix_sk(sk)->addr))
 		return -ENOENT;
-	path = unix_sk(sk)->path;
+	if (!unix_sk(sk)->path.dentry)
-	if (!path.dentry)
 		return -ENOENT;
-	path_get(&path);
 	fd = get_unused_fd_flags(O_CLOEXEC);
 	if (fd < 0)
-		goto out;
+		return fd;
-	f = dentry_open(&path, O_PATH, current_cred());
+	f = dentry_open(&unix_sk(sk)->path, O_PATH, current_cred());
 	if (IS_ERR(f)) {
 		put_unused_fd(fd);
-		fd = PTR_ERR(f);
+		return PTR_ERR(f);
-		goto out;
 	}
 	fd_install(fd, f);
-out:
-	path_put(&path);
 	return fd;
 }

tools/testing/selftests/net/af_unix/scm_pidfd.c+173−44

@@ -15,6 +15,7 @@ · 14 unmodified lines
15	#include <sys/types.h>	15	#include <sys/types.h>
16	#include <sys/wait.h>	16	#include <sys/wait.h>
17		17
		18	#include "../../pidfd/pidfd.h"
18	#include "../../kselftest_harness.h"	19	#include "../../kselftest_harness.h"
19		20
20	#define clean_errno() (errno == 0 ? "None" : strerror(errno))	21	#define clean_errno() (errno == 0 ? "None" : strerror(errno))
@@ -26,6 +27,8 @@ · 5 unmodified lines
26	#define SCM_PIDFD 0x04	27	#define SCM_PIDFD 0x04
27	#endif	28	#endif
28		29
		30	#define CHILD_EXIT_CODE_OK 123
		31
29	static void child_die()	32	static void child_die()
30	{	33	{
31	exit(1);	34	exit(1);
@@ -126,16 +129,65 @@ out: · 94 unmodified lines
126	return result;	129	return result;
127	}	130	}
128		131
		132	struct cmsg_data {
		133	struct ucred *ucred;
		134	int *pidfd;
		135	};
		136
		137	static int parse_cmsg(struct msghdr msg, struct cmsg_data res)
		138	{
		139	struct cmsghdr *cmsg;
		140	int data = 0;
		141
		142	if (msg->msg_flags & (MSG_TRUNC \| MSG_CTRUNC)) {
		143	log_err("recvmsg: truncated");
		144	return 1;
		145	}
		146
		147	for (cmsg = CMSG_FIRSTHDR(msg); cmsg != NULL;
		148	cmsg = CMSG_NXTHDR(msg, cmsg)) {
		149	if (cmsg->cmsg_level == SOL_SOCKET &&
		150	cmsg->cmsg_type == SCM_PIDFD) {
		151	if (cmsg->cmsg_len < sizeof(*res->pidfd)) {
		152	log_err("CMSG parse: SCM_PIDFD wrong len");
		153	return 1;
		154	}
		155
		156	res->pidfd = (void *)CMSG_DATA(cmsg);
		157	}
		158
		159	if (cmsg->cmsg_level == SOL_SOCKET &&
		160	cmsg->cmsg_type == SCM_CREDENTIALS) {
		161	if (cmsg->cmsg_len < sizeof(*res->ucred)) {
		162	log_err("CMSG parse: SCM_CREDENTIALS wrong len");
		163	return 1;
		164	}
		165
		166	res->ucred = (void *)CMSG_DATA(cmsg);
		167	}
		168	}
		169
		170	if (!res->pidfd) {
		171	log_err("CMSG parse: SCM_PIDFD not found");
		172	return 1;
		173	}
		174
		175	if (!res->ucred) {
		176	log_err("CMSG parse: SCM_CREDENTIALS not found");
		177	return 1;
		178	}
		179
		180	return 0;
		181	}
		182
129	static int cmsg_check(int fd)	183	static int cmsg_check(int fd)
130	{	184	{
131	struct msghdr msg = { 0 };	185	struct msghdr msg = { 0 };
132	struct cmsghdr *cmsg;	186	struct cmsg_data res;
133	struct iovec iov;	187	struct iovec iov;
134	struct ucred *ucred = NULL;
135	int data = 0;	188	int data = 0;
136	char control[CMSG_SPACE(sizeof(struct ucred)) +	189	char control[CMSG_SPACE(sizeof(struct ucred)) +
137	CMSG_SPACE(sizeof(int))] = { 0 };	190	CMSG_SPACE(sizeof(int))] = { 0 };
138	int *pidfd = NULL;
139	pid_t parent_pid;	191	pid_t parent_pid;
140	int err;	192	int err;
141		193
@@ -158,53 +210,99 @@ static int cmsg_check(int fd) · 16 unmodified lines
158	return 1;	210	return 1;
159	}	211	}
160		212
161	for (cmsg = CMSG_FIRSTHDR(&msg); cmsg != NULL;	213	/* send(pfd, "x", sizeof(char), 0) */
162	cmsg = CMSG_NXTHDR(&msg, cmsg)) {	214	if (data != 'x') {
163	if (cmsg->cmsg_level == SOL_SOCKET &&	215	log_err("recvmsg: data corruption");
164	cmsg->cmsg_type == SCM_PIDFD) {	216	return 1;
165	if (cmsg->cmsg_len < sizeof(*pidfd)) {	217	}
166	log_err("CMSG parse: SCM_PIDFD wrong len");
167	return 1;
168	}
169		218
170	pidfd = (void *)CMSG_DATA(cmsg);	219	if (parse_cmsg(&msg, &res)) {
171	}	220	log_err("CMSG parse: parse_cmsg() failed");
		221	return 1;
		222	}
172		223
173	if (cmsg->cmsg_level == SOL_SOCKET &&	224	/* pidfd from SCM_PIDFD should point to the parent process PID */
174	cmsg->cmsg_type == SCM_CREDENTIALS) {	225	parent_pid =
175	if (cmsg->cmsg_len < sizeof(*ucred)) {	226	get_pid_from_fdinfo_file(*res.pidfd, "Pid:", sizeof("Pid:") - 1);
176	log_err("CMSG parse: SCM_CREDENTIALS wrong len");	227	if (parent_pid != getppid()) {
177	return 1;	228	log_err("wrong SCM_PIDFD %d != %d", parent_pid, getppid());
178	}	229	close(*res.pidfd);
		230	return 1;
		231	}
179		232
180	ucred = (void *)CMSG_DATA(cmsg);	233	close(*res.pidfd);
181	}	234	return 0;
		235	}
		236
		237	static int cmsg_check_dead(int fd, int expected_pid)
		238	{
		239	int err;
		240	struct msghdr msg = { 0 };
		241	struct cmsg_data res;
		242	struct iovec iov;
		243	int data = 0;
		244	char control[CMSG_SPACE(sizeof(struct ucred)) +
		245	CMSG_SPACE(sizeof(int))] = { 0 };
		246	pid_t client_pid;
		247	struct pidfd_info info = {
		248	.mask = PIDFD_INFO_EXIT,
		249	};
		250
		251	iov.iov_base = &data;
		252	iov.iov_len = sizeof(data);
		253
		254	msg.msg_iov = &iov;
		255	msg.msg_iovlen = 1;
		256	msg.msg_control = control;
		257	msg.msg_controllen = sizeof(control);
		258
		259	err = recvmsg(fd, &msg, 0);
		260	if (err < 0) {
		261	log_err("recvmsg");
		262	return 1;
182	}	263	}
183		264
184	/* send(pfd, "x", sizeof(char), 0) */	265	if (msg.msg_flags & (MSG_TRUNC \| MSG_CTRUNC)) {
185	if (data != 'x') {	266	log_err("recvmsg: truncated");
		267	return 1;
		268	}
		269
		270	/* send(cfd, "y", sizeof(char), 0) */
		271	if (data != 'y') {
186	log_err("recvmsg: data corruption");	272	log_err("recvmsg: data corruption");
187	return 1;	273	return 1;
188	}	274	}
189		275
190	if (!pidfd) {	276	if (parse_cmsg(&msg, &res)) {
191	log_err("CMSG parse: SCM_PIDFD not found");	277	log_err("CMSG parse: parse_cmsg() failed");
192	return 1;	278	return 1;
193	}	279	}
194		280
195	if (!ucred) {	281	/*
196	log_err("CMSG parse: SCM_CREDENTIALS not found");	282	* pidfd from SCM_PIDFD should point to the client_pid.
		283	* Let's read exit information and check if it's what
		284	* we expect to see.
		285	*/
		286	if (ioctl(*res.pidfd, PIDFD_GET_INFO, &info)) {
		287	log_err("%s: ioctl(PIDFD_GET_INFO) failed", __func__);
		288	close(*res.pidfd);
197	return 1;	289	return 1;
198	}	290	}
199		291
200	/* pidfd from SCM_PIDFD should point to the parent process PID */	292	if (!(info.mask & PIDFD_INFO_EXIT)) {
201	parent_pid =	293	log_err("%s: No exit information from ioctl(PIDFD_GET_INFO)", __func__);
202	get_pid_from_fdinfo_file(*pidfd, "Pid:", sizeof("Pid:") - 1);	294	close(*res.pidfd);
203	if (parent_pid != getppid()) {
204	log_err("wrong SCM_PIDFD %d != %d", parent_pid, getppid());
205	return 1;	295	return 1;
206	}	296	}
207		297
		298	err = WIFEXITED(info.exit_code) ? WEXITSTATUS(info.exit_code) : 1;
		299	if (err != CHILD_EXIT_CODE_OK) {
		300	log_err("%s: wrong exit_code %d != %d", __func__, err, CHILD_EXIT_CODE_OK);
		301	close(*res.pidfd);
		302	return 1;
		303	}
		304
		305	close(*res.pidfd);
208	return 0;	306	return 0;
209	}	307	}
210		308
@@ -291,6 +389,24 @@ static void fill_sockaddr(struct sock_addr *addr, bool abstract) · 80 unmodified lines
291	memcpy(sun_path_buf, addr->sock_name, strlen(addr->sock_name));	389	memcpy(sun_path_buf, addr->sock_name, strlen(addr->sock_name));
292	}	390	}
293		391
		392	static int sk_enable_cred_pass(int sk)
		393	{
		394	int on = 0;
		395
		396	on = 1;
		397	if (setsockopt(sk, SOL_SOCKET, SO_PASSCRED, &on, sizeof(on))) {
		398	log_err("Failed to set SO_PASSCRED");
		399	return 1;
		400	}
		401
		402	if (setsockopt(sk, SOL_SOCKET, SO_PASSPIDFD, &on, sizeof(on))) {
		403	log_err("Failed to set SO_PASSPIDFD");
		404	return 1;
		405	}
		406
		407	return 0;
		408	}
		409
294	static void client(FIXTURE_DATA(scm_pidfd) *self,	410	static void client(FIXTURE_DATA(scm_pidfd) *self,
295	const FIXTURE_VARIANT(scm_pidfd) *variant)	411	const FIXTURE_VARIANT(scm_pidfd) *variant)
296	{	412	{
@@ -299,7 +415,6 @@ static void client(FIXTURE_DATA(scm_pidfd) *self, · 2 unmodified lines
299	struct ucred peer_cred;	415	struct ucred peer_cred;
300	int peer_pidfd;	416	int peer_pidfd;
301	pid_t peer_pid;	417	pid_t peer_pid;
302	int on = 0;
303		418
304	cfd = socket(AF_UNIX, variant->type, 0);	419	cfd = socket(AF_UNIX, variant->type, 0);
305	if (cfd < 0) {	420	if (cfd < 0) {
@@ -322,14 +437,8 @@ static void client(FIXTURE_DATA(scm_pidfd) *self, · 16 unmodified lines
322	child_die();	437	child_die();
323	}	438	}
324		439
325	on = 1;	440	if (sk_enable_cred_pass(cfd)) {
326	if (setsockopt(cfd, SOL_SOCKET, SO_PASSCRED, &on, sizeof(on))) {	441	log_err("sk_enable_cred_pass() failed");
327	log_err("Failed to set SO_PASSCRED");
328	child_die();
329	}
330
331	if (setsockopt(cfd, SOL_SOCKET, SO_PASSPIDFD, &on, sizeof(on))) {
332	log_err("Failed to set SO_PASSPIDFD");
333	child_die();	442	child_die();
334	}	443	}
335		444
@@ -340,6 +449,12 @@ static void client(FIXTURE_DATA(scm_pidfd) *self, · 4 unmodified lines
340	child_die();	449	child_die();
341	}	450	}
342		451
		452	/* send something to the parent so it can receive SCM_PIDFD too and validate it */
		453	if (send(cfd, "y", sizeof(char), 0) == -1) {
		454	log_err("Failed to send(cfd, \"y\", sizeof(char), 0)");
		455	child_die();
		456	}
		457
343	/* skip further for SOCK_DGRAM as it's not applicable */	458	/* skip further for SOCK_DGRAM as it's not applicable */
344	if (variant->type == SOCK_DGRAM)	459	if (variant->type == SOCK_DGRAM)
345	return;	460	return;
@@ -398,7 +513,13 @@ TEST_F(scm_pidfd, test) · 52 unmodified lines
398	close(self->server);	513	close(self->server);
399	close(self->startup_pipe[0]);	514	close(self->startup_pipe[0]);
400	client(self, variant);	515	client(self, variant);
401	exit(0);	516
		517	/*
		518	* It's a bit unusual, but in case of success we return non-zero
		519	* exit code (CHILD_EXIT_CODE_OK) and then we expect to read it
		520	* from ioctl(PIDFD_GET_INFO) in cmsg_check_dead().
		521	*/
		522	exit(CHILD_EXIT_CODE_OK);
402	}	523	}
403	close(self->startup_pipe[1]);	524	close(self->startup_pipe[1]);
404		525
@@ -421,9 +542,17 @@ TEST_F(scm_pidfd, test) · 16 unmodified lines
421	ASSERT_NE(-1, err);	542	ASSERT_NE(-1, err);
422	}	543	}
423		544
424	close(pfd);
425	waitpid(self->client_pid, &child_status, 0);	545	waitpid(self->client_pid, &child_status, 0);
426	ASSERT_EQ(0, WIFEXITED(child_status) ? WEXITSTATUS(child_status) : 1);	546	/* see comment before exit(CHILD_EXIT_CODE_OK) */
		547	ASSERT_EQ(CHILD_EXIT_CODE_OK, WIFEXITED(child_status) ? WEXITSTATUS(child_status) : 1);
		548
		549	err = sk_enable_cred_pass(pfd);
		550	ASSERT_EQ(0, err);
		551
		552	err = cmsg_check_dead(pfd, self->client_pid);
		553	ASSERT_EQ(0, err);
		554
		555	close(pfd);
427	}	556	}
428		557
429	TEST_HARNESS_MAIN	558	TEST_HARNESS_MAIN

tools/testing/selftests/pidfd/.gitignore+2−0

@@ -10,3 +10,5 @@ pidfd_file_handle_test · 9 unmodified lines
10	pidfd_bind_mount	10	pidfd_bind_mount
11	pidfd_info_test	11	pidfd_info_test
12	pidfd_exec_helper	12	pidfd_exec_helper
		13	pidfd_xattr_test
		14	pidfd_setattr_test

tools/testing/selftests/pidfd/Makefile+3−2

@@ -1,9 +1,10 @@
1	# SPDX-License-Identifier: GPL-2.0-only	1	# SPDX-License-Identifier: GPL-2.0-only
2	CFLAGS += -g $(KHDR_INCLUDES) -pthread -Wall	2	CFLAGS += -g $(KHDR_INCLUDES) $(TOOLS_INCLUDES) -pthread -Wall
3		3
4	TEST_GEN_PROGS := pidfd_test pidfd_fdinfo_test pidfd_open_test \	4	TEST_GEN_PROGS := pidfd_test pidfd_fdinfo_test pidfd_open_test \
5	pidfd_poll_test pidfd_wait pidfd_getfd_test pidfd_setns_test \	5	pidfd_poll_test pidfd_wait pidfd_getfd_test pidfd_setns_test \
6	pidfd_file_handle_test pidfd_bind_mount pidfd_info_test	6	pidfd_file_handle_test pidfd_bind_mount pidfd_info_test \
		7	pidfd_xattr_test pidfd_setattr_test
7		8
8	TEST_GEN_PROGS_EXTENDED := pidfd_exec_helper	9	TEST_GEN_PROGS_EXTENDED := pidfd_exec_helper
9		10

tools/testing/selftests/pidfd/pidfd.h+5−1

 #include "../kselftest.h"
 #include "../clone3/clone3_selftests.h"
+#ifndef FD_PIDFS_ROOT
+#define FD_PIDFS_ROOT -10002
+#endif
 #ifndef P_PIDFD
 #define P_PIDFD 3
 #endif
 #endif
 #ifndef PIDFD_SELF_THREAD_GROUP
-#define PIDFD_SELF_THREAD_GROUP		-20000 /* Current thread group leader. */
+#define PIDFD_SELF_THREAD_GROUP		-10001 /* Current thread group leader. */
 #endif
 #ifndef PIDFD_SELF

tools/testing/selftests/pidfd/pidfd_file_handle_test.c+60−0

@@ -500,4 +500,64 @@ TEST_F(file_handle, valid_name_to_handle_at_flags) · 499 unmodified lines
500	ASSERT_EQ(close(pidfd), 0);	500	ASSERT_EQ(close(pidfd), 0);
501	}	501	}
502		502
		503	/*
		504	* That we decode a file handle without having to pass a pidfd.
		505	*/
		506	TEST_F(file_handle, decode_purely_based_on_file_handle)
		507	{
		508	int mnt_id;
		509	struct file_handle *fh;
		510	int pidfd = -EBADF;
		511	struct stat st1, st2;
		512
		513	fh = malloc(sizeof(struct file_handle) + MAX_HANDLE_SZ);
		514	ASSERT_NE(fh, NULL);
		515	memset(fh, 0, sizeof(struct file_handle) + MAX_HANDLE_SZ);
		516	fh->handle_bytes = MAX_HANDLE_SZ;
		517
		518	ASSERT_EQ(name_to_handle_at(self->child_pidfd1, "", fh, &mnt_id, AT_EMPTY_PATH), 0);
		519
		520	ASSERT_EQ(fstat(self->child_pidfd1, &st1), 0);
		521
		522	pidfd = open_by_handle_at(FD_PIDFS_ROOT, fh, 0);
		523	ASSERT_GE(pidfd, 0);
		524
		525	ASSERT_EQ(fstat(pidfd, &st2), 0);
		526	ASSERT_TRUE(st1.st_dev == st2.st_dev && st1.st_ino == st2.st_ino);
		527
		528	ASSERT_EQ(close(pidfd), 0);
		529
		530	pidfd = open_by_handle_at(FD_PIDFS_ROOT, fh, O_CLOEXEC);
		531	ASSERT_GE(pidfd, 0);
		532
		533	ASSERT_EQ(fstat(pidfd, &st2), 0);
		534	ASSERT_TRUE(st1.st_dev == st2.st_dev && st1.st_ino == st2.st_ino);
		535
		536	ASSERT_EQ(close(pidfd), 0);
		537
		538	pidfd = open_by_handle_at(FD_PIDFS_ROOT, fh, O_NONBLOCK);
		539	ASSERT_GE(pidfd, 0);
		540
		541	ASSERT_EQ(fstat(pidfd, &st2), 0);
		542	ASSERT_TRUE(st1.st_dev == st2.st_dev && st1.st_ino == st2.st_ino);
		543
		544	ASSERT_EQ(close(pidfd), 0);
		545
		546	pidfd = open_by_handle_at(self->pidfd, fh, 0);
		547	ASSERT_GE(pidfd, 0);
		548
		549	ASSERT_EQ(fstat(pidfd, &st2), 0);
		550	ASSERT_TRUE(st1.st_dev == st2.st_dev && st1.st_ino == st2.st_ino);
		551
		552	ASSERT_EQ(close(pidfd), 0);
		553
		554	pidfd = open_by_handle_at(-EBADF, fh, 0);
		555	ASSERT_LT(pidfd, 0);
		556
		557	pidfd = open_by_handle_at(AT_FDCWD, fh, 0);
		558	ASSERT_LT(pidfd, 0);
		559
		560	free(fh);
		561	}
		562
503	TEST_HARNESS_MAIN	563	TEST_HARNESS_MAIN

tools/testing/selftests/pidfd/pidfd_setattr_test.c+69−0

@@ -0,0 +1,69 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2
	3	#define _GNU_SOURCE
	4	#include <errno.h>
	5	#include <fcntl.h>
	6	#include <limits.h>
	7	#include <linux/types.h>
	8	#include <poll.h>
	9	#include <pthread.h>
	10	#include <sched.h>
	11	#include <signal.h>
	12	#include <stdio.h>
	13	#include <stdlib.h>
	14	#include <string.h>
	15	#include <syscall.h>
	16	#include <sys/prctl.h>
	17	#include <sys/wait.h>
	18	#include <unistd.h>
	19	#include <sys/socket.h>
	20	#include <linux/kcmp.h>
	21	#include <sys/stat.h>
	22	#include <sys/xattr.h>
	23
	24	#include "pidfd.h"
	25	#include "../kselftest_harness.h"
	26
	27	FIXTURE(pidfs_setattr)
	28	{
	29	pid_t child_pid;
	30	int child_pidfd;
	31	};
	32
	33	FIXTURE_SETUP(pidfs_setattr)
	34	{
	35	self->child_pid = create_child(&self->child_pidfd, CLONE_NEWUSER \| CLONE_NEWPID);
	36	EXPECT_GE(self->child_pid, 0);
	37
	38	if (self->child_pid == 0)
	39	_exit(EXIT_SUCCESS);
	40	}
	41
	42	FIXTURE_TEARDOWN(pidfs_setattr)
	43	{
	44	sys_waitid(P_PID, self->child_pid, NULL, WEXITED);
	45	EXPECT_EQ(close(self->child_pidfd), 0);
	46	}
	47
	48	TEST_F(pidfs_setattr, no_chown)
	49	{
	50	ASSERT_LT(fchown(self->child_pidfd, 1234, 5678), 0);
	51	ASSERT_EQ(errno, EOPNOTSUPP);
	52	}
	53
	54	TEST_F(pidfs_setattr, no_chmod)
	55	{
	56	ASSERT_LT(fchmod(self->child_pidfd, 0777), 0);
	57	ASSERT_EQ(errno, EOPNOTSUPP);
	58	}
	59
	60	TEST_F(pidfs_setattr, no_exec)
	61	{
	62	char *const argv[] = { NULL };
	63	char *const envp[] = { NULL };
	64
	65	ASSERT_LT(execveat(self->child_pidfd, "", argv, envp, AT_EMPTY_PATH), 0);
	66	ASSERT_EQ(errno, EACCES);
	67	}
	68
	69	TEST_HARNESS_MAIN

tools/testing/selftests/pidfd/pidfd_xattr_test.c+132−0

@@ -0,0 +1,132 @@
	1	// SPDX-License-Identifier: GPL-2.0
	2
	3	#define _GNU_SOURCE
	4	#include <errno.h>
	5	#include <fcntl.h>
	6	#include <limits.h>
	7	#include <linux/types.h>
	8	#include <poll.h>
	9	#include <pthread.h>
	10	#include <sched.h>
	11	#include <signal.h>
	12	#include <stdio.h>
	13	#include <stdlib.h>
	14	#include <string.h>
	15	#include <syscall.h>
	16	#include <sys/prctl.h>
	17	#include <sys/wait.h>
	18	#include <unistd.h>
	19	#include <sys/socket.h>
	20	#include <linux/kcmp.h>
	21	#include <sys/stat.h>
	22	#include <sys/xattr.h>
	23
	24	#include "pidfd.h"
	25	#include "../kselftest_harness.h"
	26
	27	FIXTURE(pidfs_xattr)
	28	{
	29	pid_t child_pid;
	30	int child_pidfd;
	31	};
	32
	33	FIXTURE_SETUP(pidfs_xattr)
	34	{
	35	self->child_pid = create_child(&self->child_pidfd, CLONE_NEWUSER \| CLONE_NEWPID);
	36	EXPECT_GE(self->child_pid, 0);
	37
	38	if (self->child_pid == 0)
	39	_exit(EXIT_SUCCESS);
	40	}
	41
	42	FIXTURE_TEARDOWN(pidfs_xattr)
	43	{
	44	sys_waitid(P_PID, self->child_pid, NULL, WEXITED);
	45	}
	46
	47	TEST_F(pidfs_xattr, set_get_list_xattr_multiple)
	48	{
	49	int ret, i;
	50	char xattr_name[32];
	51	char xattr_value[32];
	52	char buf[32];
	53	const int num_xattrs = 10;
	54	char list[PATH_MAX] = {};
	55
	56	for (i = 0; i < num_xattrs; i++) {
	57	snprintf(xattr_name, sizeof(xattr_name), "trusted.testattr%d", i);
	58	snprintf(xattr_value, sizeof(xattr_value), "testvalue%d", i);
	59	ret = fsetxattr(self->child_pidfd, xattr_name, xattr_value, strlen(xattr_value), 0);
	60	ASSERT_EQ(ret, 0);
	61	}
	62
	63	for (i = 0; i < num_xattrs; i++) {
	64	snprintf(xattr_name, sizeof(xattr_name), "trusted.testattr%d", i);
	65	snprintf(xattr_value, sizeof(xattr_value), "testvalue%d", i);
	66	memset(buf, 0, sizeof(buf));
	67	ret = fgetxattr(self->child_pidfd, xattr_name, buf, sizeof(buf));
	68	ASSERT_EQ(ret, strlen(xattr_value));
	69	ASSERT_EQ(strcmp(buf, xattr_value), 0);
	70	}
	71
	72	ret = flistxattr(self->child_pidfd, list, sizeof(list));
	73	ASSERT_GT(ret, 0);
	74	for (i = 0; i < num_xattrs; i++) {
	75	snprintf(xattr_name, sizeof(xattr_name), "trusted.testattr%d", i);
	76	bool found = false;
	77	for (char *it = list; it < list + ret; it += strlen(it) + 1) {
	78	if (strcmp(it, xattr_name))
	79	continue;
	80	found = true;
	81	break;
	82	}
	83	ASSERT_TRUE(found);
	84	}
	85
	86	for (i = 0; i < num_xattrs; i++) {
	87	snprintf(xattr_name, sizeof(xattr_name), "trusted.testattr%d", i);
	88	ret = fremovexattr(self->child_pidfd, xattr_name);
	89	ASSERT_EQ(ret, 0);
	90
	91	ret = fgetxattr(self->child_pidfd, xattr_name, buf, sizeof(buf));
	92	ASSERT_EQ(ret, -1);
	93	ASSERT_EQ(errno, ENODATA);
	94	}
	95	}
	96
	97	TEST_F(pidfs_xattr, set_get_list_xattr_persistent)
	98	{
	99	int ret;
	100	char buf[32];
	101	char list[PATH_MAX] = {};
	102
	103	ret = fsetxattr(self->child_pidfd, "trusted.persistent", "persistent value", strlen("persistent value"), 0);
	104	ASSERT_EQ(ret, 0);
	105
	106	memset(buf, 0, sizeof(buf));
	107	ret = fgetxattr(self->child_pidfd, "trusted.persistent", buf, sizeof(buf));
	108	ASSERT_EQ(ret, strlen("persistent value"));
	109	ASSERT_EQ(strcmp(buf, "persistent value"), 0);
	110
	111	ret = flistxattr(self->child_pidfd, list, sizeof(list));
	112	ASSERT_GT(ret, 0);
	113	ASSERT_EQ(strcmp(list, "trusted.persistent"), 0)
	114
	115	ASSERT_EQ(close(self->child_pidfd), 0);
	116	self->child_pidfd = -EBADF;
	117	sleep(2);
	118
	119	self->child_pidfd = sys_pidfd_open(self->child_pid, 0);
	120	ASSERT_GE(self->child_pidfd, 0);
	121
	122	memset(buf, 0, sizeof(buf));
	123	ret = fgetxattr(self->child_pidfd, "trusted.persistent", buf, sizeof(buf));
	124	ASSERT_EQ(ret, strlen("persistent value"));
	125	ASSERT_EQ(strcmp(buf, "persistent value"), 0);
	126
	127	ret = flistxattr(self->child_pidfd, list, sizeof(list));
	128	ASSERT_GT(ret, 0);
	129	ASSERT_EQ(strcmp(list, "trusted.persistent"), 0);
	130	}
	131
	132	TEST_HARNESS_MAIN