os: add (*Process).WithHandle

[kolyshkin]

Proposal

Add a new WithHandle method for os.Process, which, if a handle is available, calls a specified callback function with the process handle as an argument. On Linux, handle is a pidfd (file descriptor referring to the process). On Windows, handle is a handle to the process.

// WithHandle calls a supplied function f with a valid process handle as an argument.
// The handle is guaranteed to refer to process p until f returns, even if p terminates.
// If no process handle is available, it returns ErrNoHandle.
//
// Currently, process handles are only available on Linux 5.4 and later
// (where it is known as pidfd) and Windows.
func (p *Process) WithHandle(f func(handle uintptr)) error

var ErrNoHandle = errors.New("process handle unavailable")

Context

This is a continuation of #62654, in particular its item 6, described in there as:

6. (Optional) Add (*Process).Handle() uintptr method to return process handle on Windows and pidfd on Linux. This might be useful for low-level operations that require handle/pidfd (e.g. pidfd_getfd on Linux), or to ensure that pidfd (rather than pid) is being used for kill/wait.

Also, a similar thing was proposed earlier here by @prattmic:

A new os.Process.Fd() could return the pid FD for additional direct use.

Use cases

1. Check if pidfd is being used on Linux.

Since Go 1.23, pidfd is used for os.Process-related operations instead of pid, if supported by the Linux kernel. This includes os.StartProcess and os.FindProcess (they obtain a pidfd), as well as (*Process).Wait, (*Process).Signal, and (*Process).Kill (they use pidfd). The main benefit of pidfd in the use cases above is a guarantee we're referring to the same process (i.e. there's no pid reuse issue).

However, since this is done in a fully transparent way, there is no way for a user to know if pidfd is being used or not. Some programs implement some protection against pid reuse (for example, runc and cri-o obtain and check process start time from /proc/<pid>/stat). They can benefit from being able to know if Go is using pidfd internally.

Another example is containerd which relies on Go 1.23 using pidfd internally, but since there's no way to check they had to recreate all the functionality checking for pidfd support here (which is still not 100% correct since the checks are slightly different from those in Go's checkPidfd, and Go checks may change over time ). Cc @fuweid.

With the proposed interface, a user can easily check if pidfd is being used:

p, err := os.FindProcess()
pidfdUsed := false
p.WithHandle(func(_ uintptr) {
       pidfdUsed = true
})

2. Use the existing pidfd directly.

Aside from use cases already covered by existing os.Process methods, pidfd can also be used to:

• obtain a duplicate of a file descriptor of another process (pidfd_getfd(2));
• select/poll/epoll on a pidfd to know when a process is terminated;
• move a process/thread into one or more of the same namespaces as the process referred to by the file descriptor (setns(2)).

Other use cases may emerge in the future.

Currently, the only way to obtain a pidfd on Linux is to execute a new process (via os.StartProcess or os/exec) with process' Attr.SysAttr.PidFD field set. This works if we're starting the process, but not in any other case (someone else starts a process for us, or it is already running).

Questions / discussion points

1. What are (could be) the additional direct use cases of Windows process handle?

A few are listed here. Apparently some git grep (GetPriorityClass, SetPriorityClass, AssignProcessToJobObject) are implemented in golang.org/x/sys/windows.

2. Should a duplicate of a handle be used, or the original handle?

Use the original one.

Arguments against duplicated handle:

• a duplicated pidfd makes the "check if pidfd is being used" use case above more complicated as the user will need to close the returned pidfd;
• a user can always use dupfd if/when needed;
• returned handle won't leak as it is still part of os.Process struct, which have a Release method and a proper finalizer;
• os.File.Fd returns the original underlying fd, pidfd is similar.

Arguments for duplicated handle:

• cleaner separation of responsibilities(?);
• a Windows process handle can be duplicated;

3. Should WithHandle provide *os.File rather than uintptr?

Raw handle makes more sense in this case, and *os.File won't work with Windows process handle as it is not a file descriptor.

4. Should this be Linux-specific?

Probably not. Since we have a boolean flag returned, we can implement it for all platforms and do nothing for those that do not support process handle (other than Windows and Linux).

[gabyhelp]

Related Issues

• os: make use of pidfd for linux #62654 (closed)
• os: racy use of pidfd #67641 (closed)

Related Code Changes

• os: overhaul handling of PID vs pidfd within Process
• os/exec: use pidfd for waiting and signaling of processes
• os: make use of pidfd on linux
• syscall: introduce SysProcAttr.ParentProcess on Windows
• os: terminate windows processes via handle directly
• os: make use of pidfd on linux

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[fuweid]

Thanks @kolyshkin for tagging me. This proposal looks good me from user perspective. I don't need to check pidfd in my side. :)

[adonovan]

It should probably return an error instead of a boolean since the operation is (I think?) a system call and could in principle fail for more reasons than ENOTSUP.

[kolyshkin]

It should probably return an error instead of a boolean since the operation is (I think?) a system call and could in principle fail for more reasons than ENOTSUP.

The pidfd is already used internally by os, and Handle does not do any syscalls, it merely returns an existing pidfd, if available. An error getting pidfd can happen way before the call to Handle.

Are you suggesting we need to save the error getting pidfd in struct Process, and have Handle return it?

I've added bool because 0 is a valid fd and the type is unsigned, so it's not trivial for the caller to distinguish between a valid pidfd and "pidfd is not available".

[adonovan]

The pidfd is already used internally by os, and Handle does not do any syscalls, it merely returns an existing pidfd, if available.

I see. Thanks for clarifying.

Are you suggesting we need to save the error getting pidfd in struct Process, and have Handle return it?

No, what you have seems fine.

I've added bool because 0 is a valid fd and the type is unsigned, so it's not trivial for the caller to distinguish between a valid pidfd and "pidfd is not available".

That seems appropriate.

[kolyshkin]

Not sure if there's anything I can do to move this forward. This seems to be the last step needed to fully support Linux pidfd in Go, and it also seems pretty trivial (except, unlike all the previous work, it amends the public API of os).

Any suggestions to move this forward @adonovan @prattmic @ianlancetaylor?

[adonovan]

I don't think there's anything lacking in this proposal; the only obstacle is that the committee is not keeping up with the volume of proposals. We'll be scheduling extra sessions in the run up to the go1.25 freeze, and longer term we're thinking about ways to improve our efficiency. Thanks for your patience.

[prattmic]

I think having an API for this makes sense.

That said, I am a bit concerned about returning the raw FD, similar to os.(*File).Fd.

One of the most common footguns users run into with finalizers (if I may be so bold, probably the most common) is that the fd returned from os.(*File).Fd does not keep the File alive. File has a finalizer that closes the FD, so if you don't keep the file alive (usually with defer f.Close() or explicit runtime.KeepAlive) then you can find the FD randomly closed out from under you.

Process also has a finalizer (well, cleanup as of 1.25) that closes the pidfd, so I believe this API will suffer from the exact same footgun. I'd like to avoid that footgun on a new API if we can.

Off the top of my head, a couple of ways to avoid this problem:

1. Handle dups the FD before returning it. You discuss this above but don't mention that it avoids the finalizer issue, which is a significant advantage.

2. Using some sort of callback API like func (*Process) WithHandle(f func(handle uintptr)). We have some existing use of this pattern with syscall.RawConn, which net uses to provide access to the underlying FD. e.g., net.(*IPConn).SyscallConn. This approach doesn't completely prevent misuse (you can keep a copy of the FD after the callback returns), but it is much safer by default.

[kolyshkin]

The callback API idea sounds really appealing to me! Perhaps I should rewrite this proposal to use it (or write a new one? or will you @prattmic write it?)