Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(postinstall): ignore systemctl commands when systemd is not running #874

Merged
merged 2 commits into from
Mar 13, 2025

Conversation

bowling233
Copy link
Contributor

Based on #767. This avoids package manager like dpkg to fail in chrooted setups.

@bowling233 bowling233 requested a review from a team as a code owner March 12, 2025 13:42
@bowling233 bowling233 requested a review from mx-psi March 12, 2025 13:42
Copy link

linux-foundation-easycla bot commented Mar 12, 2025

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: bowling233 / name: Baolin Zhu (65d9431)
  • ✅ login: mx-psi / name: Pablo Baeyens (a53413b)

Copy link
Member

@mx-psi mx-psi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, can you add a changelog note? Also, could you expand on why this is needed in chrooted setups?

@bowling233
Copy link
Contributor Author

Hi, I've added a changelog.

When working in a chrooted environment, you're effectively isolating a portion of the filesystem and running commands within that confined space. However, systemd (the init system) is not running as PID 1 inside the chroot. This leads to issues when trying to use systemctl, as it relies on communicating with the systemd daemon, which is responsible for managing services and units.

$ systemctl daemon-reload
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down

Many systemd-related commands, such as systemctl daemon-reload or systemctl restart, are not required when setting up a chrooted system.

In our scenario, we use a chroot environment in our CI/CD pipeline to build system images and install otelcol. However, during installation, otelcol executes systemctl commands, which fail in the chrooted environment.

For package managers like dpkg, this leads to the following issues:

  • dpkg enters a broken state.
  • Subsequent package installations or updates fail because dpkg keeps retrying the failing post-installation script.

Therefore, this fix would be very beneficial for CI/CD pipelines.

@mx-psi
Copy link
Member

mx-psi commented Mar 12, 2025

Understood, thanks! I understand the problem but I am not sure this is the right solution for it: unconditionally passing regardless of any errors can hide meaningful issues for users in other environments where systemd would typically work. If there is a way to check if we are in chrooted environment where systemd does not work at all, I would prefer to add such a check instead of ignoring errors

@bowling233
Copy link
Contributor Author

As @netsandbox mentioned in #767, Systemd RPM macros append || : to all systemctl commands to prevent package scripts from failing. This suggests that RPM maintainers do not consider the result of systemctl commands critical to the package installation process.

On Debian, most packages use dh_installsystemd from deb-systemd-helper to manage systemd units. As described in the man page:

Thus, it is not necessary for the machine to be running systemd during package installation. Enabling happens universally to allow seamless switching between sysvinit and systemd.

Examining the source code confirms that Debian also ignores systemctl command results: autoscripts/postinst-systemd-restart · main · Debian / debhelper · GitLab

if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ] || [ "$1" = "abort-deconfigure" ] || [ "$1" = "abort-remove" ]; then
	if [ -d /run/systemd/system ]; then
		systemctl --system daemon-reload >/dev/null || true
		if [ -n "$2" ]; then
			_dh_action=#RESTART_ACTION#
		else
			_dh_action=start
		fi
		deb-systemd-invoke $_dh_action #UNITFILES# >/dev/null || true
	fi
fi

If the main concern is suppressing meaningful errors, we could add checks like [ -d /run/systemd/system ] or systemctl is-system-running --quiet to verify whether the systemd daemon is running before executing commands.

Would you prefer this approach, or should I make any modifications?

@mx-psi
Copy link
Member

mx-psi commented Mar 12, 2025

Thanks for being patient with me and providing all this context :)

If the main concern is suppressing meaningful errors, we could add checks like [ -d /run/systemd/system ] or systemctl is-system-running --quiet to verify whether the systemd daemon is running before executing commands.

I think this is a reasonable approach. I understand and sympathize with distro maintainers not wanting to troubleshoot this every time for every package, but given that, for our particular case, this seems to work fine for most users, I would rather take the gentler approach of checking here. We can always go more aggressive with || : later

@bowling233 bowling233 changed the title fix(postinstall): add error handling for systemctl commands fix(postinstall): ignore systemctl commands when systemd is not running Mar 12, 2025
@bowling233
Copy link
Contributor Author

The changes make it much clearer now—looks great. 🚀

Copy link
Member

@mx-psi mx-psi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks! Will make a small change on the release note

@mx-psi mx-psi enabled auto-merge March 13, 2025 08:34
@mx-psi mx-psi added this pull request to the merge queue Mar 13, 2025
Merged via the queue into open-telemetry:main with commit 0b45963 Mar 13, 2025
61 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants