Multiple issues when installing on Ubuntu 24.04

[pgrenaud]

TL;DR

• Single apt package with multiple candidates prevent installation of all apt packages
• Python new Externally Managed Environments feature prevents installation of all tools that rely on pip
• Muted output during installation prevents troubleshooting

Description

I recently tried to install reconftw on a Ubuntu 24.04 and notice that something was wrong. The install.sh script would report that everything was installed while the reconftw.sh script would report a bunch of tools missing. Running install.sh again wouldn't fix the issue.

Re-enabling output during the install (vars DEBUG_STD and DEBUG_ERROR in reconftw.cfg) allowed me to notice a few problems with the install process.

The installing system packages step failed silently because the package libasound2 is a virtual package with two possible candidates (this is unique to Ubuntu 24.04). The resulting error blocks the installation of all required system packages.

Many python tools rely on PIP to install their requirements, but reconftw do it by running sudo pip3 install -r requirements.txt, which is now blocked by the Externally Managed Environments feature.

There is also a bunch more errors that are probably caused by the missing apt packages and/or the missing python dependencies.

Affected Systems

• Ubuntu 24.04 (Noble)
• Debian 12 (Bookworm)
• Kali Linux (exact versions unknown) (see Problem install on Kali raspberry pi 5 8gb #899)

Expected Results

Cloning the repo and running ./install.sh should install all the required dependencies and tools.

Actual Results

Cloning the repo and running ./install.sh is silently falling and not installing all the dependencies and tools. See screenshots above.

Steps to Reproduce

• Deploy a new Ubuntu 24.04 LTS machine
• Clone and install reconftw

  git clone https://github.com/six2dez/reconftw.git
  cd reconftw/
  ./install.sh

• Run ./reconftw.sh --help
• reconftw will report several tools missing

Discussion and Proposed Solutions

I've been using reconftw for the past year at work and it has been a great tool to have in my toolbox! I'm ready to put the time and effort to fix the issues highlighted and submit PRs for review. But since there is no contributing guide for this project, I just want to make sure we all agree on the work that need to be done before starting to work on fixes.

I'd like to propose the following changes:

1. Send all the output of the ./install.sh to a log file by default, like it is already the case with ./reconftw.sh
2. Fix the list of apt packages to install (not sure yet how to do it in a compatible manner with all the debian-based distributions, would need to look into it)
3. Install all python tools using pipx, or setup a venv for those who do not support it
4. Remove the use of sudo when it's not necessary (all tools would be installed for the local user only, like it is the case with the golang tools)
5. In the long term, validate the exit code of commands, especially for./install.sh, to make the scripts more resilient and detect abnormal states and errors

I can break down this issue in multiple issues if that would make it easier to keep track of the different issues and proposed changes.

See Also

• https://packaging.python.org/en/latest/specifications/externally-managed-environments/#guide-users-towards-virtual-environments
• https://peps.python.org/pep-0668/
• https://www.kali.org/blog/python-externally-managed/

[bileltechno]

Thanks for your detailed report!

Some tools installation is already added to the auto installer in the dev branch but not yet merged to the main branch.

adding EmailFinder to auto isntaller

add metaFinder to auto installer

add porch-pirate to auto-install

Add xnLinkFinder to auto installer

add waymore to auto-installer

The remaining issues will be analysed later .

[pgrenaud]

Some tools installation is already added to the auto installer in the dev branch but not yet merged to the main branch.

I don't think those commits address the problems highlighted in this issues. Unless I missed something, but when running on Ubuntu 22.04 (or Ubuntu 24.04 with some hotfixes to make the install.sh script work), everything is installed and the main reconftw.sh script works properly.

The remaining issues will be analysed later .

Ok, no problem! Please let me know what you think of my diagnostic and proposed solutions. I'm able to put effort into writing and sending PR to fix those issues.