Support "concurrency groups" for archiver processes

[mneudert]

The Matomo CLI archiver provides a simple concurrency control by using the --concurrent-archivers argument.

When used it will prevent the number of running archiving processes to exceed the set number by checking the server's process list for other running archivers. In multi-tenant setups the instance id (--matomo-domain) will be taken into account to group archiving processes together.

Archivers with different configurations

This works well until there is a need to add temporary archivers to work on a short-term backlog, for example after creating several new segments.

When trying to run different configurations for archivers on the same host, the current concurrency handling can also lead to unintended balancing.

For example, the following crontab:

15 * * * * console core:archive --concurrent-archivers=2 --skip-idsites=1
45 * * * * console core:archive --concurrent-archivers=2 --force-idsites=1

It could be created with the intent to have one archiver work on "site 1", and one archiver work on all other sites.

However, depending on the timing, it will eventually lead to "two archivers work on site 1, none on the rest", or vice versa. And once an archiver is started manually to work on a bigger invalidation backlog, it could prevent the crontab from being executed because concurrency slots are occupied by the manually started process.

Concurrency groups

One option to solve the mentioned problem would be to introduce and additional concurrency group flag:

15 * * * * console core:archive --concurrent-archivers=1 --skip-idsites=1 --concurrency-group=other-sites
45 * * * * console core:archive --concurrent-archivers=1 --force-idsites=1 --concurrenty-group=primary-site

The internal CronArchive::hasReachedMaxConcurrentArchivers can check if the current process was started for a given concurrency group and ignore processes in other groups, allowing a more fine-grained control over what the archiving work is spent on. And giving the option to raise the number of archivers on one server manually without changes to the existing (cron) setup.