24.8 leak scanning script

[strtgbb]

Changelog category (leave one):

• CI Fix or Improvement (changelog entry is not required)

Scan all files uploaded to S3 for [A-Z_]*(SECRET|PASSWORD)[A-Z_]*

[Enmk]

We also need to add re to match ROBOT_TOKEN from here: https://github.com/Altinity/ClickHouse/pull/693/files#diff-5399cabc63b54fc614af664aabf2e13ece95190dbc6b2442d1becc25d98c87faR14

[Enmk]

It checks for leaks after the leak happened, IMO it would be more productive to scan files before uploading anything, and failing the job if there is a leak.

That way we are preventing the leak rather than detecting it (which would require us to mitigate the leak, which is quite elaborate) and also save enormous amount of time and traffic on pulling every single artifact back from S3 to the runner.

I suggest to modify S3Helper.upload_file/S3Helper.upload_build_directory_to_s3 (maybe something else?)
https://github.com/Altinity/ClickHouse/blob/releases/24.8.14/tests/ci/s3_helper.py

[strtgbb]

Checking after is what was initially discussed.
The traffic usage is a good point. Checking before will be cheaper as long as we're using S3.
upload_build_directory_to_s3 ultimately calls upload_file. Unless it contains compressed files, I don't think it needs any special handling.
I don't think there is enough in common between that proposal and the existing script, I will create a new PR.

[strtgbb]

See #701