Skip to content

Conversation

@ikreymer
Copy link
Member

@ikreymer ikreymer commented May 16, 2025

Use V4 ('s3v4') signature version for for all presigning URLs to support backblaze, fixes #2472

  • add 'access_addressing_style' to be able to choose virtual/path addressing for access endpoint (default to 'virtual' as before)
  • fix minio presigning with v4 by using 'path' addressing style for minio
  • if path matches '/data/' for internal minio bucket, then always use 'path'
  • also make minio access path '/data/' configurable

also simplify running in any namespace with default settings:

  • don't hardcode 'local-minio.default'
  • in crawlers namespace, add a 'local-minio' externalName service which maps to the service in which the main namespace is

@ikreymer ikreymer requested a review from tw4l May 16, 2025 19:50
@ikreymer ikreymer changed the title storage: use s3v4 signature for presigning urls, storage: use s3v4 signature for presigning urls May 16, 2025
- fix minio signing with v4 by using path-style addressing for minio
- add 'access_addressing_style' to be able to choose virtual/path addressing for access endpoint
- default to 'virtual', unless path is '/data/' for internal minio bucket, then use path
- also make '/data/' configurable
- remove isV4Signature, now always using v4

simplify running in any namespace with default settings:
- don't hardcode 'local-minio.default'
- in crawlers namespace, add a 'local-minio' externalName service which maps to the service in which the main namespace is
Copy link
Member

@tw4l tw4l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working locally with minio and on dev with Backblaze, and some nice improvements here. Thanks!

@ikreymer ikreymer merged commit f1fd11c into main May 19, 2025
24 of 25 checks passed
@ikreymer ikreymer deleted the issue-2472-storage-use-v4-sig branch May 19, 2025 22:44
ikreymer added a commit that referenced this pull request May 20, 2025
remove namespace from minio config to match settings
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Incorrectly using Signature v2 API for generating presigned URLs in Backblaze B2

3 participants