Skip to content

Export AWS Access Credentials If Needed #194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jul 11, 2023
Merged

Export AWS Access Credentials If Needed #194

merged 7 commits into from
Jul 11, 2023

Conversation

ranchodeluxe
Copy link
Contributor

@ranchodeluxe ranchodeluxe commented Jul 10, 2023

Background

Workaround for #192 so that we don't have to change anything in titiler-pgstac

Changes

  • adds VEDA_RASTER_EXPORT_ASSUME_ROLE_CREDS_AS_ENVS to APISettings
  • adds VEDA_RASTER_EXPORT_ASSUME_ROLE_CREDS_AS_ENVS=True as defaults to UAH dev/staging env secrets
  • adds a quick conditional check for VEDA_RASTER_EXPORT_ASSUME_ROLE_CREDS_AS_ENVS in get_gdal_config to know if it should export the assume-role creds. The reason this check exists is b/c we know Lambdas already inject AWS_* os env vars and this is a VERY hacky solution so we don't want to get used to these things being exported and also want to have a trail of why we did this for future us

Why It Works

  • rasterio.Env uses thread local which means it inits a new empty context per thread

  • So by the the time CustomSTACReader is retrieving the image in this line it's running inside of multiple threads. Each thread's CustomSTACReader triggers self.ctx (which is just rasterio.Env) and it has to start all over again with empty context when accessing the image.

  • However, because we now export the AWS credentials into os env vars the nested rasterio.Env lookup will hit this check and re-init a new session: https://github.com/rasterio/rasterio/blob/main/rasterio/env.py#L204-L205

@smohiudd smohiudd self-requested a review July 10, 2023 23:14
@ranchodeluxe ranchodeluxe merged commit 2fae434 into develop Jul 11, 2023
@anayeaye anayeaye deleted the issue-192 branch August 23, 2023 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants