are auth credentials thread safe?

[david-gang]

I have the following code. it is running from many threads for different gcs blobs:

signed_url: str = blob.generate_signed_url(
            version="v4",
            expiration=timedelta(minutes=100),
            method=method,
            content_type=content_type,
            headers=headers,
            credentials=self._signing_credentials,
        )

where the credentials are

google.authcompute_engine.IDTokenCredentials

Now the problem is that these credentials need to be refreshed in the before_request:

google-auth-library-python/google/auth/credentials.py

Line 239 in ca94ead

self._blocking_refresh(request)

but this is not thread safe

google-auth-library-python/google/auth/credentials.py

Line 200 in ca94ead

def _blocking_refresh(self, request):

    def _blocking_refresh(self, request):
        if not self.valid:
            self.refresh(request)

do i miss something?

what do you suggest ? ignoring the issue because it is normally not a problem and can just cause multiple http calls ?

[david-gang]

My currrent solution is to have a class which wraps the credentials to be on the safe side

class CredentialsManager:
    
    def __init__(self, credentials) -> None:
        
        self._actual_credentials = credentials
        self._lock = RLock()
    
    def get_credentials(self) -> Credentials :
        """Get credentials, refreshing them as needed."""
            
        with self._lock:
            if self._actual_credentials.token_state is not TokenState.FRESH:
                auth_request = requests.Request()
                self._actual_credentials.refresh(auth_request)
            
            return self._actual_credentials 

and now i can call it from the code and be sure that refresh is just called once. No idea if this is an overkill. would be happy to hear what you think (could improve with double locking pattern but python has anyway a gil so we don't gain as much as in JAVA)

[sai-sunder-s]

I believe you could call with_non_blocking_refresh on your credential object. This should make sure only one thread does the refresh.

google-auth-library-python/google/auth/credentials.py

Line 244 in ca94ead

def with_non_blocking_refresh(self):

[david-gang]

That's interesting. After digging into it i have the following question:
this function sets: self._use_non_blocking_refresh = True

if this is true the _non_blocking_refresh function is called

google-auth-library-python/google/auth/credentials.py

Line 219 in ca94ead

def before_request(self, request, method, url, headers):

looking at this function:

google-auth-library-python/google/auth/credentials.py

Line 204 in ca94ead

def _non_blocking_refresh(self, request):

I have the following question:

if self.token_state == TokenState.INVALID or use_blocking_refresh_fallback:
            self.refresh(request)

This is not done in the refresh manager and without lock, and therefore can lead to multiple concurrent refresh calls .

I also checked the usage of this flag and i don't see it used in any place:

https://github.com/search?q=org%3Agoogleapis+with_non_blocking_refresh&type=code

https://github.com/search?q=with_non_blocking_refresh&type=code

Additionally while i am asking and have the opportunity to speak with core contributors:

why is the request deepcopied at:

google-auth-library-python/google/auth/_refresh_worker.py

Line 58 in ca94ead

self._worker = RefreshThread(cred=cred, request=copy.deepcopy(request))

isn't google.auth.request itself threadsafe? It looks like in the code adjustments for thread safety were done:

google-auth-library-python/google/auth/transport/requests.py

Line 516 in ca94ead

# Use a kwarg for this instead of an attribute to maintain

At last I would like to get an official recommendation how and if to use credentials in a multi threaded environment