-
Notifications
You must be signed in to change notification settings - Fork 6
add possibility to add a custom headers to calls to LLMs #120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -29,8 +29,6 @@ def to_url(self): | |||||||||||||||||||||||||||
| case _: | ||||||||||||||||||||||||||||
| raise ValueError(f"Unsupported endpoint: {self}") | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| COPILOT_INTEGRATION_ID = 'vscode-chat' | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| # you can also set https://api.githubcopilot.com if you prefer | ||||||||||||||||||||||||||||
| # but beware that your taskflows need to reference the correct model id | ||||||||||||||||||||||||||||
| # since different APIs use their own id schema, use -l with your desired | ||||||||||||||||||||||||||||
|
|
@@ -52,6 +50,31 @@ def get_AI_token(): | |||||||||||||||||||||||||||
| return token | ||||||||||||||||||||||||||||
| raise RuntimeError("AI_API_TOKEN environment variable is not set.") | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| def get_custom_header() -> dict[str, str]: | ||||||||||||||||||||||||||||
| """ | ||||||||||||||||||||||||||||
| Get custom header from environment variable AI_API_CUSTOM_HEADER. | ||||||||||||||||||||||||||||
p- marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||||||||||||||
| Expected format: name:value | ||||||||||||||||||||||||||||
| Returns a dictionary that can be merged into request headers. | ||||||||||||||||||||||||||||
| """ | ||||||||||||||||||||||||||||
| custom_header = os.getenv('AI_API_CUSTOM_HEADER') | ||||||||||||||||||||||||||||
| if not custom_header: | ||||||||||||||||||||||||||||
| return {} | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| # Split on first colon to handle values that might contain colons | ||||||||||||||||||||||||||||
| parts = custom_header.split(':', 1) | ||||||||||||||||||||||||||||
| if len(parts) != 2: | ||||||||||||||||||||||||||||
| logging.warning(f"Invalid AI_API_CUSTOM_HEADER format. Expected 'name:value', got: {custom_header}") | ||||||||||||||||||||||||||||
| return {} | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| name, value = parts | ||||||||||||||||||||||||||||
| name = name.strip() | ||||||||||||||||||||||||||||
| value = value.strip() | ||||||||||||||||||||||||||||
| if not name or not value: | ||||||||||||||||||||||||||||
| logging.warning(f"Invalid AI_API_CUSTOM_HEADER: header name and value must be non-empty after stripping. Got: '{custom_header}'") | ||||||||||||||||||||||||||||
| return {} | ||||||||||||||||||||||||||||
| return {name: value} | ||||||||||||||||||||||||||||
p- marked this conversation as resolved.
Show resolved
Hide resolved
Comment on lines
+64
to
+75
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there a simplified version of JSON supported by the json.loads API?
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not that I know of. I agree that it makes the syntax more clunky. But it simplifies the code a lot and adds support for multiple fields, which somebody might find useful. This isn't going to be a frequently used feature is it?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It is actually needed all the time after removing the default integration id, so I'd rather not having to do something like |
||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
| # assume we are >= python 3.9 for our type hints | ||||||||||||||||||||||||||||
| def list_capi_models(token: str) -> dict[str, dict]: | ||||||||||||||||||||||||||||
| """Retrieve a dictionary of available CAPI models""" | ||||||||||||||||||||||||||||
|
|
@@ -69,12 +92,11 @@ def list_capi_models(token: str) -> dict[str, dict]: | |||||||||||||||||||||||||||
| case _: | ||||||||||||||||||||||||||||
| raise ValueError(f"Unsupported Model Endpoint: {api_endpoint}\n" | ||||||||||||||||||||||||||||
| f"Supported endpoints: {[e.to_url() for e in AI_API_ENDPOINT_ENUM]}") | ||||||||||||||||||||||||||||
| r = httpx.get(httpx.URL(api_endpoint).join(models_catalog), | ||||||||||||||||||||||||||||
| headers={ | ||||||||||||||||||||||||||||
| 'Accept': 'application/json', | ||||||||||||||||||||||||||||
| 'Authorization': f'Bearer {token}', | ||||||||||||||||||||||||||||
| 'Copilot-Integration-Id': COPILOT_INTEGRATION_ID | ||||||||||||||||||||||||||||
| }) | ||||||||||||||||||||||||||||
| headers = { | ||||||||||||||||||||||||||||
| 'Accept': 'application/json', | ||||||||||||||||||||||||||||
| 'Authorization': f'Bearer {token}', | ||||||||||||||||||||||||||||
| } | get_custom_header() | ||||||||||||||||||||||||||||
p- marked this conversation as resolved.
Show resolved
Hide resolved
p- marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||||||||||||||
| r = httpx.get(httpx.URL(api_endpoint).join(models_catalog), headers=headers) | ||||||||||||||||||||||||||||
| r.raise_for_status() | ||||||||||||||||||||||||||||
| # CAPI vs Models API | ||||||||||||||||||||||||||||
| match netloc: | ||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||
Uh oh!
There was an error while loading. Please reload this page.