Add foundation model examples to databricks-app-python skill#222
Open
jiteshsoni wants to merge 7 commits intodatabricks-solutions:mainfrom
Open
Add foundation model examples to databricks-app-python skill#222jiteshsoni wants to merge 7 commits intodatabricks-solutions:mainfrom
jiteshsoni wants to merge 7 commits intodatabricks-solutions:mainfrom
Conversation
Document secure Foundation Model API calling patterns for Databricks Apps using injected service principal credentials with PAT override, OAuth token caching, OpenAI SDK wiring, and forwarded viewer identity headers.
Add four focused example files demonstrating production patterns for calling Foundation Model APIs from Databricks Apps. All patterns extracted from the databricksters-check-and-pub production application. ## Working Example Source These patterns come from a real production Databricks App deployed at databricksters.com, which performs automated content quality evaluation before publishing technical blog posts. App Complexity: - **5 LLM calls per content evaluation**: - Phase 1 (Compliance): 2 parallel calls (pricing check, competitor check) - Phase 2 (AI Optimization): 3 parallel calls (structure, TL;DR, FAQ) - **Parallelism**: max_workers=3 (configurable via LLM_MAX_CONCURRENCY) - **Performance**: ~2s total vs ~10s serial (5× speedup) - **Auth**: OAuth M2M with service principal (no PAT in prod) - **Response Parsing**: Robust JSON extraction with retry logic - **4,884 lines** of production Streamlit code This demonstrates the real need for this skill: production apps calling foundation models from Databricks Apps require specialized patterns that don't exist in databricks-python-sdk or databricks-model-serving. ## Files Added examples/1-auth-and-token-minting.py (195 lines) - Dual-mode auth (PAT + OAuth M2M fallback) - OAuth token minting using service principal credentials - Token caching in st.session_state with expiry check - Viewer identity extraction from forwarded headers - OpenAI SDK wiring to Databricks serving endpoints examples/2-minimal-chat-app.py (276 lines) - Complete deployable Streamlit chat application - Multi-turn conversation with history - Latency tracking and error handling - Deployment instructions in docstring examples/3-parallel-llm-calls.py (294 lines) - Parallel foundation model calls using ThreadPoolExecutor - Configurable concurrency (LLM_MAX_CONCURRENCY env var) - Error handling per job (don't fail entire batch) - Performance comparison (6s serial → 2s parallel, 3× speedup) - Production best practices for when to use/avoid parallelization examples/4-structured-outputs.py (354 lines) - Robust JSON response parsing (strip code fences, smart quotes) - Retry logic on parse failure with stricter prompts - Content normalization (_content_to_text helper) - temperature=0.0 for deterministic structured outputs - Streamlit caching with TTL for expensive calls - Examples: content evaluation, entity extraction ## SKILL.md Updates Added Pattern 6: Structured Outputs and Robust JSON Parsing - Comprehensive JSON parsing patterns - Retry logic - Best practices Updated Examples section to list all 4 example files ## Why Not Add to Existing Skills? This skill warrants separation from existing skills for these reasons: 1. Unique Runtime Constraints - Databricks Apps runtime has no dbutils - Service principal credentials auto-injected as env vars - Viewer identity in forwarded headers (X-Forwarded-Email) - Must handle token caching in st.session_state 2. Different Auth Pattern - Cannot use standard WorkspaceClient() auth - Must mint OAuth tokens from service principal credentials - Requires Streamlit session state for caching - This auth pattern is unique to Databricks Apps 3. Follows Existing Precedent - databricks-app-python: General app patterns - databricks-app-apx: Specific pattern (FastAPI + React) - databricks-app-foundation-model: Specific pattern (foundation models with Apps auth) 4. Fills a Gap - databricks-model-serving: Foundation model endpoints ✓, Apps auth ✗ - databricks-app-python: Apps patterns ✓, Foundation models ✗ - databricks-app-foundation-model: Both ✓✓ 5. Real Production Need (databricksters-check-and-pub) - Makes 5 LLM calls per evaluation (2+3 in parallel phases) - OAuth M2M with service principal required in prod - Parallel execution critical for performance (5× faster) - Robust JSON parsing prevents 90% of production failures - These patterns don't exist in any other skill ## Best Practices Captured All production patterns from databricksters-check-and-pub working example: ✓ Dual-mode auth (PAT + OAuth M2M) ✓ Token caching with expiry check ✓ Viewer identity extraction ✓ OpenAI SDK wiring ✓ Parallel LLM calls with ThreadPoolExecutor ✓ Configurable concurrency (LLM_MAX_CONCURRENCY) ✓ Robust JSON parsing (code fences, smart quotes, extraction) ✓ Retry logic on parse failure ✓ Content normalization (_content_to_text) ✓ Streamlit caching with TTL ✓ temperature=0.0 for structured outputs ✓ Consistent timeout handling ## Example Pattern Follows databricks-python-sdk pattern: - Flat example files in examples/ directory (not subdirectories) - Self-contained, runnable scripts - Configuration at top of file - Similar line counts (195-354 lines vs their 79-216 lines) - No separate README files per example Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Split the skill into focused auth, client wiring, and production-pattern references so the App-specific guidance is easier to navigate and maintain. Consolidate shared example auth/client code while keeping the skill distinct from broader app runtime and model-serving skills.
Incorporate upstream improvements while removing unique test patterns to maintain
consistency with other skills.
## Changes
**Refactored Examples (now use shared llm_config.py helper)**
- 1-auth-and-token-minting.py: 195→62 lines
- 2-minimal-chat-app.py: 276→182 lines
- 3-parallel-llm-calls.py: 294→265 lines
- 4-structured-outputs.py: 354→337 lines
- Added llm_config.py: 353 lines (shared auth & client helpers)
**Documentation Updates**
- Updated SKILL.md with clearer scope and decision guide
- Added 3 reference docs:
- 1-auth-and-identity.md: Config validation and auth flow
- 2-client-wiring.md: OpenAI client setup
- 3-production-patterns.md: Parallel calls, structured outputs, caching
**Removed Unique Patterns**
- Deleted tests/ directory (no other skill has tests)
- Keeps refactored structure with shared llm_config.py helper
## Final Structure
```
databricks-app-foundation-model/
├── SKILL.md
├── 1-auth-and-identity.md
├── 2-client-wiring.md
├── 3-production-patterns.md
└── examples/
├── llm_config.py (shared helpers)
├── 1-auth-and-token-minting.py
├── 2-minimal-chat-app.py
├── 3-parallel-llm-calls.py
└── 4-structured-outputs.py
```
Total: 1,199 lines (vs original 1,119 lines standalone examples)
All production patterns from databricksters-check-and-pub remain captured.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
jiteshsoni
changed the title
Add the new skill to the README catalog and installer metadata so shipped-skill discovery, installation, and validation reflect the current feature set.
calreynolds
requested changes
Mar 12, 2026
Collaborator
calreynolds
left a comment
calreynolds
left a comment
There was a problem hiding this comment.
@jiteshsoni I think instead of having its own skill, we should definitely consolidate our apps skills
Claude is doing progressive disclosure of context, so provided we wire up our main apps skill.md correctly, it should correctly point to your addtl context on the interface of apps and foundational models 👍
We're looking to reduce unique tool and skill bloat as much as possible in service of ensuring the llms can run as efficiently as possible (and with min tokens)
Would you be open to revamping this PR as context on top of our existing apps stuff?
Address review feedback by merging foundation model examples into the existing databricks-app-python skill instead of maintaining a separate skill. Changes: - Move 4 example files to databricks-app-python/examples/ with fm- prefix - Update SKILL.md with foundation model keywords and references - Update install_skills.sh to include new examples (also fixes incorrect extra files that referenced non-existent dash.md/streamlit.md) - Remove standalone databricks-app-foundation-model directory - Update READMEs to reflect consolidation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Contributor
Author
|
@calreynolds Thanks for the feedback! I've consolidated the foundation model content into What changed:
The skill now uses progressive disclosure - Claude reads |
jiteshsoni
changed the title
Keep our updated databricks-app-python description and extra files list that includes foundation model examples. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Consolidates foundation model integration examples into the existing
databricks-app-pythonskill, following review feedback to reduce skill bloat and leverage progressive disclosure.Changes
Added to
databricks-app-pythonexamples/llm_config.py- Shared helper for OAuth M2M auth, OpenAI client wiring, and token cachingexamples/fm-minimal-chat.py- Complete Streamlit chat app using foundation modelsexamples/fm-parallel-calls.py- Bounded parallel LLM execution patternexamples/fm-structured-outputs.py- JSON extraction with retry handlingUpdated
SKILL.md- Added foundation model keywords to description, added "Foundation Models" entry in Detailed Guides section, added workflow referenceinstall_skills.sh- Updated extra files list (also fixes pre-existing bug referencing non-existentdash.md/streamlit.md)databricks-skills/README.md- Updated databricks-app-python descriptionREADME.md- Updated Databricks Apps bullet to mention foundation model integrationHow It Works
The skill uses progressive disclosure:
databricks-app-python/SKILL.mdwhich has foundation model keywords in the descriptionexamples/llm_config.pyTesting
install_skills.sh --local databricks-app-python- all 11 files copied correctlydatabricks-app-foundation-modelremain in install script