Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

# Run logs
backend/run_logs/*
logs/

# Weird Docker setup related files
backend/backend/*
Expand All @@ -14,3 +15,5 @@ frontend/.env.local

# Mac files
.DS_Store
node_modules
yarn.lock
103 changes: 103 additions & 0 deletions PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Claude Code Pro CLI Integration

## Summary

This PR adds comprehensive Claude Code Pro CLI integration to screenshot-to-code, enabling users to generate code from screenshots using their Claude Code Pro subscription without requiring separate API keys.

### Key Features
- ✅ **Full Image Analysis**: Claude CLI integration with screenshot processing
- ✅ **Zero API Keys**: Works entirely with Claude Code Pro subscription
- ✅ **Smart Fallback**: Automatically falls back to API if available
- ✅ **Configurable Variants**: Control number of variants via `NUM_VARIANTS` env var
- ✅ **Enhanced Scripts**: Comprehensive development scripts with process management

### Technical Implementation

#### New Claude CLI Wrapper (`backend/models/claude_cli.py`)
- Base64 image extraction and temp file handling
- Streaming response support with character-by-character output
- Proper cleanup of temporary image files
- Support for both regular and video generation modes

#### Modified Route Handler (`backend/routes/generate_code.py`)
- Uses CLI by default for all Claude models
- Maintains backward compatibility with API keys
- Enhanced error handling and process management
- Smart model selection based on available resources

#### Enhanced Configuration (`backend/config.py`)
- `NUM_VARIANTS` environment variable support
- Configurable model selection based on available resources
- Default NUM_VARIANTS=1 for simplified UX when using CLI only

### Development Experience Improvements
- **Automated Scripts**: `start.sh`, `stop.sh`, `restart.sh`, `dev.sh`, `start-separate.sh`
- **Process Management**: PID tracking and graceful shutdown
- **Enhanced Logging**: Separate log files for backend/frontend with timestamps
- **Status Monitoring**: Real-time server status checking
- **Dependency Management**: Auto-install missing dependencies

### Files Changed
```
backend/models/claude_cli.py # New Claude CLI integration
backend/routes/generate_code.py # Modified to use CLI by default
backend/config.py # Added NUM_VARIANTS configuration
README.md # Updated with CLI setup instructions
.gitignore # Enhanced for better project management
start.sh # Comprehensive server startup script
stop.sh # Graceful server shutdown script
restart.sh # Combined restart functionality
dev.sh # Multi-purpose development helper
start-separate.sh # Alternative startup in separate terminals
```

### Benefits
- 🚀 **Immediate Value**: Users with Claude Code Pro can start using immediately
- 💰 **Cost Savings**: No additional API costs for Pro subscribers
- 🛠️ **Better DX**: Enhanced development scripts and process management
- 🔄 **Backward Compatible**: Existing API key setups continue to work
- 🖼️ **Full Feature Parity**: Image analysis works through CLI integration

### Setup Instructions

#### For Claude Code Pro Users (No API Keys Needed)
```bash
# Authenticate with Claude Code Pro
claude auth login

# Clone and setup
git clone https://github.com/abi/screenshot-to-code.git
cd screenshot-to-code

# Start servers (creates NUM_VARIANTS=1 automatically)
./start.sh
```

#### For API Key Users (Existing Workflow)
```bash
# Setup with API keys as before
echo "OPENAI_API_KEY=sk-your-key" > backend/.env
echo "ANTHROPIC_API_KEY=your-key" >> backend/.env
./start.sh
```

### Test Plan
- [x] Test screenshot upload and code generation with Claude Code Pro
- [x] Verify image analysis works through CLI integration
- [x] Test server startup/shutdown scripts with PID management
- [x] Validate environment variable configuration (NUM_VARIANTS)
- [x] Test graceful fallback to API when available
- [x] Verify backward compatibility with existing API key setups
- [x] Test all development helper scripts (dev.sh, status checking)

### Breaking Changes
None - this is fully backward compatible.

### Migration Notes
Existing users can continue using API keys as before. New users with Claude Code Pro can skip API key setup entirely.

---

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
47 changes: 45 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,20 @@ The app has a React/Vite frontend and a FastAPI backend.

Keys needed:

- [OpenAI API key with access to GPT-4](https://github.com/abi/screenshot-to-code/blob/main/Troubleshooting.md) or Anthropic key (optional)
- Both are recommended so you can compare results from both Claude and GPT4o
- [OpenAI API key with access to GPT-4](https://github.com/abi/screenshot-to-code/blob/main/Troubleshooting.md) (optional)
- Anthropic API key (optional) - **OR** use Claude Code Pro CLI (no API key needed!)
- If using Claude Code Pro: Make sure `claude` CLI is installed and authenticated with `claude auth login`

**✅ Full Support:** Claude Code Pro CLI now supports image analysis!
- **Screenshots & Images**: Works with Claude Code Pro CLI (no API key needed)
- **Text prompts**: Works with Claude Code Pro CLI (no API key needed)
- **Fallback**: Can still use Anthropic API key if preferred

If you'd like to run the app with Ollama open source models (not recommended due to poor quality results), [follow this comment](https://github.com/abi/screenshot-to-code/issues/354#issuecomment-2435479853).

Run the backend (I use Poetry for package management - `pip install --upgrade poetry` if you don't have it):

**Option 1: With API keys**
```bash
cd backend
echo "OPENAI_API_KEY=sk-your-key" > .env
Expand All @@ -56,6 +63,16 @@ poetry shell
poetry run uvicorn main:app --reload --port 7001
```

**Option 2: With Claude Code Pro (no API key needed)**
```bash
cd backend
# Make sure Claude CLI is authenticated
claude auth login
poetry install
poetry shell
poetry run uvicorn main:app --reload --port 7001
```

You can also set up the keys using the settings dialog on the front-end (click the gear icon after loading the frontend).

Run the frontend:
Expand All @@ -68,6 +85,32 @@ yarn dev

Open http://localhost:5173 to use the app.

## 🚀 Quick Start Commands

**Simple Commands (Recommended):**
```bash
./start.sh # Start both servers
./stop.sh # Stop both servers
./restart.sh # Restart both servers
```

**Development Helper:**
```bash
./dev.sh status # Check server status
./dev.sh backend # Start backend only
./dev.sh frontend # Start frontend only
./dev.sh install # Install dependencies
```

**Manual Commands:**
```bash
# Backend
cd backend && python -m uvicorn main:app --reload --port 7001

# Frontend
cd frontend && yarn dev
```

If you prefer to run the backend on a different port, update VITE_WS_BACKEND_URL in `frontend/.env.local`

For debugging purposes, if you don't want to waste GPT4-Vision credits, you can run the backend in mock mode (which streams a pre-recorded response):
Expand Down
5 changes: 4 additions & 1 deletion backend/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,10 @@
# TODO: Should only be set to true when value is 'True', not any abitrary truthy value
import os

NUM_VARIANTS = 4
NUM_VARIANTS = int(os.environ.get("NUM_VARIANTS", 4))
# Set to 1 if you only want Claude Code Pro CLI
# Set to 2 for Claude CLI + backup model
# Set to 4 for full comparison (requires API keys)

# LLM-related
OPENAI_API_KEY = os.environ.get("OPENAI_API_KEY", None)
Expand Down
Loading