Skip to content

💬 Visual Message Box: Two-Way Communication in DevTools Panel #71

New issue
New issue
Open
Open
💬 Visual Message Box: Two-Way Communication in DevTools Panel#71
@ahelme

Description

@ahelme
ahelme
opened on Oct 4, 2025

💬 Visual Message Box: Two-Way Communication in DevTools Panel

🎯 Concept

Add a Slack/Discord-style message box directly in the Browser Tools DevTools panel for true two-way visual communication between user and Claude!

No more context switching! Send messages + screenshots to Claude directly from the panel, and see Claude's responses right there.

🎨 UI Mockup

```
┌──────────────────────────────────────────┐
│ Browser Tools • Connected to Claude │
├──────────────────────────────────────────┤
│ │
│ 📸 Quick Capture: │
│ [📄 Full] [🖼️ Viewport] [🎯 Element] │
│ │
│ ───────────────────────────────────── │
│ │
│ 💬 Message Claude: │
│ ┌─────────────────────────────────┐ │
│ │ Type your message here... │ │
│ │ │ │
│ │ Tip: Select screenshots below │ │
│ │ to attach them │ │
│ └─────────────────────────────────┘ │
│ │
│ 📎 Attach Screenshots: │
│ ☑ nav-menu.png (5s) [👁️ View] │
│ 📦 nav.main-menu • 1400x80px │
│ │
│ ☑ button-primary.png (10s) [👁️ View] │
│ 📦 button.primary • 200x48px │
│ │
│ ☐ old-screenshot.png (5m) [👁️ View] │
│ 📄 Full page • 1400x3200px │
│ │
│ [✉️ Send to Claude] ← 2 selected │
│ │
│ ───────────────────────────────────── │
│ │
│ 💬 Conversation: │
│ ┌─────────────────────────────────┐ │
│ │ You (2m ago): │ │
│ │ Can you check the nav alignment? │ │
│ │ 📎 nav-menu.png │ │
│ │ │ │
│ │ Claude (2m ago): │ │
│ │ ✅ Nav looks perfect! Centered │ │
│ │ at 700px. The button is off │ │
│ │ though - want me to check it? │ │
│ │ │ │
│ │ You (1m ago): │ │
│ │ Yes please! │ │
│ │ 📎 button-primary.png │ │
│ │ │ │
│ │ Claude (1m ago): │ │
│ │ ❌ Button is 12px off-center. │ │
│ │ Fixed! Here's the before/after: │ │
│ │ 📸 before-fix.png │ │
│ │ 📸 after-fix.png │ │
│ │ │ │
│ │ Run browser_visual_diff to see │ │
│ │ the exact changes? │ │
│ └─────────────────────────────────┘ │
│ │
│ [Auto-scroll] [Clear History] │
└──────────────────────────────────────────┘
```

✨ Key Features

1. Message Input Box

  • ✅ Multi-line text area
  • ✅ Markdown support (optional)
  • ✅ @-mentions for specific tools?
  • ✅ Keyboard shortcuts (Cmd+Enter to send)

2. Screenshot Attachment System

  • ✅ Checkbox list of recent screenshots
  • ✅ Auto-check screenshots from last 30 seconds
  • ✅ Preview thumbnails on hover
  • ✅ Element metadata shown (selector, dimensions)
  • ✅ Select multiple screenshots

3. Two-Way Conversation Display

  • ✅ Threaded messages (user + Claude)
  • ✅ Timestamps
  • ✅ Screenshot attachments inline
  • ✅ Message status indicators (sending, sent, read)
  • ✅ Auto-scroll to latest
  • ✅ Search/filter messages

4. Claude's Responses in Panel

  • ✅ Text responses
  • ✅ Screenshot attachments (before/after, visual diffs)
  • ✅ Code snippets
  • ✅ Status updates ("Analyzing...", "Fixed!")
  • ✅ Tool suggestions ("Run browser_visual_diff?")

🔧 Technical Architecture

Message Flow:

```
User types message + selects screenshots
↓
Extension Panel UI
↓
HTTP Bridge (POST /visual-message)
↓
MCP Server
↓
Claude Code (via MCP tool)
↓
Claude processes & responds
↓
MCP Server (response)
↓
HTTP Bridge (WebSocket push)
↓
Extension Panel (display response)
↓
User sees Claude's response in panel!
```

🛠️ Implementation Details

New MCP Tool: browser_receive_visual_message

```javascript
{
name: "browser_receive_visual_message",
description: "Receive message from user with attached screenshots",
inputSchema: {
type: "object",
properties: {
message: {
type: "string",
description: "User's message text"
},
screenshots: {
type: "array",
items: {
type: "object",
properties: {
filename: { type: "string" },
imageBase64: { type: "string" },
element: {
type: "object",
properties: {
selector: { type: "string" },
dimensions: { type: "string" },
tagName: { type: "string" },
classes: { type: "array" }
}
},
timestamp: { type: "string" }
}
}
}
}
}
}
```

New WebSocket Event: visual-message-response

```javascript
// HTTP Bridge → Extension Panel
{
type: "visual-message-response",
data: {
sender: "claude",
message: "✅ Nav looks perfect! Centered at 700px.",
screenshots: [
{
filename: "analysis-overlay.png",
imageBase64: "...",
caption: "Nav with measurement overlay"
}
],
timestamp: "2025-10-05T12:35:00Z",
messageId: "msg-12345"
}
}
```

Extension Panel Component:

```javascript
class VisualMessagePanel extends EventTarget {
constructor() {
super();
this.messages = [];
this.screenshots = [];
this.websocket = null;
}

async sendMessage(text, selectedScreenshots) {
// Add to local message history
const message = {
id: generateId(),
sender: 'user',
text: text,
screenshots: selectedScreenshots,
timestamp: new Date(),
status: 'sending'
};

this.messages.push(message);
this.render();

// Send to HTTP bridge
try {
  const response = await fetch('http://localhost:3024/visual-message', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      message: text,
      screenshots: await this.loadScreenshotData(selectedScreenshots)
    })
  });
  
  message.status = 'sent';
  this.render();
} catch (error) {
  message.status = 'error';
  message.error = error.message;
  this.render();
}

}

receiveMessage(data) {
// Called when WebSocket receives response from Claude
const message = {
id: data.messageId,
sender: 'claude',
text: data.message,
screenshots: data.screenshots,
timestamp: new Date(data.timestamp),
status: 'received'
};

this.messages.push(message);
this.render();
this.scrollToBottom();

// Show desktop notification?
if (!document.hasFocus()) {
  this.showNotification('New message from Claude', data.message);
}

}

render() {
// Render conversation UI
return <div class=\"visual-message-panel\"> ${this.renderMessageInput()} ${this.renderScreenshotAttachments()} ${this.renderConversation()} </div>;
}

renderConversation() {
return this.messages.map(msg => <div class=\"message ${msg.sender}\"> <div class=\"message-header\"> <span class=\"sender\">${msg.sender === 'user' ? 'You' : 'Claude'}</span> <span class=\"timestamp\">${formatTimestamp(msg.timestamp)}</span> ${msg.status ?<span class="status ${msg.status}">${msg.status}` : ''}

    <div class=\"message-body\">
      ${this.formatMessageText(msg.text)}
    </div>
    
    ${msg.screenshots && msg.screenshots.length > 0 ? `
      <div class=\"message-attachments\">
        ${msg.screenshots.map(s => `
          <div class=\"attachment\">
            📎 ${s.filename}
            ${s.caption ? `<span class=\"caption\">${s.caption}</span>` : ''}
            <button onclick=\"viewImage('${s.imageBase64}')\">View</button>
          </div>
        `).join('')}
      </div>
    ` : ''}
  </div>
`).join('');

}
}
```

💡 Usage Examples

Example 1: Simple Question

```
Extension Panel:

Message: "Is the nav centered?"
Attach: ☑ nav-menu.png
[Send]

─────────

Conversation:
┌─────────────────────────┐
│ You (now): │
│ Is the nav centered? │
│ 📎 nav-menu.png │
└─────────────────────────┘

[30 seconds later]

┌─────────────────────────┐
│ Claude (now): │
│ ✅ Yes! The nav is │
│ perfectly centered at │
│ 700px (viewport center).│
│ │
│ Measurements: │
│ • Nav width: 1400px │
│ • Nav center: 700px │
│ • Viewport: 1400px │
└─────────────────────────┘
```

Example 2: Multi-Screenshot Issue

```
Extension Panel:

Message: "These three areas look wrong, can you check?"
Attach: ☑ header.png
☑ nav.png
☑ footer.png
[Send]

─────────

Conversation:
┌─────────────────────────┐
│ You (now): │
│ These three areas look │
│ wrong, can you check? │
│ 📎 header.png │
│ 📎 nav.png │
│ 📎 footer.png │
└─────────────────────────┘

┌─────────────────────────┐
│ Claude (1m): │
│ Checked all three: │
│ │
│ ✅ Header: Good! │
│ ❌ Nav: 12px off │
│ ❌ Footer: Padding │
│ inconsistent │
│ │
│ Should I fix both │
│ issues? │
└─────────────────────────┘

┌─────────────────────────┐
│ You (now): │
│ Yes please! │
└─────────────────────────┘

┌─────────────────────────┐
│ Claude (30s): │
│ ✅ Fixed both! │
│ │
│ Changes made: │
│ 1. Nav: Added │
│ margin-left: 12px │
│ 2. Footer: Standardized │
│ padding to 20px │
│ │
│ 📸 before-nav.png │
│ 📸 after-nav.png │
│ 📸 before-footer.png │
│ 📸 after-footer.png │
│ │
│ Want me to run │
│ browser_visual_diff? │
└─────────────────────────┘
```

Example 3: Ongoing Conversation

```
Conversation shows full thread:

┌─────────────────────────┐
│ You (5m): │
│ Subscribe to screenshots│
└─────────────────────────┘

┌─────────────────────────┐
│ Claude (5m): │
│ ✅ Subscribed! │
└─────────────────────────┘

┌─────────────────────────┐
│ You (4m): │
│ Check the nav │
│ 📎 nav.png │
└─────────────────────────┘

┌─────────────────────────┐
│ Claude (4m): │
│ Nav is 12px off. Fixed! │
└─────────────────────────┘

┌─────────────────────────┐
│ You (3m): │
│ Can you check button? │
│ 📎 button.png │
└─────────────────────────┘

┌─────────────────────────┐
│ Claude (3m): │
│ Button looks good! ✅ │
└─────────────────────────┘

[Auto-scroll keeps latest visible]
```

🎯 Benefits

For Users:

  • ✅ No context switching - Stay in DevTools
  • ✅ Visual context - Screenshots attached to messages
  • ✅ Clear communication - No ambiguity about what to check
  • ✅ Conversation history - See past discussions
  • ✅ Familiar UX - Like Slack/Discord

For Claude:

  • ✅ Clear intent - User tells me what to focus on
  • ✅ Multi-screenshot context - Understand relationships
  • ✅ Respond directly - Send screenshots back to user
  • ✅ Persistent thread - Remember conversation context

For Pair Programming:

  • ✅ True collaboration - Back-and-forth conversation
  • ✅ Visual artifacts - Screenshots as shared reference
  • ✅ Real-time feedback - No waiting for terminal responses
  • ✅ Embedded in workflow - Right where you're working

📋 Implementation Checklist

Phase 1: Basic Messaging

  • Message input text area
  • Send button + keyboard shortcut
  • Message history display
  • User/Claude message styling
  • Timestamps

Phase 2: Screenshot Attachments

  • Screenshot list with checkboxes
  • Auto-check recent screenshots
  • Load screenshot data on send
  • Display attached screenshots in messages
  • Screenshot preview/view

Phase 3: Two-Way Communication

  • WebSocket listener for Claude responses
  • Display Claude's messages in thread
  • Claude can attach screenshots
  • Message status indicators
  • Desktop notifications

Phase 4: Polish

  • Auto-scroll to latest
  • Search/filter messages
  • Clear history
  • Message persistence (localStorage?)
  • Markdown formatting?
  • Code snippet formatting

🔧 Configuration

```
Visual Message Settings:
☑ Enable two-way messaging
☑ Desktop notifications for Claude responses
☑ Auto-scroll to latest message
☑ Auto-check recent screenshots (last 30s)

Message Retention:
◉ Session only (clear on close)
○ Keep last 50 messages
○ Keep last 24 hours

Screenshot Attachments:
Max attachments per message: [5]
Auto-compress large screenshots: ☑
```

🚀 Future Enhancements

V2 Features:

  • Voice messages? (Web Speech API)
  • Screen recording attachments
  • @-mentions for specific tools
  • Message reactions (👍 ❤️ 😂)
  • Claude can ask clarifying questions
  • Shared annotation on screenshots
  • Export conversation to markdown

💬 Why This is Game-Changing

Current workflow:

  1. Take screenshot
  2. Switch to terminal
  3. Type: "Claude, can you check the nav?"
  4. Wait for response in terminal
  5. Read response
  6. Switch back to browser
  7. Repeat...

New workflow:

  1. Type in panel: "Check the nav"
  2. Select screenshot
  3. Click Send
  4. See response RIGHT THERE
  5. Continue conversation in panel!

It's pair programming, but it FEELS like chatting with a colleague who's sitting next to you! 🎉

🎯 Success Metrics

  • ✅ Reduce context switching (stay in DevTools)
  • ✅ Faster iteration loops (immediate responses)
  • ✅ Better communication (screenshots + text)
  • ✅ Natural conversation flow (like messaging)
  • ✅ Higher user satisfaction (feels collaborative!)

📚 Related Issues

This would make Browser Tools feel less like a "tool" and more like a collaborative partner! 💖

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions