LLM Chat Application Template

A simple, ready-to-deploy chat application template powered by Cloudflare Workers AI. This template provides a clean starting point for building AI chat applications with streaming responses.

Demo

This template demonstrates how to build an AI-powered chat interface using Cloudflare Workers AI with streaming responses. It features:

• Real-time streaming of AI responses using Server-Sent Events (SSE)
• Easy customization of models and system prompts
• Support for AI Gateway integration
• Clean, responsive UI that works on mobile and desktop

Features

• 💬 Simple and responsive chat interface
• ⚡ Server-Sent Events (SSE) for streaming responses
• 🧠 Powered by Cloudflare Workers AI LLMs
• 🛠️ Built with TypeScript and Cloudflare Workers
• 📱 Mobile-friendly design
• 🔄 Maintains chat history on the client
• 🔎 Built-in Observability logging

Getting Started

Prerequisites

• Node.js (v18 or newer)
• Wrangler CLI
• A Cloudflare account with Workers AI access

Installation

1. Clone this repository:

   git clone https://github.com/cloudflare/templates.git
   cd templates/llm-chat-app

2. Install dependencies:

   npm install

3. Generate Worker type definitions:

   npm run cf-typegen

Development

Start a local development server:

npm run dev

This will start a local server at http://localhost:8787.

Note: Using Workers AI accesses your Cloudflare account even during local development, which will incur usage charges.

Deployment

Deploy to Cloudflare Workers:

npm run deploy

Monitor

View real-time logs associated with any deployed Worker:

npm wrangler tail

Project Structure

/
├── public/             # Static assets
│   ├── index.html      # Chat UI HTML
│   └── chat.js         # Chat UI frontend script
├── src/
│   ├── index.ts        # Main Worker entry point
│   └── types.ts        # TypeScript type definitions
├── test/               # Test files
├── wrangler.jsonc      # Cloudflare Worker configuration
├── tsconfig.json       # TypeScript configuration
└── README.md           # This documentation

How It Works

Backend

The backend is built with Cloudflare Workers and uses the Workers AI platform to generate responses. The main components are:

1. API Endpoint (/api/chat): Accepts POST requests with chat messages and streams responses
2. Streaming: Uses Server-Sent Events (SSE) for real-time streaming of AI responses
3. Workers AI Binding: Connects to Cloudflare's AI service via the Workers AI binding

Frontend

The frontend is a simple HTML/CSS/JavaScript application that:

1. Presents a chat interface
2. Sends user messages to the API
3. Processes streaming responses in real-time
4. Maintains chat history on the client side

Customization

Changing the Model

To use a different AI model, update the MODEL_ID constant in src/index.ts. You can find available models in the Cloudflare Workers AI documentation.

Using AI Gateway

The template includes commented code for AI Gateway integration, which provides additional capabilities like rate limiting, caching, and analytics.

To enable AI Gateway:

1. Create an AI Gateway in your Cloudflare dashboard
2. Uncomment the gateway configuration in src/index.ts
3. Replace YOUR_GATEWAY_ID with your actual AI Gateway ID
4. Configure other gateway options as needed:
   • skipCache: Set to true to bypass gateway caching
   • cacheTtl: Set the cache time-to-live in seconds

Learn more about AI Gateway.

Modifying the System Prompt

The default system prompt can be changed by updating the SYSTEM_PROMPT constant in src/index.ts.

Styling

The UI styling is contained in the <style> section of public/index.html. You can modify the CSS variables at the top to quickly change the color scheme.

Resources

• Cloudflare Workers Documentation
• Cloudflare Workers AI Documentation
• Workers AI Models