diff --git a/docs/contributing/coding_guidelines.md b/docs/contributing/coding_guidelines.md index cb60dd190..5e950c436 100644 --- a/docs/contributing/coding_guidelines.md +++ b/docs/contributing/coding_guidelines.md @@ -7,12 +7,23 @@ You are an expert Python developer contributing to the Google Dotprompt project. - **Language**: Python. - **Environment Management**: Use `uv` for packaging and environment management. -### Libraries +### Python Libraries +- **JSON**: Use the built-in `json` module or `pydantic` for serialization. +- **Testing**: Use `pytest` and `unittest` for testing. + +### Java Libraries - **JSON**: Use **Jackson** for JSON parsing and serialization. Avoid Gson. - **Testing**: Use **Google Truth** (`com.google.truth.Truth`) for assertions. Use JUnit 4/5 for test runners. - **Utilities**: Use **Guava** for immutable collections and common utilities. - **Dependency Injection**: Use **Dagger** for dependency injection. +### Java Style Guidelines +- **Imports**: Always use proper imports instead of fully qualified type names. Never write `com.github.jknack.handlebars.Context` inline; instead, add an `import` statement and use `Context`. +- **Formatting**: Use `google-java-format` for code formatting. +- **Javadoc**: Write comprehensive Javadoc for all public classes and methods. +- **Doc Sync**: Keep Javadoc comments in sync with the code. When modifying method signatures, parameters, or return types, update the corresponding Javadoc. +- **Method Chaining**: Fluent builder methods should return `this` for chaining. + ## 2. Typing & Style - **Type Unions**: Use the pipe operator `|` (PEP 604) for union types (e.g., `int | str`) instead of `typing.Union`. Use `| None` for optional types. - **Generics**: Use standard collection generics (PEP 585) like `list`, `dict`, `tuple` (lowercase) for type hints instead of `typing.List`, `typing.Dict`. diff --git a/java/com/google/dotprompt/helpers/BUILD.bazel b/java/com/google/dotprompt/helpers/BUILD.bazel index 9c99d5745..0b5278168 100644 --- a/java/com/google/dotprompt/helpers/BUILD.bazel +++ b/java/com/google/dotprompt/helpers/BUILD.bazel @@ -20,6 +20,7 @@ java_library( name = "helpers", srcs = [ "Helpers.java", + "package-info.java", ], visibility = ["//visibility:public"], deps = [ diff --git a/java/com/google/dotprompt/helpers/package-info.java b/java/com/google/dotprompt/helpers/package-info.java new file mode 100644 index 000000000..908e4acc9 --- /dev/null +++ b/java/com/google/dotprompt/helpers/package-info.java @@ -0,0 +1,129 @@ +/* + * Copyright 2025 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ + +/** + * Handlebars helper functions for Dotprompt templates. + * + *

This package provides custom Handlebars helpers that enable rich template functionality in + * Dotprompt. These helpers are automatically registered when creating a {@link + * com.google.dotprompt.Dotprompt} instance. + * + *

Available Helpers

+ * + *
+ *  ┌──────────────────────────────────────────────────────────────────┐
+ *  │                    Dotprompt Handlebars Helpers                  │
+ *  ├──────────────┬───────────────────────────────────────────────────┤
+ *  │ json         │ Serialize objects to JSON                         │
+ *  │ role         │ Switch message role (user, model, system)         │
+ *  │ history      │ Insert conversation history                       │
+ *  │ section      │ Delineate named sections in output                │
+ *  │ media        │ Embed media (images, audio, video)                │
+ *  │ ifEquals     │ Conditional block if values are equal             │
+ *  │ unlessEquals │ Conditional block unless values are equal         │
+ *  └──────────────┴───────────────────────────────────────────────────┘
+ * 
+ * + *

Helper Usage Examples

+ * + *

json

+ * + *

Serializes any object to formatted JSON. Supports an optional {@code indent} parameter. + * + *

{@code
+ * {{json myObject}}
+ * {{json myObject indent=2}}
+ * }
+ * + *

role

+ * + *

Switches the role for subsequent content. Valid roles: user, model, system. + * + *

{@code
+ * {{role "system"}}You are a helpful assistant.
+ * {{role "user"}}Hello!
+ * }
+ * + *

history

+ * + *

Injects conversation history at this position. History messages are passed via the render + * data. + * + *

{@code
+ * {{role "system"}}You are a helpful assistant.
+ * {{history}}
+ * {{role "user"}}{{userMessage}}
+ * }
+ * + *

section

+ * + *

Creates a named section marker for structured output parsing. + * + *

{@code
+ * {{section "introduction"}}
+ * Welcome to the tutorial.
+ * {{section "main"}}
+ * Main content here.
+ * }
+ * + *

media

+ * + *

Embeds media content with a URL and optional content type. + * + *

{@code
+ * {{media url="https://example.com/image.png"}}
+ * {{media url=imageUrl contentType="image/jpeg"}}
+ * }
+ * + *

ifEquals / unlessEquals

+ * + *

Conditional rendering based on equality comparison. + * + *

{@code
+ * {{#ifEquals status "active"}}
+ *   User is active!
+ * {{else}}
+ *   User is inactive.
+ * {{/ifEquals}}
+ *
+ * {{#unlessEquals role "guest"}}
+ *   Welcome back, {{name}}!
+ * {{/unlessEquals}}
+ * }
+ * + *

Implementation

+ * + *

All helpers output special marker strings that are parsed by {@link + * com.google.dotprompt.parser.Parser} during rendering to produce structured {@link + * com.google.dotprompt.models.Message} objects. + * + *

+ *  Template                   Helpers                    Parser
+ *  ┌────────────┐          ┌───────────┐          ┌──────────────────┐
+ *  │ {{role     │ ──────▶  │ role()    │ ──────▶  │ Parser.parse()   │
+ *  │ "system"}} │          │           │          │                  │
+ *  │            │          │ Returns:  │          │ Produces:        │
+ *  │            │          │ marker    │          │ Message[role=    │
+ *  │            │          │ string    │          │   SYSTEM, ...]   │
+ *  └────────────┘          └───────────┘          └──────────────────┘
+ * 
+ * + * @see com.google.dotprompt.helpers.Helpers + * @see com.google.dotprompt.parser.Parser + */ +package com.google.dotprompt.helpers; diff --git a/java/com/google/dotprompt/models/BUILD.bazel b/java/com/google/dotprompt/models/BUILD.bazel index 49be9836b..f5767c9ea 100644 --- a/java/com/google/dotprompt/models/BUILD.bazel +++ b/java/com/google/dotprompt/models/BUILD.bazel @@ -53,6 +53,7 @@ java_library( "ToolDefinition.java", "ToolRequestPart.java", "ToolResponsePart.java", + "package-info.java", ], deps = [ "@maven//:com_fasterxml_jackson_core_jackson_annotations", diff --git a/java/com/google/dotprompt/models/package-info.java b/java/com/google/dotprompt/models/package-info.java new file mode 100644 index 000000000..ad8b47f52 --- /dev/null +++ b/java/com/google/dotprompt/models/package-info.java @@ -0,0 +1,131 @@ +/* + * Copyright 2025 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ + +/** + * Data models for Dotprompt prompts, messages, and content parts. + * + *

This package contains the core data structures used throughout Dotprompt for representing + * prompts, rendered messages, and their content. These models are designed to be immutable records + * for thread-safety and ease of use. + * + *

Type Hierarchy

+ * + *
+ *                        ┌─────────────────────────┐
+ *                        │         Part            │ (sealed interface)
+ *                        │  Base for all content   │
+ *                        └───────────┬─────────────┘
+ *                                    │
+ *        ┌───────────────┬───────────┼───────────┬─────────────────┐
+ *        │               │           │           │                 │
+ *        ▼               ▼           ▼           ▼                 ▼
+ *  ┌──────────┐   ┌───────────┐ ┌──────────┐ ┌────────────┐ ┌────────────────┐
+ *  │ TextPart │   │ MediaPart │ │ DataPart │ │ ToolRequest│ │ ToolResponse   │
+ *  │          │   │           │ │          │ │ Part       │ │ Part           │
+ *  └──────────┘   └───────────┘ └──────────┘ └────────────┘ └────────────────┘
+ *       │               │            │             │               │
+ *       │               ▼            │             │               │
+ *       │        ┌───────────┐       │             │               │
+ *       │        │MediaContent│      │             │               │
+ *       │        │ url,      │       │             │               │
+ *       │        │ contentType│      │             │               │
+ *       │        └───────────┘       │             │               │
+ *       │                            │             │               │
+ *       └─────────────┬──────────────┴─────────────┴───────────────┘
+ *                     │
+ *                     ▼
+ *               ┌──────────┐
+ *               │ Message  │ (role + List<Part> + metadata)
+ *               └────┬─────┘
+ *                    │
+ *                    ▼
+ *            ┌───────────────┐
+ *            │RenderedPrompt│ (config + List<Message>)
+ *            └───────────────┘
+ * 
+ * + *

Core Types

+ * + *

Content Parts

+ * + * + * + *

Messages and Prompts

+ * + * + * + *

Storage Types

+ * + * + * + *

Roles

+ * + *

The {@link com.google.dotprompt.models.Role} enum defines message roles: + * + *

+ *  ┌────────┬────────────────────────────────────────────┐
+ *  │ Role   │ Description                                │
+ *  ├────────┼────────────────────────────────────────────┤
+ *  │ USER   │ Input from the user                        │
+ *  │ MODEL  │ Output from the AI model                   │
+ *  │ SYSTEM │ System instructions                        │
+ *  │ TOOL   │ Tool/function call responses               │
+ *  └────────┴────────────────────────────────────────────┘
+ * 
+ * + *

Usage Example

+ * + *
{@code
+ * // Create a message with mixed content
+ * Message message = new Message(
+ *     Role.USER,
+ *     List.of(
+ *         new TextPart("Describe this image:"),
+ *         new MediaPart(new MediaContent("https://example.com/photo.jpg", "image/jpeg"))
+ *     ),
+ *     Map.of()
+ * );
+ *
+ * // Work with a rendered prompt
+ * RenderedPrompt rendered = promptFn.render(data).get();
+ * for (Message msg : rendered.messages()) {
+ *     System.out.println(msg.role() + ": " + msg.content());
+ * }
+ * }
+ * + * @see com.google.dotprompt.models.Part + * @see com.google.dotprompt.models.Message + * @see com.google.dotprompt.models.RenderedPrompt + */ +package com.google.dotprompt.models; diff --git a/java/com/google/dotprompt/parser/BUILD.bazel b/java/com/google/dotprompt/parser/BUILD.bazel index e2089b2a0..e359d2280 100644 --- a/java/com/google/dotprompt/parser/BUILD.bazel +++ b/java/com/google/dotprompt/parser/BUILD.bazel @@ -21,6 +21,7 @@ java_library( srcs = [ "Parser.java", "Picoschema.java", + "package-info.java", ], visibility = ["//visibility:public"], deps = [ diff --git a/java/com/google/dotprompt/parser/package-info.java b/java/com/google/dotprompt/parser/package-info.java new file mode 100644 index 000000000..923053216 --- /dev/null +++ b/java/com/google/dotprompt/parser/package-info.java @@ -0,0 +1,118 @@ +/* + * Copyright 2025 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ + +/** + * Parsing utilities for Dotprompt templates and Picoschema definitions. + * + *

This package provides the core parsing functionality for Dotprompt, including YAML frontmatter + * extraction, template body parsing, Picoschema-to-JSON-Schema conversion, and rendered string + * processing into structured messages. + * + *

Parsing Pipeline

+ * + *
+ *  ┌─────────────────────────────────────────────────────────────────────┐
+ *  │                      PROMPT SOURCE FILE                             │
+ *  │  ---                                                                │
+ *  │  model: gemini-pro                                                  │
+ *  │  input:                                                             │
+ *  │    schema:                                                          │
+ *  │      name: string                                                   │
+ *  │  ---                                                                │
+ *  │  Hello {{name}}!                                                    │
+ *  └─────────────────────────────────────────────────────────────────────┘
+ *                                    │
+ *                                    ▼
+ *  ┌─────────────────────────────────────────────────────────────────────┐
+ *  │                   Parser.parse(source)                              │
+ *  │                                                                     │
+ *  │   1. Extract YAML frontmatter  ───▶  config map                     │
+ *  │   2. Extract template body     ───▶  template string                │
+ *  │   3. Convert Picoschema        ───▶  JSON Schema (if present)       │
+ *  └─────────────────────────────────────────────────────────────────────┘
+ *                                    │
+ *                                    ▼
+ *  ┌─────────────────────────────────────────────────────────────────────┐
+ *  │                        Prompt                                       │
+ *  │   template: "Hello {{name}}!"                                       │
+ *  │   config: {model: "gemini-pro", input: {schema: {...}}}             │
+ *  └─────────────────────────────────────────────────────────────────────┘
+ * 
+ * + *

Key Components

+ * + *

Parser

+ * + *

The {@link com.google.dotprompt.parser.Parser} class provides: + * + *

+ * + *

Picoschema

+ * + *

The {@link com.google.dotprompt.parser.Picoschema} class converts the compact Picoschema + * format to full JSON Schema: + * + *

+ *  Picoschema (compact)              JSON Schema (expanded)
+ *  ─────────────────────             ─────────────────────────────
+ *  name: string                 ───▶  {"type": "object",
+ *  age?: integer                       "properties": {
+ *  tags(array): string                   "name": {"type": "string"},
+ *                                        "age": {"type": "integer"},
+ *                                        "tags": {"type": "array",
+ *                                                 "items": {"type": "string"}}
+ *                                      },
+ *                                      "required": ["name", "tags"]}
+ * 
+ * + *

Marker Processing

+ * + *

Helpers emit special marker strings that the parser processes: + * + *

+ *  ┌───────────────────────────┬────────────────────────────────────┐
+ *  │ Marker                    │ Effect                             │
+ *  ├───────────────────────────┼────────────────────────────────────┤
+ *  │ <<<dotprompt:role:X>>>   │ Start new message with role X      │
+ *  │ <<<dotprompt:history>>>  │ Insert history messages            │
+ *  │ <<<dotprompt:media:url X>>> │ Insert media part                │
+ *  │ <<<dotprompt:section X>>>│ Create named section               │
+ *  └───────────────────────────┴────────────────────────────────────┘
+ * 
+ * + *

Usage Example

+ * + *
{@code
+ * // Parse a prompt from source
+ * String source = "---\nmodel: gemini-pro\n---\nHello {{name}}!";
+ * Prompt prompt = Parser.parse(source);
+ *
+ * // Convert Picoschema to JSON Schema
+ * Map picoschema = Map.of("name", "string", "age?", "integer");
+ * Map jsonSchema = Picoschema.picoschemaToJsonSchema(picoschema, null);
+ * }
+ * + * @see com.google.dotprompt.parser.Parser + * @see com.google.dotprompt.parser.Picoschema + */ +package com.google.dotprompt.parser; diff --git a/java/com/google/dotprompt/resolvers/BUILD.bazel b/java/com/google/dotprompt/resolvers/BUILD.bazel index f2605eea0..deef133b9 100644 --- a/java/com/google/dotprompt/resolvers/BUILD.bazel +++ b/java/com/google/dotprompt/resolvers/BUILD.bazel @@ -22,6 +22,7 @@ java_library( "PartialResolver.java", "SchemaResolver.java", "ToolResolver.java", + "package-info.java", ], visibility = ["//visibility:public"], deps = [ diff --git a/java/com/google/dotprompt/resolvers/package-info.java b/java/com/google/dotprompt/resolvers/package-info.java new file mode 100644 index 000000000..12cbb679a --- /dev/null +++ b/java/com/google/dotprompt/resolvers/package-info.java @@ -0,0 +1,112 @@ +/* + * Copyright 2025 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * SPDX-License-Identifier: Apache-2.0 + */ + +/** + * Resolver interfaces for dynamic content loading in Dotprompt. + * + *

This package defines functional interfaces for resolving external resources during prompt + * compilation and rendering. Resolvers enable dynamic loading of partials, schemas, and tools from + * various sources such as databases, file systems, or remote APIs. + * + *

Resolver Architecture

+ * + *
+ *                           ┌─────────────────────┐
+ *                           │     Dotprompt       │
+ *                           │                     │
+ *                           │  - partialResolver  │
+ *                           │  - schemaResolver   │
+ *                           │  - toolResolver     │
+ *                           └─────────┬───────────┘
+ *                                     │
+ *          ┌──────────────────────────┼──────────────────────────┐
+ *          │                          │                          │
+ *          ▼                          ▼                          ▼
+ *  ┌───────────────┐        ┌─────────────────┐        ┌─────────────────┐
+ *  │PartialResolver│        │ SchemaResolver  │        │  ToolResolver   │
+ *  │               │        │                 │        │                 │
+ *  │ name ─▶ source│        │ name ─▶ schema  │        │ name ─▶ tool    │
+ *  └───────┬───────┘        └────────┬────────┘        └────────┬────────┘
+ *          │                         │                          │
+ *          ▼                         ▼                          ▼
+ *  ┌───────────────┐        ┌─────────────────┐        ┌─────────────────┐
+ *  │  File System  │        │  Schema Store   │        │  Tool Registry  │
+ *  │  Database     │        │  Database       │        │  API Gateway    │
+ *  │  Remote API   │        │  Remote API     │        │  etc.           │
+ *  └───────────────┘        └─────────────────┘        └─────────────────┘
+ * 
+ * + *

Available Resolvers

+ * + *
+ *  ┌────────────────────┬────────────────────────────────────────────────┐
+ *  │ Resolver           │ Purpose                                        │
+ *  ├────────────────────┼────────────────────────────────────────────────┤
+ *  │ PartialResolver    │ Resolve partial templates by name              │
+ *  │ SchemaResolver     │ Resolve JSON schemas by name                   │
+ *  │ ToolResolver       │ Resolve tool definitions by name               │
+ *  └────────────────────┴────────────────────────────────────────────────┘
+ * 
+ * + *

PartialResolver

+ * + *

{@link com.google.dotprompt.resolvers.PartialResolver} resolves Handlebars partials by name. + * When a template uses {@code {{> partialName}}}, the resolver fetches the partial source. + * + *

{@code
+ * PartialResolver resolver = name -> {
+ *     // Load from database, file, etc.
+ *     return loadPartialFromDatabase(name);
+ * };
+ * dotprompt.setPartialResolver(resolver);
+ * }
+ * + *

SchemaResolver

+ * + *

{@link com.google.dotprompt.resolvers.SchemaResolver} resolves JSON schema definitions by + * name. Used when templates reference schemas like {@code input.schema: MySchema}. + * + *

{@code
+ * SchemaResolver resolver = name -> {
+ *     return schemaRegistry.get(name);
+ * };
+ * dotprompt.setSchemaResolver(resolver);
+ * }
+ * + *

ToolResolver

+ * + *

{@link com.google.dotprompt.resolvers.ToolResolver} resolves tool definitions by name. Used + * when templates specify tools for function calling. + * + *

{@code
+ * ToolResolver resolver = name -> {
+ *     return toolRegistry.getDefinition(name);
+ * };
+ * dotprompt.setToolResolver(resolver);
+ * }
+ * + *

Async Support

+ * + *

All resolvers return {@link java.util.concurrent.CompletableFuture} to support asynchronous + * loading from remote sources without blocking the rendering pipeline. + * + * @see com.google.dotprompt.resolvers.PartialResolver + * @see com.google.dotprompt.resolvers.SchemaResolver + * @see com.google.dotprompt.resolvers.ToolResolver + */ +package com.google.dotprompt.resolvers; diff --git a/java/com/google/dotprompt/store/BUILD.bazel b/java/com/google/dotprompt/store/BUILD.bazel index 8ab8603c0..7f70045dd 100644 --- a/java/com/google/dotprompt/store/BUILD.bazel +++ b/java/com/google/dotprompt/store/BUILD.bazel @@ -27,6 +27,7 @@ java_library( "PromptStoreWritable.java", "PromptStoreWritableSync.java", "StoreUtils.java", + "package-info.java", ], visibility = ["//visibility:public"], deps = [