Skip to content

AgentMemory: support nested multimodal data #141

@duf59

Description

@duf59

Relates to #137, in particular this comment. AgentMemory does not serialize multimodal content (Image, PDF, Audio) so that Instructor can handle it. However we do not support it if the multimodal content is provided in a nested input schema as follows:

class Document(BaseIOSchema):
    """PDF document with owner."""

    pdf: PDF = Field(..., description="The PDF data") 
    owner: str = Field(..., description="The PDF owner")


class InputSchema(BaseIOSchema):
    """A list of documents to analyze."""

    documents: list[Document] = Field(..., description="List of documents")

In the above, the pdf will be serialized using json.dumps, which might lead to issues as discussed in #131.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions