[Feature Request] BYLLM in-memory image request #2827
Description
Feature Description
Currently, when we use by_llm to make calls that involve images, we have to first pass the path string to the Image class.
Example:
llm = Model("model_name")
@by(llm)
def classify_document(image: Image) -> str:
...
img = Image("sample.png")
result = classify_document(img)This assumes that we're always working with images that are stored on disk. However, it may be necessary to work with images that are read from memory, e.g. using by_llm as part of a pipeline.
For example, lets say that we're converting from a pdf to an image, e.g.
doc = pymupdf.Document(pdf_path)
idx = 0
final_results = []
for page in doc:
mat = pymupdf.Matrix(dpi / 72, dpi / 72)
pix = page.get_pixmap(matrix=mat, alpha=False) # type: ignore
img_data = pix.tobytes("png")
img = PIL.Image.open(io.BytesIO(img_data))
buffer = io.BytesIO()
img.save(buffer, format="PNG")
# Pass the bytes buffer to the by_llm Image class here
result = classify_document(Image(buffer))
final_results.append(results)It would be great if the Image class can be used as a pass through for images stored in memory when needed.
Metadata
Metadata
Assignees
Type
Projects
Status