Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Media type detection not working #17

Open
XInTheDark opened this issue Jun 13, 2024 · 3 comments
Open

Media type detection not working #17

XInTheDark opened this issue Jun 13, 2024 · 3 comments

Comments

@XInTheDark
Copy link

Currently only images are detected correctly. Video formats (like .mov) or plain text (.txt) documents are erroneously rejected.

Steps to reproduce:

  1. let buffer = fs.readFileSync("path/to/any/mov or txt file")
  2. pass buffer to the API as a file upload

The API will return an error saying "Error: Please provide a valid file format that is accepted by Gemini. Learn more about valid formats here: https://ai.google.dev/gemini-api/docs/prompting_with_media?lang=node#supported_file_formats".

Expected behaviour: The API should accept all these video and plaintext formats.

@XInTheDark
Copy link
Author

I have done some debugging and the issue is because fileTypeFromBuffer (used in utils.ts) wrongly returns undefined on txt, mov, etc. files.

@XInTheDark
Copy link
Author

A possible fix would be to fall back to using the file extension if the MIME type not found, using https://www.npmjs.com/package/mime (lite version) which is really compact.

@EvanZhouDev
Copy link
Owner

The issue is, Gemini AI is not in any way aware of the file extension (or even the file name at all, for that matter). This is by design, as Gemini AI was not supposed to read from files directly anyway (as that would require fs, thus involving a Node-only dep, causing browser incompatibility. In the past, we have considered dynamically importing fs, but that was just a mess)... however, what I can do is make it so that the user can manually specify a MIME type.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants