Skip to content

[bug] agent_fs and fs tool not returning the content of a file #1362

@sinedied

Description

@sinedied

When experimenting with more "ambitious" agentic refactoring, it seems I hit an issue with the agent_fs implementation. For context, I'm using gpt-4o-mini as the model.

This is my script:

script({
  tools: "agent_fs"
});

$`## Role
You're an expert JavaScript developer. You are going to refactor a JavaScript file.

## Task
1. Find all **/*.genai.js files in the workspace. When querying the content of the single file, ask "Give me the content of the file and nothing else" and provide the file path.
2. For each file, check that it has a "script({})" call at the top of the file. Create one if it doesn't exist.
3. Update the "script({})" call to include the "group" property, and add a short name based on the script's purpose. Group names for related scripts should be the same.
4. Do no change anything else in the file.`;

First, I have to explicitly tell the full glob Find all **/*.genai.js files, as stating only Find all .genai.js files would return nothing. I would expect the agent to be a bit more smarter, probably that this could be fixed in its prompt.

Now trying to run the script, it fails looping indefinitely as it cannot retrieve the content of the files. Something the content of the file gets returned in a fenced block, most of the time it only returns a description of its content and if the query is not a full file path it's erroring (which is frequent, again that may be a prompt issue? query example sent to the agent by the LLM: {"query": "Give me the content of the file and nothing else: genaisrc/refactor.genai.js"}).

I tried switching to using the "fs" tool directly, but this time it only return partial content. It seems the LLM is not aware on how to get the full content of the file, so maybe something's missing in the tool description?

Then using the tools fs_read_file and fs_find_files worked, though I got this result, while I did not hit the context limit:

It appears that the content of each **/*.genai.js file is lengthy, and due to the restrictions in the environment, I can only process one file at a time. If you want me to proceed with refactoring any specific file, please specify which one you'd like to focus on first.

I think workflows like these have a huge potential for GenAIScript, though currently it's not working as simply as expected. I'll continue exploring more, and I suspect a dedicated "refactoring" agent specialized in modifying one file at a time would work better here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions