Skip to content

Add instructions against common AI poor code quality habits#718

Open
jaredoconnell wants to merge 1 commit intovllm-project:mainfrom
jaredoconnell:docs/ai-quality-pitfalls
Open

Add instructions against common AI poor code quality habits#718
jaredoconnell wants to merge 1 commit intovllm-project:mainfrom
jaredoconnell:docs/ai-quality-pitfalls

Conversation

@jaredoconnell
Copy link
Copy Markdown
Collaborator

Summary

AI bots very frequently tend to import things inline and use getattr. But we have always found that to be a bad design, so this documents AI chatbots not to do these things.

Details

  • Inline imports are not ideal from a code style and maintainability standpoint
  • getattr is frequently used by AI to speculatively use values that don't exist on a type, reducing quality of the code.

  • "I certify that all code in this PR is my own, except as noted below."

Use of AI

  • Includes AI-assisted code completion
  • Includes code generated by an AI application
  • Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes ## WRITTEN BY AI ##)

Signed-off-by: Jared O'Connell <joconnel@redhat.com>
Copy link
Copy Markdown
Collaborator

@dbutenhof dbutenhof left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of accumulating protections and advice as we go, so this is a fantastic precedent. We need to be cautious about the wording, so I'm expressing some concerns.

Comment thread AGENTS.md
- All tests must pass before committing
- Markdown files must be properly formatted
- Public functions in `src/` code must use the reStructuredText docstring format
- All imports must be done at the top of the file unless necessary for functionality
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we be more specific? I think the only time we legitimately do that is to narrow the scope of optional extra packages; so maybe even go very specific like "unless necessary to minimize the scope of optional imports listed under the pyproject.toml "project.optional-dependencies" section."

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make this more broad, not more specific. If you give AI an excuse to ignore a rule it will twist its logic to match.

Suggested change
- All imports must be done at the top of the file unless necessary for functionality
- All imports **SHALL** be done at the top of the file

Comment thread AGENTS.md
- Markdown files must be properly formatted
- Public functions in `src/` code must use the reStructuredText docstring format
- All imports must be done at the top of the file unless necessary for functionality
- Use of `getattr` should be avoided if possible as it hides incorrect usage of types
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use getattr quite a bit -- again, I wonder if we can be more specific here. I kinda worry about how an over-eager-to-please LLM might choose to interpret "if possible". I'm not really coming up with a better condition right now, but this feels like a plot hole ... 😆

Copy link
Copy Markdown
Collaborator

@sjmonson sjmonson May 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is general guidance for AIs, not humans. If a human wants to override this guidance for a good reason they are free to do so.

Suggested change
- Use of `getattr` should be avoided if possible as it hides incorrect usage of types
- **DO NOT** use `getattr` or `setattr` as it hides incorrect usage of types

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern is that an over-eager AI assistant will notice & decide to "fix" existing instances. Ah well -- we can burn that bridge when we reach it.

Copy link
Copy Markdown
Collaborator

@sjmonson sjmonson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please restructure the sections a bit as we will probably add more guidance in the future.

### Quality Requirements

- All Python code must pass linting and formatting
- All Python code must pass type checking
- All tests must pass before committing
- Markdown files must be properly formatted

### Style Requirements

- Public functions in `src/` code must use the reStructuredText docstring format
- All imports must be done at the top of the file unless necessary for functionality
- Use of `getattr` should be avoided if possible as it hides incorrect usage of types

Also see wording suggestions below.

Comment thread AGENTS.md
- All tests must pass before committing
- Markdown files must be properly formatted
- Public functions in `src/` code must use the reStructuredText docstring format
- All imports must be done at the top of the file unless necessary for functionality
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make this more broad, not more specific. If you give AI an excuse to ignore a rule it will twist its logic to match.

Suggested change
- All imports must be done at the top of the file unless necessary for functionality
- All imports **SHALL** be done at the top of the file

Comment thread AGENTS.md
- Markdown files must be properly formatted
- Public functions in `src/` code must use the reStructuredText docstring format
- All imports must be done at the top of the file unless necessary for functionality
- Use of `getattr` should be avoided if possible as it hides incorrect usage of types
Copy link
Copy Markdown
Collaborator

@sjmonson sjmonson May 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is general guidance for AIs, not humans. If a human wants to override this guidance for a good reason they are free to do so.

Suggested change
- Use of `getattr` should be avoided if possible as it hides incorrect usage of types
- **DO NOT** use `getattr` or `setattr` as it hides incorrect usage of types

Copy link
Copy Markdown
Collaborator

@dbutenhof dbutenhof left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been playing with my GitHub PM tool, and I added an AGENTS.md to be sure it checked & fixed all formatting & tests before "finishing". I found it's lazy about keeping the rules in context. It suggested adding a section at the beginning, and that seems to be working (so far), so we should consider something like this:

## Read this file when starting or resuming work

- **Open `AGENTS.md` again** when you begin a task on this repo or return after a long gap, so required checks (below) stay in context until all required checks are green.

Comment thread AGENTS.md
- Markdown files must be properly formatted
- Public functions in `src/` code must use the reStructuredText docstring format
- All imports must be done at the top of the file unless necessary for functionality
- Use of `getattr` should be avoided if possible as it hides incorrect usage of types
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern is that an over-eager AI assistant will notice & decide to "fix" existing instances. Ah well -- we can burn that bridge when we reach it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants