Skip to content

Create proper testing package #73

@pawbana

Description

@pawbana

Currently AIBridge tests suffer from large amount of copy-paste and not well fitting helper functions that needs to be extended for new tests.
This leads to tests being less and less readable / understandable / harder to write. Such complexity slows down development and can even lead to bugs (https://github.com/coder/aibridge/pull/72)

For example most tests follows the same setup steps, setting up mock server based on fixture files + streaming option, executes request from fixture then verify results. Server setup and performing request could be generalized.
Access to data that needs verification could be better structured. Instead of having multiple recorders/accumulators one place providing this data would be much simpler to use.

Examples of tech debt / code becoming too complicated by gluing more stuff:

Now that more tests were implemented there are clear patterns that could be simplified.

Some examples (more thorough investigation is needed):

  • create dedicated structs based on fixture for:
    • upstream server (provider)
      • could provide request creation instead of createOpenAIChatCompletionsReq / createAnthropicMessagesReq
    • MCP server
    • AIB server?
  • clean up too many return values from helper functions
    • move callAccumulator into mock MCP server struct
    • move setupMCPServerProxiesForTest and return values into MCP server struct
    • More to be found
  • Create testing package containing all testing helpers, maybe it could also be used in https://github.com/coder/coder/blob/pb/aibridge-otel-trace/enterprise/aibridged/aibridged_integration_test.go

  • Align on fixture files format (maybe hardcoded instead of txtar files?)
    • separate for streaming / non streaming
    • multiple responses
    • expected requests (with headers? what about path?)?
    • multiple requests?
    • provider?
    • response headers / status?
  • Implement / clean up mock upstream
  • Implement / clean up mock MCP
    • fix initialization flake:
    • init MCP client: failed to send initialized notification: failed to send request: failed to send request: Post "http://127.0.0.1:33481": net/http: HTTP/1.x transport connection broken: http: CloseIdleConnections called
      
  • Add NewRequestBridge helper?
    • Adds all?/only needed providers?
    • Sets recorder, metrics, tracer with verification helper

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions