Support for configuring the model

[hazzlim]

Hi,

Really awesome tool! I had one suggestion - it would be great to be able to configure which model is used, for example GPT-4, as this doesn't seem to currently be possible. Apologies if I'm wrong about this - I just had a cursory look at the code and it seemed like GPT-3 is hard coded.

Cheers

[breadchris]

No you are right, gpt4 support is not fully implemented yet! I have some code started for it, but I can whip that out 👍

[hazzlim]

Cheers dude! I’m completely inexperienced in Go, but I’m keen to get to grips with it and make some contributions to the project

[hazzlim]

Ah in fact - looks like the project you are using for OpenAI API interaction, PullRequestInc/go-gpt3, only supports the old “engine” endpoints and not the new “model” endpoints - perhaps the code you started is migrating to a different project / interfacing with the APIs directly?

[breadchris]

oh interesting catch, I am not familiar with that distinction. Do you have a link that explains the difference/how I can implement it? From a brief search I can't see anything, I can dive into the API docs later, but maybe you can save me some time :)

[hazzlim]

So the engines API was deprecated in favour of models - which to be fair seems to largely be a naming difference in the endpoint URL paths. From what I read - and from my testing - it seems as though the old endpoints still work and accept the newer models.

Looks like PullRequestInc/go-gpt3 are still using the old endpoints - although there is a PR to address this, it looks like it may have been abandoned mid-review. Either way - the upside is that it looks like the model can simply be specified in the ChatCompletionRequest, for example the below:

lunapipe/internal/openai.go

Lines 93 to 98 in d1450e0

	req := gpt3.ChatCompletionRequest{
	Temperature: float32(0),
	MaxTokens: respTokenCount,
	Stream: stream,
	Messages: chatCtx,
	}

can be modified to:

 req := gpt3.ChatCompletionRequest{ 
        Model: model_string,
 	Temperature: float32(0), 
 	MaxTokens:   respTokenCount, 
 	Stream:      stream, 
 	Messages:    chatCtx, 
 } 

where model string is the name of any of the models supporting the /v1/chat/completions endpoint (documented here).

Again, thanks for working on this super cool project! I should really learn some go ... I got a bit stumped for a second there with some "slice literal" related errors haha.

[hazzlim]

Also, its somewhat amusing to me to be using a module called "gpt3" to access gpt-4 models... I feel as though they should have named that project something a bit more future-proof 😆

[breadchris]

awesome, thanks for the deets. Unlucky that the PR looks stale. Perhaps it would be worthwhile to use a more actively maintained client like https://github.com/sashabaranov/go-openai

[breadchris]

You can run lunapipe -m gpt-4 "Read values from map" now, as well as stick it in your config:

openai:
  api_key: xxx
  timeout: 5m0s
  model: gpt-4

[hazzlim]

Nice one! Thanks for getting that in so quickly :)