Add support for string content in messages

[ricardog]

Some local servers (like ml-explore/ mlx_lm server) which are based on huggingface / transformers don't seem to support message where content has a type and "data".

For example the following generates an error

{
   "messages": [
      {
         "role": "user",
         "content": [
            {
               "type": "text",
               "text": "who is mickey mouse?"
            }
         ]
      }
   ]
}

While this format works

{
   "messages": [
      {
         "role": "user",
         "content":  "who is mickey mouse?"
      }
   ]
}

Is it possible to have chatgpt-shell use the simpler format?

See for example 1195.

[xenodium]

Hi Ricardo. It may be possible, but I need to check that I don't regress other models or other functionality in the package.

Can you share how you're setting chatgpt-shell to use mlx? I'm not familiar with it.

[ricardog]

Hi Alvaro --

Thanks for looking at this issue.

You need to install the mlx-lm python package and then run the server. For me this looks something like this

pyenv virtualenv 3.12.4 mlx
pip install -U pip setup tools wheel
pip install mlx-lm

Then start the server.

mlx_lm.server

And I configure chatgpt-shell as follows

(setq chatgpt-shell-models
      (list 
       (chatgpt-shell-openai-make-model
	:version "/Users/ricardog/tmp/mlx/llama3-8b"
	:short-version "llama3.1:8b"
	:token-width 4
	:context-window 32768
	)
       (chatgpt-shell-openai-make-model
	:version "/Users/ricardog/tmp/mlx/deepseek-v2"
	:short-version "deepseek:v2"
	:token-width 4
	:context-window 8192
	)       
       (chatgpt-shell-openai-make-model
	:version "mlx-community/Mistral-Nemo-Instruct-2407-4bit"
	:short-version "nemo:2407"
	:token-width 4
	:context-window 16384
	)       
       ))

Please don't put too much attention to the actual configuration values because I don't know what I'm doing. The server is supposed to be able to take a relative path for the model name (relative to where you started the server) but I've found I need to specify the full path.
Running models locally with mlx works really well (if you have a Mac).

I thought about adding a new model type to chatgpt-shell but it seems like a lot of repeated code for a relatively small difference.