Assistants API Integration #539
Replies: 10 comments 47 replies
-
|
Support for the assistants API is not currently planned, as this is OpenAI only and I prefer to work on features that work across the board. Moving this to a discussion in case people have suggestions or thoughts. |
Beta Was this translation helpful? Give feedback.
-
|
I was playing around with setting this up. Here's my current minimal implementation of using the assistant API. My understanding is that the gptel-request API is built with single-step calls in mind. As pointed out above, the assistant API involves multiple steps. I am working around that by wrapping multiple calls to gptel-requests with different backends for each endpoint. Note that, I have created an assistant on the openai dashboard and using the id here. (defvar-local gptel-openai-assistant-thread-id nil)
(cl-defstruct (gptel-openai-assistant
(:constructor gptel-openai--make-assistant-step)
(:copier nil)
(:include gptel-openai))
step)
(cl-defmethod gptel--parse-response ((backend gptel-openai-assistant) response info)
(pcase (gptel-openai-assistant-step backend)
(:threads
(with-current-buffer (plist-get info :buffer)
(setq-local gptel-openai-assistant-thread-id (plist-get response :id))))
(:messages
"")
(:runs
;; TODO: peridically poll the run and check if it's completed. If so, get last message
;; https://platform.openai.com/docs/assistants/deep-dive#polling-for-updates
(error "not implemented yet!"))
(_ (error "Unknown step"))))
(cl-defmethod gptel--request-data ((backend gptel-openai-assistant) prompts)
(pcase (gptel-openai-assistant-step backend)
(:threads nil)
(:messages
(list :role "user"
:content `[(:type "text" :text ,(plist-get (car (last prompts)) :content))]))
(:runs
(let ((prompts-plist
;; Using cached assistant-id
`(:assistant_id ,(gethash 'openai-assistant-id configurations)
:stream ,(or (and gptel-stream gptel-use-curl
(gptel-backend-stream gptel-backend))
:json-false))))
(when gptel-temperature
(plist-put prompts-plist :temperature gptel-temperature))
(when gptel-max-tokens
(plist-put prompts-plist (if (memq gptel-model '(o1-preview o1-mini))
:max_completion_tokens :max_tokens)
gptel-max-tokens))
;; Merge request params with model and backend params.
(gptel--merge-plists
prompts-plist
(gptel-backend-request-params gptel-backend)
(gptel--model-request-params gptel-model))))
(_ (error "Unknown step"))))
(cl-defmethod gptel-curl--parse-stream ((backend gptel-openai-assistant) info)
(pcase (gptel-openai-assistant-step backend)
(:threads
(user-error "no streaming happening here!"))
(:messages
(user-error "no streaming happening here!"))
(:runs
(let* ((content-strs))
(condition-case nil
(while (re-search-forward "^data:" nil t)
(save-match-data
(unless (looking-at " *\\[DONE\\]")
;; TODO: Handle annotations
(when-let* ((response (gptel--json-read))
(content (map-nested-elt
response '(:delta :content 0 :text :value))))
(push content content-strs)))))
(error
(goto-char (match-beginning 0))))
(apply #'concat (nreverse content-strs))))))
(setq gptel-openai-assistant--threads-start-backend
(gptel-openai--make-assistant-step
:name "gptel-openai-assistant--threads-start"
:step :threads
:host "api.openai.com"
:key 'gptel-api-key
:stream nil
:header (lambda () (when-let (key (gptel--get-api-key))
`(("Authorization" . ,(concat "Bearer " key))
("OpenAI-Beta" . "assistants=v2"))))
:protocol "https"
:endpoint "/v1/threads"
:models (gptel--process-models gptel--openai-models)
:url (concat "https://api.openai.com/v1/threads")))
(setq gptel-openai-assistant--add-message-backend
(gptel-openai--make-assistant-step
:name "gptel-openai-assistant--add-message"
:step :messages
:host "api.openai.com"
:key 'gptel-api-key
:stream nil
:header (lambda () (when-let (key (gptel--get-api-key))
`(("Authorization" . ,(concat "Bearer " key))
("OpenAI-Beta" . "assistants=v2"))))
:protocol "https"
:endpoint "/v1/threads/thread_id/messages"
:models (gptel--process-models gptel--openai-models)
:url (lambda ()
(if gptel-openai-assistant-thread-id
(format "https://api.openai.com/v1/threads/%s/messages" gptel-openai-assistant-thread-id)
(user-error "No thread in current buffer to add messages!")))))
(setq gptel-openai-assistant--runs-backend
(gptel-openai--make-assistant-step
:name "gptel-openai-assistant--runs"
:step :runs
:host "api.openai.com"
:key 'gptel-api-key
:header (lambda () (when-let (key (gptel--get-api-key))
`(("Authorization" . ,(concat "Bearer " key))
("OpenAI-Beta" . "assistants=v2"))))
:protocol "https"
:endpoint "/v1/threads/thread_id/runs"
:models (gptel--process-models gptel--openai-models)
:stream t
:url (lambda ()
(if gptel-openai-assistant-thread-id
(format "https://api.openai.com/v1/threads/%s/runs" gptel-openai-assistant-thread-id)
(user-error "No thread in current buffer to start run!")))))
(defun gptel-openai-assistant-start-thread (&optional callback)
"Start a assistant thread in the current buffer."
(let ((gptel-backend gptel-openai-assistant--threads-start-backend))
(funcall (if gptel-use-curl
#'gptel-curl-get-response #'gptel--url-get-response)
`(:data ,(gptel--request-data gptel-openai-assistant--threads-start-backend nil)
:buffer ,(current-buffer))
callback)))
(defun gptel-openai-assistant-add-messages (&optional callback)
"Add a message to the current thread in the buffer."
(let ((gptel-backend gptel-openai-assistant--add-message-backend))
(gptel-request nil :stream nil :callback callback)))
(defun gptel-openai-assistant-run ()
"Start a run on the thread in the current buffer."
;; TODO: should messages already in the thread be filtered out?
(let ((gptel-backend gptel-openai-assistant--runs-backend))
(gptel-request nil :stream t)))
(defun gptel-openai-assistant-send ()
"Add message to the current thread and run it. Create thread if one is not initialized."
(interactive)
(let (prompt position)
(cl-labels ((send-message-and-run (&optional _ _)
(gptel-openai-assistant-add-messages (lambda (_ _)
(gptel-openai-assistant-run)))))
(if gptel-openai-assistant-thread-id
(send-message-and-run)
(gptel-openai-assistant-start-thread #'send-message-and-run)))))
|
Beta Was this translation helpful? Give feedback.
-
|
Wow that's clever. Nice!
Since I'm working on adding tool-use to gptel (see the
Screenshot of the request context inspector now built-into gptel: While none of the tool-use code changes the request backend dynamically, all properties of the context are mutable. There are no backward-incompatible changes so your above solution should continue to work. In fact I found and fixed several subtle state-management bugs in the main branch. But if you're interested in working on this feature further, I suggest using |
Beta Was this translation helpful? Give feedback.
-
|
The individual components are documented but the package might be missing some "big picture" explanation of how the state machine driven control flow works -- the kind of thing that's better served by a quick pencil sketch. If you need help feel free to ask here. |
Beta Was this translation helpful? Give feedback.
-
|
Oh, a couple more things, for when you dive into the code:
This might help "load" the shape of the fsm into your head, hopefully making it easier to write your own control flow for the assistants API. |
Beta Was this translation helpful? Give feedback.
-
|
Appreciate the clarifications. I just saw this, and figured this part out the hard way 😅 Since #514 is focused on tool-use, and this thread is more focused on fsm, I'll share some initial thoughts on how I am going to go about this here:
|
Beta Was this translation helpful? Give feedback.
-
As you suggest, it would be cleaner to have a single assistant gptel-backend and change the url depending on the state. You can do this via your state handler as follows:
(defun gptel-openai-assistant--handle-FOO (fsm) ;handler for state FOO
(let* ((info (gptel-fsm-info fsm))
(backend (plist-get info :backend)))
;; Change the backend url to whatever it next needs to be
(setf (gptel-backend-url backend) "api.openai.com/...")
;; Do your other updates here, possibly modifying INFO
;; Only if you need to move to the next state right away:
(gptel--fsm-transition fsm)
;; Otherwise the next call to `gptel-openai-assistant-send' etc should
;; transition the fsm forward.
))Feel free to make liberal use of |
Beta Was this translation helpful? Give feedback.
-
|
Ah, nice! Thanx for the suggestions man. I'll try this and see where it goes. |
Beta Was this translation helpful? Give feedback.
-
|
So my above solution only works if I had the fix I had discussed in #545 😅 Anyhow I ended up reimplementing everything with fsm in mind: Here's the current implementation I have;; This gets created when a new session/thread is started
(cl-defstruct (gptel-openai-assistant-session
(:constructor gptel-openai--make-assistant-session)
(:copier nil)
(:include gptel-openai))
(steps
'(:threads :messages :runs)
:documentation "The sequence of the steps.")
(step-idx -1 :documentation "Current step in steps sequence.")
cached-prompts
(step
:threads
:documentation "The current step of the session."))
(defvar gptel-openai-assistant--known-steps nil)
(cl-defstruct (gptel-openai-assistant-step
(:constructor gptel-openai-assistant--make-step))
(url
:documentation "String of function that returns the url as string")
(request-data-fn
:documentation "Takes backend and prompts as arg. See `gptel--request-data'")
(parse-response-fn
:documentation "Takes response and info as args. See `gptel--parse-response'")
(parse-stream-fn
:documentation "Takes info as args. See `gptel-curl--parse-stream'"))
(cl-defun gptel-openai-assistant--add-known-step (step &key url request-data-fn parse-response-fn parse-stream-fn)
(setf (alist-get step gptel-openai-assistant--known-steps)
(gptel-openai-assistant--make-step
:url url :request-data-fn request-data-fn :parse-response-fn parse-response-fn :parse-stream-fn parse-stream-fn)))
(gptel-openai-assistant--add-known-step
:threads
:url "https://api.openai.com/v1/threads"
:parse-response-fn (lambda (response info)
(with-current-buffer (plist-get info :buffer)
(setq-local gptel-openai-assistant-thread-id (plist-get response :id))
nil)))
(gptel-openai-assistant--add-known-step
:messages
:url (lambda ()
(if gptel-openai-assistant-thread-id
(format "https://api.openai.com/v1/threads/%s/messages" gptel-openai-assistant-thread-id)
(user-error "No thread in current buffer to add messages!")))
:request-data-fn (lambda (backend prompts)
(list :role "user"
:content
`[(:type "text"
:text ,(plist-get (car (last prompts)) :content))])))
(gptel-openai-assistant--add-known-step
:runs
:url (lambda ()
(if gptel-openai-assistant-thread-id
(format "https://api.openai.com/v1/threads/%s/runs" gptel-openai-assistant-thread-id)
(user-error "No thread in current buffer to add messages!")))
:request-data-fn (lambda (backend prompts)
(let ((_ (em gptel-stream gptel-use-curl (gptel-backend-stream backend) 333333333333))
(prompts-plist
`(:assistant_id ,(gethash 'openai-assistant-id configurations)
:stream ,(or (and gptel-stream gptel-use-curl
(gptel-backend-stream backend))
:json-false))))
(when gptel-temperature
(plist-put prompts-plist :temperature gptel-temperature))
(when gptel-max-tokens
(plist-put prompts-plist (if (memq gptel-model '(o1-preview o1-mini))
:max_completion_tokens :max_tokens)
gptel-max-tokens))
;; Merge request params with model and backend params.
(gptel--merge-plists
prompts-plist
(gptel-backend-request-params backend)
(gptel--model-request-params gptel-model))))
:parse-response-fn (lambda (response info)
;; TODO: peridically poll the run and check if it's completed. If so, get last message
;; https://platform.openai.com/docs/assistants/deep-dive#polling-for-updates
(error "not implemented yet!"))
:parse-stream-fn (lambda (info)
(let* ((content-strs))
(condition-case nil
(while (re-search-forward "^data:" nil t)
(save-match-data
(if (looking-at " *\\[DONE\\]")
;; Going to next step.....
(gptel-openai-assistant--update-session-step info)
(when-let* ((response (gptel--json-read))
(content (map-nested-elt
response '(:delta :content 0 :text :value)))
)
(push content content-strs)
(when-let (annotations (map-nested-elt
response '(:delta :content 0 :text :annotations)))
(cl-loop for annotation in annotations
for file-id = (map-nested-elt
annotation '(:file_citation :file_id))
if file-id do (push (format "(%s)" file-id) content-strs)))))))
(error
(goto-char (match-beginning 0))))
(apply #'concat (nreverse content-strs)))))
(defun gptel-openai-assistant--update-session-step (info &optional backend)
(cl-assert (or info backend))
(let* ((backend (or backend
(plist-get info :backend)))
(new-step
(when-let* ((step-idx (cl-incf (gptel-openai-assistant-session-step-idx backend)))
(_ (length> (gptel-openai-assistant-session-steps backend) step-idx)))
(nth step-idx (gptel-openai-assistant-session-steps backend))))
(target-buffer (or (and info
(plist-get info :buffer))
;; When being called first time, assuming it is
;; done from the current-buffer
(current-buffer)))
stream)
(setf
(gptel-openai-assistant-session-step backend) new-step
(gptel-backend-url backend) (with-current-buffer target-buffer
(when new-step
(if-let (step (alist-get new-step gptel-openai-assistant--known-steps))
(let ((url (em (gptel-openai-assistant-step-url step) new-step)))
(cl-typecase url
(string url)
(function (funcall url))
(t (error "Unknown value for url (%s) in step (%s)" url new-step))))
(error "Unknown step %s" new-step))))
;; This doesn't currently work unless I have set the process sentinel/fileter functions
;; stream (pcase new-step
;; (:threads nil)
;; (:messages nil)
;; (:runs t)))
)
(when new-step
(plist-put info :data (gptel--request-data backend (gptel-openai-assistant-session-cached-prompts backend))))
(when info
(plist-put info :wait new-step)
;; ;; KLUDGE: copied from `gptel-request'. Can this be managed upstream?
;; (plist-put info :stream (and stream
;; ;; TODO: model parameters?
;; gptel-stream gptel-use-curl
;; (gptel-backend-stream backend)))
;; KLUDGE: copied from `gptel-curl-get-response'. Can this be managed upstream?
;; (plist-put info :parser (cl--generic-method-function
;; (if stream
;; (cl-find-method
;; 'gptel-curl--parse-stream nil
;; (list (aref backend 0) t))
;; (cl-find-method
;; 'gptel--parse-response nil
;; (list (aref backend 0) t t)))))
)))
(cl-defmethod gptel--request-data ((backend gptel-openai-assistant-session) prompts)
(when (eq 0 (gptel-openai-assistant-session-step-idx backend))
(setf (gptel-openai-assistant-session-cached-prompts backend) prompts))
(let* ((step (gptel-openai-assistant-session-step backend))
(step-info (alist-get step gptel-openai-assistant--known-steps)))
(if step-info
(when-let (fn (gptel-openai-assistant-step-request-data-fn step-info))
(funcall fn backend prompts))
(error "Unknown step %s" step-info))))
(cl-defmethod gptel--parse-response ((_ gptel-openai-assistant-session) response info)
(prog1
(let* ((step (gptel-openai-assistant-session-step (plist-get info :backend)))
(step-info (alist-get step gptel-openai-assistant--known-steps)))
(if step-info
(when-let (fn (gptel-openai-assistant-step-parse-response-fn step-info))
(funcall fn response info))
(error "Unknown step %s" step)))
(gptel-openai-assistant--update-session-step info)))
(cl-defmethod gptel-curl--parse-stream ((_ gptel-openai-assistant-session) info)
(let* ((step (gptel-openai-assistant-session-step (plist-get info :backend)))
(step-info (alist-get step gptel-openai-assistant--known-steps)))
(if step-info
(when-let (fn (gptel-openai-assistant-step-parse-stream-fn step-info))
(funcall fn info))
(error "Unknown step %s" new-step))))
(defun gptel-openai-assistant-make-session (steps &optional stream)
(gptel-openai--make-assistant-session
:name "gptel-openai-assistant-session"
:step (car steps)
:steps steps
:host "api.openai.com"
:key 'gptel-api-key
:header (lambda () (when-let (key (gptel--get-api-key))
`(("Authorization" . ,(concat "Bearer " key))
("OpenAI-Beta" . "assistants=v2"))))
:protocol "https"
:endpoint "/v1/threads/thread_id/runs"
:stream stream))
(defun gptel-openai-assistant-start-thread ()
"Start a assistant thread in the current buffer."
(interactive)
;; The info hasn't yet been created either.
;; FIXME: is this needed?
(setq-local gptel-backend (gptel-openai-assistant-make-session '(:threads)))
(gptel-openai-assistant--update-session-step nil gptel-backend)
(gptel-request nil :stream nil))
(defun gptel-openai-assistant-add-message ()
"Start a assistant thread in the current buffer."
(interactive)
;; The info hasn't yet been created either.
;; FIXME: is this needed?
(setq-local gptel-backend (gptel-openai-assistant-make-session '(:messages)))
(gptel-openai-assistant--update-session-step nil gptel-backend)
(gptel-request nil :stream nil))
(defun gptel-openai-assistant-start-run ()
"Start a assistant thread in the current buffer."
(interactive)
;; The info hasn't yet been created either.
;; FIXME: is this needed?
(setq-local gptel-backend (gptel-openai-assistant-make-session '(:runs) t))
(gptel-openai-assistant--update-session-step nil gptel-backend)
(gptel-request nil :stream t))
(defun gptel-openai-assistant-send ()
"Add message to the current thread and run it. Create thread if one is not initialized."
(interactive)
(setq-local gptel-backend (gptel-openai-assistant-make-session '(:threads :messages)))
(gptel-openai-assistant--update-session-step nil gptel-backend)
(gptel-request nil :stream t))What have I done:
Earlier, I had used the callbacks to chain the method, which works fine as the fsm is started anew for each step (i.e., a new fsm for each of So either I should figure out how to mix streaming and non-streaming, or implement all steps for streaming/non-streaming. I am not entirely sure how to do the former. When doing the latter:
|
Beta Was this translation helpful? Give feedback.
-
|
Well, implementing the streaming based option turned out much simpler. Here's the full working code with some bug fixes on the above: EDIT: missed pre-amble (defun gptel--wait-again-p (info)
"Check if the fsm should WAIT."
(plist-get info :wait))
(defun gptel--delay-p (info)
"Test fsm transition to DELAY."
(plist-get info :delay))
(defun gptel-handle-on-wait-callback (fsm)
"Handle on-wait-callback."
(when-let ((callback (plist-get (gptel-fsm-info fsm) :on-wait-callback)))
(funcall callback)))
(defun gptel--handle-delay (fsm)
"Handle the delay state."
(let ((delay (plist-get (gptel-fsm-info fsm) :delay)))
(cl-assert (numberp delay))
(plist-put (gptel-fsm-info fsm) :delay nil)
(run-at-time delay nil (lambda () (gptel--fsm-transition fsm)))))
;; Adding TYPE -> WAIT & TYPE -> DELAY transitions
(setf (alist-get 'TYPE gptel-request--transitions) '((gptel--delay-p . DELAY)
(gptel--error-p . ERRS)
(gptel--tool-use-p . TOOL)
(gptel--wait-again-p . WAIT)
(t . DONE)))
;; MAYBE: this is useful to be connected to other states as well?
;; From DELAY go to what was expected
(setf (alist-get 'DELAY gptel-request--transitions) '((gptel--error-p . ERRS)
(gptel--tool-use-p . TOOL)
(gptel--wait-again-p . WAIT)
(t . DONE)))
(setf (alist-get 'DELAY gptel-request--handlers) '(gptel--handle-delay))
(setf (alist-get 'WAIT gptel-request--handlers) '(gptel-handle-on-wait-callback gptel--handle-wait))
(defvar-local gptel-openai-assistant-thread-id nil)(cl-defstruct (gptel-openai-assistant-session
(:constructor gptel-openai--make-assistant-session)
(:copier nil)
(:include gptel-openai))
(steps
'(:threads :messages :runs)
:documentation "The sequence of the steps.")
(step-idx -1 :documentation "Current step in steps sequence.")
cached-prompts
(step
:threads
:documentation "The current step of the session."))
(defvar gptel-openai-assistant--known-steps nil)
(cl-defstruct (gptel-openai-assistant-step
(:constructor gptel-openai-assistant--make-step))
(url
:documentation "String of function that returns the url as string")
(request-data-fn
:documentation "Takes backend and prompts as arg. See `gptel--request-data'")
(parse-response-fn
:documentation "Takes response and info as args. See `gptel--parse-response'")
(parse-stream-fn
:documentation "Takes info as args. See `gptel-curl--parse-stream'"))
(cl-defun gptel-openai-assistant--add-known-step (step &key url request-data-fn parse-response-fn parse-stream-fn)
(setf (alist-get step gptel-openai-assistant--known-steps)
(gptel-openai-assistant--make-step
:url url :request-data-fn request-data-fn :parse-response-fn parse-response-fn :parse-stream-fn parse-stream-fn)))
(gptel-openai-assistant--add-known-step
:threads
:url "https://api.openai.com/v1/threads"
:parse-response-fn (lambda (response info)
(with-current-buffer (plist-get info :buffer)
(setq-local gptel-openai-assistant-thread-id (plist-get response :id))
nil))
:parse-stream-fn (lambda (info)
(goto-char (point-min))
(re-search-forward "
?\n
?\n" nil t)
(condition-case err
(when-let* ((response (gptel--json-read))
(_ (equal "thread" (plist-get response :object))))
(gptel-openai-assistant--wait-or-update-step info)
(with-current-buffer (plist-get info :buffer)
(setq-local gptel-openai-assistant-thread-id (plist-get response :id)))
nil)
(json-parse-error
(goto-char (match-beginning 0)))
(error
(signal (car err) (cdr err))))))
(gptel-openai-assistant--add-known-step
:messages
:url (lambda ()
(if gptel-openai-assistant-thread-id
(format "https://api.openai.com/v1/threads/%s/messages" gptel-openai-assistant-thread-id)
(user-error "No thread in current buffer to add messages!")))
:request-data-fn (lambda (backend prompts)
(list :role "user"
:content
`[(:type "text"
:text ,(plist-get (car (last prompts)) :content))]))
:parse-stream-fn (lambda (info)
(goto-char (point-min))
(re-search-forward "
?\n
?\n" nil t)
(condition-case nil
(when-let* ((response (gptel--json-read))
(_ (equal "thread.message" (plist-get response :object))))
(gptel-openai-assistant--wait-or-update-step info)
nil)
(json-parse-error
(goto-char (match-beginning 0)))
(error
(signal (car err) (cdr err))))))
(gptel-openai-assistant--add-known-step
:runs
:url (lambda ()
(if gptel-openai-assistant-thread-id
(format "https://api.openai.com/v1/threads/%s/runs" gptel-openai-assistant-thread-id)
(user-error "No thread in current buffer to add messages!")))
:request-data-fn (lambda (backend prompts)
(let ((prompts-plist
`(:assistant_id ,(gethash 'openai-assistant-id configurations)
:stream ,(or (and gptel-stream gptel-use-curl
(gptel-backend-stream backend))
:json-false))))
(when gptel-temperature
(plist-put prompts-plist :temperature gptel-temperature))
(when gptel-max-tokens
(plist-put prompts-plist (if (memq gptel-model '(o1-preview o1-mini))
:max_completion_tokens :max_tokens)
gptel-max-tokens))
;; Merge request params with model and backend params.
(gptel--merge-plists
prompts-plist
(gptel-backend-request-params backend)
(gptel--model-request-params gptel-model))))
:parse-response-fn (lambda (response info)
;; TODO: peridically poll the run and check if it's completed. If so, get last message
;; https://platform.openai.com/docs/assistants/deep-dive#polling-for-updates
(error "not implemented yet!"))
:parse-stream-fn (lambda (info)
(let* ((content-strs))
(condition-case err
(while (re-search-forward "^data:" nil t)
(save-match-data
(if (looking-at " *\\[DONE\\]")
(prog1 nil
(gptel-openai-assistant--wait-or-update-step info))
(when-let* ((response (gptel--json-read))
(content (map-nested-elt
response '(:delta :content 0 :text :value)))
)
(push content content-strs)
(when-let (annotations (map-nested-elt
response '(:delta :content 0 :text :annotations)))
(cl-loop for annotation in annotations
for file-id = (map-nested-elt
annotation '(:file_citation :file_id))
if file-id do (push (format "(%s)" file-id) content-strs)))))))
((json-parse-error json-end-of-file)
(goto-char (match-beginning 0)))
(error
(signal (car err) (cdr err))))
(apply #'concat (nreverse content-strs)))))
(defun gptel-openai-assistant--wait-or-update-step (info)
(if (length>
(gptel-openai-assistant-session-steps (plist-get info :backend))
(1+ (gptel-openai-assistant-session-step-idx (plist-get info :backend))))
(progn
(plist-put info :wait t)
(plist-put info :on-wait-callback
(lambda ()
(gptel-openai-assistant--update-session-step info))))
(plist-put info :wait nil)))
(defun gptel-openai-assistant--update-session-step (info &optional backend)
(cl-assert (or info backend))
(let* ((backend (or backend
(plist-get info :backend)))
(new-step
(when-let* ((step-idx (cl-incf (gptel-openai-assistant-session-step-idx backend)))
(_ (length> (gptel-openai-assistant-session-steps backend) step-idx)))
(nth step-idx (gptel-openai-assistant-session-steps backend))))
(target-buffer (or (and info
(plist-get info :buffer))
;; When being called first time, assuming it is
;; done from the current-buffer
(current-buffer)))
stream)
(setf
(gptel-openai-assistant-session-step backend) new-step
(gptel-backend-url backend) (with-current-buffer target-buffer
(when new-step
(if-let (step (alist-get new-step gptel-openai-assistant--known-steps))
(let ((url (gptel-openai-assistant-step-url step)))
(cl-typecase url
(string url)
(function (funcall url))
(t (error "Unknown value for url (%s) in step (%s)" url new-step))))
(error "Unknown step %s" new-step))))
;; This doesn't currently work unless I have set the process sentinel/fileter functions
;; stream (pcase new-step
;; (:threads nil)
;; (:messages nil)
;; (:runs t)))
)
(when new-step
(plist-put info :data (gptel--request-data backend (gptel-openai-assistant-session-cached-prompts backend))))
(when info
(plist-put info :wait new-step)
(plist-put info :on-wait-callback nil)
;; ;; KLUDGE: copied from `gptel-request'. Can this be managed upstream?
;; (plist-put info :stream (and stream
;; ;; TODO: model parameters?
;; gptel-stream gptel-use-curl
;; (gptel-backend-stream backend)))
;; KLUDGE: copied from `gptel-curl-get-response'. Can this be managed upstream?
;; (plist-put info :parser (cl--generic-method-function
;; (if stream
;; (cl-find-method
;; 'gptel-curl--parse-stream nil
;; (list (aref backend 0) t))
;; (cl-find-method
;; 'gptel--parse-response nil
;; (list (aref backend 0) t t)))))
)))
(cl-defmethod gptel--request-data ((backend gptel-openai-assistant-session) prompts)
(when (eq 0 (gptel-openai-assistant-session-step-idx backend))
(setf (gptel-openai-assistant-session-cached-prompts backend) prompts))
(let* ((step (gptel-openai-assistant-session-step backend))
(step-info (alist-get step gptel-openai-assistant--known-steps)))
(if step-info
(when-let (fn (gptel-openai-assistant-step-request-data-fn step-info))
(funcall fn backend prompts))
(error "Unknown step %s" step-info))))
(cl-defmethod gptel--parse-response ((_ gptel-openai-assistant-session) response info)
(prog1
(let* ((step (gptel-openai-assistant-session-step (plist-get info :backend)))
(step-info (alist-get step gptel-openai-assistant--known-steps)))
(if step-info
(when-let (fn (gptel-openai-assistant-step-parse-response-fn step-info))
(funcall fn response info))
(error "Unknown step %s" step)))
(gptel-openai-assistant--update-session-step info)))
(cl-defmethod gptel-curl--parse-stream ((_ gptel-openai-assistant-session) info)
(let* ((step (gptel-openai-assistant-session-step (plist-get info :backend)))
(step-info (alist-get step gptel-openai-assistant--known-steps)))
(if step-info
(when-let (fn (gptel-openai-assistant-step-parse-stream-fn step-info))
(funcall fn info))
(error "Unknown step %s" new-step))))
(defun gptel-openai-assistant-make-session (steps &optional stream)
(gptel-openai--make-assistant-session
:name "gptel-openai-assistant-session"
:step (car steps)
:steps steps
:host "api.openai.com"
:key 'gptel-api-key
:header (lambda () (when-let (key (gptel--get-api-key))
`(("Authorization" . ,(concat "Bearer " key))
("OpenAI-Beta" . "assistants=v2"))))
:protocol "https"
:endpoint "/v1/threads/thread_id/runs"
:stream stream))
(defun gptel-openai-assistant-start-thread ()
"Start a assistant thread in the current buffer."
(interactive)
;; The info hasn't yet been created either.
;; FIXME: is this needed?
(when (or (not gptel-openai-assistant-thread-id)
(y-or-n-p "There is already a thread in the buffer. Create a new thread? "))
(setq-local gptel-backend (gptel-openai-assistant-make-session '(:threads)))
(gptel-openai-assistant--update-session-step nil gptel-backend)
(gptel-request nil :stream nil)))
(defun gptel-openai-assistant-add-message ()
"Start a assistant thread in the current buffer."
(interactive)
;; The info hasn't yet been created either.
;; FIXME: is this needed?
(setq-local gptel-backend (gptel-openai-assistant-make-session '(:messages)))
(gptel-openai-assistant--update-session-step nil gptel-backend)
(gptel-request nil :stream nil))
(defun gptel-openai-assistant-start-run ()
"Start a assistant thread in the current buffer."
(interactive)
;; The info hasn't yet been created either.
;; FIXME: is this needed?
(setq-local gptel-backend (gptel-openai-assistant-make-session '(:runs) t))
(gptel-openai-assistant--update-session-step nil gptel-backend)
(gptel-request nil :stream t))
(defun gptel-openai-assistant-send ()
"Add message to the current thread and run it. Create thread if one is not initialized."
(interactive)
(setq-local gptel-backend (gptel-openai-assistant-make-session
(if gptel-openai-assistant-thread-id
'(:messages :runs)
'(:threads :messages :runs))
t))
(gptel-openai-assistant--update-session-step nil gptel-backend)
(gptel-request nil :stream t)) |
Beta Was this translation helpful? Give feedback.
-
|
Thanx for taking a look at this. The easiest way I found to understand what is going on is to use the openai's playground. It shows the calls being made and the responses in the logs.
When there are new messages/prompts, steps (2) and (3) are repeated. You are right, this went a bit out of hand 😅 In terms of the state management, what I have here can be moved up the fsm. What I have here is to define a sequence of steps/states as opposed to defining transitions through a table. For example in |
Beta Was this translation helpful? Give feedback.
-
|
EDIT: I missed some of the pre-able where I am modifying the table and handlers a bit, added them in the above post. |
Beta Was this translation helpful? Give feedback.
-
|
I think I understand the basics of the assistants API now. I'm trying to understand your code. What is |
Beta Was this translation helpful? Give feedback.
-
|
One thing I noticed is that you're creating a new backend for each message or run, like in (setq-local gptel-backend (gptel-openai-assistant-make-session '(:messages)))Is there some reason you can't (re)use a single backend for this? |
Beta Was this translation helpful? Give feedback.
-
That's my way of storing and a bunch of kv pairs. I should have scrubbed that out when I posted it here 😅
With the code I have here, I can't. I need to reset the internal state of the backend before I start the next request. I am working on trying to get that part to work. |
Beta Was this translation helpful? Give feedback.
-
|
so, yet another implementation. What changed:
Some setup for by adding states to the fsm: (require 'gptel)
(defun gptel--wait-again-p (info)
"Check if the fsm should WAIT."
(plist-get info :wait))
(defun gptel--delay-p (info)
"Test fsm transition to DELAY."
(plist-get info :delay))
(defun gptel--await-p (info)
"Test fsm transition to AWAIT."
(plist-get info :await))
(defun gptel-handle-on-wait-again (fsm)
"Handle on-wait-callback."
(plist-put (gptel-fsm-info fsm) :wait nil))
;; Not used right now
(defun gptel-handle-on-wait-callback (fsm)
"Handle on-wait-callback."
(when-let ((callback (plist-get (gptel-fsm-info fsm) :on-wait-callback)))
(funcall callback)
(plist-put (gptel-fsm-info fsm) :on-wait-callback nil)))
(defun gptel--handle-delay (fsm)
"Handle the delay state."
(let ((delay (plist-get (gptel-fsm-info fsm) :delay)))
(cl-assert (numberp delay))
(plist-put (gptel-fsm-info fsm) :delay nil)
(run-at-time delay nil (lambda () (gptel--fsm-transition fsm)))))
;; Adding TYPE -> WAIT & TYPE -> DELAY transitions
(setf (alist-get 'TYPE gptel-request--transitions) '((gptel--await-p . AWAIT)
(gptel--delay-p . DELAY)
(gptel--error-p . ERRS)
(gptel--tool-use-p . TOOL)
(gptel--wait-again-p . WAIT)
(t . DONE)))
;; Not used right now - but will be useful when trying to poll using non streaming option
;; MAYBE: this is useful to be connected to other states as well?
;; From DELAY go to what was expected
(setf (alist-get 'DELAY gptel-request--transitions) '((gptel--await-p . AWAIT)
(gptel--error-p . ERRS)
(gptel--tool-use-p . TOOL)
(gptel--wait-again-p . WAIT)
(t . DONE)))
;; From AWAIT go to what was expected
(setf (alist-get 'AWAIT gptel-request--transitions)
'((gptel--delay-p . DELAY)
(gptel--error-p . ERRS)
(gptel--tool-use-p . TOOL)
(gptel--wait-again-p . WAIT)
(t . DONE)))
(push '(gptel--await-p . AWAIT) (alist-get 'INIT gptel-request--transitions))
(push '(gptel--await-p . AWAIT) (alist-get 'WAIT gptel-request--transitions))
(push '(gptel--await-p . AWAIT) (alist-get 'TOOL gptel-request--transitions))
(setf (alist-get 'DELAY gptel-request--handlers) '(gptel--handle-delay))
(setf (alist-get 'WAIT gptel-request--handlers) '(gptel-handle-on-wait-callback gptel-handle-on-wait-again gptel--handle-wait))
(cl-defgeneric gptel-backend--on-start-of-state (backend state info)
"When a the gptel fsm starts a new STATE, this will be called.
This method is called before the handlers of the new state are invoked.")
(cl-defmethod gptel-backend--on-start-of-state ((backend gptel-backend) state info))
(cl-defgeneric gptel-backend--on-end-of-state (backend state info)
"When a the gptel fsm transitions to a new state, this will be called with the old STATE.")
(cl-defmethod gptel-backend--on-end-of-state ((backend gptel-backend) state info))
(defun gptel--fsm-transition-handle-backend-on-start-on-end (oldfn fsm &optional new-state)
"Advice for `gptel--fsm-transition'.
Invokes the `gptel-backend--on-start-of-state' & `gptel-backend--on-end-of-state'."
(let* ((info (gptel-fsm-info fsm))
(backend (plist-get info :backend)))
(gptel-backend--on-end-of-state backend (gptel-fsm-state fsm) info)
;; the start is being called before the handlers of the next state..
(gptel-backend--on-start-of-state backend (gptel--fsm-next fsm) info)
(funcall oldfn fsm new-state)))
(advice-add #'gptel--fsm-transition :around #'gptel--fsm-transition-handle-backend-on-start-on-end)Implementation of openai-assistant: (defvar-local gptel-openai-assistant-thread-id nil)
;; TODO: Get this from api?
(defvar-local gptel-openai-assistant-assistant-id <assistant id copied from openai dashboard>)
(cl-defstruct (gptel-openai-assistant
(:constructor gptel-openai--make-assistant)
(:copier nil)
(:include gptel-openai))
messages-data)
(defvar url-http-end-of-headers)
(defun gptel--openai-assistant-url-retrive (method data url info callback)
"Get data from URL with DATA using METHOD (POST/GET).
INFO is info from gptel
CALLBACK is called with the response from calling url-retrive."
(let ((url-request-method "POST")
(url-request-extra-headers
(append `(("Content-Type" . "application/json")
("Authorization" . ,(concat "Bearer " (gptel--get-api-key)))
("OpenAI-Beta" . "assistants=v2"))))
(url-request-data data))
(url-retrieve (cl-typecase url
(string url)
(function (funcall url))
(t (error "Unknown value for url (%s)" url)))
(lambda (_)
(pcase-let ((`(,response ,http-status ,http-msg ,error)
(custom--url-parse-response))
(buf (current-buffer)))
(plist-put info :http-status http-status)
(plist-put info :status http-msg)
(when error
(plist-put info :error error))
(with-current-buffer (plist-get info :buffer)
(funcall callback response))
(kill-buffer buf)
))
nil t t)))
;; copied from `gptel--url-parse-response' beacuse we don't want the following:
;; (gptel--parse-response backend response proc-info)
(defun custom--url-parse-response ()
"Parse response from url-retrive."
(when gptel-log-level ;logging
(save-excursion
(goto-char url-http-end-of-headers)
(when (eq gptel-log-level 'debug)
(gptel--log (gptel--json-encode (buffer-substring-no-properties (point-min) (point)))
"response headers"))
(gptel--log (buffer-substring-no-properties (point) (point-max))
"response body")))
(if-let* ((http-msg (string-trim (buffer-substring (line-beginning-position)
(line-end-position))))
(http-status
(save-match-data
(and (string-match "HTTP/[.0-9]+ +\\([0-9]+\\)" http-msg)
(match-string 1 http-msg))))
(response (progn (goto-char url-http-end-of-headers)
(condition-case nil
(gptel--json-read)
(error 'json-read-error)))))
(cond
;; FIXME Handle the case where HTTP 100 is followed by HTTP (not 200) BUG #194
((or (memq url-http-response-status '(200 100))
(string-match-p "\\(?:1\\|2\\)00 OK" http-msg))
(list response
http-status http-msg))
((plist-get response :error)
(list nil http-status http-msg (plist-get response :error)))
((eq response 'json-read-error)
(list nil http-status (concat "(" http-msg ") Malformed JSON in response.") "json-read-error"))
(t (list nil http-status (concat "(" http-msg ") Could not parse HTTP response.")
"Could not parse HTTP response.")))
(list nil (concat "(" http-msg ") Could not parse HTTP response.")
"Could not parse HTTP response.")))
(defun gptel-openai-assistant-start-thread (info &optional callback)
"Use the threads endpoint to start new thread.
Set the `gptel-openai-assistant-thread-id' of the buffer.
INFO is the info plist from gptel.
CALLBACK is invoked without any args after successfully creating a thread."
(gptel--openai-assistant-url-retrive
"POST" nil "https://api.openai.com/v1/threads"
info
(lambda (response)
(with-current-buffer
(plist-get info :buffer)
(setq-local gptel-openai-assistant-thread-id (plist-get response :id))
(when callback (funcall callback))))))
(defun gptel-openai-assistant-add-message (info callback)
"Use the messages endpoint to start new thread.
Needs the `gptel-openai-assistant-thread-id' of the buffer to be set.
INFO is the info plist from gptel.
CALLBACK is invoked without any args after successfully creating a thread."
(gptel--openai-assistant-url-retrive
"POST"
(encode-coding-string
(gptel--json-encode (gptel-openai-assistant-messages-data (plist-get info :backend)))
'utf-8)
(with-current-buffer
(plist-get info :buffer)
(if gptel-openai-assistant-thread-id
(format "https://api.openai.com/v1/threads/%s/messages"
gptel-openai-assistant-thread-id)
(user-error "No thread in current buffer to add messages!")))
info
(lambda (response)
(when callback (funcall callback)))))
(cl-defmethod gptel--request-data ((backend gptel-openai-assistant) prompts)
(setf (gptel-openai-assistant-messages-data backend)
(list :role "user"
:content
`[(:type "text"
:text ,(plist-get (car (last prompts)) :content))]))
(let ((prompts-plist
`(:assistant_id ,gptel-openai-assistant-assistant-id
:stream ,(or (and gptel-stream gptel-use-curl
(gptel-backend-stream backend))
:json-false))))
(when gptel-temperature
(plist-put prompts-plist :temperature gptel-temperature))
(when gptel-max-tokens
(plist-put prompts-plist (if (memq gptel-model '(o1-preview o1-mini))
:max_completion_tokens :max_tokens)
gptel-max-tokens))
;; Merge request params with model and backend params.
(gptel--merge-plists
prompts-plist
(gptel-backend-request-params backend)
(gptel--model-request-params gptel-model))))
(cl-defmethod gptel--parse-response ((_ gptel-openai-assistant) response info)
;; TODO: peridically poll the run and check if it's completed. If so, get last message
;; https://platform.openai.com/docs/assistants/deep-dive#polling-for-updates
(error "not implemented yet!"))
(cl-defmethod gptel-curl--parse-stream ((_ gptel-openai-assistant) info)
(let* ((content-strs))
(condition-case err
(while (re-search-forward "^data:" nil t)
(save-match-data
(unless (looking-at " *\\[DONE\\]")
(when-let* ((response (gptel--json-read))
(content (map-nested-elt
response '(:delta :content 0 :text :value)))
)
(if-let* ((annotations (map-nested-elt
response '(:delta :content 0 :text :annotations)))
(_ (length> annotations 0)))
(cl-loop for annotation across annotations
for file-id = (map-nested-elt
annotation '(:file_citation :file_id))
if file-id
do (push (format "[[file_citation:%s][%s]]" file-id content) content-strs))
(push content content-strs))))))
((json-parse-error json-end-of-file search-failed)
(goto-char (match-beginning 0)))
(error
(signal (car err) (cdr err))))
(apply #'concat (nreverse content-strs))))
(cl-defmethod gptel--handle-openai-assistant-await (fsm)
(let* ((info (gptel-fsm-info fsm))
(await-manual-state (plist-get info :await)))
(when (gptel-openai-assistant-p (plist-get info :backend))
;; This tells us await manual is happening from after the init
(pcase await-manual-state
(:send-message
(plist-put info :await nil)
(cl-labels ((send-message ()
(gptel-openai-assistant-add-message
info
(lambda ()
(plist-put info :wait t)
(gptel--fsm-transition fsm)))))
(if gptel-openai-assistant-thread-id
(send-message)
(gptel-openai-assistant-start-thread
info
(lambda ()
(if (plist-get info :error)
(gptel--fsm-transition fsm)
(send-message)))))))
(:await-runs-complete
(plist-put info :await nil)
;; TODO: The polling can happen here
(error "Not implemented yet"))))))
(cl-defmethod gptel-backend--on-end-of-state ((backend gptel-openai-assistant) state info)
(when (eq state 'INIT)
(plist-put info :await :send-message)))
;;;###autoload
(defun gptel-openai-make-assistant ()
"Create a openai-assistant backend."
(gptel-openai--make-assistant
:name "gptel-openai-assistant"
:host "api.openai.com"
:key 'gptel-api-key
:models (gptel--process-models gptel--openai-models)
:header (lambda () (when-let (key (gptel--get-api-key))
`(("Authorization" . ,(concat "Bearer " key))
("OpenAI-Beta" . "assistants=v2"))))
:protocol "https"
:endpoint "/v1/threads/thread_id/runs"
:url (lambda ()
(if gptel-openai-assistant-thread-id
(format "https://api.openai.com/v1/threads/%s/runs" gptel-openai-assistant-thread-id)
(user-error "No thread in current buffer to add messages!")))
:stream t))
;;;###autoload
(defun gptel-openai-assistant-create-new-thread ()
"Create a new thread in the current buffer."
(interactive)
(gptel-openai-assistant-start-thread `(:buffer ,(buffer-name))))
(push 'gptel--handle-openai-assistant-await (alist-get 'AWAIT gptel-request--handlers))
(setf (alist-get "openai-assistant" gptel--known-backends
nil nil #'equal)
(gptel-openai-make-assistant)) |
Beta Was this translation helpful? Give feedback.
-
|
You're partially correct, but perhaps not for the reason you think?
The backend needs to be registered in Everything else just sets independent elisp variables or transient infixes, so it doesn't matter how the request is sent afterwards.
The answer depends on what exactly you're doing. If you use a custom send command (i.e. one that uses custom transitions/handlers, OR does extra processing as in Method 1)It's not clear what you mean by
Case 1:
All the variables you set in the menu will be respected by Case 2:
All the options you set in the menu are respected, but not the extra processing you do in The solution: find a way to replace If you modify
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks for taking the time to provide a detailed response, it is much appreciated.
I hadn't realized that. I'll keep that in mind.
I was referring to
I think this is something the fsm implementation affords. Otherwise, I am not sure this would have been possible without some extensive advising around some of the core functions. In this regard, I think this would be a much preferable way when building new workflows:
In a way, it would provide a standard way to implement a lot of new workflows while using the same UI. But you are right, if I am not careful, I can easily mess up the base flow of
That would be a cool feature to have with gptel. This would allow implementing new workflows without stepping on the global or even other workflows. But off the top of my head, I can see it getting a bit wild. For instance, the fsm and backend may be tied like in my case - where the transition-table and handlers are specific to the backend I am using. This would require some way to assess compatibility between backend and fsm. One naive solution might be to have a method that specializes on the backend to initialize and return an fsm which is used by gptel-request when not passed in as a parameter. But I am not sure what the implications of that are. Either way, if you need contributions to try and get this working, I'd be happy to help.
Yes, this would be the solution to go with for now.
This was why the patch I had shown was at the end of |
Beta Was this translation helpful? Give feedback.
-
Assuming the openai assistant session is tied to a buffer, an easy way to achieve this for the assistant is to just set gptel already sets the backend, model and request parameters buffer-locally anyway. |
Beta Was this translation helpful? Give feedback.
-
|
True. But that also means, that if I switch to a different backend in the same session, those variables would also need to change? |
Beta Was this translation helpful? Give feedback.
-
|
Oh yes, unfortunately. A more bespoke solution is probably required to feed
|
Beta Was this translation helpful? Give feedback.
-
|
So, here's a full working version for both streaming and non-streaming versions. One issue that I ran into with implementing the non-streaming option was with gptel always sending POST requests. However, ideally, the GET method would be used to get the messages after the run has completed. As a workaround I am using the modify messages endpoint where I request an empty modification, which also returns the messages I am looking for. ;; Openai assistant api ******************************************************************
(defvar-local gptel-openai-assistant-thread-id nil)
;; TODO: Get this from api?
(defvar-local gptel-openai-assistant-assistant-id "<assistant id from dashboard>")
;; Helper functions **********************************************************************
(defvar url-http-end-of-headers)
(defun gptel-openai-assistant--url-retrive (method data url info callback)
"Get data from URL with DATA using METHOD (POST/GET).
INFO is info from gptel
CALLBACK is called with the response from calling url-retrive."
(let ((url-request-method method)
(url-request-extra-headers
(append `(("Content-Type" . "application/json")
("Authorization" . ,(concat "Bearer " (gptel--get-api-key)))
("OpenAI-Beta" . "assistants=v2"))))
(url-request-data data))
(when gptel-log-level ;logging
(when (eq gptel-log-level 'debug)
(gptel--log (gptel--json-encode
(mapcar (lambda (pair) (cons (intern (car pair)) (cdr pair)))
url-request-extra-headers))
"request headers"))
(when url-request-data
(gptel--log url-request-data "request body")))
(url-retrieve (cl-typecase url
(string url)
(function (funcall url))
(t (error "Unknown value for url (%s) in step" url)))
(lambda (_)
(pcase-let ((`(,response ,http-status ,http-msg ,error)
(gptel-openai-assistant--url-parse-response))
(buf (current-buffer)))
(plist-put info :http-status http-status)
(plist-put info :status http-msg)
(when error
(plist-put info :error error))
(with-current-buffer (plist-get info :buffer)
(funcall callback response))
(kill-buffer buf)
))
nil t t)))
;; copied from `gptel--url-parse-response' beacuse we don't want the following:
;; (gptel--parse-response backend response proc-info)
(defun gptel-openai-assistant--url-parse-response ()
"Parse response from url-retrive."
(when gptel-log-level ;logging
(save-excursion
(goto-char url-http-end-of-headers)
(when (eq gptel-log-level 'debug)
(gptel--log (gptel--json-encode (buffer-substring-no-properties (point-min) (point)))
"response headers"))
(gptel--log (buffer-substring-no-properties (point) (point-max))
"response body")))
(if-let* ((http-msg (string-trim (buffer-substring (line-beginning-position)
(line-end-position))))
(http-status
(save-match-data
(and (string-match "HTTP/[.0-9]+ +\\([0-9]+\\)" http-msg)
(match-string 1 http-msg))))
(response (progn (goto-char url-http-end-of-headers)
(condition-case nil
(gptel--json-read)
(error 'json-read-error)))))
(cond
;; FIXME Handle the case where HTTP 100 is followed by HTTP (not 200) BUG #194
((or (memq url-http-response-status '(200 100))
(string-match-p "\\(?:1\\|2\\)00 OK" http-msg))
(list response
http-status http-msg))
((plist-get response :error)
(list nil http-status http-msg (plist-get response :error)))
((eq response 'json-read-error)
(list nil http-status (concat "(" http-msg ") Malformed JSON in response.") "json-read-error"))
(t (list nil http-status (concat "(" http-msg ") Could not parse HTTP response.")
"Could not parse HTTP response.")))
(list nil (concat "(" http-msg ") Could not parse HTTP response.")
"Could not parse HTTP response.")))
;; Core function to work with gptel ******************************************************
(cl-defstruct (gptel-openai-assistant
(:constructor gptel-openai--make-assistant)
(:copier nil)
(:include gptel-openai))
messages-data)
(defun gptel-openai-assistant-start-thread (info &optional callback)
"Use the threads endpoint to start new thread.
Set the `gptel-openai-assistant-thread-id' of the buffer.
INFO is the info plist from gptel.
CALLBACK is invoked without any args after successfully creating a thread."
(gptel-openai-assistant--url-retrive
"POST" nil "https://api.openai.com/v1/threads"
info
(lambda (response)
(with-current-buffer
(plist-get info :buffer)
(setq-local gptel-openai-assistant-thread-id (plist-get response :id))
(when callback (funcall callback))))))
(defun gptel-openai-assistant-add-message (info callback)
"Use the messages endpoint to start new thread.
Needs the `gptel-openai-assistant-thread-id' of the buffer to be set.
INFO is the info plist from gptel.
CALLBACK is invoked without any args after successfully creating a thread."
(gptel-openai-assistant--url-retrive
"POST"
(encode-coding-string
(gptel--json-encode (gptel-openai-assistant-messages-data (plist-get info :backend)))
'utf-8)
(with-current-buffer
(plist-get info :buffer)
(if gptel-openai-assistant-thread-id
(format "https://api.openai.com/v1/threads/%s/messages"
gptel-openai-assistant-thread-id)
(user-error "No thread in current buffer to add messages!")))
info
(lambda (response)
(when callback (funcall callback)))))
(cl-defmethod gptel--request-data ((backend gptel-openai-assistant) prompts)
(setf (gptel-openai-assistant-messages-data backend)
(list :role "user"
:content
`[(:type "text"
:text ,(plist-get (car (last prompts)) :content))]))
(if (and gptel-stream gptel-use-curl
(gptel-backend-stream backend))
(let ((prompts-plist
`(:assistant_id ,gptel-openai-assistant-assistant-id
:stream ,(or (and gptel-stream gptel-use-curl
(gptel-backend-stream backend))
:json-false))))
(when gptel-temperature
(plist-put prompts-plist :temperature gptel-temperature))
(when gptel-max-tokens
(plist-put prompts-plist (if (memq gptel-model '(o1-preview o1-mini))
:max_completion_tokens :max_tokens)
gptel-max-tokens))
;; Merge request params with model and backend params.
(gptel--merge-plists
prompts-plist
(gptel-backend-request-params backend)
(gptel--model-request-params gptel-model)))
`(:metadata nil)))
(cl-defmethod gptel--parse-response ((_ gptel-openai-assistant) response info)
(string-join
(mapcar
(lambda (content)
(let ((str (map-nested-elt content '(:text :value))))
(mapcar
(lambda (annotation)
(setf str
(string-replace
(plist-get annotation :text)
(format "[file_citation:%s]" (map-nested-elt annotation '(:file_citation :file_id)))
str)))
(map-nested-elt content '(:text :annotations)))
str))
(plist-get response :content))
" "))
(cl-defmethod gptel-curl--parse-stream ((_ gptel-openai-assistant) info)
(let* ((content-strs))
(condition-case err
(while (re-search-forward "^data:" nil t)
(save-match-data
(unless (looking-at " *\\[DONE\\]")
(when-let* ((response (gptel--json-read))
(content (map-nested-elt
response '(:delta :content 0 :text :value)))
)
(if-let* ((annotations (map-nested-elt
response '(:delta :content 0 :text :annotations)))
(_ (length> annotations 0)))
(cl-loop for annotation across annotations
for file-id = (map-nested-elt
annotation '(:file_citation :file_id))
if file-id
do (push (format "[file_citation:%s]" file-id content) content-strs))
(push content content-strs))))))
((json-parse-error json-end-of-file search-failed)
(goto-char (match-beginning 0)))
(error
(signal (car err) (cdr err))))
(apply #'concat (nreverse content-strs))))
(defun gptel-openai-assistant--handle-await (fsm)
(let* ((info (gptel-fsm-info fsm))
(await-manual-state
(or (plist-get info :openai-assistant-await)
(let ((history (plist-get info :history)))
(when (and
(eq (car history) 'INIT)
(null (cdr history)))
:send-message)))))
(plist-put info :openai-assistant-await nil)
(when (gptel-openai-assistant-p (plist-get info :backend))
(with-current-buffer (plist-get (gptel-fsm-info fsm) :buffer)
(gptel--update-status " Waiting..." 'warning))
;; This tells us await manual is happening from after the init
(pcase await-manual-state
(:send-message
(cl-labels ((send-message ()
(gptel-openai-assistant-add-message
info
(lambda ()
(if (plist-get info :stream)
(plist-put info :openai-assistant-wait t)
(plist-put info :openai-assistant-await :openai-assistant-runs))
(gptel--fsm-transition fsm)))))
(if gptel-openai-assistant-thread-id
(send-message)
(gptel-openai-assistant-start-thread
info
(lambda ()
(if (plist-get info :error)
(gptel--fsm-transition fsm)
(send-message)))))))
(:openai-assistant-runs
(gptel-openai-assistant--url-retrive
"POST"
(encode-coding-string
(gptel--json-encode `(:assistant_id ,gptel-openai-assistant-assistant-id))
'utf-8)
(with-current-buffer
(plist-get info :buffer)
(if gptel-openai-assistant-thread-id
(format "https://api.openai.com/v1/threads/%s/runs"
gptel-openai-assistant-thread-id)
(user-error "No thread in current buffer to add messages!")))
info
(lambda (response)
(unless (plist-get info :error)
(plist-put info :openai-assistant-run-id (plist-get response :id))
(plist-put info :openai-assistant-await :openai-assistant-runs-completed-p)
(plist-put info :openai-assistant-delay 0.5))
(gptel--fsm-transition fsm))))
(:openai-assistant-runs-completed-p
(gptel-openai-assistant--url-retrive
"GET"
nil
(with-current-buffer
(plist-get info :buffer)
(if gptel-openai-assistant-thread-id
(format "https://api.openai.com/v1/threads/%s/runs/%s"
gptel-openai-assistant-thread-id
(plist-get info :openai-assistant-run-id))
(user-error "No thread in current buffer to add messages!")))
info
(lambda (response)
(unless (plist-get info :error)
(pcase (plist-get response :status)
((or "in_progress" "queued")
(plist-put info :openai-assistant-await :openai-assistant-runs-completed-p)
(plist-put info :openai-assistant-delay 0.5))
("completed"
(plist-put info :openai-assistant-await :openai-assistant-list-messages))
(status (error "Unhandle state for run %s" status))))
(gptel--fsm-transition fsm))))
(:openai-assistant-list-messages
(gptel-openai-assistant--url-retrive
"GET"
nil
(with-current-buffer
(plist-get info :buffer)
(if gptel-openai-assistant-thread-id
(format "https://api.openai.com/v1/threads/%s/messages?run_id=%s"
gptel-openai-assistant-thread-id
(plist-get info :openai-assistant-run-id))
(user-error "No thread in current buffer to add messages!")))
info
(lambda (response)
(unless (plist-get info :error)
(plist-put info :openai-assistant-message-id (map-nested-elt response '(:data 0 :id)))
(plist-put info :openai-assistant-wait t))
(gptel--fsm-transition fsm))))
(_ (error "Unknown await state %s" await-manual-state))))))
;;;###autoload
(defun gptel-make-openai-assistant ()
"Create a openai-assistant backend."
(gptel-openai--make-assistant
:name "gptel-openai-assistant"
:host "api.openai.com"
:key 'gptel-api-key
:models (gptel--process-models gptel--openai-models)
:header (lambda () (when-let (key (gptel--get-api-key))
`(("Authorization" . ,(concat "Bearer " key))
("OpenAI-Beta" . "assistants=v2"))))
:protocol "https"
:endpoint "/v1/threads/thread_id/runs"
:url (lambda ()
(if gptel-openai-assistant-thread-id
(format "https://api.openai.com/v1/threads/%s/runs" gptel-openai-assistant-thread-id)
(user-error "No thread in current buffer to add messages!")))
:stream t))
;;;###autoload
(defun gptel-openai-assistant-create-new-thread ()
"Create a new thread in the current buffer."
(interactive)
(gptel-openai-assistant-start-thread `(:buffer ,(buffer-name))))
;; Modify gptel vars *********************************************************************
(setf (alist-get "openai-assistant" gptel--known-backends
nil nil #'equal)
(gptel-make-openai-assistant))
(defun gptel-openai-assistant--backend-is-oaia-p (info)
"Check if backend is openai-assistant."
(gptel-openai-assistant-p (plist-get info :backend)))
(defun gptel-openai-assistant--init-to-await (info)
"If in first INIT, move to AWAIT"
(and (gptel-openai-assistant--backend-is-oaia-p info)
;; If :history is not set or empty, means state is at INIT
(null (plist-get info :history))))
(defun gptel-openai-assistant--wait-again-p (info)
"Check if the fsm should WAIT."
(and (gptel-openai-assistant--backend-is-oaia-p info)
(plist-get info :openai-assistant-wait)))
(defun gptel-openai-assistant--delay-p (info)
"Test fsm transition to DELAY."
(and (gptel-openai-assistant--backend-is-oaia-p info)
(plist-get info :openai-assistant-delay)))
(defun gptel-openai-assistant--await-p (info)
"Test fsm transition to AWAIT."
(and (gptel-openai-assistant--backend-is-oaia-p info)
(plist-get info :openai-assistant-await)))
(defun gptel-openai-assistant--handle-wait (fsm)
"Handle wait-again."
(let ((info (gptel-fsm-info fsm)))
(when (gptel-openai-assistant--backend-is-oaia-p info)
(unless gptel-openai-assistant-thread-id
(user-error "No thread in current buffer to add messages!"))
(plist-put info :openai-assistant-wait nil)
(setf (gptel-backend-url (plist-get info :backend))
(if (plist-get info :stream)
(format "https://api.openai.com/v1/threads/%s/runs" gptel-openai-assistant-thread-id)
;; FIXME: gptel-always sends POST. Ideally, this should be the GET endpoint
;; So using the endpoint for updating messages to get messages related to a run.
(format "https://api.openai.com/v1/threads/%s/messages/%s"
gptel-openai-assistant-thread-id
(plist-get info :openai-assistant-message-id)))))))
(defun gptel-openai-assistant--handle-delay (fsm)
"Handle the delay state."
(let ((info (gptel-fsm-info fsm)))
(and (gptel-openai-assistant--backend-is-oaia-p info)
(let ((delay (plist-get info :openai-assistant-delay)))
(cl-assert (numberp delay))
(plist-put info :openai-assistant-delay nil)
(run-at-time delay nil (lambda () (gptel--fsm-transition fsm)))))))
;; INIT should transition to AWAIT
(push '(gptel-openai-assistant--init-to-await . AWAIT) (alist-get 'INIT gptel-request--transitions))
;; Adding TYPE -> WAIT/DELAY/AWAIT
;; DELAY & AWAIT has to happen at the very beginning
;; WAIT should be tested right before DONE
(setf (alist-get 'TYPE gptel-request--transitions)
(append
'((gptel-openai-assistant--delay-p . DELAY)
(gptel-openai-assistant--await-p . AWAIT))
;; *sigh*
(cl-loop for transition in (alist-get 'TYPE gptel-request--transitions)
;; Insert the wait again, right before DONE
if (eq (cdr transition) 'DONE)
collect '(gptel-openai-assistant--wait-again-p . WAIT) into retval
collect transition into retval
finally return retval)))
;; From DELAY go to what was expected
;; Same as TYPE but without the DELAY itself!
(setf (alist-get 'DELAY gptel-request--transitions) (cl-loop for transition in (alist-get 'TYPE gptel-request--transitions)
unless (eq (cdr transition) 'DELAY)
collect transition))
;; From AWAIT go to what was expected. Same as TYPE
(setf (alist-get 'AWAIT gptel-request--transitions) (alist-get 'TYPE gptel-request--transitions))
;; AWAIT should be tested for first in WAIT and TOOL
(push '(gptel-openai-assistant--await-p . AWAIT) (alist-get 'WAIT gptel-request--transitions))
(push '(gptel-openai-assistant--await-p . AWAIT) (alist-get 'TOOL gptel-request--transitions))
;; Adding DELAY handler
(push '(DELAY gptel-openai-assistant--handle-delay) gptel-request--handlers)
;; Handle the wait-again in WAIT
(push 'gptel-openai-assistant--handle-wait (alist-get 'WAIT gptel-request--handlers))
;; Handle AWAIT
(push '(AWAIT gptel-openai-assistant--handle-await) gptel-request--handlers) |
Beta Was this translation helpful? Give feedback.
-
|
In the above code, I took a longer look at the code today. Still not sure how many of these transitions can occur, like For things like tool-use and file-uploads, which I imagine is most of the reason to use the assistants API, you'll have to re-use |
Beta Was this translation helpful? Give feedback.
-
Right now, these don't happen. I left them in there so that I can handle tool-use/function-calls and code-interpreter as well.
Yes, that is correct. It might be better to rename that
As far as I understand the tool-use structure itself seems to be the same. Where and in which endpoint it is triggered seems to be different. For now, I am exclusively using the assistant API for file-search, I'll play around with the rest when I get some time. |
Beta Was this translation helpful? Give feedback.
-
Now that I think about it, tampering with the |
Beta Was this translation helpful? Give feedback.
-
Okay.
Perhaps. I'll leave it to you.
I forgot to mention -- it is OK to push things onto |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, I was using the |
Beta Was this translation helpful? Give feedback.
-
You can use |
Beta Was this translation helpful? Give feedback.
-
|
That's neat, I'll take a closer look at the newer implementations and see how I use that. |
Beta Was this translation helpful? Give feedback.
-
|
@ahmed-shariff Where are we on the Assistants API integration? Is there anything I need to do? |
Beta Was this translation helpful? Give feedback.
-
I don't yet have an idea of how to supply a transition table to
Yeah, that's one way to solve it. But the way it's currently engineered both |
Beta Was this translation helpful? Give feedback.
-
|
Oh, I think I found a way. Are you okay with using a buffer-local variable to store the fsm/transition table? I think I can pass it through to |
Beta Was this translation helpful? Give feedback.
-
|
That would be similar to the variables that get set based on the scope? If so shouldn't make a difference, right? Since it is all being passed to gptel-request? |
Beta Was this translation helpful? Give feedback.
-
|
Right, this problem again: #539 (reply in thread) I might have confused myself here, I need to think about it more. |
Beta Was this translation helpful? Give feedback.
-
|
To address that would mean either the transition table would need to be aware of the backend or vice versa. i.e., if the table is changed if the backend is not compatible with the table it should error and vice versa? |
Beta Was this translation helpful? Give feedback.
-
|
I've been working on adding function call feature openai assistant. The implementation is currently tucked away in my (very messy) config. I'll putt this out to a seeprate repo when I get some time. So, the good news is open ai decided to stick to the same structure of defining tools. But where and how these calls are seen are different. I had to modify the Another piece to be conscious about is the tool configured for the assistant need to be set in
(cl-defmethod gptel--parse-tool-results ((_ gptel-openai-assistant) tool-use)
"Return a prompt containing tool call results in TOOL-USE."
;; (declare (side-effect-free t))
(mapcar
(lambda (tool-call)
(list
:output (plist-get tool-call :result)
:tool_call_id (plist-get tool-call :id)))
tool-use))
(cl-defmethod gptel--inject-prompt ((_ gptel-openai-assistant) data new-prompt &optional _position)
;; The tool calls are handled differently in oai
;; FIXME: What are the other use cases for inject?
(let ((prompts (plist-get data :tool_outputs)))
(plist-put data :tool_outputs (vconcat prompts new-prompt))))
@@ -195,9 +195,7 @@ information if the stream contains it."
(args (apply #'concat (nreverse (plist-get info :partial_json))))
(func (plist-get (car tool-use) :function)))
(plist-put func :arguments args) ;Update arguments for last recorded tool
- (gptel--inject-prompt
- (plist-get info :backend) (plist-get info :data)
- `(:role "assistant" :content :null :tool_calls ,(vconcat tool-use))) ; :refusal :null
+ (plist-put info :data (list :stream (plist-get info :stream)))
(cl-loop
for tool-call in tool-use ; Construct the call specs for running the function calls
for spec = (plist-get tool-call :function)
@@ -207,13 +205,23 @@ information if the stream contains it."
(plist-get spec :arguments))))
into call-specs
finally (plist-put info :tool-use call-specs)))
- (when-let* ((response (gptel--json-read))
- (delta (map-nested-elt response '(:choices 0 :delta))))
- (if-let* ((content (plist-get delta :content))
+ (when-let* ((response (gptel--json-read)))
+ (when (and (null (plist-get info :openai-assistant-run-id))
+ (string-equal (plist-get response :object) "thread.run"))
+ (plist-put info :openai-assistant-run-id (plist-get response :id)))
+ (if-let* ((content (map-nested-elt response '(:delta :content 0 :text :value)))
((not (eq content :null))))
- (push content content-strs)
+ (if-let* ((annotations (map-nested-elt
+ response '(:delta :content 0 :text :annotations)))
+ (_ (length> annotations 0)))
+ (cl-loop for annotation across annotations
+ for file-id = (map-nested-elt
+ annotation '(:file_citation :file_id))
+ if file-id
+ do (push (format "[file_citation:%s]" file-id content) content-strs))
+ (push content content-strs))
;; No text content, so look for tool calls
- (when-let* ((tool-call (map-nested-elt delta '(:tool_calls 0)))
+ (when-let* ((tool-call (map-nested-elt response '(:delta :step_details :tool_calls 0)))The complete (cl-defmethod gptel-curl--parse-stream ((_ gptel-openai-assistant) info)
(let* ((content-strs))
(condition-case err
(while (re-search-forward "^data:" nil t)
(save-match-data
(if (looking-at " *\\[DONE\\]")
;; The stream has ended, so we do the following thing (if we found tool calls)
;; - pack tool calls into the messages prompts list to send (INFO -> :data -> :messages)
;; - collect tool calls (formatted differently) into (INFO -> :tool-use)
(when-let* ((tool-use (plist-get info :tool-use))
(args (apply #'concat (nreverse (plist-get info :partial_json))))
(func (plist-get (car tool-use) :function)))
(plist-put func :arguments args) ;Update arguments for last recorded tool
(plist-put info :data (list :stream (plist-get info :stream)))
(cl-loop
for tool-call in tool-use ; Construct the call specs for running the function calls
for spec = (plist-get tool-call :function)
collect (list :id (plist-get tool-call :id)
:name (plist-get spec :name)
:args (ignore-errors (gptel--json-read-string
(plist-get spec :arguments))))
into call-specs
finally (plist-put info :tool-use call-specs)))
(when-let* ((response (gptel--json-read)))
(when (and (null (plist-get info :openai-assistant-run-id))
(string-equal (plist-get response :object) "thread.run"))
(plist-put info :openai-assistant-run-id (plist-get response :id)))
(if-let* ((content (map-nested-elt response '(:delta :content 0 :text :value)))
((not (eq content :null))))
(if-let* ((annotations (map-nested-elt
response '(:delta :content 0 :text :annotations)))
(_ (length> annotations 0)))
(cl-loop for annotation across annotations
for file-id = (map-nested-elt
annotation '(:file_citation :file_id))
if file-id
do (push (format "[file_citation:%s]" file-id content) content-strs))
(push content content-strs))
;; No text content, so look for tool calls
(when-let* ((tool-call (map-nested-elt response '(:delta :step_details :tool_calls 0)))
(func (plist-get tool-call :function)))
(if (plist-get func :name) ;new tool block begins
(progn
(when-let* ((partial (plist-get info :partial_json)))
(let* ((prev-tool-call (car (plist-get info :tool-use)))
(prev-func (plist-get prev-tool-call :function)))
(plist-put prev-func :arguments ;update args for old tool block
(apply #'concat (nreverse (plist-get info :partial_json)))))
(plist-put info :partial_json nil)) ;clear out finished chain of partial args
;; Start new chain of partial argument strings
(plist-put info :partial_json (list (plist-get func :arguments)))
;; NOTE: Do NOT use `push' for this, it prepends and we lose the reference
(plist-put info :tool-use (cons tool-call (plist-get info :tool-use))))
;; old tool block continues, so continue collecting arguments in :partial_json
(push (plist-get func :arguments) (plist-get info :partial_json)))))))))
((json-parse-error json-end-of-file search-failed)
(goto-char (match-beginning 0)))
(error
(signal (car err) (cdr err))))
(apply #'concat (nreverse content-strs)))) |
Beta Was this translation helpful? Give feedback.
-
It looks like nobody has proposed any new issues here. Maybe it's the
challenge to bring all those new things into this great emacs ai
package. I'll just bring some basic the things into users attention. I
hope as we further learn the basics of how the Assistant API works, We
can extend our package with even more power.
The documentaion is Here OpenAI Platform
Step 1: Create an Assistant
I don't know if we can set up the assistent setting from the
web interface instead of the Python script. Because what I
found is that on the web interface, we have more easily
control over the things. Ideally, I'm thinking about once we
create a new assistant, we have a unique ID for that system,
and then we can use that ID to communicate the things that we
are interested in.
Step 2: Create a Thread
Step 3: Add a Message to a Thread
Step two and three looks to me is the same thing, which we
kind of start a new section to talk to this assistant. But,
yeah, different from our previous function in this package,
now we have ability to upload files, and in the near future we
can upload images in the coming months.
Step 4: Run the Assistant
Step 5: Display the Assistant's Response
Steps four and five is essentially we are sending our input to
API and we expect data or output from the API and then we
decode the content and properly process within Emacs, either
we do region replacement or we do some magic Org-mode
manipulations.
For me, I need to figure out a minimum elisp function to properly set
up the request information and then gather feedback from the current
API and from there I can probably tweak about the new API.
Beta Was this translation helpful? Give feedback.
All reactions