org-ql-select with custom predicate does not refresh cache between searches

[claudiusmueller]

OS/platform

openSUSE Leap 15.6

Emacs version and provenance

Emacs 29.4 - installed through openSUSE repository

Emacs command

KRunner -> "emacs"

Org version and provenance

org version 9.6.15 - builtin to emacs

org-ql package version and provenance

20240222.200

Actions taken

Before meetings I like to pull in topics I need to discuss with meeting attendees. I capture topics with a "topic" tag and a TARGET property that describes who this topic needs to be discussed with. See below:

** TODO topic 1 for Susie                                            :topic:
:PROPERTIES:
:OWNER:
:TARGET: Susie
:CREATED: [2025-01-12 Sun]
:END:
** TODO topic 2 for Bob                                              :topic:
:PROPERTIES:
:OWNER:
:TARGET: Bob
:CREATED: [2025-01-12 Sun]
:END:
** TODO topic 1 for Bob                                              :topic:
:PROPERTIES:
:OWNER:
:TARGET: Bob
:CREATED: [2025-01-12 Sun]
:END:

I wrote a custom org-ql predicate to search for items where the TARGET property matches a name:

  (org-ql-defpred target (name)
    "Search for entries where the \"TARGET\" property matches the supplied name."
    :body (and (property "TARGET")
               (string-match name (org-entry-get (point) "TARGET"))))

I then use this predicate in a simple function (simplified for a reprex here):

  (defun my-dupl-test-example ()
    "Collect open topics for a list of people and insert as headers by person."
    (interactive)

    ;; list of example persons
    (dolist (name (list "Bob" "Susie"))
      (insert (format "* %s\n" name))

      ;; get topics for a person
      (let ((results (org-ql-select (org-agenda-files)
                       '(and (target name) (tags "topic") (not (done)))
                       :action '(let ((heading (org-get-heading t t t t))
                                      (position (line-number-at-pos)))
                                  (if heading
                                      (let ((file (buffer-file-name)))
                                        (format "%s" heading)))))))

        ;; insert topics as headers
        (dolist (headline results)
          (insert (format "** %s\n" headline))))))

Observed results

The function inserts Bob's topics for both names IF the topics are found in a separate org file and the org-ql-search scope is set to "org-agenda-files":

* Bob
** topic 2 for Bob
** topic 1 for Bob
* Susie
** topic 2 for Bob
** topic 1 for Bob

If the topics are found in the same org file where they are supposed to be inserted and the org-ql-search scope is set to "current-buffer" it works correctly:

* Bob
** topic 2 for Bob
** topic 1 for Bob
* Susie
** topic 1 for Susie

I am completely guessing here that the items for Bob were returned both times because of a caching issue between searches. That does not explain why it works when the search is within current buffer though. I'm really at a loss of what to do about it or anything else I could test to find the source of the problem.

Expected results

The result should be:

* Bob
** topic 2 for Bob
** topic 1 for Bob
* Susie
** topic 1 for Susie

Backtrace

No response

Etc.

No response

[alphapapa]

Hello,

Before debugging your code, I can't help but notice that your scenario:

Before meetings I like to pull in topics I need to discuss with meeting attendees. I capture topics with a "topic" tag and a TARGET property that describes who this topic needs to be discussed with.

...is virtually identical to that of the custom predicates tutorial:

Imagine you have weekly meetings with other people, and during the week you take notes about items to discuss at each meeting. When the time for a meeting comes, you want to quickly and easily search for all of the items to discuss at the meeting. ... You’ve tried using a :person: NAME property on entries...

And that your predicate is essentially the same as the first example in the tutorial:

(org-ql-defpred person (name)
  "Search for entries with the \"person\" property being NAME."
  :body (property "person" name))

...but with superfluous code:

(org-ql-defpred target (name)
    "Search for entries where the \"TARGET\" property matches the supplied name."
    :body (and (property "TARGET")
               (string-match name (org-entry-get (point) "TARGET"))))

So my first suggestion would be to read that tutorial and follow the code examples there as closely as possible.

If you do all that and still can reproduce a problem, you'll need to reproduce it in a clean Emacs configuration, e.g. using https://github.com/alphapapa/with-emacs.sh and provide a recipe that works in a clean environment.

[alphapapa]

By the way, a problem in your code appears to be that you have quoted the query expression, rather than backquoting it and unquoting the name variable, so the name from the dolist is not inserted into the query expression. See (info "(elisp) Backquote").

[claudiusmueller]

This solved the problem! I need to get back into elisp basics to avoid mistakes like this. Thank you for taking the time to help. I truly appreciate it. Org-ql remains a game changer that constantly tempts me to improve my workflows for project and team management. I just have to be careful to walk before I run.

[alphapapa]

You're welcome. I'm glad to hear that Org QL is helpful to you.