jiri/org-ql
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge: org-ql-defpred

Browse source 
Squashed commit of the following:

commit a0ace307b0de3f5e811f042fd9f0b821a0bbc95c
Author: Adam Porter <adam@alphapapa.net>
Date:   Mon Nov 23 01:11:07 2020 -0600

    Docs: Fix TOC

commit bef9f3c9b5b762070445c76ac3c833e51dd893ba
Author: Adam Porter <adam@alphapapa.net>
Date:   Mon Nov 23 01:09:33 2020 -0600

    Docs: (README.org) Add org-ql-defpred

commit ff0257696fe87764e3289e12ddc336377a9a5d01
Author: Adam Porter <adam@alphapapa.net>
Date:   Mon Nov 23 00:52:06 2020 -0600

    Docs: Improve defpred.org

    Thanks to piyo in #emacsconf@freenode.

commit 18d6d73a99c336cd637c99309ee5d150106b73b7
Author: Adam Porter <adam@alphapapa.net>
Date:   Mon Nov 23 00:35:24 2020 -0600

    Docs: Improve custom predicate tutorial

    Closes #155.  Thanks to Merlin Göttlinger (@mgttlinger) for suggesting
    improvements.

commit b8cbe36fb3
Author: Adam Porter <adam@alphapapa.net>
Date:   Mon Nov 23 00:05:52 2020 -0600

    Docs: Add custom predicate tutorial

commit b1008d0c52
Author: Adam Porter <adam@alphapapa.net>
Date:   Sun Nov 22 19:34:16 2020 -0600

    Docs: Update readme

commit 73d4f95cd4
Merge: 9a2719f 148a193
Author: Adam Porter <adam@alphapapa.net>
Date:   Sun Nov 22 19:29:15 2020 -0600

    Merge branch 'wip/0.6-pre' into wip/define-predicate

...
This commit is contained in:
Adam Porter 2020-11-23 01:13:03 -06:00
parent 148a193ca0
commit 57b3eeba49
9 changed files with 1432 additions and 791 deletions
Download patch file Download diff file Expand all files Collapse all files

64
README.org
View file

@ -258,15 +258,19 @@ The following predicates, in addition to the keyword arguments, can also take a
** Functions / Macros ** Functions / Macros
:PROPERTIES: :PROPERTIES:
:TOC: ignore-children :TOC: :include descendants
:END:
:CONTENTS:
- [[#agenda-like-views][Agenda-like views]]
- [[#function-org-ql-block][Function: org-ql-block]]
- [[#listing--acting-on-results][Listing / acting-on results]]
- [[#caching][Caching]]
- [[#function-org-ql-select][Function: org-ql-select]]
- [[#function-org-ql-query][Function: org-ql-query]]
- [[#macro-org-ql][Macro: org-ql]]
- [[#custom-predicates][Custom predicates]]
- [[#macro-org-ql-defpred][Macro: org-ql-defpred]]
:END: :END:
- [[#agenda-like-views][Agenda-like views]]
- [[#function-org-ql-block][Function: org-ql-block]]
- [[#listing--acting-on-results][Listing / acting-on results]]
- [[#function-org-ql-select][Function: org-ql-select]]
- [[#function-org-ql-query][Function: org-ql-query]]
- [[#macro-org-ql][Macro: org-ql]]
*** Agenda-like views *** Agenda-like views
@ -405,6 +409,44 @@ Examples:
Expands into a call to ~org-ql-select~ with the same arguments. For convenience, arguments should be unquoted. Expands into a call to ~org-ql-select~ with the same arguments. For convenience, arguments should be unquoted.
*** Custom predicates
+ See: [[file:examples/defpred.org][Custom predicate tutorial]]
**** Macro: =org-ql-defpred=
/Arguments:/ ~(name args docstring &key body preambles normalizers)~
Define an ~org-ql~ selector predicate named ~org-ql--predicate-NAME~. ~NAME~ may be a symbol or a list of symbols: if a list, the first is used as ~NAME~ and the rest are aliases. ~A~ function is only created for ~NAME~, not for aliases, so a normalizer should be used to replace aliases with ~NAME~ in queries (keep reading).
~ARGS~ is a ~cl-defun~-style argument list. ~DOCSTRING~ is the function's docstring.
~BODY~ is the body of the predicate. It will be evaluated with point on the beginning of an Org heading and should return non-nil if the heading's entry is a match.
~PREAMBLES~ and ~NORMALIZERS~ are lists of ~pcase~ forms matched against Org ~QL~ query sexps. They are spliced into ~pcase~ forms in the definitions of the functions ~org-ql--query-preamble~ and ~org-ql--normalize-query~, which see. Those functions are redefined when this macro is expanded, unless variable ~org-ql-defpred-defer~ is non-nil, in which case those functions should be redefined manually after defining predicates by calling ~org-ql--define-query-preamble-fn~ and ~org-ql--define-normalize-query-fn~.
~NORMALIZERS~ are used to normalize query expressions to standard forms. For example, when the predicate has aliases, the aliases should be replaced with predicate names using a normalizer. Also, predicate arguments may be put into a more optimal form so that the predicate has less work to do at query time.
~PREAMBLES~ refer to regular expressions which may be used to search through a buffer directly to a potential match rather than testing the predicate body on each heading. (Naming things is hard.) In each ~pcase~ form in ~PREAMBLES~, the ~pcase~ expression (not the pattern) should be a plist with the following keys, each value of which should be an expression which may refer to variables bound in the pattern:
~:regexp~ Regular expression which searches directly to a potential match.
~:case-fold~ Bound to ~case-fold-search~ around the regexp search.
~:query~ Expression which should replace the query expression, or ~query~ if it should not be changed (e.g. if the regexp is insufficient to determine whether a heading matches, in which case the predicate's body needs to be tested on the heading). If the regexp guarantees a match, this may be simply ~t~, leaving the query expression with no work to do, which improves performance.
For convenience, within the ~pcase~ patterns, the symbol ~predicate-names~ is a special form which is replaced with a pattern matching any of the predicate's name and aliases. For example, if ~NAME~ were:
~(heading h)~
Then if ~NORMALIZERS~ were:
~((`(,predicate-names . ,args) `(heading ,@args)))~
It would be expanded to:
~((`(,(or 'heading 'h) . ,args) `(heading ,@args)))~
** Dynamic block ** Dynamic block
Org QL provides a dynamic block that lists entries in the current document matching a query. In the header, these parameters are supported: Org QL provides a dynamic block that lists entries in the current document matching a query. In the header, these parameters are supported:
@ -463,7 +505,11 @@ Simple links may also be written manually in either sexp or non-sexp form, like:
** 0.6-pre ** 0.6-pre
Nothing new yet. *Added*
+ Macro =org-ql-defpred=, used to define search predicates. (See [[file:examples/defpred.org][tutorial]].)
*Internal*
+ Predicates are now defined more cleanly with a macro (=org-ql-defpred=) that consolidates functionality related to each predicate. This will also allow users to more easily define custom predicates.
** 0.5 ** 0.5

5
examples.org
View file

@ -5,6 +5,7 @@
:TOC: :include siblings :ignore this :TOC: :include siblings :ignore this
:END: :END:
:CONTENTS: :CONTENTS:
- [[#custom-predicates][Custom predicates]]
- [[#agenda-like-view][Agenda-like view]] - [[#agenda-like-view][Agenda-like view]]
- [[#entries-from-the-past-week][Entries from the past week]] - [[#entries-from-the-past-week][Entries from the past week]]
- [[#find-entries-matching-a-certain-custom_id][Find entries matching a certain CUSTOM_ID]] - [[#find-entries-matching-a-certain-custom_id][Find entries matching a certain CUSTOM_ID]]
@ -18,6 +19,10 @@
- [[#task-list-for-files-in-subdirectories][Task list for files in subdirectories]] - [[#task-list-for-files-in-subdirectories][Task list for files in subdirectories]]
:END: :END:
* Custom predicates
+ [[file:examples/defpred.org][Custom predicate tutorial]]
* Agenda-like view * Agenda-like view
Show an agenda-like view, similar to a "traditional" Org Agenda with Log Mode turned on. Show an agenda-like view, similar to a "traditional" Org Agenda with Log Mode turned on.

331
examples/defpred.org Normal file
View file

@ -0,0 +1,331 @@
#+TITLE: Org QL Custom Predicates Tutorial
#+OPTIONS: author:nil creator:nil created:nil date:nil num:nil title:t
[2020-11-22 Sun 22:35] Imagine you have weekly meetings with other people, and during the week you take notes about items to discuss at each meeting. When the time for a meeting comes, you want to quickly and easily search for all of the items to discuss at the meeting.
You've been experimenting with different ways to track such data in Org. You've tried using tags, but some of the names in question conflict with other tags in your data (e.g. someone's named Charles, but you also work with a firm named Charles, Inc., and you'd prefer to continue using the tag =Charles= for entries about that firm), so you've been using tags like ~:personNAME:~, which seems awkward. You've tried using a ~:person: NAME~ property on entries, which has the advantage of not cluttering the tags list, but also the disadvantage of not being readily visible in an outline.
So you haven't decided on a long-term solution, but the meetings aren't going to wait--you need to search that data now, and you have a mix of both tags and properties in your entries. What you need is to be able to search for all of the entries about Alice (which you've tagged ~:personAlice:~) when you're meeting with her, and all of the entries about Bob (which have the property ~:person: Bob~) when you're meeting with him What do you do?
* Contents
:PROPERTIES:
:TOC: :include siblings :ignore this
:END:
:CONTENTS:
- [[#using-built-in-predicates][Using built-in predicates]]
- [[#a-custom-person-predicate][A custom (person) predicate]]
- [[#searching-for-multiple-people-at-once][Searching for multiple people at once]]
- [[#normalizing-queries-to-rewrite-arguments][Normalizing queries to rewrite arguments]]
- [[#non-sexp-query-syntax][Non-sexp query syntax]]
- [[#using-multiple-predicates][Using multiple predicates]]
- [[#predicate-aliases][Predicate aliases]]
- [[#appendix-anaphoric-macros][Appendix: Anaphoric macros]]
:END:
* Using built-in predicates
You could start by using built-in Org QL predicates to search your data. For example:
#+BEGIN_SRC elisp :results list :exports both :cache yes
(org-ql-query :select '(org-get-heading :no-tags)
:from (current-buffer)
:where '(or (tags "personAlice")
(property "person" "Bob")))
#+END_SRC
#+RESULTS[91a413cda23cb65d6bb99212e111f283e5a5c910]:
- [#A] Loud pet parakeet
- [#C] Missing sticky notes
- [#C] Dirty dishes in sink
- [#A] Stinky coffee breath
That was easy enough, but it's not very...semantic. You have to think about the implementation details: Alice uses tags, Bob uses properties, and what if Charlie uses both? It starts to feel complicated, and it's a lot to type out every time. Is there an easier way?
* A custom ~(person)~ predicate
Enter =org-ql= custom search predicates. Let's start simple, by defining a predicate to search for just the ~:person:~ property, one person at a time. The predicate will take one argument, a person's name, and search for that property. It would look like this:
#+BEGIN_SRC elisp :results silent :exports code
(org-ql-defpred person (name)
"Search for entries with the \"person\" property being NAME."
:body (property "person" name))
#+END_SRC
Now let's see what results we get for searching this file for entries about Bob:
#+BEGIN_SRC elisp :results list :exports both :cache yes
(org-ql-query :select '(org-get-heading :no-tags)
:from (current-buffer)
:where '(person "Bob"))
#+END_SRC
#+RESULTS[c11a4ce2c4f179d7487c9b46eff9f72766bc2bc4]:
- [#C] Missing sticky notes
- [#C] Dirty dishes in sink
- [#A] Stinky coffee breath
Hmm, looks like we need to remind Bob to wash his mug and take some mints after lunch. Now what do we need to discuss with Alice?
#+BEGIN_SRC elisp :results list :exports both :cache yes
(org-ql-query :select '(org-get-heading :no-tags)
:from (current-buffer)
:where '(person "Alice"))
#+END_SRC
#+RESULTS[1f12f437042bbc077a4696d707805c1367f2ca3d]:
#+BEGIN_EXAMPLE
#+END_EXAMPLE
Nothing? Oh, right, Alice's entries use the ~:personAlice:~ tag, so we'll also need to also search those kind of entries. Let's make the predicate do that too:
#+BEGIN_SRC elisp :results silent :exports code
(org-ql-defpred person (name)
"Search for entries with the \"person\" property being NAME or having the tag \"personNAME\"."
:body (or (property "person" name)
(tags (concat "person" name))))
#+END_SRC
How about now?
#+BEGIN_SRC elisp :results list :exports both :cache yes
(org-ql-query :select '(org-get-heading :no-tags)
:from (current-buffer)
:where '(person "Alice"))
#+END_SRC
#+RESULTS[1f12f437042bbc077a4696d707805c1367f2ca3d]:
- [#A] Loud pet parakeet
- [#C] Missing sticky notes
- [#C] Dirty dishes in sink
Hmm, I thought we already told her to leave Polly at home.
* Searching for multiple people at once
Oh, wait, this week is shortened due to holidays, so we're having a combined meeting. How do we search for entries about either of them? Well, this is the obvious solution:
#+BEGIN_SRC elisp :results list :exports both :cache yes
(org-ql-query :select '(org-get-heading :no-tags)
:from (current-buffer)
:where '(or (person "Alice")
(person "Bob")))
#+END_SRC
#+RESULTS[4e4c75bde4fbceaadb076a53410c1625d1283e06]:
- [#A] Loud pet parakeet
- [#C] Missing sticky notes
- [#C] Dirty dishes in sink
- [#A] Stinky coffee breath
And that works fine. But it seems like a lot to type. Could we make the =person= predicate accept multiple names instead?
#+BEGIN_SRC elisp :results silent :exports code
(org-ql-defpred person (&rest names)
"Search for entries about any of NAMES."
:body (cl-loop for name in names
thereis (or (property "person" name)
(tags (concat "person" name)))))
#+END_SRC
#+BEGIN_SRC elisp :results list :exports both :cache yes
(org-ql-query :select '(org-get-heading :no-tags)
:from (current-buffer)
:where '(person "Alice" "Bob"))
#+END_SRC
#+RESULTS[4f5971c56616f01d8d3c28a66ef380495ee3e158]:
- [#A] Loud pet parakeet
- [#C] Missing sticky notes
- [#C] Dirty dishes in sink
- [#A] Stinky coffee breath
That was easy!
* Normalizing queries to rewrite arguments
Now, all this is well and good if you don't have hundreds of thousands of Org entries in your files. But what if you do? All that =concat='ing happening on every entry could add up, and the query might take a few seconds. What if we could do that stringing-along just once, before running the query? We want to turn our ~(person "Alice" "Bob")~ query into this, with the =:personNAME:= strings already made and the per-person ~(property ...)~ predicates also included:
#+BEGIN_SRC elisp
(or (tags "personAlice" "personBob")
(property "person" "Alice")
(property "person" "Bob"))
#+END_SRC
Can we do that? In fact, we can, by using a query normalizer. Normalizers are =pcase= forms (I /know/) that normalize query expressions before execution. We can use one to rewrite the query ahead of time, like this:
#+BEGIN_SRC elisp :results silent :exports code
(org-ql-defpred person (&rest names)
"Search for entries about any of NAMES."
:normalizers ((`(person . ,names)
`(or (tags ,@(cl-loop for name in names
collect (concat "person" name)))
,@(cl-loop for name in names
collect `(property "person" ,name)))))
:body (cl-loop for name in names
thereis (or (property "person" name)
(tags (concat "person" name)))))
#+END_SRC
Now, don't faint from all the backquoting and unquoting--it's just Lisp, nothing to be afraid of! Let's slow down a moment and see what the normalized query looks like to be sure we're doing it correctly:
#+BEGIN_SRC elisp :results code :exports both :cache yes
(org-ql--normalize-query '(person "Alice" "Bob"))
#+END_SRC
#+RESULTS[ebc46fff31b72359353dda539a26c95b7d650df2]:
#+BEGIN_SRC elisp
(or (tags "personAlice" "personBob")
(property "person" "Alice")
(property "person" "Bob"))
#+END_SRC
And, as they say, Bob's your uncle! Or even if he isn't, let's see if it works:
#+BEGIN_SRC elisp :results list :exports both :cache yes
(org-ql-query :select '(org-get-heading :no-tags)
:from (current-buffer)
:where '(person "Alice" "Bob"))
#+END_SRC
#+RESULTS[4f5971c56616f01d8d3c28a66ef380495ee3e158]:
- [#A] Loud pet parakeet
- [#C] Missing sticky notes
- [#C] Dirty dishes in sink
- [#A] Stinky coffee breath
Yep, same result as the non-normalized query. And look at how much simpler it is to write ~(person "Alice" "Bob")~ than to write ~(or (tags "personAlice" "personBob") (property "person" "Alice") (property "person" "Bob"))~.
* Non-sexp query syntax
But wait, that's not all! If you order now, we'll throw in non-sexp query syntax for free! That's right, your search could be as simple as typing ~person:Alice,Bob~!
#+BEGIN_SRC elisp :results none :exports code
(org-ql-search (current-buffer) "person:Alice,Bob")
#+END_SRC
Don't believe me? Well, you see, queries in this syntax are automatically converted to the sexp syntax, like:
#+BEGIN_SRC elisp :results code :exports both :cache yes
(org-ql--query-string-to-sexp "person:Alice,Bob")
#+END_SRC
#+RESULTS[a60655544956644605c23c152570185c329faa87]:
#+BEGIN_SRC elisp
(person "Alice" "Bob")
#+END_SRC
But that happens automatically when you use a search command like =org-ql-search=. If you have =org-ql= installed already, you could even click this link: [[org-ql-search:person:Alice,Bob][Alice or Bob]]. Which, in Org syntax, looks like:
#+BEGIN_SRC org
[[org-ql-search:person:Alice,Bob]]
#+END_SRC
And that would open an Agenda Mode buffer that looks like this:
#+BEGIN_EXAMPLE
Query: (person "Alice" "Bob") In:meetings.org
[#A] Loud pet parakeet :personAlice:
[#C] Missing sticky notes :personAlice:
[#C] Dirty dishes in sink :personAlice:
[#A] Stinky coffee breath
#+END_EXAMPLE
* Using multiple predicates
Oops, you forgot that there's a birthday party in 20 minutes, so you only have time to talk about the highest priority items at this joint meeting today.
No problem, let's just select high-priority items:
#+BEGIN_SRC elisp :results silent :exports code
(org-ql-search (current-buffer) "person:Alice,Bob priority:A")
#+END_SRC
#+BEGIN_EXAMPLE
Query: (and (person "Alice" "Bob") (priority "A")) In:meetings.org
[#A] Loud pet parakeet :personAlice:
[#A] Stinky coffee breath
#+END_EXAMPLE
* Predicate aliases
And, you know what, if you're just so busy that you don't even have time to type the word =person=, you can add an abbreviated alias, =p=, like this:
#+BEGIN_SRC elisp :results silent :exports code
(org-ql-defpred (person p) (&rest names)
"Search for entries about any of NAMES."
:normalizers ((`(,predicate-names . ,names)
`(or (tags ,@(cl-loop for name in names
collect (concat "person" name)))
,@(cl-loop for name in names
collect `(property "person" ,name)))))
:body (cl-loop for name in names
thereis (or (property "person" name)
(tags (concat "person" name)))))
#+END_SRC
#+BEGIN_SRC elisp :results silent :exports code
(org-ql-search (current-buffer) "p:Alice,Bob priority:A")
#+END_SRC
#+BEGIN_EXAMPLE
Query: (and (person "Alice" "Bob") (priority "A")) In:meetings.org
[#A] Loud pet parakeet :personAlice:
[#A] Stinky coffee breath
#+END_EXAMPLE
(It's up to you to remember whether =p= means =person= or =priority=, but code can't solve everything.)
* Appendix: Anaphoric macros
Finally, if you're a Lisper who appreciates anaphora, you might prefer a more syntactically concise definition of the predicate using Dash macros:
#+BEGIN_SRC elisp :results silent :exports code
(org-ql-defpred (person p) (&rest names)
"Search for entries about any of NAMES."
:normalizers ((`(,predicate-names . ,names)
`(or (tags ,@(--map `(concat "person" ,it) names))
,@(--map `(property "person" ,it) names))))
:body (--any (or (property "person" name)
(tags (concat "person" name)))
names))
#+END_SRC
Let's make sure it works:
#+BEGIN_SRC elisp :results list :exports both :cache yes
(org-ql-query :select '(org-get-heading :no-tags)
:from (current-buffer)
:where '(person "Alice" "Bob"))
#+END_SRC
#+RESULTS[4f5971c56616f01d8d3c28a66ef380495ee3e158]:
- [#A] Loud pet parakeet
- [#C] Missing sticky notes
- [#C] Dirty dishes in sink
- [#A] Stinky coffee breath
Have fun making custom search predicates!
* Example data
:PROPERTIES:
:TOC: :ignore (this descendants)
:END:
** [#A] Loud pet parakeet :personAlice:
** [#C] Missing sticky notes :personAlice:
:PROPERTIES:
:person: Bob
:END:
** [#C] Dirty dishes in sink :personAlice:
:PROPERTIES:
:person: Bob
:END:
** [#A] Stinky coffee breath
:PROPERTIES:
:person: Bob
:END:

4
helm-org-ql.el
View file

@ -173,7 +173,7 @@ Is transformed into this query:
(interactive) (interactive)
(let ((buffers-files (with-current-buffer (helm-buffer-get) (let ((buffers-files (with-current-buffer (helm-buffer-get)
helm-org-ql-buffers-files)) helm-org-ql-buffers-files))
(query (org-ql--plain-query helm-pattern))) (query (org-ql--query-string-to-sexp helm-pattern)))
(helm-run-after-exit #'org-ql-search buffers-files query))) (helm-run-after-exit #'org-ql-search buffers-files query)))
;;;###autoload ;;;###autoload
@ -189,7 +189,7 @@ Is transformed into this query:
;; Expansion of `helm-build-sync-source' macro. ;; Expansion of `helm-build-sync-source' macro.
(helm-make-source name 'helm-source-sync (helm-make-source name 'helm-source-sync
:candidates (lambda () :candidates (lambda ()
(let* ((query (org-ql--plain-query helm-pattern)) (let* ((query (org-ql--query-string-to-sexp helm-pattern))
(window-width (window-width (helm-window)))) (window-width (window-width (helm-window))))
(when query (when query
(with-current-buffer (helm-buffer-get) (with-current-buffer (helm-buffer-get)

4
org-ql-search.el
View file

@ -158,7 +158,7 @@ necessary."
;; Read sexp query. ;; Read sexp query.
(read query) (read query)
;; Parse non-sexp query into sexp query. ;; Parse non-sexp query into sexp query.
(org-ql--plain-query query))) (org-ql--query-string-to-sexp query)))
(list query))) (list query)))
(results (org-ql-select buffers-files query (results (org-ql-select buffers-files query
:action 'element-with-markers :action 'element-with-markers
@ -273,7 +273,7 @@ For example, an org-ql dynamic block header could look like:
#+BEGIN: org-ql :query (todo \"UNDERWAY\") :columns (priority todo heading) :sort (priority date) :ts-format \"%Y-%m-%d %H:%M\"" #+BEGIN: org-ql :query (todo \"UNDERWAY\") :columns (priority todo heading) :sort (priority date) :ts-format \"%Y-%m-%d %H:%M\""
(-let* (((&plist :query :columns :sort :ts-format :take) params) (-let* (((&plist :query :columns :sort :ts-format :take) params)
(query (cl-etypecase query (query (cl-etypecase query
(string (org-ql--plain-query query)) (string (org-ql--query-string-to-sexp query))
(list ;; SAFETY: Query is in sexp form: ask for confirmation, because it could contain arbitrary code. (list ;; SAFETY: Query is in sexp form: ask for confirmation, because it could contain arbitrary code.
(org-ql--ask-unsafe-query query) (org-ql--ask-unsafe-query query)
query))) query)))

2
org-ql-view.el
View file

@ -775,7 +775,7 @@ When opened, the link searches the buffer it's opened from."
;; Read sexp query. ;; Read sexp query.
(read query) (read query)
;; Parse non-sexp query into sexp query. ;; Parse non-sexp query into sexp query.
(org-ql--plain-query query))))) (org-ql--query-string-to-sexp query)))))
(define-infix-command org-ql-view--transient-in () (define-infix-command org-ql-view--transient-in ()
:description (lambda () (org-ql-view--format-transient-lisp-argument "In buffers/files" org-ql-view-buffers-files)) :description (lambda () (org-ql-view--format-transient-lisp-argument "In buffers/files" org-ql-view-buffers-files))

1499
org-ql.el
View file

File diff suppressed because it is too large Load diff

192
org-ql.info
View file

@ -63,6 +63,7 @@ Functions / Macros
* Agenda-like views:: * Agenda-like views::
* Listing / acting-on results:: * Listing / acting-on results::
* Custom predicates::
Changelog Changelog
@ -565,11 +566,13 @@ File: README.info, Node: Functions / Macros, Next: Dynamic block, Prev: Queri
• •
• • • •
• • • • • • • •
•
* Menu: * Menu:
* Agenda-like views:: * Agenda-like views::
* Listing / acting-on results:: * Listing / acting-on results::
* Custom predicates::
 
File: README.info, Node: Agenda-like views, Next: Listing / acting-on results, Up: Functions / Macros File: README.info, Node: Agenda-like views, Next: Listing / acting-on results, Up: Functions / Macros
@ -606,7 +609,7 @@ File: README.info, Node: Agenda-like views, Next: Listing / acting-on results,
the block header, otherwise the header is formed automatically. the block header, otherwise the header is formed automatically.
 
File: README.info, Node: Listing / acting-on results, Prev: Agenda-like views, Up: Functions / Macros File: README.info, Node: Listing / acting-on results, Next: Custom predicates, Prev: Agenda-like views, Up: Functions / Macros
4.3.2 Listing / acting-on results 4.3.2 Listing / acting-on results
--------------------------------- ---------------------------------
@ -737,6 +740,84 @@ File: README.info, Node: Listing / acting-on results, Prev: Agenda-like views,
Expands into a call to ‘org-ql-select’ with the same arguments. Expands into a call to ‘org-ql-select’ with the same arguments.
For convenience, arguments should be unquoted. For convenience, arguments should be unquoted.

File: README.info, Node: Custom predicates, Prev: Listing / acting-on results, Up: Functions / Macros
4.3.3 Custom predicates
-----------------------
• See: Custom predicate tutorial (examples/defpred.org)
1. Macro: org-ql-defpred
_Arguments:_ ‘(name args docstring &key body preambles
normalizers)’
Define an ‘org-ql’ selector predicate named
‘org-ql--predicate-NAME’. ‘NAME’ may be a symbol or a list of
symbols: if a list, the first is used as ‘NAME’ and the rest are
aliases. ‘A’ function is only created for ‘NAME’, not for aliases,
so a normalizer should be used to replace aliases with ‘NAME’ in
queries (keep reading).
‘ARGS’ is a ‘cl-defun’-style argument list. ‘DOCSTRING’ is the
function’s docstring.
‘BODY’ is the body of the predicate. It will be evaluated with
point on the beginning of an Org heading and should return non-nil
if the heading’s entry is a match.
‘PREAMBLES’ and ‘NORMALIZERS’ are lists of ‘pcase’ forms matched
against Org ‘QL’ query sexps. They are spliced into ‘pcase’ forms
in the definitions of the functions ‘org-ql--query-preamble’ and
‘org-ql--normalize-query’, which see. Those functions are
redefined when this macro is expanded, unless variable
‘org-ql-defpred-defer’ is non-nil, in which case those functions
should be redefined manually after defining predicates by calling
‘org-ql--define-query-preamble-fn’ and
‘org-ql--define-normalize-query-fn’.
‘NORMALIZERS’ are used to normalize query expressions to standard
forms. For example, when the predicate has aliases, the aliases
should be replaced with predicate names using a normalizer. Also,
predicate arguments may be put into a more optimal form so that the
predicate has less work to do at query time.
‘PREAMBLES’ refer to regular expressions which may be used to
search through a buffer directly to a potential match rather than
testing the predicate body on each heading. (Naming things is
hard.) In each ‘pcase’ form in ‘PREAMBLES’, the ‘pcase’ expression
(not the pattern) should be a plist with the following keys, each
value of which should be an expression which may refer to variables
bound in the pattern:
‘:regexp’ Regular expression which searches directly to a potential
match.
‘:case-fold’ Bound to ‘case-fold-search’ around the regexp search.
‘:query’ Expression which should replace the query expression, or
‘query’ if it should not be changed (e.g. if the regexp is
insufficient to determine whether a heading matches, in which case
the predicate’s body needs to be tested on the heading). If the
regexp guarantees a match, this may be simply ‘t’, leaving the
query expression with no work to do, which improves performance.
For convenience, within the ‘pcase’ patterns, the symbol
‘predicate-names’ is a special form which is replaced with a
pattern matching any of the predicate’s name and aliases. For
example, if ‘NAME’ were:
‘(heading h)’
Then if ‘NORMALIZERS’ were:
‘((`(,predicate-names . ,args) `(heading ,@args)))’
It would be expanded to:
‘((`(,(or 'heading 'h) . ,args) `(heading ,@args)))’
 
File: README.info, Node: Dynamic block, Next: Links, Prev: Functions / Macros, Up: Usage File: README.info, Node: Dynamic block, Next: Links, Prev: Functions / Macros, Up: Usage
@ -859,7 +940,15 @@ File: README.info, Node: 06-pre, Next: 05, Up: Changelog
5.1 0.6-pre 5.1 0.6-pre
=========== ===========
Nothing new yet. *Added*
• Macro org-ql-defpred, used to define search predicates. (See
tutorial (examples/defpred.org).)
*Internal*
• Predicates are now defined more cleanly with a macro
(org-ql-defpred) that consolidates functionality related to each
predicate. This will also allow users to more easily define custom
predicates.
 
File: README.info, Node: 05, Next: 049, Prev: 06-pre, Up: Changelog File: README.info, Node: 05, Next: 049, Prev: 06-pre, Up: Changelog
@ -1348,54 +1437,55 @@ GPLv3
 
Tag Table: Tag Table:
Node: Top225 Node: Top225
Node: Contents1606 Node: Contents1628
Node: Screenshots1729 Node: Screenshots1751
Node: Installation1847 Node: Installation1869
Node: Quelpa2485 Node: Quelpa2507
Node: Usage2928 Node: Usage2950
Node: Commands3326 Node: Commands3348
Node: org-ql-search3799 Node: org-ql-search3821
Node: helm-org-ql5449 Node: helm-org-ql5471
Node: org-ql-view5861 Node: org-ql-view5883
Node: org-ql-view-sidebar6361 Node: org-ql-view-sidebar6383
Node: org-ql-view-recent-items6717 Node: org-ql-view-recent-items6739
Node: org-ql-sparse-tree7201 Node: org-ql-sparse-tree7223
Node: Queries8001 Node: Queries8023
Node: Non-sexp query syntax9112 Node: Non-sexp query syntax9134
Node: General predicates10819 Node: General predicates10841
Node: Ancestor/descendant predicates16232 Node: Ancestor/descendant predicates16254
Node: Date/time predicates17360 Node: Date/time predicates17382
Node: Functions / Macros20015 Node: Functions / Macros20037
Node: Agenda-like views20270 Node: Agenda-like views20335
Node: Listing / acting-on results21675 Node: Listing / acting-on results21740
Node: Dynamic block26870 Node: Custom predicates26961
Node: Links29568 Node: Dynamic block30452
Node: Tips30255 Node: Links33150
Node: Changelog30573 Node: Tips33837
Node: 06-pre31269 Node: Changelog34155
Node: 0531375 Node: 06-pre34851
Node: 04932852 Node: 0535291
Node: 04833126 Node: 04936768
Node: 04733473 Node: 04837042
Node: 04633868 Node: 04737389
Node: 04534268 Node: 04637784
Node: 04434627 Node: 04538184
Node: 04334984 Node: 04438543
Node: 04235179 Node: 04338900
Node: 04135340 Node: 04239095
Node: 0435581 Node: 04139256
Node: 03239514 Node: 0439497
Node: 03139893 Node: 03243430
Node: 0340090 Node: 03143809
Node: 02343065 Node: 0344006
Node: 02243293 Node: 02346981
Node: 02143561 Node: 02247209
Node: 0243760 Node: 02147477
Node: 0147795 Node: 0247676
Node: Notes47896 Node: 0151711
Node: Comparison with Org Agenda searches48058 Node: Notes51812
Node: org-sidebar48930 Node: Comparison with Org Agenda searches51974
Node: License49209 Node: org-sidebar52846
Node: License53125
 
End Tag Table End Tag Table

122
tests/test-org-ql.el
View file

@ -50,7 +50,7 @@ Set at runtime by test suite.")
('org-ql 'org-ql) ('org-ql 'org-ql)
('org-ql-expect t) ('org-ql-expect t)
('org-ql--query-preamble 'query-preamble) ('org-ql--query-preamble 'query-preamble)
('org-ql--pre-process-query t) ('org-ql--normalize-query t)
(_ nil))) (_ nil)))
(result (pcase sexp (result (pcase sexp
(`(org-ql-expect ,args) (`(org-ql-expect ,args)
@ -63,7 +63,7 @@ Set at runtime by test suite.")
:action (org-ql-test-org-get-heading)))) :action (org-ql-test-org-get-heading))))
(`(org-ql-select . _) (org-ql-test--format-result--ql sexp)) (`(org-ql-select . _) (org-ql-test--format-result--ql sexp))
(`(org-ql--query-preamble . _) (org-ql-test--format-result--query-preamble sexp)) (`(org-ql--query-preamble . _) (org-ql-test--format-result--query-preamble sexp))
(`(org-ql--pre-process-query . _) (format "'%S" (eval sexp))) (`(org-ql--normalize-query . _) (format "'%S" (eval sexp)))
(_ nil)))) (_ nil))))
(progn (progn
(backward-char 1) (backward-char 1)
@ -222,61 +222,75 @@ RESULTS should be a list of strings as returned by
(describe "(level)" (describe "(level)"
(it "with one level" (it "with one level"
(expect (org-ql--pre-process-query '(level "1")) (expect (org-ql--normalize-query '(level "1"))
:to-equal '(level 1))) :to-equal '(level 1)))
(it "with two levels" (it "with two levels"
(expect (org-ql--pre-process-query '(level "1" "2")) (expect (org-ql--normalize-query '(level "1" "2"))
:to-equal '(level 1 2))) :to-equal '(level 1 2)))
(it "with a comparator and a level" (it "with a comparator and a level"
(expect (org-ql--pre-process-query '(level ">" "1")) (expect (org-ql--normalize-query '(level ">" "1"))
:to-equal '(level > 1)))) :to-equal '(level > 1))))
(describe "(link)" (describe "(link)"
(it "with one argument" (it "with one argument"
(expect (org-ql--pre-process-query '(link "DESC-OR-TARGET")) (expect (org-ql--normalize-query '(link "DESC-OR-TARGET"))
:to-equal '(link "DESC-OR-TARGET"))) :to-equal '(link "DESC-OR-TARGET")))
(it "with one argument and :regexp-p" (it "with one argument and :regexp-p"
(expect (org-ql--pre-process-query '(link "DESC-OR-TARGET" :regexp-p t)) (expect (org-ql--normalize-query '(link "DESC-OR-TARGET" :regexp-p t))
:to-equal '(link "DESC-OR-TARGET" :regexp-p t))) :to-equal '(link "DESC-OR-TARGET" :regexp-p t)))
(it "with keyword arguments" (it "with keyword arguments"
(expect (org-ql--pre-process-query '(link :description "DESCRIPTION" :target "TARGET" (expect (org-ql--normalize-query '(link :description "DESCRIPTION" :target "TARGET"
:regexp-p t)) :regexp-p t))
:to-equal '(link :description "DESCRIPTION" :target "TARGET" :to-equal '(link :description "DESCRIPTION" :target "TARGET"
:regexp-p t)))) :regexp-p t))))
(expect (org-ql--pre-process-query '(and "string1" "string2")) (expect (org-ql--normalize-query '(and "string1" "string2"))
:to-equal '(and (regexp "string1") (regexp "string2"))) :to-equal '(and (regexp "string1") (regexp "string2")))
(expect (org-ql--pre-process-query '(or "string1" "string2")) (expect (org-ql--normalize-query '(or "string1" "string2"))
:to-equal '(or (regexp "string1") (regexp "string2"))) :to-equal '(or (regexp "string1") (regexp "string2")))
(expect (org-ql--pre-process-query '(and (todo "TODO") (expect (org-ql--normalize-query '(and (todo "TODO")
(or "string1" "string2"))) (or "string1" "string2")))
:to-equal '(and (todo "TODO") (or (regexp "string1") (regexp "string2")))) :to-equal '(and (todo "TODO") (or (regexp "string1") (regexp "string2"))))
(expect (org-ql--pre-process-query '(when (todo "TODO") (expect (org-ql--normalize-query '(when (todo "TODO")
(or "string1" "string2"))) (or "string1" "string2")))
:to-equal '(when (todo "TODO") (or (regexp "string1") (regexp "string2")))) :to-equal '(when (todo "TODO") (or (regexp "string1") (regexp "string2"))))
(expect (org-ql--pre-process-query '(when "string-cond1" (expect (org-ql--normalize-query '(when "string-cond1"
(or "string1" "string2"))) (or "string1" "string2")))
:to-equal '(when (regexp "string-cond1") (or (regexp "string1") (regexp "string2")))) :to-equal '(when (regexp "string-cond1") (or (regexp "string1") (regexp "string2"))))
(expect (org-ql--pre-process-query '(when (and "string-cond1" "string-cond2") (expect (org-ql--normalize-query '(when (and "string-cond1" "string-cond2")
(or "string1" "string2"))) (or "string1" "string2")))
:to-equal '(when (and (regexp "string-cond1") (regexp "string-cond2")) (or (regexp "string1") (regexp "string2")))) :to-equal '(when (and (regexp "string-cond1") (regexp "string-cond2")) (or (regexp "string1") (regexp "string2"))))
(expect (org-ql--pre-process-query '(unless (and "stringcondition1" "stringcond2") (expect (org-ql--normalize-query '(unless (and "stringcondition1" "stringcond2")
(or "string1" "string2"))) (or "string1" "string2")))
:to-equal '(unless (and (regexp "stringcondition1") (regexp "stringcond2")) (or (regexp "string1") (regexp "string2")))) :to-equal '(unless (and (regexp "stringcondition1") (regexp "stringcond2")) (or (regexp "string1") (regexp "string2"))))
(expect (org-ql--pre-process-query '(or (ts-active :on "2019-01-01") (expect (org-ql--normalize-query '(or (ts-active :on "2019-01-01")
(ts-a :on "2019-01-01") (ts-a :on "2019-01-01")
(ts-inactive :on "2019-01-01") (ts-inactive :on "2019-01-01")
(ts-i :on "2019-01-01"))) (ts-i :on "2019-01-01")))
:to-equal '(or (ts :type active :on "2019-01-01") :to-equal '(or (ts :type active :on "2019-01-01")
(ts :type active :on "2019-01-01") (ts :type active :on "2019-01-01")
(ts :type inactive :on "2019-01-01") (ts :type inactive :on "2019-01-01")
(ts :type inactive :on "2019-01-01")))) (ts :type inactive :on "2019-01-01"))))
(describe "Query optimizing" (describe "Query preambles"
;; TODO: Other predicates. ;; TODO: Other predicates.
(describe "(clocked)"
(it "without arguments"
(expect (org-ql--query-preamble '(clocked))
:to-equal (list :query t
:preamble org-ql-clock-regexp
:preamble-case-fold nil)))
(it "with a number of days"
(expect (org-ql--query-preamble '(clocked 1))
:to-equal (list :query t
:preamble org-ql-clock-regexp
:preamble-case-fold nil)))
;; TODO: Other arguments for (clocked).
)
(describe "(level)" (describe "(level)"
(it "with a number" (it "with a number"
(expect (org-ql--query-preamble '(level 2)) (expect (org-ql--query-preamble '(level 2))
@ -314,60 +328,60 @@ RESULTS should be a list of strings as returned by
;; TODO: Other predicates. ;; TODO: Other predicates.
(it "Negated terms" (it "Negated terms"
(expect (org-ql--plain-query "todo: !todo:CHECK,SOMEDAY") (expect (org-ql--query-string-to-sexp "todo: !todo:CHECK,SOMEDAY")
:to-equal '(and (todo) (not (todo "CHECK" "SOMEDAY")))) :to-equal '(and (todo) (not (todo "CHECK" "SOMEDAY"))))
(expect (org-ql--plain-query "!todo:CHECK,SOMEDAY todo:") (expect (org-ql--query-string-to-sexp "!todo:CHECK,SOMEDAY todo:")
:to-equal '(and (not (todo "CHECK" "SOMEDAY")) (todo))) :to-equal '(and (not (todo "CHECK" "SOMEDAY")) (todo)))
(expect (org-ql--plain-query "tags:universe !moon") (expect (org-ql--query-string-to-sexp "tags:universe !moon")
:to-equal '(and (tags "universe") (not (regexp "moon")))) :to-equal '(and (tags "universe") (not (regexp "moon"))))
(expect (org-ql--plain-query "!moon tags:universe") (expect (org-ql--query-string-to-sexp "!moon tags:universe")
:to-equal '(and (not (regexp "moon")) (tags "universe"))) :to-equal '(and (not (regexp "moon")) (tags "universe")))
(expect (org-ql--plain-query "mars !ts:on=today") (expect (org-ql--query-string-to-sexp "mars !ts:on=today")
:to-equal '(and (regexp "mars") (not (ts :on "today")))) :to-equal '(and (regexp "mars") (not (ts :on "today"))))
(expect (org-ql--plain-query "!\"quoted phrase\"") (expect (org-ql--query-string-to-sexp "!\"quoted phrase\"")
:to-equal '(not (regexp "quoted phrase")))) :to-equal '(not (regexp "quoted phrase"))))
(it "Regexp predicates" (it "Regexp predicates"
(expect (org-ql--plain-query "scheduled") (expect (org-ql--query-string-to-sexp "scheduled")
;; No colon after keyword, so not a predicate query. ;; No colon after keyword, so not a predicate query.
:to-equal '(regexp "scheduled")) :to-equal '(regexp "scheduled"))
(expect (org-ql--plain-query "\"quoted phrase\"") (expect (org-ql--query-string-to-sexp "\"quoted phrase\"")
:to-equal '(regexp "quoted phrase")) :to-equal '(regexp "quoted phrase"))
(expect (org-ql--plain-query "regexp:word") (expect (org-ql--query-string-to-sexp "regexp:word")
:to-equal '(regexp "word")) :to-equal '(regexp "word"))
(expect (org-ql--plain-query "regexp:\"quoted phrase\"") (expect (org-ql--query-string-to-sexp "regexp:\"quoted phrase\"")
:to-equal '(regexp "quoted phrase"))) :to-equal '(regexp "quoted phrase")))
(it "Timestamp-based predicates" (it "Timestamp-based predicates"
(expect (org-ql--plain-query "scheduled:on=2017-07-07") (expect (org-ql--query-string-to-sexp "scheduled:on=2017-07-07")
:to-equal '(scheduled :on "2017-07-07")) :to-equal '(scheduled :on "2017-07-07"))
(expect (org-ql--plain-query "deadline:from=2017-07-07,to=2017-07-09") (expect (org-ql--query-string-to-sexp "deadline:from=2017-07-07,to=2017-07-09")
:to-equal '(deadline :from "2017-07-07" :to "2017-07-09")) :to-equal '(deadline :from "2017-07-07" :to "2017-07-09"))
(expect (org-ql--plain-query "planning:from=2017-07-07") (expect (org-ql--query-string-to-sexp "planning:from=2017-07-07")
:to-equal '(planning :from "2017-07-07")) :to-equal '(planning :from "2017-07-07"))
(expect (org-ql--plain-query "closed:from=2017-07-07") (expect (org-ql--query-string-to-sexp "closed:from=2017-07-07")
:to-equal '(closed :from "2017-07-07")) :to-equal '(closed :from "2017-07-07"))
(expect (org-ql--plain-query "ts-active:to=2017-07-07") (expect (org-ql--query-string-to-sexp "ts-active:to=2017-07-07")
:to-equal '(ts-active :to "2017-07-07")) :to-equal '(ts-active :to "2017-07-07"))
(expect (org-ql--plain-query "ts-inactive:to=2017-07-07") (expect (org-ql--query-string-to-sexp "ts-inactive:to=2017-07-07")
:to-equal '(ts-inactive :to "2017-07-07")) :to-equal '(ts-inactive :to "2017-07-07"))
(expect (org-ql--plain-query "ts-a:to=2017-07-07") (expect (org-ql--query-string-to-sexp "ts-a:to=2017-07-07")
:to-equal '(ts-a :to "2017-07-07")) :to-equal '(ts-a :to "2017-07-07"))
(expect (org-ql--plain-query "ts-i:on=2017-07-07") (expect (org-ql--query-string-to-sexp "ts-i:on=2017-07-07")
:to-equal '(ts-i :on "2017-07-07")) :to-equal '(ts-i :on "2017-07-07"))
(expect (org-ql--plain-query "ts:") (expect (org-ql--query-string-to-sexp "ts:")
:to-equal '(ts)) :to-equal '(ts))
(expect (org-ql--plain-query "clocked:") (expect (org-ql--query-string-to-sexp "clocked:")
:to-equal '(clocked))) :to-equal '(clocked)))
(it "To-do predicates" (it "To-do predicates"
(expect (org-ql--plain-query "todo:") (expect (org-ql--query-string-to-sexp "todo:")
:to-equal '(todo)) :to-equal '(todo))
(expect (org-ql--plain-query "todo:TODO") (expect (org-ql--query-string-to-sexp "todo:TODO")
:to-equal '(todo "TODO")) :to-equal '(todo "TODO"))
(expect (org-ql--plain-query "todo:TODO,SOMEDAY") (expect (org-ql--query-string-to-sexp "todo:TODO,SOMEDAY")
:to-equal '(todo "TODO" "SOMEDAY"))) :to-equal '(todo "TODO" "SOMEDAY")))
(it "Compound queries" (it "Compound queries"
(expect (org-ql--plain-query "todo:SOMEDAY ts-a:from=2020-01-01,to=2021-01-01") (expect (org-ql--query-string-to-sexp "todo:SOMEDAY ts-a:from=2020-01-01,to=2021-01-01")
:to-equal '(and (todo "SOMEDAY") (ts-a :from "2020-01-01" :to "2021-01-01"))) :to-equal '(and (todo "SOMEDAY") (ts-a :from "2020-01-01" :to "2021-01-01")))
(expect (org-ql--plain-query "regexp:\"quoted phrase\" todo:SOMEDAY") (expect (org-ql--query-string-to-sexp "regexp:\"quoted phrase\" todo:SOMEDAY")
:to-equal '(and (regexp "quoted phrase") (todo "SOMEDAY"))))) :to-equal '(and (regexp "quoted phrase") (todo "SOMEDAY")))))
(describe "Convert sexp queries to non-sexp queries" (describe "Convert sexp queries to non-sexp queries"
@ -842,9 +856,9 @@ RESULTS should be a list of strings as returned by
(org-ql-expect ('(scheduled :to today)) (org-ql-expect ('(scheduled :to today))
'("Skype with president of Antarctica" "Practice leaping tall buildings in a single bound" "Order a pizza" "Get haircut" "Fix flux capacitor" "Shop for groceries" "Rewrite Emacs in Common Lisp"))))) '("Skype with president of Antarctica" "Practice leaping tall buildings in a single bound" "Order a pizza" "Get haircut" "Fix flux capacitor" "Shop for groceries" "Rewrite Emacs in Common Lisp")))))
;; TODO: Test (src) predicate. That will require modifying test data, which will be a ;; ;; TODO: Test (src) predicate. That will require modifying test data, which will be a
;; significant hassle. Manual testing shows that the predicate appears to work properly. ;; ;; significant hassle. Manual testing shows that the predicate appears to work properly.
;;
(describe "(todo)" (describe "(todo)"
(org-ql-it "without arguments" (org-ql-it "without arguments"