adcb96cb1e
The API for `org-get-tags-at` and `org-timestamp--to-internal-time` changed in org-mode version 9.2. Thanks to Ataias Pereira Reis (@ataias) and Daniel Kraus (@dakra). Closes #27. Closes #31.
348 lines
18 KiB
Org Mode
348 lines
18 KiB
Org Mode
#+TITLE: org-ql
|
|
|
|
~org-ql~ is a lispy query language for Org files. It allows you to find Org entries matching certain criteria and return a list of them or perform actions on them. Commands are also provided which display a buffer with matching results, similar to an Org Agenda buffer.
|
|
|
|
* Contents
|
|
:PROPERTIES:
|
|
:TOC: this
|
|
:END:
|
|
- [[#examples][Examples]]
|
|
- [[#usage][Usage]]
|
|
- [[#commands][Commands]]
|
|
- [[#queries][Queries]]
|
|
- [[#predicates][Predicates]]
|
|
- [[#functions--macros][Functions / Macros]]
|
|
- [[#changelog][Changelog]]
|
|
- [[#notes][Notes]]
|
|
|
|
* Examples
|
|
|
|
More examples are available in [[examples.org]].
|
|
|
|
#+BEGIN_SRC elisp
|
|
;; Return a list of Org entry elements in the file "~/org/main.org" which have the SOMEDAY
|
|
;; to-do keyword, are tagged "Emacs", and have priority B or higher.
|
|
(org-ql "~/org/main.org"
|
|
(and (todo "SOMEDAY")
|
|
(tags "Emacs")
|
|
(priority >= "B"))) ;=> ((headline (:raw-value "org-board" :begin 1220270 :end 1220403 ...)) ...)
|
|
|
|
;; Return a list of bills coming due, searching all Org Agenda files, sorted by deadline. Deadlines
|
|
;; are compared with configured Org warning days, which is implied by the plain `<=' in the
|
|
;; `deadline' matcher.
|
|
(org-ql (org-agenda-files)
|
|
(and (not (done))
|
|
(tags "bills")
|
|
(deadline <=))
|
|
:sort deadline)
|
|
|
|
;; Set the tag "Emacs" on every entry in the inbox file that mentions "Emacs".
|
|
(org-ql "~/org/inbox.org"
|
|
(regexp "Emacs")
|
|
:action (org-toggle-tag "Emacs" 'on))
|
|
|
|
;; If you kept a database of music in an Org file, you might run a query like this to find tracks
|
|
;; composed by Chopin that do not have their key recorded in the database:
|
|
(org-ql "~/org/music.org"
|
|
(and (property "genre" "classical")
|
|
(property "composer" "Chopin")
|
|
(not (property "key"))))
|
|
#+END_SRC
|
|
|
|
* Usage
|
|
|
|
The functionality provided may be grouped by:
|
|
|
|
+ Interactive commands :: ~org-ql-search~
|
|
+ Non-interactive functions and macros :: ~org-ql~ (macro), ~org-ql-query~ (function), and ~org-ql-agenda~ (macro)
|
|
|
|
Alternatively, they may be grouped by:
|
|
|
|
+ Showing an agenda-like view :: ~org-ql-search~ (command), and ~org-ql-agenda~ (macro)
|
|
+ Returning a list of matches or acting on them :: ~org-ql~ (macro), and ~org-ql-query~ (function)
|
|
|
|
Feedback on these APIs is welcome. Eventually, after being tested and polished, they will be considered stable.
|
|
|
|
** Commands
|
|
:PROPERTIES:
|
|
:TOC: ignore-children
|
|
:END:
|
|
|
|
*** org-ql-search
|
|
|
|
Read ~QUERY~ and search with ~org-ql~. Interactively, prompt for these variables:
|
|
|
|
~BUFFERS-FILES~: ~A~ list of buffers and/or files to search. Interactively, may also be:
|
|
|
|
+ ~buffer~: search the current buffer
|
|
+ ~all~: search all Org buffers
|
|
+ ~agenda~: search buffers returned by the function ~org-agenda-files~
|
|
+ An expression which evaluates to a list of files/buffers
|
|
+ A space-separated list of file or buffer names
|
|
|
|
~GROUPS~: An ~org-super-agenda~ group set. See variable ~org-super-agenda-groups~.
|
|
|
|
~NARROW~: When non-nil, don't widen buffers before searching. Interactively, with prefix, leave narrowed.
|
|
|
|
~SORT~: One or a list of ~org-ql~ sorting functions, like ~date~ or ~priority~.
|
|
|
|
Press =g= to refresh the results buffer.
|
|
|
|
[[images/org-ql-search.gif]]
|
|
|
|
Here's an example of using it to generate an agenda-like view for certain files in a directory tree:
|
|
|
|
[[images/org-ql-search-snippet.png]]
|
|
|
|
** Queries
|
|
|
|
A query is a lisp form which may contain arbitrary lisp forms, as well as certain built-in predicates. It is byte-compiled into a predicate function which is tested with point on each heading in an Org buffer; when it returns non-nil, the heading matches the query.
|
|
|
|
*** Predicates
|
|
|
|
Arguments are listed next to predicate names, where applicable.
|
|
|
|
Note that, for convenience, standard numeric comparator function symbols (~<~, ~=~, etc.) do not need to be quoted when passed as an argument to these predicates. The resemblance to infix notation is coincidental. See examples in documentation.
|
|
|
|
+ ~category (&optional categories)~ :: Return non-nil if current heading is in one or more of ~CATEGORIES~ (a list of strings).
|
|
+ ~clocked (&key from to on)~ :: Return non-nil if current entry was clocked in given period. If no arguments are specified, return non-nil if entry was clocked at any time. If ~FROM~, return non-nil if entry was clocked on or after ~FROM~. If ~TO~, return non-nil if entry was clocked on or before ~TO~. If ~ON~, return non-nil if entry was clocked on date ~ON~. ~FROM~, ~TO~, and ~ON~ should be strings parseable by ~parse-time-string~ but may omit the time value. Note: Clock entries are expected to be clocked out. Currently clocked entries (i.e. with unclosed timestamp ranges) are ignored.
|
|
+ ~closed (&optional comparator target-date)~ :: Return non-nil if entry's closed date compares with ~TARGET-DATE~ using ~COMPARATOR~. ~TARGET-DATE~ should be a string parseable by ~date-to-day~. ~COMPARATOR~ should be a function (like ~<=~).
|
|
+ ~date (&optional comparator target-date type)~ :: Return non-nil if Org entry at point has date of ~TYPE~ that compares with ~TARGET-DATE~ using ~COMPARATOR~. Checks all Org-formatted timestamp strings in entry. ~TYPE~ may be ~active~, ~inactive~, or ~all~, to control whether active, inactive, or all timestamps are checked. Ranges of each type are also checked. ~TARGET-DATE~ should be a string parseable by ~date-to-day~. ~COMPARATOR~ should be a function (like ~<=~).
|
|
+ ~deadline (&optional comparator target-date)~ :: Return non-nil if entry's deadline compares with ~TARGET-DATE~ using ~COMPARATOR~. ~TARGET-DATE~ should be a string parseable by ~date-to-day~; or if omitted, it is determined automatically using ~org-deadline-warning-days~. ~COMPARATOR~ should be a function (like ~<=~).
|
|
+ ~done~ :: Return non-nil if entry's ~TODO~ keyword is in ~org-done-keywords~.
|
|
+ ~habit~ :: Return non-nil if entry is a habit.
|
|
+ ~heading (regexp)~ :: Return non-nil if current entry's heading matches ~REGEXP~ (a regexp string).
|
|
+ ~level (level-or-comparator &optional level)~ :: Return non-nil if current heading's outline level matches ~LEVEL~ with ~COMPARATOR~. If ~LEVEL~ is nil, ~LEVEL-OR-COMPARATOR~ should be an integer level, which will be tested for equality to the heading's outline level. If ~LEVEL~ is non-nil, ~LEVEL-OR-COMPARATOR~ should be a comparator function (like ~<=~).
|
|
+ ~planning (&optional comparator target-date)~ :: Return non-nil if entry's planning date (deadline or scheduled) compares with ~TARGET-DATE~ using ~COMPARATOR~. ~TARGET-DATE~ should be a string parseable by ~date-to-day~. ~COMPARATOR~ should be a function (like ~<=~).
|
|
+ ~priority (&optional comparator-or-priority priority)~ :: Return non-nil if current heading has a certain priority. ~COMPARATOR-OR-PRIORITY~ should be either a comparator function, like ~<=~, or a priority string, like "A" (in which case (~=~ will be the comparator). If ~COMPARATOR-OR-PRIORITY~ is a comparator, ~PRIORITY~ should be a priority string.
|
|
+ ~property (property &optional value)~ :: Return non-nil if current entry has ~PROPERTY~ (a string), and optionally ~VALUE~ (a string).
|
|
+ ~regexp (regexp)~ :: Return non-nil if current entry matches ~REGEXP~ (a regexp string).
|
|
+ ~scheduled (&optional comparator target-date)~ :: Return non-nil if entry's scheduled date compares with ~TARGET-DATE~ using ~COMPARATOR~. ~TARGET-DATE~ should be a string parseable by ~date-to-day~. ~COMPARATOR~ should be a function (like ~<=~).
|
|
+ ~tags (&optional tags)~ :: Return non-nil if current heading has one or more of ~TAGS~ (a list of strings).
|
|
+ ~todo (&optional keywords)~ :: Return non-nil if current heading is a ~TODO~ item. With ~KEYWORDS~, return non-nil if its keyword is one of ~KEYWORDS~ (a list of strings).
|
|
+ ~ts (&key from to on)~ :: Return non-nil if current entry has a timestamp in given period. If no arguments are specified, return non-nil if entry has any timestamp. If ~FROM~, return non-nil if entry has a timestamp on or after ~FROM~. If ~TO~, return non-nil if entry has a timestamp on or before ~TO~. If ~ON~, return non-nil if entry has a timestamp on date ~ON~. ~FROM~, ~TO~, and ~ON~ should be strings parseable by ~parse-time-string~ but may omit the time value.
|
|
+ ~ts-active (&key from to on)~ :: Return non-nil if current entry has an active timestamp in given period. If no arguments are specified, return non-nil if entry has any active timestamp. If ~FROM~, return non-nil if entry has an active timestamp on or after ~FROM~. If ~TO~, return non-nil if entry has an active timestamp on or before ~TO~. If ~ON~, return non-nil if entry has an active timestamp on date ~ON~. ~FROM~, ~TO~, and ~ON~ should be strings parseable by ~parse-time-string~ but may omit the time value.
|
|
+ ~ts-inactive (&key from to on)~ :: Return non-nil if current entry has an inactive timestamp in given period. If no arguments are specified, return non-nil if entry has any inactive timestamp. If ~FROM~, return non-nil if entry has an inactive timestamp on or after ~FROM~. If ~TO~, return non-nil if entry has an inactive timestamp on or before ~TO~. If ~ON~, return non-nil if entry has an inactive timestamp on date ~ON~. ~FROM~, ~TO~, and ~ON~ should be strings parseable by ~parse-time-string~ but may omit the time value.
|
|
|
|
** Functions / Macros
|
|
:PROPERTIES:
|
|
:TOC: ignore-children
|
|
:END:
|
|
|
|
*** Agenda-like views
|
|
|
|
**** Macro: ~org-ql-agenda~
|
|
|
|
This macro is like ~org-ql~, but it presents matching entries in an Agenda-like view. It's compatible with [[https://github.com/alphapapa/org-super-agenda][org-super-agenda]], which provides grouping. For example:
|
|
|
|
#+BEGIN_SRC elisp
|
|
(org-ql-agenda "~/src/emacs/org-super-agenda/test/test.org"
|
|
(and (or (date = today)
|
|
(deadline <=)
|
|
(scheduled <= today))
|
|
(not (done)))
|
|
;; The `org-super-agenda-groups' setting is used automatically when set, or it
|
|
;; may be overriden by specifying it here:
|
|
:super-groups ((:name "Bills"
|
|
:tag "bills")
|
|
(:todo ("SOMEDAY" "TO-READ" "CHECK" "TO-WATCH" "WATCHING")
|
|
:order 7)
|
|
(:name "Personal"
|
|
:habit t
|
|
:tag "personal"
|
|
:order 3)
|
|
(:todo "WAITING"
|
|
:order 6)
|
|
(:priority "A" :order 1)
|
|
(:priority "B" :order 2)
|
|
(:priority "C" :order 2)))
|
|
#+END_SRC
|
|
|
|
Which presents this buffer:
|
|
|
|
[[images/screenshot.png]]
|
|
|
|
*Note:* The view buffer is currently put in ~org-agenda-mode~, which means that /some/ Org Agenda commands work, such as jumping to entries and changing item priorities (without necessarily updating the view). This feature is experimental and not guaranteed to work correctly with all commands. (It works to the extent it does because the appropriate text properties are placed on each item, imitating an Agenda buffer.)
|
|
|
|
Here are some other examples:
|
|
|
|
#+BEGIN_SRC elisp
|
|
;; Show an agenda-like view of items in "~/org/main.org" with TODO and SOMEDAY keywords which are
|
|
;; tagged "computer" or "Emacs" and in the category "main":
|
|
(org-ql-agenda "~/org/main.org"
|
|
(and (todo "TODO" "SOMEDAY")
|
|
(tags "computer" "Emacs")
|
|
(category "main")))
|
|
|
|
;; Show an agenda-like view of all habits in all agenda files:
|
|
(org-ql-agenda
|
|
(habit))
|
|
|
|
;; Show an agenda-like view similar to a "traditional" Org agenda.
|
|
(org-ql-agenda
|
|
(or (habit)
|
|
(date = today)
|
|
(deadline <=)
|
|
(scheduled <= today)
|
|
(and (todo "DONE" "CANCELLED")
|
|
(closed = today))))
|
|
#+END_SRC
|
|
|
|
*** Listing / acting-on results
|
|
|
|
**** Function: ~org-ql-query~
|
|
|
|
/Arguments:/ ~(buffers-or-files query &key action narrow sort)~
|
|
|
|
Return items matching ~QUERY~ in ~BUFFERS-OR-FILES~.
|
|
|
|
~BUFFERS-OR-FILES~ is a one (or a list of) file(s) or buffer(s).
|
|
|
|
~QUERY~ is an ~org-ql~ query sexp (quoted, since this is a function).
|
|
|
|
~ACTION~ is a function which is called on each matching entry, with point at the beginning of its heading. For example, ~org-element-headline-parser~ may be used to parse an entry into an Org element (note that it must be called with a limit argument, so a lambda must be used to do so). Also see ~org-ql--add-markers~, which may be used to add markers compatible with Org Agenda code.
|
|
|
|
If ~NARROW~ is non-nil, buffers are not widened.
|
|
|
|
~SORT~ is either nil, in which case items are not sorted; or one or a list of defined ~org-ql~ sorting methods: ~date~, ~deadline~, ~scheduled~, ~todo~, and ~priority~.
|
|
|
|
**** Macro: ~org-ql~
|
|
|
|
/Arguments:/ ~(buffers-or-files query &key sort narrow markers action)~
|
|
|
|
Find entries in ~BUFFERS-OR-FILES~ that match ~QUERY~, and return the results of running ~ACTION-FN~ on each matching entry.
|
|
|
|
~BUFFERS-OR-FILES~ is a form which should evaluate to one (or a list of) file(s) or buffer(s).
|
|
|
|
~QUERY~ is an ~org-ql~ query sexp, unquoted.
|
|
|
|
~ACTION~ is a sexp which will be evaluated at each matching entry with point at the beginning of its heading. It is passed to ~org-ql-query~ as a lambda. By default, ~org-element-headline-parser~ is called to return an Org element.
|
|
|
|
~SORT~ is a user defined sorting function, or an unquoted list of one or more sorting methods, including: ~date~, ~deadline~, ~scheduled~, ~todo~, and ~priority~.
|
|
|
|
If ~NARROW~ is non-nil, query will run without widening the buffer (the default is to widen and search the entire buffer).
|
|
|
|
If ~MARKERS~ is non-nil, ~org-agenda-ng--add-markers~ is used to add markers to each item, pointing to the item in its source buffer. In this case, ~ACTION~ should return an Org element.
|
|
|
|
* Changelog
|
|
:PROPERTIES:
|
|
:TOC: ignore-children
|
|
:END:
|
|
|
|
/Note:/ Breaking changes may be made before version 1.0, but in the event of major changes, attempts at backward compatibility will be made with obsolescence declarations, translation of arguments, etc. Users who need stability guarantees before 1.0 may choose to use tagged stable releases.
|
|
|
|
** 0.2-pre
|
|
|
|
*Changed*
|
|
+ ~(regexp)~ selector accepts multiple regexps to test.
|
|
|
|
*Compatibility*
|
|
+ Fixes for compatibility with Org 9.2. (Thanks to [[https://github.com/ataias][Ataias Pereira Reis]] and [[https://github.com/dakra][Daniel Kraus]].)
|
|
|
|
*Internal*
|
|
+ Optimizations for some query selectors, e.g. =regexp= and =todo=. These can provide a significant improvement for some queries. See benchmarks in [[file:notes.org][notes.org]].
|
|
|
|
** 0.1
|
|
|
|
First tagged release.
|
|
|
|
* Notes
|
|
:PROPERTIES:
|
|
:TOC: ignore-children
|
|
:END:
|
|
|
|
** Comparison with Org Agenda searches
|
|
|
|
Of course, queries like these can already be written with Org Agenda searches, but the syntax can be complex. For example, this query would be difficult to write in a standard Org Agenda search, because it matches against a to-do keyword /and/ a plain-text search. As described in the [[https://orgmode.org/worg/org-tutorials/advanced-searching.html#combining-metadata-and-full-text-queries][advanced searching tutorial]], it would require using ~org-search-view~ with a query with specific regular expression syntax, like this:
|
|
|
|
#+BEGIN_EXAMPLE
|
|
+lisp +{^\*+\s-+TO-READ\s-}
|
|
#+END_EXAMPLE
|
|
|
|
But with ~org-ql-agenda~, you would write:
|
|
|
|
#+BEGIN_SRC elisp
|
|
(org-ql-agenda
|
|
(and (regexp "lisp")
|
|
(todo "TO-READ")))
|
|
#+END_SRC
|
|
|
|
** org-sidebar
|
|
|
|
This package is used by [[https://github.com/alphapapa/org-sidebar][org-sidebar]], which presents a customizable agenda-like view in a sidebar window.
|
|
|
|
* License
|
|
:PROPERTIES:
|
|
:TOC: ignore
|
|
:END:
|
|
|
|
GPLv3
|
|
|
|
* COMMENT Code :noexport:
|
|
:PROPERTIES:
|
|
:TOC: ignore
|
|
:END:
|
|
|
|
# The COMMENT keyword prevents GitHub's renderer from showing this entry.
|
|
|
|
Code used to update this document.
|
|
|
|
** Predicates
|
|
|
|
Generates the predicate subtree.
|
|
|
|
#+BEGIN_SRC elisp :results silent :exports code
|
|
(defun org-ql--readme-update-predicates ()
|
|
"Update predicate subtree in current document."
|
|
(interactive)
|
|
(org-ql--readme-replace-node '("Usage" "Queries" "Predicates") (org-ql--readme-predicate-list)))
|
|
|
|
(defun org-ql--readme-replace-node (outline-path string)
|
|
"Replace contents of node at OUTLINE-PATH with STRING."
|
|
(org-with-wide-buffer
|
|
(-let* ((subtree-marker (org-find-olp outline-path t))
|
|
((_headline element) (progn
|
|
(goto-char subtree-marker)
|
|
(org-element-headline-parser (point-max))))
|
|
((&plist :contents-begin beg :contents-end end) element))
|
|
(goto-char beg)
|
|
(delete-region (point) (1- end))
|
|
(insert string "\n"))))
|
|
|
|
(defun org-ql--readme-predicate-list ()
|
|
"Return an Org list string documenting predicates."
|
|
(concat (unpackaged/docstring-to-org
|
|
"Arguments are listed next to predicate names, where applicable.
|
|
|
|
Note that, for convenience, standard numeric comparator function symbols (`<', `=', etc.) do not need to be quoted when passed as an argument to these predicates. The resemblance to infix notation is coincidental. See examples in documentation.\n\n")
|
|
(s-join "\n" (->> org-ql-predicates
|
|
(--sort (string< (symbol-name (plist-get it :name))
|
|
(symbol-name (plist-get other :name))))
|
|
(--map (-let* (((&plist :name name :docstring docstring :fn fn :args args) it)
|
|
(args (->> args
|
|
(--replace-where (listp it) (car it))
|
|
(--replace-where (eq '&rest it) '&optional))))
|
|
(if docstring
|
|
(progn
|
|
(setq docstring (s-replace "\n" " " docstring))
|
|
(format "+ ~%s%s~ :: %s" name
|
|
(if args
|
|
(format " %s" args)
|
|
"")
|
|
(unpackaged/docstring-to-org docstring)))
|
|
(when (s-prefix? "org-ql-" (symbol-name name))
|
|
(warn "No docstring for: %s" name))
|
|
nil)))
|
|
-non-nil))))
|
|
#+END_SRC
|
|
|
|
*** TODO Use async
|
|
|
|
If ~org-ql~ is loaded byte-compiled, the argument lists are not named properly (not sure why, as ~help-function-arglist~ is supposed to handle that). We could run the function in another Emacs process with ~async~ to avoid this.
|
|
|
|
** File-local variables
|
|
|
|
# Local Variables:
|
|
# eval: (require 'org-make-toc)
|
|
# before-save-hook: org-make-toc
|
|
# End:
|