org-ql

• Contents
• Examples
• Usage
  • Commands
    • org-ql-search
  • Queries
    • Predicates
  • Functions / Macros
    • Agenda-like views
      • Function: org-ql-block
      • Macro: org-ql-agenda
    • Listing / acting-on results
      • Function: org-ql-select
      • Function: org-ql-query
      • Macro: org-ql
• Changelog
  • 0.2-pre
  • 0.1
• Notes
  • Comparison with Org Agenda searches
  • org-sidebar
• License

<a href=https://alphapapa.github.io/dont-tread-on-emacs/><img src="images/dont-tread-on-emacs-150.png" align="right"></a>

org-ql is a lispy query language for Org files. It allows you to find Org entries matching certain criteria and return a list of them or perform actions on them. Commands are also provided which display a buffer with matching results, similar to an Org Agenda buffer.

Contents

• Examples

• Usage

  • Commands
  • Queries
  • Functions / Macros
• Changelog
• Notes

Examples

More examples are available in /jiri/org-ql/src/commit/b634647e466a7b25effddfbc535d61251c6629df/examples.org.

  ;; Return a list of Org entry elements in the file "~/org/main.org"
  ;; which have the SOMEDAY to-do keyword, are tagged "Emacs", and have
  ;; priority B or higher.
  (org-ql "~/org/main.org"
    (and (todo "SOMEDAY")
         (tags "Emacs")
         (priority >= "B")))
  ;;=> ((headline (:raw-value "org-board" :begin 1220270 :end 1220403 ...)) ...)

  ;; Set the tag "Emacs" on every entry in the inbox file that mentions
  ;; "Emacs".  `org-ql-select' works like `org-ql' but is a function
  ;; rather than a macro.  The bare-string query "Emacs" is equivalent
  ;; to (regexp "Emacs").
  (org-ql-select "~/org/inbox.org"
    "Emacs"
    :action '(org-toggle-tag "Emacs" 'on))

  ;; Return a list of bills coming due, searching all Org Agenda files,
  ;; sorted by deadline.  Deadlines are compared with
  ;; `org-deadline-warning-days', which is implied by the plain `<=' in
  ;; the `deadline' predicate.  `org-ql-query' works like `org-ql-select'
  ;; but offers arguments named like SQL queries.
  (org-ql-query
    :select #'org-get-heading
    :from (org-agenda-files)
    :where '(and (not (done))
                 (tags "bills")
                 (deadline <=))
    :order-by 'deadline)
  ;;=> ("TODO Electric bill" "TODO Water bill")

  ;; If you kept a database of music in an Org file, you could run a
  ;; query like this to find tracks composed by Chopin that do not have
  ;; their key recorded in the database.  `org-ql-search' works like
  ;; `org-ql-select' and displays results in an agenda-like buffer:
  (org-ql-search "~/org/music.org"
    '(and (property "genre" "classical")
          (property "composer" "Chopin")
          (not (property "key"))))

  ;; Integrate `org-ql' into a custom Org Agenda command which inserts
  ;; an `org-ql' block before the regular agenda:
  (setq org-agenda-custom-commands
        '(("ces" "Custom: Agenda and Emacs SOMEDAY [#A] items"
           ((org-ql-block '(and (todo "SOMEDAY")
                                (tags "Emacs")
                                (priority "A")))
            (agenda)))))

  ;; Show a "stuck projects" view: tasks that are not done and have only
  ;; non-task children.
  (org-ql-agenda (org-agenda-files)
    (and (todo)
         (children)
         (not (children (todo)))))

Usage

The functionality provided may be grouped by:

Interactive commands

org-ql-search

Non-interactive functions and macros

org-ql (macro), org-ql-select (function), org-ql-query (function), org-ql-block (agenda function), and org-ql-agenda (macro)

Alternatively, they may be grouped by:

Showing an agenda-like view

org-ql-search (command), org-ql-block (agenda function), and org-ql-agenda (macro)

Returning a list of matches or acting on them

org-ql (macro), org-ql-select (function), and org-ql-query (function)

Feedback on these APIs is welcome. Eventually, after being tested and polished, they will be considered stable.

Commands

org-ql-search

Read QUERY and search with org-ql. Interactively, prompt for these variables:

BUFFERS-FILES: A list of buffers and/or files to search. Interactively, may also be:

• buffer: search the current buffer
• all: search all Org buffers
• agenda: search buffers returned by the function org-agenda-files
• An expression which evaluates to a list of files/buffers
• A space-separated list of file or buffer names

GROUPS: An org-super-agenda group set. See variable org-super-agenda-groups.

NARROW: When non-nil, don't widen buffers before searching. Interactively, with prefix, leave narrowed.

SORT: One or a list of org-ql sorting functions, like date or priority.

Press g to refresh the results buffer.

Here's an example of using it to generate an agenda-like view for certain files in a directory tree:

Queries

A query is a lisp form which may contain arbitrary lisp forms, as well as certain built-in predicates. It is byte-compiled into a predicate function which is tested with point on each heading in an Org buffer; when it returns non-nil, the heading matches the query.

Notes:

• Bare strings like "string" are automatically converted to (regexp "string") predicates.
• Standard numeric comparator function symbols (<, <=, >, >=, = ) need not be quoted when passed as an argument to these predicates. The resemblance to infix notation is coincidental. See examples in documentation.

Predicates

Arguments are listed next to predicate names, where applicable.

category (&optional categories)

Return non-nil if current heading is in one or more of CATEGORIES (a list of strings).

children (&optional query)

Return non-nil if current heading has direct child headings. If QUERY, test it against child headings. This selector may be nested, e.g. to match grandchild headings.

clocked (&key from to on)

Return non-nil if current entry was clocked in given period. If no arguments are specified, return non-nil if entry was clocked at any time. If FROM, return non-nil if entry was clocked on or after FROM. If TO, return non-nil if entry was clocked on or before TO. If ON, return non-nil if entry was clocked on date ON. FROM, TO, and ON should be strings parseable by parse-time-string but may omit the time value. Note: Clock entries are expected to be clocked out. Currently clocked entries (i.e. with unclosed timestamp ranges) are ignored.

closed (&optional comparator target-date)

Return non-nil if entry's closed date compares with TARGET-DATE using COMPARATOR. TARGET-DATE should be a string parseable by date-to-day. COMPARATOR should be a function (like <=).

date (&optional comparator target-date type)

Return non-nil if Org entry at point has date of TYPE that compares with TARGET-DATE using COMPARATOR. Checks all Org-formatted timestamp strings in entry. TYPE may be active, inactive, or all, to control whether active, inactive, or all timestamps are checked. Ranges of each type are also checked. TARGET-DATE should be a string parseable by date-to-day. COMPARATOR should be a function (like <=).

deadline (&optional comparator target-date)

Return non-nil if entry's deadline compares with TARGET-DATE using COMPARATOR. TARGET-DATE should be a string parseable by date-to-day; or if omitted, it is determined automatically using org-deadline-warning-days. COMPARATOR should be a function (like <=).

descendants (&optional query)

Return non-nil if current heading has descendant headings. If QUERY, test it against descendant headings. This selector may be nested (if you can grok the nesting!).

done

Return non-nil if entry's TODO keyword is in org-done-keywords.

habit

Return non-nil if entry is a habit.

heading (regexp)

Return non-nil if current entry's heading matches REGEXP (a regexp string).

level (level-or-comparator &optional level)

Return non-nil if current heading's outline level matches arguments. The following forms are accepted: (level NUMBER): Matches if heading level is NUMBER. (level NUMBER NUMBER): Matches if heading level is equal to or between NUMBERs. (level COMPARATOR NUMBER): Matches if heading level compares to NUMBER with COMPARATOR. COMPARATOR may be <, <=, >, or >=.

planning (&optional comparator target-date)

Return non-nil if entry's planning date (deadline or scheduled) compares with TARGET-DATE using COMPARATOR. TARGET-DATE should be a string parseable by date-to-day. COMPARATOR should be a function (like <=).

priority (&optional comparator-or-priority priority)

Return non-nil if current heading has a certain priority. COMPARATOR-OR-PRIORITY should be either a comparator function, like <=, or a priority string, like "A" (in which case (= will be the comparator). If COMPARATOR-OR-PRIORITY is a comparator, PRIORITY should be a priority string.

property (property &optional value)

Return non-nil if current entry has PROPERTY (a string), and optionally VALUE (a string). Note that property inheritance is currently not enabled for this predicate. If you need to test with inheritance, you could use a custom predicate form, like (org-entry-get (point) "PROPERTY" 'inherit).

regexp (regexp)

Return non-nil if current entry matches REGEXP (a regexp string).

scheduled (&optional comparator target-date)

Return non-nil if entry's scheduled date compares with TARGET-DATE using COMPARATOR. TARGET-DATE should be a string parseable by date-to-day. COMPARATOR should be a function (like <=).

tags (&optional tags)

Return non-nil if current heading has one or more of TAGS (a list of strings).

todo (&optional keywords)

Return non-nil if current heading is a TODO item. With KEYWORDS, return non-nil if its keyword is one of KEYWORDS (a list of strings). When called without arguments, only matches non-done tasks (i.e. does not match keywords in org-done-keywords).

ts (&key from to on type)

Return non-nil if current entry has a timestamp in given period. If no arguments are specified, return non-nil if entry has any timestamp. If FROM, return non-nil if entry has a timestamp on or after FROM. If TO, return non-nil if entry has a timestamp on or before TO. If ON, return non-nil if entry has a timestamp on date ON. FROM, TO, and ON should be strings parseable by parse-time-string but may omit the time value. TYPE may be active to match active timestamps, inactive to match inactive ones, or both / nil to match both types.

ts-active

Like ts called with :type active.

ts-a

Like ts called with :type active.

ts-inactive

Like ts called with :type inactive.

ts-i

Like ts called with :type inactive.

Functions / Macros

Agenda-like views

Function: org-ql-block

For use as a custom agenda block type in org-agenda-custom-commands. For example, you could define a custom series command like this, which would list all priority A items tagged Emacs with to-do keyword SOMEDAY, followed by the standard agenda view, in a single buffer:

  (setq org-agenda-custom-commands
        '(("ces" "Custom: Agenda and Emacs SOMEDAY [#A] items"
           ((org-ql-block '(and (todo "SOMEDAY")
                                (tags "Emacs")
                                (priority "A")))
            (agenda)))))

Which would be equivalent to a tags-todo search like this:

  (setq org-agenda-custom-commands
        '(("ces" "Custom: Agenda and Emacs SOMEDAY [#A] items"
           ((tags-todo "PRIORITY=\"A\"+Emacs/!SOMEDAY")
            (agenda)))))

However, the org-ql-block version runs in about 1/5th the time.

Macro: org-ql-agenda

This macro is like org-ql, but it presents matching entries in an Agenda-like view. It's compatible with org-super-agenda, which provides grouping. For example:

  (org-ql-agenda "~/src/emacs/org-super-agenda/test/test.org"
    (and (or (date = today)
             (deadline <=)
             (scheduled <= today))
         (not (done)))
    ;; The `org-super-agenda-groups' setting is used automatically when set, or it
    ;; may be overriden by specifying it here:
    :super-groups ((:name "Bills"
                          :tag "bills")
                   (:todo ("SOMEDAY" "TO-READ" "CHECK" "TO-WATCH" "WATCHING")
                          :order 7)
                   (:name "Personal"
                          :habit t
                          :tag "personal"
                          :order 3)
                   (:todo "WAITING"
                          :order 6)
                   (:priority "A" :order 1)
                   (:priority "B" :order 2)
                   (:priority "C" :order 2)))

Which presents this buffer:

Note: The view buffer is currently put in org-agenda-mode, which means that some Org Agenda commands work, such as jumping to entries and changing item priorities (without necessarily updating the view). This feature is experimental and not guaranteed to work correctly with all commands. (It works to the extent it does because the appropriate text properties are placed on each item, imitating an Agenda buffer.)

Here are some other examples:

  ;; Show an agenda-like view of items in "~/org/main.org" with TODO and SOMEDAY keywords which are
  ;; tagged "computer" or "Emacs" and in the category "main":
  (org-ql-agenda "~/org/main.org"
    (and (todo "TODO" "SOMEDAY")
         (tags "computer" "Emacs")
         (category "main")))

  ;; Show an agenda-like view of all habits in all agenda files:
  (org-ql-agenda
    (habit))

  ;; Show an agenda-like view similar to a "traditional" Org agenda.
  (org-ql-agenda
    (or (habit)
        (date = today)
        (deadline <=)
        (scheduled <= today)
        (and (todo "DONE" "CANCELLED")
             (closed = today))))

Listing / acting-on results

Function: org-ql-select

Arguments: (buffers-or-files query &key action narrow sort)

Return items matching QUERY in BUFFERS-OR-FILES.

BUFFERS-OR-FILES is a one or a list of files and/or buffers.

QUERY is an org-ql query sexp (quoted, since this is a function).

ACTION is a function which is called on each matching entry with point at the beginning of its heading. It may be:

• element or nil: Equivalent to org-element-headline-parser.
• element-with-markers: Equivalent to calling org-element-headline-parser, with markers added using org-ql--add-markers. Suitable for formatting with org-ql-agenda--format-element, allowing insertion into an Org Agenda-like buffer.
• A sexp, which will be byte-compiled into a lambda function.
• A function symbol.

If NARROW is non-nil, buffers are not widened (the default is to widen and search the entire buffer).

SORT is either nil, in which case items are not sorted; or one or a list of defined org-ql sorting methods (date, deadline, scheduled, todo, or priority); or a user-defined comparator function that accepts two items as arguments and returns nil or non-nil.

Examples:

  ;; Return list of to-do headings in inbox file with tags and to-do keywords:
  (org-ql-select "~/org/inbox.org"
    '(todo)
    :action #'org-get-heading)
  ;; => ("TODO Practice leaping tall buildings in a single bound  :personal:" ...)

  ;; Without tags and to-do keywords:
  (org-ql-select "~/org/inbox.org"
    '(todo)
    :action '(org-get-heading t t))
  ;; => ("Practice leaping tall buildings in a single bound" ...)

  ;; Return WAITING heading elements in agenda files:
  (org-ql-select (org-agenda-files)
    '(todo "WAITING")
    :action 'element)
  ;; => ((headline (:raw-value "Visit the moon" ...) ...) ...)

  ;; Since `element' is the default for ACTION, it may be omitted:
  (org-ql-select (org-agenda-files)
    '(todo "WAITING"))
  ;; => ((headline (:raw-value "Visit the moon" ...) ...) ...)

Function: org-ql-query

Arguments: (&key (select 'element-with-markers) from where order-by narrow)

Like org-ql-select, but arguments are named more like a SQL query.

• SELECT corresponds to the org-ql-select argument ACTION.
• FROM corresponds to the org-ql-select argument BUFFERS-OR-FILES.
• WHERE corresponds to the org-ql-select argument QUERY.
• ORDER-BY corresponds to the org-ql-select argument SORT, which see.
• NARROW corresponds to the org-ql-select argument NARROW.

Examples:

  ;; Return list of to-do headings in inbox file with tags and to-do keywords:
  (org-ql-query
    :select #'org-get-heading
    :from "~/org/inbox.org"
    :where '(todo))
  ;; => ("TODO Practice leaping tall buildings in a single bound  :personal:" ...)

  ;; Without tags and to-do keywords:
  (org-ql-query
    :select '(org-get-heading t t)
    :from "~/org/inbox.org"
    :where '(todo))
  ;; => ("Practice leaping tall buildings in a single bound" ...)

  ;; Return WAITING heading elements in agenda files:
  (org-ql-query
    :select 'element
    :from (org-agenda-files)
    :where '(todo "WAITING"))
  ;; => ((headline (:raw-value "Visit the moon" ...) ...) ...)

  ;; Since `element' is the default for SELECT, it may be omitted:
  (org-ql-query
    :from (org-agenda-files)
    :where '(todo "WAITING"))
  ;; => ((headline (:raw-value "Visit the moon" ...) ...) ...)

Macro: org-ql

Arguments: (buffers-or-files query &key sort narrow markers action)

Expands into a call to org-ql-select with the same arguments. For convenience, arguments should be unquoted.

Changelog

Note: Breaking changes may be made before version 1.0, but in the event of major changes, attempts at backward compatibility will be made with obsolescence declarations, translation of arguments, etc. Users who need stability guarantees before 1.0 may choose to use tagged stable releases.

0.2-pre

Added

• Function org-ql-query, like org-ql-select but with arguments named more like a SQL query.
• Bare strings like "string" can be used in queries, which are converted to (regexp "string") automatically.
• Selector (regexp) accepts multiple regexps to test.
• Macro org-ql and functions org-ql-query and org-ql-select now also accept a comparator function in their :sort argument.
• Function org-ql-block, which works as an Org Agenda series/composite/block command, usable in custom agenda commands defined in variable org-agenda-custom-commands. (Inspired by Benson Chu's config.)
• Function org-ql-agenda--agenda optionally takes a list of entries as an argument.
• Selectors ts-a and ts-i, aliases for ts-active and ts-inactive.
• Selector ts now accepts a :type argument.
• Face org-ql-agenda-due-date.
• Selectors (children) and (descendants).

Changed

• Function org-ql-query renamed to org-ql-select. org-ql-query now refers to a new function.
• Macro org-ql no longer accepts a :markers argument. Instead, use argument :action element-with-markers. See function org-ql-select, which org-ql calls.
• Selector (todo) no longer matches "done" keywords when used without arguments (i.e. the ones in variable org-done-keywords).

Fixed

• Handle date ranges in date-based selectors. (Thanks to Cody Goodman, Samuel W. Flint, and Vikas Rawal.)
• Don't overwrite bindings in org-agenda-mode-map.
• Don't search buffers without headings, and show a message if the user attempts it.

Compatibility

• Fixes for compatibility with Org 9.2. (Thanks to Ataias Pereira Reis and Daniel Kraus.)

Internal

• Optimizations for some query selectors, e.g. regexp and todo. These can provide a significant improvement for some queries. See benchmarks in notes.org.
• Library ts is now used for parsing and comparing timestamps.

0.1

First tagged release.

Notes

Comparison with Org Agenda searches

Of course, queries like these can already be written with Org Agenda searches, but the syntax can be complex. For example, this query would be difficult to write in a standard Org Agenda search, because it matches against a to-do keyword and a plain-text search. As described in the advanced searching tutorial, it would require using org-search-view with a query with specific regular expression syntax, like this:

  +lisp +{^\*+\s-+TO-READ\s-}

But with org-ql-agenda, you would write:

  (org-ql-agenda
    (and (regexp "lisp")
         (todo "TO-READ")))

org-sidebar

This package is used by org-sidebar, which presents a customizable agenda-like view in a sidebar window.

License

GPLv3