jiri/org-ql
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge: wip/helm-query-rewriting

Browse source 
It's finally time to merge this branch, which adds non-sexp queries,
updates the readme, and a few other things.

Squashed commit of the following:

commit ee54e31dfa2c3335e4ef5498fc9714ddc093968b
Author: Adam Porter <adam@alphapapa.net>
Date:   Sun Oct 6 00:10:19 2019 -0500

    WIP: Depend-on/Require dash-functional

commit 42f6a47d86dfb7f8e96f3b1822e17a3eed825897
Author: Adam Porter <adam@alphapapa.net>
Date:   Sat Oct 5 23:45:06 2019 -0500

    WIP: Update docs

commit eb56e46ed8cd9ad6874dc955375be5efadb4fcf2
Author: Adam Porter <adam@alphapapa.net>
Date:   Sat Oct 5 22:04:24 2019 -0500

    WIP: Add demo script

commit cdd394f4865cd49791b2fbebb1913d9210aeaf35
Author: Adam Porter <adam@alphapapa.net>
Date:   Sat Oct 5 20:24:44 2019 -0500

    WIP: Update demo GIF

commit 7f470a7c3c7612b7718dc26044069dc2a6afdff3
Author: Adam Porter <adam@alphapapa.net>
Date:   Sat Oct 5 13:02:09 2019 -0500

    WIP: (property) Convert keyword properties to strings

    I'm a little uncomfortable with how query parsing and processing is
    getting spread out among 2 or 3 functions, but handling this in the
    plain-query parsing function would require uglier special-case
    workarounds, so this seems like the best solution for now.

commit 4e494a03f29a1d0068daa9cddf095dba15252133
Author: Adam Porter <adam@alphapapa.net>
Date:   Fri Oct 4 16:25:00 2019 -0500

    WIP: Handle query errors, move code, add tags-all, tags& aliases

    This prevents query compilation errors from displaying a warning
    buffer, especially in the Helm command.

    Close to done, but now need to figure out the "property:" predicate,
    because its value argument looks like an alternative property rather
    than a value, e.g. "property:agenda-group,plans" in which "plans" is
    the value for the property "agenda-group".

commit 9d8222d8fa9a584b1ee648c5aa38999200e7c54f
Author: Adam Porter <adam@alphapapa.net>
Date:   Thu Oct 3 22:18:29 2019 -0500

    WIP: Rewrite (priority) predicate to accept multiple priorities

    This commit also fixes a latent bug in the preamble for (priority), which
    wasn't being hit because the query (priority) was being rewritten by
    pre-process to (priority '>= C).

commit 4900470ba52b0fcf4786ab857e316403dec00481
Author: Adam Porter <adam@alphapapa.net>
Date:   Thu Oct 3 20:41:54 2019 -0500

    WIP: More documentation

commit 5041a901f6c1071a4233e5671a0926e65e29a5c0
Author: Adam Porter <adam@alphapapa.net>
Date:   Thu Oct 3 12:30:50 2019 -0500

    WIP: Mention query types for commands

commit bb2d18532cb972a54b5b9056b76b0582eb4cad03
Author: Adam Porter <adam@alphapapa.net>
Date:   Thu Oct 3 11:01:56 2019 -0500

    WIP: Remove old code

commit 8de048e531bfa9ba7058e7d540aae2d332f9be68
Author: Adam Porter <adam@alphapapa.net>
Date:   Thu Oct 3 11:01:03 2019 -0500

    WIP: TODO done

commit 6a38fc845aeb05fe3f03174c04a7011830873705
Author: Adam Porter <adam@alphapapa.net>
Date:   Thu Oct 3 10:59:45 2019 -0500

    WIP: Tidy docstrings

commit 04d0c254fc8694b1363c91a1691f3303fcf7755a
Author: Adam Porter <adam@alphapapa.net>
Date:   Thu Oct 3 10:58:21 2019 -0500

    WIP: Document plain queries

commit 23dcc9bad4a8e6660e48632a20a04adb2654e20d
Author: Adam Porter <adam@alphapapa.net>
Date:   Thu Oct 3 10:27:41 2019 -0500

    WIP: Move ts aliases into defpreds, improve docstrings and comments

commit 6ef710f25bbd430b56ce205b8f314b17cd9ac7c3
Author: Adam Porter <adam@alphapapa.net>
Date:   Thu Oct 3 10:11:17 2019 -0500

    Tidy: Whitespace

commit 1022c4b36fb03a6724d97ba4dd1b5bb68025ed7b
Author: Adam Porter <adam@alphapapa.net>
Date:   Thu Oct 3 10:01:53 2019 -0500

    WIP: (org-ql-search) Accept plain queries

    This way, helm-org-ql and org-ql-search accept queries in the same
    format.

commit 1903e8f5684269c272a2ff3984d4d262f41323c5
Author: Adam Porter <adam@alphapapa.net>
Date:   Thu Oct 3 09:22:35 2019 -0500

    WIP: Add more plain-query tests

commit 9d58e3395e871f3536296ccfbb3901f98416bd9a
Author: Adam Porter <adam@alphapapa.net>
Date:   Thu Oct 3 09:17:07 2019 -0500

    WIP: Return boolean from --plain-query, and only when needed

commit 3770e18e68eaf776aef6fbb94e6ac5648c869c12
Author: Adam Porter <adam@alphapapa.net>
Date:   Thu Oct 3 08:59:55 2019 -0500

    WIP: Parse numbers in key/value args

    e.g. "ts-a:on=-1" for yesterday, or "ts:on=0" or "ts:on=today" for
    today.

commit 0f2b71bb869980f413c7366d1b15fa35f3f6a1c5
Author: Adam Porter <adam@alphapapa.net>
Date:   Thu Oct 3 08:59:43 2019 -0500

    WIP: Rename plain-query function

commit be410a36f4a1c043de24a2d0d2fbbe22135c3385
Author: Adam Porter <adam@alphapapa.net>
Date:   Wed Oct 2 19:21:07 2019 -0500

    WIP: Reverse query properly

    PEG returns queries backwards for some reason, so reverse them back to
    the order they were input in.

commit c184091ac2e96a76696055509ca6245e9a020b03
Author: Adam Porter <adam@alphapapa.net>
Date:   Wed Oct 2 19:17:20 2019 -0500

    WIP: Fix previous commit

    Something weird happens when defining the function that's defined by
    the macro in the eval-when block: It seems that I have to eval it
    twice before it takes effect.  So when I made the previous commit, the
    function defined at the time was working properly, but the function
    defined at the time was apparently not what was written in the file,
    but some intermediate state that had been written before, so I don't
    even know what the code was that was working properly.

    After some more fiddling, this seems to work properly.

commit 4e7288d494c6cfe69f82f3aea4b88feced7232f7
Author: Adam Porter <adam@alphapapa.net>
Date:   Wed Oct 2 18:42:59 2019 -0500

    WIP: Handle keyword/value queries

commit f603d199f47c1115cf11f8661c19c73f056b414d
Author: Adam Porter <adam@alphapapa.net>
Date:   Tue Oct 1 16:45:33 2019 -0500

    WIP: Define pred aliases and input-query function with macros

commit 217ef7dff13e8464baaa50539c82843e48bc3fe8
Author: Adam Porter <adam@alphapapa.net>
Date:   Tue Oct 1 16:16:04 2019 -0500

    WIP: Tidy: Don't quote the lambda

    I don't even know why I did that.

commit 3130f6519d7c21e47a026404e3929f118b003092
Author: Adam Porter <adam@alphapapa.net>
Date:   Fri Sep 27 23:20:50 2019 -0500

    WIP: Parsing with peg

    Seems to be working pretty well.

commit 0762825c7ba8026768aec74b4fffd32b8530c42c
Author: Adam Porter <adam@alphapapa.net>
Date:   Tue Sep 10 16:17:50 2019 -0500

    WIP: Rewriting Helm queries

    Some people are unaccustomed to parentheses, so we'll try to
    accommodate them.
This commit is contained in:
Adam Porter 2019-10-06 00:17:14 -05:00
parent cea8028922
commit f4918b19dc
9 changed files with 1046 additions and 520 deletions
Download patch file Download diff file Expand all files Collapse all files

189
README.org
View file

@ -6,13 +6,15 @@
[[https://melpa.org/#/org-ql][file:https://melpa.org/packages/org-ql-badge.svg]] [[https://stable.melpa.org/#/org-ql][file:https://stable.melpa.org/packages/org-ql-badge.svg]]
~org-ql~ is a lispy query language for Org files. It allows you to find Org entries matching certain criteria and return a list of them or perform actions on them. Commands are also provided which display matching results.
This package provides a query language for Org files. It offers two syntax styles: Lisp-like sexps and search engine-like keywords.
It includes three libraries: The =org-ql= library is flexible and may be used as a backend for other tools. The libraries =org-ql-search= and =helm-org-ql= provide interactive search commands and saved views.
* Contents
:PROPERTIES:
:TOC: this
:END:
- [[#examples][Examples]]
- [[#screenshots][Screenshots]]
- [[#installation][Installation]]
- [[#usage][Usage]]
- [[#commands][Commands]]
@ -21,77 +23,15 @@
- [[#changelog][Changelog]]
- [[#notes][Notes]]
* Examples
* Screenshots
More examples are available in [[examples.org]].
[[images/org-ql-search.gif]]
#+BEGIN_SRC elisp
;; Show entries that have any timestamp within the past week. Group
;; by date using `org-super-agenda' with the `:auto-ts' group.
(org-ql-search (org-agenda-files)
'(ts :from -7 :to today)
:title "Recent Items"
:sort '(date priority todo)
:groups '((:auto-ts t)))
[[images/org-ql-search-snippet.png]]
;; Show a GTD-style "stuck projects" view: PROJECT tasks that have no
;; descendants with the NEXT keyword. If you use a "project" tag
;; instead of the to-do keyword, you could replace (todo "PROJECT")
;; with (tags "project").
(org-ql-search (org-agenda-files)
'(and (todo "PROJECT")
(not (descendants (todo "NEXT"))))
:title "Stuck Projects")
[[images/helm-org-ql.gif]]
;; Integrate `org-ql' into a custom Org Agenda command which inserts
;; an `org-ql' block before the regular agenda:
(setq org-agenda-custom-commands
'(("ces" "Custom: Agenda and Emacs SOMEDAY [#A] items"
((org-ql-block '(and (todo "SOMEDAY")
(tags "Emacs")
(priority "A")))
(agenda)))))
;; Return a list of bills coming due, searching all Org Agenda files,
;; sorted by deadline. The `auto' argument to `deadline' means to match
;; entries whose deadlines fall within `org-deadline-warning-days'.
;; `org-ql-query' works like `org-ql-select' but offers arguments named
;; like SQL queries.
(org-ql-query
:select #'org-get-heading
:from (org-agenda-files)
:where '(and (not (done))
(tags "bills")
(deadline auto))
:order-by 'deadline)
;;=> ("TODO Electric bill" "TODO Water bill")
;; If you kept a database of music in an Org file, you could run a
;; query like this to find tracks composed by Chopin that do not have
;; their key recorded in the database. `org-ql-search' works like
;; `org-ql-select' and displays results in an agenda-like buffer:
(org-ql-search "~/org/music.org"
'(and (property "genre" "classical")
(property "composer" "Chopin")
(not (property "key"))))
;; Set the tag "Emacs" on every entry in the inbox file that mentions
;; "Emacs". `org-ql-select' works like `org-ql' but is a function
;; rather than a macro. The bare-string query "Emacs" is equivalent
;; to (regexp "Emacs").
(org-ql-select "~/org/inbox.org"
"Emacs"
:action '(org-toggle-tag "Emacs" 'on))
;; Return a list of Org entry elements in the file "~/org/main.org"
;; which have the SOMEDAY to-do keyword, are tagged "Emacs", and have
;; priority B or higher.
(org-ql "~/org/main.org"
(and (todo "SOMEDAY")
(tags "Emacs")
(priority >= "B")))
;;=> ((headline (:raw-value "org-board" :begin 1220270 :end 1220403 ...)) ...)
#+END_SRC
[[images/org-ql-view-sidebar.gif]]
* Installation
:PROPERTIES:
@ -100,9 +40,9 @@ More examples are available in [[examples.org]].
The package may be installed directly from [[https://melpa.org/#/org-ql][MELPA]] or with other tools like [[https://framagit.org/steckerhalter/quelpa][Quelpa]].
After installation, you can use commands like ~org-ql-search~ immediately.
After installation, you can use the commands without additional configuration. /Note: The command =helm-org-ql= only works if the package =helm-org= is installed; Helm is not a dependency of this package, so it's not automatically installed./
To use the functions and macros in your own Elisp code, load the libraries ~org-ql~ and/or ~org-ql-agenda~ with e.g. ~(require 'org-ql)~.
To use the functions and macros in your own Elisp code, use libraries =org-ql= and =org-ql-view=.
** Quelpa
@ -134,7 +74,9 @@ These commands and functions are included:
- ~org-ql-select~ (function)
- ~org-ql-query~ (function)
Feedback on these APIs is welcome. Eventually, after being tested and polished, they will be considered stable.
Feedback on these APIs is welcome. Eventually, after being tested and polished, they will be considered stable.
Lisp code examples are in [[examples.org]].
** Commands
:PROPERTIES:
@ -143,6 +85,8 @@ Feedback on these APIs is welcome. Eventually, after being tested and polished,
*** org-ql-search
/Note: This command supports both sexp queries and [[#non-sexp-query-syntax][non-sexp queries]]./
Read ~QUERY~ and search with ~org-ql~. Interactively, prompt for these variables:
~BUFFERS-FILES~: ~A~ list of buffers and/or files to search. Interactively, may also be:
@ -163,22 +107,16 @@ Read ~QUERY~ and search with ~org-ql~. Interactively, prompt for these variable
+ =g=: Refresh results.
+ =C-x C-s=: Save query to variable ~org-ql-views~ (accessible with command ~org-ql-view~).
[[images/org-ql-search.gif]]
Here's an example of using it to generate an agenda-like view for certain files in a directory tree:
[[images/org-ql-search-snippet.png]]
*Note:* The view buffer is currently put in ~org-agenda-mode~, which means that /some/ Org Agenda commands work, such as jumping to entries and changing item priorities (without necessarily updating the view). This feature is experimental and not guaranteed to work correctly with all commands. (It works to the extent it does because the appropriate text properties are placed on each item, imitating an Agenda buffer.)
*** helm-org-ql
/Note: This command uses [[#non-sexp-query-syntax][non-sexp queries]]./
This command displays matches with Helm. *Note:* Helm is not a package dependency, so this command only works if the package =helm-org= is installed.
Note also that queries in this command are specially handled so that quotes around strings may be omitted for ease of typing.
+ Press =C-x C-s= in the Helm session to save the results to an =org-ql-search= buffer.
[[images/helm-org-ql.gif]]
*** org-ql-view
Choose and display a view stored in ~org-ql-views~.
@ -187,8 +125,6 @@ Choose and display a view stored in ~org-ql-views~.
Show a sidebar window listing views stored in =org-ql-views= for easy access. In the sidebar, press =RET= or =mouse-1= to show the view at point, and press =c= to customize the view at point.
[[images/org-ql-view-sidebar.gif]]
*** org-ql-view-recent-items
Show items in ~FILES~ from last ~DAYS~ days with timestamps of ~TYPE~. ~TYPE~ may be ~ts~, ~ts-active~, ~ts-inactive~, ~clocked~, ~closed~, ~deadline~, ~planning~, or ~scheduled~. =FILES= defaults to those returned by the function =org-agenda-files=.
@ -201,11 +137,32 @@ Show a sparse tree for ~QUERY~ in ~BUFFER~ and return number of results. The tr
** Queries
A query is a lisp form which may contain arbitrary lisp forms, as well as certain built-in predicates. It is byte-compiled into a predicate function which is tested with point on each heading in an Org buffer; when it returns non-nil, the heading matches the query.
An =org-ql= query is a lisp form which may contain arbitrary lisp forms, as well as certain built-in predicates. It is byte-compiled into a predicate function which is tested with point on each heading in an Org buffer; when it returns non-nil, the heading matches the query.
*Notes:*
+ Bare strings like ~"string"~ are automatically converted to ~(regexp "string")~ predicates.
+ Standard numeric comparator function symbols (~<~, ~<=~, ~>~, ~>=~, ~=~ ) need not be quoted when passed as an argument to these predicates. The resemblance to infix notation is coincidental. See examples in documentation.
+ Standard numeric comparator function symbols (~<~, ~<=~, ~>~, ~>=~, ~=~ ) need not be quoted when passed as an argument to predicates which accept them. The resemblance to infix notation is coincidental.
*** Non-sexp query syntax
:PROPERTIES:
:TOC: ignore
:END:
The command =org-ql-search= also accepts, and the command =helm-org-ql= only accepts, an alternative, non-sexp query syntax. The syntax is simple, and a few examples of queries in both syntaxes should suffice. By default, when multiple predicates are used, they are combined with boolean =and=.
| Sexp syntax | Non-sexp syntax |
|-------------------------------------------------+-----------------------------------------|
| ~(todo)~ | ~todo:~ |
| ~(todo "SOMEDAY")~ | ~todo:SOMEDAY~ |
| ~(todo "SOMEDAY" "WAITING")~ | ~todo:SOMEDAY,WAITING~ |
| ~(ts :on today)~ | ~ts:on=today~ |
| ~(ts-active :from "2017-01-01" :to "2018-01-01")~ | ~ts-active:from=2017-01-01,to=2018-01-01~ |
| ~(clocked :on -1)~ | ~clocked:on=-1~ |
| ~(heading "quoted phrase" "word")~ | ~heading:"quoted phrase",word~ |
| ~(and (tags "book" "books") (priority "A"))~ | ~tags:book,books priority:A~ |
| ~(priority >= B)~ | ~priority:A,B~ |
Note that the =priority= predicate does not support comparators in the non-sexp syntax, so multiple priorities should be passed instead, as seen in the last example.
*** Predicates
:PROPERTIES:
@ -297,67 +254,6 @@ However, the ~org-ql-block~ version runs in about 1/5th the time.
The variable =org-ql-block-header= may be bound to a string to use as the block header, otherwise the header is formed automatically.
**** Macro: ~org-ql-agenda~
This macro is like ~org-ql~, but it presents matching entries in an Agenda-like view. It's compatible with [[https://github.com/alphapapa/org-super-agenda][org-super-agenda]], which provides grouping. For example:
#+BEGIN_SRC elisp
(org-ql-agenda "~/src/emacs/org-super-agenda/test/test.org"
(and (or (ts-active :on today)
(deadline auto)
(scheduled :to today))
(not (done)))
:title "My Agenda View"
;; The `org-super-agenda-groups' setting is used automatically when set, or it
;; may be overriden by specifying it here:
:super-groups ((:name "Bills"
:tag "bills")
(:todo ("SOMEDAY" "TO-READ" "CHECK" "TO-WATCH" "WATCHING")
:order 7)
(:name "Personal"
:habit t
:tag "personal"
:order 3)
(:todo "WAITING"
:order 6)
(:priority "A" :order 1)
(:priority "B" :order 2)
(:priority "C" :order 2)))
#+END_SRC
Which presents this buffer:
[[images/screenshot.png]]
*Note:* The view buffer is currently put in ~org-agenda-mode~, which means that /some/ Org Agenda commands work, such as jumping to entries and changing item priorities (without necessarily updating the view). This feature is experimental and not guaranteed to work correctly with all commands. (It works to the extent it does because the appropriate text properties are placed on each item, imitating an Agenda buffer.)
Here are some other examples:
#+BEGIN_SRC elisp
;; Show an agenda-like view of items in "~/org/main.org" with TODO and
;; SOMEDAY keywords which are tagged "computer" or "Emacs" and in the
;; category "main":
(org-ql-agenda "~/org/main.org"
(and (todo "TODO" "SOMEDAY")
(tags "computer" "Emacs")
(category "main")))
;; Show an agenda-like view of all habits in all agenda files:
(org-ql-agenda
(habit))
;; Show an agenda-like view similar to a "traditional" Org Agenda with
;; Log Mode turned on.
(org-ql-agenda
(or (and (not (done))
(or (habit)
(deadline auto)
(scheduled :to today)
(ts-active :on today)))
(closed :on today))
:sort (date priority todo))
#+END_SRC
*** Listing / acting-on results
**** Function: ~org-ql-select~
@ -470,6 +366,7 @@ Expands into a call to ~org-ql-select~ with the same arguments. For convenience
** 0.3-pre
*Added*
+ Alternative, non-sexp query syntax for commands =org-ql-search= and =helm-org-ql=. See [[#non-sexp-query-syntax][documentation]].
+ Command =helm-org-ql=.
+ Command =org-ql-sparse-tree=, like =org-sparse-tree= for =org-ql= queries. (Thanks to [[https://github.com/akirak][Akira Komamura]].)
+ Command =org-ql-view-sidebar=.