WIP: Allow "with-time=" in string queries

README.org

@@ -244,29 +244,30 @@ Arguments are listed next to predicate names, where applicable.
 
 *** Date/time predicates
 
-All of these predicates take optional keyword arguments ~:from~, ~:to:~, and ~:on~:
+These predicates take optional keyword arguments:
 
-+  If ~:from~, return non-nil if entry has a timestamp on or after ~:from~.
++  ~:from~: Match entries whose timestamp is on or after timestamp ~:from~.
-+  If ~:to~, return non-nil if entry has a timestamp on or before ~:to~.
++  ~:to~: Match entries whose timestamp is on or before timestamp ~:to~.
-+  If ~:on~, return non-nil if entry has a timestamp on date ~:on~.
++  ~:on~: Match entries whose timestamp is on date ~:on~.
++  ~:with-time~: If unspecified, match timestamps with or without times (i.e. HH:MM).  If nil, match timestamps without times.  If t, match timestamps with times.
 
-Argument values should be either a number of days (positive to look forward, or negative to look backward), a ~ts~ struct, or a string parseable by ~parse-time-string~ (the string may omit the time value).
+Timestamp/date arguments should be either a number of days (positive to look forward, or negative to look backward), a string parseable by ~parse-time-string~ (the string may omit the time value), or a ~ts~ struct.
 
 +  *Predicates*
-     -  =ts= :: Return non-nil if current entry has a timestamp in given period.  If no arguments are specified, return non-nil if entry has any timestamp.
+     -  =ts= :: Return non-nil if current entry has a timestamp in given period.  Without arguments, return non-nil if entry has a timestamp.
      -  =ts-active=, =ts-a= :: Like =ts=, but only matches active timestamps.
      -  =ts-inactive=, =ts-i= :: Like =ts=, but only matches inactive timestamps.
 
 The following predicates, in addition to the keyword arguments, can also take a single argument, a number, which looks backward or forward a number of days.  The number can be negative to invert the direction.
 
 +  *Backward-looking*
-     -  =clocked= :: Return non-nil if current entry was clocked in given period.  If no arguments are specified, return non-nil if entry was clocked at any time.  Note: Clock entries are expected to be clocked out.  Currently clocked entries (i.e. with unclosed timestamp ranges) are ignored.
+     -  =clocked= :: Return non-nil if current entry was clocked in given period.  Without arguments, return non-nil if entry was ever clocked.  Note: Clock entries are expected to be clocked out.  Currently clocked entries (i.e. with unclosed timestamp ranges) are ignored.
-     -  =closed= :: Return non-nil if current entry was closed in given period.  If no arguments are specified, return non-nil if entry was closed at any time.
+     -  =closed= :: Return non-nil if current entry was closed in given period.  Without arguments, return non-nil if entry is closed.
 
 +  *Forward-looking*
-     -  =deadline= :: Return non-nil if current entry has deadline in given period.  If argument is =auto=, return non-nil if entry has deadline within =org-deadline-warning-days=.  If no arguments are specified, return non-nil if entry has any deadline.
+     -  =deadline= :: Return non-nil if current entry has deadline in given period.  If argument is =auto=, return non-nil if entry has deadline within =org-deadline-warning-days=.  Without arguments, return non-nil if entry has any deadline.
-     -  =planning= :: Return non-nil if current entry has planning timestamp in given period (i.e. its deadline, scheduled, or closed timestamp).  If no arguments are specified, return non-nil if entry is scheduled at any time.
+     -  =planning= :: Return non-nil if current entry has planning timestamp (i.e. its deadline, scheduled, or closed timestamp) in given period.  Without arguments, return non-nil if entry has any planning timestamp.
-     -  =scheduled= :: Return non-nil if current entry is scheduled in given period.  If no arguments are specified, return non-nil if entry is scheduled at any time.
+     -  =scheduled= :: Return non-nil if current entry is scheduled in given period.  Without arguments, return non-nil if entry is scheduled.
 
 ** Functions / Macros
 :PROPERTIES:
@@ -523,6 +524,7 @@ Simple links may also be written manually in either sexp or non-sexp form, like:
 +  Macro =org-ql-defpred=, used to define search predicates.  (See [[file:examples/defpred.org][tutorial]].)
 +  Predicate ~effort~.
 +  Predicate ~heading-regexp~, which matches regular expressions against heading text (alias: ~h*~).
++  Timestamp-related predicates now accept an optional ~:with-time~ argument, which allows matching timestamps with or without times (i.e. HH:MM).
 
 *Changed*
 +  Helm support (including the command =helm-org-ql=) has been moved to a separate package, =helm-org-ql=.

org-ql.el

@@ -130,6 +130,147 @@ the value returned by it at that node.")
     "Plist of predicates, their corresponding functions, and their docstrings.
 This list should not contain any duplicates."))
 
+;;;;; Timestamp regexps
+
+;; We need more specificity than the built-in Org timestamp regexps
+;; provide, and sometimes they change from version to version, so we
+;; define our own.  And by defining them with `rx', they are much
+;; easier to understand than the string-based ones in org.el (of
+;; course, `rx' probably wasn't available when most of those were
+;; written).
+
+;; MAYBE: Use newer `rx' custom expressions to define these.
+;; MAYBE: Add match groups corresponding to the ones in the "official" Org regexps.
+;; TODO: Use these new regexps in more places.
+
+(defvar org-ql-regexp-part-ts-date
+  (rx (repeat 4 digit) "-" (repeat 2 digit) "-" (repeat 2 digit)
+      ;; Day of week
+      (optional " " (1+ alpha)))
+  "Matches the inner, date part of an Org timestamp, both active and inactive.
+Also matches optional day-of-week.  Used to build other timestamp
+regexps.")
+
+(defvar org-ql-regexp-part-ts-repeaters
+  ;; Repeaters (not sure if the colon is necessary, but it's in the org.el one)
+  (rx (repeat 1 2 (seq " " (repeat 1 2 (any "-+:")) (1+ digit) (any "hdwmy"))))
+  "Matches the repeater part of an Org timestamp.
+Includes leading space character.")
+
+(defvar org-ql-regexp-part-ts-time
+  (rx " " (repeat 1 2 digit) ":" (repeat 2 digit))
+  "Matches the inner, time part of an Org timestamp (i.e. HH:MM).
+Includes leading space character.  Used to build other timestamp
+regexps.")
+
+;; NOTE: The inactive timestamp regexps don't allow repeaters.  I don't know if this is
+;; officially correct, but it seems to make sense, and would be easy to change if necessary.
+
+(defvar org-ql-regexp-ts-both
+  (rx-to-string
+   `(or (seq "<" (regexp ,org-ql-regexp-part-ts-date)
+             (optional (regexp ,org-ql-regexp-part-ts-time))
+             (optional (regexp ,org-ql-regexp-part-ts-repeaters)) ">")
+        (seq "[" (regexp ,org-ql-regexp-part-ts-date)
+             (optional (regexp ,org-ql-regexp-part-ts-time)))))
+  "Matches both active and inactive Org timestamps, with or without time.")
+
+(defvar org-ql-regexp-ts-both-with-time
+  (rx-to-string `(or (seq "<" (regexp ,org-ql-regexp-part-ts-date)
+                          (regexp ,org-ql-regexp-part-ts-time)
+                          (optional (regexp ,org-ql-regexp-part-ts-repeaters)) ">")
+                     (seq "[" (regexp ,org-ql-regexp-part-ts-date)
+                          (regexp ,org-ql-regexp-part-ts-time) "]")))
+  "Matches both active and inactive Org timestamps, with time.")
+
+(defvar org-ql-regexp-ts-both-without-time
+  (rx-to-string `(or (seq "<" (regexp ,org-ql-regexp-part-ts-date)
+                          (optional (regexp ,org-ql-regexp-part-ts-repeaters)) ">")
+                     (seq "[" (regexp ,org-ql-regexp-part-ts-date) "]")))
+  "Matches both active and inactive Org timestamps, without time.")
+
+(defvar org-ql-regexp-ts-active
+  (rx-to-string `(seq "<" (regexp ,org-ql-regexp-part-ts-date)
+                      (optional (regexp ,org-ql-regexp-part-ts-time))
+                      (optional (regexp ,org-ql-regexp-part-ts-repeaters)) ">"))
+  "Matches active Org timestamps, with or without time.")
+
+(defvar org-ql-regexp-ts-active-with-time
+  (rx-to-string `(seq "<" (regexp ,org-ql-regexp-part-ts-date)
+                      (regexp ,org-ql-regexp-part-ts-time)
+                      (optional (regexp ,org-ql-regexp-part-ts-repeaters)) ">"))
+  "Matches active Org timestamps, with time.")
+
+(defvar org-ql-regexp-ts-active-without-time
+  (rx-to-string `(seq "<" (regexp ,org-ql-regexp-part-ts-date)
+                      (optional (regexp ,org-ql-regexp-part-ts-repeaters)) ">"))
+  "Matches active Org timestamps, without time.")
+
+(defvar org-ql-regexp-ts-inactive
+  (rx-to-string `(seq "[" (regexp ,org-ql-regexp-part-ts-date)
+                      (optional (regexp ,org-ql-regexp-part-ts-time))"]"))
+  "Matches inactive Org timestamps, with or without time.")
+
+(defvar org-ql-regexp-ts-inactive-with-time
+  (rx-to-string `(seq "[" (regexp ,org-ql-regexp-part-ts-date)
+                      (regexp ,org-ql-regexp-part-ts-time)"]"))
+  "Matches inactive Org timestamps, with time.")
+
+(defvar org-ql-regexp-ts-inactive-without-time
+  (rx-to-string `(seq "[" (regexp ,org-ql-regexp-part-ts-date) "]"))
+  "Matches inactive Org timestamps, without time.")
+
+(defvar org-ql-regexp-planning
+  (rx-to-string `(seq bow (or (seq "CLOSED" ":" (0+ " ")
+                                   (group-n 1 (regexp ,org-ql-regexp-ts-inactive)))
+                              (seq (or "DEADLINE" "SCHEDULED") ":" (0+ " ")
+                                   (group-n 1 (regexp ,org-ql-regexp-ts-active))))))
+  "Matches CLOSED, DEADLINE or SCHEDULED keyword with timestamp, with or without time.")
+
+(defvar org-ql-regexp-planning-with-time
+  (rx-to-string `(seq bow (or (seq "CLOSED" ":" (0+ " ")
+                                   (group-n 1 (regexp ,org-ql-regexp-ts-inactive-with-time)))
+                              (seq (or "DEADLINE" "SCHEDULED") ":" (0+ " ")
+                                   (group-n 1 (regexp ,org-ql-regexp-ts-active-with-time))))))
+  "Matches CLOSED, DEADLINE or SCHEDULED keyword with timestamp, with time.")
+
+(defvar org-ql-regexp-planning-without-time
+  (rx-to-string `(seq bow (or (seq "CLOSED" ":" (0+ " ")
+                                   (group-n 1 (regexp ,org-ql-regexp-ts-inactive-without-time)))
+                              (seq (or "DEADLINE" "SCHEDULED") ":" (0+ " ")
+                                   (group-n 1 (regexp ,org-ql-regexp-ts-active-without-time))))))
+  "Matches CLOSED, DEADLINE or SCHEDULED keyword with timestamp, without time.")
+
+(defvar org-ql-regexp-deadline
+  (rx-to-string `(seq bow "DEADLINE" ":" (0+ " ")
+                      (group (regexp ,org-ql-regexp-ts-active))))
+  "Matches DEADLINE keyword with a time-and-hour stamp, with or without time.")
+
+(defvar org-ql-regexp-deadline-with-time
+  (rx-to-string `(seq bow "DEADLINE" ":" (0+ " ")
+                      (group (regexp ,org-ql-regexp-ts-active-with-time))))
+  "Matches DEADLINE keyword with a time-and-hour stamp, with time.")
+
+(defvar org-ql-regexp-deadline-without-time
+  (rx-to-string `(seq bow "DEADLINE" ":" (0+ " ")
+                      (group (regexp ,org-ql-regexp-ts-active-without-time))))
+  "Matches DEADLINE keyword with a time-and-hour stamp, without time.")
+
+(defvar org-ql-regexp-scheduled
+  (rx-to-string `(seq bow "SCHEDULED" ":" (0+ " ")
+                      (group (regexp ,org-ql-regexp-ts-active))))
+  "Matches SCHEDULED keyword with a time-and-hour stamp, with or without time.")
+
+(defvar org-ql-regexp-scheduled-with-time
+  (rx-to-string `(seq bow "SCHEDULED" ":" (0+ " ")
+                      (group (regexp ,org-ql-regexp-ts-active-with-time))))
+  "Matches SCHEDULED keyword with a time-and-hour stamp, with time.")
+
+(defvar org-ql-regexp-scheduled-without-time
+  (rx-to-string `(seq bow "SCHEDULED" ":" (0+ " ")
+                      (group (regexp ,org-ql-regexp-ts-active-without-time))))
+  "Matches SCHEDULED keyword with a time-and-hour stamp, without time.")
+
 ;;;; Customization
 
 (defgroup org-ql nil
@@ -536,6 +677,12 @@ returns nil."
                                    query-string " Execute it? "))
         (user-error "Query aborted by user")))))
 
+(defun org-ql--plist-get* (plist property)
+  "Return the value of PROPERTY in PLIST, or `not-found' if the property is missing."
+  (if-let ((pair (plist-member plist property)))
+      (cadr pair)
+    'not-found))
+
 ;;;;; Query processing
 
 ;; Processing, compiling, etc. for queries.
@@ -637,28 +784,57 @@ Arguments STRING, POS, FILL, and LEVEL are according to
   (let ((byte-compile-log-warning-function #'org-ql--byte-compile-warning))
     (byte-compile
      `(lambda ()
+        ;; NOTE: `clocked' and `closed' don't have WITH-TIME args, because they should always have a time.
+        ;; TODO: If possible, all of this argument processing should be done in each predicate's normalizers.
+        ;; NOTE: The pcases check for both t/nil symbols and strings, because the
+        ;; string queries always return keyword arguments' values as strings.
         (cl-macrolet ((clocked (&key from to on)
                                (org-ql--from-to-on)
                                `(org-ql--predicate-clocked :from ,from :to ,to))
-                      (closed (&key from to on)
+                      (closed (&key from to on (with-time 'not-found))
                               (org-ql--from-to-on)
                               `(org-ql--predicate-closed :from ,from :to ,to))
-                      (deadline (&key from to on)
+                      (deadline (&key from to on (with-time 'not-found))
                                 (org-ql--from-to-on)
-                                `(org-ql--predicate-deadline :from ,from :to ,to))
+                                `(org-ql--predicate-deadline
-                      (planning (&key from to on)
+                                  :from ,from :to ,to :with-time ',with-time
+                                  :regexp ,(pcase-exhaustive with-time
+                                             ((or 't "t") org-ql-regexp-deadline-with-time)
+                                             ((or 'nil "nil") org-ql-regexp-deadline-without-time)
+                                             ('not-found org-ql-regexp-deadline))))
+                      (planning (&key from to on (with-time 'not-found))
                                 (org-ql--from-to-on)
-                                `(org-ql--predicate-planning :from ,from :to ,to))
+                                `(org-ql--predicate-planning
-                      (scheduled (&key from to on)
+                                  :from ,from :to ,to :with-time ',with-time
+                                  :regexp ,(pcase-exhaustive with-time
+                                             ((or 't "t") org-ql-regexp-planning-with-time)
+                                             ((or 'nil "nil") org-ql-regexp-planning-without-time)
+                                             ('not-found org-ql-regexp-planning))))
+                      (scheduled (&key from to on (with-time 'not-found))
                                  (org-ql--from-to-on)
-                                 `(org-ql--predicate-scheduled :from ,from :to ,to))
+                                 `(org-ql--predicate-scheduled
-                      (ts (&key from to on (type 'both))
+                                   :from ,from :to ,to :with-time ',with-time
+                                   :regexp ,(pcase-exhaustive with-time
+                                              ((or 't "t") org-ql-regexp-scheduled-with-time)
+                                              ((or 'nil "nil") org-ql-regexp-scheduled-without-time)
+                                              ('not-found org-ql-regexp-scheduled))))
+                      (ts (&key from to on (type 'both) (with-time 'not-found))
                           (org-ql--from-to-on)
-                          `(org-ql--predicate-ts :from ,from :to ,to
+                          `(org-ql--predicate-ts
-                                                 :regexp ,(pcase type
+                            :from ,from :to ,to :with-time ',with-time
-                                                            ('both org-tsr-regexp-both)
+                            :regexp ,(pcase type
-                                                            ('active org-tsr-regexp)
+                                       ((or 'nil 'both) (pcase-exhaustive with-time
-                                                            ('inactive org-ql-tsr-regexp-inactive)))))
+                                                          ((or 't "t") org-ql-regexp-ts-both-with-time)
+                                                          ((or 'nil "nil") org-ql-regexp-ts-both-without-time)
+                                                          ('not-found org-ql-regexp-ts-both)))
+                                       ('active (pcase-exhaustive with-time
+                                                  ((or 't "t") org-ql-regexp-ts-active-with-time)
+                                                  ((or 'nil "nil") org-ql-regexp-ts-active-without-time)
+                                                  ('not-found org-ql-regexp-ts-active)))
+                                       ('inactive (pcase-exhaustive with-time
+                                                    ((or 't "t") org-ql-regexp-ts-inactive-with-time)
+                                                    ((or 'nil "nil") org-ql-regexp-ts-inactive-without-time)
+                                                    ('not-found org-ql-regexp-ts-inactive)))))))
           ,query)))))
 
 ;;;;; String query parsing
@@ -1023,6 +1199,7 @@ For compatibility, since Org 9.1 deprecated
 `org-duration-string-to-minutes', replacing it with
 `org-duration-to-minutes', which seems to return floats instead
 of integers."
+  ;; FIXME: Define this as an alias instead.
   ;; MAYBE: Remove if compatibility with Org 9.0 is dropped.
   (funcall (if (fboundp 'org-duration-to-minutes)
                #'org-duration-to-minutes
@@ -1671,34 +1848,41 @@ With KEYWORDS, return non-nil if its keyword is one of KEYWORDS (a list of strin
 
 ;;;;;; Timestamps
 
-;; TODO: Remove the _on vars from these arg lists.  I think they're not
+;; NOTE: The underscores before some arguments in these definitions
+;; prevent "unused lexical variable" warnings, because we pre-process
+;; them before the functions are called.
+
+;; TODO: Remove the _underscored vars from these arg lists.  I think they're not
 ;; necessary, or shouldn't be, since --pre-process-query should handle them.
 
-;; NOTE: These docstrings apply to the functions defined by `org-ql--defpref',
+;; NOTE: Arguments to these predicates are pre-processed in
-;; not necessarily to the way users are expected to call them in queries.  The
+;; `org-ql--normalize-query' and `org-ql--query-predicate'.  Some
-;; queries are pre-processed by `org-ql--normalize-query' to handle
+;; arguments are not to be given by the user in a query,
-;; arguments which are constant during a query's execution.
+;; e.g. `regexp'.  FROM and TO are actually expected to be `ts'
+;; structs.  However, the docstrings are written for users, which
+;; makes documentation easier to update.
 
 ;; TODO: Update the macro to define a user-facing docstring so I don't
 ;; have to manually update the documentation.
 
-(org-ql-defpred clocked (&key from to _on)
+;; This string is common to these predicates and is used in
-  ;; The underscore before `on' prevents "unused lexical variable"
+;; documentation; keeping it here should make it easier to update:
-  ;; warnings, because we pre-process that argument in a macro before
+"If FROM, return non-nil if entry's timestamp is on or after FROM.
-  ;; this function is called.
-  "Return non-nil if current entry was clocked in given period.
-If no arguments are specified, return non-nil if entry has any
-timestamp.
 
-If FROM, return non-nil if entry has a timestamp on or after
+If TO, return non-nil if entry's timestamp is on or before TO.
-FROM.
 
-If TO, return non-nil if entry has a timestamp on or before TO.
+If ON, return non-nil if entry's timestamp is on date ON.
-
-If ON, return non-nil if entry has a timestamp on date ON.
 
 FROM, TO, and ON should be either `ts' structs, or strings
 parseable by `parse-time-string' which may omit the time value."
+
+(org-ql-defpred clocked (&key from to _on)
+  "Return non-nil if current entry was clocked in given period.
+Without arguments, return non-nil if entry was ever clocked.
+Note: Clock entries are expected to be clocked out.  Currently
+clocked entries (i.e. with unclosed timestamp ranges) are
+ignored."
+  ;; TODO: Verify that currently clocked entries are still ignored.
   :normalizers ((`(,predicate-names ,(and num-days (pred numberp)))
                  ;; (clocked) and (closed) implicitly look into the past.
                  (let ((from (->> (ts-now)
@@ -1713,22 +1897,9 @@ parseable by `parse-time-string' which may omit the time value."
   (org-ql--predicate-ts :from from :to to :regexp org-ql-clock-regexp :match-group 1))
 
 (org-ql-defpred closed (&key from to _on)
-  ;; The underscore before `on' prevents "unused lexical variable"
+  ;; MAYBE: Use the new org-ql-regexps?
-  ;; warnings, because we pre-process that argument in a macro before
-  ;; this function is called.
   "Return non-nil if current entry was closed in given period.
-If no arguments are specified, return non-nil if entry has any
+Without arguments, return non-nil if entry is closed."
-timestamp.
-
-If FROM, return non-nil if entry has a timestamp on or after
-FROM.
-
-If TO, return non-nil if entry has a timestamp on or before TO.
-
-If ON, return non-nil if entry has a timestamp on date ON.
-
-FROM, TO, and ON should be either `ts' structs, or strings
-parseable by `parse-time-string' which may omit the time value."
   :normalizers ((`(,predicate-names ,(and num-days (pred numberp)))
                  ;; (clocked) and (closed) implicitly look into the past.
                  (let ((from (->> (ts-now)
@@ -1742,23 +1913,11 @@ parseable by `parse-time-string' which may omit the time value."
   (org-ql--predicate-ts :from from :to to :regexp org-closed-time-regexp :match-group 1
                         :limit (line-end-position 2)))
 
-(org-ql-defpred deadline (&key from to _on)
+(org-ql-defpred deadline (&key from to _on regexp _with-time)
-  ;; The underscore before `on' prevents "unused lexical variable"
-  ;; warnings, because we pre-process that argument in a macro before
-  ;; this function is called.
   "Return non-nil if current entry has deadline in given period.
-If no arguments are specified, return non-nil if entry has any
+If argument is `auto', return non-nil if entry has deadline
-timestamp.
+within `org-deadline-warning-days'.  Without arguments, return
-
+non-nil if entry has a deadline."
-If FROM, return non-nil if entry has a timestamp on or after
-FROM.
-
-If TO, return non-nil if entry has a timestamp on or before TO.
-
-If ON, return non-nil if entry has a timestamp on date ON.
-
-FROM, TO, and ON should be either `ts' structs, or strings
-parseable by `parse-time-string' which may omit the time value."
   :normalizers ((`(,predicate-names auto)
                  ;; Use `org-deadline-warning-days' as the :to arg.
                  (let ((to (->> (ts-now)
@@ -1770,15 +1929,22 @@ parseable by `parse-time-string' which may omit the time value."
                              (ts-adjust 'day num-days)
                              (ts-apply :hour 23 :minute 59 :second 59))))
                    `(deadline :to ,to))))
-  ;; NOTE: Does this normalizer cause the preamble to not be used?  (Adding one to the deadline-warning definition to be sure.)
+  ;; NOTE: Does this normalizer cause the preamble to not be used?
-  :preambles ((`(,predicate-names . ,_)
+  ;; (Adding one to the deadline-warning definition to be sure.)
-               (list :regexp org-deadline-time-regexp :query query)))
+  :preambles ((`(,predicate-names . ,rest)
+               (list :query query
+                     :regexp (pcase-exhaustive (org-ql--plist-get* rest :with-time)
+                               ((or 't "t") org-ql-regexp-deadline-with-time)
+                               ((or 'nil "nil") org-ql-regexp-deadline-without-time)
+                               ('not-found org-ql-regexp-deadline)))))
   :body
-  (org-ql--predicate-ts :from from :to to :regexp org-deadline-time-regexp :match-group 1
+  (org-ql--predicate-ts :from from :to to :regexp regexp :match-group 1
                         :limit (line-end-position 2)))
 
 (org-ql-defpred deadline-warning (&key from to)
-  "Internal selector used to handle `org-deadline-warning-days' and deadlines with warning periods."
+  ;; TODO: Should this also accept a WITH-TIME argument?
+  ;; MAYBE: Use the new org-ql-regexps?
+  "Internal predicate used to handle `org-deadline-warning-days' and deadlines with warning periods."
   :preambles ((`(,predicate-names . ,_)
                (list :regexp org-deadline-time-regexp :query query)))
   :body
@@ -1805,82 +1971,49 @@ parseable by `parse-time-string' which may omit the time value."
            (ts<= (->> ts (ts-adjust unit (- warning-value))) org-ql--today))
           ('week (ts<= (->> ts (ts-adjust 'day (* -7 warning-value))) org-ql--today)))))))
 
-(org-ql-defpred planning (&key from to _on)
+(org-ql-defpred planning (&key from to _on regexp _with-time)
-  ;; The underscore before `on' prevents "unused lexical variable"
+  "Return non-nil if current entry has planning timestamp in given period.
-  ;; warnings, because we pre-process that argument in a macro before
+Without arguments, return non-nil if entry has any planning timestamp."
-  ;; this function is called.
-  "Return non-nil if current entry has planning timestamp in given period (i.e. its deadline, scheduled, or closed timestamp).
-If no arguments are specified, return non-nil if entry has any
-timestamp.
-
-If FROM, return non-nil if entry has a timestamp on or after
-FROM.
-
-If TO, return non-nil if entry has a timestamp on or before TO.
-
-If ON, return non-nil if entry has a timestamp on date ON.
-
-FROM, TO, and ON should be either `ts' structs, or strings
-parseable by `parse-time-string' which may omit the time value."
   :normalizers ((`(,predicate-names ,(and num-days (pred numberp)))
                  (let ((to (->> (ts-now)
                              (ts-adjust 'day num-days)
                              (ts-apply :hour 23 :minute 59 :second 59))))
                    `(planning :to ,to))))
-  :preambles ((`(,predicate-names . ,_)
+  :preambles ((`(,predicate-names . ,rest)
-               (list :regexp org-ql-planning-regexp :query query)))
+               (list :query query
+                     :regexp (pcase-exhaustive (org-ql--plist-get* rest :with-time)
+                               ((or 't "t") org-ql-regexp-planning-with-time)
+                               ((or 'nil "nil") org-ql-regexp-planning-without-time)
+                               ('not-found org-ql-regexp-planning)))))
+  ;; MAYBE: Should the regexp be done in the normalizer instead?  (If
+  ;; so, also in other ts-related predicates.)
   :body
-  (org-ql--predicate-ts :from from :to to :regexp org-ql-planning-regexp :match-group 1
+  (org-ql--predicate-ts :from from :to to :regexp regexp :match-group 1
                         :limit (line-end-position 2)))
 
-(org-ql-defpred scheduled (&key from to _on)
+(org-ql-defpred scheduled (&key from to _on regexp _with-time)
-  ;; The underscore before `on' prevents "unused lexical variable"
-  ;; warnings, because we pre-process that argument in a macro before
-  ;; this function is called.
   "Return non-nil if current entry is scheduled in given period.
-If no arguments are specified, return non-nil if entry has any
+Without arguments, return non-nil if entry is scheduled."
-timestamp.
-
-If FROM, return non-nil if entry has a timestamp on or after
-FROM.
-
-If TO, return non-nil if entry has a timestamp on or before TO.
-
-If ON, return non-nil if entry has a timestamp on date ON.
-
-FROM, TO, and ON should be either `ts' structs, or strings
-parseable by `parse-time-string' which may omit the time value."
   :normalizers ((`(,predicate-names ,(and num-days (pred numberp)))
                  (let ((to (->> (ts-now)
                              (ts-adjust 'day num-days)
                              (ts-apply :hour 23 :minute 59 :second 59))))
                    `(scheduled :to ,to))))
-  :preambles ((`(,predicate-names . ,_)
+  :preambles ((`(,predicate-names . ,rest)
-               (list :regexp org-scheduled-time-regexp :query query)))
+               (list :query query
+                     :regexp (pcase-exhaustive (org-ql--plist-get* rest :with-time)
+                               ((or 't "t") org-ql-regexp-scheduled-with-time)
+                               ((or 'nil "nil") org-ql-regexp-scheduled-without-time)
+                               ('not-found org-ql-regexp-scheduled)))))
   :body
-  (org-ql--predicate-ts :from from :to to :regexp org-scheduled-time-regexp :match-group 1
+  (org-ql--predicate-ts :from from :to to :regexp regexp :match-group 1
                         :limit (line-end-position 2)))
 
 (org-ql-defpred (ts ts-active ts-a ts-inactive ts-i)
-  (&key from to _on regexp (match-group 0) (limit (org-entry-end-position)))
+  (&key from to _on regexp _with-time
-  ;; NOTE: Arguments to this predicate are pre-processed in `org-ql--normalize-query'.
+        (match-group 0) (limit (org-entry-end-position)))
-  ;; The underscore before `on' prevents "unused lexical variable" warnings due to the
-  ;; pre-processing converting that argument to FROM and TO.  The `regexp' argument is
-  ;; also provided by the pre-processing and is not to be given by the user.  FROM and
-  ;; TO are actually expected to be `ts' structs.  The docstring is written for users.
   "Return non-nil if current entry has a timestamp in given period.
-If no arguments are specified, return non-nil if entry has any
+Without arguments, return non-nil if entry has a timestamp.
-timestamp.
-
-If FROM, return non-nil if entry has a timestamp on or after
-FROM.
-
-If TO, return non-nil if entry has a timestamp on or before TO.
-
-If ON, return non-nil if entry has a timestamp on date ON.
-
-FROM, TO, and ON should be either `ts' structs, or strings
-parseable by `parse-time-string' which may omit the time value.
 
 TYPE may be `active' to match active timestamps, `inactive' to
 match inactive ones, or `both' / nil to match both types.
@@ -1888,24 +2021,39 @@ match inactive ones, or `both' / nil to match both types.
 LIMIT bounds the search for the timestamp REGEXP.  It defaults to
 the end of the entry, i.e. the position returned by
 `org-entry-end-position', but for certain searches it should be
-bound to a different positiion, e.g. for planning lines, the end
+bound to a different positiion (e.g. for planning lines, the end
-of the line after the heading."
+of the line after the heading).  MATCH-GROUP should be the number
+of REGEXP's group that matches the Org timestamp (i.e. excluding
+any planning prefix); it defaults to 0 (i.e. the whole regexp)."
   ;; MAYBE: Define active/inactive ones separately?
-  :normalizers ((`(,(or 'ts-active 'ts-a) . ,rest) `(ts :type active ,@rest))
+  :normalizers
-                (`(,(or 'ts-inactive 'ts-i) . ,rest) `(ts :type inactive ,@rest)))
+  ((`(,(or 'ts-active 'ts-a) . ,rest) `(ts :type active ,@rest))
-  :preambles ((`(,predicate-names . ,rest)
+   (`(,(or 'ts-inactive 'ts-i) . ,rest) `(ts :type inactive ,@rest)))
-               (list :regexp (pcase (plist-get rest :type)
+
-                               ((or 'nil 'both) org-tsr-regexp-both)
+  :preambles
-                               ('active org-tsr-regexp)
+  ((`(,predicate-names . ,rest)
-                               ('inactive org-ql-tsr-regexp-inactive))
+    (list :regexp (pcase (plist-get rest :type)
-                     ;; Predicate needs testing only when args are present.
+                    ((or 'nil 'both) (pcase-exhaustive (org-ql--plist-get* rest :with-time)
-                     :query (-let (((&keys :from :to :on) rest))
+                                       ((or 't "t") org-ql-regexp-ts-both-with-time)
-                              ;; FIXME: This used to be (when (or from to on) query), but that doesn't seem right, so I
+                                       ((or 'nil "nil") org-ql-regexp-ts-both-without-time)
-                              ;; changed it to this if, and the tests pass either way.  Might deserve a little scrutiny.
+                                       ('not-found org-ql-regexp-ts-both)))
-                              (if (or from to on)
+                    ('active (pcase-exhaustive (org-ql--plist-get* rest :with-time)
-                                  query
+                               ((or 't "t") org-ql-regexp-ts-active-with-time)
-                                t)))))
+                               ((or 'nil "nil") org-ql-regexp-ts-active-without-time)
-  ;; TODO: DRY this with the clocked predicate.
+                               ('not-found org-ql-regexp-ts-active)))
+                    ('inactive (pcase-exhaustive (org-ql--plist-get* rest :with-time)
+                                 ((or 't "t") org-ql-regexp-ts-inactive-with-time)
+                                 ((or 'nil "nil") org-ql-regexp-ts-inactive-without-time)
+                                 ('not-found org-ql-regexp-ts-inactive))))
+          ;; Predicate needs testing only when args are present.
+          :query (-let (((&keys :from :to :on) rest))
+                   ;; TODO: This used to be (when (or from to on) query), but
+                   ;; that doesn't seem right, so I changed it to this if, and the
+                   ;; tests pass either way.  Might deserve a little scrutiny.
+                   (if (or from to on)
+                       query
+                     t)))))
+
   :body
   (cl-macrolet ((next-timestamp ()
                                 `(when (re-search-forward regexp limit t)

org-ql.info

@@ -530,24 +530,27 @@ File: README.info,  Node: Date/time predicates,  Prev: Ancestor/descendant predi
 4.2.4 Date/time predicates
 --------------------------
 
-All of these predicates take optional keyword arguments ‘:from’, ‘:to:’,
+These predicates take optional keyword arguments:
-and ‘:on’:
 
-   • If ‘:from’, return non-nil if entry has a timestamp on or after
+   • ‘:from’: Match entries whose timestamp is on or after timestamp
      ‘:from’.
-   • If ‘:to’, return non-nil if entry has a timestamp on or before
+   • ‘:to’: Match entries whose timestamp is on or before timestamp
      ‘:to’.
-   • If ‘:on’, return non-nil if entry has a timestamp on date ‘:on’.
+   • ‘:on’: Match entries whose timestamp is on date ‘:on’.
+   • ‘:with-time’: If unspecified, match timestamps with or without
+     times (i.e.  HH:MM). If nil, match timestamps without times.  If t,
+     match timestamps with times.
 
-   Argument values should be either a number of days (positive to look
+   Timestamp/date arguments should be either a number of days (positive
-forward, or negative to look backward), a ‘ts’ struct, or a string
+to look forward, or negative to look backward), a string parseable by
-parseable by ‘parse-time-string’ (the string may omit the time value).
+‘parse-time-string’ (the string may omit the time value), or a ‘ts’
+struct.
 
    • *Predicates*
      ‘ts’
           Return non-nil if current entry has a timestamp in given
-          period.  If no arguments are specified, return non-nil if
+          period.  Without arguments, return non-nil if entry has a
-          entry has any timestamp.
+          timestamp.
      ‘ts-active, ts-a’
           Like ts, but only matches active timestamps.
      ‘ts-inactive, ts-i’
@@ -560,30 +563,28 @@ number of days.  The number can be negative to invert the direction.
    • *Backward-looking*
      ‘clocked’
           Return non-nil if current entry was clocked in given period.
-          If no arguments are specified, return non-nil if entry was
+          Without arguments, return non-nil if entry was ever clocked.
-          clocked at any time.  Note: Clock entries are expected to be
+          Note: Clock entries are expected to be clocked out.  Currently
-          clocked out.  Currently clocked entries (i.e.  with unclosed
+          clocked entries (i.e.  with unclosed timestamp ranges) are
-          timestamp ranges) are ignored.
+          ignored.
      ‘closed’
           Return non-nil if current entry was closed in given period.
-          If no arguments are specified, return non-nil if entry was
+          Without arguments, return non-nil if entry is closed.
-          closed at any time.
 
    • *Forward-looking*
      ‘deadline’
           Return non-nil if current entry has deadline in given period.
           If argument is auto, return non-nil if entry has deadline
-          within org-deadline-warning-days.  If no arguments are
+          within org-deadline-warning-days.  Without arguments, return
-          specified, return non-nil if entry has any deadline.
+          non-nil if entry has any deadline.
      ‘planning’
-          Return non-nil if current entry has planning timestamp in
+          Return non-nil if current entry has planning timestamp (i.e.
-          given period (i.e.  its deadline, scheduled, or closed
+          its deadline, scheduled, or closed timestamp) in given period.
-          timestamp).  If no arguments are specified, return non-nil if
+          Without arguments, return non-nil if entry has any planning
-          entry is scheduled at any time.
+          timestamp.
      ‘scheduled’
           Return non-nil if current entry is scheduled in given period.
-          If no arguments are specified, return non-nil if entry is
+          Without arguments, return non-nil if entry is scheduled.
-          scheduled at any time.
 
 
 File: README.info,  Node: Functions / Macros,  Next: Dynamic block,  Prev: Queries,  Up: Usage
@@ -981,6 +982,9 @@ File: README.info,  Node: 06-pre,  Next: 052,  Up: Changelog
    • Predicate ‘effort’.
    • Predicate ‘heading-regexp’, which matches regular expressions
      against heading text (alias: ‘h*’).
+   • Timestamp-related predicates now accept an optional ‘:with-time’
+     argument, which allows matching timestamps with or without times
+     (i.e.  HH:MM).
 
    *Changed*
    • Helm support (including the command helm-org-ql) has been moved to
@@ -1534,40 +1538,40 @@ Node: Non-sexp query syntax9485
 Node: General predicates11209
 Node: Ancestor/descendant predicates17430
 Node: Date/time predicates18558
-Node: Functions / Macros21213
+Node: Functions / Macros21220
-Node: Agenda-like views21511
+Node: Agenda-like views21518
-Node: Listing / acting-on results22916
+Node: Listing / acting-on results22923
-Node: Custom predicates28538
+Node: Custom predicates28545
-Node: Dynamic block32029
+Node: Dynamic block32036
-Node: Links34727
+Node: Links34734
-Node: Tips35414
+Node: Tips35421
-Node: Changelog35732
+Node: Changelog35739
-Node: 06-pre36458
+Node: 06-pre36465
-Node: 05237865
+Node: 05238038
-Node: 05138169
+Node: 05138342
-Node: 0538586
+Node: 0538759
-Node: 04940060
+Node: 04940233
-Node: 04840334
+Node: 04840507
-Node: 04740681
+Node: 04740854
-Node: 04641076
+Node: 04641249
-Node: 04541476
+Node: 04541649
-Node: 04441835
+Node: 04442008
-Node: 04342194
+Node: 04342367
-Node: 04242391
+Node: 04242564
-Node: 04142552
+Node: 04142725
-Node: 0442793
+Node: 0442966
-Node: 03246726
+Node: 03246899
-Node: 03147105
+Node: 03147278
-Node: 0347302
+Node: 0347475
-Node: 02350277
+Node: 02350450
-Node: 02250505
+Node: 02250678
-Node: 02150773
+Node: 02150946
-Node: 0250972
+Node: 0251145
-Node: 0155007
+Node: 0155180
-Node: Notes55108
+Node: Notes55281
-Node: Comparison with Org Agenda searches55270
+Node: Comparison with Org Agenda searches55443
-Node: org-sidebar56142
+Node: org-sidebar56315
-Node: License56421
+Node: License56594
 
 End Tag Table
 

tests/data.org

@@ -43,7 +43,7 @@ SCHEDULED: <2017-07-05 Wed +2d>
 :END:
 
 ** TODO [#B] Renew membership in supervillain club
-DEADLINE: <2017-07-10 Mon -1w>
+DEADLINE: <2017-07-10 Mon 23:59 -1w>
 
 ** DONE [#B] Learn universal sign language
 CLOSED: [2017-07-05 Wed 03:02]

tests/test-org-ql.el

@@ -637,7 +637,15 @@ RESULTS should be a list of strings as returned by
           '("Take over the universe" "Take over the world" "Visit Mars" "Visit the moon" "Renew membership in supervillain club" "Internet" "Spaceship lease" "/r/emacs"))
         (org-ql-then
           (org-ql-expect ('(deadline :to today))
-            '("/r/emacs")))))
+            '("/r/emacs"))))
+
+      (org-ql-it ":with-time"
+        (org-ql-expect ('(deadline :with-time nil))
+          '("Take over the universe" "Take over the world" "Visit Mars" "Visit the moon" "Internet" "Spaceship lease" "/r/emacs"))
+        (org-ql-expect ('(deadline :with-time t))
+          '("Renew membership in supervillain club"))
+        (org-ql-expect ('(deadline :to "2017-07-10" :with-time t))
+          '("Renew membership in supervillain club"))))
 
     (org-ql-it "(done)"
       (org-ql-expect ('(done))
@@ -831,7 +839,15 @@ RESULTS should be a list of strings as returned by
           '("Take over the universe" "Take over the world" "Skype with president of Antarctica" "Visit Mars" "Visit the moon" "Practice leaping tall buildings in a single bound" "Renew membership in supervillain club" "Learn universal sign language" "Order a pizza" "Get haircut" "Internet" "Spaceship lease" "Fix flux capacitor" "/r/emacs" "Shop for groceries" "Rewrite Emacs in Common Lisp"))
         (org-ql-then
           (org-ql-expect ('(planning :to today))
-            '("Skype with president of Antarctica" "Practice leaping tall buildings in a single bound" "Learn universal sign language" "Order a pizza" "Get haircut" "Fix flux capacitor" "/r/emacs" "Shop for groceries" "Rewrite Emacs in Common Lisp")))))
+            '("Skype with president of Antarctica" "Practice leaping tall buildings in a single bound" "Learn universal sign language" "Order a pizza" "Get haircut" "Fix flux capacitor" "/r/emacs" "Shop for groceries" "Rewrite Emacs in Common Lisp"))))
+
+      (org-ql-it ":with-time"
+        (org-ql-expect ('(planning :with-time nil))
+          '("Take over the universe" "Take over the world" "Visit Mars" "Visit the moon" "Practice leaping tall buildings in a single bound" "Get haircut" "Internet" "Spaceship lease" "Fix flux capacitor" "/r/emacs" "Shop for groceries" "Rewrite Emacs in Common Lisp"))
+        (org-ql-expect ('(planning :with-time t))
+          '("Skype with president of Antarctica" "Renew membership in supervillain club" "Learn universal sign language" "Order a pizza"))
+        (org-ql-expect ('(planning :to "2017-07-04" :with-time t))
+          '("Skype with president of Antarctica"))))
 
     (describe "(priority)"
 
@@ -940,7 +956,13 @@ RESULTS should be a list of strings as returned by
           '("Skype with president of Antarctica" "Practice leaping tall buildings in a single bound" "Order a pizza" "Get haircut" "Fix flux capacitor" "Shop for groceries" "Rewrite Emacs in Common Lisp"))
         (org-ql-then
           (org-ql-expect ('(scheduled :to today))
-            '("Skype with president of Antarctica" "Practice leaping tall buildings in a single bound" "Order a pizza" "Get haircut" "Fix flux capacitor" "Shop for groceries" "Rewrite Emacs in Common Lisp")))))
+            '("Skype with president of Antarctica" "Practice leaping tall buildings in a single bound" "Order a pizza" "Get haircut" "Fix flux capacitor" "Shop for groceries" "Rewrite Emacs in Common Lisp"))))
+
+      (org-ql-it ":with-time"
+        (org-ql-expect ('(scheduled :with-time t))
+          '("Skype with president of Antarctica" "Order a pizza"))
+        (org-ql-expect ('(scheduled :to "2017-07-04" :with-time t))
+          '("Skype with president of Antarctica"))))
 
     ;; ;; TODO: Test (src) predicate.  That will require modifying test data, which will be a
     ;; ;; significant hassle.  Manual testing shows that the predicate appears to work properly.
@@ -1140,7 +1162,28 @@ RESULTS should be a list of strings as returned by
         (org-ql-it ":on a number of days"
           (org-ql-then
             (org-ql-expect ('(ts-active :on 2))
-              '("Take over the world")))))
+              '("Take over the world"))))
+
+        (org-ql-it ":with-time"
+          (org-ql-expect ('(ts-active :with-time nil))
+            '("Take over the universe" "Take over the world" "Visit Mars" "Visit the moon" "Practice leaping tall buildings in a single bound" "Get haircut" "Internet" "Spaceship lease" "Fix flux capacitor" "/r/emacs" "Shop for groceries" "Rewrite Emacs in Common Lisp"))
+          (org-ql-expect ('(ts-active :with-time t))
+            '("Skype with president of Antarctica" "Renew membership in supervillain club" "Order a pizza"))
+          (org-ql-expect ('(ts-active :to "2017-07-04" :with-time t))
+            '("Skype with president of Antarctica"))
+
+          ;; Test string query syntax.  Just doing it in this predicate
+          ;; for now, rather than in all ths ts-related ones.
+
+          ;; NOTE: "with-time=" is equivalent to "with-time=nil".  It's debatable whether this is best or most
+          ;; intuitive, but making it behave as if "with-time=" were not given is too much trouble and makes the
+          ;; code too complicated in the current implementation of argument handling and string query parsing.
+          (org-ql-expect ((org-ql--query-string-to-sexp "ts-active:with-time=nil"))
+            '("Take over the universe" "Take over the world" "Visit Mars" "Visit the moon" "Practice leaping tall buildings in a single bound" "Get haircut" "Internet" "Spaceship lease" "Fix flux capacitor" "/r/emacs" "Shop for groceries" "Rewrite Emacs in Common Lisp"))
+          (org-ql-expect ((org-ql--query-string-to-sexp "ts-active:with-time="))
+            '("Take over the universe" "Take over the world" "Visit Mars" "Visit the moon" "Practice leaping tall buildings in a single bound" "Get haircut" "Internet" "Spaceship lease" "Fix flux capacitor" "/r/emacs" "Shop for groceries" "Rewrite Emacs in Common Lisp"))
+          (org-ql-expect ((org-ql--query-string-to-sexp "ts-active:with-time=t"))
+            '("Skype with president of Antarctica" "Renew membership in supervillain club" "Order a pizza"))))
 
       (describe "inactive"
 
@@ -1188,7 +1231,15 @@ RESULTS should be a list of strings as returned by
         (org-ql-it ":on a number of days"
           (org-ql-then
             (org-ql-expect ('(ts-inactive :on 19))
-              '("Visit the moon" "Rewrite Emacs in Common Lisp")))))
+              '("Visit the moon" "Rewrite Emacs in Common Lisp"))))
+
+        (org-ql-it ":with-time"
+          (org-ql-expect ('(ts-inactive :with-time nil))
+            nil)
+          (org-ql-expect ('(ts-inactive :with-time t))
+            '("Visit the moon" "Learn universal sign language" "Rewrite Emacs in Common Lisp"))
+          (org-ql-expect ('(ts-inactive :to "2017-07-04" :with-time t))
+            nil)))
 
       (describe "both"
 
@@ -1250,7 +1301,15 @@ RESULTS should be a list of strings as returned by
         (org-ql-it ":on a number of days"
           (org-ql-then
             (org-ql-expect ('(ts :on 5))
-              '("Renew membership in supervillain club"))))))
+              '("Renew membership in supervillain club"))))
+
+        (org-ql-it ":with-time"
+          (org-ql-expect ('(ts :with-time nil))
+            '("Take over the universe" "Take over the world" "Visit Mars" "Visit the moon" "Practice leaping tall buildings in a single bound" "Get haircut" "Internet" "Spaceship lease" "Fix flux capacitor" "/r/emacs" "Shop for groceries" "Rewrite Emacs in Common Lisp"))
+          (org-ql-expect ('(ts :with-time t))
+            '("Skype with president of Antarctica" "Visit the moon" "Renew membership in supervillain club" "Learn universal sign language" "Order a pizza" "Rewrite Emacs in Common Lisp"))
+          (org-ql-expect ('(ts :to "2017-07-04" :with-time t))
+            '("Skype with president of Antarctica")))))
 
     (describe "Compound queries"