Liking cljdoc? Tell your friends :D

dtype-next datetime API – summary for tablecloth.time

Namespace: tech.v3.datatype.datetime

Autogenerated wrapper namespace providing thorough, mostly vectorized bindings to java.time via dtype-next. It exposes:

  • Constants for time units (milliseconds / nanoseconds / seconds in day, hour, minute, etc.).
  • Constructors for java.time types and Duration.
  • Conversions between datetime types and several epoch-based numeric encodings.
  • Temporal arithmetic (add/subtract amounts, compute differences).
  • Field extraction (year, month, day-of-week, etc.).
  • Descriptive statistics over datetime/duration series.
  • Helpers for building time-based rolling windows.

This file focuses on how these pieces matter for lifting functionality into tablecloth.time.

1. Data model and general behavior

  • Functions typically accept either scalars or collections/column-like data (readers, sequences) and return values or readers suitable for dtype-next.
  • The API is designed around epoch representations for time:
    • :epoch-microseconds
    • :epoch-milliseconds
    • :epoch-seconds
    • :epoch-days
  • Many functions take an explicit timezone (java.time.ZoneId or nil, where nil means UTC). When no timezone is given, defaults are usually UTC or type-dependent sensible defaults.
  • datetime "datatypes" here refer to dtype-next logical dtypes (e.g. packed datetime, packed duration), not just raw Java classes.

In tablecloth.time, these functions will typically be called on columns (or column readers) rather than on individual scalar values.

2. Constants – time unit sizes

tech.v3.datatype.datetime.constants is re-exposed via vars in this namespace:

  • Milliseconds:
    • milliseconds-in-second
    • milliseconds-in-minute
    • milliseconds-in-hour
    • milliseconds-in-day
    • milliseconds-in-week
  • Seconds:
    • seconds-in-minute
    • seconds-in-hour
    • seconds-in-day
  • Nanoseconds:
    • nanoseconds-in-millisecond
    • nanoseconds-in-second
    • nanoseconds-in-minute
    • nanoseconds-in-hour
    • nanoseconds-in-day
    • nanoseconds-in-week

These constants are the canonical source for converting between metric time units. When lifting to tablecloth.time, we should use these instead of hardcoding factors like 1000, 60, etc., especially when computing bucket sizes in milliseconds.

3. Constructors for datetime and duration types

These wrap various java.time constructors and parsing facilities. They are mostly scalar-oriented but type-hinted and consistent with dtype-next:

  • instant → java.time.Instant
  • local-date → java.time.LocalDate
  • local-time → java.time.LocalTime
  • local-date-time → java.time.LocalDateTime
  • zoned-date-time → java.time.ZonedDateTime
  • duration → java.time.Duration

Each constructor usually has two arities:

  • ([] ...) – construct "now" or a default value.
  • ([arg] ...) – construct from a provided value (string, numeric, etc., depending on the specific base function).

For tablecloth.time, these are less central; we’ll primarily operate on existing datetime columns coming from users or parsed elsewhere. But they can be handy in tests or for generating synthetic time series.

4. Datatype predicates

These queries help distinguish datetime/duration columns from other numeric or object columns:

  • datetime-datatype?
    • Predicate on a dtype-next datatype descriptor to check whether it represents a datetime-like logical type.
  • duration-datatype?
    • Predicate on a dtype-next datatype descriptor for duration-like types.

These are important for column dispatch inside tablecloth.time (e.g. deciding when a column can be treated as a datetime axis versus a generic numeric column).

5. Epoch and millisecond conversions

5.1 General epoch conversions

  • datetime->epoch

    • (datetime->epoch timezone epoch-datatype data)
    • (datetime->epoch epoch-datatype data)
    • (datetime->epoch data)
    • Converts datetime data (scalar or vector) to one of the epoch encodings:
      • :epoch-microseconds
      • :epoch-milliseconds
      • :epoch-seconds
      • :epoch-days
    • Timezone handling:
      • If timezone (a ZoneId) is provided, it’s used for conversions.
      • If not, UTC is used.

  • epoch->datetime

    • (epoch->datetime timezone epoch-datatype datetime-datatype data)
    • (epoch->datetime timezone datetime-datatype data)
    • (epoch->datetime datetime-datatype data)
    • Inverse of datetime->epoch:
      • Takes numeric epoch data and returns a chosen datetime logical datatype.
      • Can infer the epoch kind from the elementwise datatype if epoch-datatype is omitted.

These are the most general conversion primitives in the API.

5.2 Millisecond-focused conversions (core for tablecloth.time)

Two functions are especially central for tablecloth.time because they implement the "millis pivot" model:

  • datetime->milliseconds

    • (datetime->milliseconds timezone data)
    • (datetime->milliseconds data)
    • Vectorized conversion of datetime data to epoch milliseconds with a given timezone (or UTC) and an implied 0 time offset.
    • Converts a wide range of datetime datatypes into a :int64 representation.

  • milliseconds->datetime

    • (milliseconds->datetime datatype timezone milli-data)
    • (milliseconds->datetime datatype milli-data)
    • Vectorized conversion of epoch milliseconds back into a specific datetime datatype, with optional timezone.

These are the main bridge between human-friendly datetime columns and the numeric millis axis we’ll use internally for bucketing, slicing, and rolling operations in tablecloth.time.

5.3 Specific helpers and convenience conversions

The namespace also exposes a large set of more specific helpers, for scalar or vector data:

  • Epoch <-> Instant:

    • instant->microseconds-since-epoch
    • instant->milliseconds-since-epoch
    • instant->seconds-since-epoch
    • microseconds-since-epoch->instant
    • milliseconds-since-epoch->instant
    • seconds-since-epoch->instant

  • Days/months since epoch:

    • days-since-epoch->local-date
    • local-date->days-since-epoch
    • epoch-days->epoch-months
    • epoch-months->epoch-days
    • epoch-months->local-date
    • local-date->epoch-months

  • Local date/time -> epoch millis/seconds (and back):

    • local-date->milliseconds-since-epoch
    • local-date-time->milliseconds-since-epoch
    • milliseconds-since-epoch->local-date
    • milliseconds-since-epoch->local-date-time
    • milliseconds-since-epoch->local-time
    • milliseconds-since-epoch->zoned-date-time

  • Local time conversions:

    • local-time->milliseconds
    • local-time->seconds
    • seconds->local-time

  • Composition between LocalDate, LocalTime, LocalDateTime, ZonedDateTime, Instant:

    • local-date->instant, local-date->local-date-time, local-date->zoned-date-time
    • local-date-time->instant, local-date-time->local-date, local-date-time->local-time, local-date-time->zoned-date-time
    • local-time->instant, local-time->local-date-time
    • instant->local-date-time, instant->zoned-date-time
    • zoned-date-time->instant, zoned-date-time->milliseconds-since-epoch

For tablecloth.time, we will rarely need these directly; instead we’ll standardize on millis and use datetime->milliseconds / milliseconds->datetime. However, these helpers are useful when we need calendar-aware operations (e.g. converting millis buckets to LocalDate ends).

6. Duration and unit conversions

Duration-related functions work with java.time.Duration and associated numeric encodings:

  • Construction and inspection:

    • duration → construct a Duration.
    • duration->milliseconds
    • duration->nanoseconds

  • Conversions from primitive counts:

    • milliseconds->duration
    • nanoseconds->duration

Together with the unit constants, this lets us mix duration objects and numeric millisecond/nanosecond representations. Tablecloth’s time API will probably treat most user-facing durations as {:amount n :unit :minutes} or similar, but we can use these functions internally when we need true Duration objects.

7. Temporal arithmetic and differences

7.1 Adding / subtracting temporal amounts

  • plus-temporal-amount
    • (plus-temporal-amount datetime-data long-data tf)
    • (plus-temporal-amount datetime-data long-data)
  • minus-temporal-amount
    • (minus-temporal-amount datetime-data long-data tf)
    • (minus-temporal-amount datetime-data long-data)

Common characteristics:

  • datetime-data: scalar or vector of datetime values.
  • long-data: scalar or vector of integer counts (e.g. number of days / hours to add).
  • tf: temporal field / unit. Supported units (from the docstrings):
    • #{:months :days :seconds :hours :years :milliseconds :minutes :weeks}

These functions handle the details of adding/subtracting amounts across the different supported datetime datatypes. They are natural building blocks for tablecloth.time operations like:

  • "shift this time column forward 3 days".
  • "subtract 2 hours from each timestamp".
  • "add N days from another numeric column to a datetime column".

7.2 Computing differences between datetimes

  • between
    • (between lhs rhs units)
    • Computes the time between two datetime series (or scalar + series) in requested units.
    • lhs and rhs must have the same datatype.
    • units may be a ChronoUnit or a keyword in:
      • #{:milliseconds :seconds :minutes :hours :days :weeks :months :years}
    • Returns longs or long readers.

between is the canonical way to compute durations between columns (or rows) without manually going through milliseconds. Units closely match those supported by plus-temporal-amount / minus-temporal-amount.

8. Field extraction and statistics

8.1 Temporal fields

  • long-temporal-field
    • (long-temporal-field tf data)
    • Extracts a numeric temporal field from datetime data.
    • Supported fields include (keywords):
      • :iso-day-of-week
      • :iso-week-of-year
      • :day-of-week
      • :months
      • :days
      • :seconds
      • :epoch-days
      • :day-of-year
      • :hours
      • :years
      • :milliseconds
      • :minutes
      • :week-of-year

This is the main hook for derived calendar columns in tablecloth.time: year, month, day-of-week, week-of-year, etc., derived from a datetime column.

8.2 Descriptive statistics

  • millisecond-descriptive-statistics
    • (millisecond-descriptive-statistics stats-seq options data)
    • (millisecond-descriptive-statistics data)
    • Computes descriptive statistics for datetime or duration series in millisecond space, then converts selected metrics back to datetime/duration types:
      • For datetime data, min, mean, max are returned as unpacked datetime objects.
      • Other stats are returned in milliseconds unless the input is a duration type, in which case standard deviation is also returned as a duration.

This function can be used to implement tablecloth.time helpers like "per-group datetime min/mean/max" while keeping millisecond semantics consistent.

9. Timezone helpers

The namespace re-exports a few simple helpers for working with zones and offsets:

  • system-zone-id → java.time.ZoneId for the system default.
  • system-zone-offset → java.time.ZoneOffset for the system default.
  • utc-zone-id → UTC ZoneId.
  • utc-zone-offset → UTC ZoneOffset.

In tablecloth.time, we may surface UTC as the default and allow options maps to override the zone using these helpers.

10. Rolling window index helpers

  • variable-rolling-window-ranges
    • (variable-rolling-window-ranges src-data window-length units options)
    • (variable-rolling-window-ranges src-data window-length units)

Behavior:

  • src-data must be a monotonically increasing datetime series (reader).
  • Windows are expressed in microseconds and return an iterable of index ranges over the source data.
  • There is one window per source element.
  • Windows near the end may be incomplete (cannot satisfy full window length).
  • Options are passed through to tech.v3.datatype.rolling/variable-rolling-window-ranges, including :stepsize and others.

This is the critical low-level primitive for time-based rolling operations in tablecloth.time (e.g. rolling means over a 5-minute window on a time axis).

11. How this maps onto tablecloth.time

When lifting functionality into tablecloth.time, we likely want to build a small, coherent subset of higher-level operations around these core pieces:

  1. Millis pivot conversions

    • Use datetime->milliseconds / milliseconds->datetime as the primary bridge between logical datetime columns and numeric millis columns.
    • Lean on the unit constants (milliseconds-in-*, seconds-in-*, etc.) to derive bucket sizes and windows.

  2. Column-wise temporal arithmetic

    • Wrap plus-temporal-amount / minus-temporal-amount to implement operations such as:
      • shift-time (add/subtract fixed offsets in various units).
      • "add N days from this numeric column to a datetime column".

  3. Differences and durations

    • Wrap between for column-wise differences (e.g. time since previous event, latency between two timestamps).
    • Express results in user-chosen units (defaults to milliseconds or seconds).

  4. Derived calendar features

    • Use long-temporal-field to build column helpers like:
      • year, month, day-of-month, day-of-week, week-of-year.
    • These can be exposed as simple column transforms in tablecloth.column.time and dataset-level helpers in tablecloth.time.api.*.

  5. Descriptive statistics

    • Use millisecond-descriptive-statistics for summarizing datetime columns, especially when we want:
      • group-wise temporal min/mean/max as datetime values,
      • but with millisecond-based computation semantics.

  6. Rolling windows

    • Use variable-rolling-window-ranges as the basis for time-based rolling operations, combined with dataset-level aggregation:
      • E.g. compute a rolling 5-minute average over a time axis column.
    • This complements dtype-next’s generic rolling machinery by adding time-unit semantics rather than pure index-based windows.

  7. Timezone semantics

    • Decide clear defaults (likely UTC) and accept an optional :timezone in tablecloth.time options.
    • Implement conversions and bucketing by consistently threading that option into datetime->milliseconds / milliseconds->datetime / epoch->datetime as needed.

Overall, tech.v3.datatype.datetime provides the low-level, type-correct, vectorized building blocks for time handling. tablecloth.time should:

  • Treat columns as the primary operands.
  • Normalize to millis where possible for simplicity and performance.
  • Use these primitives (especially datetime->milliseconds, milliseconds->datetime, between, plus-temporal-amount, long-temporal-field, and variable-rolling-window-ranges) to implement higher-level column and dataset APIs with clear, documented semantics.
❮Design Decisionsfpp3 Workthrough — Forecasting: Principles and Practice (3rd ed)❯

Can you improve this documentation?Edit on GitHub

cljdoc builds & hosts documentation for Clojure/Script libraries

Keyboard shortcuts
Ctrl+kJump to recent docs
←Move to previous article
→Move to next article
Ctrl+/Jump to the search field
Raise an issueBrowse cljdoc sourceChat on Slack
× close