Skip to content

Working with dates in dataflows

Preferred format for dates

The preferred format for working with dates is ISO 8601 short format. For example: 2022-12-25T00:00:00Z.

This format is useful because it sorts well as a string, it truncates easily into date, month or year, and it is can be merged with other dates avoiding issues of inconsistent time zones.

Date Functions

Included in rvrdata is a jinja2 function now() which returns the current date and time in UTC ISO8601 short format (e.g. 2022-12-25T00:00:00Z). This can be used anywhere in dataflow YAML.

Here are some examples using this function:

    current_timestamp: "{{ now() }}"  # returns UTC YYYY-MM-DDTHH:mm:SSZ
    todays_date: "{{ now()[0:10] }}"  # returns UTC YYYY-MM-DD
    current_month: "{{ now()[0:7] }}"  # returns YYYY-MM

Date Filters

A jinja2 custom filter is available that converts date strings into UTC ISO8601 Shorthand format. Note that the current implementation is timezone naive and assumes UTC.

Here are some examples using this filter:

    # the following yields "2001-02-03T00:00:00Z"
    timestamp: "{{ '2001-02-03'|str_to_utc_str('%Y-%m-%d') }}"

    # the following yields "2001-02-03T04:05:00Z"
    timestamp: "{{ '2001-02-03 04:05'|str_to_utc_str('%Y-%m-%d %H:%M') }}"

For a list of date format codes, see: https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes

  • How to do date formatting
  • How to do date calcs (e.g. now - 1d, or first day of week)