rvrdata.adapters.yaml

Data adapter for reading and writing YAML files.

class RvrdataYaml

Data adapter for YAML files.

read()

Read yaml file into memory.

• Parameters:
• filename (str) – Path to file.
• typ (str) – Loader style (defaults to ‘rt’ for round-trip)

The action yaml.read in a dataflow step reads the contents of a YAML file into the .data property of the step.

If the filename is not correct or the file is not accessible due to permissions, this action will fail.

Example: Dataflow reading a YAML file

steps:
  - name: get_yaml
    notes: Load YAML file contents into 'steps.get_yaml.data'.
    action: yaml.read
    params:
        filename: "data/example.yml"

This leverages the ruamel.yaml python module. See the ruamel.yaml api for more background on loaders including the round-trip loader which is a derivative of safe loader with support for comments.

Read Metrics

The following metrics are included in the step results:

• rvrdata.dataflow.action.yaml.read.size: Size of data in bytes (int).
• rvrdata.dataflow.action.yaml.read.file_count: Number of files read (int).

One row is added to the metrics each time the action is run.

write()

Write data to a YAML file.

• Parameters:
• filename (str) – Path to YAML file.
• data (Any) – Data to be saved in the file.
• mode (str) – Mode used to open the file - defaults to ‘w’.

Example: Dataflow writing YAML file with referenced data

steps:
  - name: save_data_from_prv_step
    notes: Write data from context reference to file.
    action: yaml.write
    params:
        filename: "data/example.yaml"
        data__ref: steps.some_previous_step.data

Writing a file automatically creates parent directories if they are not present.

This leverages the ruamel.yaml python module.

Write Metrics

The following metrics are included in the step results:

• rvrdata.dataflow.action.yaml.write.size: Size of data in bytes (int).
• rvrdata.dataflow.action.yaml.write.file_count: Number of files read (int).

One row is added to the metrics each time the action is run.

create()

Create YAML file only if it is not already present.

This action works similar to a POST action in a REST API. It is intended for situations that need to avoid overwriting existing data. This will fail if there is an existing file in the path specified.

Syntax is the same as yaml.write.

See yaml.write metrics for create metrics.

delete()

Delete file.

This works the same as text.delete.

Delete Metrics

The following metrics are included in the step results:

• rvrdata.dataflow.action.yaml.delete.file_count: Number of files deleted (int).

One row is added to the metrics each time the action is run.

update()

Update YAML file with supplied data.

• Parameters:
• filename (str) – Path to text file.
• data (Any) – Data used to update the YAML.

This works similar to doing a PATCH in a REST API. Partial data can be supplied and the keys will be either added if new, or updated if they already exist.

The operation works as follows:

• read a existing file into a dictionary in memory,
• update the dictionary with the supplied data, and
• write the updated dictionary back to the file path.

Example: Dataflow to update YAML data

steps:
  - name: update_file
    notes: Updates YAML file with new/changed data.
    action: yaml.update
    params:
        filename: "data/example.yaml"
        data:
            key1: value
            key2: value

Note that this does not guarantee any order for the YAML.

See yaml.write metrics for update metrics.

force_pass()

Fake successful action that does nothing.

See force_pass in stdout adapter.