rvrdata.adapters.text

Data adapter for reading and writing files.

Defaults to working with text files but can also write binary files.

class RvrdataText

Data adapter for text files.

read()

Read text file into memory.

• Parameters:
• filename (str) – Path to file.
• add_indent (int) – Number of spaces to indent lines of text (optional)

The action ‘text.read’ in a dataflow step reads the contents of the file into the .data property of the step.

If the filename is not correct or the file is not accessible due to permissions, this action will fail.

Example: Dataflow reading a file

steps:
  - name: get_file
    notes: Load file contents into 'steps.get_file.data'.
    action: text.read
    params:
        filename: "data/simple_list.txt"
        add_indent: 2

This leverages the standard file open and file read features of python. For more information on how the f.read() function works, see https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files

Read Metrics

The following metrics are included in the step results:

• rvrdata.dataflow.action.text.read.size: Length of the string read (int).
• rvrdata.dataflow.action.text.read.line_count: Number of newlines read (int).
• rvrdata.dataflow.action.text.read.file_count: Number of files read (int).

One row is added to the metrics each time the action is run.

write()

Write data to a file.

• Parameters:
• filename (str) – Path to text file.
• data (Any) – Data to be saved as the file contents.
• mode (str) – Mode used to open the file - defaults to ‘w’.

Example: Dataflow to write text file

steps:
  - name: save_data
    notes: Write data to file.
    action: text.write
    params:
        filename: "data/simple_list.txt"
        data: |
            item1
            item2
            item3

Example: Dataflow to write text file with referenced data

steps:
  - name: save_data_from_prv_step
    notes: Write data from context reference to file.
    action: text.write
    params:
        filename: "data/example.txt"
        data__ref: steps.some_previous_step.data

• Returns: success = True if file is written or False if the write fails.

Writing a file automatically creates parent directories if they are not present.

The ‘mode’ parameter can be useful to change the write behavior. By default, an existing file with the same name will be overwritten. Use ‘mode: a’ to open the file for appending ( see append ). Appending a ‘b’ to the mode opens the file in binary mode.

For more information on options for writing files, see https://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files

Write Metrics

The following metrics are included in the step results:

• rvrdata.dataflow.action.text.write.size: Length of the string read (int).
• rvrdata.dataflow.action.text.write.line_count: Number of newlines read (int).
• rvrdata.dataflow.action.text.write.file_count: Number of files written (int).

One row is added to the metrics each time the action is run.

append()

Append data to file.

Syntax is the same as text.write but the file is opened with ‘mode: a’ so that data is appended instead of overwriting the file.

See text.write metrics for append metrics. The metrics names replace write with append for this action.

create()

Check if file is present and only write if it is not.

This action is used to avoid overwriting existing data. It will fail if there is a file in the path specified in filename. If no data is supplied, it will create an empty file.

Syntax is the same as text.write.

• Parameters:
• filename (str) – Path to text file.
• data (Any) – Data to be saved as the file contents.

Example: Dataflow to create text file

steps:
  - name: create_file
    notes: Creates text file only if no file exists.
    action: text.create
    params:
        filename: "data/simple_list.txt"
        data: |
            Text to be written.

See text.write metrics for create metrics.

delete()

Delete file.

• Parameters:
• filename (str) – Path to file.
• missing_ok (bool) – Option to pass if file missing (defaults to False).

Example: Dataflow to delete file

steps:
  - name: delete_file
    action: text.delete
    params:
        filename: "data/file_to_be_deleted"
        missing_ok: True

For more information, see pathlib.Path.unlink

Delete Metrics

The following metrics are included in the step results:

• rvrdata.dataflow.action.text.delete.file_count: Number of files deleted (int).

One row is added to the metrics each time the action is run.

force_pass()

Fake successful action that does nothing.

See force_pass in stdout adapter.