Skip to content

Rover Micro Installation

Revision: 0.4.0

Pre-requisites

Rovers run in Docker containers. This means that some form of infrastructure needs to be in place to run containers. It could be a shared model such as Docker Enterprise using Swarm, a Kubernetes cluster, or some other platform like Jenkins Pipelines.

Docker can also run standalone on Linux including on Virtual Machines (VMs), or locally on MacOS and Windows machines using Docker Desktop. In these cases, docker needs to be installed and available for use.

Follow the steps from Install Docker Engine for the appropriate OS. For linux, be sure to also follow the Post-installation steps for Linux

Confirm docker is running with docker ps and docker --version.

Deployment Overview

The following components need to be in place for the solution to run:

  1. Rover Micro Docker Image
  2. The image contains all of the core code for rover.

  3. These images can be pulled using the procedure outlined below

  4. Environment Variables

  5. Environment variables can be set individually or via an env-file
  6. At run time, these variables need to be passed in to the docker container.

  7. Included is sensitive information such as user ids and passwords, as such, this data should be appropriately protected.

  8. Configuration files (.yml) - typically delivered as part of an overall solution

  9. The YAML files contain the dataflows which act as data pipelines.
  10. The configuration files can be maintained in source code control and delivered via the Rover Cloud.

Accessing Docker Image using AWS

AWS Elastic Container Registry (ECR) provides a private image repo for Rover images.

Install the AWS CLI

To access the AWS ECR, the AWS CLI version 2 can be used. Follow the instructions for the AWS CLI install for the appropriate OS.

Once installed, use the following command to verify it is successful:

aws --version

Rover Credentials

Each rover has a corresponding AWS IAM account with an access key. ASI will provide an AWS access key id and secret access key.

Configure the AWS CLI

The AWS CLI must be configured and there are many options for this - see https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html

One of the easiest options is to run the following:

aws configure

The default AWS region for rover is us-east-1.

Pulling the Image

The image tag for rover micro is as follows:

076052596660.dkr.ecr.us-east-1.amazonaws.com/rover-micro

The following commands can be used to access this with AWS CLI:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 076052596660.dkr.ecr.us-east-1.amazonaws.com

This does a docker login to aws ecr which allows the next command to work:

docker pull \ 
 076052596660.dkr.ecr.us-east-1.amazonaws.com/rover-micro

Volume Mounts

The data for rover can be persisted to volumes or bind mounts. Configuration files can also be pre-populated to rovers using volumes or bind mounts. See Docker Volumes for more info on how these work.

The steps below can be used to create volumes and pre-populate the configuration data.

  1. Export the Rover ID (useful to ensure volumes for different rovers do not conflict)

    export ROVER_ID=<rover_ROVERID>

  2. Create volumes for data and, optionally, logs

    docker volume create $ROVER_ID-data
docker volume create $ROVER_ID-logs

  3. Confirm that the volumes are present:

    docker volume ls | grep $ROVER_ID

    Optionally, to get more details for a given volume use:

    docker volume inspect $ROVER_ID-data

Setting the Environment Variables

The following is an example of the contents of the env file. Replace the variable values with local credentials.

ROVER_ACCESS_KEY_ID=<REPLACE_WITH_ACCESS_KEY>
ROVER_SECRET_ACCESS_KEY=<REPLACE_WITH_ACCESS_SECRET>

Append other environment variables as needed to the above list.

Running the Docker container

To run Rover with the volumes, replace the environment file path in the command below and run it from the command-line:

docker run -d --rm --name $ROVER_ID \
    --env-file <REPLACE_WITH_ENV_FILE_PATH> \
    -v $ROVER_ID-data:/home/rover/data \
    -v $ROVER_ID-logs:/home/rover/logs \
    076052596660.dkr.ecr.us-east-1.amazonaws.com/rover-micro

Running dataflows from the command line

To run the dataflows manually in the container CLI, login to the container using the following:

docker exec -it <container_id> /bin/bash

To see the help for running dataflows, use the following command:

rvrdata -h