HomeGuidesAPI ReferenceChangelog
Guides

Orchestration and Scheduling

Suggest Edits

CLI Orchestration and Scheduling

The Command Line Interface (CLI) is intended to run as a "sidecar" to the main RegScale platform where a separate Virtual Machine (VM) can be used to schedule and orchestrate bulk processing using the command line. This documentation is intended to demonstrate methods for using the CLI effectively to automate various compliance processing jobs.

Types of Orchestration

There are multiple types of orchestration that are deployed to schedule jobs via the CLI. These options include:

  • Cron Jobs - job scheduler available on Unix/Linux/Mac systems
  • Serverless Functions - on demand functions that can be called to kick off a CLI job (typically available in cloud environments)
  • Task Scheduler - job scheduler available on Windows systems

These job schedulers can be used to handle orchestration that determines exactly when and how RegScale CLI jobs are scheduled.

Authentication

Authentication to the APIs in RegScale uses an API bearer token. For security reasons, these tokens expire daily and are therefore inappropriate for bulk processing. To address this limitation, RegScale recommends running scheduled or orchestrated bulk processing in the context of a Service Account. See regscale.readme.io/edit/seService Accounts for instructions on how to create an account.

Calls to the CLI load environments from the configuration file init.yaml which is colocated in the directory with the CLI. Copy the Service Account bearer token from the RegScale interface to the 'token' field in init.yaml. This will configure the CLI to use that user context to run commands and RegScale API calls.

Cron Job on Ubuntu

This section will cover an example of scheduling the RegScale Wiz CLI using a cron job on a Linux Ubuntu VM. In this example, we will do the following:

  • Create a simple Bash script to execute multiple CLI commands
  • Determine the Bash path and file path for the script to execute
  • Set the schedule for the job to run
  • Create the cron job and assign the schedule
  • Look at the logs to verify it executed successfully

The first step is to create a Bash script that will execute the applicable commands for your job. Below is an example Bash script that authenticates to Wiz, syncs issues to a specific security plan, and then outputs the timestamp that the job finished and dumps the results to the logs:

#!/bin/sh
echo "Authenticating to Wiz"
regscale wiz authenticate
echo "Syncing issues between Wiz and RegScale"
regscale wiz issues --regscale_id 5  --regscale_module securityplans
echo "Synchronization complete"
# put current date as yyyy-mm-dd HH:MM:SS in $date
echo $(date '+%Y-%m-%d %H:%M:%S')

For ease of use, you can also pull this file directly from the command line with the following command - wget https://atlasity-io.web.app/assets/kb/yaml/cron-wiz.sh

The next step is to determine the absolute paths of your Bash directory and the directory where you have the actual script to execute. Navigate via the command line to the folder with your script, then run the following commands:

  • pwd will retrieve the absolute path of your current directory
  • Determine the path to Bash, typically either "/bin/bash" or "/usr/bin/bash"

Store this information for later use. Next, determine the schedule you want to run the cron job. Crontab Guru can be used as a visual aid to help you understand the cron scheduler. Once you have the appropriate schedule established, save this information for later use.

The next step is schedule the cron job. First, open up a new cron tab with crontab -e. Next, enter the command to schedule the cron job, an example is provided below:

0 23 * * 6 /usr/bin/sh /home/azureuser/regscaleScheduler.sh >> scheduler.log

In this example, we have set the script to execute at a specific time every Saturday which is the first part of the cron job 0 23 * * 6. Next, we provide a path to Bash /usr/bin/sh. The third component is to provide the path of the script we want to execute /home/azureuser/regscaleScheduler.sh. Finally, we dump the logs from this command out to a local file for monitoring >> scheduler.log.

This process can be repeated to run any number of jobs with the RegScale CLI on any number of schedules. In so doing, you can provide for sophisticated automations that process large volumes of data using our pre-built CLI commands.

Updated over 1 year ago