Airflow
What is Airflow?
Links to relevant documentation
- Links to Airflow Documentation
Installation Instructions
To install airflow simply pip install "RegScale-CLI[airflow]"
.
RegScale-CLI Airflow Customizations
The configs and scripts held within this folder are example scripts using the SDK, and are set up to make the deployment
of Airflow for RegScale much simpler, so that your compliance can be left shifted automatically beyond belief!
DAGs
These DAGs are provided as examples for what can be done with RegScale for Airflow. For customized DAGs to meet your
specific needs, contact our sales team.
RegScale-CLI as Airflow PythonOperators
Through the convenience of structuring our CLI via click
, we can now dynamically generate
Airflow's PythonOperator
to execute a CLI command.
These operators and default arguments can be found in
from regscale.airflow.hierarchy
's AIRFLOW_CLICK_OPERATORS
.
Below is an example DAG to query CISA and then send reminders.
"""An example RegScale-CLI based DAG."""
from airflow import DAG
from regscale.airflow.config import DEFAULT_ARGS, yesterday
from regscale.airflow.hierarchy import AIRFLOW_CLICK_OPERATORS as OPERATORS
from regscale.airflow.tasks.groups import setup_task_group
dag = DAG(
'query_cisa_and_send_reminders',
default_args=DEFAULT_ARGS,
description="Query CISA and send reminders",
schedule_interval='@daily',
start_date=yesterday(),
is_paused_upon_creation=False,
)
setup = setup_task_group(dag)
cisa_alerts = OPERATORS['cisa__ingest_cisa_alerts']['lambda'](dag=dag)
cisa_kev = OPERATORS['cisa__ingest_cisa_kev']['lambda'](dag=dag)
send_reminders = OPERATORS['admin_actions__send_reminders']['lambda'](dag=dag)
setup >> [cisa_alerts, cisa_kev] >> send_reminders
In the above example, we use double underscores __
(dunderscores) to simulate spaces between the command groups passed
to the RegScale-CLI entrypoint, so no need to use regscale about
, we can just index the key about
.
Under the hood, we call a python lambda
that is a factory function for creating a dynamic PythonOperator
object
available for use within a DAG. The pre-built setup_task_group
is also a factory function for generating a setup task
group. The group will share the same first 8 characters from a UUID4, allowing the setups to avoid collision when
multiple setup tasks may be used.
Contact Information
To contact us regarding customizing your shift left compliance via Airflow, contact us.
License Information
This is limited to RegScale Enterprise on the platform side, but we provide an airflow image for use by unlicensed users.
Updated 11 months ago