GitHub Actions Integration

Run data quality checks in your CI/CD pipeline with GitHub Actions

Integrate AnomalyArmor into your GitHub Actions workflows to automatically run data quality checks on pull requests, scheduled jobs, or deployments.

Prerequisites

• AnomalyArmor account with connected data source
• GitHub repository
• API key stored as a GitHub secret

Setup

1. Create API Key

Generate an API key in Settings > API Keys with read-only scope (or read-write if you need to trigger checks).

2. Add Secret to GitHub

Go to your repository’s Settings > Secrets and variables > Actions and add:

• Name: ARMOR_API_KEY
• Value: aa_live_your_key_here

Basic Workflow

Add this workflow file to .github/workflows/data-quality.yml:

name: Data Quality Checks

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  schedule:
    # Run daily at 6 AM UTC
    - cron: '0 6 * * *'
  workflow_dispatch:  # Manual trigger

jobs:
  quality-checks:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install AnomalyArmor CLI
        run: pip install anomalyarmor-cli

      - name: Check data freshness
        env:
          ARMOR_API_KEY: ${{ secrets.ARMOR_API_KEY }}
        run: |
          armor freshness check snowflake.prod.warehouse.orders
          armor freshness check snowflake.prod.warehouse.customers

      - name: Run validity checks
        env:
          ARMOR_API_KEY: ${{ secrets.ARMOR_API_KEY }}
        run: |
          armor validity summary ${{ vars.ASSET_ID }}

Workflow Patterns

Pattern 1: Pre-deployment Gate

Block deployments if data quality checks fail:

name: Deploy with Quality Gate

on:
  push:
    branches: [main]

jobs:
  quality-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install CLI
        run: pip install anomalyarmor-cli

      - name: Quality Gate
        env:
          ARMOR_API_KEY: ${{ secrets.ARMOR_API_KEY }}
        run: |
          # Check all critical tables
          armor freshness check snowflake.prod.warehouse.orders
          armor freshness check snowflake.prod.warehouse.customers
          armor freshness check snowflake.prod.warehouse.products

          echo "All quality checks passed!"

  deploy:
    needs: quality-gate
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Deploy application
        run: ./deploy.sh

Pattern 2: Scheduled Quality Report

Generate a daily quality report:

name: Daily Quality Report

on:
  schedule:
    - cron: '0 8 * * *'  # 8 AM UTC daily
  workflow_dispatch:

jobs:
  quality-report:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: pip install anomalyarmor-cli

      - name: Generate Quality Report
        env:
          ARMOR_API_KEY: ${{ secrets.ARMOR_API_KEY }}
        run: |
          echo "# Daily Data Quality Report" >> $GITHUB_STEP_SUMMARY
          echo "" >> $GITHUB_STEP_SUMMARY
          echo "## Freshness Summary" >> $GITHUB_STEP_SUMMARY
          armor freshness summary >> $GITHUB_STEP_SUMMARY
          echo "" >> $GITHUB_STEP_SUMMARY
          echo "## Validity Summary" >> $GITHUB_STEP_SUMMARY
          armor validity summary ${{ vars.ASSET_ID }} >> $GITHUB_STEP_SUMMARY
          echo "" >> $GITHUB_STEP_SUMMARY
          echo "## Referential Summary" >> $GITHUB_STEP_SUMMARY
          armor referential summary ${{ vars.ASSET_ID }} >> $GITHUB_STEP_SUMMARY

Pattern 3: PR Comment with Quality Status

Post quality status as a PR comment:

name: PR Quality Check

on:
  pull_request:
    branches: [main]

jobs:
  quality-check:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write

    steps:
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install CLI
        run: pip install anomalyarmor-cli

      - name: Run Quality Checks
        id: quality
        env:
          ARMOR_API_KEY: ${{ secrets.ARMOR_API_KEY }}
        run: |
          # Capture results
          FRESHNESS=$(armor freshness summary 2>&1) || true
          VALIDITY=$(armor validity summary ${{ vars.ASSET_ID }} 2>&1) || true

          # Build comment
          echo "COMMENT<<EOF" >> $GITHUB_OUTPUT
          echo "## Data Quality Status" >> $GITHUB_OUTPUT
          echo "" >> $GITHUB_OUTPUT
          echo "### Freshness" >> $GITHUB_OUTPUT
          echo '```' >> $GITHUB_OUTPUT
          echo "$FRESHNESS" >> $GITHUB_OUTPUT
          echo '```' >> $GITHUB_OUTPUT
          echo "" >> $GITHUB_OUTPUT
          echo "### Validity" >> $GITHUB_OUTPUT
          echo '```' >> $GITHUB_OUTPUT
          echo "$VALIDITY" >> $GITHUB_OUTPUT
          echo '```' >> $GITHUB_OUTPUT
          echo "EOF" >> $GITHUB_OUTPUT

      - name: Post PR Comment
        uses: actions/github-script@v7
        with:
          script: |
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: `${{ steps.quality.outputs.COMMENT }}`
            })

Pattern 4: dbt + Quality Checks

Combine dbt runs with quality validation:

name: dbt with Quality Gates

on:
  push:
    branches: [main]
    paths:
      - 'dbt/**'
  workflow_dispatch:

jobs:
  dbt-run:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: |
          pip install dbt-snowflake anomalyarmor-cli

      - name: Pre-flight Quality Check
        env:
          ARMOR_API_KEY: ${{ secrets.ARMOR_API_KEY }}
        run: |
          echo "Checking source data freshness..."
          armor freshness check snowflake.raw.stripe.payments
          armor freshness check snowflake.raw.crm.customers

      - name: Run dbt
        working-directory: ./dbt
        env:
          DBT_PROFILES_DIR: ${{ github.workspace }}/dbt
        run: |
          dbt deps
          dbt run

      - name: Post-run Validation
        env:
          ARMOR_API_KEY: ${{ secrets.ARMOR_API_KEY }}
        run: |
          echo "Validating dbt outputs..."
          armor validity summary ${{ vars.ASSET_ID }}
          armor referential summary ${{ vars.ASSET_ID }}

Pattern 5: Python Script for Complex Logic

For complex quality gates, use a Python script:

name: Advanced Quality Gate

on:
  push:
    branches: [main]

jobs:
  quality-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: pip install anomalyarmor-cli

      - name: Run Quality Gate
        env:
          ARMOR_API_KEY: ${{ secrets.ARMOR_API_KEY }}
          ASSET_ID: ${{ vars.ASSET_ID }}
        run: python scripts/quality_gate.py

Create scripts/quality_gate.py:

#!/usr/bin/env python
"""
Advanced quality gate for GitHub Actions.
Exits with code 0 if all checks pass, 1 otherwise.
"""
import os
import sys
from anomalyarmor import Client
from anomalyarmor.exceptions import StalenessError

def main():
    client = Client()
    asset_id = os.environ["ASSET_ID"]
    passed = True

    print("=" * 50)
    print("DATA QUALITY GATE")
    print("=" * 50)

    # 1. Freshness checks
    print("\n## Freshness Checks")
    critical_tables = [
        "snowflake.prod.warehouse.orders",
        "snowflake.prod.warehouse.customers",
    ]
    for table in critical_tables:
        try:
            status = client.freshness.require_fresh(table)
            print(f"[PASS] {table} ({status.hours_since_update:.1f}h old)")
        except StalenessError as e:
            print(f"[FAIL] {table} ({e.hours_since_update:.1f}h stale)")
            passed = False

    # 2. Validity checks
    print("\n## Validity Checks")
    validity = client.validity.summary(asset_id)
    if validity.failing == 0:
        print(f"[PASS] {validity.total_rules} rules, all passing")
    else:
        print(f"[FAIL] {validity.failing}/{validity.total_rules} rules failing")
        # List failing rules
        for rule in client.validity.list(asset_id):
            result = client.validity.check(asset_id, rule.uuid)
            if result.status == "fail":
                print(f"       - {rule.name}: {result.invalid_count} invalid")
        passed = False

    # 3. Referential integrity
    print("\n## Referential Integrity")
    ref = client.referential.summary(asset_id)
    if ref.failing_checks == 0:
        print(f"[PASS] {ref.total_checks} checks, all passing")
    else:
        print(f"[FAIL] {ref.failing_checks}/{ref.total_checks} failing")
        passed = False

    # 4. Summary
    print("\n" + "=" * 50)
    if passed:
        print("RESULT: ALL CHECKS PASSED")
        print("=" * 50)
        sys.exit(0)
    else:
        print("RESULT: QUALITY GATE FAILED")
        print("=" * 50)
        sys.exit(1)

if __name__ == "__main__":
    main()

Environment Variables

Reference these in your workflows:

Variable	Required	Description
ARMOR_API_KEY	Yes	Your AnomalyArmor API key (store as secret)
ASSET_ID	For some commands	Asset UUID (store as variable)

Using GitHub Variables

Store non-sensitive config as repository variables:

1. Go to Settings > Secrets and variables > Actions
2. Click Variables tab
3. Add variables like ASSET_ID, CRITICAL_TABLES, etc.

Reference in workflows:

env:
  ASSET_ID: ${{ vars.ASSET_ID }}

Best Practices

1. Use Secrets for API Keys

Never hardcode API keys:

# Good
env:
  ARMOR_API_KEY: ${{ secrets.ARMOR_API_KEY }}

# Bad - Never do this!
env:
  ARMOR_API_KEY: aa_live_xxxxx

2. Fail Fast

Put quality checks early in workflows:

jobs:
  quality-gate:
    # Runs first
    runs-on: ubuntu-latest
    ...

  build:
    needs: quality-gate  # Only runs if quality passes
    ...

  deploy:
    needs: build
    ...

3. Cache Dependencies

Speed up workflows by caching:

- uses: actions/setup-python@v5
  with:
    python-version: '3.11'
    cache: 'pip'

- name: Install dependencies
  run: pip install anomalyarmor-cli

4. Use Job Summaries

Write results to $GITHUB_STEP_SUMMARY for visibility:

- name: Quality Summary
  run: |
    echo "## Quality Gate Results" >> $GITHUB_STEP_SUMMARY
    armor freshness summary >> $GITHUB_STEP_SUMMARY

5. Set Timeouts

Prevent hanging jobs:

jobs:
  quality-checks:
    runs-on: ubuntu-latest
    timeout-minutes: 10

Troubleshooting

”Command not found: armor”

Ensure you’ve installed the CLI before using it:

- name: Install CLI
  run: pip install anomalyarmor-cli

- name: Use CLI
  run: armor --version  # Should work now

“Authentication failed”

Check that:

1. ARMOR_API_KEY secret is set correctly
2. The secret name matches your workflow reference
3. API key hasn’t been revoked

- name: Debug auth
  env:
    ARMOR_API_KEY: ${{ secrets.ARMOR_API_KEY }}
  run: |
    armor auth status

Workflow not triggering

Check your on: triggers and branch patterns match your setup.

Next Steps

Airflow Integration

Orchestrate with Airflow

dbt Integration

Add gates to dbt workflows

CLI Reference

Full CLI documentation

API Reference

Full API documentation