Schema Monitoring Overview

Detect and track schema changes across your data assets

Schema monitoring is AnomalyArmor’s core capability. It automatically detects changes to your database structure (columns added, removed, renamed, or modified) before they break your pipelines and dashboards.

Why Schema Monitoring Matters

Schema changes are one of the most common causes of data pipeline failures:

What We Monitor

AnomalyArmor detects all structural changes to your database:

Change Type	Description	Impact Level
Column Removed	Existing column no longer exists	High - breaks queries
Column Added	New column appears	Low - usually safe
Column Type Changed	Data type modified	Medium - may affect logic
Column Renamed	Name changed (detected as remove + add)	High - breaks queries
Table Added	New table discovered	Low - informational
Table Removed	Table no longer exists	High - breaks queries
Constraint Changed	Primary key, foreign key, unique modified	Medium - may affect joins

How It Works

Schema monitoring happens during discovery:

1. Discovery scans your database’s system catalogs
2. Comparison against the previous discovery state
3. Changes recorded with full before/after details
4. Alerts triggered based on your rules

Key Features

Automatic Detection

Changes are detected automatically during scheduled discovery, no manual work required.

• Set discovery to run hourly, daily, or custom
• Changes detected on next run after they occur
• Zero configuration for basic monitoring

Complete Change History

Every change is recorded with full context:

• What changed: Column name, type, constraints
• When detected: Timestamp of discovery
• Before/after: Previous and new values
• Asset context: Which table, schema, database

Flexible Alerting

Route different changes to different places:

Production + Column Removed  →  PagerDuty + Slack #incidents
Production + Column Added    →  Slack #data-changes
Development + Any Change     →  Email digest

Change Visualization

See schema changes at a glance:

Getting Started

1. Connect Your Database

Schema monitoring starts when you add a data source.

2. Run Discovery

Trigger discovery to establish a baseline.

3. Set Up Alerts

Create alert rules for schema changes:

Event: Schema Change Detected
Scope: Your production database
Destinations: Slack, Email, or PagerDuty

4. Schedule Discovery

Configure discovery to run regularly:

Environment	Recommended Schedule
Production	Hourly
Staging	Every 6 hours
Development	Daily

Viewing Schema Changes

In the Asset Catalog

1. Go to Assets
2. Click on a table or view
3. Select the Schema History tab

You’ll see:

• Timeline of all changes
• Before/after comparison for each change
• Which discovery run detected it

In Alert History

1. Go to Alerts → History
2. Filter by “Schema Change”
3. Click an alert to see change details

Schema Change Types

Column Removed

What: A column that existed before is no longer present.

Risk: High, any query referencing this column will fail.

Example:

Table: orders
Column removed: shipping_status (varchar)
Detected: 2024-01-15 08:00 UTC

Action: Update downstream queries, models, and reports.

Column Added

What: A new column appears that didn’t exist before.

Risk: Low. Existing queries usually continue working.

Example:

Table: orders
Column added: delivery_estimate (timestamp)
Detected: 2024-01-15 08:00 UTC

Action: Consider adding to models if relevant.

Column Type Changed

What: A column’s data type was modified.

Risk: Medium. May affect type-specific logic.

Example:

Table: products
Column: price
Type change: integer → decimal(10,2)
Detected: 2024-01-15 08:00 UTC

Action: Review aggregations, comparisons, and type casting.

Table Removed

What: An entire table no longer exists.

Risk: High. All queries to this table will fail.

Example:

Table removed: legacy_orders
Last seen: 2024-01-14 08:00 UTC
Detected: 2024-01-15 08:00 UTC

Action: Urgent. Investigate if intentional.

Best Practices

Monitor Production Most Closely

Production schema changes have the highest impact. Run discovery more frequently and set up immediate alerts.

Route by Change Type

Not all changes are equally urgent:

Change	Alert Destination
Column/Table removed	PagerDuty + Slack
Type changed	Slack
Column/Table added	Email or Slack (low priority)

Coordinate with Change Management

When you know schema changes are coming:

1. Inform your team
2. Update downstream code first (if possible)
3. Make the schema change
4. Verify AnomalyArmor detects it
5. Confirm no unexpected alerts

Use Schema History for Debugging

When a pipeline fails, check schema history:

1. Find the asset in AnomalyArmor
2. Look at recent schema changes
3. Correlate timing with pipeline failure
4. Identify the breaking change

Related Topics

Schema Drift

Deep dive into detecting and managing drift

Asset Discovery

How discovery finds and catalogs assets

Alert Rules

Configure schema change alerts

Core Concepts

Understand schema changes in depth