Data Quality Guided Tour Data Quality

This interactive guide walks through the full Data Quality workflow: review quality health, profile datasets, define rules, schedule checks, monitor operational status, explore results, and use LLM observability for deeper insight.

DQ Dashboard

Review quality health

Use the DQ Dashboard to understand the current health of data quality across projects, domains, and datasets.

What to check

Area	Purpose
Quality score	Overall pass/fail health across monitored assets.
Issue summary	Count of open, resolved, warning, and failed validations.
Trend charts	Quality movement over time.
High-risk datasets	Datasets with recurring failures or severe issues.

Typical actions

1. Open the Data Quality Dashboard.
2. Filter by project, domain, dataset, or owner.
3. Review scorecards and trend indicators.
4. Drill into failed or warning validations.
5. Identify the datasets that need profiling or new rules.

:::tip Dashboard first Start from the dashboard when you need a broad health view before creating or editing rules. :::

Profile

Profile a dataset

Use Profile to inspect the structure, distribution, completeness, and anomalies of a dataset before defining quality rules.

What to review

Profile area	Purpose
Schema	Column names, types, nullable fields, and detected structure.
Completeness	Null count, empty values, and missing data percentage.
Uniqueness	Duplicate count and approximate distinct values.
Distribution	Min, max, average, frequency, and outlier patterns.
Samples	Example records used to validate expectations.

Typical actions

1. Select the target dataset.
2. Run or refresh the profile.
3. Review completeness and uniqueness indicators.
4. Identify candidate columns for validation.
5. Use profile findings to define DQ rules.

Common profile-driven rule ideas

Finding	Suggested rule
High null count	Not null or completeness rule.
Duplicate business keys	Uniqueness rule.
Unexpected value range	Range rule.
Invalid text patterns	Regex rule.
Missing reference values	Referential integrity rule.

DQ Rule

Create a DQ rule

Use DQ Rule to define the validation logic that checks whether a dataset meets expected quality standards.

Rule configuration

Field	Description
Rule Name	A descriptive name for the validation rule.
Dataset	The table or data asset to validate.
Column	Optional target column for column-level rules.
Rule Type	Validation type such as null check, uniqueness, range, regex, or referential integrity.
Threshold	Allowed failure limit before the rule is marked failed.
Severity	Impact level such as low, medium, high, or critical.
Owner	Person or team responsible for remediation.

Typical actions

1. Select Create Rule.
2. Choose the dataset and column.
3. Select the rule type.
4. Configure thresholds and severity.
5. Save the rule and run a validation test.

:::info Rule design Use profiling results to make rules specific and actionable instead of creating broad rules that generate noisy failures. :::

Schedule

Schedule validations

Use Schedule to run data quality validations automatically at the required cadence.

Schedule settings

Field	Description
Frequency	How often the validation runs, such as hourly, daily, weekly, or custom.
Start Time	Date and time when the schedule becomes active.
Timezone	Timezone used for scheduled runs.
Retry Policy	Number of retries and delay between retries.
Alert On	Events that trigger notifications, such as failure or warning.
Recipients	Users, teams, email addresses, or webhook targets to notify.

Typical actions

1. Open the schedule configuration for a rule or rule group.
2. Select the execution frequency.
3. Configure retry and timeout behavior.
4. Add notification recipients.
5. Save and enable the schedule.

Scheduling guidance

Dataset type	Suggested cadence
Batch daily tables	Daily after ingestion completes.
Critical operational tables	Hourly or after each pipeline run.
Reference datasets	Weekly or on source refresh.
Development datasets	Manual or ad hoc validation.

OpsHub

Monitor quality operations

Use OpsHub to monitor data quality validation runs, failures, retries, and operational health.

What to monitor

Area	Purpose
Run status	Current and historical validation execution status.
Failures	Failed validations grouped by dataset, owner, severity, or rule type.
Retries	Automatic retry attempts and final outcomes.
Alerts	Notifications sent for warnings, failures, and timeouts.
SLA impact	Rules or datasets affecting operational commitments.

Typical actions

1. Open OpsHub from the Data Quality workflow.
2. Filter by failed or warning runs.
3. Review error details and affected records.
4. Assign or track remediation actions.
5. Re-run validations after fixes are applied.

:::tip Operational triage Use severity, owner, and dataset filters to quickly separate critical failures from low-priority warnings. :::

Explore

Explore validation results

Use Explore to investigate data quality results in detail and understand why a rule passed, failed, or produced warnings.

Exploration areas

Area	Purpose
Failed records	Inspect rows that violated the rule.
Rule output	Review counts, percentages, and threshold comparisons.
Dataset context	Check related schema, profile, lineage, and ownership.
Historical results	Compare current failures with previous runs.
Remediation notes	Capture findings and next actions.

Typical actions

1. Open a failed or warning validation result.
2. Review summary metrics and thresholds.
3. Inspect failed records.
4. Compare against profile and historical run data.
5. Decide whether to fix data, adjust the rule, or update thresholds.

Investigation checklist

Question	Why it matters
Did the source data change?	Helps identify upstream ingestion or source-system issues.
Did the rule threshold change?	Explains new failures after configuration updates.
Are failures concentrated in one partition?	Points to time-window or batch-specific issues.
Is the failed column governed?	Determines ownership and remediation path.

LLM Observability

Use LLM Observability

Use LLM Observability to review AI-assisted quality explanations, recommendations, and rule insights when available.

What to review

Area	Purpose
Failure explanation	Summarized reason for validation failure.
Suggested remediation	Recommended action based on rule output and dataset context.
Rule recommendation	Suggested new rules or threshold changes.
Confidence	Confidence level of the generated recommendation.
Evidence	Supporting metrics, samples, or lineage references.

Typical actions

1. Open the LLM Observability view for a quality result.
2. Review the generated explanation.
3. Check evidence and confidence before acting.
4. Convert reliable suggestions into remediation tasks or rule updates.
5. Track recurring recommendations for systemic fixes.

:::caution Validate recommendations Treat LLM output as guidance. Confirm recommendations against source data, profile metrics, and governance ownership before applying changes. :::