Skip to content
Docs
Open Mammoth
Learn › Clean and shape data › Duplicates and checks: open the docs menu

Data Check types: row count, nulls, unique values and ranges

A Data Check in the Controls category tests the output of the task before it. Choose a type from the tiles when you open the form, or from the type dropdown on each check. To add a Data Check, see Remove duplicate rows and add data quality checks.

Check type dropdown open, listing Row count, Null percentage and Unique values with a short description for each

Check the percentage of empty values (Null percentage)

Tests the share of null or empty values in a column. You set the column and a percentage threshold.

ConditionPasses when
is less thanNull % is below the threshold
is greater thanNull % is above the threshold
is exactlyNull % equals the threshold

Example: make sure customer_email has fewer than 1% nulls after a join, to catch a broken join before it drops records.

Check the number of rows (Row count)

Tests the total number of rows in the output. You set one value, or a min and max for "is between".

ConditionPasses when
is at leastRow count ≥ value
is at mostRow count ≤ value
is exactlyRow count equals value
is betweenRow count is within the min–max range

Example: make sure a daily sales feed has at least 1,000 rows, to catch a source failure before it produces empty reports.

Check that values are unique (Unique values)

Tests whether a column's values are unique. Empty values are not counted. You set the column, plus a percentage for the second condition.

ConditionPasses when
100% unique (no duplicates)Every value is distinct
Uniqueness at least...At least the percentage you enter is unique

Example: make sure order_id has no duplicates after a join, to catch fan-out that would inflate totals.

Check that numbers fall within a range (Value range)

Tests whether a number column stays within bounds. You set the column, a value or min and max, and a tolerance.

ConditionPasses when
betweenValues are within the min–max range
is at leastValues are ≥ value
is at mostValues are ≤ value
is positiveValues are > 0
is negativeValues are < 0

The tolerance options are Fail if any row violates, Fail if > 1% violate, Fail if > 5% violate and Fail if > 10% violate. A 1% tolerance means the check passes if at most 1% of rows break the rule. Use it for real-world data where a few edge cases are fine.

Example: require transaction_amount to be positive with a 1% tolerance, so rare refunds don't fail the whole pipeline.

Write your own rule in plain words (Custom prompt)

Describe a rule and Mammoth writes the SQL for it.

Data Check with Custom prompt selected, showing a plain-language prompt box and a Generate SQL button
  1. Select Custom prompt as the check type. The tile reads Write your own rule.
  2. Type the rule, for example "all prices must be positive" or "revenue should not exceed 10 million".
  3. Click Generate SQL. Mammoth sends the description to an AI model, which writes a SQL condition.
  4. Review the Generated SQL under the text box.
  5. Click Apply.

The SQL runs as a condition against the whole pipeline output. For example, "the discount percentage should never exceed 100" becomes discount_pct <= 100.

Use a parameter as a check threshold

Bind a check's number to a parameter instead of a fixed value, so the same pipeline works with different thresholds in different runs.

Data Check configuration form showing a Row count check with an is at least operator and a threshold of rows

Example: a pipeline runs on small test data in development and full data in production.

  1. Create a Number parameter min_rows.
  2. In the Data Check, click the parameter icon next to the row count value.
  3. Select min_rows.
  4. Set min_rows = 10 for development runs and min_rows = 50000 for production runs.

See Parameters and snippets to create a parameter.

Was this page helpful?

Something wrong? support@mammoth.io