Skip to content
Data Integration & ETL · 4 min

Write the Logic Once

The same threshold is hardcoded in nine pipelines. Somebody changes it in seven. Parameters and snippets exist so that the value and the logic live in one place, and every pipeline that uses them tells you when it has not caught up yet.

There is a number somewhere in your pipelines that means something.

A defect threshold. A discount rate. The list of region codes that count as EMEA this year. It is not a secret and it is not complicated — it is just a value that several pipelines need to agree on.

So it got typed into all of them. And now it is in nine places, and when it changes somebody updates seven, and the two that were missed keep producing output that looks completely normal and is quietly using last year's number.

Nothing errors. That is the whole problem.

The value lives in one place

A parameter is a project-scoped named value. You define it once, and pipeline tasks reference it rather than containing it.

Inside a supported task editor, value fields gain a # button that opens a picker. Choose a parameter and the static input is replaced by a green token — {defect_threshold} — and the task now points at the definition instead of holding a copy.

At MVP the tasks that take parameters are the ones where hardcoded values actually accumulate:

TaskWhere it bindsWhat it looks like
Conditional FilterCondition valuedefect_count >= {defect_threshold}
Math FunctionOperand in a formulaprice * (1 - {discount_rate})
Add ColumnConstant value for a new columnRegion = {target_region}
Label & Insert ValuesCondition and output valuesIF sales > {min_sales} THEN "High"

Snippets do the same thing one level up. Where a parameter holds a value, a snippet holds a block of reusable SQL or expression logic, inserted into a task editor as {{snippet_name}}. The join condition that four pipelines share, the case statement that encodes how your business classifies something — written once, referenced everywhere.

The part that actually matters: when it resolves

The pipeline stores the reference, not the value.

{defect_threshold} is resolved at execution time, not at the moment you bound it. So a pipeline built in March and run in August uses August's number, without anyone opening it.

That sounds like a small implementation detail and it is the entire point. A configuration system that resolves at bind time is just a faster way of hardcoding — it copies the value into nine pipelines more conveniently. Resolving at run time is what makes "change it once" true rather than aspirational.

What happens when you change it

Change a parameter and every task referencing it is flagged stale.

The token turns amber. Hovering it shows what it was, struck through, and what it is now. A banner appears above the task: changed from X to Y — this pipeline needs rerun, with a rerun button on it. And the Parameters page carries a Rerun stale pipelines action for doing the lot.

Here is the design decision worth understanding: stale does not block execution.

A stale pipeline is still valid and will run. When it next executes — on its schedule, or when you trigger it — it uses the new value. The amber state is not a lock, it is an answer to a question you would otherwise have to work out by hand: which outputs on this screen were produced before the change, and therefore still reflect the old number?

Blocking would have been the easier thing to build and the wrong behaviour. A configuration change should not silently freeze your automation. It should tell you exactly what has not caught up yet, and let you decide whether that matters this hour or this week.

Where it sits

Parameters and snippets are a configuration layer, deliberately separate from execution control. They define what values and what logic tasks use. They do not decide when anything runs.

The same idea runs through the presentation layer, where styles are token bundles rather than per-artifact settings, for exactly the reasons above.

That keeps them independent of Draft Mode, view and export auto-sync, checkpoints and data checks — all of which control when and whether things execute. You can reason about one without holding the other in your head, which is not true of systems where configuration and scheduling are tangled together.

They live on the Data surface, under Data > Parameters — not in the Transform menu, because they are not a transformation. Managing them is an Admin and Editor capability; viewers do not see the definitions.

When this is worth doing

Not for everything. A value used in one pipeline is fine where it is, and turning it into a parameter is ceremony.

It earns its place at the second use. The moment a number, a threshold or a piece of classification logic exists in two pipelines, it has started drifting — not immediately, but on the first change nobody propagates. That is the point to pull it out.

The test is simple: if this value changed tomorrow, would you be confident you had found every place it lives? If the honest answer is no, it belongs in one place with everything else pointing at it.

In production at

  • Arla Foods logo
  • Bacardi logo
  • British American Tobacco logo
  • Everest Detection logo
  • Golden Acre logo
  • Handlangers logo
  • Inspired Learning Group logo
  • Kantar logo
  • MUFG logo
  • NielsenIQ logo
  • PTI Digital logo
  • RethinkFirst logo
  • Starbucks logo
  • The Specialist Works logo

Also in production at Bacardi, which cut manual data work by 70% across 170+ markets.

Connect your sources directly.

Tested connectors for the common sources, plus anything with a REST API, cleaning and combining on the way in.

  • 21-day Pro trial
  • No credit card
  • Viewers always free