A Refresh That Tells You When It Breaks
Automation that fails loudly is annoying. Automation that fails silently is dangerous, because the report still renders and the number is still wrong. The interesting part of a scheduled refresh is not the happy path — it is what happens the week the source changes shape.
Everybody demos the happy path.
A source updates, a pipeline runs, a dashboard refreshes, and someone says the word "automatically" while a green tick appears. Every tool on the market can do that, and it tells you almost nothing about whether the thing will still be right in November.
The interesting question is the other one. What happens the week the source changes shape?
Because it will. Somebody adds a column. A field gets renamed. An export that has produced the same eleven columns for two years produces twelve. And at that moment your automation does one of three things: it stops, it tells you, or it quietly carries on producing output that is subtly wrong.
The third is the one that costs money, and it is the default in more systems than you would like.
The five things that run on a schedule
Mammoth has five orchestration types, all in production:
Dataset Refresh pulls the most recent data from connected sources, manually or on a schedule, across multiple datasets with per-dataset update behaviour — combine or replace.
Data Consolidation watches a folder and combines or replaces dataset data as new files land in it, matching on file name.
File Collection goes and fetches files from somewhere else — Google Drive, Dropbox, OneDrive, SFTP — into a Mammoth folder on a schedule, filtered by name pattern.
Messaging sends email on a schedule or on an activity event, with dataset views attached.
PDF & Image Extraction runs the same extraction you would do by hand, on a folder, automatically, with the prompt saved so every document is read the same way.
Those are the mechanics. They are not the point.
The point is the failure path
Schema-mismatch handling is fixed behaviour. It is not a setting somebody could have switched off in a hurry six months ago.
If a refresh cannot reproduce the committed schema — a required field stops resolving, or a new column appears that the recipe has no mapping for — the run is rejected. Your last good data stays exactly where it is, untouched. Nothing half-written, nothing coerced into the wrong column, no nulls quietly standing in for values that never arrived.
Then it tells you, in the Workflow Monitor, in words: "discount: new column found in source with no mapping in the recipe." Not a stack trace, not a row number, not ERR_SCHEMA_VALIDATION. The field, and what happened to it.
That sentence is doing more work than it looks. The reason schema failures go unresolved for weeks in most stacks is not that nobody cares — it is that the error is written for the person who built the system, and the person who notices the dashboard is stale is not that person.
Strict by default, and the option to be flexible
When a replay meets something the mappings have never seen, the recipe's policy decides what happens.
Strict is the default. The run is held, with a plain-language reason, and nothing moves until a person looks at it.
Interpret-and-flag is the alternative: the AI resolves the change using the prompt you saved, and flags that it did so. Useful where the source is genuinely noisy and a held run every fortnight would train everyone to stop reading the notifications.
Strict being the default is the deliberate part. The safe behaviour is the one you get without making a decision, and the flexible behaviour is the one you opt into for a specific source where you have judged it worth it. Most systems ship this the other way round, because the demo is smoother.
A held run resolves once, and the recipe re-baselines afterwards — you are not re-answering the same question every week.
The small thing that prevents a large mess
If you have automated collection running, the same file dropped twice does not produce two runs. Dedup is by file content hash.
This sounds like housekeeping. It is the difference between a re-sent attachment being recognised as a duplicate and a month's revenue being counted twice, and duplicate-counting is a bug that survives review — because every individual number in the pipeline is correct.
What you actually get from this
Not "set it and forget it." Nobody should sell you that, and the honest version is less exciting.
What you get is automation that fails in a direction you can see. When the source changes, the run stops rather than guessing; your existing data stays valid rather than being overwritten with something worse; and the reason is written in a sentence naming the field.
Pair it with a check on the data itself and you cover both halves: the shape changing, and the values going wrong inside a shape that did not.
The dashboard being briefly stale is a visible, fixable problem. The dashboard being confidently wrong is not — and the entire design here is a series of choices to convert the second kind of failure into the first.
That is the trade. It is worth making.
