AI-powered workflows

Mammoth's AI Powered transformations apply language-model intelligence to your data: Generative AI writes new column values row by row, AI SQL Query turns a plain-English description into SQL, Bulk Replace proposes groupings for messy values, and the AI Prompt bar ("Describe your transformation") turns a typed instruction into pipeline tasks. This guide is for analysts and data engineers. It covers when to use each, how to write prompts that produce consistent output, and how to design pipelines that use AI reliably in production.

---

Which AI transform to use

Transform	What it does	Use when
Generative AI	Writes a new (or overwrites an existing) column from a prompt, using the context columns you select	Classification, extraction, translation, or scoring that needs reasoning about meaning
AI SQL Query	Generates SQL from a plain-English description, which you review before applying	Multi-column logic, conditions, or aggregations that are easier to describe than to build task by task
Bulk Replace	Groups similar values in a column and replaces them with a standard name	Many spellings of the same value (company names, statuses, locations)
AI Prompt bar	Picks the right transformation and fills in its parameters from your description	You know what you want but not which task to use

The key distinction: Generative AI calls a language model for each row, so it is flexible but needs careful prompt design for consistent output. The other three produce deterministic pipeline tasks (a SQL query, a replacement rule, or a standard function) that you can inspect and re-run with identical results.

Generative AI uses an external AI key saved in your workspace, and it has limits: 300 characters per generated value, up to 20 context columns, and only the first 50,000 rows of the view are processed. See Generative AI for the full reference.

---

Classification patterns with Generative AI

Designing categories

The quality of classification depends heavily on how the categories are defined.

Poor category set:

• Sales, Marketing, Other: vague and overlapping, and Other will be overused

Good category set:

• Pricing Inquiry, Feature Request, Bug Report, Billing Issue, Account Access, Integration Help, General Feedback, Onboarding
• Each category is distinct, has a clear definition, and covers a specific scenario

Rules for good categories:

1. Mutually exclusive: a row should clearly belong to one category
2. Collectively exhaustive: every possible input has a natural home (avoid needing Other)
3. Operationally meaningful: the output drives a decision or action
4. Few enough to validate: fewer than 3 categories rarely needs AI, and a long list is hard to check

Category definitions

Put the category list and a short definition of each in the prompt itself:

• Bug Report: "The product is behaving incorrectly or unexpectedly. Examples: 'the export button does nothing', 'I got an error when uploading'"
• Feature Request: "The user wants functionality that doesn't exist. Examples: 'can you add Excel export?', 'I wish I could filter by date'"

Definitions and examples remove ambiguity and make the output more consistent. Finish the prompt with an instruction such as Return only the category name.

Validation approach

After building a Generative AI task:

1. Use Preview to check a 5-row sample before you apply the task to the whole dataset
2. After applying, sample rows from each category—are the classifications accurate?
3. Find the rows the AI got wrong—do they reveal an ambiguous category definition?
4. Refine the prompt until accuracy is acceptable for the use case
5. Document the accepted accuracy level and the date reviewed

---

Prompt patterns

Prompt engineering for data

A Generative AI prompt is applied to every row, using the context columns you select. Write the instruction once; it runs on each row's values.

Principle: be specific about format

Bad: "Extract the product name from this order note"
Good: "Extract the product name from this order note. Return only the product name, with no extra words, punctuation, or explanation. If no product name is mentioned, return 'Unknown'."

The second version constrains the output format, making downstream tasks predictable.

Principle: specify what to do when data is missing

Always include a null/empty case:
"If the input is empty or unclear, return 'Not specified'"

Without this, the AI may return None, null, N/A, not available, or a sentence, all of which look different in downstream filtering.

Principle: show an example

For complex extractions:
"Extract the numeric value and unit from this measurement string. Examples: '14.5 kg' → '14.5', '200ml' → '200', 'N/A' → ''"

Common use cases

Select the referenced columns as the task's context columns.

Entity extraction:
"Using the email column, extract the company name from the email address domain and return only the company name (e.g. 'acme.co.uk' → 'Acme'). If unclear, return the domain without the TLD."

Standardization across sources:
"Standardize the job_title column to a standard function. Map to one of: Engineering, Sales, Marketing, Finance, Operations, HR, Legal, Executive, Other."

Address parsing:
"Parse the address column into JSON with keys: street, city, state, postal_code, country. If a component is missing, use null. Return only valid JSON." Then use Extract JSON to pull each key into its own column.

Sentiment to score:
"Rate the sentiment of the comment column from -2 (very negative) to +2 (very positive). Return only the number." The output is still a text column, so convert its type before doing math on it.

---

Deriving new columns

A Generative AI task can derive a column from several existing ones, for example:

• Description: "Customer risk tier based on days since last purchase, total spend, and account age"

  • Context columns: DaysSinceLastPurchase, TotalLifetimeSpend, AccountAgeDays
  • Output: High, Medium, Low risk tier

When the logic is a fixed rule (thresholds and arithmetic), prefer AI SQL Query or a standard task: the rule is visible, repeatable, and costs nothing per row. Use Generative AI when the derivation depends on understanding meaning.

---

Production AI pipeline design

Cost management

Generative AI processes rows individually using your own AI provider key, so usage is proportional to rows processed × input length.

Cost reduction strategies:

1. Filter before AI: Apply non-AI filters first to reduce the row count before the AI task
2. Limit the context columns: Select only the columns the prompt needs rather than leaving every column as context
3. Keep results you have already generated: Once a classification is accepted, don't re-run it on old rows—store the result and apply AI only to new rows
4. Use incremental processing: Run AI tasks only on new or changed rows, using the time-window extraction pattern

Consistency across runs

AI models can produce slightly different outputs for the same input across runs. For production pipelines:

1. Run the classification once and store the result
2. In subsequent runs, apply AI only to new rows and use the stored result for existing rows
3. Periodically re-validate a sample to detect model drift

Monitoring AI tasks

If the selected key hits its provider's rate limit, the Generative AI panel shows a warning. Add a Data Check after the AI task (for example, a Null percentage check on the generated column) so empty or failed generations are caught instead of silently flowing downstream.

When not to use AI transforms

AI transforms add latency and cost. Use standard transforms when they are sufficient:

Task	Use this standard task instead
Mapping known values to categories	Bulk Replace or Find & replace: faster and deterministic
Arithmetic on numeric columns	Math function: exact, no cost
Filtering by exact value	Conditional Filter: instant
Extracting structured substrings	Extract Text
Renaming or recasing text	Text formatting