> AI agents: this is one page from Mammoth Analytics documentation. The index of all pages as Markdown is https://mammoth.io/docs/llms.txt. Append `.md` to any docs URL, or send `Accept: text/markdown`, to get Markdown.

# Import Apache Kafka topic messages into Mammoth

> Connect Mammoth to an Apache Kafka cluster with bootstrap servers and SASL, read messages from a topic with a consumer group, and save them as a dataset.

The Apache Kafka connector reads messages from a topic on your Kafka cluster and saves them in Mammoth as a dataset. Open it from **Data Library** > **New dataset** > **Connectors** and search for **Apache Kafka**. Project Admins create the connection; Project Editors can start datasets from a connection the project already has. Which connectors you can add depends on your plan.

## What you can bring in from Kafka

- Messages from one topic, picked from the topics Mammoth finds on the cluster.
- Each message's value, plus columns for **Kafka Offset**, **Kafka Partition**, **Kafka Timestamp** and **Kafka Topic**.

## Before you connect Kafka

You need:

- The cluster's bootstrap servers, reachable from the internet.
- Its security protocol, and a SASL username and password if it uses SASL.
- Mammoth's IP address allowed through your firewall for every bootstrap server. The connection form shows the address. See [Allow Mammoth through your firewall](https://mammoth.io/docs/learn/get-data-in/network-requirements/).

## Connect Apache Kafka to Mammoth

1. Go to the **Data Library**, click **New dataset**, and choose **Connectors**.
2. Search for **Apache Kafka** and click its tile. The connector workspace opens with the **Chat** showing. See [Connect a database or app](https://mammoth.io/docs/learn/get-data-in/connect-a-source/) for the shared steps.
3. Enter the connection details below and connect.

| Field | What to enter |
| --- | --- |
| **Bootstrap Servers** | The broker addresses, comma-separated, such as `broker1:9092,broker2:9092` |
| **Security Protocol** | Default `PLAINTEXT`. Use the protocol your cluster requires (see below) |
| **Username (SASL only)** | SASL username, if your cluster uses SASL |
| **Password (SASL only)** | SASL password, if your cluster uses SASL |
| **Display Name** | Optional label for this connection in Mammoth |



| Security protocol | When to use it |
| --- | --- |
| `PLAINTEXT` | The cluster needs no sign-in |
| `SASL_PLAINTEXT` | Username and password sign-in without TLS |
| `SASL_SSL` | Username and password sign-in over TLS |

With either SASL protocol, Mammoth signs in with the SASL `PLAIN` mechanism.

## Choose the Kafka topic, offset and message limit

The Kafka connector's own setup form asks for these settings:

| Setting | What it does | Default |
| --- | --- | --- |
| **Topic** | The topic to read, picked from the topics Mammoth finds on the cluster | |
| **Consumer Group ID** | The consumer group Mammoth reads as | `mammoth-consumer` |
| **Offset Reset** | **Earliest (all messages)** reads from the start of the topic; **Latest (new messages only)** reads only new messages | **Earliest (all messages)** |
| **Max Messages Per Run** | The most messages Mammoth reads in one run | `10000` |

In the connector workspace, you can tell the chat which topic you want.

## Refresh Kafka data and choose an update method

After you save, a Kafka dataset's **Update method** offers two options:

| Update method | What it does |
| --- | --- |
| **Replace all data** | Rebuilds the dataset from the messages read on every refresh |
| **Add to existing data** | Adds the messages read on each refresh to the rows you already have |

Kafka refreshes don't use a key column. To read new messages on a schedule, use a [Dataset Refresh](https://mammoth.io/docs/learn/automate/refresh-a-dataset/) automation.

## Kafka limits

- One run reads at most the **Max Messages Per Run** value (default `10000`).

## Troubleshoot Kafka connections

| Symptom | Likely cause | Fix |
| --- | --- | --- |
| "Failed to connect to Kafka broker. Please check your connection details." | Mammoth can't reach a broker | Check Mammoth's IP address can reach every bootstrap server, and the addresses are right |
| "Authentication failed. Please check your username and password." | SASL details don't match | Check **Security Protocol** and the SASL username and password match the cluster's settings |
| A message says the topic wasn't found and lists available topics | The topic name is wrong | Pick a topic from the list Mammoth shows |

## Related

- [MQTT](https://mammoth.io/docs/learn/get-data-in/connectors/mqtt/): collect messages from an MQTT broker.
- [Receive data by webhook](https://mammoth.io/docs/learn/get-data-in/webhooks/): let another system push data to Mammoth.
- [Connect a database or app](https://mammoth.io/docs/learn/get-data-in/connect-a-source/): the shared connector workspace steps.
- [Allow Mammoth through your firewall](https://mammoth.io/docs/learn/get-data-in/network-requirements/): the IP address to allow.
- [Refresh a dataset](https://mammoth.io/docs/learn/automate/refresh-a-dataset/): change when the dataset refreshes.

---
Source: https://mammoth.io/docs/learn/get-data-in/connectors/kafka.md · Updated: 2026-11-08