Apache Kafka connector
This page shows how to connect Mammoth to an Apache Kafka cluster and read messages from a topic into a dataset.
Which connectors you can add depends on your plan.
What you need before connecting Kafka
- The cluster's bootstrap servers, reachable from the internet
- Its security protocol, and a SASL username and password if it uses SASL
- Mammoth's IP address allowed through your firewall. The connection form shows the address. See Network requirements.
Connect Apache Kafka
- Open the Connectors launcher and click Apache Kafka (see Add a connector).
- Enter the connection details:
| Field | What to enter |
|---|---|
| Bootstrap Servers | The broker addresses, such as broker1:9092 |
| Security Protocol | Default PLAINTEXT. Use the protocol your cluster requires |
| Username (SASL only) | SASL username, if your cluster uses SASL |
| Password (SASL only) | SASL password, if your cluster uses SASL |
| Display Name | Optional label for this connection in Mammoth |
Choose what to read from Kafka
The Kafka connector's own setup form asks:
- Topic: the topic to read, picked from the topics Mammoth finds on the cluster
- Consumer Group ID: default
mammoth-consumer - Offset Reset: Earliest (all messages) (default) or Latest (new messages only)
- Max Messages Per Run: default
10000
In the connector workspace, you can tell the chat which topic you want.
Kafka update methods
After you save, a Kafka dataset's Update method offers two options: Replace all data or Add to existing data. Kafka refreshes don't use a key column.
Troubleshoot Kafka connections
- Connection fails: check that Mammoth's IP address can reach every bootstrap server, and that Security Protocol and the SASL credentials match the cluster's settings.