Apache Hadoop connector

The Apache Hadoop connector connects Mammoth to your Hadoop cluster via HiveServer2 (the standard SQL interface for Hadoop). Import Hive tables or write HiveQL queries to bring structured data from your data lake into Mammoth datasets.

Before you start

You need:

• A Hadoop cluster running HiveServer2 (accessible from the internet)
• HiveServer2 set up to accept username and password authentication
• A Hive user with SELECT access on the relevant tables
• Your HiveServer2 hostname, port (default: 10000), database name, username, and password
• Mammoth's IP address allowed in your cluster's firewall—see Network requirements

---

Step 1: Select the Hadoop connector

1. In Mammoth, go to Data Library and click + New Dataset
2. Select Connectors
3. Search for Hadoop
4. Click the Apache Hadoop tile

Step 2: Create a connection

Click New Connection to open the credential form.

Field	Description	Example
Host URL	Hostname or IP of your HiveServer2 node	hive.cluster.example.com
Port	HiveServer2 TCP port	10000 (default)
Username	Hive user with SELECT access	mammoth_reader
Password	Password for the Hive user	—
Database	Name of the Hive database	analytics
Display Name	Optional label for this connection	Hadoop Cluster Prod

After filling in the fields, click Connect. Mammoth validates the credentials and saves the connection; if validation fails, the error message appears at the bottom of the form.

Step 3: Select a Hive schema and query your data

After you connect, select the connection under Existing Connections, then click a schema in the Schema list (the default is default). If there is only one schema, Mammoth opens it automatically. The List of tables panel shows the tables in the schema.

Click a table to load a SELECT * query for it and its preview, or write a HiveQL query in the Enter SQL query editor:

SELECT order_id, customer_id, amount, order_date
FROM analytics.orders
WHERE order_date >= '2024-01-01'
LIMIT 10000

See Understanding data preview & query for a full guide.

Step 4: Save the Hadoop dataset

1. Enter a dataset name in the Save table as field
2. Check the Update Method (the default is Replace all data). To change it, click the edit icon, pick an option, and click Save changes:
   • Replace all data—Replaces all existing data
   • Add new data—Adds new data based on a unique sequential column
   • Replace with new data—Replaces existing data with new data based on a unique sequential column For Add new data and Replace with new data, also choose a Key column—a numeric or date column with incrementing values, such as an ID or a date.
3. Click Save as dataset. When the dataset is saved, click Done

Troubleshoot Hadoop connections

Connection fails

• Confirm port 10000 is open inbound from Mammoth's IP address
• Check the user has access: run a test query in Beeline from your cluster

Slow queries

Large Hive scans can be slow. Use LIMIT clauses in your query during setup to get a fast preview, then remove them for the full dataset import.