Skip to content
Docs
Open Mammoth
Learn › Get data in › Connectors: open the docs menu

Import files from Google Cloud Storage into Mammoth

The Google Cloud Storage (GCS) connector opens a file from a bucket and saves it in Mammoth as a dataset. It signs in with a Google Cloud service account, with no OAuth popup. Open it from Data Library > New dataset > Connectors and search for Google Cloud Storage. Project Admins create the connection; Project Editors can start datasets from a connection the project already has. Which connectors you can add depends on your plan.

What you can bring in from Google Cloud Storage

Files in any bucket the service account can see, of these types: .txt, .csv, .tsv, .psv, .xls, .xlsx, .xlsm, .zip, .bz2, .gz, .tar, .7z, .parquet, .json.

A spreadsheet file opens to its tabs.

Before you connect Google Cloud Storage

You need:

  • A GCS bucket with the files you want to import.
  • A Google Cloud service account that can list buckets (the storage.buckets.list permission) and read the objects in your bucket.
  • The service account's JSON key file, downloaded to your computer.

Create a Google Cloud service account for GCS

  1. In Google Cloud Console, go to IAM & Admin → Service Accounts.
  2. Click Create Service Account and give it a name, for example mammoth-gcs.
  3. Grant a role that includes storage.buckets.list and read access to the bucket's objects.
  4. Under Keys, click Add Key → Create new key → JSON and download the file.

Connect Google Cloud Storage to Mammoth

  1. Go to the Data Library, click New dataset, and choose Connectors.
  2. Search for Google Cloud Storage and click its tile. The connector workspace opens with the Chat connecting Google Cloud Storage.
  3. Click Upload service account file, select the JSON key file, and connect.

If you already have a GCS connection, pick it under Your connections in the launcher instead. See Connect a database or app for the shared steps.

Google Cloud Storage New Connection dialog with a service account file attached and the Connect button enabled Google Cloud Storage Existing Connections list showing one service account connection and the New Connection button

Open a file from your GCS bucket

  1. In the left panel, open a bucket and browse to the file you want. A spreadsheet opens to its tabs.
  2. Tick the columns you want. They gather in Your table.
  3. If a file can't open in the panel yet, ask the agent in the Chat to import it.
Google Cloud Storage New Dataset dialog with file tree, Exact Sheet name, Future pulls mode, and Create new dataset

Save a GCS file as a dataset

  1. Click Save as dataset.
  2. Enter a Dataset name and pick the folder under Save in.
  3. Click Save.

A dataset from a GCS file is saved once. Mammoth shows Saved once. Scheduled refresh for files is coming. A Dataset Refresh automation can't re-read GCS files.

Bring in new GCS files on a schedule

If your bucket receives new files regularly, such as daily exports named orders_2024-01-15.csv, use two automations:

  1. A File Collection automation collects new files from the bucket into Mammoth on a schedule. See What you can automate.
  2. A Data Consolidation automation adds the collected files to your dataset.

Future pulls mode in the older import dialog

The older Google Cloud Storage - New Dataset dialog, which Mammoth still opens in some cases, shows a Future pulls mode list after you select a file:

OptionWhat it does
Pull same fileEvery refresh re-reads the file you selected
Pull next file based on name patternAvailable only when a single file is selected. Choose the Folder to pull from and enter a File pattern.

In a file pattern, put a date format inside <>. For the file sftp_Tue-01-November-2022_daily.csv, the pattern is sftp_<dy>-<dd>-<mon>-<yyyy>_daily.csv.

  • Supported date formats: dd, mm, yyyy, yy, month, mon, day, dy, hh, mi and ss, in full lower case, upper case or capitalized.
  • The selected file name must contain at least one date format.
  • The pattern must match the selected file name.

Troubleshoot Google Cloud Storage connections

SymptomLikely causeFix
"Permission denied"The service account can't list bucketsGrant it a role that includes storage.buckets.list and access to the bucket's objects. Check permissions in GCS, then the bucket, then Permissions.
"Invalid service account file. Please upload a valid JSON key file."The file isn't a service account JSON keyDownload a new key from Keys in Google Cloud Console and upload it
"Authentication failed. Please verify your service account JSON file."Google rejected the keyConfirm the key still exists and the service account is enabled
"No buckets found for this service account."The service account can't see any bucketGive it access to the bucket
"No supported files found in bucket"The bucket is empty or holds only unsupported typesAdd files of a supported type (see above)
Was this page helpful?

Something wrong? support@mammoth.io