GCP + Pachyderm
Learn how to deploy to Pachyderm to the cloud with GCP.
March 22, 2023
Before You Start #
This guide assumes that:
- You have already tried Pachyderm locally and have some familiarity with Kubectl, Helm, Google Cloud SDK and jq.
- You have access to a Google Cloud account linked to an active billing account.
⚠️
This is not a production-level setup guide; see the Google Cloud Platform deploy guide for in-depth setup.
1. Create a New Project #
- Log in to Google Cloud Console.
- Create a new project (e.g.,
pachyderm-quickstart-project
). - Enable the Compute Engine API.
You are now ready to create a GKE Cluster.
2. Run Setup Script #
You can run this setup script either through the Cloud Shell or in a local terminal via the gcloud
cli. Running this script creates all of the following:
- One GKE cluster
- Workload identity service accounts
- Permissions
- A static IP address
- The cloud SQL instance and databases
- One cloud storage bucket
- One file called
${NAME}.values.yaml
in the current directory
It also installs Pachyderm into the cluster.
3. Connect to Cluster #
- Run the following:
pachctl config import-kube local --overwrite
pachctl config set active-context local
pachctl port-forward
- Open your browser at localhost:4000.
💡
You can also connect to Console via Google’s Cloud Shell: