Reference
PachCTL

Server Setup

Learn how to set up a Pachyderm enterprise server.

March 23, 2023

The Enterprise Server is a component in Pachyderm which manages Enterprise Licensing and the integration with a company’s Identity Providers (IDPs).

An organization can have many Pachyderm clusters registered with one single Enterprise Server. Administrators activate the Enterprise Server with an Enterprise License Key from Pachyderm sales, and optionally configure authentication with their IDP via SAML, OIDC, LDAP, etc…

An Enterprise Server uses the same binary/images as a regular Pachyderm cluster, therefore, it is deployed like a typical cluster (See our Deploy/Manage section) with minor differences:

The following diagram gives you a quick overview of an organization with multiple Pachyderm clusters behind a single Enterprise Server.

Enterprise Server General Deployment

ℹī¸

For POCs and smaller organizations with one single Pachyderm cluster, the Enterprise Server services can be run embedded in pachd. A separate deployment is not necessary. An organization with a single Pachyderm cluster can run the Enterprise Server services embedded within pachd.

The setup of an Enterprise Server requires to:

  1. Deploy it.
  2. Activate your Enterprise Key and enable Auth.
  3. Register your newly created or existing Pachyderm clusters with your enterprise server.
  4. Optional: Enable Auth on each cluster.
⚠ī¸

We are now shipping Pachyderm with an embedded proxy allowing your nterprise server to expose one single port externally. This deployment setup is optional.

If you choose to deploy your enterprise server with a Proxy, check out our new recommended architecture and deployment instructions as they alter the instructions below.

1. Deploy An Enterprise Server #

Deploying and configuring an enterprise server can be done in one of two flavors:

  1. Provide all licensing and authentication configurations as a part of the Helm deployment.
  2. Or, install a bare-bones version of Pachyderm with Helm, then use pachctl commands to set up licensing and authentication.

As Part Of A Regular Pachyderm Helm Deployment #

Update your values.yaml with your enterprise license key and auth configurations (for an example on localhost, see the example values.yaml here) or check our minimal example below to your values.yaml.

⚠ī¸
  • If a pachyderm cluster will also be installed in the same kubernetes cluster, they should be installed in different namespaces:
kubectl create namespace enterprise
helm install ... --set enterpriseServer.enabled=true  --namespace enterprise

This command deploys postgres, etcd and a deployment and service called pach-enterprise. pach-enterprise uses the same docker image and pachd binary, but it listens on a different set of ports (31650, 31657, 31658) to avoid conflicts with pachd.

  • Check the state of your deployment by running:
kubectl get all --namespace enterprise

System Response

NAME                                   READY   STATUS    RESTARTS   AGE
pod/etcd-5fd7c675b6-46kz7              1/1     Running   0          113m
pod/pach-enterprise-6dc9cb8f66-rs44t   1/1     Running   0          105m
pod/postgres-6bfd7bfc47-9mz28          1/1     Running   0          113m

values.yaml for a stand-alone Enterprise Server as part of a multi-cluster deployment #

Deploying a stand-alone enterprise server requires setting the helm parameter `enterpriseServer.enabled` to `true` and the `pachd.enabled` to `false`. 
	enterpriseServer:
		enabled: true
	pachd:
		enabled: false
		enterpriseLicenseKey: "<ENTERPRISE-LICENSE-KEY>"
		# Alternatively, you can pass your license in a secret
		enterpriseLicenseKeySecretName: "<enterprise License key secret name>"
		oauthClientID: "pachd"
		oauthRedirectURI: "http://<PACHD-IP>:30657/authorization-code/callback"
		## if a secret name is not provided in `oauthClientSecretSecretName`, a secret containing `oauthClientSecret` (or a randomly generated value if empty) will be created on install and stored in the k8s secret 'pachyderm-auth` under the key `auth-config'
		oauthClientSecret: ""
		oauthClientSecretSecretName: ""
		## if a secret name is not provided in `enterpriseSecretSecretName`, a secret containing `enterpriseSecret` (or a randomly generated value if empty) will be created on install and stored in the k8s secret 'pachyderm-enterprise` under the key `enterprise-secret'
		enterpriseSecretSecretName: ""
		enterpriseSecret: ""
		activateAuth: true
		## if a secret name is not provided in `rootTokenSecretName`, a secret containing `rootToken` (or a randomly generated value if empty) will be created on install and stored in the k8s secret 'pachyderm-auth` under the key `rootToken'
		rootTokenSecretName: ""
		rootToken: ""
    externalService:
      enabled: true
	oidc:
		issuerURI: "http://<PACHD-IP>:30658/"
		## userAccessibleOauthIssuerHost is necessary in localhost settings or anytime the registered Issuer address isn't accessible outside the cluster
		# userAccessibleOauthIssuerHost: "localhost:30658"
		## if `mockIDP` is set to true, `pachd.upstreamIDPs` will be ignored in favor of a testing placeholder IDP with username/password: admin/password
		mockIDP: false
		## to set up upstream IDPs, set pachd.mockIDP to false,
		## and populate the pachd.upstreamIDPs with an array of Dex Connector configurations.
		## See the example below or https://dexidp.io/docs/connectors/
		upstreamIDPs:
		  - id: idpConnector
		    jsonConfig: >-
		      {
		          "issuer": "<ISSUER>",
		          "clientID": "<CLIENT-ID>",
		          "clientSecret": "<CLIENT-SECRET>",
		          "redirectURI": "http://<PACHD-IP>:30658/callback",
		          "insecureEnableGroups": true,
		          "insecureSkipEmailVerified": true,
		          "insecureSkipIssuerCallbackDomainCheck": true,
		          "forwardedLoginParams": ["login_hint"]
		      }
		    name: idpConnector
		    type: oidc

values.yaml for an embedded single-cluster deployment #

	pachd:
		enterpriseLicenseKey: "<ENTERPRISE-LICENSE-KEY>"
		# Alternatively, you can pass your license in a secret
		enterpriseLicenseKeySecretName: "<enterprise License key secret name>"
		oauthClientID: "pachd"
		oauthRedirectURI: "http://<PACHD-IP>:30657/authorization-code/callback"
		## if a secret name is not provided in `oauthClientSecretSecretName`, a secret containing `oauthClientSecret` (or a randomly generated value if empty) will be created on install and stored in the k8s secret 'pachyderm-auth` under the key `auth-config'
		oauthClientSecret: ""
		oauthClientSecretSecretName: ""
		## if a secret name is not provided in `enterpriseSecretSecretName`, a secret containing `enterpriseSecret` (or a randomly generated value if empty) will be created on install and stored in the k8s secret 'pachyderm-enterprise` under the key `enterprise-secret'
		enterpriseSecretSecretName: ""
		enterpriseSecret: ""
		activateAuth: true
		## if a secret name is not provided in `rootTokenSecretName`, a secret containing `rootToken` (or a randomly generated value if empty) will be created on install and stored in the k8s secret 'pachyderm-auth` under the key `rootToken'
		rootTokenSecretName: ""
		rootToken: ""
    externalService:
      enabled: true
	oidc:
		issuerURI: "http://<PACHD-IP>:30658/"
		## userAccessibleOauthIssuerHost is necessary in localhost settings or anytime the registered Issuer address isn't accessible outside the cluster
		# userAccessibleOauthIssuerHost: "localhost:30658"
		## if `mockIDP` is set to true, `pachd.upstreamIDPs` will be ignored in favor of a testing placeholder IDP with username/password: admin/password
		mockIDP: false
		## to set up upstream IDPs, set pachd.mockIDP to false,
		## and populate the pachd.upstreamIDPs with an array of Dex Connector configurations.
		## See the example below or https://dexidp.io/docs/connectors/
		upstreamIDPs:
		  - id: idpConnector
		    jsonConfig: >-
		      {
		          "issuer": "<ISSUER>",
		          "clientID": "<CLIENT-ID>",
		          "clientSecret": "<CLIENT-SECRET>",
		          "redirectURI": "http://<PACHD-IP>:30658/callback",
		          "insecureEnableGroups": true,
		          "insecureSkipEmailVerified": true,
		          "insecureSkipIssuerCallbackDomainCheck": true,
		          "forwardedLoginParams": ["login_hint"]
		      }
		    name: idpConnector
		    type: oidc

This results in a single pachd pod, with authentication enabled, and an IDP integration configured.

ℹī¸

Update the following values as follows:

PACHD-IP: The address of Pachyderm’s IP. Retrieve Pachyderm external IP address if necessary. ISSUER, CLIENT-ID, CLIENT-SECRET: Refer to our Identity Provider Configuration page.

Check the list of all available helm values at your disposal in our reference documentation or on Github.

⚠ī¸
  • When enterprise is enabled through Helm, auth is automatically activated (i.e., you do not need to run pachctl auth activate) and a pachyderm-auth k8s secret is created containing a rootToken key. Use {{"kubectl get secret pachyderm-auth -o go-template='{{.data.rootToken | base64decode }}'"}} to retrieve it and save it where you see fit.

However, this secret is only used when configuring through helm:

  • If you run pachctl auth activate, the secret is not updated. Instead, the rootToken is printed in your STDOUT for you to save.

  • Set the helm value pachd.activateAuth to false to prevent the automatic bootstrap of auth on the cluster.

On An Existing Pachyderm Cluster #

To enable the Enterprise Server on an existing cluster:

2. Activate Enterprise Licensing And Enable Authentication #

⚠ī¸

Enabling Auth will return a root token for the enterprise server. This is separate from the root tokens for each pachd (cluster). They should all be stored securely.

Once the enterprise server is deployed, deploy your cluster(s) helm install... and register it(them) with the enterprise server. Note that you have the option to register your clusters directly in your values.yaml when deploying or after its deployment, using pachctl.

You might want to expose your cluster(s) to the internet. Check the setup of a Load Balancer in our deployment section.

3. Register Your Cluster With The Enterprise Server #

Similarly to the enterprise server, we can configure our pachyderm clusters to leverage Helm for licensing and authentication in one of two flavors:

  1. Provide enterprise registration information as a part of the Helm deployment of a cluster.
  2. Register a cluster with the Enterprise Server using pachctl commands.

Register Clusters With Helm #

Add the enterprise server’s root token, and network addresses to the values.yaml of each cluster you plan to deploy and register, for the cluster and enterprise server to communicate (for an example on localhost, see the example values.yaml here), or insert our minimal example below to your values.yaml.

values.yaml with activation of an enterprise license and authentication #

pachd:
  activateEnterpriseMember: true
	enterpriseServerAddress: "grpc://<ENTERPRISE_SERVER_ADDRESS>"
	enterpriseCallbackAddress: "grpc://<PACHD_ADDRESS>"
	enterpriseServerToken: "<ENTERPRISE-SERVER-TOKEN>" # the same root token of the enterprise cluster
# Alternatively, use a secret
enterpriseServerTokenSecretName: "<Name of you secret containing enterpriseServerToken>" 
⚠ī¸

When setting your enterprise server info as part of the Helm deployment of a cluster, auth is automatically activated unless the helm value pachd.activateAuth was intentionally set to false. (i.e., you can skip step 4).

In this case, a pachyderm-auth k8s secret is automatically created containing an entry for your rootToken in the key rootToken. Use the following to retrieve it and save it where you see fit:

{{"kubectl get secret pachyderm-auth -o go-template='{{.data.rootToken | base64decode }}'"}}

Register Clusters With pachctl #

4. Enable Auth On Each Cluster #

Finally, if your clusters were registered with the Enterprise Server using pachctl, you might choose to activate auth on each (or some) of them. This is an optional step. Clusters can be registered with the enterprise server without authentication being enabled.

ℹī¸
  • Note the /authorization-code/callback appended after <pachd-IP>:657 in --redirect.
  • --client-id is to pachctl auth activate what --id is to pachctl enterprise register: In both cases, enter <my-pachd-config-name>.
pachctl config get active-enterprise-context

If not:

pachctl config set active-enterprise-context <my-enterprise-context-name>

To manage you server, its context, or connect your IdP, visit the Manage your Enterprise Server page.