Unified Deployment

In this guide, we’ll show you how to update your cluster configuration to support a unified installation and deployment of Pachyderm + .

Before You Start

This guide assumes that you have already completed all of the following:

  1. Deployed Pachyderm using one of the cloud deployment guides ( AWS, GCP, or Azure).
  2. Added an active Enterprise License Key.
  3. Set up TLS (SSL, HTTPS) for your Pachyderm cluster.
  4. Set up an OIDC connector for your Pachyderm cluster.

Self-Signed Certificates

If you are using a self-signed certificate for your implementation, you must update the Determined Helm values.yaml file at .Values.externalCaCertSecretName to include the name of the secret containing the root certificate.

How to Configure a Unified Setup

1. Create Necessary Secrets

You will need to create two secrets for Determined:

  1. A Determined Enterprise docker image credentials secret (e.g., det-image)

    kubectl create secret docker-registry det-image \
    --docker-server=https://index.docker.io/v1/ \
    --docker-username=<username> \
    --docker-password=<password> \
    --docker-email=<email> \
    --output=json > det-image-secret.json
  2. A Determined Enterprise admin credentials secret (e.g., det-creds)

    kubectl create secret generic det-creds \
    --from-literal=determined-username=admin \
    --from-literal=determined-password="" \
    --output=json > det-creds-secret.json

2. Update the Pachyderm Helm Chart

  1. Open your values.yaml file or generate a local copy using the following command:
    helm get values pachyderm > values.yaml
  2. Update the determined.enabled field in values.yaml file to true.
  3. Update the determined.oidc section of your values.yaml file to include the OIDC provider, client ID, and URLs:
        enabled: true
        provider: "" # your oidc.upstreamIDPs.config.id; e.g., Auth0 or Okta
        idpRecipientUrl: "" # https://<proxy.host.value.com>:8080 
        idpSsoUrl: "" # https://<proxy.host.value.com>/dex
        clientId: "determined"
        clientSecretKey: ""
        clientSecretName: ""
        authenticationClaim: ""
        scimAuthenticationAttribute: ""
        autoProvisionUsers: false
        groupsAttributeName: ""
        displayNameAttributeName: ""
  4. Provide a determined.tlsSecret if applicable.
  5. Update the pachd section of your values.yaml file to include the full endpoint address and the name of the Determined admin credentials secret:
        apiEndpoint: # https://determined-master-service-internal-<HELM RELEASE NAME>:8082
        credentialsSecretName: det-creds 
      activateEnterprise: true

Adding Users to Pipelines

You can add a determined section to your pipeline specification file and make use of a user via the $DET_USER and $DET_PASS environment variables. This can be used by the user code that run determined work to talk back to Pachyderm and can be used with the Pachyderm SDK.

    "pipeline": {
      "name": "<PIPELINE NAME>"
    "description": "<PIPELINE DESCRIPTION>",
    "input": {
      "pfs": {
        "name": "data",
        "repo": "input",
        "branch": "master",
        "glob": "/",
        "emptyFiles": true
    "transform": {
      "cmd": ["/bin/sh"],
      "stdin": ["pip install determined && echo $DET_PASS | det user login $DET_USER && det model list -w WORKSPACE-NAME  > /pfs/out/WORKSPACE-NAME.txt"],
      "image": "python:3.8"
    "determined": {
      "workspaces": ["WORKSPACE-NAME"]
These tokens life cycles are scoped to the jobs and are revoked after the job ends.