Release Notes Highlights for Pachyderm

2.7.0

January 1, 0001

  • Feature: The new Pachyderm SDK is now available. Check out the reference documentation, install guide, and example starter project.
  • Feature: Console now has a runtime visualization for jobs in your pipeline.
  • Feature: The documentation site now has a chatbot to help you find what you’re looking for. This feature is in beta, so please let us know if you have any feedback through our Slack community.
  • Feature: Pachyderm’s helm chart now has a section for preflight checks, allowing you to easily validate whether the upgrade/migrations will be successful. This section can be found at pachd.preflightchecks. Simply set enabled: true and set the image.tag to the new version you want to upgrade to. If created the pod named pachyderm-preflight-check shows a status of Completed, you are ready to perform the upgrade. See the Upgrade steps for more information.
  • Enhancement: Console’s scalability has been improved to handle more concurrent users (50+) and power users who have many pipelines.
  • Enhancement: Console’s DAG visualization has been upgraded to include more information about the state of your pipelines.
  • Enhancement: The Jupyterlab Pipeline Specification Extension now supports GPUs.
  • Refactor: The functionality of the Branch Cron Trigger has been refactored to work more intuitively. Previously, cron triggers functioned more like rate limiters; now, they enable you to set up a scheduled reoccurring event on a repo branch that evaluates and fires the trigger. When a Cron Trigger fires, but no new data has been added, there are no new downstream commits or jobs. See our Cron glossary entry for more information on crons in Pachyderm
  • Deprecation: The original Python SDK (python-pachyderm) will be deprecated in 9 months (May 2024). We recommend that you start trying out the new Pachyderm SDK (pachyderm-sdk) and begin planning your transition.

2.7.1

January 1, 0001

  • Fix: Resolved an issue with the pachyderm-sdk’s debug.dump method where it was using the incorrect argument type.
  • Fix: Full timestamps in pachctl have been standardized for consistency.
  • Fix: Resolved some issues in the Console log viewer UI.
  • Enhancement Tooltips have been added to Console for file table actions (Download, Delete).

2.7.2

January 1, 0001

  • Fix: The Community Edition of Console now shows the pipeline count across projects in the banner.
  • Fix: Previously when a user’s active context was still set to a deleted project, the terminal would return an error – now, it returns a warning instead.
  • Enhancement: The Pachyderm SDK now catches serialization errors and converts them into human-readable errors.

2.7.3

January 1, 0001

  • Enhancement: Refactored the Egress node to display as much of the URL as possible; for a full URL, you can find it in the Spec tab.
  • Feature: A new case for the Node component has been created to display connected projects and repos. These nodes are clickable and will route you to the appropriate resource.

2.7.4

January 1, 0001

  • Enhancement: Console has expanded download preview support for more filetypes, such as HTML/XML. Users can also now click “View Raw” to preview any unsupported filetypes. The following table is a list of all supported filetypes:

    Type Extensions Preview
    markdown .markdn, .markdown, .md, .mdown, .mdwn, .mkdn, .mkdown, .mkd (preview and view source)
    html .html, .htm (preview and view source)
    xml .xml, .xsl (preview and view source)
    code .yaml, .yml, .json, .jsonl, .py, .js, .jsx, .ts, .tsx, .cjs, .mjs, .c, .cpp, .java, .php, .rs, .sql, .go, .sh, .jl, .rb Dockerfile, .css (view source)
    text .text, .txt, .textpb (view source)
    csv .csv, .tsv, .tab (preview and view source)
    image .jpeg, .jpg, .jfif, .jif, .jpe, .pjpg, .png, .apng, .gif, .avif, .avifs, .webp, .bmp, .ico, .tiff (Safari only), .tif (Safari only) (preview)
    svg .svg, .svgz (preview and view source)
    video .mpg, .mpeg, .mpe, .m1v, .m2v, .mpa, .mp4, .mp4v, .mpg4, .avi, .wmv, .mov, .qt, .rm, .ra, .ram, .webm (preview based on your browser/OS)
    audio .mp3, .m2a, .m3a, .mp2, .mp2a, .mpga, .wav, .ogg, .oga, .spx (preview based on your browser/OS)

  • Enhancement: Timestamps in Console now use 0-23 hour notation.

2.7.5

January 1, 0001

  • Fix: Fixed a bug where the pipeline status did not display in the Console DAG when a Global ID was applied.
  • Fix: Fixed a bug where input repos were not being listed in a Pipeline’s details sidebar.
  • Enhancement: You can now add commit messages for file uploads in the Console UI.
  • Security: Added security enhancements to prevent HTTP/2 Stream Cancellation Attacks

2.9.0

January 1, 0001

  • Enhancement: The Jupyter Mount Extension is now FUSE-less. This means that the Jupyter Mount Extension no longer requires FUSE to be installed on the host machine.
  • Enhancement: We’ve migrated to using goCDK to handle blob storage configuration in the Helm chart. This means that you can now use storage URLs and pass in supported query parameters for easy setup.