Learn about the concept of a service pipeline.

Service is a special type of pipeline that does not process data but provides a capability to expose it to the outside world. For example, you can use a service to serve a machine learning model as an API that has the most up-to-date version of your data.

The following pipeline spec extract is an example of how you can expose your Jupyter notebook as a service by adding a service field:

    "pipeline": {
      "name": "notebook"
    "input": {
        "pfs": {
            "glob": "/",
            "repo": "input"
    "service": {
        "external_port": 30888,
        "internal_port": 8888
    "transform": {
        "cmd": [
        "image": "jupyter/datascience-notebook"

The service section specifies the following parameters:

"internal_port"The port that the code running inside the container binds to.
"external_port"The port that is exposed outside of the container. You must set this value in the range of 30000 — 32767. You can access the service from any Kubernetes node through the following address: http://<kubernetes-host>:<external_port>.

The Service starts running at the first commit in the input repo.


See Also