Skip to main content

JupyterHub

Dynamic YAML Badge

The JupyterHub service is deployed on the VRE cluster using the upstream Zero to JupyterHub (z2jh) Helm charts. The cluster and the manifest are kept synchronised via Flux.

  • The current z2jh Helm chart version deployed on the CERN VRE is depicted on the badge at the top of this page.
  • The JupyterHub manifest deployed on the VRE can be found on the VRE repository - JupyterHub deployment.

For general details on the deployment of JupyterHub, we refer the user to the official documentation of Zero to JupyterHub with Kubernetes.

CERN VRE Customization​

User Authentication and Authorization​

The VRE uses the ESCAPE Indico IAM instance as Identity Provider (IdP). For further details, please go to the AAI section or visit the official ESCAPE IAM documentation.

To use the ESCAPE IAM as the JupyterHub authenticator, you will need to:

  1. Register a new client on the ESCAPE IAM instance.
  2. Configure the the JupyterHub authenticator to use the previously registered client, that makes use of an OIDC based IdP.

VRE hub.config configuration for the custom OAuth2 authenticator:

values:
  hub:
    config:
      JupyterHub:
        authenticator_class: "generic-oauth"
      RucioAuthenticator:
        # client_id: "" # set through secret
        # client_secret: "" # set through secret
        authorize_url: https://iam-escape.cloud.cnaf.infn.it/authorize
        token_url: https://iam-escape.cloud.cnaf.infn.it/token
        userdata_url: https://iam-escape.cloud.cnaf.infn.it/userinfo
        username_key: preferred_username
        enable_auth_state: true
        allow_all: true
        scope:
          - openid
          - profile
          - email
  extraConfig:
    (...)

OIDC token exchange - Rucio JupyterLab extension configuration​

The configuration of the Rucio JupyterLab extension can be found in the eponymous extension section.

To use OIDC tokens as the authentication method for the extension, the Jupyter server (i.e., the user session) needs to exchange an access token with the IdP. To do so:

  1. Enable the token-exchange grant type on the registered client (requires admin privileges).
  2. Add the token exchange configuration on the jupyterhub deployment (see below).

VRE hub.extraConfig configuration for the OIDC token exchange:

values:
  hub:
    extraConfig:
      token-exchange: |
        import pprint
        import os
        import warnings
        import requests
        from oauthenticator.generic import GenericOAuthenticator

        # custom authenticator to enable auth_state and get access token to set as env var for rucio extension
        class RucioAuthenticator(GenericOAuthenticator):
            def __init__(self, **kwargs):
                super().__init__(**kwargs)
                self.enable_auth_state = True

            def exchange_token(self, token):
                params = {
                    'client_id': self.client_id,
                    'client_secret': self.client_secret,
                    'grant_type': 'urn:ietf:params:oauth:grant-type:token-exchange',
                    'subject_token_type': 'urn:ietf:params:oauth:token-type:access_token',
                    'subject_token': token,
                    'scope': 'openid profile',
                    'audience': 'rucio'
                }
                response = requests.post(self.token_url, data=params)
                rucio_token = response.json()['access_token']

                return rucio_token
                
            async def pre_spawn_start(self, user, spawner):
                auth_state = await user.get_auth_state()
                #print("AUTH_state")
                #pprint.pprint(auth_state)
                if not auth_state:
                    # user has no auth state
                    return False
                
                # define token environment variable from auth_state
                spawner.environment['RUCIO_ACCESS_TOKEN'] = self.exchange_token(auth_state['access_token'])
        
        # set the above authenticator as the default
        c.JupyterHub.authenticator_class = RucioAuthenticator

        # enable authentication state
        c.GenericOAuthenticator.enable_auth_state = True

Finally, some extra environment variables can be set on the hub side to configure the Rucio extension:

tip

Check an example of how these variables are propagated from the hub to the user environmnet here.

VRE singleuser.extraEnv example:

values:
  singleuser:
    extraEnv:
      RUCIO_MODE: "replica"
      RUCIO_WILDCARD_ENABLED: "1"
      RUCIO_BASE_URL: "https://vre-rucio.cern.ch"
      RUCIO_AUTH_URL: "https://vre-rucio-auth.cern.ch"
      RUCIO_WEBUI_URL: "https://vre-rucio-ui.cern.ch"
      RUCIO_DISPLAY_NAME: "RUCIO - CERN VRE"
      RUCIO_NAME: "vre-rucio.cern.ch"
      RUCIO_SITE_NAME: "CERN"
      RUCIO_OIDC_AUTH: "env"
      RUCIO_OIDC_ENV_NAME: "RUCIO_ACCESS_TOKEN"
      RUCIO_DEFAULT_AUTH_TYPE: "oidc"
      RUCIO_OAUTH_ID: "rucio"
      RUCIO_DEFAULT_INSTANCE: "vre-rucio.cern.ch"
      RUCIO_DESTINATION_RSE: "CERN-EOSPILOT"
      RUCIO_RSE_MOUNT_PATH: "/eos/eulake"
      RUCIO_PATH_BEGINS_AT: "5" # Substitute /eos/pilot/eulake/escape/data with /eos/eulake
      RUCIO_CA_CERT: "/certs/rucio_ca.pem"
PRO TIP

Because both the Rucio and JupyterHub authentication use the same IdP, you could automate the creation of a rucio.cfg file at the start of an user session.

CERN VRE example to automatically create a rucio.cfg file (singleuser.lifecycleHooks.postStart configuration):

values:
  singleuser:
    lifecycleHooks:
      postStart:
        exec:
          command:
            - "sh"
            - "-c"
            - |
              mkdir -p /tmp;
              echo -n $RUCIO_ACCESS_TOKEN > /tmp/rucio_oauth.token;
              mkdir -p /opt/rucio/etc;
              echo "[client]" >> /opt/rucio/etc/rucio.cfg;
              echo "rucio_host = https://vre-rucio.cern.ch" >> /opt/rucio/etc/rucio.cfg;
              echo "auth_host = https://vre-rucio-auth.cern.ch" >> /opt/rucio/etc/rucio.cfg;
              echo "ca_cert = /certs/rucio_ca.pem" >> /opt/rucio/etc/rucio.cfg;
              echo "account = $JUPYTERHUB_USER" >> /opt/rucio/etc/rucio.cfg;
              echo "auth_type = oidc" >> /opt/rucio/etc/rucio.cfg;
              echo "oidc_audience = rucio" >> /opt/rucio/etc/rucio.cfg;
              echo "oidc_polling = true" >> /opt/rucio/etc/rucio.cfg;
              echo "oidc_issuer = escape" >> /opt/rucio/etc/rucio.cfg;
              echo "oidc_scope = openid profile offline_access" >> /opt/rucio/etc/rucio.cfg;
              echo "auth_token_file_path = /tmp/rucio_oauth.token" >> /opt/rucio/etc/rucio.cfg;

Connection to CERN Resources​

TBD:

  • Storage
  • DB
  • Network
  • Ingress
  • User Management