JupyterHub
The JupyterHub service is deployed on the VRE cluster using the upstream Zero to JupyterHub (z2jh
) Helm charts. The cluster and the manifest are kept synchronised via Flux.
- The current
z2jh
Helm chart version deployed on the CERN VRE is depicted on the badge at the top of this page.- The Application version can be checked on the official
z2jh
Helm chart repository.
- The Application version can be checked on the official
- The JupyterHub manifest deployed on the VRE can be found on the VRE repository - JupyterHub deployment.
For general details on the deployment of JupyterHub, we refer the user to the official documentation of Zero to JupyterHub with Kubernetes.
CERN VRE Customization
User Authentication and Authorization
The VRE uses the ESCAPE Indico IAM instance as Identity Provider (IdP). For further details, please go to the AAI section or visit the official ESCAPE IAM documentation.
To use the ESCAPE IAM as the JupyterHub authenticator, you will need to:
- Register a new client on the ESCAPE IAM instance.
- Configure the the JupyterHub authenticator to use the previously registered client, that makes use of an OIDC based IdP.
VRE hub.config
configuration for the custom OAuth2 authenticator:
values:
hub:
config:
JupyterHub:
authenticator_class: "generic-oauth"
RucioAuthenticator:
# client_id: "" # set through secret
# client_secret: "" # set through secret
authorize_url: https://iam-escape.cloud.cnaf.infn.it/authorize
token_url: https://iam-escape.cloud.cnaf.infn.it/token
userdata_url: https://iam-escape.cloud.cnaf.infn.it/userinfo
username_key: preferred_username
enable_auth_state: true
allow_all: true
scope:
- openid
- profile
- email
extraConfig:
(...)
OIDC token exchange - Rucio JupyterLab extension configuration
The configuration of the Rucio JupyterLab extension can be found in the eponymous extension section.
To use OIDC tokens as the authentication method for the extension, the Jupyter server (i.e., the user session) needs to exchange an access token with the IdP. To do so:
- Enable the
token-exchange
grant type on the registered client (requires admin privileges). - Add the token exchange configuration on the jupyterhub deployment (see below).
VRE hub.extraConfig
configuration for the OIDC token exchange:
values:
hub:
extraConfig:
token-exchange: |
import pprint
import os
import warnings
import requests
from oauthenticator.generic import GenericOAuthenticator
# custom authenticator to enable auth_state and get access token to set as env var for rucio extension
class RucioAuthenticator(GenericOAuthenticator):
def __init__(self, **kwargs):
super().__init__(**kwargs)
self.enable_auth_state = True
def exchange_token(self, token):
params = {
'client_id': self.client_id,
'client_secret': self.client_secret,
'grant_type': 'urn:ietf:params:oauth:grant-type:token-exchange',
'subject_token_type': 'urn:ietf:params:oauth:token-type:access_token',
'subject_token': token,
'scope': 'openid profile',
'audience': 'rucio'
}
response = requests.post(self.token_url, data=params)
rucio_token = response.json()['access_token']
return rucio_token
async def pre_spawn_start(self, user, spawner):
auth_state = await user.get_auth_state()
#print("AUTH_state")
#pprint.pprint(auth_state)
if not auth_state:
# user has no auth state
return False
# define token environment variable from auth_state
spawner.environment['RUCIO_ACCESS_TOKEN'] = self.exchange_token(auth_state['access_token'])
# set the above authenticator as the default
c.JupyterHub.authenticator_class = RucioAuthenticator
# enable authentication state
c.GenericOAuthenticator.enable_auth_state = True
Finally, some extra environment variables can be set on the hub side to configure the Rucio extension:
Check an example of how these variables are propagated from the hub to the user environmnet here.
VRE singleuser.extraEnv
example:
values:
singleuser:
extraEnv:
RUCIO_MODE: "replica"
RUCIO_WILDCARD_ENABLED: "1"
RUCIO_BASE_URL: "https://vre-rucio.cern.ch"
RUCIO_AUTH_URL: "https://vre-rucio-auth.cern.ch"
RUCIO_WEBUI_URL: "https://vre-rucio-ui.cern.ch"
RUCIO_DISPLAY_NAME: "RUCIO - CERN VRE"
RUCIO_NAME: "vre-rucio.cern.ch"
RUCIO_SITE_NAME: "CERN"
RUCIO_OIDC_AUTH: "env"
RUCIO_OIDC_ENV_NAME: "RUCIO_ACCESS_TOKEN"
RUCIO_DEFAULT_AUTH_TYPE: "oidc"
RUCIO_OAUTH_ID: "rucio"
RUCIO_DEFAULT_INSTANCE: "vre-rucio.cern.ch"
RUCIO_DESTINATION_RSE: "CERN-EOSPILOT"
RUCIO_RSE_MOUNT_PATH: "/eos/eulake"
RUCIO_PATH_BEGINS_AT: "5" # Substitute /eos/pilot/eulake/escape/data with /eos/eulake
RUCIO_CA_CERT: "/certs/rucio_ca.pem"
Because both the Rucio and JupyterHub authentication use the same IdP, you could automate the creation of a rucio.cfg
file at the start of an user session.
CERN VRE example to automatically create a rucio.cfg
file (singleuser.lifecycleHooks.postStart
configuration):
values:
singleuser:
lifecycleHooks:
postStart:
exec:
command:
- "sh"
- "-c"
- |
mkdir -p /tmp;
echo -n $RUCIO_ACCESS_TOKEN > /tmp/rucio_oauth.token;
mkdir -p /opt/rucio/etc;
echo "[client]" >> /opt/rucio/etc/rucio.cfg;
echo "rucio_host = https://vre-rucio.cern.ch" >> /opt/rucio/etc/rucio.cfg;
echo "auth_host = https://vre-rucio-auth.cern.ch" >> /opt/rucio/etc/rucio.cfg;
echo "ca_cert = /certs/rucio_ca.pem" >> /opt/rucio/etc/rucio.cfg;
echo "account = $JUPYTERHUB_USER" >> /opt/rucio/etc/rucio.cfg;
echo "auth_type = oidc" >> /opt/rucio/etc/rucio.cfg;
echo "oidc_audience = rucio" >> /opt/rucio/etc/rucio.cfg;
echo "oidc_polling = true" >> /opt/rucio/etc/rucio.cfg;
echo "oidc_issuer = escape" >> /opt/rucio/etc/rucio.cfg;
echo "oidc_scope = openid profile offline_access" >> /opt/rucio/etc/rucio.cfg;
echo "auth_token_file_path = /tmp/rucio_oauth.token" >> /opt/rucio/etc/rucio.cfg;
Connection to CERN Resources
TBD:
- Storage
- DB
- Network
- Ingress
- User Management