Serverless environment version 2
This article outlines the system environment information for serverless environment version 2.
To ensure compatibility for the application, serverless workloads use a versioned API, known as the environment version, which remains compatible with newer server versions.
You can select the environment version using the Environment panel in your serverless notebooks. See Install notebook dependencies.
For more on environment versions, see Serverless compute release notes.
New features and improvements
The following new features and improvements are available in serverless environment 2.
Notebooks are supported as workspace files
January 23, 2025
Notebooks are supported as workspace files in serverless environment 2. You can programmatically write, read, and delete notebooks just as you would any other file. This allows for programmatic interaction with notebooks from anywhere the workspace filesystem is available. For more information, see Notebooks as workspace files.
Task progress bar added to serverless compute
December 16, 2024
A new task progress bar has been added to notebook cells running on serverless compute environment version 2. This progress bar indicates the execution progress of the cell’s Spark code.
System environment
- Operating System: Ubuntu 22.04.4 LTS
- Python: 3.11.10
- Databricks Connect: 15.4.2
Installed Python libraries
| Library | Version | Library | Version | Library | Version |
|---|---|---|---|---|---|
| asttokens | 2.0.5 | astunparse | 1.6.3 | autocommand | 2.2.2 |
| azure-core | 1.31.0 | azure-storage-blob | 12.19.1 | azure-storage-file-datalake | 12.14.0 |
| backports.tarfile | 1.2.0 | black | 23.3.0 | blinker | 1.4 |
| boto3 | 1.34.39 | botocore | 1.34.39 | cachetools | 5.5.0 |
| certifi | 2023.7.22 | cffi | 1.15.1 | chardet | 4.0.0 |
| charset-normalizer | 2.0.4 | click | 8.0.4 | cloudpickle | 3.0.0 |
| comm | 0.1.2 | contourpy | 1.0.5 | cryptography | 41.0.3 |
| cycler | 0.11.0 | Cython | 0.29.32 | databricks-connect | 15.4.2 |
| databricks-sdk | 0.36.0 | dbus-python | 1.2.18 | debugpy | 1.6.7 |
| decorator | 5.1.1 | dill | 0.3.6 | distlib | 0.3.9 |
| entrypoints | 0.4 | executing | 0.8.3 | facets-overview | 1.1.1 |
| filelock | 3.13.4 | fonttools | 4.25.0 | gitdb | 4.0.11 |
| GitPython | 3.1.43 | google-api-core | 2.18.0 | google-auth | 2.35.0 |
| google-cloud-core | 2.4.1 | google-cloud-storage | 2.18.2 | google-crc32c | 1.6.0 |
| google-resumable-media | 2.7.2 | googleapis-common-protos | 1.65.0 | grpcio | 1.67.0 |
| grpcio-status | 1.67.0 | httplib2 | 0.20.2 | idna | 3.4 |
| importlib-metadata | 6.0.0 | importlib_resources | 6.4.0 | inflect | 7.3.1 |
| ipyflow-core | 0.0.201 | ipykernel | 6.28.0 | ipython | 8.25.0 |
| ipython-genutils | 0.2.0 | ipywidgets | 7.7.2 | isodate | 0.7.2 |
| jaraco.collections | 5.1.0 | jaraco.context | 5.3.0 | jaraco.functools | 4.0.1 |
| jaraco.text | 3.12.1 | jedi | 0.18.1 | jeepney | 0.7.1 |
| jmespath | 0.10.0 | joblib | 1.2.0 | jupyter_client | 7.4.9 |
| jupyter_core | 5.3.0 | keyring | 23.5.0 | kiwisolver | 1.4.4 |
| launchpadlib | 1.10.16 | lazr.restfulclient | 0.14.4 | lazr.uri | 1.0.6 |
| matplotlib | 3.7.2 | matplotlib-inline | 0.1.6 | mlflow-skinny | 2.11.4 |
| more-itertools | 8.10.0 | mypy-extensions | 0.4.3 | nest-asyncio | 1.5.6 |
| numpy | 1.23.5 | oauthlib | 3.2.0 | packaging | 23.2 |
| pandas | 1.5.3 | parso | 0.8.3 | pathspec | 0.10.3 |
| patsy | 0.5.3 | pexpect | 4.8.0 | pillow | 10.3.0 |
| pip | 24.2 | platformdirs | 3.10.0 | plotly | 5.9.0 |
| prompt_toolkit | 3.0.48 | proto-plus | 1.25.0 | protobuf | 5.28.3 |
| psutil | 5.9.0 | psycopg2 | 2.9.3 | ptyprocess | 0.7.0 |
| pure-eval | 0.2.2 | py4j | 0.10.9.7 | pyarrow | 14.0.1 |
| pyasn1 | 0.4.8 | pyasn1-modules | 0.2.8 | pyccolo | 0.0.65 |
| pycparser | 2.21 | pydantic | 1.10.6 | Pygments | 2.15.1 |
| PyGObject | 3.42.1 | PyJWT | 2.3.0 | pyodbc | 4.0.39 |
| pyparsing | 3.0.9 | python-dateutil | 2.8.2 | python-lsp-jsonrpc | 1.1.2 |
| pytz | 2022.7 | PyYAML | 6.0 | pyzmq | 25.1.2 |
| requests | 2.31.0 | rsa | 4.9 | s3transfer | 0.10.3 |
| scikit-learn | 1.3.0 | scipy | 1.11.1 | seaborn | 0.12.2 |
| SecretStorage | 3.3.1 | setuptools | 75.1.0 | six | 1.16.0 |
| smmap | 5.0.1 | sqlparse | 0.5.1 | ssh-import-id | 5.11 |
| stack-data | 0.2.0 | statsmodels | 0.14.0 | tenacity | 8.2.2 |
| threadpoolctl | 2.2.0 | tokenize-rt | 4.2.1 | tomli | 2.0.1 |
| tornado | 6.3.2 | traitlets | 5.13.0 | typeguard | 4.3.0 |
| typing_extensions | 4.10.0 | tzdata | 2022.1 | ujson | 5.4.0 |
| unattended-upgrades | 0.1 | urllib3 | 1.26.16 | virtualenv | 20.26.6 |
| wadllib | 1.3.6 | wcwidth | 0.2.5 | wheel | 0.38.4 |
| zipp | 3.11.0 | zstandard | 0.23.0 |