Databricks Runtime 16.1 voor Machine Learning
Databricks Runtime 16.1 voor Machine Learning biedt een kant-en-klareto-go omgeving voor machine learning en gegevenswetenschap op basis van Databricks Runtime 16.1. Databricks Runtime ML bevat veel populaire machine learning-bibliotheken, waaronder TensorFlow, PyTorch en XGBoost. Databricks Runtime ML bevat AutoML-, een hulpprogramma voor het automatisch trainen van machine learning-pijplijnen. Databricks Runtime ML ondersteunt ook gedistribueerde deep learning-training met behulp van TorchDistributor, DeepSpeed en Ray.
Tip
Zie de release-opmerkingen voor Databricks Runtime-versies die het einde van de ondersteuning (EoS) hebben bereikt, zie Einde van de ondersteuning voor Databricks Runtime release-opmerkingen. De EoS Databricks Runtime-versies zijn buiten gebruik gesteld en worden mogelijk niet bijgewerkt.
Nieuwe functies en verbeteringen
Databricks Runtime 16.1 ML is gebouwd op Databricks Runtime 16.1. Zie de releaseopmerkingen van Databricks Runtime 16.1 voor informatie over de nieuwe functies in Databricks Runtime 16.1, inclusief Apache Spark MLlib en SparkR.
Systeemomgeving
De systeemomgeving in Databricks Runtime 16.1 ML verschilt als volgt van Databricks Runtime 16.1.
- Voor GPU-clusters bevat Databricks Runtime ML de volgende NVIDIA GPU-bibliotheken:
- CUDA 12.6
- cublas 12.6.0.22-1
- cusolver 11.6.4.38-1
- cupti 12.6.37-1
- cusparse 12.5.2.23-1
- cuDNN 9.3.0.75-1
- NCCL 2.22.3
- TensorRT 10.2.0.19-1
Bibliotheken
In de volgende secties worden de bibliotheken vermeld die zijn opgenomen in Databricks Runtime 16.1 ML die verschillen van de bibliotheken die zijn opgenomen in Databricks Runtime 16.1.
In deze sectie:
- bibliotheken van topniveau
- Python-bibliotheken
- R-bibliotheken
- Java- en Scala-bibliotheken (Scala 2.12-cluster)
Topbibliotheken
Databricks Runtime 16.1 ML bevat de volgende uitmuntende bibliotheken:
- gegevenssets
- GraphFrames
- MLflow
- PyTorch-
- spark-tensorflow-connector
- Scikit-learn
- streaming
- TensorFlow
- TensorBoard-
- transformatoren
Python-bibliotheken
Databricks Runtime 16.1 ML maakt gebruik van virtualenv voor Python-pakketbeheer en bevat veel populaire ML-pakketten.
Naast de pakketten die in de volgende secties zijn opgegeven, bevat Databricks Runtime 16.1 ML ook de volgende pakketten:
- hyperopt 0.2.9+db5
- automl 1.29.0
Als u de Databricks Runtime ML Python-omgeving in uw lokale virtuele Python-omgeving wilt reproduceren, downloadt u het requirements-16.1.txt-bestand en voert u pip install -r requirements-16.1.txtuit. Met deze opdracht worden alle opensourcebibliotheken geïnstalleerd die door Databricks Runtime ML worden gebruikt, maar worden geen bibliotheken geïnstalleerd die zijn ontwikkeld door Databricks, zoals databricks-automl, databricks-feature-engineeringof de Databricks-fork van hyperopt.
Python-bibliotheken op CPU-clusters
| Bibliotheek | Versie | Bibliotheek | Versie | Bibliotheek | Versie |
|---|---|---|---|---|---|
| absl-py | 1.0.0 | versnellen | 1.1.1 | aiohttp | 3.9.5 |
| aiohttp-cors | 0.7.0 | aiosignal | 1.2.0 | alembiek | 1.13.3 |
| geannoteerde typen | 0.7.0 | anyio | 4.2.0 | argcomplete | 3.5.0 |
| argon2-cffi | 21.3.0 | argon2-cffi-bindingen | 21.2.0 | pijl | 1.2.3 |
| Astor | 0.8.1 | asttokens | 2.0.5 | astunparse | 1.6.3 |
| async-lru | 2.0.4 | attrs | 23.1.0 | audioread | 3.0.1 |
| autocommando | 2.2.2 | azure-core | 1.31.0 | azure-cosmos | 4.3.1 |
| azure-identity | 1.18.0 | azure-storage-blob | 12.23.1 | azure-storage-file-datalake | 12.17.0 |
| Babel | 2.11.0 | terugtrekken | 2.2.1 | backports.tarfile | 1.2.0 |
| bcrypt | 3.2.0 | beautifulsoup4 | 4.12.3 | zwart | 24.4.2 |
| bleekmiddel | 4.1.0 | richtingaanwijzer | 1.7.0 | blis | 0.7.11 |
| boto3 | 1.34.69 | botocore | 1.34.69 | Brotli | 1.0.9 |
| cachetools | 5.3.3 | catalogus | 2.0.10 | category-encoders | 2.6.3 |
| certifi | 2024.6.2 | cffi | 1.16.0 | chardet | 4.0.0 |
| charset-normalizer | 2.0.4 | stroomonderbreker | 2.0.0 | klikken | 8.1.7 |
| cloudpathlib | 0.19.0 | cloudpickle | 2.2.1 | cmdstanpy | 1.2.4 |
| kleurrijk | 0.5.6 | kleurlog | 6.8.2 | Communicatie | 0.2.1 |
| componist | 0.27.0 | suikergoed | 0.1.5 | configparser | 5.2.0 |
| contourpy | 1.2.0 | coolname | 2.2.0 | cryptografie | 42.0.5 |
| wielrijder | 0.11.0 | cymem | 2.0.10 | Cython | 3.0.11 |
| daciet | 1.8.1 | databricks-automl-runtime | 0.2.21 | databricks-feature-engineering | 0.7.0 |
| databricks-sdk | 0.30.0 | Datasets | 3.1.0 | dbl-tempo | 0.1.26 |
| dbus-python | 1.3.2 | debugpy | 1.6.7 | decorateur | 5.1.1 |
| deepspeed | 0.16.0 | defusedxml | 0.7.1 | Verouderd | 1.2.14 |
| dille | 0.3.8 | distlib | 0.3.8 | dm-tree | 0.1.8 |
| docstring-to-markdown | 0.11 | invoerpunten | 0.4 | evalueren | 0.4.3 |
| Uitvoeren | 0.8.3 | overzicht van facetten | 1.1.1 | Farama-Notifications | 0.0.4 |
| fastjsonschema | 2.21.1 | fasttext-wheel | 0.9.2 | bestandsvergrendeling | 3.13.1 |
| Fles | 2.2.5 | flatbuffers | 24.3.25 | fonttools | 4.51.0 |
| Fqdn | 1.5.1 | bevroren lijst | 1.4.0 | fsspec | 2023.5.0 |
| toekomst | 0.18.3 | gast | 0.4.0 | gitdb | 4.0.11 |
| GitPython | 3.1.37 | google-api-core | 2.20.0 | google-auth | 2.21.0 |
| google-auth-oauthlib | 1.0.0 | google-cloud-core | 2.4.1 | google-cloud-storage | 2.10.0 |
| google-crc32c | 1.6.0 | google-pasta | 0.2.0 | google-resumable-media | 2.7.2 |
| googleapis-common-protos | 1.65.0 | gql | 3.5.0 | graphql-core | 3.2.4 |
| greenlet | 3.0.1 | grpcio | 1.60.0 | grpcio-status | 1.60.0 |
| gunicorn | 20.1.0 | gviz-api | 1.10.0 | gymnastiekzaal | 0.28.1 |
| h11 | 0.14.0 | h5py | 3.11.0 | hjson | 3.1.0 |
| vakantie | 0.54 | htmlmin | 0.1.12 | httpcore | 1.0.5 |
| httplib2 | 0.20.4 | httpx | 0.28.0 | huggingface-hub | 0.24.5 |
| idna | 3.7 | ImageHash | 4.3.1 | imageio | 2.33.1 |
| onevenwichtig leren | 0.12.3 | importlib-metadata | 6.0.0 | importlib_resources | 6.4.5 |
| vervoegen | 7.3.1 | ipyflow-core | 0.0.201 | ipykernel | 6.28.0 |
| ipython | 8.25.0 | ipython-genutils | 0.2.0 | ipywidgets | 7.7.2 |
| isodate | 0.6.1 | isoduratie | 20.11.0 | itsdangerous | 2.2.0 |
| jaraco.context | 5.3.0 | jaraco.functools | 4.0.1 | jaraco.text | 3.12.1 |
| jax-jumpy | 1.0.0 | jedi | 0.19.1 | Jinja2 | 3.1.4 |
| jiter | 0.8.0 | jmespath | 1.0.1 | joblib | 1.4.2 |
| joblibspark | 0.5.1 | json5 | 0.9.6 | jsonpatch | 1.33 |
| jsonpointer | 3.0.0 | jsonschema | 4.19.2 | jsonschema-specifications | 2023.7.1 |
| jupyter-events | 0.10.0 | jupyter-lsp | 2.2.0 | jupyter_client | 8.6.0 |
| jupyter_core | 5.7.2 | jupyter_server | 2.14.1 | Jupyter-serverterminals | 0.4.4 |
| jupyterlab | 4.0.11 | jupyterlab-pygments | 0.1.2 | jupyterlab_server | 2.25.1 |
| keras | 3.6.0 | kiwisolver | 1.4.4 | langchain | 0.3.8 |
| langchain-core | 0.2.41 | langchain-text-splitters | 0.2.4 | langcodes | 3.4.1 |
| langsmith | 0.1.129 | taalgegevens | 1.2.0 | launchpadlib | 1.11.0 |
| lazr.restfulclient | 0.14.6 | lazr.uri | 1.0.6 | lazy_loader | 0.4 |
| libclang | 15.0.6.1 | librosa | 0.10.2 | lightgbm | 4.5.0 |
| bliksemhulpprogramma's | 0.11.9 | linkify-it-py | 2.0.0 | llvmlite | 0.42.0 |
| lz4 | 4.3.2 | Mako | 1.2.0 | marisa-trie | 1.2.0 |
| Markdown | 3.4.1 | markdown-it-py | 2.2.0 | MarkupSafe | 2.1.3 |
| matplotlib | 3.8.4 | matplotlib-inline | 0.1.6 | Mccabe | 0.7.0 |
| mdit-py-plugins | 0.3.0 | mdurl | 0.1.0 | memray | 1.14.0 |
| verkeerd afstemmen | 2.0.4 | ml-dtypes | 0.4.1 | mlflow-skinny | 2.15.1 |
| more-itertools | 10.3.0 | mosaicml-cli | 0.6.41 | mosaicml-streaming | 0.8.0 |
| mpmath | 1.3.0 | msal | 1.31.1 | msal-extensions | 1.2.0 |
| msgpack | 1.1.0 | multidict | 6.0.4 | multimethode | 1.12 |
| meerdere processen | 0.70.16 | murmurhash | 1.0.11 | mypy | 1.10.0 |
| mypy-extensions | 1.0.0 | namex | 0.0.8 | nbclient | 0.8.0 |
| nbconvert | 7.10.0 | nbformat | 5.9.2 | nest-asyncio | 1.6.0 |
| networkx | 3.2.1 | ninja | 1.11.1.1 | nltk | 3.8.1 |
| nodeenv | 1.9.1 | schrift | 7.0.8 | notebook_shim | 0.2.3 |
| numba | 0.59.1 | numpy | 1.26.4 | nvidia-ml-py | 12.560.30 |
| oauthlib | 3.2.0 | Oci | 2.139.0 | openai | 1.55.1 |
| opencensus | 0.11.4 | opencensus-context | 0.1.3 | opentelemetry-api | 1.28.2 |
| opentelemetry-sdk | 1.28.2 | opentelemetry-semantic-conventions | 0,49b2 | opt_einsum | 3.4.0 |
| optree | 0.12.1 | optuna | 3.6.1 | optuna-integration | 3.6.0 |
| orjson | 3.10.12 | Overschrijft | 7.4.0 | verpakking | 24.1 |
| pandas | 1.5.3 | pandocfilters | 1.5.0 | paramiko | 3.4.0 |
| parso | 0.8.3 | pathspec | 0.10.3 | Patsy | 0.5.6 |
| pexpect | 4.8.0 | phik | 0.12.4 | kussen | 10.3.0 |
| pit | 24.2 | platformdirs | 3.10.0 | plotly | 5.22.0 |
| plugachtig | 1.0.0 | pmdarima | 2.0.4 | hondje | 1.8.2 |
| portalocker | 2.10.1 | vooraf geslagen | 3.0.9 | prometheus-client | 0.14.1 |
| prompt-toolkit | 3.0.43 | profeet | 1.1.5 | proto-plus | 1.24.0 |
| protobuf | 4.24.1 | psutil | 5.9.0 | psycopg2 | 2.9.3 |
| ptyprocess | 0.7.0 | pure-eval | 0.2.2 | py-cpuinfo | 9.0.0 |
| py-spy | 0.3.14 | pyarrow | 15.0.2 | pyarrow-hotfix | 0.6 |
| pyasn1 | 0.4.8 | pyasn1-modules | 0.2.8 | pybind11 | 2.13.6 |
| pyccolo | 0.0.65 | pycparser | 2.21 | pydantic | 2.8.2 |
| pydantic_core | 2.20.1 | pyflakes | 3.2.0 | Pygments | 2.15.1 |
| PyGObject | 3.48.2 | PyJWT | 2.7.0 | PyNaCl | 1.5.0 |
| pyodbc | 5.0.1 | pyOpenSSL | 24.0.0 | pyparsing | 3.0.9 |
| pyright | 1.1.294 | pytesseract | 0.3.10 | python-dateutil | 2.9.0.post0 |
| Python-bewerker | 1.0.4 | python-json-logger | 2.0.7 | python-lsp-jsonrpc | 1.1.2 |
| python-lsp-server | 1.10.0 | python-snappy | 0.6.1 | pytoolconfig | 1.2.6 |
| pytorch-ranger | 0.1.1 | pytz | 2024.1 | PyWavelets | 1.5.0 |
| PyYAML | 6.0.1 | pyzmq | 25.1.2 | vraagteken | 1.10.0 |
| straal | 2.35.0 | Verwijzen naar | 0.30.2 | regex | 2023.10.3 |
| Verzoeken | 2.32.2 | requests-oauthlib | 1.3.1 | rfc3339-validator | 0.1.4 |
| rfc3986-validator | 0.1.1 | rijk | 13.3.5 | touw | 1.12.0 |
| rpds-py | 0.10.6 | RSA | 4.9 | ruamel.yaml | 0.18.6 |
| ruamel.yaml.clib | 0.2.8 | s3transfer | 0.10.2 | safetensors | 0.4.4 |
| scikit-image | 0.23.2 | scikit-learn | 1.4.2 | scipy | 1.13.1 |
| seaborn | 0.13.2 | Send2Trash | 1.8.2 | sentence-transformers | 3.3.1 |
| zinsstuk | 0.2.0 | setuptools | 74.0.0 | Shap | 0.46.0 |
| shellingham | 1.5.4 | simplejson | 3.17.6 | Zes | 1.16.0 |
| Snijmachine | 0.0.8 | intelligent openen | 5.2.1 | smmap | 5.0.0 |
| sniffio | 1.3.0 | geluidsbestand | 0.12.1 | soepzeef | 2.5 |
| soxr | 0.5.0.post1 | spatie | 3.7.5 | spacy-legacy | 3.0.12 |
| spacy-loggers | 1.0.5 | SQLAlchemy | 2.0.30 | sqlparse | 0.4.2 |
| srsly | 2.4.8 | ssh-import-id | 5.11 | stapelgegevens | 0.2.0 |
| stanio | 0.5.1 | statsmodels | 0.14.2 | sympy | 1.13.1 |
| Tabulate | 0.9.0 | tangled-up-in-unicode | 0.2.0 | vasthoudendheid | 8.2.2 |
| tensorboard | 2.18.0 | tensorboard-data-server | 0.7.2 | tensorboard-plugin-profile | 2.18.0 |
| tensorboardX | 2.6.2.2 | tensorflow | 2.18.0 | tensorflow-estimator | 2.15.0 |
| termcolor | 2.4.0 | afgerond | 0.17.1 | Tekstuele | 0.87.1 |
| tf_keras | 2.18.0 | thinc | 8.2.5 | threadpoolctl | 2.2.0 |
| tifffile | 2023.4.12 | tiktoken | 0.7.0 | tinycss2 | 1.2.1 |
| tokenize-rt | 4.2.1 | tokenisatoren | 0.20.0 | tomli | 2.0.1 |
| fakkel | 2.5.0+cpu | torch-optimizer | 0.3.0 | torcheval | 0.0.7 |
| torchmetrics | 1.4.0.post0 | torchvision | 0.20.0+cpu | tornado | 6.4.1 |
| tqdm | 4.66.4 | traitlets | 5.14.3 | Transformatoren | 4.46.3 |
| typeguard | 4.3.0 | typist | 0.12.5 | types-protobuf | 3.20.3 |
| types-psutil | 5.9.0 | types-pytz | 2023.3.1.1 | types-PyYAML | 6.0.0 |
| typen verzoeken | 2.31.0.0 | types-setuptools | 68.0.0.0 | types-zes | 1.16.0 |
| types-urllib3 | 1.26.25.14 | typing_extensions | 4.11.0 | uc-micro-py | 1.0.1 |
| ujson | 5.10.0 | onbeheerde upgrades | 0.1 | URI-sjabloon | 1.3.0 |
| urllib3 | 1.26.16 | validerende entiteiten | 0.34.0 | virtualenv | 20.26.2 |
| Visies | 0.7.5 | wadllib | 1.3.6 | wasabi | 1.1.3 |
| wcwidth | 0.2.5 | wezel | 0.4.1 | webcolors | 24.8.0 |
| webcoderingen | 0.5.1 | websocket-client | 1.8.0 | websockets | 11.0.3 |
| gereedschap | 3.0.3 | whatthepatch | 1.0.2 | wiel | 0.43.0 |
| woordwolk | 1.9.3 | ingepakt | 1.14.1 | xgboost | 2.0.3 |
| xgboost-ray | 0.1.19 | xxhash | 3.4.1 | yapf | 0.33.0 |
| yarl | 1.9.3 | ydata-profiling | 4.9.0 | zipp | 3.17.0 |
| zstd | 1.5.5.1 |
Python-bibliotheken op GPU-clusters
Notitie
PyTorch maakt gebruik van de CUDA PyPI-afhankelijkheden om CUDA-ondersteuning te bieden in plaats van de CUDA-bibliotheekversies die zijn ingebouwd in Databricks Runtime 16.1 ML.
| Bibliotheek | Versie | Bibliotheek | Versie | Bibliotheek | Versie |
|---|---|---|---|---|---|
| absl-py | 1.0.0 | versnellen | 1.1.1 | aiohttp | 3.9.5 |
| aiohttp-cors | 0.7.0 | aiosignal | 1.2.0 | geannoteerde typen | 0.7.0 |
| anyio | 4.2.0 | argcomplete | 3.5.0 | argon2-cffi | 21.3.0 |
| argon2-cffi-bindingen | 21.2.0 | pijl | 1.2.3 | Astor | 0.8.1 |
| asttokens | 2.0.5 | astunparse | 1.6.3 | async-lru | 2.0.4 |
| attrs | 23.1.0 | audioread | 3.0.1 | autocommando | 2.2.2 |
| azure-core | 1.31.0 | azure-cosmos | 4.3.1 | azure-identity | 1.18.0 |
| azure-storage-blob | 12.23.1 | azure-storage-file-datalake | 12.17.0 | Babel | 2.11.0 |
| terugtrekken | 2.2.1 | backports.tarfile | 1.2.0 | bcrypt | 3.2.0 |
| beautifulsoup4 | 4.12.3 | zwart | 24.4.2 | bleekmiddel | 4.1.0 |
| richtingaanwijzer | 1.7.0 | blis | 0.7.11 | boto3 | 1.34.69 |
| botocore | 1.34.69 | Brotli | 1.0.9 | cachetools | 5.3.3 |
| catalogus | 2.0.10 | category-encoders | 2.6.3 | certifi | 2024.6.2 |
| cffi | 1.16.0 | chardet | 4.0.0 | charset-normalizer | 2.0.4 |
| stroomonderbreker | 2.0.0 | klikken | 8.1.7 | cloudpathlib | 0.19.0 |
| cloudpickle | 2.2.1 | cmdstanpy | 1.2.4 | kleurrijk | 0.5.6 |
| kleurlog | 6.8.2 | Communicatie | 0.2.1 | componist | 0.27.0 |
| suikergoed | 0.1.5 | configparser | 5.2.0 | contourpy | 1.2.0 |
| coolname | 2.2.0 | cryptografie | 42.0.5 | wielrijder | 0.11.0 |
| cymem | 2.0.10 | Cython | 3.0.11 | daciet | 1.8.1 |
| databricks-automl-runtime | 0.2.21 | databricks-feature-engineering | 0.7.0 | databricks-sdk | 0.30.0 |
| Datasets | 3.1.0 | dbl-tempo | 0.1.26 | dbus-python | 1.3.2 |
| debugpy | 1.6.7 | decorateur | 5.1.1 | deepspeed | 0.16.0 |
| defusedxml | 0.7.1 | Verouderd | 1.2.14 | dille | 0.3.8 |
| distlib | 0.3.8 | dm-tree | 0.1.8 | docstring-to-markdown | 0.11 |
| einops | 0.8.0 | invoerpunten | 0.4 | evalueren | 0.4.3 |
| Uitvoeren | 0.8.3 | overzicht van facetten | 1.1.1 | Farama-Notifications | 0.0.4 |
| fastjsonschema | 2.21.1 | fasttext-wheel | 0.9.2 | bestandsvergrendeling | 3.13.1 |
| flash_attn | 2.7.0 | Fles | 2.2.5 | flatbuffers | 24.3.25 |
| fonttools | 4.51.0 | Fqdn | 1.5.1 | bevroren lijst | 1.4.0 |
| fsspec | 2023.5.0 | toekomst | 0.18.3 | gast | 0.4.0 |
| gitdb | 4.0.11 | GitPython | 3.1.37 | google-api-core | 2.20.0 |
| google-auth | 2.21.0 | google-auth-oauthlib | 1.0.0 | google-cloud-core | 2.4.1 |
| google-cloud-storage | 2.10.0 | google-crc32c | 1.6.0 | google-pasta | 0.2.0 |
| google-resumable-media | 2.7.2 | googleapis-common-protos | 1.65.0 | gql | 3.5.0 |
| graphql-core | 3.2.4 | greenlet | 3.0.1 | grpcio | 1.60.0 |
| grpcio-status | 1.60.0 | gunicorn | 20.1.0 | gviz-api | 1.10.0 |
| gymnastiekzaal | 0.28.1 | h11 | 0.14.0 | h5py | 3.11.0 |
| hjson | 3.1.0 | vakantie | 0.54 | htmlmin | 0.1.12 |
| httpcore | 1.0.5 | httplib2 | 0.20.4 | httpx | 0.27.2 |
| huggingface-hub | 0.24.5 | idna | 3.7 | ImageHash | 4.3.1 |
| imageio | 2.33.1 | onevenwichtig leren | 0.12.3 | importlib-metadata | 6.0.0 |
| importlib_resources | 6.4.5 | vervoegen | 7.3.1 | ipyflow-core | 0.0.201 |
| ipykernel | 6.28.0 | ipython | 8.25.0 | ipython-genutils | 0.2.0 |
| ipywidgets | 7.7.2 | isodate | 0.6.1 | isoduratie | 20.11.0 |
| itsdangerous | 2.2.0 | jaraco.context | 5.3.0 | jaraco.functools | 4.0.1 |
| jaraco.text | 3.12.1 | jax-jumpy | 1.0.0 | jedi | 0.19.1 |
| Jinja2 | 3.1.4 | jiter | 0.8.0 | jmespath | 1.0.1 |
| joblib | 1.4.2 | joblibspark | 0.5.1 | json5 | 0.9.6 |
| jsonpatch | 1.33 | jsonpointer | 3.0.0 | jsonschema | 4.19.2 |
| jsonschema-specifications | 2023.7.1 | jupyter-events | 0.10.0 | jupyter-lsp | 2.2.0 |
| jupyter_client | 8.6.0 | jupyter_core | 5.7.2 | jupyter_server | 2.14.1 |
| Jupyter-serverterminals | 0.4.4 | jupyterlab | 4.0.11 | jupyterlab-pygments | 0.1.2 |
| jupyterlab_server | 2.25.1 | keras | 3.6.0 | kiwisolver | 1.4.4 |
| langchain | 0.3.8 | langchain-core | 0.2.41 | langchain-text-splitters | 0.2.4 |
| langcodes | 3.4.1 | langsmith | 0.1.129 | taalgegevens | 1.2.0 |
| launchpadlib | 1.11.0 | lazr.restfulclient | 0.14.6 | lazr.uri | 1.0.6 |
| lazy_loader | 0.4 | libclang | 15.0.6.1 | librosa | 0.10.2 |
| lightgbm | 4.5.0 | bliksemhulpprogramma's | 0.11.9 | linkify-it-py | 2.0.0 |
| llvmlite | 0.42.0 | lz4 | 4.3.2 | Mako | 1.2.0 |
| marisa-trie | 1.2.0 | Markdown | 3.4.1 | markdown-it-py | 2.2.0 |
| MarkupSafe | 2.1.3 | matplotlib | 3.8.4 | matplotlib-inline | 0.1.6 |
| Mccabe | 0.7.0 | mdit-py-plugins | 0.3.0 | mdurl | 0.1.0 |
| memray | 1.14.0 | verkeerd afstemmen | 2.0.4 | ml-dtypes | 0.4.1 |
| mlflow-skinny | 2.15.1 | more-itertools | 10.3.0 | mosaicml-cli | 0.6.41 |
| mosaicml-streaming | 0.8.0 | mpmath | 1.3.0 | msal | 1.31.1 |
| msal-extensions | 1.2.0 | msgpack | 1.1.0 | multidict | 6.0.4 |
| multimethode | 1.12 | meerdere processen | 0.70.16 | murmurhash | 1.0.11 |
| mypy | 1.10.0 | mypy-extensions | 1.0.0 | namex | 0.0.8 |
| nbclient | 0.8.0 | nbconvert | 7.10.0 | nbformat | 5.9.2 |
| nest-asyncio | 1.6.0 | networkx | 3.2.1 | ninja | 1.11.1.1 |
| nltk | 3.8.1 | nodeenv | 1.9.1 | schrift | 7.0.8 |
| notebook_shim | 0.2.3 | numba | 0.59.1 | numpy | 1.26.4 |
| nvidia-cublas-cu12 | 12.4.5.8 | nvidia-cuda-cupti-cu12 | 12.4.127 | nvidia-cuda-nvrtc-cu12 | 12.4.127 |
| nvidia-cuda-runtime-cu12 | 12.4.127 | nvidia-cudnn-cu12 | 9.1.0.70 | nvidia-cufft-cu12 | 11.2.1.3 |
| nvidia-curand-cu12 | 10.3.5.147 | nvidia-cusolver-cu12 | 11.6.1.9 | nvidia-cusparse-cu12 | 12.3.1.170 |
| nvidia-ml-py | 12.560.30 | nvidia-nccl-cu12 | 2.21.5 | nvidia-nvjitlink-cu12 | 12.4.127 |
| nvidia-nvtx-cu12 | 12.4.127 | oauthlib | 3.2.0 | Oci | 2.139.0 |
| openai | 1.55.1 | opencensus | 0.11.4 | opencensus-context | 0.1.3 |
| opentelemetry-api | 1.28.2 | opentelemetry-sdk | 1.28.2 | opentelemetry-semantic-conventions | 0,49b2 |
| opt_einsum | 3.4.0 | optree | 0.12.1 | optuna | 3.6.1 |
| optuna-integration | 3.6.0 | orjson | 3.10.12 | Overschrijft | 7.4.0 |
| verpakking | 24.1 | pandas | 1.5.3 | pandocfilters | 1.5.0 |
| paramiko | 3.4.0 | parso | 0.8.3 | pathspec | 0.10.3 |
| Patsy | 0.5.6 | pexpect | 4.8.0 | phik | 0.12.4 |
| kussen | 10.3.0 | pit | 24.2 | platformdirs | 3.10.0 |
| plotly | 5.22.0 | plugachtig | 1.0.0 | pmdarima | 2.0.4 |
| hondje | 1.8.2 | portalocker | 2.10.1 | vooraf geslagen | 3.0.9 |
| prometheus-client | 0.14.1 | prompt-toolkit | 3.0.43 | profeet | 1.1.5 |
| proto-plus | 1.24.0 | protobuf | 4.24.1 | psutil | 5.9.0 |
| psycopg2 | 2.9.3 | ptyprocess | 0.7.0 | pure-eval | 0.2.2 |
| py-cpuinfo | 9.0.0 | py-spy | 0.3.14 | pyarrow | 15.0.2 |
| pyarrow-hotfix | 0.6 | pyasn1 | 0.4.8 | pyasn1-modules | 0.2.8 |
| pybind11 | 2.13.6 | pyccolo | 0.0.65 | pycparser | 2.21 |
| pydantic | 2.8.2 | pydantic_core | 2.20.1 | pyflakes | 3.2.0 |
| Pygments | 2.15.1 | PyGObject | 3.48.2 | PyJWT | 2.7.0 |
| PyNaCl | 1.5.0 | pyodbc | 5.0.1 | pyOpenSSL | 24.0.0 |
| pyparsing | 3.0.9 | pyright | 1.1.294 | pytesseract | 0.3.10 |
| python-dateutil | 2.9.0.post0 | Python-bewerker | 1.0.4 | python-json-logger | 2.0.7 |
| python-lsp-jsonrpc | 1.1.2 | python-lsp-server | 1.10.0 | python-snappy | 0.6.1 |
| pytoolconfig | 1.2.6 | pytorch-ranger | 0.1.1 | pytz | 2024.1 |
| PyWavelets | 1.5.0 | PyYAML | 6.0.1 | pyzmq | 25.1.2 |
| vraagteken | 1.10.0 | straal | 2.35.0 | Verwijzen naar | 0.30.2 |
| regex | 2023.10.3 | Verzoeken | 2.32.2 | requests-oauthlib | 1.3.1 |
| rfc3339-validator | 0.1.4 | rfc3986-validator | 0.1.1 | rijk | 13.3.5 |
| touw | 1.12.0 | rpds-py | 0.10.6 | RSA | 4.9 |
| ruamel.yaml | 0.18.6 | ruamel.yaml.clib | 0.2.8 | s3transfer | 0.10.2 |
| safetensors | 0.4.4 | scikit-image | 0.23.2 | scikit-learn | 1.4.2 |
| scipy | 1.13.1 | seaborn | 0.13.2 | Send2Trash | 1.8.2 |
| sentence-transformers | 3.3.1 | zinsstuk | 0.2.0 | setuptools | 74.0.0 |
| Shap | 0.46.0 | shellingham | 1.5.4 | simplejson | 3.17.6 |
| Zes | 1.16.0 | Snijmachine | 0.0.8 | intelligent openen | 5.2.1 |
| smmap | 5.0.0 | sniffio | 1.3.0 | geluidsbestand | 0.12.1 |
| soepzeef | 2.5 | soxr | 0.5.0.post1 | spatie | 3.7.5 |
| spacy-legacy | 3.0.12 | spacy-loggers | 1.0.5 | SQLAlchemy | 2.0.30 |
| sqlparse | 0.4.2 | srsly | 2.4.8 | ssh-import-id | 5.11 |
| stapelgegevens | 0.2.0 | stanio | 0.5.1 | statsmodels | 0.14.2 |
| sympy | 1.13.1 | Tabulate | 0.9.0 | tangled-up-in-unicode | 0.2.0 |
| vasthoudendheid | 8.2.2 | tensorboard | 2.17.0 | tensorboard-data-server | 0.7.2 |
| tensorboard-plugin-profile | 2.18.0 | tensorboardX | 2.6.2.2 | tensorflow | 2.18.0 |
| tensorflow-estimator | 2.15.0 | termcolor | 2.4.0 | afgerond | 0.17.1 |
| Tekstuele | 0.87.1 | tf_keras | 2.17.0 | thinc | 8.2.5 |
| threadpoolctl | 2.2.0 | tifffile | 2023.4.12 | tiktoken | 0.7.0 |
| tinycss2 | 1.2.1 | tokenize-rt | 4.2.1 | tokenisatoren | 0.20.0 |
| tomli | 2.0.1 | fakkel | 2.5.0+cu124 | torch-optimizer | 0.3.0 |
| torcheval | 0.0.7 | torchmetrics | 1.4.0.post0 | torchvision | 0.20.0+cu124 |
| tornado | 6.4.1 | tqdm | 4.66.4 | traitlets | 5.14.3 |
| Transformatoren | 4.46.3 | Triton | 3.1.0 | typeguard | 4.3.0 |
| typist | 0.14.0 | types-protobuf | 3.20.3 | types-psutil | 5.9.0 |
| types-pytz | 2023.3.1.1 | types-PyYAML | 6.0.0 | typen verzoeken | 2.31.0.0 |
| types-setuptools | 68.0.0.0 | types-zes | 1.16.0 | types-urllib3 | 1.26.25.14 |
| typing_extensions | 4.11.0 | uc-micro-py | 1.0.1 | ujson | 5.10.0 |
| onbeheerde upgrades | 0.1 | URI-sjabloon | 1.3.0 | urllib3 | 1.26.16 |
| validerende entiteiten | 0.34.0 | virtualenv | 20.26.2 | Visies | 0.7.5 |
| wadllib | 1.3.6 | wasabi | 1.1.3 | wcwidth | 0.2.5 |
| wezel | 0.4.1 | webcolors | 24.8.0 | webcoderingen | 0.5.1 |
| websocket-client | 1.8.0 | websockets | 11.0.3 | gereedschap | 3.0.3 |
| whatthepatch | 1.0.2 | wiel | 0.43.0 | woordwolk | 1.9.3 |
| ingepakt | 1.14.1 | xgboost | 2.0.3 | xgboost-ray | 0.1.19 |
| xxhash | 3.4.1 | yapf | 0.33.0 | yarl | 1.9.3 |
| ydata-profiling | 4.9.0 | zipp | 3.17.0 | zstd | 1.5.5.1 |
R-bibliotheken
De R-bibliotheken zijn identiek aan de R-bibliotheken in Databricks Runtime 16.1.
Java- en Scala-bibliotheken (Scala 2.12-cluster)
Naast Java- en Scala-bibliotheken in Databricks Runtime 16.1 bevat Databricks Runtime 16.1 ML de volgende JAR's:
CPU-clusters
| Groeps-id | Artefact-id | Versie |
|---|---|---|
| com.typesafe.akka | akka-actor_2.12 | 2.5.23 |
| ml.dmlc | xgboost4j-spark_2.12 | 1.7.3 |
| ml.dmlc | xgboost4j_2.12 | 1.7.3 |
| org.graphframes | graphframes_2.12 | 0.8.4-db1-spark3.5 |
| org.mlflow | mlflow-client | 2.15.1 |
| org.tensorflow | spark-tensorflow-connector_2.12 | 1.15.0 |
GPU-clusters
| Groeps-id | Artefact-id | Versie |
|---|---|---|
| com.typesafe.akka | akka-actor_2.12 | 2.5.23 |
| ml.dmlc | xgboost4j-gpu_2.12 | 1.7.3 |
| ml.dmlc | xgboost4j-spark-gpu_2.12 | 1.7.3 |
| org.graphframes | graphframes_2.12 | 0.8.4-db1-spark3.5 |
| org.mlflow | mlflow-client | 2.15.1 |
| org.tensorflow | spark-tensorflow-connector_2.12 | 1.15.0 |