Jaa

Databricks Runtime 10.5 (EoS)

  • Artikkeli

Note

Support for this Databricks Runtime version has ended. For the end-of-support date, see End-of-support history. For all supported Databricks Runtime versions, see Databricks Runtime release notes versions and compatibility.

The following release notes provide information about Databricks Runtime 10.5, powered by Apache Spark 3.2.1. Databricks released this version in May 2022.

New features and improvements

SQL: New functions to_number and try_to_number

The to_number function and try_to_number function allow you to convert character representations of numbers in various formats to DECIMAL.

SQL: New function try_sum

Unlike sum, the try_sum aggregate function returns NULL instead of an error when an overflow occurs.

SQL: Update to ceil, ceiling, and floor functions

The ceil function, ceiling function, and floor function now support a second optional argument to specify a target scale.

SQL: New aggregate functions regr_avgx and regr_avgy

The new regr_avgx aggregate function and regr_avgy aggregate function support linear regressions.

SQL: Update to contains, startswith, and endswith functions

The contains function, startswith function, and endswith function now support binary data types as input in addition to strings.

SQL: New function array_size

The new array_size function calculates the size of an array.

Unity Catalog: SQL LIST output removes is_directory column and adds trailing / for directory paths (Public Preview)

The output of the SQL LIST statement for listing paths governed by Unity Catalog has changed. The is_directory column is no longer displayed. Instead, paths that represent directories now end with a trailing slash (/) for the path and name columns. This change is for consistency with the output of the dbutils.fs.ls command.

Auto Loader: new SQL function CLOUD_FILES_STATE

You can use the new CLOUD_FILES_STATE function to query the internal state of an Auto Loader stream; see Querying files discovered by Auto Loader.

Auto Loader: file notification support is now GA

File notification support for What is Auto Loader? on Google Cloud is now generally available.

Delta Lake: new maxRecordsPerFile option for maximum records written to a single file

When you use the DataFrame APIs to write to a Delta table, you can use the maxRecordsPerFile option to specify the maximum number of records to write to a single file. Specifying a value of zero or a negative value represents no limit. See Configure Delta Lake to control data file size.

Deprecation of Koalas

Koalas is now deprecated. Koalas provides pandas APIs on top of Apache Spark. Koalas was contributed to by the community and has been available as pandas API on Spark beginning in Apache Spark 3.2. Koalas continues to be preinstalled for Databricks Runtime 10.5, but Koalas will no longer be preinstalled beginning with the next Databricks Runtime version.

To learn more about pandas API on Spark, see:

New _metadata column for all file formats

You can get metadata information for input files with the _metadata column. The _metadata column is hidden. This column is available for all input file formats. To include the _metadata column in a returned DataFrame, you must explicitly reference this column in your query. See File metadata column.

Test your Delta Live Tables pipeline with Databricks Runtime 10.5

Databricks Runtime 10.5 is now the supported preview version for Delta Live Tables. To test your pipeline with Databricks Runtime 10.5, set the channel setting to preview in your pipeline settings.

Bug fixes

  • Fixed the display of numUpdatedRows and numCopiedRows metric results for update operations on Delta tables that have change data capture (CDC) enabled.

Library upgrades

  • Upgraded Python libraries:
    • plotly from 5.5.0 to 5.6.0
  • Upgraded R libraries:
    • base from 4.1.2 to 4.1.3
    • caret from 6.0-90 to 6.0-91
    • cluster from 2.1.2 to 2.1.3
    • commonmark from 1.7 to 1.8.0
    • compiler from 4.1.2 to 4.1.3
    • crayon from 1.5.0 to 1.5.1
    • datasets from 4.1.2 to 4.1.3
    • desc from 1.4.0 to 1.4.1
    • fansi from 1.0.2 to 1.0.3
    • glue from 1.6.1 to 1.6.2
    • graphics from 4.1.2 to 4.1.3
    • grDevices from 4.1.2 to 4.1.3
    • grid from 4.1.2 to 4.1.3
    • knitr from 1.37 to 1.38
    • MASS from 7.3-55 to 7.3-56
    • Matrix from 1.4-0 to 1.4-1
    • methods from 4.1.2 to 4.1.3
    • mgcv from 1.8-39 to 1.8-40
    • nlme from 3.1-155 to 3.1-157
    • openssl from 1.4.6 to 2.0.0
    • parallel from 4.1.2 to 4.1.3
    • plyr from 1.8.6 to 1.8.7
    • processx from 3.5.2 to 3.5.3
    • r2d3 from 0.2.5 to 0.2.6
    • Rcpp from 1.0.8 to 1.0.8.3
    • rlang from 1.0.1 to 1.0.2
    • rmarkdown from 2.11 to 2.13
    • RSQLite from 2.2.10 to 2.2.11
    • sass from 0.4.0 to 0.4.1
    • splines from 4.1.2 to 4.1.3
    • stats from 4.1.2 to 4.1.3
    • stats4 from 4.1.2 to 4.1.3
    • tcltk from 4.1.2 to 4.1.3
    • tools from 4.1.2 to 4.1.3
    • utils from 4.1.2 to 4.1.3
    • uuid from 1.0-3 to 1.0-4
    • waldo from 0.3.1 to 0.4.0
    • withr from 2.4.3 to 2.5.0
    • xfun from 0.29 to 0.30
  • Upgraded Java libraries:
    • com.h2database.h2 from 1.4.195 to 2.1.210
    • org.apache.hadoop.hadoop-client-api from 3.3.1-databricks to 3.3.2-databricks
    • org.apache.hadoop.hadoop-client-runtime from 3.3.1 to 3.3.2
    • org.rocksdb.rocksdbjni from 6.24.2 to 6.20.3

Apache Spark

Databricks Runtime 10.5 includes Apache Spark 3.2.1. This release includes all Spark fixes and improvements included in Databricks Runtime 10.4 LTS, as well as the following additional bug fixes and improvements made to Spark:

  • [SPARK-38354] [SC-97886] Revert “[SQL] Add hash probes metric for shuffled hash join”
  • [SPARK-38484] [SAS-103][SAS-111][SAS-113][SC-98102][Cherry-Pick] Instrument PySpark SQL module with general usage logger
  • [SPARK-38333] [SQL] PlanExpression expression should skip addExprTree function in Executor
  • [SPARK-38646] [PYTHON][10.x] Pull a trait out for Python functions
  • [SPARK-38674] [PHOTON] Add a metric to PhotonSubqueryBroadcastExec
  • [SPARK-38787] [SS] Replace found value with non-null element in the remaining list for key and remove remaining null elements from values in keyWithIndexToValue store for stream-stream joins
  • [SPARK-38446] [Core] Fix deadlock between ExecutorClassLoader and FileDownloadCallback caused by Log4j
  • [SPARK-38705] [SQL] Use function identifier in create and drop function command
  • [SPARK-38684] [SS] Fix correctness issue on stream-stream outer join with RocksDB state store provider
  • [SPARK-38655] [SQL] OffsetWindowFunctionFrameBase cannot find the offset row whose input is not-null
  • [SPARK-38204] [SS] Use StatefulOpClusteredDistribution for stateful operators with respecting backward compatibility
  • [SPARK-38616] [SQL] Keep track of SQL query text in Catalyst TreeNode
  • [SPARK-38600] [SQL] Include unit into the sql string of TIMESTAMPADD/DIFF
  • [SPARK-34805] [SQL] Propagate metadata from nested columns in Alias
  • [SPARK-38535] [SQL] Add the datetimeUnit enum and use it in TIMESTAMPADD/DIFF
  • [SPARK-38415] [SQL] Update the histogram_numeric (x, y) result type to make x == the input type
  • [SPARK-38148] [SQL] Do not add dynamic partition pruning if there exists static partition pruning
  • [SPARK-37527] [SQL] Compile COVAR_POP, COVAR_SAMP and CORR in H2Dialet
  • [SPARK-38510] [SQL] Retry ClassSymbol.selfType to work around cyclic references
  • [SPARK-38509] [SQL] Unregister the TIMESTAMPADD/DIFF functions and remove DATE_ADD/DIFF
  • [SPARK-38504] [SQL] Cannot read TimestampNTZ as TimestampLTZ
  • [SPARK-38526] [SQL] Fix misleading function alias name for RuntimeReplaceable
  • [SPARK-38335] [SQL] Implement parser support for DEFAULT column values
  • [SPARK-38593] [SS] Carry over the metric of the number of dropped late events into SessionWindowStateStoreSaveExec
  • [SPARK-38549] [SS] Add numRowsDroppedByWatermark to SessionWindowStateStoreRestoreExec
  • [SPARK-38583] [SQL] Restore the behavior of to_timestamp that allows numeric types
  • [SPARK-38481] [SQL] Substitute Java overflow exception from TIMESTAMPADD by Spark exception
  • [SPARK-37753] [SQL] Fine tune logic to demote Broadcast hash join in DynamicJoinSelection
  • [SPARK-37582] [SPARK-37583][SQL] CONTAINS, STARTSWITH, ENDSWITH should support both string and binary types
  • [SPARK-38345] [SQL] Introduce SQL function ARRAY_SIZE
  • [SPARK-38385] [SQL] Improve error messages of empty statement and <EOF> in ParseException
  • [SPARK-37614] [SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
  • [SPARK-38560] [SQL] If Sum, Count, Any accompany with distinct, cannot do partial agg push down.
  • [SPARK-38360] [SQL][SS][PYTHON] Introduce a exists function for TreeNode to eliminate duplicate code patterns
  • [SPARK-38385] [SQL] Improve error messages of ‘mismatched input’ cases from ANTLR
  • [SPARK-38240] [SQL] Improve RuntimeReplaceable and add a guideline for adding new functions
  • [SPARK-38542] [SQL]UnsafeHashedRelation should serialize numKeys out
  • [SPARK-38521] [SQL] Change partitionOverwriteMode from string to variable in Scala
  • [SPARK-38558] [SQL] Remove unnecessary casts between IntegerType and IntDecimal
  • [SPARK-38489] [SQL] Aggregate.groupOnly support foldable expressions
  • [SPARK-38410] [SQL] Support specify initial partition number for rebalance
  • [SPARK-38107] [SQL] Use error classes in the compilation errors of python/pandas UDFs
  • [SPARK-38354] [SQL] Add hash probes metric for shuffled hash join
  • [SPARK-38112] [SQL] Use error classes in the execution errors of date/timestamp handling
  • [SPARK-38361] [SQL] Add factory method getConnection into JDBCDialect.
  • [SPARK-37865] [SQL] Fix union deduplication correctness bug
  • [SPARK-38298] [SQL][Tests] Fix DataExpressionSuite, NullExpressionsSuite, StringExpressionsSuite, complexTypesSuite, CastSuite under ANSI mode
  • [SPARK-38407] [SQL] ANSI Cast: loosen the limitation of casting non-null complex types
  • [SPARK-38352] [SQL] Fix DataFrameAggregateSuite/DataFrameSetOperationsSuite/DataFrameWindowFunctionsSuite under ANSI mode
  • [SPARK-38054] [SQL] Supports list namespaces in JDBC v2 MySQL dialect
  • [SPARK-37960] [SQL] A new framework to represent catalyst expressions in DS v2 APIs
  • [SPARK-38103] [SQL] Migrate parsing errors of transform into the new error framework
  • [SPARK-38519] [SQL] AQE throw exception should respect SparkFatalException
  • [SPARK-38528] [SQL] Eagerly iterate over aggregate sequence when building project list in ExtractGenerator
  • [SPARK-38320] [SS] Fix flatMapGroupsWithState timeout in batch with data for key
  • [SPARK-38442] [SQL] Fix ConstantFoldingSuite/ColumnExpressionSuite/DataFrameSuite/AdaptiveQueryExecSuite under ANSI mode
  • [SPARK-38268] [SQL] Hide the “failOnError” field in the toString method of Abs/CheckOverflow
  • [SPARK-37949] [SQL] Improve Rebalance statistics estimation
  • [SPARK-38450] [SQL] Fix HiveQuerySuite//PushFoldableIntoBranchesSuite/TransposeWindowSuite under ANSI mode
  • [SPARK-38501] [SQL] Fix thriftserver test failures under ANSI mode
  • [SPARK-37895] [SQL] Filter push down column with quoted columns
  • [SPARK-38449] [SQL] Avoid call createTable when ignoreIfExists=true and table exists
  • [SPARK-38406] [SQL] Improve performance of ShufflePartitionsUtil createSkewPartitionSpecs
  • [SPARK-37947] [SQL] Extract generator from GeneratorOuter expression contained by a Generate operator.
  • [SPARK-38309] [CORE] Fix SHS shuffleTotalReads and shuffleTotalBlocks percentile metrics
  • [SPARK-38412] [SS] Fix the swapped sequence of from and to in StateSchemaCompatibilityChecker
  • [SPARK-38267] [CORE][SQL][SS] Replace pattern matches on boolean expressions with conditional statements
  • [SPARK-38104] [SQL] Migrate parsing errors of window into the new error framework
  • [SPARK-38434] [SQL] Correct semantic of CheckAnalysis.getDataTypesAreCompatibleFn method
  • [SPARK-38033] [SS] The SS processing cannot be started because the com…
  • [SPARK-38378] [SQL] Refactoring of the ANTLR grammar definition into separate Parser and Lexer files
  • [SPARK-37475] [SQL] Add scale parameter to floor and ceil functions
  • [SPARK-38411] [CORE] Use UTF-8 when doMergeApplicationListingInternal reads event logs
  • [SPARK-36553] [ML] KMeans avoid compute auxiliary statistics for large K
  • [SPARK-38393] [SQL] Clean up deprecated usage of GenSeq/GenMap
  • [SPARK-38389] [SQL] Add the DATEDIFF() and DATE_DIFF() aliases for TIMESTAMPDIFF()
  • [SPARK-33206] [CORE][3.2] Fix shuffle index cache weight calculation for small index files
  • [SPARK-38353] [PYTHON] Instrument enter and exit magic methods for Pandas API on Spark
  • [SPARK-38323] [SQL][Streaming] Support the hidden file metadata in Streaming
  • [SPARK-38138] [SQL] Materialize QueryPlan subqueries
  • [SPARK-38094] Enable matching schema column names by field ids
  • [SPARK-37923] [SQL] Generate partition transforms for BucketSpec inside parser
  • [SPARK-38363] [SQL] Avoid runtime error in Dataset.summary()/Dataset.describe() when ANSI mode is on
  • [SPARK-38284] [SQL] Add the TIMESTAMPDIFF() function
  • [SPARK-38251] [SQL] Change Cast.toString as “cast” instead of “ansi_cast” under ANSI mode
  • [SPARK-38314] [SQL] Fix of failing to read parquet files after writing the hidden file metadata in

See Databricks Runtime 10.5 maintenance updates.

System environment

  • Operating System: Ubuntu 20.04.4 LTS
  • Java: Zulu 8.56.0.21-CA-linux64
  • Scala: 2.12.14
  • Python: 3.8.10
  • R: 4.1.3
  • Delta Lake: 1.2.1

Installed Python libraries

Library Version Library Version Library Version
Antergos Linux 2015.10 (ISO-Rolling) appdirs 1.4.4 argon2-cffi 20.1.0
async-generator 1.10 attrs 20.3.0 backcall 0.2.0
bidict 0.21.4 bleach 3.3.0 boto3 1.16.7
botocore 1.19.7 certifi 2020.12.5 cffi 1.14.5
chardet 4.0.0 cycler 0.10.0 Cython 0.29.23
dbus-python 1.2.16 decorator 5.0.6 defusedxml 0.7.1
distlib 0.3.4 distro-info 0.23ubuntu1 entrypoints 0.3
facets-overview 1.0.0 filelock 3.6.0 idna 2.10
ipykernel 5.3.4 ipython 7.22.0 ipython-genutils 0.2.0
ipywidgets 7.6.3 jedi 0.17.2 Jinja2 2.11.3
jmespath 0.10.0 joblib 1.0.1 jsonschema 3.2.0
jupyter-client 6.1.12 jupyter-core 4.7.1 jupyterlab-pygments 0.1.2
jupyterlab-widgets 1.0.0 kiwisolver 1.3.1 koalas 1.8.2
MarkupSafe 2.0.1 matplotlib 3.4.2 mistune 0.8.4
nbclient 0.5.3 nbconvert 6.0.7 nbformat 5.1.3
nest-asyncio 1.5.1 notebook 6.3.0 numpy 1.20.1
packaging 20.9 pandas 1.2.4 pandocfilters 1.4.3
parso 0.7.0 patsy 0.5.1 pexpect 4.8.0
pickleshare 0.7.5 Pillow 8.2.0 pip 21.0.1
plotly 5.6.0 prometheus-client 0.10.1 prompt-toolkit 3.0.17
protobuf 3.17.2 psycopg2 2.8.5 ptyprocess 0.7.0
pyarrow 4.0.0 pycparser 2.20 Pygments 2.8.1
PyGObject 3.36.0 pyparsing 2.4.7 pyrsistent 0.17.3
python-apt 2.0.0+ubuntu0.20.4.7 python-dateutil 2.8.1 python-engineio 4.3.0
python-socketio 5.4.1 pytz 2020.5 pyzmq 20.0.0
requests 2.25.1 requests-unixsocket 0.2.0 s3transfer 0.3.7
scikit-learn 0.24.1 scipy 1.6.2 seaborn 0.11.1
Send2Trash 1.5.0 setuptools 52.0.0 six 1.15.0
ssh-import-id 5.10 statsmodels 0.12.2 tenacity 8.0.1
terminado 0.9.4 testpath 0.4.4 threadpoolctl 2.1.0
tornado 6.1 traitlets 5.0.5 unattended-upgrades 0.1
urllib3 1.25.11 virtualenv 20.4.1 wcwidth 0.2.5
webencodings 0.5.1 wheel 0.36.2 widgetsnbextension 3.5.1

Installed R libraries

R libraries are installed from the Microsoft CRAN snapshot on 2022-03-28.

Library Version Library Version Library Version
askpass 1.1 assertthat 0.2.1 backports 1.4.1
base 4.1.3 base64enc 0.1-3 bit 4.0.4
bit64 4.0.5 blob 1.2.2 boot 1.3-28
brew 1.0-7 brio 1.1.3 broom 0.7.12
bslib 0.3.1 cachem 1.0.6 callr 3.7.0
caret 6.0-91 cellranger 1.1.0 chron 2.3-56
class 7.3-20 cli 3.2.0 clipr 0.8.0
cluster 2.1.3 codetools 0.2-18 colorspace 2.0-3
commonmark 1.8.0 compiler 4.1.3 config 0.3.1
cpp11 0.4.2 crayon 1.5.1 credentials 1.3.2
curl 4.3.2 data.table 1.14.2 datasets 4.1.3
DBI 1.1.2 dbplyr 2.1.1 desc 1.4.1
devtools 2.4.3 diffobj 0.3.5 digest 0.6.29
dplyr 1.0.8 dtplyr 1.2.1 e1071 1.7-9
ellipsis 0.3.2 evaluate 0.15 fansi 1.0.3
farver 2.1.0 fastmap 1.1.0 fontawesome 0.2.2
forcats 0.5.1 foreach 1.5.2 foreign 0.8-82
forge 0.2.0 fs 1.5.2 future 1.24.0
future.apply 1.8.1 gargle 1.2.0 generics 0.1.2
gert 1.5.0 ggplot2 3.3.5 gh 1.3.0
gitcreds 0.1.1 glmnet 4.1-3 globals 0.14.0
glue 1.6.2 googledrive 2.0.0 googlesheets4 1.0.0
gower 1.0.0 graphics 4.1.3 grDevices 4.1.3
grid 4.1.3 gridExtra 2.3 gsubfn 0.7
gtable 0.3.0 hardhat 0.2.0 haven 2.4.3
highr 0.9 hms 1.1.1 htmltools 0.5.2
htmlwidgets 1.5.4 httpuv 1.6.5 httr 1.4.2
hwriter 1.3.2 hwriterPlus 1.0-3 ids 1.0.1
ini 0.3.1 ipred 0.9-12 isoband 0.2.5
iterators 1.0.14 jquerylib 0.1.4 jsonlite 1.8.0
KernSmooth 2.23-20 knitr 1.38 labeling 0.4.2
later 1.3.0 lattice 0.20-45 lava 1.6.10
lifecycle 1.0.1 listenv 0.8.0 lubridate 1.8.0
magrittr 2.0.2 markdown 1.1 MASS 7.3-56
Matrix 1.4-1 memoise 2.0.1 methods 4.1.3
mgcv 1.8-40 mime 0.12 ModelMetrics 1.2.2.2
modelr 0.1.8 munsell 0.5.0 nlme 3.1-157
nnet 7.3-17 numDeriv 2016.8-1.1 openssl 2.0.0
parallel 4.1.3 parallelly 1.30.0 pillar 1.7.0
pkgbuild 1.3.1 pkgconfig 2.0.3 pkgload 1.2.4
plogr 0.2.0 plyr 1.8.7 praise 1.0.0
prettyunits 1.1.1 pROC 1.18.0 processx 3.5.3
prodlim 2019.11.13 progress 1.2.2 progressr 0.10.0
promises 1.2.0.1 proto 1.0.0 proxy 0.4-26
ps 1.6.0 purrr 0.3.4 r2d3 0.2.6
R6 2.5.1 randomForest 4.7-1 rappdirs 0.3.3
rcmdcheck 1.4.0 RColorBrewer 1.1-2 Rcpp 1.0.8.3
RcppEigen 0.3.3.9.1 readr 2.1.2 readxl 1.3.1
recipes 0.2.0 rematch 1.0.1 rematch2 2.1.2
remotes 2.4.2 reprex 2.0.1 reshape2 1.4.4
rlang 1.0.2 rmarkdown 2.13 RODBC 1.3-19
roxygen2 7.1.2 rpart 4.1.16 rprojroot 2.0.2
Rserve 1.8-10 RSQLite 2.2.11 rstudioapi 0.13
rversions 2.1.1 rvest 1.0.2 sass 0.4.1
scales 1.1.1 selectr 0.4-2 sessioninfo 1.2.2
shape 1.4.6 shiny 1.7.1 sourcetools 0.1.7
sparklyr 1.7.5 SparkR 3.2.0 spatial 7.3-11
splines 4.1.3 sqldf 0.4-11 SQUAREM 2021.1
stats 4.1.3 stats4 4.1.3 stringi 1.7.6
stringr 1.4.0 survival 3.2-13 sys 3.4
tcltk 4.1.3 TeachingDemos 2.10 testthat 3.1.2
tibble 3.1.6 tidyr 1.2.0 tidyselect 1.1.2
tidyverse 1.3.1 timeDate 3043.102 tinytex 0.37
tools 4.1.3 tzdb 0.2.0 usethis 2.1.5
utf8 1.2.2 utils 4.1.3 uuid 1.0-4
vctrs 0.3.8 viridisLite 0.4.0 vroom 1.5.7
waldo 0.4.0 whisker 0.4 withr 2.5.0
xfun 0.30 xml2 1.3.3 xopen 1.0.0
xtable 1.8-4 yaml 2.3.5 zip 2.2.0

Installed Java and Scala libraries (Scala 2.12 cluster version)

Group ID Artifact ID Version
antlr antlr 2.7.7
com.amazonaws amazon-kinesis-client 1.12.0
com.amazonaws aws-java-sdk-autoscaling 1.12.189
com.amazonaws aws-java-sdk-cloudformation 1.12.189
com.amazonaws aws-java-sdk-cloudfront 1.12.189
com.amazonaws aws-java-sdk-cloudhsm 1.12.189
com.amazonaws aws-java-sdk-cloudsearch 1.12.189
com.amazonaws aws-java-sdk-cloudtrail 1.12.189
com.amazonaws aws-java-sdk-cloudwatch 1.12.189
com.amazonaws aws-java-sdk-cloudwatchmetrics 1.12.189
com.amazonaws aws-java-sdk-codedeploy 1.12.189
com.amazonaws aws-java-sdk-cognitoidentity 1.12.189
com.amazonaws aws-java-sdk-cognitosync 1.12.189
com.amazonaws aws-java-sdk-config 1.12.189
com.amazonaws aws-java-sdk-core 1.12.189
com.amazonaws aws-java-sdk-datapipeline 1.12.189
com.amazonaws aws-java-sdk-directconnect 1.12.189
com.amazonaws aws-java-sdk-directory 1.12.189
com.amazonaws aws-java-sdk-dynamodb 1.12.189
com.amazonaws aws-java-sdk-ec2 1.12.189
com.amazonaws aws-java-sdk-ecs 1.12.189
com.amazonaws aws-java-sdk-efs 1.12.189
com.amazonaws aws-java-sdk-elasticache 1.12.189
com.amazonaws aws-java-sdk-elasticbeanstalk 1.12.189
com.amazonaws aws-java-sdk-elasticloadbalancing 1.12.189
com.amazonaws aws-java-sdk-elastictranscoder 1.12.189
com.amazonaws aws-java-sdk-emr 1.12.189
com.amazonaws aws-java-sdk-glacier 1.12.189
com.amazonaws aws-java-sdk-glue 1.12.189
com.amazonaws aws-java-sdk-iam 1.12.189
com.amazonaws aws-java-sdk-importexport 1.12.189
com.amazonaws aws-java-sdk-kinesis 1.12.189
com.amazonaws aws-java-sdk-kms 1.12.189
com.amazonaws aws-java-sdk-lambda 1.12.189
com.amazonaws aws-java-sdk-logs 1.12.189
com.amazonaws aws-java-sdk-machinelearning 1.12.189
com.amazonaws aws-java-sdk-opsworks 1.12.189
com.amazonaws aws-java-sdk-rds 1.12.189
com.amazonaws aws-java-sdk-redshift 1.12.189
com.amazonaws aws-java-sdk-route53 1.12.189
com.amazonaws aws-java-sdk-s3 1.12.189
com.amazonaws aws-java-sdk-ses 1.12.189
com.amazonaws aws-java-sdk-simpledb 1.12.189
com.amazonaws aws-java-sdk-simpleworkflow 1.12.189
com.amazonaws aws-java-sdk-sns 1.12.189
com.amazonaws aws-java-sdk-sqs 1.12.189
com.amazonaws aws-java-sdk-ssm 1.12.189
com.amazonaws aws-java-sdk-storagegateway 1.12.189
com.amazonaws aws-java-sdk-sts 1.12.189
com.amazonaws aws-java-sdk-support 1.12.189
com.amazonaws aws-java-sdk-swf-libraries 1.11.22
com.amazonaws aws-java-sdk-workspaces 1.12.189
com.amazonaws jmespath-java 1.12.189
com.chuusai shapeless_2.12 2.3.3
com.clearspring.analytics stream 2.9.6
com.databricks Rserve 1.8-3
com.databricks jets3t 0.7.1-0
com.databricks.scalapb compilerplugin_2.12 0.4.15-10
com.databricks.scalapb scalapb-runtime_2.12 0.4.15-10
com.esotericsoftware kryo-shaded 4.0.2
com.esotericsoftware minlog 1.3.0
com.fasterxml classmate 1.3.4
com.fasterxml.jackson.core jackson-annotations 2.12.3
com.fasterxml.jackson.core jackson-core 2.12.3
com.fasterxml.jackson.core jackson-databind 2.12.3
com.fasterxml.jackson.dataformat jackson-dataformat-cbor 2.12.3
com.fasterxml.jackson.datatype jackson-datatype-joda 2.12.3
com.fasterxml.jackson.module jackson-module-paranamer 2.12.3
com.fasterxml.jackson.module jackson-module-scala_2.12 2.12.3
com.github.ben-manes.caffeine caffeine 2.3.4
com.github.fommil jniloader 1.1
com.github.fommil.netlib core 1.1.2
com.github.fommil.netlib native_ref-java 1.1
com.github.fommil.netlib native_ref-java-natives 1.1
com.github.fommil.netlib native_system-java 1.1
com.github.fommil.netlib native_system-java-natives 1.1
com.github.fommil.netlib netlib-native_ref-linux-x86_64-natives 1.1
com.github.fommil.netlib netlib-native_system-linux-x86_64-natives 1.1
com.github.luben zstd-jni 1.5.0-4
com.github.wendykierp JTransforms 3.1
com.google.code.findbugs jsr305 3.0.0
com.google.code.gson gson 2.8.6
com.google.crypto.tink tink 1.6.0
com.google.flatbuffers flatbuffers-java 1.9.0
com.google.guava guava 15.0
com.google.protobuf protobuf-java 2.6.1
com.h2database h2 2.1.210
com.helger profiler 1.1.1
com.jcraft jsch 0.1.50
com.jolbox bonecp 0.8.0.RELEASE
com.lihaoyi sourcecode_2.12 0.1.9
com.microsoft.azure azure-data-lake-store-sdk 2.3.9
com.ning compress-lzf 1.0.3
com.sun.istack istack-commons-runtime 3.0.8
com.sun.mail javax.mail 1.5.2
com.tdunning json 1.8
com.thoughtworks.paranamer paranamer 2.8
com.trueaccord.lenses lenses_2.12 0.4.12
com.twitter chill-java 0.10.0
com.twitter chill_2.12 0.10.0
com.twitter util-app_2.12 7.1.0
com.twitter util-core_2.12 7.1.0
com.twitter util-function_2.12 7.1.0
com.twitter util-jvm_2.12 7.1.0
com.twitter util-lint_2.12 7.1.0
com.twitter util-registry_2.12 7.1.0
com.twitter util-stats_2.12 7.1.0
com.typesafe config 1.2.1
com.typesafe.scala-logging scala-logging_2.12 3.7.2
com.univocity univocity-parsers 2.9.1
com.zaxxer HikariCP 4.0.3
commons-cli commons-cli 1.2
commons-codec commons-codec 1.15
commons-collections commons-collections 3.2.2
commons-dbcp commons-dbcp 1.4
commons-fileupload commons-fileupload 1.3.3
commons-httpclient commons-httpclient 3.1
commons-io commons-io 2.8.0
commons-lang commons-lang 2.6
commons-logging commons-logging 1.1.3
commons-net commons-net 3.1
commons-pool commons-pool 1.5.4
dev.ludovic.netlib arpack 2.2.1
dev.ludovic.netlib blas 2.2.1
dev.ludovic.netlib lapack 2.2.1
hive-2.3__hadoop-3.2 jets3t-0.7 liball_deps_2.12
info.ganglia.gmetric4j gmetric4j 1.0.10
io.airlift aircompressor 0.21
io.delta delta-sharing-spark_2.12 0.4.0
io.dropwizard.metrics metrics-core 4.1.1
io.dropwizard.metrics metrics-graphite 4.1.1
io.dropwizard.metrics metrics-healthchecks 4.1.1
io.dropwizard.metrics metrics-jetty9 4.1.1
io.dropwizard.metrics metrics-jmx 4.1.1
io.dropwizard.metrics metrics-json 4.1.1
io.dropwizard.metrics metrics-jvm 4.1.1
io.dropwizard.metrics metrics-servlets 4.1.1
io.netty netty-all 4.1.68.Final
io.prometheus simpleclient 0.7.0
io.prometheus simpleclient_common 0.7.0
io.prometheus simpleclient_dropwizard 0.7.0
io.prometheus simpleclient_pushgateway 0.7.0
io.prometheus simpleclient_servlet 0.7.0
io.prometheus.jmx collector 0.12.0
jakarta.annotation jakarta.annotation-api 1.3.5
jakarta.servlet jakarta.servlet-api 4.0.3
jakarta.validation jakarta.validation-api 2.0.2
jakarta.ws.rs jakarta.ws.rs-api 2.1.6
javax.activation activation 1.1.1
javax.annotation javax.annotation-api 1.3.2
javax.el javax.el-api 2.2.4
javax.jdo jdo-api 3.0.1
javax.transaction jta 1.1
javax.transaction transaction-api 1.1
javax.xml.bind jaxb-api 2.2.2
javax.xml.stream stax-api 1.0-2
javolution javolution 5.5.1
jline jline 2.14.6
joda-time joda-time 2.10.10
log4j apache-log4j-extras 1.2.17
log4j log4j 1.2.17
maven-trees hive-2.3__hadoop-3.2 liball_deps_2.12
net.java.dev.jna jna 5.8.0
net.razorvine pyrolite 4.30
net.sf.jpam jpam 1.1
net.sf.opencsv opencsv 2.3
net.sf.supercsv super-csv 2.2.0
net.snowflake snowflake-ingest-sdk 0.9.6
net.snowflake snowflake-jdbc 3.13.3
net.snowflake spark-snowflake_2.12 2.9.0-spark_3.1
net.sourceforge.f2j arpack_combined_all 0.1
org.acplt.remotetea remotetea-oncrpc 1.1.2
org.antlr ST4 4.0.4
org.antlr antlr-runtime 3.5.2
org.antlr antlr4-runtime 4.8
org.antlr stringtemplate 3.2.1
org.apache.ant ant 1.9.2
org.apache.ant ant-jsch 1.9.2
org.apache.ant ant-launcher 1.9.2
org.apache.arrow arrow-format 2.0.0
org.apache.arrow arrow-memory-core 2.0.0
org.apache.arrow arrow-memory-netty 2.0.0
org.apache.arrow arrow-vector 2.0.0
org.apache.avro avro 1.10.2
org.apache.avro avro-ipc 1.10.2
org.apache.avro avro-mapred 1.10.2
org.apache.commons commons-compress 1.21
org.apache.commons commons-crypto 1.1.0
org.apache.commons commons-lang3 3.12.0
org.apache.commons commons-math3 3.4.1
org.apache.commons commons-text 1.6
org.apache.curator curator-client 2.13.0
org.apache.curator curator-framework 2.13.0
org.apache.curator curator-recipes 2.13.0
org.apache.derby derby 10.14.2.0
org.apache.hadoop hadoop-client-api 3.3.2-databricks
org.apache.hadoop hadoop-client-runtime 3.3.2
org.apache.hive hive-beeline 2.3.9
org.apache.hive hive-cli 2.3.9
org.apache.hive hive-jdbc 2.3.9
org.apache.hive hive-llap-client 2.3.9
org.apache.hive hive-llap-common 2.3.9
org.apache.hive hive-serde 2.3.9
org.apache.hive hive-shims 2.3.9
org.apache.hive hive-storage-api 2.7.2
org.apache.hive.shims hive-shims-0.23 2.3.9
org.apache.hive.shims hive-shims-common 2.3.9
org.apache.hive.shims hive-shims-scheduler 2.3.9
org.apache.httpcomponents httpclient 4.5.13
org.apache.httpcomponents httpcore 4.4.12
org.apache.ivy ivy 2.5.0
org.apache.mesos mesos-shaded-protobuf 1.4.0
org.apache.orc orc-core 1.6.13
org.apache.orc orc-mapreduce 1.6.13
org.apache.orc orc-shims 1.6.13
org.apache.parquet parquet-column 1.12.0-databricks-0004
org.apache.parquet parquet-common 1.12.0-databricks-0004
org.apache.parquet parquet-encoding 1.12.0-databricks-0004
org.apache.parquet parquet-format-structures 1.12.0-databricks-0004
org.apache.parquet parquet-hadoop 1.12.0-databricks-0004
org.apache.parquet parquet-jackson 1.12.0-databricks-0004
org.apache.thrift libfb303 0.9.3
org.apache.thrift libthrift 0.12.0
org.apache.xbean xbean-asm9-shaded 4.20
org.apache.yetus audience-annotations 0.5.0
org.apache.zookeeper zookeeper 3.6.2
org.apache.zookeeper zookeeper-jute 3.6.2
org.checkerframework checker-qual 3.5.0
org.codehaus.jackson jackson-core-asl 1.9.13
org.codehaus.jackson jackson-mapper-asl 1.9.13
org.codehaus.janino commons-compiler 3.0.16
org.codehaus.janino janino 3.0.16
org.datanucleus datanucleus-api-jdo 4.2.4
org.datanucleus datanucleus-core 4.1.17
org.datanucleus datanucleus-rdbms 4.1.19
org.datanucleus javax.jdo 3.2.0-m3
org.eclipse.jetty jetty-client 9.4.43.v20210629
org.eclipse.jetty jetty-continuation 9.4.43.v20210629
org.eclipse.jetty jetty-http 9.4.43.v20210629
org.eclipse.jetty jetty-io 9.4.43.v20210629
org.eclipse.jetty jetty-jndi 9.4.43.v20210629
org.eclipse.jetty jetty-plus 9.4.43.v20210629
org.eclipse.jetty jetty-proxy 9.4.43.v20210629
org.eclipse.jetty jetty-security 9.4.43.v20210629
org.eclipse.jetty jetty-server 9.4.43.v20210629
org.eclipse.jetty jetty-servlet 9.4.43.v20210629
org.eclipse.jetty jetty-servlets 9.4.43.v20210629
org.eclipse.jetty jetty-util 9.4.43.v20210629
org.eclipse.jetty jetty-util-ajax 9.4.43.v20210629
org.eclipse.jetty jetty-webapp 9.4.43.v20210629
org.eclipse.jetty jetty-xml 9.4.43.v20210629
org.eclipse.jetty.websocket websocket-api 9.4.43.v20210629
org.eclipse.jetty.websocket websocket-client 9.4.43.v20210629
org.eclipse.jetty.websocket websocket-common 9.4.43.v20210629
org.eclipse.jetty.websocket websocket-server 9.4.43.v20210629
org.eclipse.jetty.websocket websocket-servlet 9.4.43.v20210629
org.fusesource.leveldbjni leveldbjni-all 1.8
org.glassfish.hk2 hk2-api 2.6.1
org.glassfish.hk2 hk2-locator 2.6.1
org.glassfish.hk2 hk2-utils 2.6.1
org.glassfish.hk2 osgi-resource-locator 1.0.3
org.glassfish.hk2.external aopalliance-repackaged 2.6.1
org.glassfish.hk2.external jakarta.inject 2.6.1
org.glassfish.jaxb jaxb-runtime 2.3.2
org.glassfish.jersey.containers jersey-container-servlet 2.34
org.glassfish.jersey.containers jersey-container-servlet-core 2.34
org.glassfish.jersey.core jersey-client 2.34
org.glassfish.jersey.core jersey-common 2.34
org.glassfish.jersey.core jersey-server 2.34
org.glassfish.jersey.inject jersey-hk2 2.34
org.hibernate.validator hibernate-validator 6.1.0.Final
org.javassist javassist 3.25.0-GA
org.jboss.logging jboss-logging 3.3.2.Final
org.jdbi jdbi 2.63.1
org.jetbrains annotations 17.0.0
org.joda joda-convert 1.7
org.jodd jodd-core 3.5.2
org.json4s json4s-ast_2.12 3.7.0-M11
org.json4s json4s-core_2.12 3.7.0-M11
org.json4s json4s-jackson_2.12 3.7.0-M11
org.json4s json4s-scalap_2.12 3.7.0-M11
org.lz4 lz4-java 1.7.1
org.mariadb.jdbc mariadb-java-client 2.2.5
org.objenesis objenesis 2.5.1
org.postgresql postgresql 42.2.19
org.roaringbitmap RoaringBitmap 0.9.14
org.roaringbitmap shims 0.9.14
org.rocksdb rocksdbjni 6.20.3
org.rosuda.REngine REngine 2.1.0
org.scala-lang scala-compiler_2.12 2.12.14
org.scala-lang scala-library_2.12 2.12.14
org.scala-lang scala-reflect_2.12 2.12.14
org.scala-lang.modules scala-collection-compat_2.12 2.4.3
org.scala-lang.modules scala-parser-combinators_2.12 1.1.2
org.scala-lang.modules scala-xml_2.12 1.2.0
org.scala-sbt test-interface 1.0
org.scalacheck scalacheck_2.12 1.14.2
org.scalactic scalactic_2.12 3.0.8
org.scalanlp breeze-macros_2.12 1.2
org.scalanlp breeze_2.12 1.2
org.scalatest scalatest_2.12 3.0.8
org.slf4j jcl-over-slf4j 1.7.30
org.slf4j jul-to-slf4j 1.7.30
org.slf4j slf4j-api 1.7.30
org.slf4j slf4j-log4j12 1.7.30
org.spark-project.spark unused 1.0.0
org.threeten threeten-extra 1.5.0
org.tukaani xz 1.8
org.typelevel algebra_2.12 2.0.1
org.typelevel cats-kernel_2.12 2.1.1
org.typelevel macro-compat_2.12 1.1.1
org.typelevel spire-macros_2.12 0.17.0
org.typelevel spire-platform_2.12 0.17.0
org.typelevel spire-util_2.12 0.17.0
org.typelevel spire_2.12 0.17.0
org.wildfly.openssl wildfly-openssl 1.0.7.Final
org.xerial sqlite-jdbc 3.8.11.2
org.xerial.snappy snappy-java 1.1.8.4
org.yaml snakeyaml 1.24
oro oro 2.0.8
pl.edu.icm JLargeArrays 1.5
software.amazon.ion ion-java 1.0.2
stax stax-api 1.0.1