Executing mutiple queries in a single transaction of a notebook on databricks

Question

I'm working on executing multiple queries within a single transaction on Databricks using Golang. However, I'm encountering a 'not implemented' error. Upon reviewing the library code, I found the following:

 

// Not supported in Databricks.

func (c *conn) Begin() (driver.Tx, error) {

    return nil, dbsqlerrint.NewDriverError(context.TODO(), dbsqlerr.ErrNotImplemented, nil)

}

 

// Not supported in Databricks.

func (c *conn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) {

    return nil, dbsqlerrint.NewDriverError(context.TODO(), dbsqlerr.ErrNotImplemented, nil)

}

 

 

Is there an available method or workaround to execute multiple queries within a single transaction in Databricks?

Answer 1

Hi @Gabriel-2005

Welcome to Microsoft Q&A platform and thanks for posting your query here.

This functionality is not supported by Databricks. For more details, refer to the guide on How are transactions scoped on Databricks?

Databricks manages transactions at the table level, meaning that each transaction operates on a single table. When it comes to handling concurrent transactions, Databricks employs optimistic concurrency control. This approach avoids locking mechanisms on table reads or writes, ensuring that deadlocks cannot occur.

By default, Databricks offers snapshot isolation for read operations and write-serializable isolation for write operations. While write-serializable isolation provides stronger guarantees than snapshot isolation, these guarantees are specifically applied to write transactions.

For read operations that involve multiple tables, Databricks returns the current version of each table as of the time they are accessed. This behavior does not interfere with other concurrent transactions that may be modifying the tables in question.

It’s important to note that Databricks does not support BEGIN/END constructs for grouping multiple operations into a single transaction.

When applications need to modify multiple tables, transactions are committed individually to each table in a sequential manner. However, you can use the MERGE INTO command to combine inserts, updates, and deletes into a single write transaction for a table. Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.