Azure Synapse Notebook does't fail immediately in case of error in code

Peter Polacek 20

Hello,

I am struggling with really weird behavior when executing a notebook within Azure Synapse. I am utilizing pyspark and sql within this notebook and noticed that the notebook doesn't fail immediately in case of a bug or let's say typo in a code. For example I create a temp view and if I then make a SELECT from that view in another cell and type wrong view name (view which doesn't exist) the cell runs anyway very long time (around 30 minutes) and then it fails on some generic error:

"org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Got exception:"

I'd expect that it fails immediately on "object doesn't exist" or whatever.

I encountered the same behavior also when using python code which makes all the debugging and testing very hard.

Do you please know why it's like that and how to eventually avoid that ?

Thanks

Chandra Boorla 7,260 Reputation points Microsoft Vendor

2025-01-23T20:48:32.16+00:00

Hi @Peter Polacek

Greetings & Welcome to Microsoft Q&A forum! Thanks for posting your query!

The behavior you're experiencing in Azure Synapse Notebooks, where errors do not cause immediate failures and instead result in long execution times before failing, is not uncommon in distributed computing environments like Spark. This can occur due to the way Spark processes jobs. When you submit a job, it may take time to schedule and execute, especially if it involves querying metadata or executing tasks across a cluster.

In your case, when you create a temporary view and then attempt to query it with a non-existent name, Spark may not immediately check for the existence of the view until it tries to execute the query, which can take a significant amount of time, especially if the query involves complex operations or large datasets.

To mitigate this issue, you can implement the following strategies:

Check for View Existence - Before executing a query that references a temporary view, you can check if the view exists programmatically. This can be done by querying the metadata or using a try-except block to catch exceptions early.

Use Smaller Queries - Break down your queries into smaller parts. This way, if there's an error in a specific part, it will fail faster, allowing you to debug more efficiently.

Error Handling - Implement error handling in your code to catch exceptions and provide more informative messages, which can help in debugging.

Update Spark and Hive Versions - If you're using an older Spark pool, consider upgrading to the latest version in Synapse, as newer versions may have improved error reporting and reduced retries.

For more details, please refer: https://learn.microsoft.com/en-us/azure/hdinsight/share-hive-metastore-with-synapse

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.

Thank you.
Chandra Boorla 7,260 Reputation points Microsoft Vendor

2025-01-24T19:21:41.7766667+00:00

@Peter Polacek

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Peter Polacek 20 Reputation points

2025-01-24T19:51:43.0433333+00:00

Hello @Chandra Boorla

thank you for you response.

Unfortunately I didn't find any satisfying solution so far.

I am posting another example where the execution of a cell took more than 20 minutes until it failed even I would expect that it would fail immediately on such kind of error.
Chandra Boorla 7,260 Reputation points Microsoft Vendor

2025-01-24T21:49:13.04+00:00

@Peter Polacek

Apologies for the inconvenience.

The error message you're encountering indicates that there is an issue with accessing Azure Blob Storage (specifically, the Azure Data Lake Storage Gen2) from your Hive query. The InvalidAbfsRestOperationException suggests that there was a problem with the REST operation being performed on the Azure Blob File System. This can happen due to several reasons such as permissions issues, incorrect configurations, or even network problems.

Please refer to the following similar thread links on Q&A platform to get some useful insights.

https://learn.microsoft.com/en-us/answers/questions/565482/i-am-getting-error-while-running-the-spark-job-wit

https://learn.microsoft.com/en-us/answers/questions/1605142/data-flow-run-fails-with-invalidabfsrestoperatione

For additional information, please refer: https://azureplayer.net/2022/12/invalidabfsrestoperationexception-in-azure-synapse-notebook/

Disclaimer: This response contains a reference to a third-party World Wide Web site. Microsoft is providing this information as a convenience to you. Microsoft does not control these sites and has not tested any software or information found on these sites; therefore, Microsoft cannot make any representations regarding the quality, safety, or suitability of any software or information found there. There are inherent dangers in the use of any software found on the Internet, and Microsoft cautions you to make sure that you completely understand the risk before retrieving any software from the Internet.

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.

Thank you.
Smaran Thoomu 19,310 Reputation points Microsoft Vendor

2025-01-27T11:06:49.0133333+00:00

@Peter Polacek We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Share via

Azure Synapse Notebook does't fail immediately in case of error in code

Your answer