Azure Logic Apps Incident - SQL Connector

Summary :  Starting at 06:30 UTC on 07 Mar 2017, engineers identified an issue with the Azure Logic Apps SQL Connector. A recent deployment impacted existing Azure Logic Apps SQL connections.  Mitigation: Users should self-mitigate by updating their Logic Apps that contain SQL actions.  Instructions on how to do this are available below.  In addition, you may contact the Logic Apps team for questions at logicappsemail@microsoft.com.

We are sending emails to impacted subscriptions on impacted apps.

Root Cause Analysis:

Summary of impact: Between 06:30 and 13:37 UTC on 7th March 2017, a limited subset of Logic Apps customers may have experienced Logic App failures due to a SQL connector issue. Impacted customers had the ability to mitigate the impact by updating their Logic Apps that contained SQL connections. Root cause is attributed to a deployment error which engineers quickly identified and remediated.

Root cause and mitigation: A change made to a backend configuration setting caused SQL connections to enter into an invalid state. This in turn caused SQL actions to return a 404 error code and the corresponding Logic Apps to encounter an unhealthy state. Azure Engineers received alerts about the issue and initiated investigation process accordingly. Engineers immediately deployed a fix to correct the connector configuration. Upon mitigation, a limited subset of customers may have needed to recreate their Logic Apps SQL connectors to fully mitigate the issue.

Next steps: We sincerely apologize for the impact to affected customers. We are continuously taking steps to improve the Microsoft Azure Platform, and also our processes to help ensure such incidents do not occur in the future. In this case, this includes (but is not limited to): Hardening the change management procedure trying to avoid similar regressions; Take snapshot for key configurations to improve recoverability; Further enforcing safe deployment practices to minimize the scope of the impact introduced by software update; Improving systemic monitoring platform helping to better detect API call failures and trigger mitigation process at minimal customer impact.

Provide feedback: Please help us improve the Azure customer communications experience by taking our survey https://survey.microsoft.com/319140

--

To mitigate with the visual designer (could also re-deploy an ARM template if available)

1. First, open the visual workflow designer

2. The impacted action or triggers will show as degraded / with the yellow warning due to the connection issue (SQL Connection in this case)

3. Since the connection was deleted, we will need to recreate it using the normal UX

4. We need to provide the connection configuration parameters

5. Now that the connection is created, the action or trigger should be fixed

6. For other actions that were using the same connection that was deleted, you can now reference the newly created connection in other actions as well (no need to recreate)

7. After making these changes to the connection references of the Logic App, press Save. The issue should now be resolved.

8. Resubmitting any failed runs will now work as well, so you can reprocess anything that was missed