Logic App: Parallel Processing
Introduction
In this article, we would see one of the new feature of Logic Apps which helps a Workflow to process your list of actions in parallel. This feature was supported earlier also, but now it is possible to achieve in Logic App Designer. In a normal workflow, Logic Apps will process our actions one by one in sequence. Sometimes, we need to perform some actions which is independent. For example, send confirmations in email also update a status flag in SQL table. In that case, we could trigger SMTP action and SQL actions in parallel.
Parallel Processing is a mechanism by which two or more actions can take place concurrently but independently and the parent Logic App processing does not continue until all have completed.
How to use it
There are couple of ways we could bring the parallel action in Logic Apps work flow. One way is through the designer, and another way is through the code window.
To add the parallel shape through the designer.
Create a blank Logic app and add an Request shape which takes our HTTP request.
https://www.codeproject.com/KB/Articles/1183577/Working/3.JPG
Now we wont see parallel shape because there are no other actions available, continue adding one more action (in my case I have added HTTP action)
?https://www.codeproject.com/KB/Articles/1183577/Working/5.JPG
Now by clicking the `+' symbol parallel shape would be visible in between the Request and HTTP actions.
https://www.codeproject.com/KB/Articles/1183577/Working/6.JPG
Now your Logic App would looks like below, we could add as many as actions we required in each branch of parallel shape.
https://www.codeproject.com/KB/Articles/1183577/Working/7.JPG
Using Code Window
To do the same in code window, we need to add any 2 actions in Workflow then change the shape's runafter() element like below (choose the same name of previous action)
Experiments
Fire and forget calls would be performing good but what if we use request response calls inside parallel shape. To see how it behaves, We are going to create a Logic App API in normal method(Sequential) and also using parallel shape that accepts one parameter (pageNumber) in the URL and returns five records per call. These records will be selected based on the "Page Number" parameter that we pass in the URL.
- If we pass 1, first 5 records of data will be returned.
- If the "Page Number is 2 then it would be second set of Records, that is 6-10 records.
Logic App gets those records by calling the below external REST API - GetPosts() five times based on the "Page Number" parameter and returns the cumulative result.
External GetPosts() Rest Service
Our Logic App calls this External GetPost() Rest Service, below screenshot is from the PostMan showing the same.
https://howtologicapp.files.wordpress.com/2017/03/41.png?w=736
External REST Service's behavior
https://howtologicapp.files.wordpress.com/2017/04/7.png?w=736
Parallel Shape? Implementation
Code view of this parallel shape implementation
https://howtologicapp.files.wordpress.com/2017/03/52.png?w=736
Logic App API Implementation
In our Logic App, We have one HTTP GET receive shape that accepts one URL parameter called "pageNumber", based on this page number we need to return the results. Create a parallel shape that has 5 branches(each one of them will look like the below pic.) and create the REST calls in each branch of the parallel shape to the mentioned API to get respective posts. REST API Request using "PostNumber" variable for getting respective posts through the URI. API results are in Latin language so we call Google's translate API service to convert it in English, Accumulate the result and from a JSON response and return to the caller.
The overall workflow would be as follows using Parallel shape,
https://howtologicapp.files.wordpress.com/2017/04/17.png?w=736
Our PostMan Http call would look like below,
We could do the same using normal(Sequential) implementation and it would be like below,
Parallel shape only ends if all the listed actions are successfully ended in all branches, otherwise main process will wait others in parallel shape to complete processing or it will be skipped. This could affect our processing, use parallel shape when it is really required otherwise Sequential processing is our best fit.
If there are any failures then the main flow will be skipped.
I have tested bot the APIs using POSTMAN and below are the results and processing time.
Using Parallel Shape,
Using Sequential Method,
Using parallel shape it took almost 5 seconds average to process the request but it came down to 3 seconds when Sequentially processed. If any call delays inside the Parallel shape the whole process is getting delayed.
References
Since this article assumes that the reader has the basic knowledge of Azure logic apps and does not focuses on Logic apps overview, this can be learned by visiting the MSDN documentation here.
- https://blogs.msdn.microsoft.com/logicappsupdate/2017/04/21/release-update-2017-04-21/
- https://feedback.azure.com/forums/287593-logic-apps/suggestions/8759932-ability-to-have-parallel-flows
- https://feedback.azure.com/forums/287593-logic-apps/suggestions/14104464-creation-and-modification-of-variables
Code
Use the below code in your empty Logic App that creates the above mentioned REST service.
001.{
002. "definition": {
003. "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
004. "actions": {
005. "Http_GetPosts_1": {
006. "inputs": {
007. "method": "GET",
008. "uri": "https://jsonplaceholder.typicode.com/posts/@{variables('PostNumber')}"
009. },
010. "runAfter": {
011. "Increment_PostNumberBy1": [
012. "Succeeded"
013. ]
014. },
015. "type": "Http"
016. },
017. "Http_GetPosts_1_Translate": {
018. "inputs": {
019. "method": "GET",
020. "parameters": {},
021. "uri": "https://www.googleapis.com/language/translate/v2?format=text&key={your Key}&q=@{body('Http_GetPosts_1')?['body']}&source=la&target=en"
022. },
023. "runAfter": {
024. "Http_GetPosts_1": [
025. "Succeeded"
026. ]
027. },
028. "type": "Http"
029. },
030. "Http_GetPosts_2": {
031. "inputs": {
032. "method": "GET",
033. "uri": "https://jsonplaceholder.typicode.com/posts/@{variables('PostNumber')}"
034. },
035. "runAfter": {
036. "Increment_PostNumberBy2": [
037. "Succeeded"
038. ]
039. },
040. "type": "Http"
041. },
042. "Http_GetPosts_2_Translate": {
043. "inputs": {
044. "method": "GET",
045. "uri": "https://www.googleapis.com/language/translate/v2?format=text&key={YourKey}&q=@{body('Http_GetPosts_2')?['body']}&source=la&target=en"
046. },
047. "runAfter": {
048. "Http_GetPosts_2": [
049. "Succeeded"
050. ]
051. },
052. "type": "Http"
053. },
054. "Http_GetPosts_3": {
055. "inputs": {
056. "method": "GET",
057. "uri": "https://jsonplaceholder.typicode.com/posts/@{variables('PostNumber')}"
058. },
059. "runAfter": {
060. "Increment_PostNumberBy3": [
061. "Succeeded"
062. ]
063. },
064. "type": "Http"
065. },
066. "Http_GetPosts_3_Translate": {
067. "inputs": {
068. "method": "GET",
069. "uri": "https://www.googleapis.com/language/translate/v2?format=text&key={Your Key}&q=@{body('Http_GetPosts_3')?['body']}&source=la&target=en"
070. },
071. "runAfter": {
072. "Http_GetPosts_3": [
073. "Succeeded"
074. ]
075. },
076. "type": "Http"
077. },
078. "Http_GetPosts_4": {
079. "inputs": {
080. "method": "GET",
081. "uri": "https://jsonplaceholder.typicode.com/posts/@{variables('PostNumber')}"
082. },
083. "runAfter": {
084. "Increment_PostNumberBy4": [
085. "Succeeded"
086. ]
087. },
088. "type": "Http"
089. },
090. "Http_GetPosts_4_Translate": {
091. "inputs": {
092. "method": "GET",
093. "uri": "https://www.googleapis.com/language/translate/v2?format=text&key={Your Key}&q=@{body('Http_GetPosts_4')?['body']}&source=la&target=en"
094. },
095. "runAfter": {
096. "Http_GetPosts_4": [
097. "Succeeded"
098. ]
099. },
100. "type": "Http"
101. },
102. "Http_GetPosts_5": {
103. "inputs": {
104. "method": "GET",
105. "uri": "https://jsonplaceholder.typicode.com/posts/@{variables('PostNumber')}"
106. },
107. "runAfter": {
108. "Increment_PostNumberBy5": [
109. "Succeeded"
110. ]
111. },
112. "type": "Http"
113. },
114. "Http_GetPosts_5_Translate": {
115. "inputs": {
116. "method": "GET",
117. "uri": "https://www.googleapis.com/language/translate/v2?format=text&key={Your Key}&q=@{body('Http_GetPosts_5')?['body']}&source=la&target=en"
118. },
119. "runAfter": {
120. "Http_GetPosts_5": [
121. "Succeeded"
122. ]
123. },
124. "type": "Http"
125. },
126. "Increment_PostNumberBy1": {
127. "inputs": {
128. "name": "PostNumber",
129. "value": 1
130. },
131. "runAfter": {
132. "Initialize_variable": [
133. "Succeeded"
134. ]
135. },
136. "type": "IncrementVariable"
137. },
138. "Increment_PostNumberBy2": {
139. "inputs": {
140. "name": "PostNumber",
141. "value": 1
142. },
143. "runAfter": {
144. "Initialize_variable": [
145. "Succeeded"
146. ]
147. },
148. "type": "IncrementVariable"
149. },
150. "Increment_PostNumberBy3": {
151. "inputs": {
152. "name": "PostNumber",
153. "value": 1
154. },
155. "runAfter": {
156. "Initialize_variable": [
157. "Succeeded"
158. ]
159. },
160. "type": "IncrementVariable"
161. },
162. "Increment_PostNumberBy4": {
163. "inputs": {
164. "name": "PostNumber",
165. "value": 1
166. },
167. "runAfter": {
168. "Initialize_variable": [
169. "Succeeded"
170. ]
171. },
172. "type": "IncrementVariable"
173. },
174. "Increment_PostNumberBy5": {
175. "inputs": {
176. "name": "PostNumber",
177. "value": 1
178. },
179. "runAfter": {
180. "Initialize_variable": [
181. "Succeeded"
182. ]
183. },
184. "type": "IncrementVariable"
185. },
186. "Initialize_variable": {
187. "inputs": {
188. "variables": [
189. {
190. "name": "PostNumber",
191. "type": "Integer",
192. "value": "@int(triggerOutputs()['relativePathParameters']['pageNumber'])"
193. }
194. ]
195. },
196. "runAfter": {},
197. "type": "InitializeVariable"
198. },
199. "Response": {
200. "inputs": {
201. "body": {
202. "Post1": "@body('Http_GetPosts_1_Translate')?['data']?['translations']",
203. "Post2": "@body('Http_GetPosts_2_Translate')?['data']?['translations']",
204. "Post3": "@body('Http_GetPosts_3_Translate')?['data']?['translations']",
205. "Post4": "@body('Http_GetPosts_4_Translate')?['data']?['translations']",
206. "Post5": "@body('Http_GetPosts_5_Translate')?['data']?['translations']"
207. },
208. "statusCode": 200
209. },
210. "runAfter": {
211. "Http_GetPosts_1_Translate": [
212. "Succeeded"
213. ],
214. "Http_GetPosts_2_Translate": [
215. "Succeeded"
216. ],
217. "Http_GetPosts_3_Translate": [
218. "Succeeded"
219. ],
220. "Http_GetPosts_4_Translate": [
221. "Succeeded"
222. ],
223. "Http_GetPosts_5_Translate": [
224. "Succeeded"
225. ]
226. },
227. "type": "Response"
228. }
229. },
230. "contentVersion": "1.0.0.0",
231. "outputs": {},
232. "parameters": {},
233. "triggers": {
234. "manual": {
235. "inputs": {
236. "method": "GET",
237. "relativePath": "Posts/{pageNumber}",
238. "schema": {}
239. },
240. "kind": "Http",
241. "type": "Request"
242. }
243. }
244. }
245.}