練習 - 加入程式碼以監視和記錄應用程式中的 Azure Batch 作業進度
Batch 用戶端 API 可讓應用程式監視集區、節點、作業和工作的目前狀態。
為了完成您公司用於轉換影片的主控台應用程式,您想要讓應用程式監視並回報檔案轉換的狀態。 您也想要透過新增應用程式在影片轉換之後刪除作業和集區的功能,減少 Batch 將會產生的成本。 為了減少檔案儲存成本,您也想要移除已上傳的影片檔案。
應用程式會檢查現有的集區,並建立一個集區 (如果不存在)。 作業和工作即會啟動,然後受到監視。 一旦工作成功完成之後,應用程式會提供刪除已建立作業和集區的選項。 應用程式會自動刪除已上傳的影片,以節省 Blob 儲存體成本。
新增監視功能
在 Cloud Shell 中,於編輯器內編輯
Program.cs
檔案:code Program.cs
將
MonitorTasksAsync()
方法新增至 Program.cs 以監視作業工作。private static async Task<bool> MonitorTasksAsync(BatchClient batchClient, string jobId, TimeSpan timeout) { bool allTasksSuccessful = true; const string completeMessage = "All tasks reached state Completed."; const string incompleteMessage = "One or more tasks failed to reach the Completed state within the timeout period."; const string successMessage = "Success! All tasks completed successfully. Output files uploaded to output container."; const string failureMessage = "One or more tasks failed."; Console.WriteLine("Monitoring all tasks for 'Completed' state, timeout in {0}...", timeout.ToString()); // We use a TaskStateMonitor to monitor the state of our tasks. In this case, we will wait for all tasks to // reach the Completed state. IEnumerable<CloudTask> addedTasks = batchClient.JobOperations.ListTasks(JobId); TaskStateMonitor taskStateMonitor = batchClient.Utilities.CreateTaskStateMonitor(); try { await taskStateMonitor.WhenAll(addedTasks, TaskState.Completed, timeout); } catch (TimeoutException) { await batchClient.JobOperations.TerminateJobAsync(jobId); Console.WriteLine(incompleteMessage); return false; } await batchClient.JobOperations.TerminateJobAsync(jobId); Console.WriteLine(completeMessage); // All tasks have reached the "Completed" state, however, this does not guarantee all tasks completed successfully. // Here we further check for any tasks with an execution result of "Failure". // Obtain the collection of tasks currently managed by the job. // Use a detail level to specify that only the "id" property of each task should be populated. // See https://learn.microsoft.com/azure/batch/batch-efficient-list-queries ODATADetailLevel detail = new ODATADetailLevel(selectClause: "executionInfo"); // Filter for tasks with 'Failure' result. detail.FilterClause = "executionInfo/result eq 'Failure'"; List<CloudTask> failedTasks = await batchClient.JobOperations.ListTasks(jobId, detail).ToListAsync(); if (failedTasks.Any()) { allTasksSuccessful = false; Console.WriteLine(failureMessage); } else { Console.WriteLine(successMessage); } return allTasksSuccessful; }
TaskStateMonitor
物件會遭到呼叫,並在所有工作的狀態為 (TaskState.Completed
) 時傳回。 如果應用程式等候的時間超過timeout
值,則會逾時。此方法會利用
batchClient.JobOperations.ListTasks
取得 Batch 帳戶上工作的目前狀態。 我們可以透過傳遞ODATADetailLevel
參數篩選這些呼叫,只傳回我們需要的資訊。 完成所有工作之後,程式碼必須檢查所有工作是否都成功完成,因此使用"executionInfo/result eq 'Failure'"
的篩選搭配ListTasks
呼叫會傳回所有失敗的工作。將呼叫加入至
Main()
中 using 區塊內的新MonitorTasksAsync()
方法,並儲存從AddTaskAsync
呼叫傳回的工作清單。using (BatchClient batchClient = BatchClient.Open(sharedKeyCredentials)) { // Create the Batch pool, which contains the compute nodes that execute the tasks. await CreatePoolIfNotExistAsync(batchClient, PoolId); // Create the job that runs the tasks. await CreateJobAsync(batchClient, JobId, PoolId); // Create a collection of tasks and add them to the Batch job. // Provide a shared access signature for the tasks so that they can upload their output // to the Storage container. List<CloudTask> runningTasks = await AddTasksAsync(batchClient, JobId, inputFiles, outputContainerSasUrl); // Monitor task success or failure, specifying a maximum amount of time to wait for // the tasks to complete. await MonitorTasksAsync(batchClient, JobId, TimeSpan.FromMinutes(30)); }
清理
在 using 區塊內部,
MonitorTasks
方法呼叫的呼叫底下,加入此清除程式碼。// Delete input container in storage Console.WriteLine("Deleting container [{0}]...", inputContainerName); CloudBlobContainer container = blobClient.GetContainerReference(inputContainerName); await container.DeleteIfExistsAsync(); // Clean up the job (if the user so chooses) Console.WriteLine(); Console.Write("Delete job? [yes] no: "); string response = Console.ReadLine().ToLower(); if (response != "n" && response != "no") { await batchClient.JobOperations.DeleteJobAsync(JobId); } // Clean up the pool (if the user so chooses - do not delete the pool if new batches of videos are ready to process) Console.Write("Delete pool? [yes] no: "); response = Console.ReadLine().ToLower(); if (response != "n" && response != "no") { Console.WriteLine("Deleting pool ..."); await batchClient.PoolOperations.DeletePoolAsync(PoolId); Console.WriteLine("Pool deleted."); }
上述程式碼會刪除輸入容器,其中包含所有已上傳的影片。
接著,終端會提示使用者選擇刪除作業和集區。
batchClient
可讓應用程式刪除這些元件。
測試主控台應用程式
在程式碼編輯器中,以滑鼠右鍵按一下並選取 [儲存],然後選取 [結束]。
建置並執行應用程式。
dotnet run
您應該會看到類似下列的輸出:
Creating container [input]. Creating container [output]. Uploading file ~\cutifypets\InputFiles\3.mp4 to container [input]... Uploading file ~\cutifypets\InputFiles\2.mp4 to container [input]... Uploading file ~\cutifypets\InputFiles\4.mp4 to container [input]... Uploading file ~\cutifypets\InputFiles\1.mp4 to container [input]... Uploading file ~\cutifypets\InputFiles\5.mp4 to container [input]... Uploading file ~\cutifypets\InputFiles\6.mp4 to container [input]... Creating pool [WinFFmpegPool]... Creating job [WinFFmpegJob]... Adding 6 tasks to job [WinFFmpegJob]... Monitoring all tasks for 'Completed' state, timeout in 00:30:00... All tasks reached state Completed. Success! All tasks completed successfully. Output files uploaded to output container. Deleting container [input]... Delete job? [yes] no: y Delete pool? [yes] no: y Sample complete, hit ENTER to exit...
不過,如果先前執行中的作業仍然存在,應用程式將會失敗,因為先前的應用程式沒有作業清除程式碼。 您可以在 Azure 入口網站中刪除該作業,或在 Cloud Shell 中使用下列方式刪除該作業:
az batch job delete --job-id WinFFmpegJob \ --account-name $BATCH_NAME \ --account-key $BATCH_KEY \ --account-endpoint $BATCH_URL Are you sure you want to perform this operation? (y/n):
在出現提示時,輸入 y。
主控台應用程式這次將會執行得更快,因為這些節點 (三部執行 Windows 2012 的虛擬機器) 將會閒置,並等待執行作業。