練習 - 加入程式碼以監視和記錄應用程式中的 Azure Batch 作業進度

已完成

Batch 用戶端 API 可讓應用程式監視集區、節點、作業和工作的目前狀態。

為了完成您公司用於轉換影片的主控台應用程式,您想要讓應用程式監視並回報檔案轉換的狀態。 您也想要透過新增應用程式在影片轉換之後刪除作業和集區的功能,減少 Batch 將會產生的成本。 為了減少檔案儲存成本,您也想要移除已上傳的影片檔案。

應用程式會檢查現有的集區,並建立一個集區 (如果不存在)。 作業和工作即會啟動,然後受到監視。 一旦工作成功完成之後,應用程式會提供刪除已建立作業和集區的選項。 應用程式會自動刪除已上傳的影片,以節省 Blob 儲存體成本。

新增監視功能

  1. 在 Cloud Shell 中,於編輯器內編輯 Program.cs 檔案:

    code Program.cs
    
  2. MonitorTasksAsync() 方法新增至 Program.cs 以監視作業工作。

    private static async Task<bool> MonitorTasksAsync(BatchClient batchClient, string jobId, TimeSpan timeout)
    {
        bool allTasksSuccessful = true;
        const string completeMessage = "All tasks reached state Completed.";
        const string incompleteMessage = "One or more tasks failed to reach the Completed state within the timeout period.";
        const string successMessage = "Success! All tasks completed successfully. Output files uploaded to output container.";
        const string failureMessage = "One or more tasks failed.";
    
        Console.WriteLine("Monitoring all tasks for 'Completed' state, timeout in {0}...", timeout.ToString());
    
        // We use a TaskStateMonitor to monitor the state of our tasks. In this case, we will wait for all tasks to
        // reach the Completed state.
        IEnumerable<CloudTask> addedTasks = batchClient.JobOperations.ListTasks(JobId);
    
        TaskStateMonitor taskStateMonitor = batchClient.Utilities.CreateTaskStateMonitor();
        try
        {
            await taskStateMonitor.WhenAll(addedTasks, TaskState.Completed, timeout);
        }
        catch (TimeoutException)
        {
            await batchClient.JobOperations.TerminateJobAsync(jobId);
            Console.WriteLine(incompleteMessage);
            return false;
        }
        await batchClient.JobOperations.TerminateJobAsync(jobId);
        Console.WriteLine(completeMessage);
    
        // All tasks have reached the "Completed" state, however, this does not guarantee all tasks completed successfully.
        // Here we further check for any tasks with an execution result of "Failure".
    
        // Obtain the collection of tasks currently managed by the job. 
        // Use a detail level to specify that only the "id" property of each task should be populated. 
        // See https://learn.microsoft.com/azure/batch/batch-efficient-list-queries
        ODATADetailLevel detail = new ODATADetailLevel(selectClause: "executionInfo");
    
        // Filter for tasks with 'Failure' result.
        detail.FilterClause = "executionInfo/result eq 'Failure'";
    
        List<CloudTask> failedTasks = await batchClient.JobOperations.ListTasks(jobId, detail).ToListAsync();
    
        if (failedTasks.Any())
        {
            allTasksSuccessful = false;
            Console.WriteLine(failureMessage);
        }
        else
        {
            Console.WriteLine(successMessage);
        }
        return allTasksSuccessful;
    }
    

    TaskStateMonitor 物件會遭到呼叫,並在所有工作的狀態為 (TaskState.Completed) 時傳回。 如果應用程式等候的時間超過 timeout 值,則會逾時。

    此方法會利用 batchClient.JobOperations.ListTasks 取得 Batch 帳戶上工作的目前狀態。 我們可以透過傳遞 ODATADetailLevel 參數篩選這些呼叫,只傳回我們需要的資訊。 完成所有工作之後,程式碼必須檢查所有工作是否都成功完成,因此使用 "executionInfo/result eq 'Failure'" 的篩選搭配 ListTasks 呼叫會傳回所有失敗的工作。

  3. 將呼叫加入至 Main() 中 using 區塊內的新 MonitorTasksAsync() 方法,並儲存從 AddTaskAsync 呼叫傳回的工作清單。

    using (BatchClient batchClient = BatchClient.Open(sharedKeyCredentials))
    {
        // Create the Batch pool, which contains the compute nodes that execute the tasks.
        await CreatePoolIfNotExistAsync(batchClient, PoolId);
    
        // Create the job that runs the tasks.
        await CreateJobAsync(batchClient, JobId, PoolId);
    
        // Create a collection of tasks and add them to the Batch job. 
        // Provide a shared access signature for the tasks so that they can upload their output
        // to the Storage container.
        List<CloudTask> runningTasks = await AddTasksAsync(batchClient, JobId, inputFiles, outputContainerSasUrl);
    
        // Monitor task success or failure, specifying a maximum amount of time to wait for
        // the tasks to complete.
        await MonitorTasksAsync(batchClient, JobId, TimeSpan.FromMinutes(30));
    }
    

清理

  1. 在 using 區塊內部,MonitorTasks 方法呼叫的呼叫底下,加入此清除程式碼。

    // Delete input container in storage
    Console.WriteLine("Deleting container [{0}]...", inputContainerName);
    CloudBlobContainer container = blobClient.GetContainerReference(inputContainerName);
    await container.DeleteIfExistsAsync();
    
    // Clean up the job (if the user so chooses)
    Console.WriteLine();
    Console.Write("Delete job? [yes] no: ");
    string response = Console.ReadLine().ToLower();
    if (response != "n" && response != "no")
    {
        await batchClient.JobOperations.DeleteJobAsync(JobId);
    }
    
    // Clean up the pool (if the user so chooses - do not delete the pool if new batches of videos are ready to process)
    Console.Write("Delete pool? [yes] no: ");
    response = Console.ReadLine().ToLower();
    if (response != "n" && response != "no")
    {
        Console.WriteLine("Deleting pool ...");
        await batchClient.PoolOperations.DeletePoolAsync(PoolId);
        Console.WriteLine("Pool deleted.");
    }
    

    上述程式碼會刪除輸入容器,其中包含所有已上傳的影片。

    接著,終端會提示使用者選擇刪除作業和集區。 batchClient 可讓應用程式刪除這些元件。

測試主控台應用程式

  1. 在程式碼編輯器中,以滑鼠右鍵按一下並選取 [儲存],然後選取 [結束]

  2. 建置並執行應用程式。

    dotnet run
    
  3. 您應該會看到類似下列的輸出:

    Creating container [input].
    Creating container [output].
    Uploading file ~\cutifypets\InputFiles\3.mp4 to container [input]...
    Uploading file ~\cutifypets\InputFiles\2.mp4 to container [input]...
    Uploading file ~\cutifypets\InputFiles\4.mp4 to container [input]...
    Uploading file ~\cutifypets\InputFiles\1.mp4 to container [input]...
    Uploading file ~\cutifypets\InputFiles\5.mp4 to container [input]...
    Uploading file ~\cutifypets\InputFiles\6.mp4 to container [input]...
    Creating pool [WinFFmpegPool]...
    Creating job [WinFFmpegJob]...
    Adding 6 tasks to job [WinFFmpegJob]...
    Monitoring all tasks for 'Completed' state, timeout in 00:30:00...
    All tasks reached state Completed.
    Success! All tasks completed successfully. Output files uploaded to output container.
    Deleting container [input]...
    
    Delete job? [yes] no: y
    Delete pool? [yes] no: y
    
    Sample complete, hit ENTER to exit...
    
  4. 不過,如果先前執行中的作業仍然存在,應用程式將會失敗,因為先前的應用程式沒有作業清除程式碼。 您可以在 Azure 入口網站中刪除該作業,或在 Cloud Shell 中使用下列方式刪除該作業:

    az batch job delete --job-id WinFFmpegJob \
    --account-name $BATCH_NAME \
    --account-key $BATCH_KEY \
    --account-endpoint $BATCH_URL
    
    Are you sure you want to perform this operation? (y/n):
    

    在出現提示時,輸入 y

  5. 主控台應用程式這次將會執行得更快,因為這些節點 (三部執行 Windows 2012 的虛擬機器) 將會閒置,並等待執行作業。