Edit

Share via

On-upload malware scanning

  • Article

On-upload malware scanning in Microsoft Defender for Storage automatically scans blobs when they're uploaded or modified, providing near real-time detection against malicious content. This cloud-native, SaaS-based solution uses Microsoft Defender Antivirus to perform comprehensive malware scans, ensuring your storage accounts remain secure without the need for extra infrastructure or maintenance.

By integrating on-upload scanning into your storage accounts, you can:

  • Prevent malicious uploads: Stop malware from entering your storage environment at the point of upload.
  • Simplify security management: Benefit from automatic scanning without deploying or managing agents.
  • Enhance compliance: Meet regulatory requirements by ensuring all uploaded data is scanned for malware.

Malware upload is a top threat for cloud storage, as malicious files can enter and spread within an organization through cloud storage services. Microsoft Defender for Storage provides a built-in solution to mitigate this risk with comprehensive anti-malware capabilities.

Diagram showing how malware scanning protects your data from malicious code.

Common use cases for on-upload malware scanning

  • Web applications: Secure user-generated content uploads in web applications such as tax apps, CV upload sites, and receipt uploads.

  • Content distribution: Protect assets like images and videos shared at scale through content hubs or CDNs (Content Delivery Networks), which can be common malware distribution points.

  • Compliance requirements: Meet regulatory standards, such as NIST, SWIFT, and GDPR, by scanning untrusted content, especially for regulated industries.

  • Third-party integration: Ensure third-party data, such as content from business partners or contractors, is scanned to prevent security risks.

  • Collaborative platforms: Ensure safe collaboration across teams and organizations by scanning shared content.

  • Data pipelines: Maintain data integrity in ETL (Extract, Transform, Load) processes by ensuring no malware enters through multiple data sources.

  • Machine learning training data: Protect the quality of training data by ensuring data sets are clean and safe, particularly if they contain user-generated content.

    animated GIF showing user-generated-content and data from external sources.

Note

Malware scanning is a near real-time service. Scan times can vary depending on file size, file type, service load, and storage account activity.

Enable on-upload malware scanning

Prerequisites

  • Permissions: Owner or Contributor role on the subscription or storage account, or specific roles with the necessary permissions.
  • Defender for Storage: Must be enabled on the subscription or individual storage accounts.

To enable and configure malware scanning across your subscriptions while retaining detailed control over individual storage accounts, you can use one of the following methods:

When malware scanning is enabled, an Event Grid System Topic resource is automatically created in the same resource group as the storage account. This is used by the malware scanning service to listen to blob upload triggers.

For detailed instructions, see Deploy Microsoft Defender for Storage.

Cost control for on-upload malware scanning

Malware scanning is billed per GB scanned. To provide cost predictability, malware scanning supports setting a cap on the amount of GB scanned in a single month per storage account.

Important

Malware scanning in Defender for Storage is not included in the first 30-day trial for free and will be charged from the first day in accordance with the pricing scheme available on the Defender for Cloud pricing page.

The capping mechanism sets a monthly scanning limit, measured in gigabytes (GB), for each storage account. This serves as an effective cost control measure. If a predefined scanning limit is reached for a storage account within a single calendar month, the scanning operation automatically halts. This halt occurs once the threshold is reached, with up to a 20-GB deviation. Files aren't scanned for malware beyond this point. The cap resets at the end of every month at midnight UTC. Updating the cap typically takes up to an hour to take effect.

By default, a limit of 5 TB (5,000 GB) is established if no specific capping mechanism is defined.

Tip

You can set the capping mechanism on either individual storage accounts or across an entire subscription (every storage account on the subscription will be allocated the limit defined on the subscription level).

How malware scanning works

On-upload malware scanning flow

On-upload scans are triggered by any operation that results in a BlobCreated event, as specified in the Azure Blob Storage as an Event Grid source documentation. These operations include:

  • Uploading new blobs: When a new blob is added to a container
  • Overwriting existing blobs: When an existing blob is replaced with new content
  • Finalizing changes to blobs: Operations like PutBlockList or FlushWithClose that commit changes to a blob

Note

Incremental operations, such as AppendFile in Azure Data Lake Storage Gen2 and PutBlock in Azure BlockBlob, don't trigger a malware scan independently. A malware scan occurs only when these additions are finalized through commit operations like PutBlockList or FlushWithClose. Each commit can initiate a new scan, which might increase costs if the same data is scanned multiple times due to incremental updates.

Scanning process

  1. Event detection: When a BlobCreated event occurs, the malware scanning service detects the change.
  2. Blob retrieval: The service securely reads the blob content within the same region as your storage account.
  3. In-memory scanning: The content is scanned in-memory using Microsoft Defender Antivirus with up-to-date malware definitions.
  4. Result generation: The scan result is generated, and appropriate actions are taken based on the findings.
  5. Content disposal: Scanned content isn't retained and is deleted immediately after scanning.

Throughput and capacity for on-upload malware scanning

On-upload malware scanning has specific throughput and capacity limits to ensure performance and efficiency across large-scale operations. These limits help control the volume of data that can be processed per minute, ensuring a balance between near real-time protection and system load.

  • Scan throughput rate limit: On-upload malware scanning can process up to 50 GB per minute per storage account. If the rate of blob uploads exceeds this threshold momentarily, the system queues the files and attempt to scan them. However, if the upload rate consistently surpasses the limit, some blobs might not be scanned.

Shared aspects with on-demand scanning

The following sections are applicable to both on-demand and on-upload malware scanning.

For detailed information on these topics, see the Introduction to malware scanning page.

Best practices and tips

  • Set cost control caps for storage accounts, particularly those with high upload traffic, to manage and optimize expenses effectively.
  • Use Log Analytics to track scan history for compliance and auditing purposes.
  • If your use case requires a response mechanism, consider setting up automated responses (for example, quarantine or deletion actions) using Event Grid and Logic Apps. For detailed setup guidance, see Set up response in malware scanning.

Tip

We encourage you to explore the malware scanning feature in Defender for Storage through our hands-on lab. Follow the Ninja training instructions for a detailed guide on setup, testing, and response configuration.

Related content