Azure Web Application Firewall- Microsoft_BotManagerRuleSet_1.0

Question

Azure Web Application Firewall- Microsoft_BotManagerRuleSet_1.0

rohith v 0

This post is regarding the azure WAF unknown bots and its rules at the moment for us rule id 300700 'other bots' is being logging with errors for various API'S, i didn't see any information can anyone has more inputs on the existing issue will be helpful.

GitaraniSharma-MSFT 49,846 Reputation points Microsoft Employee

2023-11-09T18:34:11.9033333+00:00

@Anonymous , Just checking in to see if you had a chance to see the below answer. To help the community find the right answers, please do mark the post which was helpful by clicking on Accept Answer.

And, if you have any further questions do let us know.
GitaraniSharma-MSFT 49,846 Reputation points Microsoft Employee

2023-11-13T16:39:52.5766667+00:00

@Anonymous , Just checking in to see if you had a chance to see the below answer. To help the community find the right answers, please do mark the post which was helpful by clicking on Accept Answer.

And, if you have any further questions do let us know.

2 answers

Your answer

GitaraniSharma-MSFT 49,846 Reputation points Microsoft Employee

2023-11-09T18:34:11.9033333+00:00

@Anonymous , Just checking in to see if you had a chance to see the below answer. To help the community find the right answers, please do mark the post which was helpful by clicking on Accept Answer.

And, if you have any further questions do let us know.
GitaraniSharma-MSFT 49,846 Reputation points Microsoft Employee

2023-11-13T16:39:52.5766667+00:00

@Anonymous , Just checking in to see if you had a chance to see the below answer. To help the community find the right answers, please do mark the post which was helpful by clicking on Accept Answer.

And, if you have any further questions do let us know.

Answer 1

Hello @Anonymous ,

Welcome to Microsoft Q&A Platform. Thank you for reaching out & hope you are doing well.

I understand that your Azure WAF is logging errors for various APIs with rule id 300700 'other bots' and you would like to know more details about the same.

I discussed this internally with the Azure WAF Product Group team and below is the update on same:

We won't be able to publish the internal definitions of our rules, as it would enable Bot creators to circumvent the rules. The bot manager rules are broken down into three categories as below:

BadBots (Bot100*) - Bot with malicious intent (bad IP reputation or modified user agents)
GoodBots (Bot200*) - Search engine traffic
UnknownBots (Bot300*) - Other bot user agents that may or may not be malicious.

The Bot300* rules are just to inform the customer that the traffic is originating from a bot (non-browser traffic). The intent of the bot is unknown (it could be valid requests from users of the service, or it could be malicious attackers using a custom tool).

Refer: https://learn.microsoft.com/en-us/azure/web-application-firewall/ag/application-gateway-crs-rulegroups-rules?tabs=bot#bot-rules

https://learn.microsoft.com/en-us/azure/web-application-firewall/afds/waf-front-door-drs?tabs=bot#bot-rules

Microsoft_BotManagerRuleSet-1.0-UnknownBots-Bot300700 - this is classified as an "unknown" bot which are published user agents without additional validation.

If your requests are getting blocked by this rule ID, then you can check the details or message field of your WAF logs and see what is the matchVariableName and matchVariableValue triggered for those requests.

Refer: https://learn.microsoft.com/en-us/azure/web-application-firewall/ag/bot-protection-overview#log-example

https://learn.microsoft.com/en-us/azure/web-application-firewall/ag/web-application-firewall-logs#firewall-log

In most cases, I've seen headers with "non-browser user agent" getting blocked by this rule.

So, I would request you to go through your WAF rules and validate the requests and if you believe that legitimate traffic is getting blocked, you can create custom rules to allow that particular traffic.

Refer: https://learn.microsoft.com/en-us/azure/web-application-firewall/ag/web-application-firewall-troubleshoot

Application gateway WAF custom rules: https://learn.microsoft.com/en-us/azure/web-application-firewall/ag/custom-waf-rules-overview

Azure Front Door WAF custom rules: https://learn.microsoft.com/en-us/azure/web-application-firewall/afds/waf-front-door-custom-rules

If you need help in understanding your WAF log, please share the particular WAF log and we can discuss about it further.

Kindly let us know if the above helps or you need further assistance on this issue.

Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

rohith v 0 Reputation points

2023-11-10T05:57:02.7333333+00:00

Hi @Anonymous ,

Thanks for your reply and suitable suggestion for the problem, in our case we are facing issue with the headers which is blocking multiple api's could you please elaborate on the custom rule how can create a custom rule if this is blocking for multiple api's ?

Thanks.
GitaraniSharma-MSFT 49,846 Reputation points Microsoft Employee

2023-11-10T19:52:53.77+00:00

Hello @Anonymous ,

We already have an example for a custom rule which shows how to block a bot named "evilbot". In this case, you block on the User-Agent "evilbot" in the request headers.

In your case, you need to replace the value of the User-Agent which is getting blocked/allowed and create a custom rule to allow/block it in action as per your requirement.

https://learn.microsoft.com/en-us/azure/web-application-firewall/ag/create-custom-waf-rules#example-1

The above example is shown using Azure PowerShell.

However, you can also configure this in Azure portal as below:

Go to the WAF policy.

Select Custom rules tab.

Select Add custom rule.

Add a custom rule name, type (match) and priority.

And add the condition and action as below:

Regards,

Gita

Answer 2

Matthew Mckenzie 15

An awful lot of traffic does not come from Browsers so to deny anything without a Browser User Agent String is madness.

Share via

Azure Web Application Firewall- Microsoft_BotManagerRuleSet_1.0

2 answers

Your answer