Freigeben über


TMG Safe search feature may not work as expected sometimes

In this post, I would like to talk about a TMG safe search problem which I dealt with recently. The problem was that the TMG safesearch feature wasn’t working properly even though the all the requirements were met. We decided to collect dynamic logs while reproducing the problem (TMG data packager + client side logs). Then I started analyzing the logs:

 

- The Safesearch related rule was already enabled:

 

SafeSearch Enabled

   Action Allow

Applies Always true

Default false

Description The SafeSearch rule allows access to the "Search Engines" URL category, and filters out adult content from search results of supported search engines.

ID 

Logging Enabled

Source Port Limits Disabled

Type Access Rule

From

  Network Internal, Included, Array scope

Applies to all content

Protocols Specified Protocols

 HTTP, Included, Array scope

 HTTPS, Included, Array scope

UrlCategory Search Engines, Included, Enterprise Scope

Users All Users, Included, Array scope

 

- It already included "All users"

- TMG server version was the latest (SP2 + rollup2)  

- Safesearch XML file was in place with the correct settings:

 

<Configuration>

                <provider domainPattern=".google." safeSearchSuffix="&amp;safe=strict">

                               <searchQuery pattern="/search?"/>

                               <searchQuery pattern="/images?"/>

                </provider>

                <provider domainPattern=".yahoo.com" safeSearchSuffix="&amp;vm=r">

                               <searchQuery pattern="/search?"/>

                               <searchQuery pattern="/search;"/>

                               <searchQuery pattern="/search/images?"/>

                               <searchQuery pattern="/search/images;"/>

                               <searchQuery pattern="/search/video?"/>

                               <searchQuery pattern="/search/video;"/>

                </provider>

                <provider domainPattern="www.bing.com" safeSearchSuffix="&amp;adlt=strict">

                               <searchQuery pattern="/search?"/>

                </provider>

</Configuration>

 

- It's the same as the one provided in ISA blog:

 

https://blogs.technet.com/b/isablog/archive/2010/09/21/new-in-forefront-tmg-update-1-safesearch-enforcement.aspx

 

<provider domainPattern="www.bing.com" safeSearchSuffix="&amp;adlt=strict" >

         <searchQuery pattern="/search?" />

 </provider>

 

=> Even though all configuration was correct, TMG server was still not forwarding the correct search URL towards Bing (which would help Bing filter search results)

 

- We see the search requests sent by the client on the client side network trace. Some examples:

 

2857 12:46:22.524738 50.167033 0.000000 10.96.96.6 10.110.0.121 HTTP GET https://www.bing.com/search?q=xxx&qs=n&form=QBLH&pq=xxx&sc=0-0&sp=-1&sk= HTTP/1.1        

3642 12:46:44.659253 72.301548 0.054843 10.96.96.6 10.110.0.121 HTTP GET https://www.bing.com/images/search?q=xxx&FORM=HDRSC2 HTTP/1.1          

3832 12:46:48.339288 75.981583 3.680035 10.96.96.6 10.110.0.121 HTTP GET https://www.bing.com/search?q=xxx&FORM=HDRSC1 HTTP/1.1           

 

=> Relevant session in TMG ETL Logs: (collected as a result of TMG data packager)

 

...

[0]197c.1cc8 10/03/2012-10:46:48.780 [1a2c7b91 1a2c7d03] [WP proxyext...] Info: WPPISAPUBLIC:Context property:WPPISAPUBLIC:HTTP URL = https://www.bing.com/search?q=xxx&FORM=HDRSC1

...  

 

=> Safesearch filter decides that there's no need for safesearch analysis:

 

[0]197c.1cc8 10/03/2012-10:46:48.781 [1a2c7b91 1a2c7d03] [HTTPFLT...] Entering CHttpFilterSafeSearchEnforcer::IsSafeSearchAnalysisRequired

...

[0]197c.1cc8 10/03/2012-10:46:48.781 [1a2c7b91 1a2c7d03] [HTTPFLT...] Info:SafeSearch analysis is not required, exiting

 

=> Network trace collected on the external interface of TMG server:

 

- We can see that the request is being sent as is without any modifications towards the Bing server:

 

256368 06:46:49.2700830 875.6370830 10.110.7.54 212.252.126.59 HTTP HTTP:Request, GET /search, Query:q=xxx&FORM=HDRSC1 {HTTP:3723, TCP:3721, IPv4:3651}

256370 06:46:49.3080860 875.6750860 212.252.126.59 10.110.7.54 TCP TCP:Flags=...A...., SrcPort=HTTP(80), DstPort=58199, PayloadLen=0, Seq=3768470858, Ack=3665294676, Win=25515 (scale factor 0x0) = 25515 {TCP:3721, IPv4:3651}

256372 06:46:49.3600890 875.7270890 212.252.126.59 10.110.7.54 HTTP HTTP:Response, HTTP/1.1, Status: Ok, URL: /search {HTTP:3723, TCP:3721, IPv4:3651}

 

https://www.bing.com/search?q=sex&FORM=HDRSC1

 

=> Normally it should have been sent something like below so that adult content wouldn't have been returned by Bing:

 

https://www.bing.com/search?q=sex&FORM=HDRSC1&adlt=strict

 

At this point everything seemed to be correctly configured but for some reason the safesearch filter wasn’t kicking in. After some more research and with the help of an escalation engineer from TMG team, we found out that in order for safesearch filter to kick in, the request itself also should be matching the Safesearch rule which is automatically created when Safesearch is enabled.

 

In my customer scenario, problem was that the access request sent by the user was hitting an enterprise level access rule and hence the safesearch rule wasn’t hit and the filter wasn’t activated as a result. After my customer re-arranged the enterprise level rules so that search engine related requests don’t hit any enterprise level access rule but hits the array level “Safesearch” rule created automatically, the problem was resolved.

Hope this helps

 

Thanks,

Murat