Removing DFSR Filters

Hi folks, Ned here again. DFSR administrators usually know about its built-in filtering mechanism. You can configure each filter based on file name and extension; by default, files named like “*.bak, *.tmp, ~*” are filtered, as they are unlikely to be permanent or useful between servers. You can also filter out folders; this is less common, as you cannot provide a full path - only a name. Sometimes though, it is useful for a specific working folder used by applications.

When you enable a filter, each server scans its database for that replicated folder path and removes any records of files and folders that match the filter. Because the database doesn’t care about filtered objects, DFSR ignores any future changes to the files. In practical filtering terms, this means:

1. Any new files/folders added do not replicate to any other server.

2. Files/folders that already exist through previous replication stay on all servers.

3. Files/folders previously replicated and then later modified after enabling the filter are quasi-deleted from all other servers, in the sense that they are added to the database tombstone list. After all, they are filtered and therefore “no longer exist” when updated, according to the downstream servers. But the files do not actually get deleted from the replicated folder; they simply stop replicating and if anyone changes them on various server nodes, they diverge.

But what about when you remove filters?

This is rare enough that we've never bothered to document it. There are key issues to understand:

1. You must install the latest DFSR hotfixes for your operating system, on all DFSR servers.

List of currently available hotfixes for Distributed File System (DFS) technologies in Windows Server 2008 and in Windows Server 2008 R2 - https://support.microsoft.com/kb/968429

List of currently available hotfixes for Distributed File System (DFS) technologies in Windows Server 2003 and in Windows Server 2003 R2 - https://support.microsoft.com/kb/958802

Do not continue with any filter changes until installing those hotfixes and restarting the DFSR servers*. There is a very nasty bug that leads to folders refusing to replicate their previously filtered-contents or files that disappear from partner servers.

* Alternatively, you can stop the DFSR service and install the hotfixes. Generally, the removes the need to restart and no prompt displays.

Note:

I'm often asked if DFSR hotfixes are recommended preemptively and the answer is YES. Data loss hotfixes do not fix your lost data, only make further data loss stop! You should probably keep on top of NTFS hotfixes too.

2. Any changes made between servers after enabling the filter are going to result in some deleted or overwritten files, and that is by design. By setting a DFSR filter, you told DFSR that those folders and their contents no longer existed in DFSR terms. When the filter comes off and after making the first change in some server’s copy of that folder, the conflict resolution algorithm is going to kick in and the two folders synchronize, regardless of your wishes as to which folder will win.

Here I filtered subfolder2 and created – on each server – different bmp files named madeon1 (on server 01) and madeon2 (on server 02):

Then I removed the filter, forced AD replication, and forced DFSR to poll AD. Those files are different, so nothing “bad” happened and they sync:

However, if I repeat that un-filtering test with two files with the same name, but different contents (the file was originally replicated to both server, filtered out, and then later modified on server 02):

Then the newer file will win, overwriting what’s on 01 (even if it was “good” data for some user). This is why when you filter folders from replication, you should make sure it’s not user data that is still modified on multiple servers. Someday it may have to re-converge.

3. It is critical that you back up filtered folders on all servers replicating its parent, as you are very likely to need to restore some files in order to undo some of the conflict resolution damage. Some users are going to be very unhappy otherwise – and if they are the Vice President of TV Programming and Microwave Oven Technologies, they will come down on you like a load of bricks.

A minority of customers use DFSR folder filters – they are not very flexible, due to their lack of path support. They are safe only if users cannot alter them or their contents – then at least they will not have a negative experience.

Until next time,

- Ned “Director of Directories” Pyle

Comments

• Anonymous
  November 14, 2011
  I love DFSR filters ( file nobody uses folders!), the algorithm while adding new filters is tremendous improvement on FRS, right?

• Anonymous
  November 15, 2011
  Hey look, the canary got an upgrade to Canary 2.0!  Much more minimalist, and actually looks a little like you were influenced by John Lennon... http://www.lennonart.com/ :)

• Anonymous
  November 15, 2011
  Now I have a fallback career in case this whole computer thing doesn't pan out.

• Anonymous
  November 19, 2011
  I think it is a pitty that in specific circumstances, DFSR looses data. We are looking into DFRS as a solution to bring data from remote sites to a central location that is backed up in a decent way. The more I read, the more I am convinced DFSR will not enable us to make less backups, instead we will need more :(

• Anonymous
  November 25, 2011
  Dude, if you say “You must install the latest DFSR hotfixes for your operating system” then KB968429 is not the very best source since it's constantly outdated. Way better link iswww.bing.com/search Well, Here's a more readable result for those who are too lazy for Bing: social.technet.microsoft.com/.../5801.aspx

• Anonymous
  November 28, 2011
  @SenneVL - well, they were bugs. It's not like it was intentional or expected. Based on that logic, you should not be using NTFS either, as it has data loss bugs. All file systems do though. @ Artem -  That's a cool link, very good job. The KB being out of date is a constant battle, (ongoing as we speak). In this case though, it includes the necessary updates to prevent the issue.

• Anonymous
  November 28, 2011
  Updated my last reply to be more useful. :) I might also suggest adding the other DFSR-related binaries mentioned to your wiki post; that's the nice thing about the KB: it covers all related files, such as minimum NTFS.SYS.

• Anonymous
  November 29, 2011
  The comment has been removed

• Anonymous
  November 29, 2011
  Awesome, thanks. Anything listed on the DFSR hotfix KB is relevant to DFSR, so if you wanted to expand this wiki post to become "all DFSR-related QFEs" that's all it would take.

• Anonymous
  December 29, 2011
  I've just run into this issue - I removed the default filters but they're still active which is causing problems with office files (because the ~$ lockfiles aren't replicating, it's allowing concurrent open files). Is there a recommended order to apply the hotfixes?  e.g. should I recreate the filters or will applying the hotfixes alone be enough to fix the issue.

• Anonymous
  January 04, 2012
  The comment has been removed