Live and Learn

Learned a few little tidy bits this week, or maybe relearned what was known before. But some of it was painful discovery. I was building a lab with multiple physical servers and one big virtual server host. One of the virtual guests is TMG 2010 which is used in this lab as firewall/router between multiple subnets. You know, we need to simulate production as much as possible, couple datacenters with specific server roles in it, field sites with workstations and other server roles in it, all separated, ports blocked, fun stuff like that. Well, to start, I opened all subnets to all subnets all traffic open. Everything seams to be working, DCs talking to each other, workstations joined to domain, everything happy. In one site I deployed ADCS cluster, enabled auto-enrollment, checked clients on the local subnet, they enrolled just fine. Checked clients on another subnet, they are not enrolling, getting RPC server is not available error. Hmm. Rechecked firewall – everything looks good. Rechecked all rules, groups memberships – all looking good. Enrollment is still not working. Trying to enroll manually – not working. Move one of the troubled clients to the same subnet as ADCS – it enrolls without any issues. Got to be firewall/router issue. I remembered that ISA 2006 enforced Strict RPC, so TMG should as well, but again, I checked it and all rules allowed all traffic. Search again on the forums and finally got a reminder that this darn “Strict RPC” enforcement is done in system policy and on the rule itself – have to right click on it and there on one of the tubs you’ll see check box for Strict RPC enforcement. Disabled it. Now everything works as expected. One check box to produce so much extra investigation and troubleshooting work. Fun, fun.

One of the server roles in this lab requires SQL 2008 server. No big deal. All servers in our lab are Windows 2008 R2. So SQL 2008 must be at SP1 level to be supported on R2. Not whole lot documentation on how to install it on R2, but again found a supported way to install it. Unzip SP1 bits into its own directory, run support msi from there to install SP1 SQL Support files on 2008R2 system, then run SQL setup with slipstreaming option to install it with SP1. Great. Trying to install it, when it asks to select accounts for SQL service account, I’d choose pre created AD SQLSRV account, click next and boom, SQL wizard does not like it. Hmm. Type it again. Does not like it. Check and recheck. Does not like it. Reset password in AD on this account. SQL does not like it. OK, time to Bing it. Can’t find much info about this specific issue. The only valid suggestion is to use strong password. OK, I use strong passwords for all accounts, but never mind, reset it again and made it super strong. Nope. SQL does not like it. OK, heck with it, choose to use Network Service account to complete installation and then figure out why it is not working with AD account. Go next, install. About 10 minutes later SQL installation failed. Pretty much all components. Aahh. It is late now and I ‘m going home.

I’m back in the lab and I’m working on that certificate enrollment issue. Solve that problem and return back to SQL installation. Meanwhile, we decided to roll back the server that requires SQL to 2008 SP2, knowing that it is fully supported with SQL 2008. Running install there, it accepts AD service account (good), then during install it fails exactly the same way as 2008R2 did. Frustrating… Checking logs, Binging it with error codes, and a few hits I get point to the same potential problem – installation must have “Debug Programs” privilege. aha. Btw, these images are super hardened with security scripts, so every time we install new or old application on it, and if there is no specific guide for this hardening – we might have issues with installation, and then usually fun begins. So, anyway, I look at the local security policy and of course hardening removes Admin group out from “Debug Programs” privilege. All good. Put it back. Install SQL and it works like a charm. We discuss the situation and decide to test 2008R2 with installation again (fixing the Debug Programs privilege).

Running installation of SQL 2008 on 2008R2, providing AD SQLSRV account and it accepts it. At this point I kind of guessed that it will accept it. It was working on 2008 and the code behind it is more likely identical. The only difference that I can think about between first attempts to install it and this attempt is TMG firewall configuration. In first attempt, it was still configured with Strict RPC, but by the time we got to second attempt to install SQL it was not enforcing Strict RPC. BTW, the site that SQL is installed did not have local DC – they were on other subnets. I haven’t seen a single bit of information on any forums about Strict RPC and SQL Service account validation, but it appears that it was the problem, because I can’t think of any other.

Anyway, I’ll remember these little gotchas for a long time.