Setting up various topologies to test with Visual Studio Lab Management – Part 1
The article will cover various ways in which customers can setup their topologies for testing with Lab Management. We will cover this in a series picking up one topology at a time, but before we get into details let us get few terms defined that we would use in the rest of the article:
- Corp network – Would refer to the corp. network where TFS is hosted. A Corp network would have one or more ATs, DTs, Load balancers, Proxy, VMM server. Clients are typically connected to this network while at work.
- Test network – Would refer to the network where the testing of the apps would happen. Test controller, Build controller, VMM server, VMM Library VMM hosts are present in this network. Please note that VMM server can either be in Corp network on in Test network depending on your choice.
- Corp Domain – the domain controller in corp. network
- Private Lab Domain – the domain controller in Private Test network.
Some of the acronyms that would be used in the article and in diagrams are:
- AT – Application Tier
- DT – Data Tier
- VMM – System Centre Virtual Machine Manager
- TC – Test controller
- BC – Build Controller
- BSH – Build Service Host
Let’s be clear about a very basic rule before we take up topologies for discussion. In Visual Studio Lab Management 2010, there should always be a full domain trust between all of the listed components: TFS, Test Controller, Build Controller, VMM server and the VMM host. Partial trust or Un-trusted setups between Corp network and Test network is not officially supported for Lab Management scenarios.
Now that the ground rules are clear, let us get into details. In this article we will pick a sample topology and walk through the configurations specifically the ones required for getting Lab Management working.
Topology #1 – Topology with multiple AT, Load Balancer and Test network with firewall settings controlling the test traffic in and out into the Corp network.
The Corp network in our example here has two application tiers each running with same domain account. There is a common DT with default SQL instance and a ARR NLB. The NLB node exposes both HTTP and HTTPS end points to cater to intranet and internet users. We would assume the requirement here is to have NTLM authentication while connecting to TFS.
The Test network has all the required components to test with Lab Management. We assume that there are firewall settings required to be configured such that only the required traffic flows in and out of test network into the Corp network. This can be accomplished either using windows firewall or by configuring ACLs in the LAN network switch. In this article we will cover the firewall settings using windows firewall.
Setting up ARR NLB
Configure ARR as explained in https://technet.microsoft.com/en-us/library/ee683953(WS.10).aspx to configure RR server farm to load balance HTTP.
Add both the application tier machines to the ARR server farm. While adding the application tiers to the server farm, remember to provide the appropriate HTTP port with which the TFS application tiers are configured with. If you had configured application tiers with default values it would be 8080.
Since the ARR node is exposed to internet, add a HTTPS binding to the default website that would be used by the internet users. Add an explicit IP, Port and SSL certificate binding for the website. You may choose to have a friendly name for the ARR node while it is exposed in the internet, let’s say it is https://www.mycompanytfs.org:8443 for our discussion here. In such case you need to have the SSL certificate for this name.
Add a HTTP binding for the intranet users. You can choose allow all IP address and bind without any specific host record.
You can take a look at https://blogs.msdn.com/ablock/archive/2009/08/24/exposing-tfs-2010-beta-2-to-the-internet.aspx which talks about ways to expose TFS to internet.
Setting up Application Tiers
- Installation and configuration of primary AT
Follow the Team Foundation 2010 Setup instructions to install the primary AT and DT. In TFS you can opt to allow all IP address, bound to port 8080 and with no host records.
Launch your TFS Admin console and ensure the below settings are done:
a) By default the notification URL will be set to HTTP endpoint of the application tier itself. Click on Change URL and set the Notification URL to the HTTPS endpoint of ARR NLB, which is https://www.mycompanytfs.org:8443 in our example.
b) Leave the “Server URL” value as it is. By default it will be https://localhost:8080/tfs
c) Click on Application Tier –> Lab Management –> Re configure Lab Management –> Advanced and check the Lab URL. The Lab URL will be same as Notification URL, in our example we should see https://www.mycompanytfs.org:8443 which was set in step a) explained above. Lab URL is used by build agent in Lab environment while connecting to the TFS. In our topology, we will assume that the golden templates will be configured to reach TFS via HTTP end point. Hence edit the Lab URL to https://<ARR_FQDN:8080/tfs. If you wish to configure build agent to communicate to TFS via HTTPS, then you should install appropriate client SSL certificate before you prepare the golden templates.
d) Our topology assumes NTLM authentication. Click on Application Tier –> Authentication Settings and set to “NTLM” if not present already. By default it would be NTLM.
Enable following windows firewall rules in application tier to enable packet flow required for Lab Management scenarios.
#TFS AT Inbound
CALL netsh advfirewall firewall add rule name="TFS Website 8443" protocol=TCP dir=in localport=8443 action=allow
CALL netsh advfirewall firewall add rule name="TFS Website 8080" protocol=TCP dir=in localport=8080 action=allow#TFS AT Outbound
CALL netsh advfirewall firewall add rule name="VMM" protocol=TCP dir=out remoteport=8100 action=allow
CALL netsh advfirewall firewall add rule name="BSH" protocol=TCP dir=out remoteport=9191 action=allow
CALL netsh advfirewall firewall add rule name="WMI Port range" protocol=TCP dir=out remoteport=135, 49152-65535 action=allow
CALL netsh advfirewall firewall add rule name="MS-SQL" protocol=TCP dir=out remoteport=1433 action=allow
CALL netsh advfirewall firewall add rule name="TC DotNet Remoting" protocol=TCP dir=out remoteport=6901 action=allow
- Installation and configuration of secondary AT
Follow the TFS 2010 setup instructions to install and configure “AT only” configuration. For our topology we assume all the application tiers are running with same domain account MyDomain\TfsATServiceAccount.
Launch TFS Admin Console and check if the AT service account is same. If not change it to MyDomain\TfsATServiceAccount by clicking on Application Tier –> Change Account.
Ensure that the authentication setting is set to “NTLM”.
Just as in Primary AT, you can have similar IIS bindings for Team Foundation Server Web site that allows all IP address, bound to port 8080 and with no host records.
Add the same firewall rules as done in primary AT.
Configuring Controllers
Take a look at the Visual Studio 2010 setup instructions for Test controller setup and TFS 2010 setup instructions for Build controller setup. The getting started video of Lab Management should help https://blogs.msdn.com/lab_management/archive/2010/02/11/visual-studio-lab-management-videos.aspx.
Specifically for this topology, while configuring the TFS URL in test and build controllers point to the HTTP end point of ARR NLB; :8080/tfs">:8080/tfs">:8080/tfs">:8080/tfs">:8080/tfs">https://<ARR_FQDN>:8080/tfs
Add the below windows firewall rules in your test and build controller machines which allows explicit ports that are required for Lab Management’s Build, deploy test scenarios to work. If you wish to have a tight control over what packets flow out of test network to corp network, you can block all other packets. However please remember to allow the packets required for infrastructure needs like Domain authentication, DNS resolution, NetBios resolution, RDP etc.
#Test Controller Inbound
CALL netsh advfirewall firewall add rule name="TC DotNet Remoting" protocol=TCP dir=in localport=6901 action=allow
#Test Controller Outbound
CALL netsh advfirewall firewall add rule name="TFS Website 8443" protocol=TCP dir=out remoteport=8443 action=allow
CALL netsh advfirewall firewall add rule name="TFS Website 8080" protocol=TCP dir=out remoteport=8080 action=allow
CALL netsh advfirewall firewall add rule name="TA DotNet Remoting" protocol=TCP dir=out remoteport=6910 action=allow
#Build Controller Inbound
CALL netsh advfirewall firewall add rule name="BSH BC" protocol=TCP dir=in localport=9191 action=allow
#Build Controller Outbound
CALL netsh advfirewall firewall add rule name="TFS Website 8443" protocol=TCP dir=out remoteport=8443 action=allow
CALL netsh advfirewall firewall add rule name="TFS Website 8080" protocol=TCP dir=out remoteport=8080 action=allow
CALL netsh advfirewall firewall add rule name="BSH BA" protocol=TCP dir=out remoteport=9191 action=allow
VMM server, VMM library and VMM host
Take a look at the TFS 2010 setup instructions for configuring Lab Management and SCVMM setup instructions. The getting started video of Lab Management should help https://blogs.msdn.com/lab_management/archive/2010/02/11/visual-studio-lab-management-videos.aspx.
Add the below windows firewall rules in VMM server, library and hosts, which are required for Lab Management. Again you can block other ports if you wish to have tight control over packets that flows out of Test network to Corp network. Remember to allow the packets required for infrastructure needs like Domain authentication, DNS resolution, NetBios resolution, RDP etc.
#VMM Server Inbound
CALL netsh advfirewall firewall add rule name="VMM Server" protocol=TCP dir=in localport=8100 action=allow
#VMM Server Outbound
CALL netsh advfirewall firewall add rule name="VMM service" protocol=TCP dir=out remoteport=8100 action=allow
CALL netsh advfirewall firewall add rule name="WinRm" protocol=TCP dir=out remoteport=80 action=allow
CALL netsh advfirewall firewall add rule name="BITS" protocol=TCP dir=out remoteport=443 action=allow
#VMM Library Inbound
CALL netsh advfirewall firewall add rule name="VMM Agent" protocol=TCP dir=in localport=8100 action=allow
CALL netsh advfirewall firewall add rule name="WinRm" protocol=TCP dir=in localport=80 action=allow
CALL netsh advfirewall firewall add rule name="BITS" protocol=TCP dir=in localport=443 action=allow
#VMM LIbrary Outbound
CALL netsh advfirewall firewall add rule name="VMM service" protocol=TCP dir=out remoteport=8100 action=allow
#VMM Host Inbound
CALL netsh advfirewall firewall add rule name="VMM Agent" protocol=TCP dir=in localport=8100 action=allow
CALL netsh advfirewall firewall add rule name="WMI Port range" protocol=TCP dir=in localport=135, 49152-65535 action=allow
CALL netsh advfirewall firewall add rule name="RDP" protocol=TCP dir=in localport=3389 action=allow
#VMM Host Outbound
CALL netsh advfirewall firewall add rule name="VMM service" protocol=TCP dir=out remoteport=8100 action=allow
Getting the Golden Templates ready
Take a look at VMPrep tool at https://code.msdn.microsoft.com/vslabmgmt. Prepare the Golden templates using VM prep tool as suggested in the link.
Add the below windows firewall rules in the golden templates, which are required for Lab Management. Again you can block other ports if you wish to have tight control over packets that flows out of Test network to Corp network. Remember to allow the packets required for infrastructure needs like Domain authentication, DNS resolution, NetBios resolution, RDP etc.
#Data VM Inbound
CALL netsh advfirewall firewall add rule name="BSH BA" protocol=TCP dir=in localport=9191 action=allow
CALL netsh advfirewall firewall add rule name="TA DotNet Remoting" protocol=TCP dir=in localport=6910 action=allow
CALL netsh advfirewall firewall add rule name="Lab Agent" protocol=TCP dir=in localport=9050 action=allow
#Data VM Outbound
CALL netsh advfirewall firewall add rule name="TFS Website 8443" protocol=TCP dir=out remoteport=8443 action=allow
CALL netsh advfirewall firewall add rule name="TFS Website 8080" protocol=TCP dir=out remoteport=8080 action=allow
CALL netsh advfirewall firewall add rule name="TC DotNet Remoting" protocol=TCP dir=out remoteport=6901 action=allow
CALL netsh advfirewall firewall add rule name="Lab Agent" protocol=TCP dir=out remoteport=9050 action=allow
CALL netsh advfirewall firewall add rule name="BSH BC" protocol=TCP dir=out remoteport=9191 action=allow
In case of Golden Network isolated environment template, add the below set of firewall rules in addition to the above, which would enable traffic flow in the internal network in isolated environments.
CALL netsh advfirewall firewall add rule name="Allow TCP incoming Internal" protocol=TCP dir=in remoteip=192.168.23.0/24 action=allow
CALL netsh advfirewall firewall add rule name="Allow UDP incoming Internal" protocol=UDP dir=in remoteip=192.168.23.0/24 action=allow
CALL netsh advfirewall firewall add rule name="Allow TCP outgoing Internal" protocol=TCP dir=out remoteip=192.168.23.0/24 action=allow
CALL netsh advfirewall firewall add rule name="Allow UDP outgoing Internal" protocol=UDP dir=out remoteip=192.168.23.0/24 action=allow
I have assumed the internal network subnet is 192.168.23.0/24, which is the default value. If you have modified subnet range while configuring Lab Management in TFS, then use appropriate subnet range in the rules.
Clients
Clients in Corp network can connect either to ARR NLB HTTP endpoint or directly to the application tier HTTP endpoint.
Clients in Internet will need to have the client certificate to connect to the ARR NLB HTTPS endpoint that the NLB is bound to. If you are in un trusted domain but accessing the exposed TFS website, you will have to authenticate yourselves while connecting to the TFS.
With respect to Lab Management while outside of Corp Network, but if you are in VPN or DA, you can pretty much do all Lab operations as you were within the Corp.
When your client is in Internet and if it is not in VPN / DA, then from MTM you will be able to do all Lab operations which are internally web service calls. Hence pretty much all Lab operations can be done via MTM. However, you will not be able to connect to the environments via LE viewer. From Visual studio, you will be able to run a complete build, deploy test workflow on Lab environments.