Azure - Puppet on-premises deploying to Azure
HowTo: Puppet on-premises to deploy resources into Azure IaaS infrastructure.
Terms:
Puppet - is an Open Source configuration management system that allows you to define the state of
your IT infrastructure, then automatically enforces the correct state.
Alternatives you may know of are Chef, or perhaps Windows PowerShell DSC.
Tennant ID - the unique identity guid that matches to your company name that owns your Azure environment.
Subscription ID - the guid for the sub-area of purchased functionality.
This might be a department or a group or some other logical unit within the Tennant,
with its own billing and users.
ARM - ("Azure Resource Manager") is the way to do things in Azure, and assign
Resources to groups and combine resources to create things, report, etc.
You assign permissions to resources, etc. This is what we will use.
ASM - the original Azure "service" based model (and matching API's) which can be thought of as V1
and should no longer be used. We decided NOT to be using/creating V1 anything.
Often the documentation refers to this as "Classic", or even RDFE (Red Dog Front-End).
IaaS - 'Infrastructure as a Service' is the platform within Azure to create a "virtual datacentre",
with virtual networks, virtual machines, virtual NICs, virtual load-balancers, etc.
You still run/control/manage absolutely all things in an IaaS environment.
Actions to be done: (noted down from memory - we did this a few days ago)
0. refer to this site for all the information on the Azure module for Puppet.
See: https://github.com/puppetlabs/puppetlabs-azure
Note - do not forget to run the GEM step when installing things.
(yes, the site I was using had missed this and we got plenty of weird errors)
To install the module:
a) puppet module install puppetlabs-azure
b) gem install azure
azure_mgmt_compute azure_mgmt_storage azure_mgmt_resources azure_mgmt_network
hocon retries --no-ri --no-rdoc
This is what Puppet uses to translate the template PP's into Azure-speak and interface to Azure.
1. To connect/configure/query Azure, you will want to install "Azure CLI",
onto your Linux/Windows/OS-X machine that is hosting the Puppet server.
See: https://azure.microsoft.com/en-gb/documentation/articles/xplat-cli-install/
2. using the Azure CLI, (the command line interface to Azure) you need to login to the target subscription.
Obviously, we need to have appropriate rights within the Subscription to create stuff.
If all is well, you can login and confirm functionality and connectivity.
NOTE: when you login from the CLI, it requests that you open a web page and enter the provided code.
Once completed, you can close the browser window.
cli: azure config mode arm
cli: azure login
3. to confirm you are in the right subscription, tenant, etc., issue a query to azure.
cli: azure account show
cut/paste the tenant and subscription GUIDs into somewhere handy - we'll need these later.
4. using Azure CLI and a simple JSON template for Azure,
create some ARM resource group, network, storage,VM etc.
see: https://github.com/Azure/azure-quickstart-templates
If that all worked, then all is good for connectivity, permissions, etc. and we are ready to turn to Puppet.
cli: azure group deployment create --template-file <template.json> --parameters-file <parameters.json> <resource-group> <deployment-name>
cli: azure group log show -l <resource-group>
5. Puppet is not going to login as you, nor get prompted for your credentials or MFA. So instead, we need
an Azure equivalent of a Service Account. This is known as a Service Principal.
6. You create the Service Principal in the Azure Directory (AAD) by creating a "dummy app" and it generates
the related "Service Principal". If you do not have permissions/ownership of the Azure directory (we didn't) , this will
will have to be done by someone who does. We had to get the Top Guy to login to the Classic UI interface to do this.
See: https://azure.microsoft.com/en-us/documentation/articles/resource-group-authenticate-service-principal/
See: https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/#create-application
When the Service Principal is created, it will show the "client secret" which we need to copy for Puppet.
Don't forget to grab it - cut/paste for later.
7. This "Service Principal" will need to be granted "Contributor" rights to the target Subscription,
so that it/Puppet can create things within our Subscription. RBAC (role based access control) will be used to set it.
NOTE: curiously, to assign this role, we need to find out/refer to the Object-ID of the Service Principal,
not the "client id" that it reports upon creation in the UI. (both are GUIDs)
We were able to get this property by querying the AAD for all Service Principals and finding our one.
cli: azure ad sp list
cli: azure role assignment create --objId <applications's object id> --role <name of role> --scope <subscription/subscription id>
8. If all worked, we should be able to login with the Service Principal to the subscription.
cli: azure login --service-principal -u client-id -p secret --tenant tenant-guid
9. another thing Puppet will ask for is the ASM (ie "Classic") certificate guid.pem file to authenticate into ASM.
(even though we don't want to use ASM, Puppet still needs this)
cli: azure config mode asm
cli: azure account cert export
cli: azure config mode arm
10. Once we have all the information, we can create the Azure.conf file for Puppet, to tell it how to connect.
We created /etc/puppetlabs/puppet/azure.conf
the file will look like this:
azure: {
tenant_id: 'your-tenant-id'
subscription_id: "your-subscription-id"
management_certificate: "\path\to\file.pem"
client_id: 'your-client-id'
client_secret: 'your-client-secret'
}
11. We should now have all the infrastructure within Puppet ready to go.
Create a TEST.pp file with some simple configuration to create and launch from Puppet.
e.g. the file might simply have
azure_vm { 'TestUbuntu':
ensure => present,
location => 'australiaeast',
image => 'canonical:ubuntuserver:14.04.2-LTS:latest',
user => 'azureuser',
password => 'P@ssw0rd!',
size => 'Standard_A0',
resource_group => 'MyTestRG',
}
$ puppet apply TEST.pp --debug
Within the Azure portal, go to the target Resource Group and you should see things appear.
We had all sorts of uninformative errors from Puppet due to some clashes. I wasted plenty of
time trying to understand what was wrong in Puppet or Azure or whatever. Make sure you
are testing using a completely empty Resource Group!