The Mooney Project

This is an ongoing series on Window’s Azure.  The series starts here, and all code is located on GitHub: https://github.com/mmooney/MMDB.AzureSample.  GitHub commit checkpoints are referenced throughout the posts.

In the previous post we covered deploying an app to Azure through the Azure web portal and from Visual Studio.  In this post, we’ll show you how to deploy to Azure from PowerShell.  This comes in really handy if you want to be able to deploy to right from your build server, and who doesn’t want to do that?

Why now?

So we have not really gotten into much detail about Azure, and our app is stupidly simple, why are we getting into mundane operational gold-plating like automating deployments from a build server?

Because it’s really important to automate your whole build/deploy pipeline as soon as possible.  The later you automated it, the more time you are flushing down the toilet.  Even if you don’t want to deploy automatically from your build server, if you don’t at least boil your whole deployment down to single one-click script file, you’re stealing from yourself.

When I started out with SportsCommander, I was building all the code locally in Visual Studio and then deploying through the Azure web portal (I know, caveman stuff right?).  Anyhow, pretty soon I got everything built and versioned through a TeamCity build server, and even had the site being FTPed to our shared hosting test server (hello, LFC Hosting), but for production deployments to Azure I would still remote into the build server and upload the latest package from the hard drive to the Azure website.  Part of this was that I wanted to be able to test everything in the test server before deploying to production, and part of this was that I wanted to make sure it didn’t get screwed up, but part of it was also the logical fallacy that I didn’t have time to sit down and spend the time to figure out how to get the Azure deployment working.

And I was wrong.  Way wrong.  Deploying to Azure manually doesn’t take too long, but it adds up.  If it took me 15 minutes to remote into the server, browse to the Azure site, select the package, select the config, and yadda yadda yadda, it only takes a handful of times before you are bleeding whole hours.  If you are deploying several times per week, this can get really expensive.  Not only are you getting fewer fixes and features done, you aren’t even deploying the ones that you do have done, because you don’t have time to deploy and it’s a pain anyway.  Plus, really the only reason we wanted to deploy to the test server first was to smoke test, because deploying again was such a pain that I didn’t want to have to do a whole second deployment to fix a line of code; but if I could fix that line of code and redeploy with one click, I don’t even need to waste time with the test server.

So I didn’t want to spend the time figuring out who to deploy to Azure automatically.  Well I did, but it took me more than 5 minutes to Google it, find the right answer among the plethora of other answers, so it took a while to get done.

Hopefully you found this post in under 5 minutes of Googling so you don’t have any excuses.

Prerequisites

If you’ve been Googling around, you may have seen some posts about installing certificates.  Don’t bother.  This approach doesn’t require it, which is good, because that’s no fun.

First, go install the Windows Azure Cmdlets (http://www.windowsazure.com/en-us/manage/downloads/).  Go go go.

Second, make sure you can run remote signed scripts in PowerShell.  You only need to do this once, and if you have played around with PowerShell you’ve probably already done this.  Open up PowerShell in Administator mode (Start Button->type powershell->CTRL+Shift+Enter).  Then type:

Set-ExecutionPolicy RemoteSigned

and hit Enter.  You will get a message along the lines of “OMG Scary Scary Bad Bad Are you Sure!?!?! This is Scary!”.  Hit “Y” to continue.

Now comes the tricky part.  There is a whole bunch of PowerShell commands and certificate stuff that can get confusing.  Thankfully Scott Kirkland wrote a great blog post and even put a sample script up on GitHub.  I had to make a few tweaks to it to get it working for me, so here goes.

Fire up PowerShell again (doesn’t need to be Administrator mode any more), browse to your solution directory, and run:

Get-AzurePublishSettingsFile

This will launch a browser window, prompt you to log into your Azure account, and then prompt you to download a file named something fun like “3-Month Free Trial-1-23-2013-credentials.publishsettings”.  Take that file, move it to your solution directory, and name it something less fun like “Azure.publishsettings”.  If you open that fella up, you’ll see something like:

<?xml version="1.0" encoding="utf-8"?>
<PublishData>
  <PublishProfile
    PublishMethod="AzureServiceManagementAPI"
    Url="https://management.core.windows.net/"
    ManagementCertificate="[Redacted]">
    <Subscription
      Id="[Redacted]"
      Name="3-Month Free Trial" />
  </PublishProfile>
</PublishData>

So on to the script.  In the root of your project, create a PowerShell script (just a text file named something like DeployAzure.ps1):

#Modified and simplified version of https://www.windowsazure.com/en-us/develop/net/common-tasks/continuous-delivery/
#From: #https://gist.github.com/3694398
$subscription = "3-Month Free Trial" #this the name from your .publishsettings file
$service = "mmdbazuresample" #this is the name of the cloud service you created
$storageAccount = "mmdbazuresamplestorage" #this is the name of the storage service you created
$slot = "production" #staging or production
$package = "C:\Projects\MMDB.AzureSample\MMDB.AzureSample.Web.Azure\bin\Release\app.publish

\MMDB.AzureSample.Web.Azure.cspkg"
$configuration = "C:\Projects\MMDB.AzureSample\MMDB.AzureSample.Web.Azure\bin\Release\app.publish

\ServiceConfiguration.Cloud.cscfg"
$publishSettingsFile = "Azure.publishsettings"
$timeStampFormat = "g"
$deploymentLabel = "PowerShell Deploy to $service"
 
Write-Output "Slot: $slot"
Write-Output "Subscription: $subscription"
Write-Output "Service: $service"
Write-Output "Storage Account: $storageAccount"
Write-Output "Slot: $slot"
Write-Output "Package: $package"
Write-Output "Configuration: $configuration"

Write-Output "Running Azure Imports"
Import-Module "C:\Program Files (x86)\Microsoft SDKs\Windows Azure\PowerShell\Azure\*.psd1"
Import-AzurePublishSettingsFile $publishSettingsFile
Set-AzureSubscription -CurrentStorageAccount $storageAccount -SubscriptionName $subscription
Set-AzureService -ServiceName $service -Label $deploymentLabel
 
function Publish(){
 $deployment = Get-AzureDeployment -ServiceName $service -Slot $slot -ErrorVariable a -ErrorAction silentlycontinue 
 
 if ($a[0] -ne $null) {
    Write-Output "$(Get-Date -f $timeStampFormat) - No deployment is detected. Creating a new deployment. "
 }
 
 if ($deployment.Name -ne $null) {
    #Update deployment inplace (usually faster, cheaper, won't destroy VIP)
    Write-Output "$(Get-Date -f $timeStampFormat) - Deployment exists in $servicename.  Upgrading deployment."
    UpgradeDeployment
 } else {
    CreateNewDeployment
 }
}
 
function CreateNewDeployment()
{
    write-progress -id 3 -activity "Creating New Deployment" -Status "In progress"
    Write-Output "$(Get-Date -f $timeStampFormat) - Creating New Deployment: In progress"
 
    $opstat = New-AzureDeployment -Slot $slot -Package $package -Configuration $configuration -label $deploymentLabel -

ServiceName $service
 
    $completeDeployment = Get-AzureDeployment -ServiceName $service -Slot $slot
    $completeDeploymentID = $completeDeployment.deploymentid
 
    write-progress -id 3 -activity "Creating New Deployment" -completed -Status "Complete"
    Write-Output "$(Get-Date -f $timeStampFormat) - Creating New Deployment: Complete, Deployment ID: 

$completeDeploymentID"
}
 
function UpgradeDeployment()
{
    write-progress -id 3 -activity "Upgrading Deployment" -Status "In progress"
    Write-Output "$(Get-Date -f $timeStampFormat) - Upgrading Deployment: In progress"
 
    # perform Update-Deployment
    $setdeployment = Set-AzureDeployment -Upgrade -Slot $slot -Package $package -Configuration $configuration -label 

$deploymentLabel -ServiceName $service -Force
 
    $completeDeployment = Get-AzureDeployment -ServiceName $service -Slot $slot
    $completeDeploymentID = $completeDeployment.deploymentid
 
    write-progress -id 3 -activity "Upgrading Deployment" -completed -Status "Complete"
    Write-Output "$(Get-Date -f $timeStampFormat) - Upgrading Deployment: Complete, Deployment ID: $completeDeploymentID"
}
 
Write-Output "Create Azure Deployment"
Publish

https://github.com/mmooney/MMDB.AzureSample/tree/6177c18320f3ed35d41dc20d4f7247e1b3af05ef

Run that guy from the PowerShell command line, and you’re watch the bits fly.  Yes, it will take several minutes to run.

A generic version of this can be found here: https://gist.github.com/4539567.  Again, I borrowed from Scott Kirkland’s version, but his script assumed that your storage and cloud service were the same name, so I added a separate field for storage account name.  Also to alleviate my insanity, I added a little more diagnostic logging.

Next

This was the post that I started out to write, before I decided to backfill with the more beginner stuff.  From here, it’s going to be a little more ad-hoc.

Anyhow, the next post will probably be setting up your own DNS and SSL for your Azure site.

If I had a nickel for every time our deployment strategy for a new or different environment was to edit a few config files and then run some batch files and then edit some more config files, and then it goes down in a steaming pile of failure, I would buy a LOT of Sriracha.

(Picture http://theoatmeal.com/)

Here’s a config file.  Lets say we need to edit that connection string:

<Setting name="ConnectionString" 
value="Data Source=(local); Initial Catalog=SportsCommander; Integrated Security=true;" />

Now let’s say we are deploying to our QA server.  So after we deploy, we fire up our handy Notepad, and edit it:

<Setting name="ConnectionString" 
value="Data Source=SCQAServ; Initial Catalog=SportsCommander; Integrated Security=true;" />

OK good.  Actually not good.  The server name is SCQASrv not SCQAServ.

<Setting name="ConnectionString" 
value="Data Source=SCQASrv; Initial Catalog=SportsCommander; Integrated Security=true;" />

OK better.  But wait, integrated security works great in your local dev environment, but in QA we need to use a username and password.

<Setting name="ConnectionString" 
value="Data Source=SCQASrv; Initial Catalog=SportsCommander; UserID=qasite; Password=&SE&RW#$" />

OK cool.  Except you can’t put & in an XML file.  So we have to encode that.

<Setting name="ConnectionString" 
value="Data Source=SCQASrv; Initial Catalog=SportsCommander; UserID=qasite; Password=&amp;SE&amp;RW#$" />

And you know what?  It’s User ID, not User ID.

<Setting name="ConnectionString" 
value="Data Source=SCQASrv; Initial Catalog=SportsCommander; User ID=qasite; Password=&amp;SE&amp;RW#$" />

OK, that’s all there is too it!  Let’s do it again tomorrow.  Make sure you don’t burn you don’t burn your fingers on this blistering fast development productivity.

I know this sounds absurd, but the reality is that for a lot of people, this really is their deployment methodology.  The might have production deployments automated, but their lower environments (DEV/QA/etc) are full of manual steps.  Or better yet, they have automated their lower environments because they deploy there every day, but their production deployments is manual because they only do it once per month.

And you know know what I’ve learned, the hard and maddeningly painful way?  Manual process fails.  Consistently.  And more importantly, it can’t be avoided. 

Storytime

A common scenario you see is a developer or an operations person (but of course never both at the same time, that would ruin the blame game)  is charged with deploying an application.  After many iterations, the deployment process has been clearly defined out as 17 manual steps.  This has been done enough times that the whole process is fully documented, with a checklist, and the folks running the deployment have done it enough times that they could do it in their sleep. 

The only problem is that in the last deployment, one of the files didn’t get copied.  The time before that, the staging file was copied instead of the production file.  And the time before that, they put a typo into the config.

Is the deployer an idiot?  No, as a matter of fact, the reason that he or she was entrusted with such an important role was that he or she was the most experienced and disciplined person on the team and was intimately familiar with the workings of the entire system.

Were the instructions wrong?  Nope, if the instructions were followed to the letter.

Was the process new?  No again, the same people have been doing this for a year.

At this point, the managers are exasperated, because no matter how much effort we put into formalizing the process, no matter how much documentation and how many checklists, we’re still getting failures.  It’s hard for the mangers to not assume that the deployers are morons, and the deployers are faced with the awful reality of going into every deployment knowing that it WILL be painful, and they WILL get blamed.

Note to management: Good people don’t stick around for this kind of abuse.  Some people will put up with it.  But trust me, you don’t want those people.

The lesson

The kick in the pants is, people are human.  They make mistakes.  A LOT Of mistakes.  And when you jump down their throat on every mistake, they learn to stop making mistakes by not doing anything.

This leads us to Mooney’s Law Of Guaranteed Failure (TM):

In the software business, every manual process will suffer at least a 10% failure rate, no matter how smart the person executing the process.  No amount of documentation or formalization will truly fix this, the only resolution is automation.

So the next time Jimmy screws up the production deployment, don’t yell at him (or sneer behind his back) “how hard is it to follow the 52-step 28-page instructions!”  Just remember that it is virtually impossible.

Also, step back and look at your day to day development process.  Almost everything you do during the day besides writing code is a manual process full of failure (coding is too, but that’s what you’re actually get getting paid for).  Like:

• When you are partially checking in some changes to source control but trying to leave other changes checked out
• When you need to edit a web.config connection string every time you get latest or check in
• When you are interactively merging branches
• When you are doing any deployment that involves editing a config or running certain batch files in order or entering values into an MSI interface, or is anything more than “click the big red button”
• When you are setting up a new server and creating user or editing folder permissions or creating MSMQ queues or setting up IIS virtual directories.
• When you are copying your hours from Excel into the ridiculously fancy but still completely unusable timesheet website
• When, instead of entering your hours into a timesheet website, you are emailing them to somebody
• When you are trying to figure out which version of “FeatureRequirements_New_Latest_Latest.docx” is actually the “latest”
• When you are updating deploying database changes by trying to remember which tables you added to your local database or which scripts have or have not been run against production yet

It’s actually easier to find these things than you think.  The reason is, again, it is just about everything you do all day besides coding.  It’s all waste.  It’s all manual.  And it’s all guaranteed to fail.  Find a way to take that failure out of your hands and bath it in the white purifying light of automation.  Sure it takes time, but with a little time investment, you’ll be amazed how much time you have when you are not wasting it with amazing stupid busywork and guaranteed failure all day.