At CodeGarden09 I had mentioned during one of the open space sessions that we had developed a 2 click deployment strategy for some of our projects and someone asked if I could share the solution, so here it goes!
The tools we use to do this are:
Get Zip File here:
TheFARMDeployUtilities.zip (513.89 kb) We generally have 3 different environments for our websites: Development, Staging, Live (Production). For 2 click deployments to work, we then need to have CruiseControl.Net server installed on each server environment. Your development machine or the machine that is performing the 2 click deploy should have access to all of the CruiseControl.Net servers so that each project can be added to your CCTray application. We lock down our CruiseControl.Net servers via firewall and running them on non-standard ports to only allowing traffic to them from our office environment.
This strategy is 2 steps because:
- Gives you the control over when the project is built
- Gives you control over when the built files are deployed live
- Though it could be done in 1 step, I feel that it’s safer to ensure that the project is built properly before it goes live
This deployment process also takes into account:
- A rollback process which ensures any build that is deployed can be rolled back to a previous build very quickly
- The problem of multiple application restarts (or long application restarts) when uploading files via FTP since each DLL is essentially copied up one at a time, this process ensures that all built files are deployed to the environment without delay.
Basic Workflow
In order to understand the next steps its best to get a brief overview of what steps are involved when the deployment occurs:
- All code is committed to the repository (to a branch, or wherever the stable code should be) that you wish to deploy
Click 1
- A CruiseControl ‘Force Build’ is initiated for the project (normally via CCTray)
- CruiseControl checks out the latest version from Subversion and puts it in it’s working folder for the project
- CruiseControl runs a NANT task which:
- Creates a build date time stamp (i.e. 20090710AM)
- Runs an MSBuild process for each deployment environment (dev/stage/live) to build and deploy the web application. Depending on the environment, it will build in Release vs. Debug mode.
- Formats configuration files to include only the relevant settings for each environment
- Zips the builds for each deployment environment named with the build date time stamp
- FTPs the Zip file to a deploy folder on the deployment servers (stage/live)
Click 2
- A CruiseControl ‘ForceBuild’ is initiated on the environment’s server you wish to deploy to (normally via CCTray)
- CruiseControl runs a NANT task which:
- Unzips the latest build files into the staging/live folders for IIS.
CruiseControl Server Setup
The CruiseControl (build) server requires a few bits of software installed and configured:
The CruiseControl servers on staging/live environments only require that CruiseControl & NANT is installed.
Folder Structure
On the CruiseControl build server, we create this folder structure for each project:
- Artifact
- Debug
- buildlogs (stores the CruiseControl build logs for the project)
- Deployment
- Dev (the folder for the Dev environment builds)
- Live (the folder for the Live environment builds)
- Staging (the folder for the Staging environment builds)
- Working (where all of the source control files are checked out to)
On the staging/live servers we have a folder in the root of the website labelled: “_DEPLOY” which is where the deployment zip files get FTP’d to. We ensure that this folder doesn’t have read access so public people can’t go downloading your build files! In this _DEPLOY folder we have a deploy.build NANT task (see below).
Configuration File Formatting
Everyone seems to have their own way of managing different configuration settings for different environments and so do we! I like to be able to see all of the configuration settings for each environment in one file so that i know what’s being defined without looking through different files. To do this, our configuration settings look something like this:
<!--<DEV>-->
<add key="mySetting" value="DevEnvironmentSettingValue" />
<!--</DEV>-->
<!--<STAGING>
<add key="mySetting" value="StagingEnvironmentSettingValue" />
</STAGING>-->
<!--<LIVE>
<add key="mySetting" value="LiveEnvironmentSettingValue" />
</LIVE>-->
With the above design, development settings are always active when developing. When CruiseControl runs it’s build one of the NANT tasks that are run is The FARM’s configuration settings parser plugin which removes any configuration settings in config files that are not required for the current build environment.
We’ve also developed a handy little Visual Studio plugin to make formatting these configuration settings very easy. All that is required is highlighting a configuration element, clicking Tools –> The Farm Config –> Format for Dev (or staging/live)
CruiseControl.Net Setup
Deployment/Build Server
I’ve put documentation about each item directly in the XML:
<project name="MYPROJECT" category="WEB" queue="MYPROJECT" queuePriority="1" >
<!-- The working folder, this is where our repository files will be checked out to -->
<workingDirectory>D:\CruiseControl\Projects\MYPROJECT\Working</workingDirectory>
<!-- This is where our log files, and debug info will be saved -->
<artifactDirectory>D:\CruiseControl\Projects\MYPROJECT\Artifact\Debug</artifactDirectory>
<sourcecontrol type="svn">
<!-- We use a file based Subversion repository... here's an example of the location -->
<trunkUrl>file:///\myserver.mydomain.local/MYPROJECT/DEV_Repository/trunk/</trunkUrl>
<!-- Our Subversion install is here -->
<executable>C:\Program Files\CollabNet Subversion\svn.exe</executable>
<autoGetSource>True</autoGetSource>
</sourcecontrol>
<!-- We always do Force Builds, instead of automatic builds based on source control so our
server doesn't die when people commit multiple times for different projects, therefore
triggers is set to nothing -->
<triggers/>
<tasks>
<nant>
<!-- We use NANT to do all of our build. NANT in turn runs MSBuild and other
methods that we require such as FTP, etc... -->
<executable>C:\Program Files\Nant\bin\nant.exe</executable>
<baseDirectory>D:\CruiseControl\Projects\MYPROJECT\Working</baseDirectory>
<!-- This is the location of the NANT build file... it is in our working folder
which means that it is committed into the repository so that our devs are free
to modify it to suit their needs -->
<buildFile>D:\CruiseControl\Projects\MYPROJECT\Working\cc.build</buildFile>
<buildTimeoutSeconds>3600</buildTimeoutSeconds>
</nant>
</tasks>
<publishers>
<xmllogger />
<!-- This ensures that our log files don't build up too big -->
<artifactcleanup cleanUpMethod="KeepLastXBuilds" cleanUpValue="50" />
</publishers>
</project>
Staging/Live Servers
For each site, just define another project which simply runs a NANT script which basically just unzips the zip build file into the working IIS site folder (see below)
<project name="_STAGING_MYSITE">
<workingDirectory>E:\Inetpub\MYSITE</workingDirectory>
<artifactDirectory>E:\Inetpub\MYSITE\_DEPLOY\Debug</artifactDirectory>
<tasks>
<nant>
<executable>C:\Nant\bin\nant.exe</executable>
<baseDirectory>E:\Inetpub\MYSITE</baseDirectory>
<buildFile>E:\Inetpub\MYSITE\_DEPLOY\deploy.build</buildFile>
</nant>
</tasks>
<publishers>
<xmllogger />
<artifactcleanup cleanUpMethod="KeepLastXBuilds" cleanUpValue="50" />
</publishers>
</project>
NANT Build File
Deployment/Build Server
I’ve put documentation about each item directly in the XML:
<!-- Ensure NO namespace is declared as FTPTask for NANT
has a bug when namespaces are declared -->
<project name="Site" default="DeployAll">
<!-- ** PROPERTY DEFINITIONS, These are the only things you should need to change -->
<!-- define the folder containing the web application project -->
<property name="location.working.web" value="${CCNetWorkingDirectory}\_net\MySite.Web"/>
<!-- define the web applicatoin project project file -->
<property name="project.web" value="${location.working.web}\MySite.Web.csproj"/>
<!-- define the deployment folders for each environment -->
<property name="location.deploy.dev" value="${CCNetWorkingDirectory}\..\Deployment\Dev"/>
<property name="location.deploy.staging" value="${CCNetWorkingDirectory}\..\Deployment\Staging"/>
<property name="location.deploy.live" value="${CCNetWorkingDirectory}\..\Deployment\Live"/>
<!-- define the MSBuild executable -->
<property name="exe.msconfig" value="C:\WINDOWS\Microsoft.NET\Framework\v3.5\MSBuild.exe"/>
<!-- define the ftp property connections for the stage/live environments -->
<property name="ftp.staging.server" value="123.123.123.123"/>
<property name="ftp.staging.user" value="admin"/>
<property name="ftp.staging.password" value="hello"/>
<property name="ftp.staging.path" value="MYSITE/FILES/_DEPLOY"/>
<property name="ftp.live.server" value="321.321.321.321"/>
<property name="ftp.live.user" value="admin"/>
<property name="ftp.live.password" value="hello"/>
<property name="ftp.live.path" value="MYSITE/FILES/_DEPLOY"/>
<!-- ** END PROPERTY DEFINITIONS -->
<!-- the primary build target which does the entire build -->
<target name="DeployAll" depends="SetBuildDate,Init,DevSite,StagingSite,LiveSite,CopyStaticFiles" />
<!-- sets some GLOBAL properties for use on all builds -->
<target name="Init" description="Sets the global working and project properties">
<property name="GLOBAL.location.working" value="${location.working.web}"/>
<property name="GLOBAL.project" value="${project.web}"/>
</target>
<!-- set a build date property to today's date with am/pm stored in ${build.date}-->
<target name="SetBuildDate" description="Creates a build date property for use in our deployment scripts">
<tstamp property="build.date" pattern="yyyyMMddtt" verbose="true" />
<echo message="Current build label: ${build.date}" level="Debug" />
</target>
<!-- Deploy Dev with Debug, this doesn't time stamp or zip the output
since it's a dev environment, the location should be set to your
IIS folder -->
<target name="DevSite" description="Compile Dev Web MySite.">
<property name="GLOBAL.location.deploy" value="${location.deploy.dev}"/>
<!-- build project in Debug mode -->
<property name="GLOBAL.project.config" value="Debug"/>
<property name="GLOBAL.deploy.type" value="DEV"/>
<!-- build the web application (see below) -->
<call target="BuildWebProject" />
</target>
<!-- Deploy Staging with Debug -->
<target name="StagingSite" description="Compile Staging Web MySite.">
<!-- build the project to the time stamped folder -->
<property name="GLOBAL.location.deploy" value="${location.deploy.staging}\${build.date}"/>
<!-- build project in Debug mode -->
<property name="GLOBAL.project.config" value="Debug"/>
<!-- define that the STAGING settings should be active on deploy -->
<property name="GLOBAL.deploy.type" value="STAGING"/>
<!-- build the web application (see below) -->
<call target="BuildWebProject" />
<!-- format the configuration files for the project to have the correct
environments settings active -->
<call target="FormatFileForDeploy" />
<!-- zip the build -->
<call target="ZipDeployFiles" />
<!-- transfer the build to the staging environment's deploy folder -->
<connection id="myconn" server="${ftp.staging.server}" username="${ftp.staging.user}" password="${ftp.staging.password}" />
<ftp connection="myconn" verbose="true" showdironconnect="true" remotedir="${ftp.staging.path}">
<put localdir="${location.deploy.staging}" type="bin">
<include name="${build.date}.zip" />
</put>
</ftp>
</target>
<!-- Deploy Live with Release -->
<target name="LiveSite" description="Compile Live Web MySite.">
<!-- build the project to the time stamped folder -->
<property name="GLOBAL.location.deploy" value="${location.deploy.live}\${build.date}"/>
<!-- build project in RELEASE mode -->
<property name="GLOBAL.project.config" value="Release"/>
<!-- define that the LIVE settings should be active on deploy -->
<property name="GLOBAL.deploy.type" value="LIVE"/>
<!-- build the web application (see below) -->
<call target="BuildWebProject" />
<!-- format the configuration files for the project to have the correct
environments settings active -->
<call target="FormatFileForDeploy" />
<!-- zip the build -->
<call target="ZipDeployFiles" />
<!-- transfer the build to the live environment's deploy folder -->
<connection id="myconn" server="${ftp.live.server}" username="${ftp.live.user}" password="${ftp.live.password}" />
<ftp connection="myconn" verbose="true" showdironconnect="true" remotedir="${ftp.live.path}">
<put localdir="${location.deploy.live}" type="bin">
<include name="${build.date}.zip" />
</put>
</ftp>
</target>
<!-- ** BUILD METHODS-->
<!-- Global method to build web applicatio projects using MSBuild -->
<target name="BuildWebProject">
<exec basedir="${GLOBAL.location.working}" program="${exe.msconfig}" workingdir="." failonerror="true">
<arg value="${GLOBAL.project}" />
<arg value="/nologo" />
<arg value="/t:Clean;Build" />
<arg value="/t:ResolveReferences;_CopyWebApplication"/>
<arg value="/p:OutDir=${GLOBAL.location.deploy}\bin\;Configuration=${GLOBAL.project.config}"/>
<arg value="/p:WebProjectOutputDir=${GLOBAL.location.deploy}"/>
</exec>
</target>
<!-- This formats any files that are included in our include property -->
<!-- to be in the correct format for dev,staging,live -->
<target name="FormatFileForDeploy">
<thefarm-process-config configtype="${GLOBAL.deploy.type}">
<fileset basedir="${GLOBAL.location.deploy}">
<include name="**/Regions.xml" />
<include name="**/EmailFormDefinitions.xml" />
<include name="**/*.config" />
</fileset>
</thefarm-process-config>
</target>
<!-- Zips all files in the current deployment folder -->
<target name="ZipDeployFiles">
<zip zipfile="${GLOBAL.location.deploy}.zip">
<fileset basedir="${GLOBAL.location.deploy}">
<include name="**/*" />
</fileset>
</zip>
<delete dir="${GLOBAL.location.deploy}" />
</target>
<!-- ** END BUILD METHODS-->
</project>
Staging/Live Servers
<project name="MyStagingSite" default="DoDeploy">
<target name="DoDeploy" depends="SetBuildDate,UnzipDeployFiles" />
<!-- set a build date property to today's date with am/pm-->
<target name="SetBuildDate">
<tstamp property="build.date" pattern="yyyyMMddtt" verbose="true" />
<echo message="Current build label: ${build.date}" level="Debug" />
</target>
<!-- unzip the build file into the IIS working folder -->
<target name="UnzipDeployFiles">
<unzip zipfile="${CCNetWorkingDirectory}\_DEPLOY\${build.date}.zip" todir="${CCNetWorkingDirectory}" />
</target>
</project>
Rollback
Since each build is contained in a date/time stamped ZIP file it is obviously fairly easy to rollback your site to a previous build, you would just have to know what date/time the last stable build was.
Conclusion
Though the above seems like a lot of work, it really isn’t since all the work is already done for you! All you really need to do once you have your CruiseControl servers setup is create your projects, modify the NANT build script properties for each project and that’s it.
This makes deployment extremely easy and safe since there’s no room for human error. All of the files will go exactly where they need to go. Developers can’t upload the wrong configuration files to the wrong servers or the wrong files to the wrong places. Another great benefit is that when the server unzips the build, the .Net application restarts quickly because it has all of the files to start compiling instantly.
Obviously to do this, you would need access to your staging and development servers, but even if you didn’t you could still get the first step done (which is the major step). The NANT scripts above are quite basic, we use them to build Sandcastle .Net documentation and all sorts of other things.
Always great to know how everyone else is doing the things described in this doc, so please let me know!
Subscribe