Today I started a new gig in my old company Dell.  I am joining the edell transactional team as one of the architects.  I am experiencing some of the slowness that comes with a big company like this.  However I am really enjoying the conversations I have been a part of so far and am really enjoying all the energetic conversations that the engineers around me are having.

I am very happy I chose to stay with Dell.  It looks like this could be more fun than most people think Dell is.  More to come soon.

I have to go brush up on nservicebus, angular, bootstrap, gridsetapp, and sherpa.

I am trying my best to get back into blogging.  After having moved to Texas and starting my own ranch/farm…I have had difficulties staying focused on technology when I am not work.  For that reason I will try to do quick posts while at work…while I am focused on technology.

To start this quick story I have to first mention that we have released several iterations of our product “Marketvine Picks” where Marketvine is the company we are spinning out of Dell as and Picks is the social commerce-y product we have created.  Currently you can see this product on www.citysports.com, www.eyeslipsface.com, and as of last night on www.surlatable.com.

If you aren’t close to me and my work world you would be interested in knowing that our application is running on .net and javascript in the Amazon cloud.  Thank god for the Amazon cloud!  With each new release we take a substantial amount of additional traffic.  With each new set of traffic we encounter interesting scaling issues.  We have done the normal query tuning and coding optimizations and have enjoyed the dynamic scaling abilities of our infrastructure that AWS gives us.  But in preparation for future releases where we might anticipate some bottle necks at the database we have preemptively decided to get ready for separating our reads and writes.

Using read replicas from an infrastructure point of view is not always easy.  However, we are in AWS…so this feature is as easy as adding a new read replica.  Right click on an RDS instance and add a read replica.  Quick modal configuration.  GO…minutes later we have a read replica.

Depending on how you chose to structure your application you may or may not be ready to utilize this easy to use feature for horizontal scaling.  Going wide with your database if you will…rather than standing up a more powerful database server.  You can have up to 5 read replicas per each master instance.  NICE!

In our case we made an decision early on to at least contain our read queries and our write queries in separate containers.  We pretty successfully conform to the concept of command query separation (CQS).  This allowed us to easily tackle the concept of a read connection string and a write connection string later on.

Well…later on is here now!

We now have two separate connection handlers 1) MySQLReadonlyDatabaseFactory and 2) MySQLReadWriteDatabaseFactory.  But how to get this new concept into our code?  Thank god for IoC (Ninject in our case).

We have marker interfaces on all the classes that do reads vs. writes.  If the class does solely reads they are marked with IQueries.  If they do solely writes then they are marked with ICommander.  This has then allowed us to swap in our new functionality with just a few lines of IoC configuration code:

   1:  _kernel.Bind<IDatabaseFactory>().To<MySQLReadonlyDatabaseFactory>()

   2:      .WhenInjectedInto<IQueries>();

   3:  _kernel.Bind<IDatabaseFactory>().To<MySQLReadWriteDatabaseFactory>()

   4:      .WhenInjectedInto<ICommander>();

   5:  _kernel.Bind<IDatabaseFactory>().To<MySQLReadWriteDatabaseFactory>();

This is a repost from here in case I need it again: http://www.codeproject.com/Articles/539088/Amazon-AWSDeploy-To-Provision-Multiple-Websites

Introduction

Recently I've been on a project where we are implementing BlueGreen deployments using Amazon CloudFormation, and came across a number of unanswered questions:

• How do you deploy multiple applications to a single instance and support auto-scaling?
• How do you deploy non-website applications using AWSDeploy?
• How can you pass in environmental config during provisioning?

The official answer to the first 2 questions appears to be that you can't without additional scripts/tools. The third (with some caveats) is that you need to bake them into your package prior to your deployment.

This seems frustrating for such common real world use cases, but luckily there is a solution which requires minimal effort.

First Some Background

CloudFormation is Amazon's offering for provisioning a collection of AWS resources on the amazon infrastructure through declarative templates. The collection of resources is known as a stack and can be used for reliable and repeatable infrastructure deployments.

AWSDeploy is the tool which can be used with the Microsoft stack to provision CloudFormation templates and to deploy web applications onto these provisioned resources. This uses Microsoft WebDeploy under the hood, and as such a WebDeploy package must be provided when provisioning a CloudFormation stack using the tool.

The WebDeploy package provided by the user is uploaded to AWS by the tool and stored in an S3 bucket. When the EC2 web server spins up, AWSDeploy installs the package on the web server as part of initialising the stack. By default any failures will result in a stack rollback.

Once a web application is provisioned via AWSDeploy, the stack will manage failover and auto-scaling scenarios automatically by redeploying the associated WebDepoy package stored on S3 to any new instances. In doing so it is able to manage resources hands-free and replicate perfect copies of your application on demand.

With Great Power Comes Great Limitations

As described above, AWSDeploy is incredibly useful for failover and scaling scenarios. Your web application can be replicated and auto-provisioned without manual interaction.

The downside is a number of limitations:

• Only 1 WebDeploy package can be associated with a web server instance
• No parameters can be passed when AWSDeploy installs the WebDeploy package
• As such WebDeploy parameterisation cannot be used for config transforms
• Environmental config is limited to a small set of pre-defined appSettings keys, which can be injected during provisioning

The impact is that to work with AWSDeploy you generally have to modify your build artifact for every deployment environment and bake in configuration settings. This is against Continuous Delivery principles and adds complexity to any deployment scripts.

An Idea

WebDeploy in it's own right is an extremely flexible and versatile tool. In a single WebDeploy package it's possible to deploy multiple websites, deploy directories of arbitrary files, execute commands, and change Acl permissions.

So, instead of baking environment configuration into the WebDeploy packages to support AWSDeploy, why not instead create a wrapper WebDeploy package and use an embedded RunCommand to pass environment variables to a child package during provisioning. In this way the original build artifact is unmodified and the full power of WebDeploy can be used to inject parameters during installation.

This approach has additional benefits in that it allows the install of multiple web applications via a single AWSDeploy package, as well as the ability to deploy Windows Services and Console applications during auto-provisioning. Additionally it's then relatively easy to cheery-pick applications for deployment to a particular instance with very little rework of deployment scripts.

The Problem

On creating my first wrapper package I immediately ran into a problem when provisioning. The AWSDeploy logs revealed the issue.

Collapse | Copy Code

ERROR 10 AWSDeploymentHostManager.Tasks.UpdateAppVersionTask - Deploy failed, not iisApp.

It seems AWSDeploy expects the WebDeploy package to contain an iisApp within it's manifest. So I included a stub website in my WebDeploy package and tried again. Surprisingly I hit the same error!

After much investigation it turns out that when AWSDeploy executes the WebDeploy package, it passes the following parameter to the package. This is to enable customisation of the target IIS Website name during provisioning, and can be set via a AWSDeploy configuration setting.

Collapse | Copy Code

-setParam:"IIS Web Application Name"="Default Web Site/"

The Solution

Knowing the parameter requirement, the solution is simple. Provided the WebDeploy package contains a parameters file with this parameter defined within, then AWSDeploy will successfully deploy the package regardless of it's contents. It also means that the WebDeploy package does not actually need to include an iisApp instance as the error first suggested.

Putting It All Together

First place any WebDeploy packages to be installed on the server instance in a folder called C:\Packages. Include any set-parameter configuration files for any environmental settings and config transforms.

Create a PowerShell install script called Install.ps1 and add this to the C:\Packages directory. The script will be executed on the server during provisioning. It will be used to deploy the WebDeploy packages in this folder on the server, and can be used to invoke explicit WebDeploy commands during install; such as to leverage environment specific parameterisation.

Collapse | Copy Code

Set-Alias msdeploy "C:\Program Files\IIS\Microsoft Web Deploy V3\msdpeloy.exe"

msdeploy -verb:sync -source:package=WebsiteA.zip -dest:auto -setParamFile:WebSiteA-Params.xml
msdeploy -verb:sync -source:package=WebsiteB.zip -dest:auto -setParamFile:WebSiteB-Params.xml

Next create the parameters file for inclusion in the WebDeploy wrapper package as follows. Name the fileparameters.xml and place it in a folder called C:\Build.

Collapse | Copy Code

<parameters>
 <parameter name="IIS Web Application Name" defaultValue="Default Web Site/" tags="IisApp" />
</parameters>

Now create a WebDeploy manifest file for the wrapper package as follows. This will provision the contents ofC:\Packages to the server instance and use the RunCommand provider to execute the Install.ps1 PowerShell script in the root of this directory. Name the file manifest.xml and place it in the C:\Build folder.

Collapse | Copy Code

<siteManifest>
  <contentPath path="C:\Packages" />
  <runCommand path="%SystemRoot%\System32\WindowsPowerShell\v1.0\PowerShell.exe
       -inputformat none -command &quot;Set-ExecutionPolicy Unrestricted -force&quot;" 
       waitInterval="10000" />
  <runCommand path="%SystemRoot%\System32\WindowsPowerShell\v1.0\PowerShell.exe 
       -inputformat none -command C:\Packages\Install.ps1" 
       waitInterval="300000" />
</siteManifest>

Finally package up the WebDeploy wrapper package using the command line as follows, which should be executed from the C:\Build folder. The trick is the declareParamFile parameter and parameters.xml file which keeps AWSDeploy happy.

Collapse | Copy Code

msdeploy -verb:sync -source:manifest=manifest.xml -dest:package=Deploy.zip
         -declareParamFile=parameters.xml

The WebDeploy package produced is now ready to be used with the AWSDeploy tool to provision 1 or more applications on your EC2 server. It is also fully compatible with auto-scaling and failover scenarios.

You can test the install locally by executing the following command in the C:\Build directory.

Collapse | Copy Code

msdeploy -verb:sync -source:package=Deploy.zip -dest:auto

Further Enhancements

The solution minimises the customisation of deployment scripts for different environments, and results in a solution much less coupled to Amazon AWSDeploy. It also works around the limitations of AWSDeploy and provides a much greater level of flexibility in how instances can be configured to support auto-provisioning.

This is an improvement, but we are still having to build different artifacts per environment which is not ideal. A further enhancement would be to leverage AWS UserData from within the PowerShell script. UserData is configured during provisioning and made available to each server instance via a web service at a fixed address:

Collapse | Copy Code

http://169.254.169.254/latest/user-data

UserData is persisted during failover & scaling, and could be used to either feed the PowerShell script entire set-parameter files dynamically, or provide flags to indicate to the PowerShell script which pre-deployed set-parameter file to use per environment when installing packages. In this way a single artifact could be used across all environments inline with best practice, as well as a single install and deployment script.

I am loving the Amazon Web Services platform!  They deliver new features constantly to make the platform that much more adoptable.  We are to a point now where we have all sorts of disconnected systems floating about in the cloud.  All of them have their own scalability story so that they can easily expand horizontally.  This translates to having many stacks all with many running instances.

We have provided a mechanism so that these systems have three layers of configuration. 

1. The local configuration that is deployed as part of the build output.  This is usually config data that doesn’t change as the application moves from one environment to the next (localhost, dev, qa, staging, sandbox, prod).  These are all the keys, usually stored in app.config or web.config, that each app is dependent upon.
2. Configuration data that is generated by the stack itself which is written to the local disk as an instance is built by AWS mechanisms. This allows us to express instance specific or AWS environmental data to the running application inside of a given instance.  We also use this as a mechanism to strip out certain data that might be pushed out as part of the build process.  A specific example of this would be when a developer might store their AWS tokens in the local config.  We use the generated config to remove those keys so that we can use security groups in the AWS environments directly instead of dev credentials.
3. Configuration data stored in a centralized location that each app uses to overlay its .config key/values at run time.  When the application starts up it makes a call to the centralized location and overlays that config data over the data it had in its local config file and the generated config file.  This is the config that we use to wire up a specific environment with the data that it might need as the applciation moves from QA to Stage and on out to Prod.

As we add more and more to our configuration story, and we add more and more new separately deployable applications across many different stacks, managing configuration has become increasingly more complex.  This one specific area in our deployment story for example has turned into the biggest possibility of wrong doing in our applications deployment story.

As I tinker with ways to simplify and manage this we have talked about a few different notions.

1. Creating configuration status pages in both html and json so that we can easily see what an application is running at any given time.  This allows us to crawl all of our end points and do interesting things like compare from one instance to the next, from one environment to the next (QA vs. Stage vs. Prod for example).
2. Create the notion of command and control using SNS where by each instance registers itself to an SNS topic for its given module and environment.  This then allows us to tell all instances in a given environment to go and update their configuration from the centralized config.  Or we can test single instances by telling them directly to get new config. Or we can tell an entire module within our world to get latest on their config. This of course requires that the end point subscribed to the topic is listening and receiving SNS calls.
3. Lastly just have the apps reload their own config every 30 seconds.  I don’t personally like this in that someone could write bad config half way through their work and it would get swallowed right away…whether you meant for it to happen or not.  This is appealing though in that it is by far the easiest to implement.

I will come back to this topic as I explore more.

This is going to be a very busy weekend for me! 

Saturday I am hosting CrossFit Strive (Bastrop) at my house again to see if we can bring CrossFit to Elgin Texas.  That starts at 8am.  I am then rushing off to the VA Memorial Park in Elgin Texas to set up an Obstacle Course for the Relay for Life kick off event at 10am.  That event will take up most of Saturday. 

Then Sunday I am up at 6am and driving down to New Braunfels Texas to pick up my (roughly 30 mile) leg of the Ruck Across America that GoRuck is putting on.  That will go from roughly 8am till 4pm.  Sunday gone.

Busy weekend!

This isn’t overly interesting but I feel the need to toss up a post on my new site.  Today I needed to create a hash and ended up writing the following code.

private string ComputeHash(string valueToHash)
{
    SHA256 sha = SHA256Managed.Create();

    //convert value to byte array
    byte[] bytes = new byte[valueToHash.Length * sizeof(char)];
    System.Buffer.BlockCopy(valueToHash.ToCharArray(), 0, bytes, 0, bytes.Length);

    //hash the bytes
    byte[] computedSha = sha.ComputeHash(bytes);

    return Convert.ToBase64String(computedSha);
}

This handily takes an input string and hands back a simple hash!