ARCHIVE NOTICE

My website can still be found at industrialcuriosity.com, but I have not been posting on this blog as I've been primarily focused on therightstuff.medium.com - please head over there and take a look!
Showing posts with label deployment. Show all posts
Showing posts with label deployment. Show all posts

Saturday, 8 January 2022

Managing site certificates with NGINX and Certbot

And removing a single domain certificate without breaking everything else

Do you operate multiple domains from the same webserver? Do you have a webserver operated by NGINX? Do you have Certbot managing your certificates? This is a set of instructions for creating your certificates correctly and removing a single domain from your configuration, after I found some confusing ones that resulted in me knocking out my server for a little while…

A note before we begin

If you’re rather in a hurry to remove a domain from a messy configuration, STOP. Re-organizing your sites and regenerating your certificates is not only pretty quick and mostly painless — and required, if you want to remove a single domain without making NGINX break down and throw a wobbly — it’s very much the same process.

Organizing your existing NGINX sites

Ensure that you know which domains are configured in which site files, in particular make sure that you do not include servers for multiple domains in the same file.

To do this, look through your enabled site files under /etc/nginx/sites-enabled to find relevant server entries. While you’re there, you might want to note any certificates which are already used by those server entries; those will be the lines starting with ssl_certificate.

If you need to reorganize your site files, remember that their actual location must be in the /etc/nginx/sites-available path. To enable a site /etc/nginx/sites-available/example, create a symlink in the /etc/nginx/sites-enabled path with

> ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/

and to disable a site, remove it from the /etc/nginx/sites-enabled path with

> rm /etc/nginx/sites-enabled/example.com

Generating certificates with Certbot

Once your sites are organized in a way that each domain has its own file, generate certificates for each domain and its subdomains with

> sudo certbot --nginx -d example.com -d www.example.com

This will generate a new certificate if needed and update the site file accordingly.

To ensure that everything is as it should be, review the updated site files and then validate them with

> sudo nginx -t

To restart NGINX once you’re ready, run

> sudo service nginx restart

Removing obsolete domains and certificates

Now that your site files and certificates are configured correctly, it’s time to remove any obsolete certificates that are no longer referenced.

Run sudo certbot certificates to list the existing certificates, paying attention to their names as well as their certificate and key paths. These paths will be registered in your NGINX site files so you can review what’s active and required and be certain that the certificate(s) you’re removing are unused.

When you’re confident that a certificate example.com is no longer in use, simply remove it by running

> sudo certbot delete --cert-name example.com 

...

Originally published at https://therightstuff.medium.com.

Friday, 6 August 2021

Bamboo YAML Specs Tips and Tricks (For Fun and Profit)

A cute panda eating bamboo
[UPDATED 2021.10.01 WITH ADDITIONAL DEPLOYMENT PLAN LEARNINGS]

Introduction

“In for a penny, in for a pound” — that’s our team’s current approach regarding Atlassian’s products. As part of our efforts to code as much of our infrastructure as possible, we’ve recently begun migrating old build plans to Bamboo Specs and, as non-Java devs who don’t want more headaches than are absolutely necessary, we’ve chosen to work with the Bamboo Specs YAML offering.

YAML specs are pretty great. Migrations are assisted by the fact that manually-created plans will show you their YAML translations, so in most cases a simple copy/paste is all you need. I mean, aside from migrating to linked repositories and ensuring they’re configured correctly…

Having said that, Bamboo’s YAML specs are an incomplete product with undocumented critical features, and fail to provide out-of-the-box support for a surprising number of standard use cases that one would expect software engineers building software for other software engineers to appreciate the value of. 
They’re still pretty great in spite of that, but overcoming the limitations of incomplete specs definitions and deployment plans is not exactly intuitive. This article attempts to cover some of the missing pieces and suggest some workarounds that we’ve found useful.


Configuring a linked repository for Bamboo Specs

Bamboo Specs require the configuration of a linked repository. Head to settings -> linked repositories (this will require admin permissions) to create or update a linked repository configuration. The two most important steps to be taken here are as follows:
  1. Determine which branch of your repository will be used by Bamboo to read the specs file. Only one branch can be considered the source of truth, my personal recommendation is to make it the development branch.

  2. Two sets of permissions need to be configured correctly in order for Bamboo Specs to be able to do their job:

    a. First, the Bamboo project must give permissions to the linked repository to create and modify the build and deployment plans. Head to the project you want your linked repository to operate in, go to Project Settings, then Bamboo Specs repositories, and add the linked repository.

    b. Under the Bamboo Specs tab of the linked repository, enable both Access all projects and Access all repositories. I’m pretty confident that this requirement is not a security best-practice, but in my experience Bamboo Specs won’t work without them.

Testing Bamboo Specs

When experimenting with changes to the specs, I’ve found it useful to do the following:
  1. Create a testing branch (a feature / bugfix branch).

  2. Change the build and deployment plans’ names and keys and push them to origin so that your changes won’t overwrite the existing plans.

  3. Set the testing branch to be the linked repository’s branch in the General tab and proceed with your changes.

  4. Once you’re happy with the changes and they’ve been reviewed: set the linked repository’s branch back to the development branch, then revert the plan names and keys in the testing branch to the existing plans’.

  5. Delete the test plans manually.

Unexpected Limitations of Bamboo Specs

Plan dependencies

The most glaring omission in the YAML specs is the inability to set up plan dependencies. While we were initially upset by this, we quickly realized that at the end of the day plan dependencies are a nice hack that we shouldn’t really have been relying on in the first place. Bamboo appears to encourage the download of artifacts from matching branches on other build plans, but this can quickly break down into chaos with dependency versioning and management. I warmly recommend using Bamboo artifacts for build debugging and deployment plans exclusively, and proper package repositories for storing and retrieving versioned build artifacts.

When exporting Bamboo Specs from existing plans, build plans and deployment plans are not considered strongly related so you will need to gather and combine the specs from both into a single file. To do this, copy in the deployment plan specs beneath the build plan specs, retaining the --- as separators.
NOTE: the ordering of the sections is important! The build plan definition must be followed by the build plan permissions, then the deployment plan definition(s), with each deployment plan definition followed by a section for its permissions. See the outline at the end of the article for clarity.
An interesting omission is the inability to include unset plan variables. This actually makes sense, as manually maintained plans need some way to ensure that they’re all using the same variable names, but with Bamboo Specs it’s really on you to be consistent and it’s obviously much* easier to search through a single file than it is to hunt for variables across different plan branches via the Bamboo interface.

* infinitely easier

Deployment Plans — sharing build variables and tooling

The principal idea behind a deployment plan is to separate deployment from the build process. Bamboo implements deployment plans as distinct entities with entirely different environments, with the intention that your only interaction with the related build plan is to download your artifacts from it.

For us, this proved problematic as we require shared environment variables and tooling to deploy our builds. To work around this, we required the following mechanisms:
  1. Environment variable injection. Early in our build plan tasks, we prepare an environment variable file in the following format and include values like build versions and git branch names.

    version=1.0.2
    git_branch=feature/example


    WARNING: variable values MUST NOT be surrounded by quotations, as this leads to unpredictable behaviour.


    It is recommended to use the inject namespace for injections. When the scope of the injection is RESULT, the variables will be available to all subsequent build tasks as well as the attached deployment plan. In Bamboo Specs they’ll be available in the form ${bamboo.inject.git_branch} and in inline scripts as $bamboo_inject_git_branch (on Unix agents) or %bamboo_inject_git_branch% (on Windows agents).

    One of my favourite uses of this technique is the ability to name releases automatically based on the build version (see the following section for the example).

  2. Deployment plans are not really designed to use git directly, but we have found that we sometimes require non-build folders to be available for deployment, such as documentation. In these cases, we simply zip the desired folders and make them available as artifacts as well.

  3. Running the deployment in a docker container. I find it disconcerting that such an extremely useful feature is undocumented! Deployment plan environments can be configured to run in a docker container just like a build plan, which provides us with all requisite tooling and context.

    DevEnvironment:
      docker:
        image: golang:1.16.6-buster
        docker-run-arguments:
        - --net=host
      tasks:

Linking multiple branches to a single deployment plan

Ironically, while deployment plans are supposed to operate independently from build plans, they only really function well when linked to specific build branches. If your intention is to build once, then deploy the build artifacts to multiple stages, you're out of luck!

UPDATE: it turns out we missed an important option! If release-naming is set to an environment variable, it only works for the specified branch (even if that specified branch is the default branch). If you want release-naming to be set to an environment variable for any branch, then it needs to be configured as follows:

release-naming:
  next-version-name: ${bamboo.inject.version}
  applies-to-branches: true

The disadvantage of using a single deployment plan is that the link to the deployment plan will only be available from the default build plan branch, but in my experience this is a very minor price to pay for the simplicity. The alternative - a single deployment plan for each branch of interest - is not only messier, but is also annoying to configure as you have to know the branch keys in advance so the branches cannot be automatically managed (plan branch keys are autoincremented and uneditable).

Regardless of your choice, it's probably a good idea to handle your branch management manually:

branches:
  create: manually
  delete: never

Putting it all together

My recommendation for the general outline of a YAML specs file is as follows:

---

version: 2

# build plan definition

plan:

  project-key: PROJECTKEY

  key: PLANKEY

  name: Product Name

  ...

branches:

  create: manually

  delete: never

---

version: 2

# build plan permissions

plan:

  key: PROJECTKEY-PLANKEY

plan-permissions:

  ...

---

version: 2

# deployment plan definition

deployment:

  # NOTE: deployment plan names must be unique

  name: Product Name

  source-plan: PROJECTKEY-PLANKEY

release-naming:
  next-version-name: ${bamboo.inject.version}
  applies-to-branches: true

...

---

version: 2

# deployment plan permissions

deployment:

  name: Product Name

deployment-permissions:

  ...

These are the tips and tricks that have helped us overcome our biggest migration challenges so far, I hope they can help others as well. If you have any others that come to mind, or improvements over the above, please let me know in the comments!
...

Originally published at https://therightstuff.medium.com.

Saturday, 23 January 2021

Towards a Python version of my Javascript AWS CDK guide

After successfully using a Typescript CDK project to deploy a python lambda on Thursday, I decided to spend some time this evening creating a Python CDK guide. It's very limited at the moment (just a simple function and a basic lambda layer), but it's a start!

Tuesday, 28 July 2020

Resolving private PyPI configuration issues

The Story

I've spent the last few days wrestling with the serverless framework, growing fonder and fonder of AWS' CDK at every turn. I appreciate that serverless has been, until recently, very necessary and very useful, but I feel confident in suggesting that when deployment mechanisms become more complex than the code they're deploying things aren't moving in the right direction.

That said, this post is about overcoming what for us was the final hurdle: getting Docker able to connect to a private PyPI repository for our Python lambdas and lambda layers.

Python is great, kind of, but is marred by two rather severe complications: the split between python and python3, and pip… so it's a great language, with an awful tooling experience. Anyway, our Docker container couldn't communicate with our PyPI repo, it took way too long to figure out why, here's what we learned:

The Solution

If you want to use a private PyPI repository without typing in your credentials at every turn, there are two options:

1.


~/.pip/pip.conf:



2.


~/.pip/pip.conf:



~/.netrc:



It is important that ~/.netrc has the same owner:group as pip.conf, and that its permissions are 0600 (chmod 0600 ~/.netrc).

What's not obvious - or even discussed anywhere - is that special characters are problematic and are handled differently by the two mechanisms.

In pip.conf, the password MUST be URL encoded.
In .netrc, the password MUST NOT be URL encoded.

The Docker Exception

For whatever reason, solution 2 (the combination of pip.conf and .netrc) does NOT work with Docker.

Conclusion

Amazon's CDK is excellent, and unless you have a very specific use-case that it doesn't support it really is worth trying out!

Oh! And that Python is Very Nice, but simply isn't nice enough to justify the cost of its tooling.

Monday, 22 June 2020

A templated guide to AWS serverless development with CDK

If all you're looking for is a no-frills, quick, step-by-step guide, just scroll on down to The Guide!


The Story

Or, how I learned to let go of my laptop and embrace The Cloud...


Motivation


A while ago I outlined a project that I intend to build, one that I'm rather enthusiastic about, and after spending a few hours evaluating my options I decided that an AWS solution was the right way to go: their options are configurable and solid, their pricing is very reasonable and, as an added bonus, I get to familiarize myself with tech I'll be using for my current contract!

All good.

Once I'd made the decision to use AWS, becoming generally familiar with the products on offer was easy enough and it took me very little time to flesh out a design. I was ready to get coding!

Or so I thought...


Hurdle 1

Serverless. And possibly other frameworks, I'm pretty sure I looked at a few but it seems like forever ago. Serverless shows lots of promise, and I know people who swear by it, but it's only free up to a point and I immediately ran into strange issues with my first deployment. I found the interface surprisingly unhelpful, and it looks like once you're using it you're somewhat locked in.

Pass.


Hurdle 2

Cognito. Security first. Cognito sounds like a great solution, but once I'd gotten a handle on how it works I was severely disappointed by how limiting it is, and getting everything set up takes real developer and user effort (and I suffer enough from poor interface fatigue). After playing around with user pools and looking at various code samples, I realized that I'd rather allow users to register using their email addresses / phone numbers as 2nd-factor authentication (mailgun and twilio are both straightforward options), or use oauth providers like Facebook, Google and GitHub, and I certainly want to encourage my users to use strong, memorable passwords (easily enforced with zxcvbn, and why is this still a thing?!) which Cognito doesn't allow for.

You'll need to configure Lambda authorizers either way, so I really don't think Cognito adds much value.


Hurdle 3

Lambda / DynamoDB. Okay, so writing a lambda function is really easy, and guides and examples for code that reads to and writes from DynamoDB abound. Great! Except for the part where you need to test your functions before deploying them.

My first big mistake - and so far it's proved to be the most expensive one - was not understanding at this point that "testing locally" is simply not a feasible strategy for a cloud solution.


Hurdle 4

The first code I wrote for my project was effectively a lambda environment emulator to test my lambda functions. It was far from perfect, and it did take me a couple of hours to cobble together, but it did the job and I used it to successfully test lambda functions against DynamoDB running in Docker.


Hurdle 5

Lambda Layers. Why do most guides not touch on these? Why are there so few layers guides written for Javascript? It took me a little while to get a handle on layers and build a simple script to create them from package.json files, but as far as hurdles go this was a relatively short one.


Hurdle 6

Deployment. It's nice to have code running locally, but uploading it to the cloud? Configuring API Gateway was a mixed bag of Good Interface / Bad Interface, same with the Lambda functions, and what eventually stumped me was setting up IAM to make everything play nicely together. What's the opposite of intuitive? Not counter-intuitive, in this case, as I don't feel that that word evokes nearly enough frustration.

Anyway, it became abundantly clear at that point that manual deployment of AWS configurations and components is not a viable strategy.


Hurdle 7

A coworker introduced me to two tools that could supposedly Solve All My Problems: CDK and SAM. This seemed like a worthy rabbit-hole to crawl into, but I couldn't find any examples of the two working together!

I began to build my own little framework, one that would allow me to configure my stack using CDK, synthesize the CloudFormation templates, and test locally using SAM. Piece by piece I put together this wonderful little tool, hour by hour, first handling DynamoDB, then Lambda functions, then Lambda Layers...

It was at that point that realization dawned: not only are SAM and CDK not interoperable by design, but SAM does not, in fact, provide meaningful "local" testing. Sure, you can invoke your lambda functions on your local machine, but the intention is to invoke them against your deployed cloud infrastructure. Once I got that through my head, it was revelation time: testing in the cloud is cheaper and better than testing locally.


The Guide

If you're like me, and you intend your first foray into cloud computing to be simple, yet reasonably production-ready, CDK is the easiest way forward and it's completely free (assuming you don't factor in your time figuring it out as valuable, but that's what I'm here for!).

Over the course of the past couple of weeks, I've put together the aws-cdk-js-dev-guide. It's a work in progress (next stop, lambda authenticators!), but at the time of writing this guide it's functional enough to put together a simple stack that includes Lambda Layers, DynamoDB, functions using both of those, API Gateway routes to those functions and the IAM permissions that bind them.

And that's just the tip of the CDK iceberg.


Step 1 - Tooling

It is both valuable and necessary to go through the following steps prior to creating your first CDK project:

  • Create a programmatic user in IAM with admin permissions
  • If you're using Visual Studio Code (recommended), configure the aws toolkit
  • Set up credentials with the profile ID "default"
  • Get your 12 digit account ID from My Account in the AWS console
  • Follow the CDK hello world tutorial


Step 2 - Creating a new CDK project

The first step to creating a CDK project is initializing it with cdk init, and a CDK project cannot be initialized if the project directory isn't empty. If you would like to use an existing project (like aws-cdk-js-dev-guide) as a template, bear in mind that you will have to rename the stack in multiple locations and it would probably be safer and easier to create a new project from scratch and copy and paste in whatever bits you need.


Step 3 - Stack Definition

There are many viable approaches to setting up stages, mine is to replicate my entire stack for development, testing and production. If my stack wasn't entirely serverless - if it included EC2 or Fargate instances, for example - then this approach might not be feasible from a cost point of view.

Stack definitions are located in the /lib folder, and this is where the stacks and all their component relationships are configured. Here you define your stacks programmatically, creating components and connecting them.



I find that the /lib folder is a good place to put your region configurations.



Once you have completed this, the code to synthesize the stack(s) is located in the /bin folder. If you intend, like I do, to deploy multiple replications of your stack, this is the place to configure that.



See the AWS CDK API documentation for reference.


Step 4 - Lambda Layers

I've got much more to learn about layers at the time of writing, but my present needs are simple: the ability to make npm packages available to my lambda functions. CDK's Code.fromAsset method requires the original folder (as opposed to an archive file, which is apparently an option), and that folder needs to have the following internal structure:

.
.nodejs
.nodejs/package.json
.nodejs/node_modules/*

You can manually create this folder anywhere in your project, maintain the package.json file and remember to run npm install and npm prune every time you update it... or you can just copy in the build-layers script, maintain a package.json file in the layers/src/my-layer-name directory and run the script as part of your build process.


Step 5 - Lambda Functions

There are a number of ways to construct lambda functions, I prefer to write mine as promises. There are two important things to note when putting together your lambda functions:

1. Error handling: if you don't handle your errors, lambda will handle them for you... and not gracefully. If you do want to implement your lambda functions as promises, as I have, try to use "resolve" to return your errors.

2. Response format: your response object MUST be in the following structure:


This example lambda demonstrates using a promise to return both success and failure responses:



Step 6 - Deployment

There are three steps to deploying and redeploying your stacks:
  1. Build your project
    If you're using Typescript, run
    > tsc
    If you're using layers, run the build-layers script (or perform the manual steps)
  2. Synthesize your project
    > cdk synth
  3. Deploy your stacks
    > cdk deploy stack-name
    Note: you can deploy multiple stacks simultaneously using wildcards
At the end of deployment, you will be presented with your endpoints. Unless your lambda has its own routing configured - see the sample dynamodb API examples that follow - simply make your HTTP requests to those URLs as is.


    Sample dynamodb API call examples:
  • List objects
    GET https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/prod/objects
  • Get specific object
    GET https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/prod/objects/b4e01973-053c-459d-9bc1-48eeaa37486e
  • Create a new object
    POST https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/prod/objects
  • Update an existing object
    POST https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/prod/objects/b4e01973-053c-459d-9bc1-48eeaa37486e

Step 7 - Debugging

Error reports from the API Gateway tend to hide useful details. If a function is not behaving correctly or is failing, go to your CloudWatch dashboard and find the log group for the function.

Step 8 - Profit!!!

I hope you've gotten good use of this guide! If you have any comments, questions or criticism, please leave them in the comments section or create issues at https://github.com/therightstuff/aws-cdk-js-dev-guide.

Monday, 23 September 2019

@mysql/xdevapi joy!

after hitting a wall last night with the existing node.js mysql packages, which for some reason have been years behind integrating with mysql 8's authentication, i discovered @mysql/xdevapi. the documentation's a bit heavy and not really comprehensive, but now that i've figured out how to use it, i'm very pleased with how it operates! unfortunately, i'm going to have to roll my own migrations management, but that's a small price to pay for being able to interface with the latest mysql databases in an intelligent way.

in the interests of reducing friction for other adopters, i've rolled some sample queries into my database creation script. enjoy!

UPDATE: i've subsequently learned that table joins have not been implemented, so to perform those you'd have to use the session.sql method and use it in the same way (the same promise chaining) as the CRUD methods. seems like a serious oversight, but whatever.

Thursday, 22 August 2019

Mail forwarding and piping emails with Postfix for multiple domains

The past few weeks I've been learning about mail servers, and the biggest takeaway for me is that it's generally worth paying someone else to handle the headache. As usual, the obstacles to configuring a mail server correctly are primarily in the lack of useful documentation and examples, so I'm putting this down here in the hopes that it'll be helpful to like-minded constructively-lazy devs.

While I'm happily using mailgun for mail sending (after much frustration I threw in the towel trying to integrate DKIM packages with Postfix to get my outgoing emails secured) I was certain that I could at least have my mail server handle mail-forwarding for my multiple domains, and while that proved to be fairly straightforward I then tumbled down a rabbit-hole trying to get Postfix to pipe certain emails to a node.js script for processing.

    Here are the steps you'll need to take:

  • Set up your A and MX records for your domain, the A record @ pointing to the IP address of the server you’re going to be receiving emails on and MX with the hostname @ and the value 10 mail.your-domain-name

    If your mail server is not the same as your primary A record, simply create an additional A record mail pointing to the correct IP address.
  • sudo apt-get install postfix
    Select "Internet Site" and enter your-domain-name (fully qualified)
  • sudo vi /etc/postfix/main.cf
    • Add mail.your-domain-name to the list of mydestination values
    • Append
      virtual_alias_domains = hash:/etc/postfix/virtual_domains
      virtual_alias_maps = hash:/etc/postfix/virtual
      to the end of the file
  • sudo vi /etc/aliases
    curl_email: "|curl --data-binary @- http://your-domain-name/email"
  • sudo newaliases
  • sudo vi /etc/postfix/virtual_domains
    example.net   #domain
    example.com   #domain
    your-domain-name   #domain
    (the #domain fields suppress warnings)
  • sudo postmap /etc/postfix/virtual_domains
  • sudo vi /etc/postfix/virtual
    info@your-domain-name bob@gmail.com
    everyone@your-domain-name bob@gmail.com jim@gmail.com
    email_processor@your-domain-name curl_email@localhost
    @your-domain-name catchall@whereveryouwant.com
    ted@example.net jane@outlook.com
  • sudo postmap /etc/postfix/virtual
  • sudo /etc/init.d/postfix reload

You should be able to find your postfix logs at /var/log/mail.log. Good luck!

Monday, 5 March 2018

Let's Encrypt site deletion (Apache on Ubuntu)

Let's Encrypt is brilliant, but why site deletion makes things so complicated is beyond me. I had a bunch of sites running on Apache that Let's Encrypt automagically generated certificates for, which was fine right up until I needed to make one of them unavailable. What follows are the instructions for regenerating the certificates quickly and (relatively) painlessly.

  1. Ensure that Apache site configurations are removed from
    /etc/apache2/sites-enabled
  2. Move unwanted site configurations from
    /etc/apache2/sites-available to a backup location if needed
  3. Remove all certificate files from
    /etc/letsencrypt/archive/
    /etc/letsencrypt/live/
    /etc/letsencrypt/renewal/
  4. Restart Apache
    service apache2 restart
  5. Run letsencrypt

Sunday, 19 November 2017

'How to create an Azure SQL Database programmatically' with less frustration

Microsoft's devs have done yet another "almost-but-not-quite-great" job with their sample code for creating an Azure database programmatically.

In addition to the headache of setting up an Azure subscription and Azure Active Directory correctly, it took a silly amount of investigation and trial and error before I could figure out what values the code was expecting as the variable names (almost predictably) don't match their counterparts as displayed in the Azure portal.
    Instructions:
  1. Follow the instructions from the original code's page up to item 4 ("Add your variables to the program")
  2. Replace Program.cs with my modified code.
  3. Follow the instructions in the 4th item, using the modified code's documentation in case of confusion or ambiguity.
Once a database has been created, you can log in via SQL Server Management Studio using the server's fully qualified domain name (xxxxx.database.windows.net) and the server's admin credentials which were created along with the server. For this you will need to update the server's firewall to allow your client IP's connections.

Sunday, 23 April 2017

How to code Entity Framework programmable migrations

The internets are eerily quiet when it comes to running Entity Framework's migrations through code, which is odd because Entity Framework's Code First migrations are extremely powerful and calling them (or rolling them back) through the code seems an obvious choice to someone like me, who prefers to reduce the amount of complexity for the poor IT people handling deployment.

(And for poor me, too, I'm very lazy and I like being able to just press a button to make magic happen.)

The most important quirk of running migrations programmatically that you'll need to know before we begin is that the DbMigrator class, for reasons unclear, requires the Entity Framework Configuration class but uses it in a way that's incompatible with the Update-Database and Add-Migration scripts, even after manually configuring the ContextType and ContextKey properties. We'll set those anyway for consistency, but no matter where you store your migrations and however diligently you inform the DbMigrator (the MigrationsDirectory property was ignored too) you're going to be stuck with automatic migrations. And while we're fiddling with the Configuration class, be sure to make the internal Configuration class public.

Automatic migrations ignore the explicit migration files and detect changes between the context's models and the database. As with the Update-Database script, the database will be created if it hasn't been already. It will not function reliably unless the AutomaticMigrationDataLossAllowed property is set to true, and you'll also need to update the OnModelCreating method of your context class to allow for database calls when the models are out of sync to give you a chance to run the migrations:





Migrating up and down

Using automatic migrations to update your database is great, but what do you do if you need to roll back? DbMigrator requires a migration name to update to, and unlike the case with explicit migrations, the migration name is not established by the developer but is unique to the database in question.

Fortunately, the DbMigrator includes the GetDatabaseMigrations method, which returns a list of applied migrations; so while you won't be able to roll back to a predefined named state, you will be able to roll back to a previously run migration. Here it is important to note that the order of the migrations returned is not guaranteed, but the names begin with a timestamp so they're not too difficult to sort.



So rolling back the first of that list is as simple as



Seeding

Seeding must be performed manually after calling DbMigrator.Update(). The Configuration class' Seed method is protected, so add the following wrapper method to your Configuration class



and call it once the migration is complete.