Weaning Off The Google

 (Or, How I’m Continuing to use Google’s Products Without A Sense of Existential Dread)

Photo by Pixabay on Pexels

My First Brush With Account Recovery Tyranny

Just over five years ago, my wife and I decided to leave Canada for South Africa, and with all the madness of the migration (after six months of being new parents with no support, we left in a big rush to be close to our family) it never occurred to me to ensure that our email accounts were all configured correctly for where we were going. When we finally arrived after two weeks of frenzied selling and packing (and deciding which half of our lives to leave behind) and two days of hard travelling, I thought I’d log in to check my emails.

Google detected that we were logging in from a different country, and wanted us to authenticate using our mobile numbers. Which had been disconnected the day before we left. As some of you may have had the misfortune of discovering, there are no human beings to speak to when attempting to recover access to a Google account, the entire process is automated and even knowing everything about your account (including your personal information, who you last communicated with, previous passwords, etc.) is no guarantee that you’re going to get it back.

We were very, VERY lucky that a really kind support agent of our Canadian mobile carrier was willing (and able) to reinstate my released number temporarily and read me back the verification codes, and even though we were forced to pay out the nose for that little service it was well worth not losing… pretty much everything that matters to a 21st century digital person.

A Reminder That It’s A False Sense Of Security

After successfully recovering our accounts, we put the experience behind us and made a mental note to sort out account verification before the next time we move. We have continued to use our gmail accounts as primary email addresses for a thousand other services, further entrenching our already-heavy reliance on a service that’s “easy come, easy go”.

What The Google Giveth, The Google Taketh Away

A few months ago I read an article on Medium, What it’s like to get locked out of Google indefinitely, and that sense of dread came rushing back as I realized that we still have absolutely no way out if we ever get stuck like that again. I’ve been a bit preoccupied with other things, though, so I haven’t really done much about it. Every once in a while I’d look at my task list, be reminded that we’re still at the mercy of a heartless, mindless system, and continue my day carrying just a teensy bit more anxiety than usual.

(If that article’s not convincing enough, please also take a look at Google has Threatened to Delete all our Google Accounts Over Nothing)

But I Like GMail!

Okay, here’s the deal. I like Google’s products. I like them better than other products. They’re good products. Good enough that I’ll ignore the data mining, the ads, even the fact that it’s Google providing them.

Honestly, sometimes I think the only two reasons I prefer GMail to any other provider are the fact that organization is by labels instead of folders, and their custom filtering is excellent. I guess that’s all any other provider would have to offer for me to be ready to jump ship.

Separating Email From Account Management

The first step to safeguarding all my other accounts was to establish one that nobody could take away from me. Fortunately, I already have a domain under my control, but then… a conundrum. What email address do I use to secure the account that manages the server that manages my email address?

Fortunately, that’s easy — multiple accounts can be used to secure that one. I signed up for a reliable, secure email address from a different provider (ProtonMail), so I at least have backup access in case either of them fails me.

Having taken care of that, I set up my own email address (which will be described in a separate post specific to configuring a postfix server — my post about Mail forwarding and piping emails with Postfix for multiple domains needs a bit of an update since I learned how to set the outgoing encryption, and even that isn’t sufficient for getting past spam filters), which now forwards to both of the other email accounts (my GMail and my new accounts email).

At this point, I was finally ready to begin the laborious process of switching the primary email of all my other accounts. It’s been an educational experience, with some services easier to update than others, but after investing a good few hours I believe I’m finally through the worst of it and have the essential services covered.

Backing Up Account Content

I’ve been considering the fact that while getting locked out of my accounts is one of my greatest fears, losing access to gigabytes of email history, documents, and videos wouldn’t be too much fun either.

During the course of the last couple of weeks, I was struggling to find an old video that I was *certain* I’d uploaded to YouTube, and eventually found it on an unclaimed channel. Google has a channel claim process, though… but it’s also fully automated. After trying and failing to claim it with my active accounts, I realized that it must have been attached to an old account that I’d deleted many years ago.

Did you know that a deleted Google account is completely unrecoverable? It is literally impossible to reinstate it, and the username will be locked forever so there’s not even a possibility of recreating it.

Over the course of this weekend I came across another Medium article, How to Quit Gmail and Reclaim Your Privacy. There’s a lot of good advice in there, but No. 7, “Don’t Delete Your Old Address”? Consider that a golden rule.

Personally, I have a terabyte drive (or two) that I use for backups, but I’ve come to the conclusion that I’m not nearly as capable of protecting my physical disks as the professionals. I’m a big fan of DropBox, which has an excellent interface and syncing tools, but I’m not a fan of their pricing models. I’ve now resorted to uploading my backups to an AWS S3 bucket, treating it as cold storage only to be used in case of emergency.

For the low prices (for my purposes, anything from the standard storage plans to glacier will do), and the safety guarantee, I’m sold.

Next Steps

I’ve now set myself a regular reminder to download my Google data and upload it to my backup bucket. At this point, I’m considering this little adventure complete and I’m ready to relax and enjoy the remainder of our long weekend in celebration of South Africa’s National Women’s Day.

I hope this article has been helpful, if you have anything you’d like to add (or disagree with) please let me know in the comments!

...

Originally published at https://therightstuff.medium.com.

Bamboo YAML Specs Tips and Tricks (For Fun and Profit)

[UPDATED 2021.10.01 WITH ADDITIONAL DEPLOYMENT PLAN LEARNINGS]

Introduction

“In for a penny, in for a pound” — that’s our team’s current approach regarding Atlassian’s products. As part of our efforts to code as much of our infrastructure as possible, we’ve recently begun migrating old build plans to Bamboo Specs and, as non-Java devs who don’t want more headaches than are absolutely necessary, we’ve chosen to work with the Bamboo Specs YAML offering.

YAML specs are pretty great. Migrations are assisted by the fact that manually-created plans will show you their YAML translations, so in most cases a simple copy/paste is all you need. I mean, aside from migrating to linked repositories and ensuring they’re configured correctly…

Having said that, Bamboo’s YAML specs are an incomplete product with undocumented critical features, and fail to provide out-of-the-box support for a surprising number of standard use cases that one would expect software engineers building software for other software engineers to appreciate the value of. 

They’re still pretty great in spite of that, but overcoming the limitations of incomplete specs definitions and deployment plans is not exactly intuitive. This article attempts to cover some of the missing pieces and suggest some workarounds that we’ve found useful.

Configuring a linked repository for Bamboo Specs

Bamboo Specs require the configuration of a linked repository. Head to settings -> linked repositories (this will require admin permissions) to create or update a linked repository configuration. The two most important steps to be taken here are as follows:

1. Determine which branch of your repository will be used by Bamboo to read the specs file. Only one branch can be considered the source of truth, my personal recommendation is to make it the development branch.
   
   
2. Two sets of permissions need to be configured correctly in order for Bamboo Specs to be able to do their job:
   
   a. First, the Bamboo project must give permissions to the linked repository to create and modify the build and deployment plans. Head to the project you want your linked repository to operate in, go to Project Settings, then Bamboo Specs repositories, and add the linked repository.
   
   b. Under the Bamboo Specs tab of the linked repository, enable both Access all projects and Access all repositories. I’m pretty confident that this requirement is not a security best-practice, but in my experience Bamboo Specs won’t work without them.

Testing Bamboo Specs

When experimenting with changes to the specs, I’ve found it useful to do the following:

1. Create a testing branch (a feature / bugfix branch).
   
   
2. Change the build and deployment plans’ names and keys and push them to origin so that your changes won’t overwrite the existing plans.
   
   
3. Set the testing branch to be the linked repository’s branch in the General tab and proceed with your changes.
   
   
4. Once you’re happy with the changes and they’ve been reviewed: set the linked repository’s branch back to the development branch, then revert the plan names and keys in the testing branch to the existing plans’.
   
   
5. Delete the test plans manually.

Unexpected Limitations of Bamboo Specs

Plan dependencies

The most glaring omission in the YAML specs is the inability to set up plan dependencies. While we were initially upset by this, we quickly realized that at the end of the day plan dependencies are a nice hack that we shouldn’t really have been relying on in the first place. Bamboo appears to encourage the download of artifacts from matching branches on other build plans, but this can quickly break down into chaos with dependency versioning and management. I warmly recommend using Bamboo artifacts for build debugging and deployment plans exclusively, and proper package repositories for storing and retrieving versioned build artifacts.

When exporting Bamboo Specs from existing plans, build plans and deployment plans are not considered strongly related so you will need to gather and combine the specs from both into a single file. To do this, copy in the deployment plan specs beneath the build plan specs, retaining the --- as separators.

NOTE: the ordering of the sections is important! The build plan definition must be followed by the build plan permissions, then the deployment plan definition(s), with each deployment plan definition followed by a section for its permissions. See the outline at the end of the article for clarity.

An interesting omission is the inability to include unset plan variables. This actually makes sense, as manually maintained plans need some way to ensure that they’re all using the same variable names, but with Bamboo Specs it’s really on you to be consistent and it’s obviously much* easier to search through a single file than it is to hunt for variables across different plan branches via the Bamboo interface.

* infinitely easier

Deployment Plans — sharing build variables and tooling

The principal idea behind a deployment plan is to separate deployment from the build process. Bamboo implements deployment plans as distinct entities with entirely different environments, with the intention that your only interaction with the related build plan is to download your artifacts from it.

For us, this proved problematic as we require shared environment variables and tooling to deploy our builds. To work around this, we required the following mechanisms:

1. Environment variable injection. Early in our build plan tasks, we prepare an environment variable file in the following format and include values like build versions and git branch names.
   
   version=1.0.2
   git_branch=feature/example
   
   WARNING: variable values MUST NOT be surrounded by quotations, as this leads to unpredictable behaviour.
   
   It is recommended to use the inject namespace for injections. When the scope of the injection is RESULT, the variables will be available to all subsequent build tasks as well as the attached deployment plan. In Bamboo Specs they’ll be available in the form ${bamboo.inject.git_branch} and in inline scripts as $bamboo_inject_git_branch (on Unix agents) or %bamboo_inject_git_branch% (on Windows agents).
   
   One of my favourite uses of this technique is the ability to name releases automatically based on the build version (see the following section for the example).
   
   
2. Deployment plans are not really designed to use git directly, but we have found that we sometimes require non-build folders to be available for deployment, such as documentation. In these cases, we simply zip the desired folders and make them available as artifacts as well.
   
   
3. Running the deployment in a docker container. I find it disconcerting that such an extremely useful feature is undocumented! Deployment plan environments can be configured to run in a docker container just like a build plan, which provides us with all requisite tooling and context.
   
   DevEnvironment:
     docker:
       image: golang:1.16.6-buster
       docker-run-arguments:
       - --net=host
     tasks:

Linking multiple branches to a single deployment plan

Ironically, while deployment plans are supposed to operate independently from build plans, they only really function well when linked to specific build branches. If your intention is to build once, then deploy the build artifacts to multiple stages, you're out of luck!

UPDATE: it turns out we missed an important option! If release-naming is set to an environment variable, it only works for the specified branch (even if that specified branch is the default branch). If you want release-naming to be set to an environment variable for any branch, then it needs to be configured as follows:

release-naming:
  next-version-name: ${bamboo.inject.version}
  applies-to-branches: true

The disadvantage of using a single deployment plan is that the link to the deployment plan will only be available from the default build plan branch, but in my experience this is a very minor price to pay for the simplicity. The alternative - a single deployment plan for each branch of interest - is not only messier, but is also annoying to configure as you have to know the branch keys in advance so the branches cannot be automatically managed (plan branch keys are autoincremented and uneditable).

Regardless of your choice, it's probably a good idea to handle your branch management manually:

branches:
  create: manually
  delete: never

Putting it all together

My recommendation for the general outline of a YAML specs file is as follows:

---

version: 2

# build plan definition

plan:

  project-key: PROJECTKEY

  key: PLANKEY

  name: Product Name

  ...

branches:

  create: manually

  delete: never

---

version: 2

# build plan permissions

plan:

  key: PROJECTKEY-PLANKEY

plan-permissions:

  ...

---

version: 2

# deployment plan definition

deployment:

  # NOTE: deployment plan names must be unique

  name: Product Name

  source-plan: PROJECTKEY-PLANKEY

release-naming:
  next-version-name: ${bamboo.inject.version}
  applies-to-branches: true

...

---

version: 2

# deployment plan permissions

deployment:

  name: Product Name

deployment-permissions:

  ...

These are the tips and tricks that have helped us overcome our biggest migration challenges so far, I hope they can help others as well. If you have any others that come to mind, or improvements over the above, please let me know in the comments!

...

Originally published at https://therightstuff.medium.com.