Resolving private PyPI configuration issues

The Story

I've spent the last few days wrestling with the serverless framework, growing fonder and fonder of AWS' CDK at every turn. I appreciate that serverless has been, until recently, very necessary and very useful, but I feel confident in suggesting that when deployment mechanisms become more complex than the code they're deploying things aren't moving in the right direction.

That said, this post is about overcoming what for us was the final hurdle: getting Docker able to connect to a private PyPI repository for our Python lambdas and lambda layers.

Python is great, kind of, but is marred by two rather severe complications: the split between python and python3, and pip… so it's a great language, with an awful tooling experience. Anyway, our Docker container couldn't communicate with our PyPI repo, it took way too long to figure out why, here's what we learned:

The Solution

If you want to use a private PyPI repository without typing in your credentials at every turn, there are two options:

1.

~/.pip/pip.conf:

2.

~/.pip/pip.conf:



~/.netrc:



It is important that ~/.netrc has the same owner:group as pip.conf, and that its permissions are 0600 (chmod 0600 ~/.netrc).

What's not obvious - or even discussed anywhere - is that special characters are problematic and are handled differently by the two mechanisms.

In pip.conf, the password MUST be URL encoded.
In .netrc, the password MUST NOT be URL encoded.

The Docker Exception

For whatever reason, solution 2 (the combination of pip.conf and .netrc) does NOT work with Docker.

Conclusion

Amazon's CDK is excellent, and unless you have a very specific use-case that it doesn't support it really is worth trying out!

Oh! And that Python is Very Nice, but simply isn't nice enough to justify the cost of its tooling.

the rights and wrongs of git push with tags

My employer uses tags for versioning builds.

git push && git push --tags

I'm guessing this is standard practice, but we've been running into an issue with our build server - Atlassian's Bamboo - picking up commits without their tags. The reason was obvious: we've been using git push && git push --tags, pushing tags after the commits, and Bamboo doesn't trigger builds on tags, only on commits. This means that every build would be versioned with the previous version's tag!

The solution should have been obvious, too: push tags with the commits. Or before the commits? This week, we played around with an experimental repo and learned the following:

git push --tags && git push

To the uninitiated (like us), the result of running git push --tags can be quite surprising! We created a commit locally, tagged it*, and ran git push --tags. This resulted in the commit being pushed to the server (Bitbucket, in our case) along with its tag, but the commit was rendered invisible. Not even git ls-remote --tags origin would return it, and it was not listed under the commits on its branch, although it showed up with its commit on Bitbucket's feature to search commits by tag quite clearly:

* All tags are annotated, automatically, courtesy of SourceTree. If you're not using SourceTree then you should be, and I promise you I'm not being paid to say that. If you must insist on the barbaric practice of using the command line, just add -a (and -m to specify the message inline if you don't want the editor to pop up, check out the documentation for more details).

Pushing a tag first isn't the end of the world - simply pushing the commit with git push afterwards puts everything in order - unless someone else pushes a different commit first. At that point the original commit will need to be merged, with merge conflicts to be expected. Alternatively - and relevant for us - any scripts that perform commits automatically might fail between pushing the tags and their commits, leading to "lost" commits.

git push --tags origin refs/heads/develop:refs/heads/develop

This ugly-to-type command does what we want it to do: it pushes the commit and its tags together. From the documentation:

When the command line does not specify what to push with <refspec>... arguments or --all, --mirror, --tags options, the command finds the default <refspec> by consulting remote.*.push configuration, and if it is not found, honors push.default configuration to decide what to push (See git-config[1] for the meaning of push.default).

Okay, so that works. But there's no way I'm typing that each time, or asking any of my coworkers to. I want English. I'm funny that way.

git push --follow-tags

Here we go: the real solution to our problems. As long as the tags are annotated - which they should be anyway, see the earlier footnote on tagging - running git push --follow-tags will push any outstanding commits from the current branch along with their annotated tags. Any tags not annotated, or not attached to a commit being pushed, will be left behind.

Resolved!