A quick-start guide to setting up a Debian guest on VMWare WorkStation 15/16 Player

I don’t know why everything needs to be subtly non-standard, but over the course of the last twenty or so virtual machine reconstructions I’ve come up with a simple checklist for setting up a Debian guest on VMWare’s WorkStation (Windows) and I thought I’d share it here.

1. Download your Debian .iso here (I recommend the netinst CD image)
2. Create a new virtual machine using the downloaded .iso and check that the default configuration is satisfactory (I tend to need a little more power, usually 2 CPUs does it for me). Note that increasing the size of a hard disk is for more complicated and risky than a regular user would expect, so give yourself a healthy buffer. In my experience, it’s less painful to rebuild a bigger machine than it is to extend the disk size.
3. Install the OS — I find the graphical installer to be just fine for my purposes. The two configuration options that are the most impactful are your choice of desktop environment, I usually choose Xfce but I’m starting to like GNOME again. It’s probably a good idea to install the ssh server as well.
4. Once installation is complete, click the VMWare button at the bottom of the screen to signal that and restart the machine.
5. To grant yourself the ability to use sudo, open the terminal and run either
   > su -
   or
   > su -c ‘su -’
   if the first isn’t allowed (I’m not sure how to use the correct single quotes in Medium, so note that the above apostrophes aren’t correct).
   Then run
   > usermod -aG sudo <username>
   to add yourself to the sudoers group. You will need to log out and back in for this to take effect.
6. Install the following to be able to install VMWare Tools, which enables things like copying and pasting between host and guest machines:
   > sudo apt install -y open-vm-tools open-vm-tools-desktop linux-source

Installing VMWare Tools in WorkStation Player 16

Open the Virtual Machine Settings, select the Options tab, then select VMWare Tools: select “Sychronize guest time with host” and “Update automatically”, then restart the virtual machine.

Installing VMWare Tools in WorkStation Player 15

1. Open the VM menu, select Install VMWare Tools.
2. Mount the VMWare Tools CD:
   > sudo mount /dev/cdrom
3. Extract the installer to your current directory (or maybe create a subdirectory for it) using tab auto-complete in place of the ellipse:
   > tar -xf /media/cdrom/VMWareTools…
4. Install the required build tools:
   > sudo apt-get install -y autoconf automake binutils cpp gcc linux-headers-$(uname -r) make psmisc
5. Try to run the .pl script in the extracted folder, expect it to fail, restart the machine anyway.

At this point you should have your VM up and running and be able to copy / paste / drag files between your machines. Now go grab yourself another cup of coffee, you deserve it! 

...

Originally published at https://therightstuff.medium.com.

An Impatient Developer’s Guide to Debian Maintenance (Installation) Scripts and Package Diverts

Photo by Andrea Piacquadio from Pexels

 The people involved in coming up with the dpkg scheme for installing / upgrading / downgrading / removing packages are very clever. While unintuitive to the uninitiated, the scheme is mostly logical and reasonable, though there are some points where I feel a little more effort and consideration could have made a world of difference.

“The evil that men do lives on and on” — Iron Maiden

In addition to regular installation behaviour, I needed to wrap my head around “package diverts”, which is a very clever system for enabling packages to handle file conflicts. Except that it doesn’t handle what I would consider to be a very basic use case:

1. Install an initial version of our package.
   
   
2. Discover that our package needs to overwrite a file that’s installed by an upstream dependency.
   
   
3. Create a new version of our package that includes the file and configures a “package divert” to safely stow the dependency’s version.
   
   
4. Remove the “package divert” on the file if the newer version of the package is uninstalled or downgraded to the previous version that doesn’t include it.

That last part, in italics? That’s the kicker right there. Read on to understand why.

Debian Installation Script Logic In Plain English

After poring over the Debian maintainer scripts flowcharts, I felt I had a pretty good handle on things but there’re a couple of little “gotcha”s, so I feel like it’s worth providing a brief summary of the general flow in common English.

Debian maintenance scripts are run by apt and dpkg at specific points in the installation process:

https://www.debian.org/doc/debian-policy/ap-flowcharts.html

1. If the package is being removed or upgraded (meaning that a different version is being installed, “upgraded” also means “downgraded” to Debian maintainers), the previously installed version’s prerm script is called.
   
   
2. If the package is being installed or upgraded, the new version’s preinst script is run.
   
   
3. If the package is not being removed, the new version’s package contents are unpacked. This will overwrite existing files if they were installed by the same package, or fail the installation if it attempts to overwrite another package’s files without a package divert configured.
   
   
4. If the package is being removed, its files are deleted.
   
   
5. The previously installed version’s postrm script is called if the package is being removed or upgraded.
   
   
6. If the package is not being removed, any package contents belonging to the previous version that do not also exist in the new one are removed.
   
   
7. If the package is not being removed, the new version’s postinst script is called.

It is important to note that if a maintenance script fails with a non-zero exit code, the package will be in a broken state that can be very difficult (sometimes impossible) to recover. From our experience, it’s best to catch all exceptions, log them, “exit gracefully” with an exit code of 0, and hope for the best.

Also from our experience, it’s a good idea for maintenance scripts to log everything to the stderr stream in order to preserve chronological order.

Debian Package Diverts for the Uninitiated

The principle of Debian package diverts is straightforward enough: when you want to include a file in your package contents that conflicts with another package’s file (i.e. the absolute paths are identical), you create a “divert” on that file so that any other package’s versions of that file is “diverted” to a different file name.

Creating a Package Divert

To create a package divert, your package’s preinst script should run the following command:

dpkg-divert --package my-package-name --add --rename \

    --divert /path/to/original.divert-ext /path/to/original

The preinst script is the place to do this because the divert must be in place before the package contents are unpacked. The dpkg-divert commands are idempotent, so having it called in the preinst of every install is fine.

Removing a Package Divert

When your package is uninstalled, it’s good practice to remove the package divert and rename the diverted files back to their original file names. It’s recommended to remove the package divert in the postrm script, which makes perfect sense when uninstalling a package because the files are deleted before postrm is called.

dpkg-divert --package my-package-name --remove --rename \

    --divert /path/to/original.divert-ext /path/to/original

When removing a package, the package’s files have been deleted already and removing a divert simply renames the diverted file back to its original file name.

When upgrading a package, however, the files are only deleted after postrm has been called. This means that a call to dpkg-divert — remove will fail because it would have to overwrite the upgraded package’s copy of the file that hasn’t yet been removed.

It also means that if you delete your package’s file in the postrm in order to remove the divert, the original package’s file will be deleted after your postrm because it will have been identified as belonging to the upgraded package.

“Insanity is contagious.” ― Joseph Heller, Catch-22

It is for this reason that if we remove the divert in the postrm during a downgrade to a version that does not include the file in its package contents, we will lose the original file. If we do not remove the package divert, we will retain the diverted original file, but it will be renamed and therefore not serve its purpose. In our downgrading scenario, the postinst script that’s run after the file removal phase of an installation belongs to the older version of the package that didn’t know about the file, or package diverts, so that won’t be of any use to us. In short, the only way to downgrade our package is to completely remove it and install the older version, which for us is simply not an option.

Epilogue

Fortunately, my team and I are in the position that the original file also belongs to a package that we maintain, and we are able to overwrite the original file in the postinst script* with confidence and impunity. That means no rolling back without removing and then reinstalling the original package, which in our case happens to be impossible.

* Of course, the file can no longer be included in the package contents with its original path or the installation will fail.

Hope you’ve found this helpful! Please share in the comments if you’ve had similar experiences, or if you know of any other workarounds!

...

Originally published at https://therightstuff.medium.com.

Priority pinning in apt-preferences with different versions and architectures

I'm posting this because I've lost too many hours figuring it out myself, the documentation is missing several important notes and I haven't found any forum posts that really relate to this:

Question: How do I prioritize specific package versions for multiple architectures? In particular, I have a number of different packages which I would like to download for multiple architectures, and I would like to prioritize the versions so that if they're not explicitly provided, apt will try to get a version matching my arbitrary requirements (in my case, the current git branch name) and fall back to our develop branch versions.

eg. I would like to download package my-package for both i386 and amd64 architectures and I would like to pull the latest version that includes my-git-branch-name before falling back to the latest that includes develop.

Answer:

The official documentation is here.

1. In order to support multiple architectures, all packages being pinned must have their architecture specified, and there must be an entry for each architecture. A pinning for the package name without the architecture specified will only influence the default (platform) architecture:

Package: my-package

Pin: version /your regex here/

Pin-Priority: 1001

2. The entries are whitespace-sensitive, although no errors will be reported if you have whitespace. The following pinning will be silently disregarded:

Package: my-package:amd64

    Pin: version /your regex here/

    Pin-Priority: 1001

3. apt update must be called after updating the preferences file in order for them to be respected and after adding additional architectures using (for example) dpkg --add-architecture i386

The following excerpt from /etc/apt/preferences solves the stated problem:

Package: my-package:amd64

Pin: version /-my-git-branch-name-/

Pin-Priority: 1001

Package: my-package:i386

Pin: version /-my-git-branch-name-/

Pin-Priority: 1001

Package: my-package:amd64

Pin: version /-develop-/

Pin-Priority: 900

Package: my-package:i386

Pin: version /-develop-/

Pin-Priority: 900 

It may be worthwhile noting that to download or install a package with a specified architecture and version use the command apt download package-name:arch=version

Mail forwarding and piping emails with Postfix for multiple domains

The past few weeks I've been learning about mail servers, and the biggest takeaway for me is that it's generally worth paying someone else to handle the headache. As usual, the obstacles to configuring a mail server correctly are primarily in the lack of useful documentation and examples, so I'm putting this down here in the hopes that it'll be helpful to like-minded constructively-lazy devs.

While I'm happily using mailgun for mail sending (after much frustration I threw in the towel trying to integrate DKIM packages with Postfix to get my outgoing emails secured) I was certain that I could at least have my mail server handle mail-forwarding for my multiple domains, and while that proved to be fairly straightforward I then tumbled down a rabbit-hole trying to get Postfix to pipe certain emails to a node.js script for processing.

Here are the steps you'll need to take:


• Set up your A and MX records for your domain, the A record @ pointing to the IP address of the server you’re going to be receiving emails on and MX with the hostname @ and the value 10 mail.your-domain-name
  
  If your mail server is not the same as your primary A record, simply create an additional A record mail pointing to the correct IP address.
• sudo apt-get install postfix
  Select "Internet Site" and enter your-domain-name (fully qualified)
• sudo vi /etc/postfix/main.cf
  • Add mail.your-domain-name to the list of mydestination values
  • Append

    virtual_alias_domains = hash:/etc/postfix/virtual_domains
    virtual_alias_maps = hash:/etc/postfix/virtual

    to the end of the file
• sudo vi /etc/aliases
  

  curl_email: "|curl --data-binary @- http://your-domain-name/email"

• sudo newaliases
• sudo vi /etc/postfix/virtual_domains
  

  example.net   #domain
  example.com   #domain
  your-domain-name   #domain

  (the #domain fields suppress warnings)
• sudo postmap /etc/postfix/virtual_domains
• sudo vi /etc/postfix/virtual
  

  info@your-domain-name bob@gmail.com
  everyone@your-domain-name bob@gmail.com jim@gmail.com
  email_processor@your-domain-name curl_email@localhost
  @your-domain-name catchall@whereveryouwant.com
  ted@example.net jane@outlook.com

• sudo postmap /etc/postfix/virtual
• sudo /etc/init.d/postfix reload

You should be able to find your postfix logs at /var/log/mail.log. Good luck!

Setting up Squiz Labs code sniffer for PHPStorm on Windows

PHPStorm is a brilliant piece of software that's highly extensible. One of the extensions that's particularly useful is Squiz Labs' PHP Code Sniffer. It wasn't quite apparent from other documentation I found online that it's really simple to set up for Windows! So with the help of a co-worker I put together the following tutorial:

To install the code sniffer for PHPStorm:

• Download and install PHP for Windows
• Download https://squizlabs.github.io/PHP_CodeSniffer/phpcs.phar
• Create a file called phpcs.bat in the C:\php directory
  @echo off
  "C:\php\php.exe" -f "C:\PATH_TO_DOWNLOADED_PHAR\phpcs.phar" -- %*
• In PHPStorm
• Open File->Settings
• Select Languages & Frameworks->PHP
• Add a local interpreter that points to C:\php\php.exe
• Select Languages & Frameworks->PHP->Code Sniffer
• Update the local configuration to point to C:\php\phpcs.bat
• increase the Tool process timeout if you're working on remote code
• Click Validate to ensure that the code sniffer is loaded correctly
• Click Apply
• Select Editor->Inspections
• Select the checkbox for PHP Code Sniffer validation
• Select PHP Code Sniffer validation
• Set the coding standard (you'll probably want PSR2)
• To run the code sniffer
• Press CTRL + ALT + SHIFT + I
• Type "code sniffer" and press enter
• Select the current file

And there you go!