Category: extremely controversial

For a Jenkins environment I had to automate the creation of a lot of identical build agents. Identical up until of course the network configuration. Sure I could have used Docker or what not. But the organization standardized on VMWare ESXi. So I had to work with the tools I got.

A neat trick that you can do with VMWare is to write so called guestinfo variables in the VMX file of your guests.

You can get SSH access to the UNIX-like environment of a VMWare ESXi host. In that environment you can do typical UNIX scripting.

First we prepare a template that has VMWare guest tools installed. We punch the zeros of the vmdk file and all that stuff. So that it’s nicely packaged and quick to make clones from. On the guest you do:

dd if=/dev/zero of=/largefile bs=10M ; rm /largefile

On the ESXi host you do:

vmkfstools --punchzero /vmfs/volumes/STORAGE/template/DISK.vmdk

Now you can for example do this (on the ESXi host’s UNIX environment):

SRC=/vmfs/volumes/STORAGE/template
DST=/vmfs/volumes/STORAGE/auto
mkdir -p $DST/$1

# Don't use cp to make copies of vmdk files. It'll just
# take ages longer as it will copy 0x0 bytes too.
# vmkfstools is what you should use instead
vmkfstools -i $SRC/DISK.vmdk $DST/$1/DISK.vmdk -d thin

# Replace some values in the destination VMX file
cat $SRC/TEMPLATE.vmx | sed s/TEMPLATE/$1/g > $DST/$1/$1.vmx

And now of course you add the guestinfo variables:

echo "guestinfo.HOSTN=$1" >> $DST/$1/$1.vmx
echo "guestinfo.EXTRA=$2" >> $DST/$1/$1.vmx

Now when the guest boots, you can make a script to read those guestinfo things out and let it for example configure itself (on the guest):

#! /bin/sh
HOSTN=`vmtoolsd --cmd "info-get guestinfo.HOSTN"`
EXTRA=`vmtoolsd --cmd "info-get guestinfo.EXTRA"`
if test "$EXTRA" = "provision"; then
   echo $HOSTN > /etc/hostname
   reboot
fi

Some other useful VMWare ESXi commands:

# Register the VMX as a new virtual machine
VIMID=`vim-cmd /solo/register $DST/$1/$1.vmx`

# Turn it on
vim-cmd /vmsvc/power.on $VIMID &

# Answer 'Copied' on the question whether it got
# copied or moved
sleep 2
VMMSG=`vim-cmd /vmsvc/message $VIMID | grep "Virtual machine message" | cut -d : -f -1 | cut -d " " -f 4`
if [ ! -z $VMMSG ]; then
    vim-cmd /vmsvc/message $VIMID $VMMSG 2
fi

That should be all you need. I’m sure we can adapt the $1.vmx file such that the question doesn’t get asked. But my solution with answering the question also worked for me.

Next thing we know you’re putting a loop around this and you just ‘programmed’ creating a few hundred Jenkins build agents on some powerful piece of ESXi equipment. Imagine that. Bread on the table and the entire flock of programmers of your customer happy.

But! Please don’t hire me to do your DevOps. I’ve been there before several times. It sucks. You get to herd brogrammers. They suck the blood out of you with their massive ignorance on almost all really simple standard things (like versioning, building, packaging, branching, etc. Anything that must not be invented here). Instead of people who take the time to be professional about their job and read five lines of documentation, they’ll waste your time with their nonsense self invented crap. Which you end up having to automate. Which they always make utterly impossible and (of course) non-standard. While the standard techniques are ten million times better and more easy.