Ya' row ze: Lets go!

You are here

Adding MD5 support to Varnish using the DIGEST library

I have a need to hash a few header values together in varnish to verify a security key before either presenting from cache, or passing the request through to origin. The downside is, varnish doesn't support md5 (or any crypto functions for that matter) out of the box. There appears to be a mod library for varnish that seems to be reasonably supported by the organization, so doing some research I found this library - https://github.com/varnish/libvmod-digest .

After pulling the source and reading instructions in the README file, this looks like it could be a chore.

Installation requires the Varnish source tree (only the source matching the binary installation).

./autogen.sh (for git-installation)
./configure VARNISHSRC=/path/to/your/varnish/source/varnish-cache
make install (may require root: sudo make install)
make check (Optional for regression tests)
VARNISHSRCDIR is the directory of the Varnish source tree for which to compile your vmod. Both the VARNISHSRCDIR and VARNISHSRCDIR/include will be added to the include search paths for your module.

Optionally you can also set the vmod install dir by adding VMODDIR=DIR (defaults to the pkg-config discovered directory from your Varnish installation)

This sounds like I'm going to have to do a few things:

Get the Varnish version

varnishd -V
varnishd (varnish-3.0.4 revision 9f83e8f)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2011 Varnish Software AS

Get the Source for Varnish [version]
At the time of writing, the source can be found here: https://repo.varnish-cache.org/source/ . for my particular version of varnish (3.0.4) the link is here: https://repo.varnish-cache.org/source/varnish-3.0.4.tar.gz

*note that just getting the source isn't enough. As you can see from the following message, you will also have to build the varnish source with "make". Just leave out the "make install" option so you don't overwrite your existing macports version of Varnish.

checking for /opt/local/etc/varnish/source/varnish-3.0.4/bin/varnishtest/varnishtest... no
configure: error: in `/opt/local/etc/varnish/source/libvmod-digest-3.0':
configure: error: Can't find "/opt/local/etc/varnish/source/varnish-3.0.4/bin/varnishtest/varnishtest". Please build your varnish source directory
See `config.log' for more details

To take care of this, do the following from the varnish source directory:

#that's it, no need to install!

Build Digest against Varnish
Now that the Varnish source has been downloaded and built, its time to switch back to the libvmod-digest-3.0 directory to build that library. This should be as simple as:

./configure VARNISHSRC=../varnish-3.0.4
make install

But... natuarally it's not quite that simple.
First Snag

When trying to run the ./configure command, I hit a snag:

checking dynamic linker characteristics... darwin14.1.0 dyld
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking whether make sets $(MAKE)... (cached) yes
checking for mhash_count in -lmhash... no
configure: error: libvmod-digest requires libmhash.

It looks like "libmhash" is a required library that is not on my system. It also appears that there is no macport for libmhash, so now I need to build that from source too.

I tracked down the home of the library to: http://sourceforge.net/projects/mhash/ and found this file: http://iweb.dl.sourceforge.net/project/mhash/mhash/

the instructions in the INSTALL file show me this:

The simplest way to compile this package is:

1. `cd' to the directory containing the package's source code and type
`./configure' to configure the package for your system. If you're
using `csh' on an old version of System V, you might need to type
`sh ./configure' instead to prevent `csh' from trying to execute
`configure' itself.

Running `configure' takes awhile. While running, it prints some
messages telling which features it is checking for.

2. Type `make' to compile the package.

3. Optionally, type `make check' to run any self-tests that come with
the package.

4. Type `make install' to install the programs and any data files and

This seems simple enough, but when I attempted to run the .configure, for this one, I hit a strange snag. (I'm guessing it was probably just me doing something crazy, but I'm not sure how) so here's what happened anyways.

Snag when building libmhash:
root# ./configure
checking build system type... i386-apple-darwin14.1.0
checking host system type... i386-apple-darwin14.1.0
checking target system type... i386-apple-darwin14.1.0
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
configure: error: source directory already configured; run "make distclean" there first

running "make distclean" seems to have no effect:
root# make distclean
make: *** No rule to make target `distclean'. Stop.

It seems that this was because I somehow landed in the wrong directory (not sure how i pulled that off). traversing back to the mhash- directory and then running "make distclean" seemed to help. I was then able to successfully run the expected commands:

make install

Now, with mhash built, I can go attempt to build the digest-3.0 library and hopefully install it to my active varnish instance.

Build Digest-3.0 and apply it to the macports varnish instance

This turned out to be easier than expected once all the dependencies were met. not only did the expected sequence of commands work:

./configure VARNISHSRC=../varnish-3.0.4
make install

I sort of expected that "make install" would wind up putting the lib files in the wrong place - specifically the varnish-3.0.4 source/build folder I had created earlier. Instead however, the build somehow detected my actual installed version of varnish, and correctly copied the extension files over to the right location!

Nothing is ever *this easy* so giving this odds of success around 10%, I fired up varnish, and loaded the VCL I've been working on, and rather than get the compilation errors previously returned, to my amazement, it fired right up!

Thanks interewebs!

Back to top