Varnish Configuration

Below you will find detailed Varnish configuration recommendations for the features provided by this library. The configuration is provided for Varnish 4 and newer, and - when it is different - for Varnish 3.

Basic Varnish Configuration

To invalidate cached objects in Varnish, begin by adding an ACL (for Varnish 3 see ACL for Varnish 3) to your Varnish configuration. This ACL determines which IPs are allowed to issue invalidation requests. To use the provided configuration fragments, this ACL has to be named invalidators. The most simple ACL, valid for Varnish version 3 or better, looks as follows:

# /etc/varnish/your_varnish.vcl

acl invalidators {
    "localhost";
    # Add any other IP addresses that your application runs on and that you
    # want to allow invalidation requests from. For instance:
    # "192.168.1.0"/24;
}

Important

Make sure that all web servers running your application that may trigger invalidation are whitelisted here. Otherwise, lost cache invalidation requests will lead to lots of confusion.

Provided VCL Subroutines

In order to ease configuration we provide a set of VCL subroutines in the resources/config directory of FOSHttpCache. These can be included from your main Varnish configuration file, typically default.vcl. Then you need to make your vcl_* subroutines call the fos_* routines.

Tip

When including one of the provided VCL, you need to call all the defined subroutines or your configuration will not be valid.

See the respective sections below on how to configure usage of each of the provided VCLs.

Purge

Purge removes a specific URL (including query strings) in all its variants (as specified by the Vary header).

Subroutines are provided in resources/config/varnish-[version]/fos_purge.vcl. To enable this feature, add the following to your_varnish.vcl:

Varnish 4-6Varnish 3

include "path-to-config/varnish/fos_purge.vcl";

sub vcl_recv {
    call fos_purge_recv;
}

Read more on handling PURGE requests in the Varnish documentation (for Varnish 3, see purging for Varnish 3).

Refresh

Refresh fetches a page from the backend even if it would still be in the cache, resulting in an updated cache without a cache miss on the next request.

Refreshing applies only to a specific URL including the query string, but not its variants.

Subroutines are provided in resources/config/varnish-[version]/fos_refresh.vcl. To enable this feature, add the following to your_varnish.vcl:

Varnish 4-6Varnish 3

include "path-to-config/varnish/fos_refresh.vcl";

sub vcl_recv {
    call fos_refresh_recv;
}

Read more on forcing a refresh in the Varnish documentation (for Varnish 3, see refreshing for Varnish 3).

Ban

Banning invalidates whole groups of cached entries with regular expressions.

Subroutines are provided in resources/config/varnish-[version]/fos_ban.vcl To enable this feature, add the following to your_varnish.vcl:

Varnish 4-6Varnish 3

include "path-to-config/varnish/fos_ban.vcl";

sub vcl_recv {
    call fos_ban_recv;
}

sub vcl_backend_response {
    call fos_ban_backend_response;
}

sub vcl_deliver {
    call fos_ban_deliver;
}

This subroutine also sets the X-Url and X-Host headers on the cache object. These headers are used by the Varnish ban lurker that crawls the content to eventually throw out banned data even when it’s not requested by any client. Read more on handling BAN requests in the Varnish documentation (for Varnish 3, see banning for Varnish 3).

Tagging

Feature: cache tagging

For this feature you need to choose between either BAN _(default)_ and the more performant xkey based tag system.

Tag Invalidation Using xkey

Since Varnish 4.1, you can use the official xkey Varnish module for better performance with cache tags.

Tip

If you are using the payed “Varnish Plus” offering, you should use the improved ykey feature instead of xkey. The ykey documentation explains how to use the xkey headers to set up the ykey information. Instead of including the fos_tags_xkey.vcl you would need to write your own VCL to handle ykey tagging and invalidation requests.

As explained in the Cache Invalidation chapter of the Varnish documentation:: > (..), hashtwo/xkey is much more efficient than bans because of two reasons: > 1) looking up hash keys is much more efficient than traversing ban-lists, and > 2) every time you test a ban expression, it checks every object in the cache > that is older than the ban itself.

With xkey, you can also soft purge tagged responses to allow for a grace period where invalidated (stale) content is still delivered to the client.

Install the varnish modules with your distribution if available, or refer to the Varnish documentation for installing xkey. Version 0.10.2 is the first version to support purging more than one tag at once. With older versions you can only invalidate one tag at a time, and soft purge is not available.

To use xkey, configure the Varnish Client for xkey and the response tagger to use the xkey header, and include resources/config/varnish/fos_tags_xkey.vcl in your VCL:

include "path-to-config/varnish/fos_tags_xkey.vcl";

sub vcl_recv {
    call fos_tags_xkey_recv;
}

sub vcl_deliver {
    call fos_tags_xkey_deliver;
}

Note that there is no xkey VCL file for Varnish version 3 because the varnish-modules are only available for Varnish 4.1 or newer.

Tag Invalidation Using BAN

If you have included fos_ban.vcl, tagging will be automatically enabled with the X-Cache-Tags header for both marking the tags on the response and for the invalidation request to tell what tags to invalidate.

If you use a different name for response tagging than the default X-Cache-Tags or a different name for specifying which tags to invalidate in your cache invalidator configuration you have to write your own VCL code for tag invalidation. Your custom VCL will look like this:

Varnish 4-6Varnish 3

.. literalinclude:: ../resources/config/varnish/fos_ban.vcl
    :language: C
    :emphasize-lines: 17-23,50-51
    :linenos:

Hint

The line you need to adjust from the code above is line 21. The left side is the header used to tag the response, the right side is the header used when sending invalidation requests. If you change one or the other header name, make sure to adjust the configuration accordingly.

User Context

Feature: user context hashing

The fos_user_context.vcl needs the user_context_hash_url subroutine that sets the URL to do the hash lookup. The default URL is /_fos_user_context_hash and you can simply include resources/config/varnish-[version]/fos_user_context_url.vcl in your configuration to provide this. If you need a different URL, write your own user_context_hash_url subroutine instead.

Tip

The provided VCL to fetch the user hash restarts GET/HEAD requests. It would be more efficient to do the hash lookup request with curl, using the curl Varnish plugin. If you can enable curl support, the recommended way is to implement your own VCL to do a curl request for the hash lookup instead of using the VCL provided here.

Also note that restarting a GET request leads to Varnish discarding the body of the request. If you have some special case where you have GET requests with a body, use curl.

To enable this feature, add the following to your_varnish.vcl:

Varnish 4-6Varnish 3

include "path-to-config/varnish/fos_user_context.vcl";
include "path-to-config/varnish/fos_user_context_url.vcl";

sub vcl_recv {
    call fos_user_context_recv;
}

sub vcl_hash {
    call fos_user_context_hash;
}

sub vcl_backend_response {
    call fos_user_context_backend_response;
}

sub vcl_deliver {
    call fos_user_context_deliver;
}

Caching User Specific Content

By default, Varnish does not check for cached data as soon as the request has a Cookie or Authorization header, as per the builtin VCL (for Varnish 3, see default VCL). For the user context, you make Varnish cache even when there are credentials present.

You need to be very careful when doing this: Your application is responsible for properly specifying what may or may not be shared. If a content only depends on the hash, Vary on the header containing the hash and set a Cache-Control header to make Varnish cache the request. If the response is individual however, you need to Vary on the Cookie and/or Authorization header and probably want to send a header like Cache-Control: s-maxage=0 to prevent Varnish from caching.

Your backend application needs to respond to the application/vnd.fos.user-context-hash request with a proper user hash.

Tip

The provided VCL assumes that you want the context hash to be cached, so we set the req.url to a fixed URL. Otherwise Varnish would cache every hash lookup separately.

The fos_user_context_hash should be used to separate the cache of the hash lookup. If you don’t do that, Varnish can run into performance issues because the user hash lookup creates a large number of variants. If your hash is taking into account other headers than Authorization and Cookie, create your own vcl_hash function that adds all those headers to hash_data for user context hash lookup requests.

However, if you have a paywall scenario, you need to leave the original URL unchanged. For that case, you would need to write your own VCL.

Cleaning the Cookie Header

In the examples above, an unaltered Cookie header is passed to the backend to use for determining the user context hash. However, cookies as they are sent by a browser are unreliable. For instance, when using Google Analytics, cookie values are different for each request. Because of this, the hash request would not be cached, but multiple hashes would be generated for one and the same user.

To make the hash request cacheable, you must extract a stable user session id before calling fos_user_context_recv. You can do this as explained in the Varnish documentation:

sub vcl_recv {
    # ...

    set req.http.cookie = ";" + req.http.cookie;
    set req.http.cookie = regsuball(req.http.cookie, "; +", ";");
    set req.http.cookie = regsuball(req.http.cookie, ";(PHPSESSID)=", "; \1=");
    set req.http.cookie = regsuball(req.http.cookie, ";[^ ][^;]*", "");
    set req.http.cookie = regsuball(req.http.cookie, "^[; ]+|[; ]+$", "");

    # ...
}

Note

If your application’s user authentication is based on a cookie other than PHPSESSID, change PHPSESSID to your cookie name.

Custom TTL

By default, the proxy server looks at the s-maxage instruction in the Cache-Control header to know for how long it should cache a page. But the Cache-Control header is also sent to the client. Any caches on the Internet, for example the Internet provider or from a cooperate network might look at s-maxage and cache the page. This can be a problem, notably when you do explicit cache invalidation. In that scenario, you want your proxy server to keep a page in cache for a long time, but caches outside your control must not keep the page for a long duration.

One option could be to set a high s-maxage for the proxy and simply rewrite the response to remove or reduce the s-maxage. This is not a good solution however, as you start to duplicate your caching rule definitions.

The solution to this issue provided here is to use a separate, different header called X-Reverse-Proxy-TTL that controls the TTL of the proxy server to keep s-maxage for other proxies. Because this is not a standard feature, you need to add configuration to your proxy server.

Subroutines are provided in resources/config/varnish-[version]/fos_custom_ttl.vcl. Add the following to your_varnish.vcl:

Varnish 4-6Varnish 3

include "path-to-config/varnish/fos_custom_ttl.vcl";

sub vcl_backend_response {
    call fos_custom_ttl_backend_response;
}

The custom TTL header is removed before sending the response to the client.

Note

If you are using Varnish 3, this feature is using inline C. Inline C is enabled for Varnish 3 by default. Check for the vcc_allow_inline_c setting. If you are using Varnish 4 or newer, you are using the varnish/fos_custom_ttl.vcl which uses a Varnish function instead of inline C.

Debugging

Configure your Varnish to set a custom header (X-Cache) that shows whether a cache hit or miss occurred. This header will only be set if your application sends an X-Cache-Debug header:

Subroutines are provided in fos_debug.vcl.

To enable this feature, add the following to your_varnish.vcl:

Varnish 4-6Varnish 3

include "path-to-config/varnish/fos_debug.vcl";

sub vcl_deliver {
    call fos_debug_deliver;
}