Varnish Configuration

Below you will find detailed Varnish configuration recommendations for the features provided by this library. The configuration is provided for Varnish 3, 4 and 5.

Basic Varnish Configuration

To invalidate cached objects in Varnish, begin by adding an ACL (for Varnish 3 see ACL for Varnish 3) to your Varnish configuration. This ACL determines which IPs are allowed to issue invalidation requests. To use the provided configuration fragments, this ACL has to be named invalidators. The most simple ACL, valid for all Varnish versions from 3 onwards, looks as follows:

# /etc/varnish/your_varnish.vcl

acl invalidators {
    "localhost";
    # Add any other IP addresses that your application runs on and that you
    # want to allow invalidation requests from. For instance:
    # "192.168.1.0"/24;
}

Important

Make sure that all web servers running your application that may trigger invalidation are whitelisted here. Otherwise, lost cache invalidation requests will lead to lots of confusion.

Provided VCL Subroutines

In order to ease configuration we provide a set of VCL subroutines in the resources/config directory. These can be included from your main Varnish configuration file, typically default.vcl. Then you need to make your VCL_* subroutines call the fos_* routines.

Tip

When including one of the provided VCL, you need to call all the defined subroutines or your configuration will not be valid.

See the respective sections below on how to configure usage of each of the provided VCLs.

Purge

Purge removes a specific URL (including query strings) in all its variants (as specified by the Vary header).

Subroutines are provided in resources/config/varnish-[version]/fos_purge.vcl. To enable this feature, add the following to your_varnish.vcl:

  • Varnish 4 & 5
    include "path-to-config/varnish/fos_purge.vcl";
    
    sub vcl_recv {
        call fos_purge_recv;
    }
    
  • Varnish 3
    include "path-to-config/varnish-3/fos_purge.vcl";
    
    sub vcl_recv {
        call fos_purge_recv;
    }
    
    sub vcl_hit {
        call fos_purge_hit;
    }
    
    sub vcl_miss {
        call fos_purge_miss;
    }
    

Read more on handling PURGE requests in the Varnish documentation (for Varnish 3, see purging for Varnish 3).

Refresh

Refresh fetches a page from the backend even if it would still be in the cache, resulting in an updated cache without a cache miss on the next request.

Refreshing applies only to a specific URL including the query string, but not its variants.

Subroutines are provided in resources/config/varnish-[version]/fos_refresh.vcl. To enable this feature, add the following to your_varnish.vcl:

  • Varnish 4 & 5
    include "path-to-config/varnish/fos_refresh.vcl";
    
    sub vcl_recv {
        call fos_refresh_recv;
    }
    
  • Varnish 3
    include "path-to-config/varnish-3/fos_refresh.vcl";
    
    sub vcl_recv {
        call fos_refresh_recv;
    }
    

Read more on forcing a refresh in the Varnish documentation (for Varnish 3, see refreshing for Varnish 3).

Ban

Banning invalidates whole groups of cached entries with regular expressions.

Subroutines are provided in resources/config/varnish-[version]/fos_ban.vcl To enable this feature, add the following to your_varnish.vcl:

  • Varnish 4 & 5
    include "path-to-config/varnish/fos_ban.vcl";
    
    sub vcl_recv {
        call fos_ban_recv;
    }
    
    sub vcl_backend_response {
        call fos_ban_backend_response;
    }
    
    sub vcl_deliver {
        call fos_ban_deliver;
    }
    
  • Varnish 3
    include "path-to-config/varnish-3/fos_ban.vcl";
    
    sub vcl_recv {
        call fos_ban_recv;
    }
    
    sub vcl_fetch {
        call fos_ban_fetch;
    }
    
    sub vcl_deliver {
        call fos_ban_deliver;
    }
    

This subroutine also sets the X-Url and X-Host headers on the cache object. These headers are used by the Varnish ban lurker that crawls the content to eventually throw out banned data even when it’s not requested by any client. Read more on handling BAN requests in the Varnish documentation (for Varnish 3, see banning for Varnish 3).

Tagging

Feature: cache tagging

If you have included fos_ban.vcl, tagging will be automatically enabled with the X-Cache-Tags header for both marking the tags on the response and for the invalidation request to tell what tags to invalidate.

If you use a different name for response tagging than the default X-Cache-Tags or a different name for specifying which tags to invalidate in your cache invalidator configuration you have to write your own VCL code for tag invalidation. Your custom VCL will look like this:

  • Varnish 4 & 5
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    /*
     * This file is part of the FOSHttpCache package.
     *
     * (c) FriendsOfSymfony <http://friendsofsymfony.github.com/>
     *
     * For the full copyright and license information, please view the LICENSE
     * file that was distributed with this source code.
     */
    
    sub fos_ban_recv {
    
        if (req.method == "BAN") {
            if (!client.ip ~ invalidators) {
                return (synth(405, "Not allowed"));
            }
    
            if (req.http.X-Cache-Tags) {
                ban("obj.http.X-Host ~ " + req.http.X-Host
                    + " && obj.http.X-Url ~ " + req.http.X-Url
                    + " && obj.http.content-type ~ " + req.http.X-Content-Type
                    // the left side is the response header, the right side the invalidation header
                    + " && obj.http.X-Cache-Tags ~ " + req.http.X-Cache-Tags
                );
            } else {
                ban("obj.http.X-Host ~ " + req.http.X-Host
                    + " && obj.http.X-Url ~ " + req.http.X-Url
                    + " && obj.http.content-type ~ " + req.http.X-Content-Type
                );
            }
    
            return (synth(200, "Banned"));
        }
    }
    
    sub fos_ban_backend_response {
    
        # Set ban-lurker friendly custom headers
        set beresp.http.X-Url = bereq.url;
        set beresp.http.X-Host = bereq.http.host;
    }
    
    sub fos_ban_deliver {
    
        # Keep ban-lurker headers only if debugging is enabled
        if (!resp.http.X-Cache-Debug) {
            # Remove ban-lurker friendly custom headers when delivering to client
            unset resp.http.X-Url;
            unset resp.http.X-Host;
    
            # Unset the tagged cache headers
            unset resp.http.X-Cache-Tags;
        }
    }
    
  • Varnish 3
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    /*
     * This file is part of the FOSHttpCache package.
     *
     * (c) FriendsOfSymfony <http://friendsofsymfony.github.com/>
     *
     * For the full copyright and license information, please view the LICENSE
     * file that was distributed with this source code.
     */
    
    sub fos_ban_recv {
    
        if (req.request == "BAN") {
            if (!client.ip ~ invalidators) {
                error 405 "Not allowed.";
            }
    
            if (req.http.X-Cache-Tags) {
                ban("obj.http.X-Host ~ " + req.http.X-Host
                    + " && obj.http.X-Url ~ " + req.http.X-Url
                    + " && obj.http.content-type ~ " + req.http.X-Content-Type
                    // the left side is the response header, the right side the invalidation header
                    + " && obj.http.X-Cache-Tags ~ " + req.http.X-Cache-Tags
                );
            } else {
                ban("obj.http.X-Host ~ " + req.http.X-Host
                    + " && obj.http.X-Url ~ " + req.http.X-Url
                    + " && obj.http.content-type ~ " + req.http.X-Content-Type
                );
            }
    
            error 200 "Banned";
        }
    }
    
    sub fos_ban_fetch {
    
        # Set ban-lurker friendly custom headers
        set beresp.http.X-Url = req.url;
        set beresp.http.X-Host = req.http.host;
    }
    
    sub fos_ban_deliver {
    
        # Keep ban-lurker headers only if debugging is enabled
        if (!resp.http.X-Cache-Debug) {
            # Remove ban-lurker friendly custom headers when delivering to client
            unset resp.http.X-Url;
            unset resp.http.X-Host;
    
            # Unset the tagged cache headers
            unset resp.http.X-Cache-Tags;
        }
    }
    

Hint

The line you need to adjust from the code above is line 21. The left side is the header used to tag the response, the right side is the header used when sending invalidation requests. If you change one or the other header name, make sure to adjust the configuration accordingly.

User Context

Feature: user context hashing

The fos_user_context.vcl needs the user_context_hash_url subroutine that sets the URL to do the hash lookup. The default URL is /_fos_user_context_hash and you can simply include resources/config/varnish-[version]/fos_user_context_url.vcl in your configuration to provide this. If you need a different URL, write your own user_context_hash_url subroutine instead.

Tip

The provided VCL to fetch the user hash restarts GET/HEAD requests. It would be more efficient to do the hash lookup request with curl, using the curl Varnish plugin. If you can enable curl support, the recommended way is to implement your own VCL to do a curl request for the hash lookup instead of using the VCL provided here.

Also note that restarting a GET request leads to Varnish discarding the body of the request. If you have some special case where you have GET requests with a body, use curl.

To enable this feature, add the following to your_varnish.vcl:

  • Varnish 4 & 5
    include "path-to-config/varnish/fos_user_context.vcl";
    include "path-to-config/varnish/fos_user_context_url.vcl";
    
    sub vcl_recv {
        call fos_user_context_recv;
    }
    
    sub vcl_backend_response {
        call fos_user_context_backend_response;
    }
    
    sub vcl_deliver {
        call fos_user_context_deliver;
    }
    
  • Varnish 3
    include "path-to-config/varnish-3/fos_user_context.vcl";
    include "path-to-config/varnish/fos_user_context_url.vcl";
    
    sub vcl_recv {
        call fos_user_context_recv;
    }
    
    sub vcl_fetch {
        call fos_user_context_fetch;
    }
    
    sub vcl_deliver {
        call fos_user_context_deliver;
    }
    

Your backend application needs to respond to the application/vnd.fos.user-context-hash request with a proper user hash.

Tip

The provided VCL assumes that you want the context hash to be cached, so we set the req.url to a fixed URL. Otherwise Varnish would cache every hash lookup separately.

However, if you have a paywall scenario, you need to leave the original URL unchanged. For that case, you would need to write your own VCL.

Custom TTL

By default, the proxy server looks at the s-maxage instruction in the Cache-Control header to know for how long it should cache a page. But the Cache-Control header is also sent to the client. Any caches on the Internet, for example the Internet provider or from a cooperate network might look at s-maxage and cache the page. This can be a problem, notably when you do explicit cache invalidation. In that scenario, you want your proxy server to keep a page in cache for a long time, but caches outside your control must not keep the page for a long duration.

One option could be to set a high s-maxage for the proxy and simply rewrite the response to remove or reduce the s-maxage. This is not a good solution however, as you start to duplicate your caching rule definitions.

The solution to this issue provided here is to use a separate, different header called X-Reverse-Proxy-TTL that controls the TTL of the proxy server to keep s-maxage for other proxies. Because this is not a standard feature, you need to add configuration to your proxy server.

Subroutines are provided in resources/config/varnish-[version]/fos_custom_ttl.vcl. The configuration needs to use inline C, which is disabled by default since Varnish 4.0. To use the custom TTL feature, you need to start your Varnish with inline C enabled: -p vcc_allow_inline_c=on. Then add the following to your_varnish.vcl:

  • Varnish 4 & 5
    include "path-to-config/varnish/fos_custom_ttl.vcl";
    
    sub vcl_backend_response {
        call fos_custom_ttl_backend_response;
    }
    
  • Varnish 3
    include "path-to-config/varnish-3/fos_custom_ttl.vcl";
    
    sub vcl_fetch {
        call fos_custom_ttl_fetch;
    }
    

The custom TTL header is removed before sending the response to the client.

Debugging

Configure your Varnish to set a custom header (X-Cache) that shows whether a cache hit or miss occurred. This header will only be set if your application sends an X-Cache-Debug header:

Subroutines are provided in fos_debug.vcl.

To enable this feature, add the following to your_varnish.vcl:

  • Varnish 4 & 5
    include "path-to-config/varnish/fos_debug.vcl";
    
    sub vcl_deliver {
        call fos_debug_deliver;
    }
    
  • Varnish 3
    include "path-to-config/varnish-3/fos_debug.vcl";
    
    sub vcl_deliver {
        call fos_debug_deliver;
    }