Determining the real client IP with Varnish (w/ X-Forwarded-For)

If you implement Varnish in your application one of the early things that you discover is that  any IP functionalities you had are now gone. Some examples are:

• GEOIP does not resolve the country
• Apache logs write 127.0.0.1 as request client IP (or another IP of your LAN)
• My own PHP logic cannot longer apply filters by IP

Why? Well, it's easy to answer. That's because your Apache or Ngnix will receive the request from Varnish, not from the real client as it used to be.

But that has an easy fix. All you need to do is to tell Varnish to send the client IP to the backend. This can be easily accomplished by setting in your VLC (/etc/varnish/default.vcl) the following:

sub vcl_recv {

# IP forwarding.
if (req.restarts == 0) {
        if (req.http.x-forwarded-for) {
                set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
        } else {
                set req.http.X-Forwarded-For = client.ip;
        }
}
...

With this change your  application will be able to get the client IP and resolve correctly the country using GeoIP. When your application needs the client IP you only have to access to the header X-Forwareded-For.

But caution! If you ALWAYS want that header set in every request, you need to add this:

 sub vcl_pipe {
     # Note that only the first request to the backend will have
     # X-Forwarded-For set.  If you use X-Forwarded-For and want to
     # have it set for all requests, make sure to have:
     # set bereq.http.connection = "close";
     # here.  It is not set by default as it might break some broken web
     # applications, like IIS with NTLM authentication.

     set bereq.http.connection = "close";
     return (pipe);
 }

Now,  you might want to see the original IP in the apache logs too. To do that you have to use a custom log that prints the X-Forwareded-For header. Just add inside your Virtualhost declaration:

LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" varnish
CustomLog /var/log/httpd/access_combined_SOMEDOMAIN.log varnish

You should also have in mind that this configuration is appended, so it might happen that you vcl has already a line like this, so you are doing it twice. See the commented code in the default.vcl file.

The latest VCL file includes already the client.ip, just that it might appear twice! Copy all the file and tune the lines you like

Mine looks more or less like this

sub vcl_recv {

    # IP forwarding.
    if (req.restarts == 0) {
        if (req.http.x-forwarded-for) {
            set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }

     # Ignore all requests from Working Copies
     if ( req.http.host ~ "([a-z]{2,3}\.development\.lan)$" )
     {
        return(pipe);
     }

     unset req.http.Cookie;

#Copy paste...

     if (req.request != "GET" &&
       req.request != "HEAD" &&
       req.request != "PUT" &&
       req.request != "POST" &&
       req.request != "TRACE" &&
       req.request != "OPTIONS" &&
       req.request != "DELETE") {
         /* Non-RFC2616 or CONNECT which is weird. */
         return (pipe);
     }
     if (req.request != "GET" && req.request != "HEAD") {
         /* We only deal with GET and HEAD by default */
         return (pass);
     }
     if (req.http.Authorization || req.http.Cookie) {
         /* Not cacheable by default */
         return (pass);
     }
     return (lookup);

 }
 sub vcl_pipe {
     # Note that only the first request to the backend will have
     # X-Forwarded-For set.  If you use X-Forwarded-For and want to
     # have it set for all requests, make sure to have:
     # set bereq.http.connection = "close";
     # here.  It is not set by default as it might break some broken web
     # applications, like IIS with NTLM authentication.

     set bereq.http.connection = "close";
     return (pipe);
 }

After that Varnish will shine again.