Report inadequate content

Determining the real client IP with Varnish (w/ X-Forwarded-For)

{
}

If you implement Varnish in your application one of the early things that you discover is that  any IP functionalities you had are now gone. Some examples are:

  • GEOIP does not resolve the country
  • Apache logs write 127.0.0.1 as request client IP (or another IP of your LAN)
  • My own PHP logic cannot longer apply filters by IP

Why? Well, it's easy to answer. That's because your Apache or Ngnix will receive the request from Varnish, not from the real client as it used to be.

But that has an easy fix. All you need to do is to tell Varnish to send the client IP to the backend. This can be easily accomplished by setting in your VLC (/etc/varnish/default.vcl) the following:

sub vcl_recv {

# IP forwarding.
if (req.restarts == 0) {
        if (req.http.x-forwarded-for) {
                set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
        } else {
                set req.http.X-Forwarded-For = client.ip;
        }
}
...
UPDATE: IP forwarding code updated to make it work with people behind proxies, as noted by Albert and Zillur in the comments.

With this change your  application will be able to get the client IP and resolve correctly the country using GeoIP. When your application needs the client IP you only have to access to the header X-Forwareded-For.

But caution! If you ALWAYS want that header set in every request, you need to add this:

 sub vcl_pipe {
     # Note that only the first request to the backend will have
     # X-Forwarded-For set.  If you use X-Forwarded-For and want to
     # have it set for all requests, make sure to have:
     # set bereq.http.connection = "close";
     # here.  It is not set by default as it might break some broken web
     # applications, like IIS with NTLM authentication.

     set bereq.http.connection = "close";
     return (pipe);
 }

Now,  you might want to see the original IP in the apache logs too. To do that you have to use a custom log that prints the X-Forwareded-For header. Just add inside your Virtualhost declaration:

LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" varnish
CustomLog /var/log/httpd/access_combined_SOMEDOMAIN.log varnish

You should also have in mind that this configuration is appended, so it might happen that you vcl has already a line like this, so you are doing it twice. See the commented code in the default.vcl file.

The latest VCL file includes already the client.ip, just that it might appear twice! Copy all the file and tune the lines you like

Mine looks more or less like this

sub vcl_recv {

    # IP forwarding.
    if (req.restarts == 0) {
        if (req.http.x-forwarded-for) {
            set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }

     # Ignore all requests from Working Copies
     if ( req.http.host ~ "([a-z]{2,3}\.development\.lan)$" )
     {
        return(pipe);
     }

     unset req.http.Cookie;

#Copy paste...

     if (req.request != "GET" &&
       req.request != "HEAD" &&
       req.request != "PUT" &&
       req.request != "POST" &&
       req.request != "TRACE" &&
       req.request != "OPTIONS" &&
       req.request != "DELETE") {
         /* Non-RFC2616 or CONNECT which is weird. */
         return (pipe);
     }
     if (req.request != "GET" && req.request != "HEAD") {
         /* We only deal with GET and HEAD by default */
         return (pass);
     }
     if (req.http.Authorization || req.http.Cookie) {
         /* Not cacheable by default */
         return (pass);
     }
     return (lookup);

 }
 sub vcl_pipe {
     # Note that only the first request to the backend will have
     # X-Forwarded-For set.  If you use X-Forwarded-For and want to
     # have it set for all requests, make sure to have:
     # set bereq.http.connection = "close";
     # here.  It is not set by default as it might break some broken web
     # applications, like IIS with NTLM authentication.

     set bereq.http.connection = "close";
     return (pipe);
 }

After that Varnish will shine again.

{
}
{
}

Comments Determining the real client IP with Varnish (w/ X-Forwarded-For)

I have racked my brain over this for days. There is virtually noting usable about this issue on the web, except for what you provide here. YOU DESERVE A MEDAL !!!
Although I do have a question: "Does this impact performance, and if so, who much and why?
CONGRATS ...
BIG TOE BIG TOE 14/12/2013 at 05:49
For getting client ip in application like PHP, ASP visit getting-real-client-ip-through-varnish/
zillur rahman zillur rahman 28/06/2014 at 15:54
Good post . I also wrote an article on same topics but more elaborately. Please visit my blog Getting real client IP through Varnish http://www.techinfobest.com/getting-real-client-ip-through-varnish/
zillur rahman zillur rahman 06/07/2014 at 18:39
Hi! We've recently found some limitations of this implementation: since we are overwritting the value of x-forwarded-for variable, we are in fact losing information. For those browsing behind proxies, we will recover the proxy IP instead of the real client IP.
This code should do the work, concatenating the actual value of x-forwarded-for, if exists to client.ip value:
# IP forwarding.
if (req.restarts == 0) {
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
Zillur's article explains a little bit more about this. Albert, update the content: your article is so well positioned in Google results. ;-)
Updated the article with your suggestions. Nice one :)

Leave your comment Determining the real client IP with Varnish (w/ X-Forwarded-For)

Log in to Obolog, or create your free blog if you are not registered yet.

User avatar Your name