Archive for April, 2008

Chunked Decoding

April 29, 2008

We have been getting a growing number of reports and bugs from people using Squid 3.0 described as ‘squid producing a blank page’ when bypassing squid apparently works.

Sounds familiar to some yes? I’m bringing it up now because while it is an old problem, its not the TCP issues Adrian wrote about earlier and you should also check if you find its not this. Which incidentally can have exactly the same visible effects for end-users.

This ‘new’ issue is caused by certain widely-used web servers which shall remain nameless and unadvertised by me. Which always respond with HTTP/1.1 chunked-encoding of pages.

Servers are explicitly forbidden from sending that particular encoding type to software announcing itself as HTTP/1.0 (such as squid). But the broken server is doing it anyway!

Ironically: The authors use this server on their own help and support website. So those who are having this problem both see it as a squid problem, and can’t find or see any solution they may have posted anyway.

How to tell if this is your problem?

Use squidclient to make a web request that bypasses the squid proxy. It should send out the HTTP/1.0 request and get a page back. If the headers of the response include “Transfer-Encoding: chunked” there is your problem.

This is currently only an issue in Squid 2.5 or earlier and 3.0, which is still highly modeled around 2.5.

The solutions are varied depending on your capabilities.

Simplest for some will be to just bypass squid for those domains.

[ UPDATE: (thanks Michael Graham)

Apparently several people are having success with simply dropping the Accept-Encoding header to certain of these broken servers. Adding this to their squid.conf :

# Fix broken sites by removing Accept-Encoding header
acl broken dstdomain …
request_header_access Accept-Encoding deny broken

NP: don’t forget to remove it again when you upgrade out of 3.0

]

Next best is to use peer-routing to divert those domain requests at a squid 2.6 (or if you are feeling experimental a 3.1 build)

If its a serious issue and you are accelerating for one of these broken web servers. Then you will need to stick with Squid 2.6 until 3.1 is available for production use.

Why does it work for 2.6 and 3.1 but not 3.0?

Well, things are a bit messy I’ll have to write it up one day. Suffice to say that 3.1 has a lot more HTTP/1.1 support where the chunked-encoding/decoding was intended for. But 2.6 needed it a bit earlier so a version of the decoding (only!) was done to fit 2.6 needs at solving this same issue for high-performance users earlier last year.

The 3.0 code is just different enough that it would need a whole new back-port project to get it going well. The time and work that would take is being used instead to get 3.1 out faster. Which should be within a month of this writing so procrastinating could solve the problem for you.

[UPDATE: Thanks to the Gentoo Project for their work back-porting this will be available from 3.0.STABLE16-RC1 ]

Advertisements

Squid-2.6 + TPROXY + Debian

April 7, 2008

Jason Healy posted some useful information to the squid-users list a week or so ago.

Quoting:

I’ve been a happy user of Squid for the past 10 years or so, and I’d like to take a second to thank everyone who has worked so hard to make such a great piece of software!  I’d like to give back to the Squid community, but unfortunately I’m not much of a C hacker.  However, I’m hoping I can still help.

I’ve just spent a few days getting my school’s Squid install up to date (we were running 2.5 on Debian Woody).  I switched to using tproxy this time around (we used to do policy routing on our core, but it was spiking the CPU too much).  Thanks to the mailing list, some articles on the web, and a little messing around I was able to get the whole system up and running.  I’ve documented the steps here:

http://web.suffieldacademy.org/ils/netadmin/docs/software/squid/

The document is written for someone with a decent grasp of Linux, and is specifically geared to Debian Etch.  There are some tweaks that are pecific to our install (compile-time flags, mostly), but otherwise it’s pretty generic.  Hopefully, this will help someone else out who’s trying to build a similar system, so I’m posting so it will hit the archives.

Squid Updates – April 2008

April 6, 2008

University studies have begun for me and so my available time has been limited. But to summarise:

  • Squid-3.0 has been released, for people who are interested in playing with it
  • Kinkie has updated the Wiki theme in a big way – http://wiki.squid-cache.org/
  • Squid-3 development has migrated to bzr
  • Alex is looking to merge in the first set of eCAP related changes into Squid-3.HEAD
  • Squid-2.7 is on track to be released – there’s one outstanding bug and its unfortunately difficult to fix. http://www.squid-cache.org/bugs/show_bug.cgi?id=2160 is the bug to watch.
  • Funded Squid-2 development will continue for the time being; mostly from projects I’m working on. We’ll see how things progress there. The Squid-2 Roadmap is slowly changing, evolving and being completed.