Don’t upgrade to Squid-2.6.STABLE15, skip straight to Squid-2.6.STABLE16

The title pretty much says it all. Squid-2.6.STABLE15 has a number of important fixes to the previous version but has a couple of teething problems and may be unstable in the real world.

Leave Squid-2.6.STABLE15 right well alone and wait until Henrik rolls the Squid-2.6.STABLE16 release.

(You can blame me for one of the bugs introduced into Squid-2.6.STABLE14 if you run select(); oops. Henrik’s corrected that in STABLE15 and STABLE16.)

If you run NTLM authentication then please upgrade to STABLE16 and report any bugs or bad behaviour you see. Some people have reported broken behaviour between earlier squid-2.6 STABLE versions and the current STABLE release - if you just stay on the old version and don’t tell us why you’re doing it we can’t fix the bugs!

Reverse Proxying with Squid

A Squid user posted about their little “CDN” installation to speed up their content delivery to the clients of a particular ISP.

You can read more about it here.

Blocking Ads in Squid

One of the more bandwidth-intensive “features” of the Web is the proliferation of ad images and flash media which has a nasty habit of wasting bandwidth and increasing loading times.

Squid has been able to filter ads and other unwanted media for a number of years. Various articles have been written to cover how exactly its done and so I won’t bother covering the how-to here.

The original method involved the “redirector”. A redirector was simply an external program which would read in URLs on STDIN and spit out “alternate” URLs on STDOUT. This could be used for a number of things - the initial use being to rewrite URLs when using Squid as a web server accelerator - but people quickly realised they could rewrite “ad” URLs to filter them out.

Another method is to simply build a text file with identified ad content URLs and hostnames and simply deny the traffic. This is simple but can scale poorly if you try filtering thousands of URLs against regular expression matches.

Finally, another method involves using the more recent “external ACL” helper. It is an external program which can be passed a variety of information about a request (URL, client IP, authenticated username, arbitrary HTTP headers, ident to name a few, but its very customizable!) and spit back a YES or a NO, with an optional message. Content can then be filtered by simply denying access to it, but it currently doesn’t let you return modified content. One of the most popular uses of the external ACL helper is actually to implement ACL groups from sources like LDAP/Windows Active Directory.

How you do it is up to you. Here’s a few links explaining whats involved.

• Tutorial: How to block Ads with AdZap
• Example: Basic Squid Setup with File, Domain and Ad Blocking
• Example: Block Banner Ads Now!
• Example: Squid w/ plenty of External ACL file example
• Squid Wiki: Squid Redirectors
• Squid Wiki: Squid ACLs

Web Cache Whitepapers/Articles

Why bother with Squid as a purely proxy server? Isn’t most of the content on the Internet today dynamic?

Perhaps; perhaps not. A few years ago “media caching” required licenced software to handle WMA and RealMedia streams; today the heavy bandwidth users are flash videos from popular sites such as YouTube. The HTML may not be cachable but all those thumbnail images, all those previews and all those large flash video files are very cachable. The problem isn’t that the Internet is “dynamic”; the problem is that website designers view caching as “evil” - they’re suddenly not 100% in control of their content - and try as hard as possible to dodge caching.

Squid has a few knobs which can be set to cache this so-called “dynamic” content. Squid has to treat everything which may be dynamic as uncacheable - the telltail “?” in the URL identifying the output as being from a script - when in fact the content isn’t all that dynamic. More on that will be covered in a future article.

ISPs who run Squid with a well-tuned configuration have shown web traffic savings of around 30%. Thats 30% of their traffic, not just hits. And thats not with any attempt at caching the “dynamic” content which can actually be cached - Youtube and Windows Updates are two big offenders here.

So Squid isn’t that useless at all!

A couple of articles which give an overview of caching follow. They’re dated - the technology isn’t new after all - and just as applicable today.

• ComputerWorld (Traditional forward proxy/caching) - QuickStudy: Web Caching
• Tech Republic (Application Acceleration) - 8 Simple Caching Rules: Web Caching Your Application For Fast Response Time

WebDAV tester wanted.

One of the IPv6 squid testers has reported strange errors with a WebDAV enabled squid3-ipv6 build. Unfortunately he had no time available to track these down, and I don’t have WebDAV capability setup for use or testing.

I am seeking someone who does have the time and setup to test WebDAV in squid under an IPv6 setup. I am willing to act as a free consultant in the IPv6 side of the setup in exchange for this testing if needed.

Proxying with Squid: A Users Perspective

Someone pointed me over to sial.org where the author wrote up a quick Howto for various Squid tasks - basic refresh_patterns for controlling cacheability of files, filetypes and web URLs; remote refreshing; performance review; and an example reverse accelerator setup.

I think its a nice high-level introduction to using Squid as an website accelerator.

The Squid Configuration Manual

There’s been plenty of attempts at writing a Squid “configuration manual” (including the visolve documentation) which serves a good purpose (how do I do X?) but aren’t kept up to date when configuration options grow, change or die.

I whipped up a quick hacky bit of Perl which takes the squid.conf.default file (the “canonical” description of each configuration option, hopefully with examples!) and breaks it out into more manageable HTML pages. The result is available here for Squid-2.6 and here for Squid-3.0. All possible Squid configuration options are listed on the front page with each option populating its own page. This is so much easier to read than the squid.conf.default file. They’re updated whenever the squid configuration file is updated to reflect new, changed and deleted configuration options.

It does require the configuration file to be kept up to date, and there might be a t-shirt in it for you if you’re willing to spend some time tidying up the configuration file (cf.data.pre in the Squid release source tree) and expanding on various options. Some are well documented (see acl) whislt some aren’t documented at all..

Squid Wiki Articles for T-shirts!

There’s a few articles which should be written up in the Squid Wiki to cover commonly-asked questions.

I’m happy to donate a Squid mug, mouse-pad or t-shirt (http://www.cafepress.com/squidproxy/) to anyone who volunteers their time to help the Squid project writing up some documentation.

The first article I’d like to see is one covering how to write URL and MIME type filters to filter specific file types and extensions. That keeps popping up on IRC and the squid-users email list.

Squid and Phishy filtering

A month or two ago I had a random thought, “Wouldn’t it be cool to integrate phishtank.com’s database of online “phishy” URLs into Squid somehow?”

My next target is to extend it to include Google’s SafeBrowsing API and downloadable hash database, which will provide enterprise organisations, ISPs, educational instutions and the like more of a reason to run Squid to provide extra safety against all the evil crap thats out there.

Further Info on IPv6 - Where the official site actually is…

Since people seem to be redirected here in preference to the official pages on the squid IPv6 branch. I think its about time I made some quick references back there so all of you trying to use this wonderful branch can find the actual code and know how to do so.

The IPv6 work in squid is all currently documented at http://devel.squid-cache.org/squid3-ipv6/ and related pages. My contacts, or those of any developer is kept on to maintain it should be referenced from there.

How-To’s, configuration, patches, etc, etc, ‘all the guff’ as they say, will be available there shortly as well.