I’ve committed my logfile handling improvements to Squid-2-HEAD. Essentially, it lets people write self-contained code modules to implement different logging methods. The three supported methods now are:
- STDIO, which is how Squid currently does its logging;
- Syslog, which is compiled in if you enable it; and
- Daemon, which uses a simple external helper to write logfiles to disk.
Those of you who have run Squid may have noticed that it couldn’t support writing more than a hundred or so requests a second to disk before performance suffered. There’s no reason it shouldn’t handle this – a hundred requests a second is only 16 kilobytes a second to write – but the use of STDIO routines to do this had a negative impact on performance.
The logfile daemon allows the blocking disk IO to occur outside of the main Squid process; which basically means Squid can continue doing what its doing well (all the other stuff) and any blocking disk activity occurs in a seperate process.
To use? Compile and install Squid-2-HEAD, then include the following line into your configuration:
In reality, Squid with the logging daemon can now handle writing -thousands of requests a second- to disk without any performance impact. Furthermore, if the logging daemon can’t write to disk fast enough Squid will log a error message stating its falling behind and drop logging entries.
I’ve tested this up to three thousand requests a second over the course of a few hours (to a dedicated logging disk however) and it handles it without a problem.
If enterprising souls wished, they could write a UDP logging helper, or a MySQL external logging helper, without needing to modify the Squid codebase.
This code will eventually also appear in Squid-3 after 3.0 is released.