[squid-dev] New caching policy on squid proxy server

[Amos Jeffries]

On 27/03/2015 12:26 a.m., Anil Prajapati wrote:
> Hello Sir
> We are PG students and want your suggestions for our research work. We have
> configured  a squid proxy server on Ubuntu machine. We want to cache the
> PDF files on squid server from amazon S3. Can we apply new caching policy
> to cache the files?

Caching policy is explicitly defined by the HTTP protocol caching
mechanisms <http://tools.ietf.org/html/rfc7234>. Everything which can be
stored is already done so. Altering that behavior can only be done if it
complies with the standard. There are some bugs in this code, but simply
bug fixing is probably not a good choice for research.


FWIW: A lot of non-cached cloud content (such as Amazon S3) cannot be
cached because of how the servers to do not implement HTTP caching
controls fully. Cacheing that content can only be done by fixing the
servers, not the proxy.
 You can use tools such as the one at http://redbot.org to analyse the
URI server messages to see what the cacheability and other HTTP protocol
problems may be.


One area of potential research is caching of both private and shared
content in a single cache system. Squid currently implements a shared
cache (many users/clients). With the protections against CVE-2009-0801
and growth in per-user private data on the web an annoyingly large
amount of traffic on some networks is not cacheable by a proxy
performing shared cache.
 I believe it should be possible to isolate the private and shared data
within the storage system, but some research and development is needed
to determine; a) how that could be done, and b) which (if any)
mechanisms meet the criteria of being safe, effective and efficient.
Even a proof against using certain mechanisms would be useful at this point.


> 
> Can we modify existing policies of squid server?
> Currently squid work on LRU, LFUDA , GDSF caching policies. We want add new
> caching policy on our squid proxy server. Can we do it? Please Provide the
> way to do it if yes.

Those are not caching policies. They are removal policies, determining
how Squid decides what to throw away when more storage space is needed.
Anything may be thrown away at any time in a cache, so those are of
course modifiable or extendable. The algorithm code is in src/repl/

Note: if you do go with editing Squid code, please work with the latest
Squid development version. Today that is 4.0.0.
Links to useful developer info can be found at
<http://wiki.squid-cache.org/DeveloperResources>

Also, please subscribe to the squid-dev mailing list (cc'd) for future
correspondence. That is where code discussion takes place and others
among the developers know more about the storage systems internals than
myself.
<http://www.squid-cache.org/Support/mailing-lists.html#squid-dev>


Amos Jeffries
Squid Software Foundation