[squid-users] Pull/Fetch high level URL requests from Squid access.log without getting all the object hits

[Swapneel Patnekar]

Thank you Amos for your detailed explanation. That helped. Appreciate it.

On Thu, Nov 20, 2014 at 8:28 PM, Amos Jeffries <squid3 at treenet.co.nz> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 21/11/2014 3:31 a.m., Swapneel Patnekar wrote:
> > Dear Amos,
> >
> > Thank you for your input.
> >
> > The reason I had this requirement is to build a analytics app on
> > top of Squid which will only show typed URL's so that the end user
> > has the precise information w.r.t evaluating web access.
> >
> > Currently, since everything gets logged into the access.log, there
> > is no way to differentiate between typed/bookmark accessed URL's
> > and URL's which are references for the main URL i.e CDN URL's for
> > js, css etc.
> >
> > Sorry if I am pressing here, but do you have any
> > suggestions/alternate solutions for how I could achieve this ?
>
> No sorry. What you ae asking for is not possible. The information you
> are seeking is private to the browser. It rarely goes anywhere else,
> and applications that attempt to make it get a CVE security
> vulnerability of "information leakage" classification thrown at them
> for doing it.
>
> You will have to go with imprecise data and heuristic algorithms
> estimating actions based on the logged URLs and their relative timing.
>
>
> It may help if you discard the idea that you can point at a single URL
> and say that is "the page". There are no such things as "pages" in
> HTTP. "Page" is one of several abstract concepts representing groups
> of URL fetches. In other words a "page" is a whole collection of
> transactions in the HTTP log.
>  Each time a user views a "page". Some *part* of the URLs forming that
> page are fetched. Some parts of the collection may not be fetched. Or
> different parts fetched at different "page" views. It depends on what
> the browser needs to fetch at the time of each different viewing
> (since it has its own cache).
>
> Amos
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.22 (MingW32)
>
> iQEcBAEBAgAGBQJUbgGXAAoJELJo5wb/XPRj24QH+wXf8G02wlscUiMO2s99q4I2
> ROM1PA16Ye6EieD4I0p+vydIint4nIh10qiuuV+BnsMPgDK95ADdWgl6p2HFWYyK
> ZwxzYZ7k4HW9cvJujkGKyJEgIv9UEQmduzljJu35yig50fNIbdYhbMNch4lmjZHg
> CYs3TTglE0VOXh5+vYxKLRBR7CzZcYcHzQ2epj5kuSkCfFQSRTyZTsO2bZsTmv38
> CufUrkzH6i0XnhYpp9tTwFdXBNPJs2yJI5I0UcDbPBI44IAH+bMCq0HJ83dV1CJA
> c4+4EXE+vnWuc67HzmsSWx9KveQvx/2sddex3HqLtWwSXYp328JqHp22oTErwXE=
> =dTiW
> -----END PGP SIGNATURE-----
>



-- 
Best,
Swapneel Patnekar
+91 98866 37820
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20141122/91a9defa/attachment.html>