[squid-users] Pull/Fetch high level URL requests from Squid access.log without getting all the object hits

[Amos Jeffries]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 21/11/2014 3:31 a.m., Swapneel Patnekar wrote:
> Dear Amos,
> 
> Thank you for your input.
> 
> The reason I had this requirement is to build a analytics app on
> top of Squid which will only show typed URL's so that the end user
> has the precise information w.r.t evaluating web access.
> 
> Currently, since everything gets logged into the access.log, there
> is no way to differentiate between typed/bookmark accessed URL's
> and URL's which are references for the main URL i.e CDN URL's for
> js, css etc.
> 
> Sorry if I am pressing here, but do you have any
> suggestions/alternate solutions for how I could achieve this ?

No sorry. What you ae asking for is not possible. The information you
are seeking is private to the browser. It rarely goes anywhere else,
and applications that attempt to make it get a CVE security
vulnerability of "information leakage" classification thrown at them
for doing it.

You will have to go with imprecise data and heuristic algorithms
estimating actions based on the logged URLs and their relative timing.


It may help if you discard the idea that you can point at a single URL
and say that is "the page". There are no such things as "pages" in
HTTP. "Page" is one of several abstract concepts representing groups
of URL fetches. In other words a "page" is a whole collection of
transactions in the HTTP log.
 Each time a user views a "page". Some *part* of the URLs forming that
page are fetched. Some parts of the collection may not be fetched. Or
different parts fetched at different "page" views. It depends on what
the browser needs to fetch at the time of each different viewing
(since it has its own cache).

Amos
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)

iQEcBAEBAgAGBQJUbgGXAAoJELJo5wb/XPRj24QH+wXf8G02wlscUiMO2s99q4I2
ROM1PA16Ye6EieD4I0p+vydIint4nIh10qiuuV+BnsMPgDK95ADdWgl6p2HFWYyK
ZwxzYZ7k4HW9cvJujkGKyJEgIv9UEQmduzljJu35yig50fNIbdYhbMNch4lmjZHg
CYs3TTglE0VOXh5+vYxKLRBR7CzZcYcHzQ2epj5kuSkCfFQSRTyZTsO2bZsTmv38
CufUrkzH6i0XnhYpp9tTwFdXBNPJs2yJI5I0UcDbPBI44IAH+bMCq0HJ83dV1CJA
c4+4EXE+vnWuc67HzmsSWx9KveQvx/2sddex3HqLtWwSXYp328JqHp22oTErwXE=
=dTiW
-----END PGP SIGNATURE-----