[squid-users] Pull/Fetch high level URL requests from Squid access.log without getting all the object hits

[Swapneel Patnekar]

Dear Amos,

Thank you for your input.

The reason I had this requirement is to build a analytics app on top of
Squid which will only show typed URL's so that the end user has the precise
information w.r.t evaluating web access.

Currently, since everything gets logged into the access.log, there is no
way to differentiate between typed/bookmark accessed URL's and URL's which
are references for the main URL i.e CDN URL's for js, css etc.

Sorry if I am pressing here, but do you have any suggestions/alternate
solutions for how I could achieve this ?

On Thu, Nov 20, 2014 at 7:37 PM, Amos Jeffries <squid3 at treenet.co.nz> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 21/11/2014 1:18 a.m., Swapneel Patnekar wrote:
> > Hi there,
> >
> > I need to pull/fetch high level URL requests from the Squid
> > access.log i.e URL requests which were typed by the user in the
> > browser.
> >
> > For example, if the user had typed facebook.com, I want to
> > pull/fetch only facebook.com from the access.log and not
> > https://fbstatic-a.akamaihd.net/rsrc.php/v2/yV/r/aXwjx2fqSf4.css
> > etc which was not typed by the user in the browser but was
> > referenced by facebook.com for the CSS.
> >
> > Can this be done ?
>
> No it can't.
>
> There is absolutely no way for Squid to identify what the user (if one
> even exists) has done with their keyboard (or shortcuts, or bookmarks,
> or search bar or...) that started the HTTP to happen.
>
> You can log the "Referer" header contents, requests with no Referer
> value are usually "first" requests. But that header is not always sent
> or is sent when users change "page". So the accuracy is very low for
> what you are asking to get out of it.
>
> FWIW: "facebook.com" is a rarely visited page. Most of the users
> search for it by company name and click the results, or use history
> and bookmarks, even the emailed links FB sends out. All of which dump
> them straight into the middle of some sub-section of the FB site.
>
> Amos
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.22 (MingW32)
>
> iQEcBAEBAgAGBQJUbfWhAAoJELJo5wb/XPRjaeQH/0uWTCtDq9DNvNNXUVo103g4
> nFAvaT5kcaEJxRNOCTMerLwSAIrqyhT+SpqnmbSJURskwmW8vZRodIAnQPXPfiCj
> QuAazwDywVq0n7SyAtyIzyK0I6qlVtuKD+3VHcCJ30AXMv4RUv3ne8WenVLYggOq
> KGfTAS5rXUQvnAKpSz+jRGY4ZS7ZJ7dxrrPZwUxsBqXiNAwJpesZVScnxMtiXdsN
> Ko+/CMUDA4i35pAsc/l/GtGQozPtlsMtiXm7V5Vg+p9r01gIIaUWA6DuatBuB6RR
> QqMsMWKm66HpeD8Kw/MMx81yKOXpEZYRpuGxoxs9CU/rrAd1uugjUR/3D1QLcgs=
> =7n1w
> -----END PGP SIGNATURE-----
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20141120/7142d20c/attachment.html>