[squid-users] Get IP of denied request
magri at web.de
magri at web.de
Fri Oct 27 11:14:56 UTC 2023
Hi Alex,
Am 26.10.23 um 21:11 schrieb Alex Rousskov:
> On 2023-10-26 08:37, magri at web.de wrote:
>
>> TL;DR: is there a way to get/log the resolved ip of a denied request?
>
> TLDR: Bugs notwithstanding, use %<a.
>
%<a doesn't work :-(
Tested with attached minimal-squid.conf:
- config blacklists the ip of www.example.org
- logformat uses %<a
Test-Call:
http_proxy=http://127.0.0.1:3128 https_proxy=http://127.0.0.1:3128 wget
http://www.example.org
--2023-10-27 09:52:27-- http://www.example.org/
Connecting to 127.0.0.1:3128... connected.
Proxy request sent, awaiting response... 403 Forbidden
2023-10-27 09:52:31 ERROR 403: Forbidden.
Log contains:
1698393151.066 3903 127.0.0.1 TCP_DENIED/- 3889 GET
http://www.example.org/ - HIER_NONE/- text/html HTTP/1.1 37030 182 "-"
"Wget/1.21.3" ERR_ACCESS_DENIED "-" 53
>
>> We have a rather large ip based malware blacklist (dst acl) and
>> sometimes a destination is blocked inadvertantly because of a false
>> positive entry in this list.
>> This happens most often with CDNs where the ips of a destination change
>> often and even move between different sites.
>>
>> Because of this rapid change it's difficult to determine the blocked ip
>> in hindsight when analyzing access problems and makes it impossible to
>> correct the blacklist.
>>
>> For normal requests the resolved and accessed ip is be logged with %<a,
>> but that doesn't happen when the request is denied.
>
>
> If a request was denied by a dst ACL based on its successfully resolved
> destination IP address but %<a was logged as "-", then it is a Squid bug
> that should be fixed IMO. Meanwhile, you can annotate every dst match
> and log that annotation. Here is an untested sketch:
>
> acl matchDst1 dst 127.0.0.1
> acl markDst1 note matched=127.0.0.1
> acl all-of dst1 matchDst1 markDst1
> http_access deny dst1
>
> acl matchDst2 dst 127.0.0.2
> acl markDst2 note matched=127.0.0.2
> acl all-of dst2 matchDst2 markDst2
> http_access deny dst2
>
> logformat myFormat ... matched_dst=%note{matched}
> access_log ...
>
>
> The same thing with fewer lines (but with fewer ways to group dst1 and
> dst2 with other ACLs):
>
> acl matchDst1 dst 127.0.0.1
> acl markDst1 note matched=127.0.0.1
> http_access deny matchDst1 markDst1
>
> acl matchDst2 dst 127.0.0.2
> acl markDst2 note matched=127.0.0.2
> http_access deny matchDst2 markDst2
>
> logformat myFormat ... matched_dst=%note{matched}
> access_log ...
>
> For long dst lists, the above approach will require scripting the
> generation of the corresponding squid.conf portions or include files, of
> course.
>
I don't think this scales to blacklists with 6-digit count sizes and it
also doesn't work for blacklisted networks :-(
I hoped there would be a way to get the ip as some kind of variable like
the header fields in logformat.
>
> If a request was denied by a dst ACL because its destination IP address
> could not be resolved, then %<a should be logged as "-". I cannot think
> of a way to distinguish this case from other cases where %<a is "-". It
> feels like address resolution failures should be available via
> %err_detail, but I doubt Squid code populates that information in these
> cases. Another problem to fix!
I'm not familiar with the code but from staring at it I get that '%<a'
is fetched from hier.tcpServer->remote and this isn't necessarily the ip
of the resolved URI FQDN (e.g. when using a parent proxy).
There doesn't seem to be any code that stores or reuses a once resolved
ip in acl/DestinationIP.cc (at least if a request exists).
As far as I understand it for every dst ACL the ip is fetched
(asyncronously) from the ipcache and compared to the ACL but never
stored for later use.
Any ideas?
Martin
>
>
> HTH,
>
> Alex.
>
>
>
>> Is there any way to get the ip logged that was used in the dst-acl aside
>> from debug logging? Maybe through some annotation mechanism?
>>
>> Squid version is 6.2, as 6.4 crashes with assertion errors here, too.
>>
>> thanks,
>> Martin
>>
>> _______________________________________________
>> squid-users mailing list
>> squid-users at lists.squid-cache.org
>> https://lists.squid-cache.org/listinfo/squid-users
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> https://lists.squid-cache.org/listinfo/squid-users
-------------- next part --------------
http_port 3128
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 443
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
acl www_example_org dst 93.184.216.34
http_access deny www_example_org
http_access allow all
never_direct deny all
always_direct allow all
logformat debug_squid %ts.%03tu %6tr %>a %Ss/%<Hs %<st %rm %ru %un %Sh/%<a %mt HTTP/%rv %>p %>st "%{Referer}>h" "%{User-Agent}>h" %err_code "%err_detail" %master_xaction
access_log daemon:/var/log/squid/test.log logformat=debug_squid
cache_log /var/log/squid/cache.log
debug_options ALL,2
More information about the squid-users
mailing list