[squid-users] TProxy and client_dst_passthru

Omid Kosari omidkosari at yahoo.com
Sun Sep 11 15:04:07 UTC 2016


Antony Stone wrote
> On Thursday 08 September 2016 at 12:27:42, Omid Kosari wrote:
> 
>> Hi Fred,
>> 
>> Same problem here . Do you found any solution or workaround ?
> 
> Please clarify which message you are reply / referring to.
> 
> Thanks,
> 
> 
> Antony.
> 
> -- 
> Archaeologists have found a previously-unknown dinosaur which seems to
> have 
> had a very large vocabulary.  They've named it Thesaurus.
> 
>                                                    Please reply to the
> list;
>                                                          please *don't* CC
> me.
> _______________________________________________
> squid-users mailing list

> squid-users at .squid-cache

> http://lists.squid-cache.org/listinfo/squid-users

I refer to following messages .i have same problem


FredT wrote
> Hi Amos,
> 
> We have done additional tests in production with ISPs and the ORIGINAL_DST
> in tproxy cannot be cached.
> In normal mode (not tproxy), ORIGINAL_DST can be cached, no problem.
> But once in tproxy (http_port 3128 tproxy), no way, it's impossible to get
> TCP_HIT.
> 
> We have played with the client_dst_passthru and the host_verify_strict,
> many combinaisons on/off.
> By settings client_dst_passthru ON and host_verify_strict OFF, we can
> reduce the number of ORIGINAL_DST (generating DNS "alerts" in the
> cache.log) but it makes issues with HTTPS websites (facebook, hotmail,
> gmail, etc...).
> We have also tried many DNS servers (internals and/or externals), same
> issue.
> 
> I read what you explain in your previous email but it seems there is
> something weird.
> The problem is that the ORIGINAL_DST could be up to 25% of the traffic
> with some installations meaning this part is "out-of-control" in term of
> cache potential.
> 
> All help is welcome here
> Thanks in advance.
> 
> Bye Fred 


FredT wrote
> Hi Eliezer,
> 
> Well, we have done many tests with Squid (3.1 to 3.5.x), disabling
> "client_dst_passthru" (off) will stop the DNS entry as explained in the
> wiki, the option directly acts on the flag "ORIGINAL_DST".
> As you know, ORIGINAL_DST switches the optimization off (ex: StoreID) then
> it's not possible to cache the URL (ex:
> http://cdn2.example.com/mypic.png).
> 
> In no tproxy/NAT mode, the client_dst_passthru works perfectly by
> disabling the DNS entry control, so optimization is done correctly.
> But in tproxy/NAT, the client_dst_passthru has no effect, we see
> ORIGINAL_DST in logs.
> 
> So, maybe I'm totaly wrong here the client_dst_passthru is not related to
> the ORIGINAL_DST, or there is an explaination why the client_dst_passthru
> does not act in tproxy/NAT...
> 
> Bye Fred

please look at following results 
As you know the following command shows statistics of line which only have
ORIGINAL_DST

tail -n 1000000 /var/log/squid/access.log | grep -a ORIGINAL_DST | calamaris 
--config-file /etc/calamaris/calamaris.conf --all-useful-reports | more


----------------------------------------------------- --------------
----------
Proxy statistics
----------------------------------------------------- --------------
----------
Total amount:                                               requests    
378310
unique hosts/users:                                            hosts      
1859
Total Bandwidth:                                                Byte    
16453M
Proxy efficiency (HIT [kB/sec] / DIRECT [kB/sec]):            factor      
1.22
Average speed increase:                                            %      
0.39
TCP response time of 100% requests:                             msec        
0M
----------------------------------------------------- --------------
----------
Cache statistics
----------------------------------------------------- --------------
----------
Total amount cached:                                        requests     
11945
Request hit rate:                                                  %      
3.16
Bandwidth savings:                                              Byte      
355M
Bandwidth savings in Percent (Byte hit rate):                      %      
2.16
Average cached object size:                                     Byte        
0M
Average direct object size:                                     Byte        
0M
Average object size:                                            Byte        
0M
----------------------------------------------------- --------------
----------

# Incoming TCP-requests by status
status                          request      %  sec/req   Byte       % 
kB/sec
------------------------------ --------- ------ ------- -------- ------
-------
HIT                                11945   3.16    1.94     355M   2.16  
15.66
 TCP_REFRESH_UNMODIFIED_ABORTED
                                     104   0.03   44.89     158M   0.96  
34.55
 TCP_REFRESH_UNMODIFIED            11795   3.12    0.77     119M   0.72  
13.47
 TCP_REFRESH_UNMODIFIED_TIMEDOUT
                                       8   0.00 1108.82      79M   0.48   
9.09
 TCP_HIT_ABORTED                      38   0.01   16.89       0M   0.00   
0.00
MISS                              366365  96.84    3.49   16098M  97.84  
12.87
 TCP_MISS                         342321  90.49    2.11   12723M  77.33  
18.02
 TCP_MISS_ABORTED                  20588   5.44   24.61    2902M  17.64   
5.86
 TCP_REFRESH_MODIFIED_ABORTED         55   0.01   99.34     193M   1.17  
36.14
 TCP_REFRESH_MODIFIED                941   0.25    1.41     118M   0.72  
91.05
 TCP_MISS_TIMEDOUT                    45   0.01  949.92      80M   0.49   
1.92
 TCP_SWAPFAIL_MISS                  2107   0.56    0.34      49M   0.30  
71.08
 TCP_REFRESH_FAIL_OLD_ABORTED         12   0.00   31.61      29M   0.18  
78.96
 TCP_CLIENT_REFRESH_MISS             284   0.08    0.31       3M   0.02  
35.74
 TCP_SWAPFAIL_MISS_ABORTED            11   0.00    0.49       0M   0.00   
1.47
 TCP_REFRESH_FAIL_OLD                  1   0.00   59.05       0M   0.00   
0.06
ERROR                                  0   0.00    0.00        0   0.00   
0.00
------------------------------ --------- ------ ------- -------- ------
-------
Sum                               378310 100.00    3.45   16453M 100.00  
12.92


# Outgoing requests by status
status                          request      %  sec/req   Byte       % 
kB/sec
------------------------------ --------- ------ ------- -------- ------
-------
DIRECT Fetch from Source          378310 100.00    3.45   16453M 100.00  
12.92
 ORIGINAL_DST                     377424  99.77    3.45   16424M  99.82  
12.92
 TIMEOUT_ORIGINAL_DST                886   0.23    2.51      29M   0.18  
13.31
SIBLING                                0   0.00    0.00        0   0.00   
0.00
PARENT                                 0   0.00    0.00        0   0.00   
0.00
------------------------------ --------- ------ ------- -------- ------
-------
Sum                               378310 100.00    3.45   16453M 100.00  
12.92


# Outgoing requests by destination
neighbor type                   request      %  sec/req   Byte       % 
kB/sec
------------------------------ --------- ------ ------- -------- ------
-------
DIRECT                            378310 100.00    3.45   16453M 100.00  
12.92
------------------------------ --------- ------ ------- -------- ------
-------
Sum                               378310 100.00    3.45   16453M 100.00  
12.92



As you know the following command shows statistics of line which DOES NOT
have ORIGINAL_DST

tail -n 1000000 /var/log/squid/access.log | grep -a -v ORIGINAL_DST |
calamaris  --config-file /etc/calamaris/calamaris.conf --all-useful-reports
| more

----------------------------------------------------- --------------
----------
Proxy statistics
----------------------------------------------------- --------------
----------
Total amount:                                               requests    
624866
unique hosts/users:                                            hosts      
1427
Total Bandwidth:                                                Byte     
6864M
Proxy efficiency (HIT [kB/sec] / DIRECT [kB/sec]):            factor      
1.62
Average speed increase:                                            %     
29.24
TCP response time of 100% requests:                             msec        
0M
----------------------------------------------------- --------------
----------
Cache statistics
----------------------------------------------------- --------------
----------
Total amount cached:                                        requests    
134472
Request hit rate:                                                  %     
21.52
Bandwidth savings:                                              Byte     
4067M
Bandwidth savings in Percent (Byte hit rate):                      %     
59.26
Average cached object size:                                     Byte        
0M
Average direct object size:                                     Byte        
0M
Average object size:                                            Byte        
0M
----------------------------------------------------- --------------
----------

# Incoming TCP-requests by status
status                          request      %  sec/req   Byte       % 
kB/sec
------------------------------ --------- ------ ------- -------- ------
-------
HIT                               134472  21.52    0.42    4067M  59.26  
74.04
 TCP_HIT                           72883  11.66    0.23    3092M  45.04 
191.49
 TCP_MEM_HIT                       33078   5.29    0.08     507M   7.38 
193.01
 TCP_HIT_ABORTED                     556   0.09   16.68     344M   5.01  
37.97
 TCP_REFRESH_UNMODIFIED             3432   0.55    1.25     111M   1.62  
26.39
 TCP_IMS_HIT                       24390   3.90    0.00       6M   0.08  
67.65
 TCP_HIT_TIMEDOUT                     23   0.00  907.61       5M   0.07   
0.23
 TCP_MEM_HIT_ABORTED                  82   0.01    7.15       3M   0.04   
5.20
 TCP_REFRESH_UNMODIFIED_ABORTED
                                      27   0.00   70.41       0M   0.00   
0.18
 TCP_IMS_HIT_ABORTED                   1   0.00    0.00       0M   0.00
1689.45
MISS                               62944  10.07    0.88    2713M  39.52  
50.42
 TCP_MISS                          59824   9.57    0.41    2425M  35.32 
100.93
 TCP_MISS_ABORTED                    451   0.07   11.23     157M   2.29  
31.82
 TCP_SWAPFAIL_MISS                  2069   0.33    0.38      51M   0.75  
66.22
 TCP_MISS_TIMEDOUT                    21   0.00  913.96      43M   0.63   
2.29
 TCP_REFRESH_MODIFIED_ABORTED          6   0.00  138.09      32M   0.46  
38.99
 TCP_REFRESH_MODIFIED                366   0.06    0.18       3M   0.04  
45.45
 TCP_CLIENT_REFRESH_MISS             160   0.03    0.13       2M   0.02  
83.19
 TCP_REFRESH_FAIL_OLD                 33   0.01  135.57       0M   0.01   
0.09
 TCP_REFRESH_FAIL_OLD_ABORTED          1   0.00   59.66       0M   0.00   
1.94
 TCP_SWAPFAIL_MISS_ABORTED            13   0.00    0.07       0M   0.00   
0.00
ERROR                             427450  68.41    0.02      84M   1.22  
11.52
 TCP_MISS                            761   0.12    0.61      42M   0.61  
93.13
 TCP_MISS_ABORTED                    284   0.05   16.29      36M   0.53   
8.02
 TCP_DENIED                        19564   3.13    0.00       4M   0.07  
71.49
 TAG_NONE                           8096   1.30    0.00       1M   0.02
1484.81
 TCP_DENIED_ABORTED                    7   0.00    0.00       0M   0.00
2327.01
 TAG_NONE_ABORTED                    732   0.12    3.11       0M   0.00   
0.00
 UDP_MISS                         334769  53.57    0.00       0M   0.00   
0.00
 UDP_HIT                           63237  10.12    0.00       0M   0.00   
0.00
------------------------------ --------- ------ ------- -------- ------
-------
Sum                               624866 100.00    0.19    6864M 100.00  
59.16



60% vs 2% hit ratio(bytes) . The problem is ORIGINAL_DST










--
View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/TProxy-and-client-dst-passthru-tp4670189p4679450.html
Sent from the Squid - Users mailing list archive at Nabble.com.


More information about the squid-users mailing list