[squid-users] Transparent Squid Proxy Server
Yuri Voinov
yvoinov at gmail.com
Wed Jun 24 09:56:28 UTC 2015
Man,
3.5.x don't work with server-first. It must be for backward
compatibility - but don't be.
Also, AFAIK, 3.5.x series don't work with transparent NAT interception
in bump mode. Fake certs are generated, but with IP against hostnames
(in all my test installations).
So, if you strictly need working bump with transparent interception,
rollback to 3.4.
WBR, Yuri.
24.06.15 12:04, Reet Vyas пишет:
> Hi
> Below is my squid file , I have configured squid 3.5.3 with ssl, but
> I cant filter https traffic and also in access log I cant see https in
> access logs.
>
>
> #
> # Recommended minimum configuration:
> #
>
> # Example rule allowing access from your local networks.
> # Adapt to list your (internal) IP networks from where browsing
> # should be allowed
> acl localnet src 116.72.152.37 192.168.0.0/24 <http://192.168.0.0/24>
> # Sesuaikan dengan ip client/local
>
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> # storeid *test*
> acl urlrewrite dstdomain .fbcdn.net <http://fbcdn.net> .akamaihd.net
> <http://akamaihd.net>
> acl speedtest url_regex -i speedtest\/.*\.(jpg|txt)\?.*
> acl reverbnation url_regex -i
> reverbnation.*audio_player.*ec_stream_song.*$
> acl utmgif url_regex -i utm.gif.*
> acl playstoreandroid url_regex -i
> c.android.clients.google.com.market.GetBinary.GetBinary.*
> acl idyoutube url_regex -i
> youtube.*(ptracking|stream_204|player_204).*(v\=|docid\=|video_id\=).*$
> acl videoyoutube url_regex -i (youtube|googlevideo).*videoplayback\?
> acl videoyoutube url_regex -i (youtube|googlevideo).*videoplayback\?
> acl CONNECT method CONNECT
> acl getmethod method GET
> acl loop_302 http_status 302
> acl step1 at_step SslBump1
> acl youtube dstdomain .youtube.com <http://youtube.com>
> acl blocksites dstdomain "/etc/squid/restricted-sites.squid"
> # TAG: QUERY
> #
> -----------------------------------------------------------------------------
> acl QUERY urlpath_regex -i
> (hackshield|blank.html|infinity.js|hshield.da|renew_session_token.php|recaptcha.js|dat.asp|notice.swf|patchlist.txt|hackshield|captcha|reset.css|update.ver|notice.html|updates.txt|gamenotice|images.kom|patchinfo.xml|noupdate.ui|\.Xtp|\.htc|\.txt)
> acl QUERY urlpath_regex -i
> (patch.conf|uiimageset.xml.iop|gashaponwnd.xml.iop|loading.swf|download.swf|version.list|version.ini|launch.jnlp|server_patch.cfg.iop|core.swf|Loading.swf|resouececheck.sq|mainloading.swf|config.xml|gemmaze.swf|xml.png|size.xml|resourcesbar.swf|version.xml|version.list|delete.ini)
> acl QUERY urlpath_regex -i \.(jsp|asp|aspx|cfg|iop|zip|php|xml|html)(\?|$)
> cache deny QUERY
> cache deny youtube
>
> #
> acl dontstore url_regex
> ^http:\/\/(([\d\w-]*(\.[^\.\-]*?\..*?))(\/\mosalsal\/[\d]{4}\/.*\/)(.*\.flv))\?start.*
> acl dontstore url_regex redbot\.org \.php
> acl dontstore url_regex -i ^http:\/\/.*gemscool\.com\/.*
> acl dontstore url_regex \.(aspx|php)\?
> acl dontstore url_regex goldprice\.org\/NewCharts\/gold\/images\/.*\.png
> acl dontstore url_regex google\.co(m|\.[a-z]{2})\/complete\/search\?
> acl dontstore url_regex
> redirector\.([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com)\/(get_video\?|videodownload\?|videoplayback.*id|get_video_info\?|ptracking\?|player_204\?|stream_204\?).*
>
> acl store_yt_id url_regex -i
> youtube.*(ptracking|stream_204|playback|player_204|watchtime|set_awesome|s\?|ads).*(video_id|docid|\&v|content_v)\=([^\&\s]*).*$
> acl store_id_list_yt url_regex -i (youtube|googlevideo).*videoplayback.*$
> acl store_id_list_yt url_regex
> ^https?\:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com)\/(get_video\?|videodownload\?|videoplayback.*id).*
>
> acl store-id_list urlpath_regex -i dl\.sourceforge\.net
> acl store-id_list urlpath_regex -i \.ytimg\.com
> acl store-id_list urlpath_regex -i \.(akamaihd|fbcdn)\.net
> acl store_id_list urlpath_regex -i
> [a-zA-Z]{2}[0-9]*\.4shared\.com\/download\/
>
> acl store_id_list_url url_regex
> ^http:\/\/[0-9]\.bp\.blogspot\.com.*\.(jpeg|jpg|png|gif|ico)
> acl store_id_list_url url_regex
> ^http[s]?:\/\/.*\.twimg\.com\/(.*)\.(gif|jpeg|jpg|png|js|css)
> acl store_id_list_url url_regex
> ^http[s]?:\/\/(media|static)\.licdn\.com\/.*\.(png|jpg|gif|woff)
> acl store_id_list_url url_regex
> ^https:\/\/fb(static|cdn)\-.*\-a.akamaihd.net
> <http://a.akamaihd.net>\/(.*)\.(gif|jpeg|jpg|png|js|css|mp4)
> acl store_id_list_url url_regex
> ^http:\/\/.*\.ak\.fbcdn\.net\/.*\.(gif|jpg|png|js|mp4)
>
> # pass requests
> url_rewrite_program /etc/squid/phpredir.php
> url_rewrite_access allow youtube
>
> request_header_access Range deny store_id_list_yt
> range_offset_limit 10 KB store_id_list_yt
>
>
> ###############################################################################
> # Recommended minimum Access Permission configuration:
> #
> # Deny requests to certain unsafe ports
> ###############################################################################
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access deny blocksites
> http_access allow localhost manager
> http_access deny manager
> http_access allow localnet
> http_access allow localhost
> http_access deny all
>
> ###############################################################################
> # squid ssl_bump option
> ###############################################################################
> always_direct allow all
> ssl_bump server-first all
> sslproxy_cert_error deny all
> sslproxy_flags DONT_VERIFY_PEER
> sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB
> sslcrtd_children 8 startup=1 idle=1
> #ssl_bump peek step1
> #ssl_bump bump all
> ###############################################################################
> # Squid normally listens to port 3128
> ###############################################################################
> https_port 3130 intercept ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_certs/squid.crt
> key=/etc/squid/ssl_certs/squid.key
> http_port 3129 intercept
> http_port 3128
>
> # TAG: Store-id Program
> #
> -----------------------------------------------------------------------------
> store_id_program /usr/bin/perl /etc/squid/store-id.pl <http://store-id.pl>
> store_id_children 100 startup=0 idle=1 concurrency=1000
>
> # TAG: Store-id Access
> #
> -----------------------------------------------------------------------------
> store_id_access allow urlrewrite
> store_id_access allow speedtest
> store_id_access allow reverbnation
> store_id_access allow utmgif
> store_id_access allow playstoreandroid
> store_id_access allow idyoutube
> store_id_access allow videoyoutube
> store_id_access deny dontstore
> store_id_access deny !getmethod
> store_id_access allow store_id_list_yt
> store_id_access allow store_yt_id
> store_id_access allow store-id_list
> store_id_access deny all
> store_id_bypass on
>
> # TAG: Youtube 302
> #
> -----------------------------------------------------------------------------
> store_miss deny store_id_list_yt loop_302
> send_hit deny store_id_list_yt loop_302
>
> ###############################################################################
> ## MEMORY CACHE OPTIONS
> ###############################################################################
> client_dst_passthru on
> cache_mem 1024 MB
> maximum_object_size_in_memory 1024 KB
> memory_cache_shared off
> memory_cache_mode disk
> memory_replacement_policy heap GDSF
>
> ###############################################################################
> ## DISK CACHE OPTIONS
> ###############################################################################
> cache_replacement_policy heap LFUDA
> minimum_object_size 1 bytes
> maximum_object_size 10 GB
>
> ###############################################################################
> # Uncomment and adjust the following to add a disk cache directory.
> ###############################################################################
> cache_dir aufs /usr/local/cache_proxy 25000 16 256 # sesuaikan dengan
> drive penyimpanan cache
> store_dir_select_algorithm round-robin
> cache_swap_low 90
> cache_swap_high 95
>
> ###############################################################################
> # Leave coredumps in the first cache dir
> ###############################################################################
> coredump_dir /var/spool/squid
>
> ###############################################################################
> ## LOGFILE OPTIONS
> ###############################################################################
> #access_log daemon:/tmp/access.log !log
> #logfile_daemon /usr/lib/squid/log_file_daemon
> cache_store_log none
> logfile_rotate 1
> mime_table /etc/squid/mime.conf
> pid_filename /var/run/squid.pid
> strip_query_terms off
> buffered_logs off
>
> ###############################################################################
> ## OPTIONS FOR TROUBLESHOOTING
> ###############################################################################
> #cache_log /tmp/cache.log
> cache_log /dev/null
> #debug_options ALL,1 22,3
> coredump_dir /var/spool/squid
>
> ###############################################################################
> ## OPTIONS FOR TUNING THE CACHE
> ###############################################################################
> max_stale 1 years
> vary_ignore_expire on
> shutdown_lifetime 10 seconds
>
> ###############################################################################
> # Add any of your own refresh_pattern entries above these.
> ###############################################################################
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> # Youtube Video
> refresh_pattern -i
> (get_video\?|videoplayback\?|videodownload\?|\.mp4|\.webm|\.flv|((audio|video)\/(webm|mp4)))
> 241920 100% 241920 override-expire ignore-reload ignore-private
> ignore-no-store ignore-must-revalidate reload-into-ims ignore-auth
> store-stale
> refresh_pattern -i ^https?\:\/\/.*\.googlevideo\.com\/videoplayback.*
> 10080 99% 43200 override-lastmod override-expire ignore-reload
> reload-into-ims ignore-private reload-into-ims ignore-auth store-stale
> refresh_pattern -i
> ^https?\:\/\/.*\.googlevideo\.com\/videoplayback.*$ 241920 100%
> 241920 override-expire ignore-reload ignore-private ignore-no-store
> ignore-must-revalidate reload-into-ims ignore-auth store-stale
>
> refresh_pattern (akamaihd|fbcdn)\.net 14400 99% 518400 ignore-no-store
> ignore-private ignore-reload ignore-must-revalidate store-stale
> refresh_pattern -i squid\.internal 14400 99% 518400 ignore-no-store
> ignore-private ignore-reload ignore-must-revalidate store-stale
> refresh_pattern \.(jpg|png|gif|css|ico)($|\?) 14400 99% 518400
> ignore-no-store ignore-private reload-into-ims ignore-must-revalidate
> store-stale
> refresh_pattern . 0 99% 518400 ignore-no-store ignore-private
> reload-into-ims store-stale
> # Image Youtube
> refresh_pattern -i (yimg|twimg)\.com\.* 1440 100% 129600
> override-expire ignore-reload reload-into-ims
> refresh_pattern -i (ytimg|ggpht)\.com\.* 1440 80% 129600
> override-expire override-lastmod ignore-auth ignore-reload reload-into-ims
>
> #images facebook
> refresh_pattern -i
> fbcdn.*net\/.*\.((jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)|(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)(\?|.*$))
> 241920 99% 241920 ignore-no-store ignore-private override-expire
> override-lastmod reload-into-ims ignore-auth
> refresh_pattern -i pixel\.facebook\.com.*\.(jpg|png|gif|ico|css|js)
> 241920 80% 241920 override-expire ignore-reload reload-into-ims
> ignore-auth
> refresh_pattern -i \.akamaihd\.net.*\.(jpg|png|gif|ico|css|js) 241920
> 80% 241920 override-expire ignore-reload reload-into-ims ignore-auth
> refresh_pattern -i ((facebook.com
> <http://facebook.com>)|(85.131.151.39))\.(jpg|png|gif) 241920 99%
> 241920 ignore-reload override-expire ignore-no-store store-stale
> refresh_pattern -i
> fbcdn\.net\/.*\.((jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)|(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)(\?|.*$))
> 241920 99% 241920 ignore-no-store ignore-private override-expire
> override-lastmod reload-into-ims ignore-auth
> refresh_pattern static\.(xx|ak)\.fbcdn\.net*\.(jpg|gif|png) 241920 99%
> 241920 ignore-reload override-expire ignore-no-store
> refresh_pattern ^https?\:\/\/profile\.ak\.fbcdn.net
> <http://fbcdn.net>*\.(jpg|gif|png) 241920 99% 241920 ignore-reload
> override-expire ignore-no-store
>
> # Video Facebook
> refresh_pattern -i \.video.ak.fbcdn.net.*\.(mp4|flv|mp3|amf) 10080 80%
> 43200 override-expire ignore-reload reload-into-ims ignore-private
> ignore-no-store ignore-must-revalidate
> refresh_pattern (audio|video)\/(webm|mp4) 129600 99% 129600
> ignore-reload override-expire override-lastmod ignore-must-revalidate
> ignore-private ignore-no-store ignore-auth store-stale
> refresh_pattern -i ^http://.*squid\.internal.* 241920 100% 241920
> override-lastmod override-expire ignore-reload ignore-must-revalidate
> ignore-private ignore-no-store ignore-auth store-stale
>
> # All File
> refresh_pattern -i
> \.(3gp|7z|ace|asx|bin|deb|divx|dvr-ms|ram|rpm|exe|inc|cab|qt) 10080
> 80% 10080 override-expire override-lastmod reload-into-ims
> refresh_pattern -i
> \.(rar|jar|gz|tgz|bz2|iso|m1v|m2(v|p)|mo(d|v)|arj|lha|lzh|zip|tar|iop|nzp|pak|mar|msp)
> 10080 80% 10080 override-expire override-lastmod reload-into-ims
> ignore-reload
> refresh_pattern -i
> \.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|dat|ad|txt|dll) 10080 80%
> 10080 override-expire override-lastmod reload-into-ims
> refresh_pattern -i
> \.(avi|ac4|mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p)|og(x|v|a|g)|rm|r(a|p)m|snd|vob|webm)
> 10080 80% 10080 override-expire override-lastmod reload-into-ims
> refresh_pattern -i
> \.(pp(t?x)|s|t)|pdf|rtf|wax|wm(a|v)|wmx|wpl|cb(r|z|t)|xl(s?x)|do(c?x)|flv|x-flv)
> 10080 80% 10080 override-expire override-lastmod reload-into-ims
> refresh_pattern . 0 20% 4320
>
> ###############################################################################
> ## ADMINISTRATIVE PARAMETERS
> ###############################################################################
> cache_mgr reetika at foxymoron.org <mailto:reetika at foxymoron.org>
> cache_effective_user proxy
> cache_effective_group proxy
> visible_hostname foxysquid.foxymoron.tv <http://foxysquid.foxymoron.tv>
> unique_hostname foxysquid.foxymoron.tv <http://foxysquid.foxymoron.tv>
>
> ###############################################################################
> ## PERSISTENT CONNECTION HANDLING
> ###############################################################################
> detect_broken_pconn on
> client_persistent_connections off
> server_persistent_connections on
>
> ###############################################################################
> ## ERROR PAGE OPTIONS
> ###############################################################################
> error_directory /usr/share/squid/errors/en
> error_log_languages off
>
> ###############################################################################
> ## DNS OPTIONS
> ###############################################################################
> check_hostnames off
> hosts_file /etc/hosts
> connect_retries 2
> ipcache_low 90
> ipcache_high 95
> ipcache_size 84024 # 2x Besar RAM
> fqdncache_size 64024 # real RAM Hardware
> pipeline_prefetch 100
>
> ###############################################################################
> ## MISCELLANEOUS
> ###############################################################################
> memory_pools off
> reload_into_ims on
> uri_whitespace strip
> max_filedescriptors 65536
>
> IPtable rules :
>
> ................................................
>
> My IPtables Rules
>
> Chain PREROUTING (policy ACCEPT 27405 packets, 1872K bytes)
> pkts bytes target prot opt in out source
> destination
> 76873 4457K DNAT tcp -- eth1 * 0.0.0.0/0 <http://0.0.0.0/0>
> 0.0.0.0/0 <http://0.0.0.0/0> tcp dpt:80 to:192.168.0.200:3129
> <http://192.168.0.200:3129>
> 26 1184 REDIRECT tcp -- eth0 * 0.0.0.0/0 <http://0.0.0.0/0>
> 0.0.0.0/0 <http://0.0.0.0/0> tcp dpt:80 redir ports 3129
> 0 0 DNAT tcp -- eth0 * 0.0.0.0/0 <http://0.0.0.0/0>
> 0.0.0.0/0 <http://0.0.0.0/0> tcp dpt:443 to:192.168.0.200:3130
> <http://192.168.0.200:3130>
>
> Chain INPUT (policy ACCEPT 9321 packets, 543K bytes)
> pkts bytes target prot opt in out source
> destination
>
> Chain OUTPUT (policy ACCEPT 1426 packets, 85560 bytes)
> pkts bytes target prot opt in out source
> destination
>
> Chain POSTROUTING (policy ACCEPT 1426 packets, 85560 bytes)
> pkts bytes target prot opt in out source
> destination
> 81432 14M MASQUERADE all -- * eth0 192.168.0.0/24
> <http://192.168.0.0/24> 0.0.0.0/0 <http://0.0.0.0/0>
>
> On Fri, Jun 5, 2015 at 1:43 PM, Reet Vyas <reet.vyas28 at gmail.com
> <mailto:reet.vyas28 at gmail.com>> wrote:
>
> Hi
>
> Thanks for reply. I am trying to cache youtube using this wiki
> http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube
> but I cant cache youtube.
>
> I want to cache facebook and youtube. SSl certificate installation
> that I have to do . Please suggest some links.
>
> On Thu, Jun 4, 2015 at 6:48 PM, Amos Jeffries
> <squid3 at treenet.co.nz <mailto:squid3 at treenet.co.nz>> wrote:
>
> On 5/06/2015 12:55 a.m., Reet Vyas wrote:
> > Thank you everyone for helping me to setup squid , Now its
> working but in
> > access.logs I only see tcp_miss if m using same website. I
> mean squid is
> > not caching
>
> You will get MISS a fair bit more with intercepted traffic
> than with
> normal proxied traffic. Particularly on certain major CDN who play
> tricks with DNS.
>
> The reasons and some workarounds to need to be doing are
> explained in
> <http://wiki.squid-cache.org/KnowledgeBase/HostHeaderForgery>
>
> Amos
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> <mailto:squid-users at lists.squid-cache.org>
> http://lists.squid-cache.org/listinfo/squid-users
>
>
>
>
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20150624/bd9cdc9c/attachment-0001.html>
More information about the squid-users
mailing list