[squid-users] Transparent Squid Proxy Server
Nathan Hoad
nathan at getoffmalawn.com
Sun Jul 12 01:02:35 UTC 2015
I'm using 3.5 with transparent server first bumping in ~100 deployments so
far, it works just fine, excluding with SNI and everything.
On 12/07/2015 10:58 am, "Yuri Voinov" <yvoinov at gmail.com> wrote:
> Man,
>
> 3.5.x don't work with server-first. It must be for backward compatibility
> - but don't be.
>
> Also, AFAIK, 3.5.x series don't work with transparent NAT interception in
> bump mode. Fake certs are generated, but with IP against hostnames (in all
> my test installations).
>
> So, if you strictly need working bump with transparent interception,
> rollback to 3.4.
>
> WBR, Yuri.
>
> 24.06.15 12:04, Reet Vyas пишет:
>
> Hi
> Below is my squid file , I have configured squid 3.5.3 with ssl, but I
> cant filter https traffic and also in access log I cant see https in access
> logs.
>
>
> #
> # Recommended minimum configuration:
> #
>
> # Example rule allowing access from your local networks.
> # Adapt to list your (internal) IP networks from where browsing
> # should be allowed
> acl localnet src 116.72.152.37 192.168.0.0/24 # Sesuaikan dengan ip
> client/local
>
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> # storeid *test*
> acl urlrewrite dstdomain .fbcdn.net .akamaihd.net
> acl speedtest url_regex -i speedtest\/.*\.(jpg|txt)\?.*
> acl reverbnation url_regex -i reverbnation.*audio_player.*ec_stream_song.*$
> acl utmgif url_regex -i utm.gif.*
> acl playstoreandroid url_regex -i
> c.android.clients.google.com.market.GetBinary.GetBinary.*
> acl idyoutube url_regex -i
> youtube.*(ptracking|stream_204|player_204).*(v\=|docid\=|video_id\=).*$
> acl videoyoutube url_regex -i (youtube|googlevideo).*videoplayback\?
> acl videoyoutube url_regex -i (youtube|googlevideo).*videoplayback\?
> acl CONNECT method CONNECT
> acl getmethod method GET
> acl loop_302 http_status 302
> acl step1 at_step SslBump1
> acl youtube dstdomain .youtube.com
> acl blocksites dstdomain "/etc/squid/restricted-sites.squid"
> # TAG: QUERY
> #
> -----------------------------------------------------------------------------
> acl QUERY urlpath_regex -i
> (hackshield|blank.html|infinity.js|hshield.da|renew_session_token.php|recaptcha.js|dat.asp|notice.swf|patchlist.txt|hackshield|captcha|reset.css|update.ver|notice.html|updates.txt|gamenotice|images.kom|patchinfo.xml|noupdate.ui|\.Xtp|\.htc|\.txt)
> acl QUERY urlpath_regex -i
> (patch.conf|uiimageset.xml.iop|gashaponwnd.xml.iop|loading.swf|download.swf|version.list|version.ini|launch.jnlp|server_patch.cfg.iop|core.swf|Loading.swf|resouececheck.sq|mainloading.swf|config.xml|gemmaze.swf|xml.png|size.xml|resourcesbar.swf|version.xml|version.list|delete.ini)
> acl QUERY urlpath_regex -i \.(jsp|asp|aspx|cfg|iop|zip|php|xml|html)(\?|$)
> cache deny QUERY
> cache deny youtube
>
> #
> acl dontstore url_regex ^http:\/\/((
> [\d\w-]*(\.[^\.\-]*?\..*?))(\/\mosalsal\/[\d]{4}\/.*\/)(.*\.flv))\?start.*
> acl dontstore url_regex redbot\.org \.php
> acl dontstore url_regex -i ^http:\/\/.*gemscool\.com\/.*
> acl dontstore url_regex \.(aspx|php)\?
> acl dontstore url_regex goldprice\.org\/NewCharts\/gold\/images\/.*\.png
> acl dontstore url_regex google\.co(m|\.[a-z]{2})\/complete\/search\?
> acl dontstore url_regex
> redirector\.([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com)\/(get_video\?|videodownload\?|videoplayback.*id|get_video_info\?|ptracking\?|player_204\?|stream_204\?).*
>
> acl store_yt_id url_regex -i
> youtube.*(ptracking|stream_204|playback|player_204|watchtime|set_awesome|s\?|ads).*(video_id|docid|\&v|content_v)\=([^\&\s]*).*$
> acl store_id_list_yt url_regex -i (youtube|googlevideo).*videoplayback.*$
> acl store_id_list_yt url_regex
> ^https?\:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com)\/(get_video\?|videodownload\?|videoplayback.*id).*
>
> acl store-id_list urlpath_regex -i dl\.sourceforge\.net
> acl store-id_list urlpath_regex -i \.ytimg\.com
> acl store-id_list urlpath_regex -i \.(akamaihd|fbcdn)\.net
> acl store_id_list urlpath_regex -i
> [a-zA-Z]{2}[0-9]*\.4shared\.com\/download\/
>
> acl store_id_list_url url_regex ^http:\/\/
> [0-9]\.bp\.blogspot\.com.*\.(jpeg|jpg|png|gif|ico)
> acl store_id_list_url url_regex
> ^http[s]?:\/\/.*\.twimg\.com\/(.*)\.(gif|jpeg|jpg|png|js|css)
> acl store_id_list_url url_regex
> ^http[s]?:\/\/(media|static)\.licdn\.com\/.*\.(png|jpg|gif|woff)
> acl store_id_list_url url_regex ^https:\/\/fb(static|cdn)\-.*\-
> a.akamaihd.net\/(.*)\.(gif|jpeg|jpg|png|js|css|mp4)
> acl store_id_list_url url_regex ^http:\/\/.*\.ak\.fbcdn\.net\/.*\.(gif
> |jpg|png|js|mp4)
>
> # pass requests
> url_rewrite_program /etc/squid/phpredir.php
> url_rewrite_access allow youtube
>
> request_header_access Range deny store_id_list_yt
> range_offset_limit 10 KB store_id_list_yt
>
>
>
> ###############################################################################
> # Recommended minimum Access Permission configuration:
> #
> # Deny requests to certain unsafe ports
>
> ###############################################################################
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access deny blocksites
> http_access allow localhost manager
> http_access deny manager
> http_access allow localnet
> http_access allow localhost
> http_access deny all
>
>
> ###############################################################################
> # squid ssl_bump option
>
> ###############################################################################
> always_direct allow all
> ssl_bump server-first all
> sslproxy_cert_error deny all
> sslproxy_flags DONT_VERIFY_PEER
> sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB
> sslcrtd_children 8 startup=1 idle=1
> #ssl_bump peek step1
> #ssl_bump bump all
>
> ###############################################################################
> # Squid normally listens to port 3128
>
> ###############################################################################
> https_port 3130 intercept ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_certs/squid.crt
> key=/etc/squid/ssl_certs/squid.key
> http_port 3129 intercept
> http_port 3128
>
> # TAG: Store-id Program
> #
> -----------------------------------------------------------------------------
> store_id_program /usr/bin/perl /etc/squid/store-id.pl
> store_id_children 100 startup=0 idle=1 concurrency=1000
>
> # TAG: Store-id Access
> #
> -----------------------------------------------------------------------------
> store_id_access allow urlrewrite
> store_id_access allow speedtest
> store_id_access allow reverbnation
> store_id_access allow utmgif
> store_id_access allow playstoreandroid
> store_id_access allow idyoutube
> store_id_access allow videoyoutube
> store_id_access deny dontstore
> store_id_access deny !getmethod
> store_id_access allow store_id_list_yt
> store_id_access allow store_yt_id
> store_id_access allow store-id_list
> store_id_access deny all
> store_id_bypass on
>
> # TAG: Youtube 302
> #
> -----------------------------------------------------------------------------
> store_miss deny store_id_list_yt loop_302
> send_hit deny store_id_list_yt loop_302
>
>
> ###############################################################################
> ## MEMORY CACHE OPTIONS
>
> ###############################################################################
> client_dst_passthru on
> cache_mem 1024 MB
> maximum_object_size_in_memory 1024 KB
> memory_cache_shared off
> memory_cache_mode disk
> memory_replacement_policy heap GDSF
>
>
> ###############################################################################
> ## DISK CACHE OPTIONS
>
> ###############################################################################
> cache_replacement_policy heap LFUDA
> minimum_object_size 1 bytes
> maximum_object_size 10 GB
>
>
> ###############################################################################
> # Uncomment and adjust the following to add a disk cache directory.
>
> ###############################################################################
> cache_dir aufs /usr/local/cache_proxy 25000 16 256 # sesuaikan dengan
> drive penyimpanan cache
> store_dir_select_algorithm round-robin
> cache_swap_low 90
> cache_swap_high 95
>
>
> ###############################################################################
> # Leave coredumps in the first cache dir
>
> ###############################################################################
> coredump_dir /var/spool/squid
>
>
> ###############################################################################
> ## LOGFILE OPTIONS
>
> ###############################################################################
> #access_log daemon:/tmp/access.log !log
> #logfile_daemon /usr/lib/squid/log_file_daemon
> cache_store_log none
> logfile_rotate 1
> mime_table /etc/squid/mime.conf
> pid_filename /var/run/squid.pid
> strip_query_terms off
> buffered_logs off
>
>
> ###############################################################################
> ## OPTIONS FOR TROUBLESHOOTING
>
> ###############################################################################
> #cache_log /tmp/cache.log
> cache_log /dev/null
> #debug_options ALL,1 22,3
> coredump_dir /var/spool/squid
>
>
> ###############################################################################
> ## OPTIONS FOR TUNING THE CACHE
>
> ###############################################################################
> max_stale 1 years
> vary_ignore_expire on
> shutdown_lifetime 10 seconds
>
>
> ###############################################################################
> # Add any of your own refresh_pattern entries above these.
>
> ###############################################################################
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> # Youtube Video
> refresh_pattern -i
> (get_video\?|videoplayback\?|videodownload\?|\.mp4|\.webm|\.flv|((audio|video)\/(webm|mp4)))
> 241920 100% 241920 override-expire ignore-reload ignore-private
> ignore-no-store ignore-must-revalidate reload-into-ims ignore-auth
> store-stale
> refresh_pattern -i ^https?\:\/\/.*\.googlevideo\.com\/videoplayback.*
> 10080 99% 43200 override-lastmod override-expire ignore-reload
> reload-into-ims ignore-private reload-into-ims ignore-auth store-stale
> refresh_pattern -i ^https?\:\/\/.*\.googlevideo\.com\/videoplayback.*$
> 241920 100% 241920 override-expire ignore-reload ignore-private
> ignore-no-store ignore-must-revalidate reload-into-ims ignore-auth
> store-stale
>
> refresh_pattern (akamaihd|fbcdn)\.net 14400 99% 518400 ignore-no-store
> ignore-private ignore-reload ignore-must-revalidate store-stale
> refresh_pattern -i squid\.internal 14400 99% 518400 ignore-no-store
> ignore-private ignore-reload ignore-must-revalidate store-stale
> refresh_pattern \.(jpg|png|gif|css|ico)($|\?) 14400 99% 518400
> ignore-no-store ignore-private reload-into-ims ignore-must-revalidate
> store-stale
> refresh_pattern . 0 99% 518400 ignore-no-store ignore-private
> reload-into-ims store-stale
> # Image Youtube
> refresh_pattern -i (yimg|twimg)\.com\.* 1440 100% 129600
> override-expire ignore-reload reload-into-ims
> refresh_pattern -i (ytimg|ggpht)\.com\.* 1440 80% 129600
> override-expire override-lastmod ignore-auth ignore-reload reload-into-ims
>
> #images facebook
> refresh_pattern -i
> fbcdn.*net\/.*\.((jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)|(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)(\?|.*$))
> 241920 99% 241920 ignore-no-store ignore-private override-expire
> override-lastmod reload-into-ims ignore-auth
> refresh_pattern -i pixel\.facebook\.com.*\.(jpg|png|gif|ico|css|js) 241920
> 80% 241920 override-expire ignore-reload reload-into-ims ignore-auth
> refresh_pattern -i \.akamaihd\.net.*\.(jpg|png|gif|ico|css|js) 241920 80%
> 241920 override-expire ignore-reload reload-into-ims ignore-auth
> refresh_pattern -i ((facebook.com)|(85.131.151.39))\.(jpg|png|gif) 241920
> 99% 241920 ignore-reload override-expire ignore-no-store store-stale
> refresh_pattern -i
> fbcdn\.net\/.*\.((jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)|(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)(\?|.*$))
> 241920 99% 241920 ignore-no-store ignore-private override-expire
> override-lastmod reload-into-ims ignore-auth
> refresh_pattern static\.(xx|ak)\.fbcdn\.net*\.(jpg|gif|png) 241920 99%
> 241920 ignore-reload override-expire ignore-no-store
> refresh_pattern ^https?\:\/\/profile\.ak\.fbcdn.net*\.(jpg|gif|png)
> 241920 99% 241920 ignore-reload override-expire ignore-no-store
>
> # Video Facebook
> refresh_pattern -i
> \.video.ak.fbcdn.net.*\.(mp4|flv|mp3|amf) 10080 80%
> 43200 override-expire ignore-reload reload-into-ims ignore-private
> ignore-no-store ignore-must-revalidate
> refresh_pattern (audio|video)\/(webm|mp4) 129600 99% 129600 ignore-reload
> override-expire override-lastmod ignore-must-revalidate ignore-private
> ignore-no-store ignore-auth store-stale
> refresh_pattern -i ^http://.*squid\.internal.* 241920 100% 241920
> override-lastmod override-expire ignore-reload ignore-must-revalidate
> ignore-private ignore-no-store ignore-auth store-stale
>
> # All File
> refresh_pattern -i
> \.(3gp|7z|ace|asx|bin|deb|divx|dvr-ms|ram|rpm|exe|inc|cab|qt) 10080 80%
> 10080 override-expire override-lastmod reload-into-ims
> refresh_pattern -i
> \.(rar|jar|gz|tgz|bz2|iso|m1v|m2(v|p)|mo(d|v)|arj|lha|lzh|zip|tar|iop|nzp|pak|mar|msp)
> 10080 80% 10080 override-expire override-lastmod reload-into-ims
> ignore-reload
> refresh_pattern -i
> \.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|dat|ad|txt|dll) 10080 80% 10080
> override-expire override-lastmod reload-into-ims
> refresh_pattern -i
> \.(avi|ac4|mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p)|og(x|v|a|g)|rm|r(a|p)m|snd|vob|webm)
> 10080 80% 10080 override-expire override-lastmod reload-into-ims
> refresh_pattern -i
> \.(pp(t?x)|s|t)|pdf|rtf|wax|wm(a|v)|wmx|wpl|cb(r|z|t)|xl(s?x)|do(c?x)|flv|x-flv)
> 10080 80% 10080 override-expire override-lastmod reload-into-ims
> refresh_pattern . 0 20% 4320
>
>
> ###############################################################################
> ## ADMINISTRATIVE PARAMETERS
>
> ###############################################################################
> cache_mgr reetika at foxymoron.org
> cache_effective_user proxy
> cache_effective_group proxy
> visible_hostname foxysquid.foxymoron.tv
> unique_hostname foxysquid.foxymoron.tv
>
>
> ###############################################################################
> ## PERSISTENT CONNECTION HANDLING
>
> ###############################################################################
> detect_broken_pconn on
> client_persistent_connections off
> server_persistent_connections on
>
>
> ###############################################################################
> ## ERROR PAGE OPTIONS
>
> ###############################################################################
> error_directory /usr/share/squid/errors/en
> error_log_languages off
>
>
> ###############################################################################
> ## DNS OPTIONS
>
> ###############################################################################
> check_hostnames off
> hosts_file /etc/hosts
> connect_retries 2
> ipcache_low 90
> ipcache_high 95
> ipcache_size 84024 # 2x Besar RAM
> fqdncache_size 64024 # real RAM Hardware
> pipeline_prefetch 100
>
>
> ###############################################################################
> ## MISCELLANEOUS
>
> ###############################################################################
> memory_pools off
> reload_into_ims on
> uri_whitespace strip
> max_filedescriptors 65536
>
> IPtable rules :
>
> ................................................
>
> My IPtables Rules
>
> Chain PREROUTING (policy ACCEPT 27405 packets, 1872K bytes)
> pkts bytes target prot opt in out source
> destination
> 76873 4457K DNAT tcp -- eth1 * 0.0.0.0/0
> 0.0.0.0/0 tcp dpt:80 to:192.168.0.200:3129
> 26 1184 REDIRECT tcp -- eth0 * 0.0.0.0/0
> 0.0.0.0/0 tcp dpt:80 redir ports 3129
> 0 0 DNAT tcp -- eth0 * 0.0.0.0/0
> 0.0.0.0/0 tcp dpt:443 to:192.168.0.200:3130
>
> Chain INPUT (policy ACCEPT 9321 packets, 543K bytes)
> pkts bytes target prot opt in out source
> destination
>
> Chain OUTPUT (policy ACCEPT 1426 packets, 85560 bytes)
> pkts bytes target prot opt in out source
> destination
>
> Chain POSTROUTING (policy ACCEPT 1426 packets, 85560 bytes)
> pkts bytes target prot opt in out source
> destination
> 81432 14M MASQUERADE all -- * eth0 192.168.0.0/24
> 0.0.0.0/0
>
> On Fri, Jun 5, 2015 at 1:43 PM, Reet Vyas <reet.vyas28 at gmail.com> wrote:
>
>> Hi
>>
>> Thanks for reply. I am trying to cache youtube using this wiki
>> http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube but I
>> cant cache youtube.
>>
>> I want to cache facebook and youtube. SSl certificate installation that
>> I have to do . Please suggest some links.
>>
>> On Thu, Jun 4, 2015 at 6:48 PM, Amos Jeffries <squid3 at treenet.co.nz>
>> wrote:
>>
>>> On 5/06/2015 12:55 a.m., Reet Vyas wrote:
>>> > Thank you everyone for helping me to setup squid , Now its working but
>>> in
>>> > access.logs I only see tcp_miss if m using same website. I mean squid
>>> is
>>> > not caching
>>>
>>> You will get MISS a fair bit more with intercepted traffic than with
>>> normal proxied traffic. Particularly on certain major CDN who play
>>> tricks with DNS.
>>>
>>> The reasons and some workarounds to need to be doing are explained in
>>> <http://wiki.squid-cache.org/KnowledgeBase/HostHeaderForgery>
>>>
>>> Amos
>>>
>>> _______________________________________________
>>> squid-users mailing list
>>> squid-users at lists.squid-cache.org
>>> http://lists.squid-cache.org/listinfo/squid-users
>>>
>>
>>
>
>
> _______________________________________________
> squid-users mailing listsquid-users at lists.squid-cache.orghttp://lists.squid-cache.org/listinfo/squid-users
>
>
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20150712/7ff761e0/attachment-0001.html>
More information about the squid-users
mailing list