[squid-users] Transparent Squid Proxy Server
Reet Vyas
reet.vyas28 at gmail.com
Wed Jun 24 06:04:37 UTC 2015
Hi
Below is my squid file , I have configured squid 3.5.3 with ssl, but I
cant filter https traffic and also in access log I cant see https in access
logs.
#
# Recommended minimum configuration:
#
# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 116.72.152.37 192.168.0.0/24 # Sesuaikan dengan ip
client/local
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
# storeid *test*
acl urlrewrite dstdomain .fbcdn.net .akamaihd.net
acl speedtest url_regex -i speedtest\/.*\.(jpg|txt)\?.*
acl reverbnation url_regex -i reverbnation.*audio_player.*ec_stream_song.*$
acl utmgif url_regex -i utm.gif.*
acl playstoreandroid url_regex -i
c.android.clients.google.com.market.GetBinary.GetBinary.*
acl idyoutube url_regex -i
youtube.*(ptracking|stream_204|player_204).*(v\=|docid\=|video_id\=).*$
acl videoyoutube url_regex -i (youtube|googlevideo).*videoplayback\?
acl videoyoutube url_regex -i (youtube|googlevideo).*videoplayback\?
acl CONNECT method CONNECT
acl getmethod method GET
acl loop_302 http_status 302
acl step1 at_step SslBump1
acl youtube dstdomain .youtube.com
acl blocksites dstdomain "/etc/squid/restricted-sites.squid"
# TAG: QUERY
#
-----------------------------------------------------------------------------
acl QUERY urlpath_regex -i
(hackshield|blank.html|infinity.js|hshield.da|renew_session_token.php|recaptcha.js|dat.asp|notice.swf|patchlist.txt|hackshield|captcha|reset.css|update.ver|notice.html|updates.txt|gamenotice|images.kom|patchinfo.xml|noupdate.ui|\.Xtp|\.htc|\.txt)
acl QUERY urlpath_regex -i
(patch.conf|uiimageset.xml.iop|gashaponwnd.xml.iop|loading.swf|download.swf|version.list|version.ini|launch.jnlp|server_patch.cfg.iop|core.swf|Loading.swf|resouececheck.sq|mainloading.swf|config.xml|gemmaze.swf|xml.png|size.xml|resourcesbar.swf|version.xml|version.list|delete.ini)
acl QUERY urlpath_regex -i \.(jsp|asp|aspx|cfg|iop|zip|php|xml|html)(\?|$)
cache deny QUERY
cache deny youtube
#
acl dontstore url_regex
^http:\/\/(([\d\w-]*(\.[^\.\-]*?\..*?))(\/\mosalsal\/[\d]{4}\/.*\/)(.*\.flv))\?start.*
acl dontstore url_regex redbot\.org \.php
acl dontstore url_regex -i ^http:\/\/.*gemscool\.com\/.*
acl dontstore url_regex \.(aspx|php)\?
acl dontstore url_regex goldprice\.org\/NewCharts\/gold\/images\/.*\.png
acl dontstore url_regex google\.co(m|\.[a-z]{2})\/complete\/search\?
acl dontstore url_regex
redirector\.([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com)\/(get_video\?|videodownload\?|videoplayback.*id|get_video_info\?|ptracking\?|player_204\?|stream_204\?).*
acl store_yt_id url_regex -i
youtube.*(ptracking|stream_204|playback|player_204|watchtime|set_awesome|s\?|ads).*(video_id|docid|\&v|content_v)\=([^\&\s]*).*$
acl store_id_list_yt url_regex -i (youtube|googlevideo).*videoplayback.*$
acl store_id_list_yt url_regex
^https?\:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com)\/(get_video\?|videodownload\?|videoplayback.*id).*
acl store-id_list urlpath_regex -i dl\.sourceforge\.net
acl store-id_list urlpath_regex -i \.ytimg\.com
acl store-id_list urlpath_regex -i \.(akamaihd|fbcdn)\.net
acl store_id_list urlpath_regex -i
[a-zA-Z]{2}[0-9]*\.4shared\.com\/download\/
acl store_id_list_url url_regex
^http:\/\/[0-9]\.bp\.blogspot\.com.*\.(jpeg|jpg|png|gif|ico)
acl store_id_list_url url_regex
^http[s]?:\/\/.*\.twimg\.com\/(.*)\.(gif|jpeg|jpg|png|js|css)
acl store_id_list_url url_regex
^http[s]?:\/\/(media|static)\.licdn\.com\/.*\.(png|jpg|gif|woff)
acl store_id_list_url url_regex ^https:\/\/fb(static|cdn)\-.*\-
a.akamaihd.net\/(.*)\.(gif|jpeg|jpg|png|js|css|mp4)
acl store_id_list_url url_regex
^http:\/\/.*\.ak\.fbcdn\.net\/.*\.(gif|jpg|png|js|mp4)
# pass requests
url_rewrite_program /etc/squid/phpredir.php
url_rewrite_access allow youtube
request_header_access Range deny store_id_list_yt
range_offset_limit 10 KB store_id_list_yt
###############################################################################
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
###############################################################################
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny blocksites
http_access allow localhost manager
http_access deny manager
http_access allow localnet
http_access allow localhost
http_access deny all
###############################################################################
# squid ssl_bump option
###############################################################################
always_direct allow all
ssl_bump server-first all
sslproxy_cert_error deny all
sslproxy_flags DONT_VERIFY_PEER
sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB
sslcrtd_children 8 startup=1 idle=1
#ssl_bump peek step1
#ssl_bump bump all
###############################################################################
# Squid normally listens to port 3128
###############################################################################
https_port 3130 intercept ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_certs/squid.crt
key=/etc/squid/ssl_certs/squid.key
http_port 3129 intercept
http_port 3128
# TAG: Store-id Program
#
-----------------------------------------------------------------------------
store_id_program /usr/bin/perl /etc/squid/store-id.pl
store_id_children 100 startup=0 idle=1 concurrency=1000
# TAG: Store-id Access
#
-----------------------------------------------------------------------------
store_id_access allow urlrewrite
store_id_access allow speedtest
store_id_access allow reverbnation
store_id_access allow utmgif
store_id_access allow playstoreandroid
store_id_access allow idyoutube
store_id_access allow videoyoutube
store_id_access deny dontstore
store_id_access deny !getmethod
store_id_access allow store_id_list_yt
store_id_access allow store_yt_id
store_id_access allow store-id_list
store_id_access deny all
store_id_bypass on
# TAG: Youtube 302
#
-----------------------------------------------------------------------------
store_miss deny store_id_list_yt loop_302
send_hit deny store_id_list_yt loop_302
###############################################################################
## MEMORY CACHE OPTIONS
###############################################################################
client_dst_passthru on
cache_mem 1024 MB
maximum_object_size_in_memory 1024 KB
memory_cache_shared off
memory_cache_mode disk
memory_replacement_policy heap GDSF
###############################################################################
## DISK CACHE OPTIONS
###############################################################################
cache_replacement_policy heap LFUDA
minimum_object_size 1 bytes
maximum_object_size 10 GB
###############################################################################
# Uncomment and adjust the following to add a disk cache directory.
###############################################################################
cache_dir aufs /usr/local/cache_proxy 25000 16 256 # sesuaikan dengan drive
penyimpanan cache
store_dir_select_algorithm round-robin
cache_swap_low 90
cache_swap_high 95
###############################################################################
# Leave coredumps in the first cache dir
###############################################################################
coredump_dir /var/spool/squid
###############################################################################
## LOGFILE OPTIONS
###############################################################################
#access_log daemon:/tmp/access.log !log
#logfile_daemon /usr/lib/squid/log_file_daemon
cache_store_log none
logfile_rotate 1
mime_table /etc/squid/mime.conf
pid_filename /var/run/squid.pid
strip_query_terms off
buffered_logs off
###############################################################################
## OPTIONS FOR TROUBLESHOOTING
###############################################################################
#cache_log /tmp/cache.log
cache_log /dev/null
#debug_options ALL,1 22,3
coredump_dir /var/spool/squid
###############################################################################
## OPTIONS FOR TUNING THE CACHE
###############################################################################
max_stale 1 years
vary_ignore_expire on
shutdown_lifetime 10 seconds
###############################################################################
# Add any of your own refresh_pattern entries above these.
###############################################################################
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
# Youtube Video
refresh_pattern -i
(get_video\?|videoplayback\?|videodownload\?|\.mp4|\.webm|\.flv|((audio|video)\/(webm|mp4)))
241920 100% 241920 override-expire ignore-reload ignore-private
ignore-no-store ignore-must-revalidate reload-into-ims ignore-auth
store-stale
refresh_pattern -i ^https?\:\/\/.*\.googlevideo\.com\/videoplayback.*
10080 99% 43200 override-lastmod override-expire ignore-reload
reload-into-ims ignore-private reload-into-ims ignore-auth store-stale
refresh_pattern -i ^https?\:\/\/.*\.googlevideo\.com\/videoplayback.*$
241920 100% 241920 override-expire ignore-reload ignore-private
ignore-no-store ignore-must-revalidate reload-into-ims ignore-auth
store-stale
refresh_pattern (akamaihd|fbcdn)\.net 14400 99% 518400 ignore-no-store
ignore-private ignore-reload ignore-must-revalidate store-stale
refresh_pattern -i squid\.internal 14400 99% 518400 ignore-no-store
ignore-private ignore-reload ignore-must-revalidate store-stale
refresh_pattern \.(jpg|png|gif|css|ico)($|\?) 14400 99% 518400
ignore-no-store ignore-private reload-into-ims ignore-must-revalidate
store-stale
refresh_pattern . 0 99% 518400 ignore-no-store ignore-private
reload-into-ims store-stale
# Image Youtube
refresh_pattern -i (yimg|twimg)\.com\.* 1440 100% 129600
override-expire ignore-reload reload-into-ims
refresh_pattern -i (ytimg|ggpht)\.com\.* 1440 80% 129600
override-expire override-lastmod ignore-auth ignore-reload reload-into-ims
#images facebook
refresh_pattern -i
fbcdn.*net\/.*\.((jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)|(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)(\?|.*$))
241920 99% 241920 ignore-no-store ignore-private override-expire
override-lastmod reload-into-ims ignore-auth
refresh_pattern -i pixel\.facebook\.com.*\.(jpg|png|gif|ico|css|js) 241920
80% 241920 override-expire ignore-reload reload-into-ims ignore-auth
refresh_pattern -i \.akamaihd\.net.*\.(jpg|png|gif|ico|css|js) 241920 80%
241920 override-expire ignore-reload reload-into-ims ignore-auth
refresh_pattern -i ((facebook.com)|(85.131.151.39))\.(jpg|png|gif) 241920
99% 241920 ignore-reload override-expire ignore-no-store store-stale
refresh_pattern -i
fbcdn\.net\/.*\.((jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)|(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|css|js)(\?|.*$))
241920 99% 241920 ignore-no-store ignore-private override-expire
override-lastmod reload-into-ims ignore-auth
refresh_pattern static\.(xx|ak)\.fbcdn\.net*\.(jpg|gif|png) 241920 99%
241920 ignore-reload override-expire ignore-no-store
refresh_pattern ^https?\:\/\/profile\.ak\.fbcdn.net*\.(jpg|gif|png) 241920
99% 241920 ignore-reload override-expire ignore-no-store
# Video Facebook
refresh_pattern -i
\.video.ak.fbcdn.net.*\.(mp4|flv|mp3|amf) 10080 80%
43200 override-expire ignore-reload reload-into-ims ignore-private
ignore-no-store ignore-must-revalidate
refresh_pattern (audio|video)\/(webm|mp4) 129600 99% 129600 ignore-reload
override-expire override-lastmod ignore-must-revalidate ignore-private
ignore-no-store ignore-auth store-stale
refresh_pattern -i ^http://.*squid\.internal.* 241920 100% 241920
override-lastmod override-expire ignore-reload ignore-must-revalidate
ignore-private ignore-no-store ignore-auth store-stale
# All File
refresh_pattern -i
\.(3gp|7z|ace|asx|bin|deb|divx|dvr-ms|ram|rpm|exe|inc|cab|qt) 10080 80%
10080 override-expire override-lastmod reload-into-ims
refresh_pattern -i
\.(rar|jar|gz|tgz|bz2|iso|m1v|m2(v|p)|mo(d|v)|arj|lha|lzh|zip|tar|iop|nzp|pak|mar|msp)
10080 80% 10080 override-expire override-lastmod reload-into-ims
ignore-reload
refresh_pattern -i
\.(jp(e?g|e|2)|gif|pn[pg]|bm?|tiff?|ico|swf|dat|ad|txt|dll) 10080 80% 10080
override-expire override-lastmod reload-into-ims
refresh_pattern -i
\.(avi|ac4|mp(e?g|a|e|1|2|3|4)|mk(a|v)|ms(i|u|p)|og(x|v|a|g)|rm|r(a|p)m|snd|vob|webm)
10080 80% 10080 override-expire override-lastmod reload-into-ims
refresh_pattern -i
\.(pp(t?x)|s|t)|pdf|rtf|wax|wm(a|v)|wmx|wpl|cb(r|z|t)|xl(s?x)|do(c?x)|flv|x-flv)
10080 80% 10080 override-expire override-lastmod reload-into-ims
refresh_pattern . 0 20% 4320
###############################################################################
## ADMINISTRATIVE PARAMETERS
###############################################################################
cache_mgr reetika at foxymoron.org
cache_effective_user proxy
cache_effective_group proxy
visible_hostname foxysquid.foxymoron.tv
unique_hostname foxysquid.foxymoron.tv
###############################################################################
## PERSISTENT CONNECTION HANDLING
###############################################################################
detect_broken_pconn on
client_persistent_connections off
server_persistent_connections on
###############################################################################
## ERROR PAGE OPTIONS
###############################################################################
error_directory /usr/share/squid/errors/en
error_log_languages off
###############################################################################
## DNS OPTIONS
###############################################################################
check_hostnames off
hosts_file /etc/hosts
connect_retries 2
ipcache_low 90
ipcache_high 95
ipcache_size 84024 # 2x Besar RAM
fqdncache_size 64024 # real RAM Hardware
pipeline_prefetch 100
###############################################################################
## MISCELLANEOUS
###############################################################################
memory_pools off
reload_into_ims on
uri_whitespace strip
max_filedescriptors 65536
IPtable rules :
................................................
My IPtables Rules
Chain PREROUTING (policy ACCEPT 27405 packets, 1872K bytes)
pkts bytes target prot opt in out source
destination
76873 4457K DNAT tcp -- eth1 * 0.0.0.0/0
0.0.0.0/0 tcp dpt:80 to:192.168.0.200:3129
26 1184 REDIRECT tcp -- eth0 * 0.0.0.0/0
0.0.0.0/0 tcp dpt:80 redir ports 3129
0 0 DNAT tcp -- eth0 * 0.0.0.0/0
0.0.0.0/0 tcp dpt:443 to:192.168.0.200:3130
Chain INPUT (policy ACCEPT 9321 packets, 543K bytes)
pkts bytes target prot opt in out source
destination
Chain OUTPUT (policy ACCEPT 1426 packets, 85560 bytes)
pkts bytes target prot opt in out source
destination
Chain POSTROUTING (policy ACCEPT 1426 packets, 85560 bytes)
pkts bytes target prot opt in out source
destination
81432 14M MASQUERADE all -- * eth0 192.168.0.0/24
0.0.0.0/0
On Fri, Jun 5, 2015 at 1:43 PM, Reet Vyas <reet.vyas28 at gmail.com> wrote:
> Hi
>
> Thanks for reply. I am trying to cache youtube using this wiki
> http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube but I
> cant cache youtube.
>
> I want to cache facebook and youtube. SSl certificate installation that I
> have to do . Please suggest some links.
>
> On Thu, Jun 4, 2015 at 6:48 PM, Amos Jeffries <squid3 at treenet.co.nz>
> wrote:
>
>> On 5/06/2015 12:55 a.m., Reet Vyas wrote:
>> > Thank you everyone for helping me to setup squid , Now its working but
>> in
>> > access.logs I only see tcp_miss if m using same website. I mean squid
>> is
>> > not caching
>>
>> You will get MISS a fair bit more with intercepted traffic than with
>> normal proxied traffic. Particularly on certain major CDN who play
>> tricks with DNS.
>>
>> The reasons and some workarounds to need to be doing are explained in
>> <http://wiki.squid-cache.org/KnowledgeBase/HostHeaderForgery>
>>
>> Amos
>>
>> _______________________________________________
>> squid-users mailing list
>> squid-users at lists.squid-cache.org
>> http://lists.squid-cache.org/listinfo/squid-users
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20150624/4aa3fba1/attachment-0001.html>
More information about the squid-users
mailing list