[squid-users] Youtube redirection loop?

Yuri Voinov yvoinov at gmail.com
Tue May 5 09:59:09 UTC 2015



05.05.15 4:07, HackXBack пишет:
> Okay Sir,
> this is the solution
>
> 1st: put this conf in your squid.conf
>
> ####for looping 302 on youtube
> acl text-html rep_mime_type text/html
> acl http302 http_status 302
> store_miss deny text-html
> store_miss deny http302
> send_hit deny text-html
> send_hit deny http302
This works on 3.5.x and above only. store_* directives absent in 3.4.x 
series.
>
>
> 2nd: use this patch:
>
>
>
> --- src/client_side_request.cc  2014-03-09 06:40:56.000000000 -0300
> +++ src/client_side_request.cc  2014-04-21 02:53:11.277155130 -0300
> @@ -545,6 +545,16 @@
>               }
>               debugs(85, 3, HERE << "validate IP " << clientConn->local << "
> non-match from Host: IP " << ia->in_addrs[i]);
>           }
> +
> +        if (true) {
> +            unsigned short port = clientConn->local.port();
> +            debugs(85, 3, HERE << "[anti-forgery] Host-non-matched remote
> IP (" << clientConn->local << ") was replaced with the first Host resolved
> IP (" << ia->in_addrs[0] << ":" << clientConn->local.port() << ")");
> +            clientConn->local = ia->in_addrs[0];
> +            clientConn->local.port(port);
> +            http->request->flags.hostVerified = true;
> +            http->doCallouts();
> +            return;
> +        }
>       }
>       debugs(85, 3, HERE << "FAIL: validate IP " << clientConn->local << "
> possible from Host:");
>       hostHeaderVerifyFailed("local IP", "any domain IP");
>
>
> --- src/Server.cc
> +++ src/Server.cc
> @@ -31,6 +31,7 @@
>    */
>   
>   #include "squid.h"
> +#include "acl/FilledChecklist.h"
>   #include "acl/Gadgets.h"
>   #include "base/TextException.h"
>   #include "comm/Connection.h"
> @@ -174,6 +175,8 @@
>       // give entry the reply because haveParsedReplyHeaders() expects it
> there
>       entry->replaceHttpReply(theFinalReply, false); // but do not write yet
>       haveParsedReplyHeaders(); // update the entry/reply (e.g., set
> timestamps)
> +    if (EBIT_TEST(entry->flags, ENTRY_CACHABLE) && blockCaching())
> +        entry->release();
>       entry->startWriting(); // write the updated entry to store
>   
>       return theFinalReply;
> @@ -533,6 +536,24 @@
>       currentOffset = partial ? theFinalReply->content_range->spec.offset :
> 0;
>   }
>   
> +/// whether to prevent caching of an otherwise cachable response
> +bool
> +ServerStateData::blockCaching()
> +{
> +    if (const Acl::Tree *acl = Config.accessList.storeMiss) {
> +        // This relatively expensive check is not in
> StoreEntry::checkCachable:
> +        // That method lacks HttpRequest and may be called too many times.
> +        ACLFilledChecklist ch(acl, originalRequest(), NULL);
> +        ch.reply = const_cast<HttpReply*>(entry->getReply()); //
> ACLFilledChecklist API bug
> +        HTTPMSGLOCK(ch.reply);
> +        if (ch.fastCheck() != ACCESS_ALLOWED) { // when in doubt, block
> +            debugs(20, 3, "store_miss prohibits caching");
> +            return true;
> +        }
> +    }
> +    return false;
> +}
> +
>   HttpRequest *
>   ServerStateData::originalRequest()
>   {
> --- src/Server.h
> +++ src/Server.h
> @@ -131,6 +131,8 @@
>       /// Entry-dependent callbacks use this check to quit if the entry went
> bad
>       bool abortOnBadEntry(const char *abortReason);
>   
> +    bool blockCaching();
> +
>   #if USE_ADAPTATION
>       void startAdaptation(const Adaptation::ServiceGroupPointer &group,
> HttpRequest *cause);
>       void adaptVirginReplyBody(const char *buf, ssize_t len);
> --- src/SquidConfig.h
> +++ src/SquidConfig.h
> @@ -375,6 +375,8 @@
>           acl_access *AlwaysDirect;
>           acl_access *ASlists;
>           acl_access *noCache;
> +        acl_access *sendHit;
> +        acl_access *storeMiss;
>           acl_access *stats_collection;
>   #if SQUID_SNMP
>   
> --- src/cf.data.pre
> +++ src/cf.data.pre
> @@ -4843,18 +4843,97 @@
>   NAME: cache no_cache
>   TYPE: acl_access
>   DEFAULT: none
> -DEFAULT_DOC: Allow caching, unless rules exist in squid.conf.
> +DEFAULT_DOC: By default, this directive is unused and has no effect.
>   LOC: Config.accessList.noCache
>   DOC_START
> -	A list of ACL elements which, if matched and denied, cause the request to
> -	not be satisfied from the cache and the reply to not be cached.
> -	In other words, use this to force certain objects to never be cached.
> -
> -	You must use the words 'allow' or 'deny' to indicate whether items
> -	matching the ACL should be allowed or denied into the cache.
> +	Requests denied by this directive will not be served from the cache
> +	and their responses will not be stored in the cache. This directive
> +	has no effect on other transactions and on already cached responses.
>   
>   	This clause supports both fast and slow acl types.
>   	See http://wiki.squid-cache.org/SquidFaq/SquidAcl for details.
> +
> +	This and the two other similar caching directives listed below are
> +	checked at different transaction processing stages, have different
> +	access to response information, affect different cache operations,
> +	and differ in slow ACLs support:
> +
> +	* cache: Checked before Squid makes a hit/miss determination.
> +		No access to reply information!
> +		Denies both serving a hit and storing a miss.
> +		Supports both fast and slow ACLs.
> +	* send_hit: Checked after a hit was detected.
> +		Has access to reply (hit) information.
> +		Denies serving a hit only.
> +		Supports fast ACLs only.
> +	* store_miss: Checked before storing a cachable miss.
> +		Has access to reply (miss) information.
> +		Denies storing a miss only.
> +		Supports fast ACLs only.
> +
> +	If you are not sure which of the three directives to use, apply the
> +	following decision logic:
> +
> +	* If your ACL(s) are of slow type _and_ need response info, redesign.
> +	  Squid does not support that particular combination at this time.
> +        Otherwise:
> +	* If your directive ACL(s) are of slow type, use "cache"; and/or
> +	* if your directive ACL(s) need no response info, use "cache".
> +        Otherwise:
> +	* If you do not want the response cached, use store_miss; and/or
> +	* if you do not want a hit on a cached response, use send_hit.
> +DOC_END
> +
> +NAME: send_hit
> +TYPE: acl_access
> +DEFAULT: none
> +DEFAULT_DOC: By default, this directive is unused and has no effect.
> +LOC: Config.accessList.sendHit
> +DOC_START
> +	Responses denied by this directive will not be served from the cache
> +	(but may still be cached, see store_miss). This directive has no
> +	effect on the responses it allows and on the cached objects.
> +
> +	Please see the "cache" directive for a summary of differences among
> +	store_miss, send_hit, and cache directives.
> +
> +	Unlike the "cache" directive, send_hit only supports fast acl
> +	types.  See http://wiki.squid-cache.org/SquidFaq/SquidAcl for details.
> +
> +	For example:
> +
> +		# apply custom Store ID mapping to some URLs
> +		acl MapMe dstdomain .c.example.com
> +		store_id_program ...
> +		store_id_access allow MapMe
> +
> +		# but prevent caching of special responses
> +		# such as 302 redirects that cause StoreID loops
> +		acl Ordinary http_status 200-299
> +		store_miss deny MapMe !Ordinary
> +
> +		# and do not serve any previously stored special responses
> +		# from the cache (in case they were already cached before
> +		# the above store_miss rule was in effect).
> +		send_hit deny MapMe !Ordinary
> +DOC_END
> +
> +NAME: store_miss
> +TYPE: acl_access
> +DEFAULT: none
> +DEFAULT_DOC: By default, this directive is unused and has no effect.
> +LOC: Config.accessList.storeMiss
> +DOC_START
> +	Responses denied by this directive will not be cached (but may still
> +	be served from the cache, see send_hit). This directive has no
> +	effect on the responses it allows and on the already cached responses.
> +
> +	Please see the "cache" directive for a summary of differences among
> +	store_miss, send_hit, and cache directives. See the
> +	send_hit directive for a usage example.
> +
> +	Unlike the "cache" directive, store_miss only supports fast acl
> +	types.  See http://wiki.squid-cache.org/SquidFaq/SquidAcl for details.
>   DOC_END
>   
>   NAME: max_stale
> --- src/client_side_reply.cc
> +++ src/client_side_reply.cc
> @@ -545,6 +545,11 @@
>          ) {
>           http->logType = LOG_TCP_NEGATIVE_HIT;
>           sendMoreData(result);
> +    } else if (blockedHit()) {
> +        debugs(88, 5, "send_hit forces a MISS");
> +        http->logType = LOG_TCP_MISS;
> +        processMiss();
> +        return;
>       } else if (!http->flags.internal && refreshCheckHTTP(e, r)) {
>           debugs(88, 5, "clientCacheHit: in refreshCheck() block");
>           /*
> @@ -773,6 +778,30 @@
>       }
>   }
>   
> +/// whether squid.conf send_hit prevents us from serving this hit
> +bool
> +clientReplyContext::blockedHit() const
> +{
> +    if (!Config.accessList.sendHit)
> +        return false; // hits are not blocked by default
> +
> +    if (http->flags.internal)
> +        return false; // internal content "hits" cannot be blocked
> +
> +    if (const HttpReply *rep = http->storeEntry()->getReply()) {
> +        std::auto_ptr<ACLFilledChecklist>
> chl(clientAclChecklistCreate(Config.accessList.sendHit, http));
> +        chl->reply = const_cast<HttpReply*>(rep); // ACLChecklist API bug
> +        HTTPMSGLOCK(chl->reply);
> +        return chl->fastCheck() != ACCESS_ALLOWED; // when in doubt, block
> +    }
> +
> +    // This does not happen, I hope, because we are called from CacheHit,
> which
> +    // is called via a storeClientCopy() callback, and store should
> initialize
> +    // the reply before calling that callback.
> +    debugs(88, 3, "Missing reply!");
> +    return false;
> +}
> +
>   void
>   clientReplyContext::purgeRequestFindObjectToPurge()
>   {
> --- src/client_side_reply.h
> +++ src/client_side_reply.h
> @@ -140,6 +140,7 @@
>       void triggerInitialStoreRead();
>       void sendClientOldEntry();
>       void purgeAllCached();
> +    bool blockedHit() const;
>   
>       void sendBodyTooLargeError();
>       void sendPreconditionFailedError();
This is also not solution. One of most biggest traffic source must have 
native solution in proxy. Not a crutch.
>
>
>
> --
> View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-redirection-loop-tp4671084p4671103.html
> Sent from the Squid - Users mailing list archive at Nabble.com.
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

Again. I want simple thing: to limit selected mime-types caching from 
selected domains. Point.
On my opinion, this will completely solve YT caching problem without any 
crutches/patches.



More information about the squid-users mailing list