[squid-users] Youtube redirection loop?
Yuri Voinov
yvoinov at gmail.com
Tue May 5 09:59:09 UTC 2015
05.05.15 4:07, HackXBack пишет:
> Okay Sir,
> this is the solution
>
> 1st: put this conf in your squid.conf
>
> ####for looping 302 on youtube
> acl text-html rep_mime_type text/html
> acl http302 http_status 302
> store_miss deny text-html
> store_miss deny http302
> send_hit deny text-html
> send_hit deny http302
This works on 3.5.x and above only. store_* directives absent in 3.4.x
series.
>
>
> 2nd: use this patch:
>
>
>
> --- src/client_side_request.cc 2014-03-09 06:40:56.000000000 -0300
> +++ src/client_side_request.cc 2014-04-21 02:53:11.277155130 -0300
> @@ -545,6 +545,16 @@
> }
> debugs(85, 3, HERE << "validate IP " << clientConn->local << "
> non-match from Host: IP " << ia->in_addrs[i]);
> }
> +
> + if (true) {
> + unsigned short port = clientConn->local.port();
> + debugs(85, 3, HERE << "[anti-forgery] Host-non-matched remote
> IP (" << clientConn->local << ") was replaced with the first Host resolved
> IP (" << ia->in_addrs[0] << ":" << clientConn->local.port() << ")");
> + clientConn->local = ia->in_addrs[0];
> + clientConn->local.port(port);
> + http->request->flags.hostVerified = true;
> + http->doCallouts();
> + return;
> + }
> }
> debugs(85, 3, HERE << "FAIL: validate IP " << clientConn->local << "
> possible from Host:");
> hostHeaderVerifyFailed("local IP", "any domain IP");
>
>
> --- src/Server.cc
> +++ src/Server.cc
> @@ -31,6 +31,7 @@
> */
>
> #include "squid.h"
> +#include "acl/FilledChecklist.h"
> #include "acl/Gadgets.h"
> #include "base/TextException.h"
> #include "comm/Connection.h"
> @@ -174,6 +175,8 @@
> // give entry the reply because haveParsedReplyHeaders() expects it
> there
> entry->replaceHttpReply(theFinalReply, false); // but do not write yet
> haveParsedReplyHeaders(); // update the entry/reply (e.g., set
> timestamps)
> + if (EBIT_TEST(entry->flags, ENTRY_CACHABLE) && blockCaching())
> + entry->release();
> entry->startWriting(); // write the updated entry to store
>
> return theFinalReply;
> @@ -533,6 +536,24 @@
> currentOffset = partial ? theFinalReply->content_range->spec.offset :
> 0;
> }
>
> +/// whether to prevent caching of an otherwise cachable response
> +bool
> +ServerStateData::blockCaching()
> +{
> + if (const Acl::Tree *acl = Config.accessList.storeMiss) {
> + // This relatively expensive check is not in
> StoreEntry::checkCachable:
> + // That method lacks HttpRequest and may be called too many times.
> + ACLFilledChecklist ch(acl, originalRequest(), NULL);
> + ch.reply = const_cast<HttpReply*>(entry->getReply()); //
> ACLFilledChecklist API bug
> + HTTPMSGLOCK(ch.reply);
> + if (ch.fastCheck() != ACCESS_ALLOWED) { // when in doubt, block
> + debugs(20, 3, "store_miss prohibits caching");
> + return true;
> + }
> + }
> + return false;
> +}
> +
> HttpRequest *
> ServerStateData::originalRequest()
> {
> --- src/Server.h
> +++ src/Server.h
> @@ -131,6 +131,8 @@
> /// Entry-dependent callbacks use this check to quit if the entry went
> bad
> bool abortOnBadEntry(const char *abortReason);
>
> + bool blockCaching();
> +
> #if USE_ADAPTATION
> void startAdaptation(const Adaptation::ServiceGroupPointer &group,
> HttpRequest *cause);
> void adaptVirginReplyBody(const char *buf, ssize_t len);
> --- src/SquidConfig.h
> +++ src/SquidConfig.h
> @@ -375,6 +375,8 @@
> acl_access *AlwaysDirect;
> acl_access *ASlists;
> acl_access *noCache;
> + acl_access *sendHit;
> + acl_access *storeMiss;
> acl_access *stats_collection;
> #if SQUID_SNMP
>
> --- src/cf.data.pre
> +++ src/cf.data.pre
> @@ -4843,18 +4843,97 @@
> NAME: cache no_cache
> TYPE: acl_access
> DEFAULT: none
> -DEFAULT_DOC: Allow caching, unless rules exist in squid.conf.
> +DEFAULT_DOC: By default, this directive is unused and has no effect.
> LOC: Config.accessList.noCache
> DOC_START
> - A list of ACL elements which, if matched and denied, cause the request to
> - not be satisfied from the cache and the reply to not be cached.
> - In other words, use this to force certain objects to never be cached.
> -
> - You must use the words 'allow' or 'deny' to indicate whether items
> - matching the ACL should be allowed or denied into the cache.
> + Requests denied by this directive will not be served from the cache
> + and their responses will not be stored in the cache. This directive
> + has no effect on other transactions and on already cached responses.
>
> This clause supports both fast and slow acl types.
> See http://wiki.squid-cache.org/SquidFaq/SquidAcl for details.
> +
> + This and the two other similar caching directives listed below are
> + checked at different transaction processing stages, have different
> + access to response information, affect different cache operations,
> + and differ in slow ACLs support:
> +
> + * cache: Checked before Squid makes a hit/miss determination.
> + No access to reply information!
> + Denies both serving a hit and storing a miss.
> + Supports both fast and slow ACLs.
> + * send_hit: Checked after a hit was detected.
> + Has access to reply (hit) information.
> + Denies serving a hit only.
> + Supports fast ACLs only.
> + * store_miss: Checked before storing a cachable miss.
> + Has access to reply (miss) information.
> + Denies storing a miss only.
> + Supports fast ACLs only.
> +
> + If you are not sure which of the three directives to use, apply the
> + following decision logic:
> +
> + * If your ACL(s) are of slow type _and_ need response info, redesign.
> + Squid does not support that particular combination at this time.
> + Otherwise:
> + * If your directive ACL(s) are of slow type, use "cache"; and/or
> + * if your directive ACL(s) need no response info, use "cache".
> + Otherwise:
> + * If you do not want the response cached, use store_miss; and/or
> + * if you do not want a hit on a cached response, use send_hit.
> +DOC_END
> +
> +NAME: send_hit
> +TYPE: acl_access
> +DEFAULT: none
> +DEFAULT_DOC: By default, this directive is unused and has no effect.
> +LOC: Config.accessList.sendHit
> +DOC_START
> + Responses denied by this directive will not be served from the cache
> + (but may still be cached, see store_miss). This directive has no
> + effect on the responses it allows and on the cached objects.
> +
> + Please see the "cache" directive for a summary of differences among
> + store_miss, send_hit, and cache directives.
> +
> + Unlike the "cache" directive, send_hit only supports fast acl
> + types. See http://wiki.squid-cache.org/SquidFaq/SquidAcl for details.
> +
> + For example:
> +
> + # apply custom Store ID mapping to some URLs
> + acl MapMe dstdomain .c.example.com
> + store_id_program ...
> + store_id_access allow MapMe
> +
> + # but prevent caching of special responses
> + # such as 302 redirects that cause StoreID loops
> + acl Ordinary http_status 200-299
> + store_miss deny MapMe !Ordinary
> +
> + # and do not serve any previously stored special responses
> + # from the cache (in case they were already cached before
> + # the above store_miss rule was in effect).
> + send_hit deny MapMe !Ordinary
> +DOC_END
> +
> +NAME: store_miss
> +TYPE: acl_access
> +DEFAULT: none
> +DEFAULT_DOC: By default, this directive is unused and has no effect.
> +LOC: Config.accessList.storeMiss
> +DOC_START
> + Responses denied by this directive will not be cached (but may still
> + be served from the cache, see send_hit). This directive has no
> + effect on the responses it allows and on the already cached responses.
> +
> + Please see the "cache" directive for a summary of differences among
> + store_miss, send_hit, and cache directives. See the
> + send_hit directive for a usage example.
> +
> + Unlike the "cache" directive, store_miss only supports fast acl
> + types. See http://wiki.squid-cache.org/SquidFaq/SquidAcl for details.
> DOC_END
>
> NAME: max_stale
> --- src/client_side_reply.cc
> +++ src/client_side_reply.cc
> @@ -545,6 +545,11 @@
> ) {
> http->logType = LOG_TCP_NEGATIVE_HIT;
> sendMoreData(result);
> + } else if (blockedHit()) {
> + debugs(88, 5, "send_hit forces a MISS");
> + http->logType = LOG_TCP_MISS;
> + processMiss();
> + return;
> } else if (!http->flags.internal && refreshCheckHTTP(e, r)) {
> debugs(88, 5, "clientCacheHit: in refreshCheck() block");
> /*
> @@ -773,6 +778,30 @@
> }
> }
>
> +/// whether squid.conf send_hit prevents us from serving this hit
> +bool
> +clientReplyContext::blockedHit() const
> +{
> + if (!Config.accessList.sendHit)
> + return false; // hits are not blocked by default
> +
> + if (http->flags.internal)
> + return false; // internal content "hits" cannot be blocked
> +
> + if (const HttpReply *rep = http->storeEntry()->getReply()) {
> + std::auto_ptr<ACLFilledChecklist>
> chl(clientAclChecklistCreate(Config.accessList.sendHit, http));
> + chl->reply = const_cast<HttpReply*>(rep); // ACLChecklist API bug
> + HTTPMSGLOCK(chl->reply);
> + return chl->fastCheck() != ACCESS_ALLOWED; // when in doubt, block
> + }
> +
> + // This does not happen, I hope, because we are called from CacheHit,
> which
> + // is called via a storeClientCopy() callback, and store should
> initialize
> + // the reply before calling that callback.
> + debugs(88, 3, "Missing reply!");
> + return false;
> +}
> +
> void
> clientReplyContext::purgeRequestFindObjectToPurge()
> {
> --- src/client_side_reply.h
> +++ src/client_side_reply.h
> @@ -140,6 +140,7 @@
> void triggerInitialStoreRead();
> void sendClientOldEntry();
> void purgeAllCached();
> + bool blockedHit() const;
>
> void sendBodyTooLargeError();
> void sendPreconditionFailedError();
This is also not solution. One of most biggest traffic source must have
native solution in proxy. Not a crutch.
>
>
>
> --
> View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Youtube-redirection-loop-tp4671084p4671103.html
> Sent from the Squid - Users mailing list archive at Nabble.com.
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
Again. I want simple thing: to limit selected mime-types caching from
selected domains. Point.
On my opinion, this will completely solve YT caching problem without any
crutches/patches.
More information about the squid-users
mailing list