[squid-dev] [PATCH] HTTP Parser upgrade
Amos Jeffries
squid3 at treenet.co.nz
Thu Oct 16 13:27:52 UTC 2014
(if there are no objections this will be applied in a couple of days)
This update is to lay the groundwork for several planned future projects
implementing protocol specific parsers, seperating the currently
intertwinned client connection management and HTTP protocol parsing
logics, and for zero-copy processing of transactions using SBuf.
This patch renames the HttpParser class as RequestParser and moves it
into the Http::One:: namespace as child of an Http::Parser class with
generic API accessors for shared use by other HTTP message parsers.
The class API
- is updated to process both the request-line and HTTP mime headers,
returning an incomplete parse result until the entire headers parts of
the message have been received.
- now contains accessor methods for retrieving the method, URI,
protocol, mime headers block (as an SBuf) and some metrics about those.
- the old request_offsets structure and similar offset details are no
longer exposed.
- now emits 414 and 431 HTTP status codes as appropriate.
The parser is made partially incremental and some use of Tokenizer is
added for faster parsing. There is more that can be done to further
speed up parsing of slow or very large requests.
Much of the code from client_side.cc parseHttpRequest() and also the
header-field code from mime_headers.cc has been moved into the parser
class. The client_side.cc code now simply runs the main
Http1::RequestParser::parse() method then uses accessors to retrieve
and process the parse results.
A unit test for incremental parsing has been added to testHttpParser.
Also, the HttpRequestMethod class is moved into the Http:: namespace and
library to reduce dependencies on the parser class outside the library.
The gains made so far from incremental parse, reducing parser passes and
zero-copy SBuf are offset by several temporary performance regressions
added converting SBuf for legacy code use. These are marked for later
removal.
Amos
On 15/10/2014 10:34 p.m., Amos Jeffries wrote:
> On 15/10/2014 10:19 a.m., Alex Rousskov wrote:
>> On 10/11/2014 02:17 PM, Amos Jeffries wrote:
>>> On 7/06/2014 4:55 a.m., Amos Jeffries wrote:
>>>> Latest version of the Parser-NG project branch. Since last
>>>> message I have dropped the copy-constructor related pieces.
>>>
>>>> Polygraph testing on the branch showed almost no performance
>>>> impact (+0.04ms/req, -0.03% request rate). Nevertheless a half
>>>> dozen points of added memory reallocate or data copy are
>>>> marked for future removal as the Parser-NG and SBuf
>>>> improvements continue.
>
>> Just to double check: Is that really 0.04ms or 40ms? The former is
>> beyond Polygraph precision (i.e. it is the same as "zero"). The
>> latter is a significant overhead. I hope it is the former!
>
> Yes the former (insignificant).
>
> ParserNG (Oct 11th):
> Mean response number: 1925.64
> Mean response time: 103.36
>
> trunk (Oct 11th):
> Mean response number: 1926.03
> Mean response time: 103.34
>
> trunk (Oct )
> Mean response number: 1926.19
> Mean response time: 103.33
>
>
>>> + * \deprecated use SBuf constructor instead */
>>> -HttpRequestMethod::HttpRequestMethod(char const *begin, char
>>> const *end) : theMethod(Http::METHOD_NONE)
>>> +HttpRequestMethod::HttpRequestMethod(char const *begin) :
>>> theMethod(Http::METHOD_NONE)
>
>> Please rework this constructor into a HttpRequestMethodXXX()
>> function call. Otherwise, it is very difficult to spot places where
>> it is still being used, including in future patches. Since you are
>> also changing the constructor profile, it should be possible to do
>> that replacement directly in your patch, avoiding re-editing many
>> files by hand.
>
>> This is especially important if this old HttpRequestMethod(char*)
>> has a somewhat different semantics/compliance than the new
>> HttpRequestMethod(SBuf).
>
>
> Done. This is not quite as simple as suggested, the renaming causes it
> to become a method instead of constructor, so code needs adjusting to
> allocate HttpRequestMethod objects separately before calling it.
>
>
>
>>> - method = HttpRequestMethod(start, t); + SBuf m(start,
>>> start-t); // XXX: SBuf ctor allocates and data-copies.
>>> performance regression. + method = HttpRequestMethod(m);
>
>
>> What is the purpose of introducing this SBuf performance
>> regression?
>
>
> To avoid repeated strcspn(X, w_space) over the input buffer.
>
>> * If the deprecated HttpRequestMethod constructor works fine, use
>> that and avoid this regression while waiting for callers to pass in
>> an SBuf.
>
>> * If the deprecated HttpRequestMethod constructor is not as good
>> as the new HttpRequestMethod constructor for some reason (other
>> than SBuf parameter itself), then should we re-implement the
>> deprecated HttpRequestMethod constructor to create an SBuf and then
>> call the new HttpRequestMethod constructor with that SBuf instead?
>
>
> It works fine. Done.
>
>
>>> + * or from a range of chars such as "GET" from "GETFOOBARBAZ" +
>>> * + * Assumes the s parameter contains only the method string
>
>> This description is not clear, especially because GETFOOBARBAZ
>> does not "contain only the method string" GET. The
>> HttpRequestMethod() code does not seem to match the documented
>> "GETFOOBARBAZ" example because it uses caseCmp(s) without any
>> prefix size limits.
>
>
> s is expected to contain an SBuf whose contents is *only* the chars
> forming the method (buf="GET", length=3) not the "FOOBARBAZ" chars
> which follow in the I/O buffer.
>
> This is assumed/needs to be true whether or not the chars which
> preceed or follow are the expected whitespace delimiters or some
> "FOOBARBAZ" fluff.
>
> How else would you write that?
>
>
>
>>> + // XXX: still check for missing method name?
>
>> Please rephrase. It is not clear what you are trying to ask here.
>> FWIW, the new code does handle the case of an empty method name as
>> well as a method name not matching any known header.
>
>
> That was it. Dropped.
>
>
>>> const char *t = start + strcspn(start, w_space);
>> ...
>>> - start = t + strspn(t, w_space); + start = t + strspn(t,
>>> w_space); // XXX: breaks if whitespace exists in URL
>
>> Wrong XXX? The code appears to skip the space found at t
>> declaration time. That is, after the declaration, "t" should point
>> to either a space character or, if there was not one in a URL, to a
>> terminating zero character. The XXX seems to be wrong in either
>> case.
>
> Er yes. mistakes strspn() for strcspn(). corrected.
>
>
>>> + SBuf url = hp->requestUri(); // use full provided URI if we
>>> abort + do { // use a loop so we can break out of it +
>>> ::Parser::Tokenizer tok(url); + if (tok.remaining()[0] ==
>>> '/') + break;
>
>> Are we guaranteed to have a non-empty URL at this point? If not, it
>> is unsafe to dereference its first character.
>
> Right. Using skip('/') instead with current Tokenizer guarantees.
>
>
>>> + // reject URI which are not well-formed even after the
>>> processing above + if (url[0] != '/') {
>
>> Similar concern here.
>
>
> Thats an old bug in dead code, but adding isEmpty() check just for
> completeness when someone gets around to makeing it non-dead.
>
>
>>> - char ipbuf[MAX_IPSTRLEN]; + static char
>>> ipbuf[MAX_IPSTRLEN];
>
>> If you are changing this, please move the declaration down, where
>> it is actually needed.
>
>
> Done.
>
>
>>> + static const CharacterSet authority =
>>> CharacterSet("authority","-._~%:@[]!$&'()*+,;=") + +
>>> CharacterSet::HEXDIG + CharacterSet::ALPHA +
>>> CharacterSet::DIGIT;
>
>> Syntax error (missing "+" before CharacterSet::HEXDIG)? If yes,
>> does that imply that the above code has not been tested?
>
> It is on the end of the preceeding line.
>
>
>>> + const char *url = SBuf(hp->requestUri()).c_str();
>
>> Semantically, this returns a pointer inside the now-gone object
>> (the temporary anonymous SBuf). Do not do that even if it usually
>> "works" with the current SBuf implementation.
>
> Okay, using a pair of local-scope variables instead.
>
>
>>> - const bool isFtp = !hp; + const bool isFtp = (hp ==
>>> NULL);
>
>> Why this change? The existing code was correct and arguably more
>> "future proof" if we start using some kind of smart pointers for
>> "hp".
>
>
> Just consistency between X==NULL and X!=NULL checks.
>
> hp is already a smart pointer in the branch and being non-C++11 it
> does not provide operator true(), just operator !(). So we end up with
> a mix of if(!X) and if(X != NULL).
>
> Reverting since it was the other way to start with.
>
>
>
>>> + const HttpRequestMethod &method = !isFtp ? hp->method() :
>>> Http::METHOD_NONE; // XXX: or should this be GET ?
>
>> The answer is in the caller you modified, I think:
>
>>> - clientProcessRequest(this, NULL /*parser*/,
>>> context.getRaw(), - request->method,
>>> request->http_ver); + clientProcessRequest(this,
>>> Http1::RequestParserPointer(), context.getRaw());
>
>
>> So it should be something like http->request->method or
>> equivalent. For FTP, we already have a parsed request at this
>> point... However, please double check that
>
>> a) you are actually using that new "method" variable -- it is
>> difficult to say by looking from the patch alone, but I see many
>> cases where the old "method" is getting replaced with
>> "hp->method()". Perhaps all of the uses are gone now?
>
>> b) the new "method" declaration is as close to the uses in (a) as
>> possible.
>
>
> Confirmed. It looks like the HTTP-only paths should all be using
> hp->method(), the shared and FTP-only paths using request->method.
>
> Removing method local.
>
>
>>> Http::ProtocolVersion http_ver; - if (ClientSocketContext
>>> *context = parseOneRequest(http_ver)) { + if
>>> (ClientSocketContext *context = parseOneRequest()) {
>
>> Do you still need the http_ver variable after removing the
>> corresponding parameter?
>
>
> No. Dropped.
>
>
>> * clientProcessRequest() no longer consumes parsed input, right?
>> Where does that consumption happen now?
>
>
> Inside parseOneRequest().
>
> * HttpServer syncs the I/O buffer to the Tokenizer in
> parseHttpRequest() after each parse attempt.
>
> * FtpServer uses byte counting by the Tokenizer to manage them
> separately with consumeInput().
>
>
>>> + * BUG: limits output to just 1KB when Squid accepts up to
>>> 64KB line length. * - * \retval -1 an error occurred.
>>> request_parse_status indicates HTTP status result. - *
>>> \retval 1 successful parse. member fields contain the
>>> request-line items - * \retval 0 more data is needed to
>>> complete the parse + * \return A pointer to a field-value of
>>> the first matching field-name, or NULL.
>
>> In what sense does the Parser limit the output? It is documented
>> to return a pointer to the field-value. Are we truncating that
>> field-value?
>
>
> Copying it into a static array of size 1KB.
>
> ...
> // arbitrary maximum-length for headers which can be found by
> Http1Parser::getHeaderField()
> #define GET_HDR_SZ 1024
>
> ...
> LOCAL_ARRAY(char, header, GET_HDR_SZ);
>
>
> For this patching I have left that old limit in place. It appears to
> be only used for a few things like Host: header value fetching in the
> prepare*URL() functions, so not a problem for most users.
>
>
>
>> * Http1::Parser class has virtual methods but is missing a virtual
>> destructor.
>
>
> Gah. C++11'ism. Default d'tor can be assumed virtual and the children
> provide the virtuals.
>
>
>>> + /// raw copy of the origina client reqeust-line URI field
>
>> "origina" typo
>
>
> Fixed.
>
>
>>> + Parser() : parsingStage_(HTTP_PARSE_NONE) {}
>
>>> + RequestParser() : Parser() {clear();} + virtual void
>>> clear();
>
>
>>> +Http::One::Parser::clear() { - clear(); // empty the state.
>>> - state = HTTP_PARSE_NEW; - buf = aBuf; - bufsiz = len; -
>>> debugs(74, 5, HERE << "Request buffer is " << buf); +
>>> parsingStage_ = HTTP_PARSE_NONE; + buf_ = NULL; + msgProtocol_
>>> = AnyP::ProtocolVersion(); + mimeHeaderBlock_.clear(); }
>
>
>>> +void +Http::One::RequestParser::clear() +{ +
>>> Http1::Parser::clear(); + + request_parse_status =
>>> Http::scNone; + req.start = req.end = -1; + req.m_start =
>>> req.m_end = -1; + req.u_start = req.u_end = -1; +
>>> req.v_start = req.v_end = -1; + method_ = HttpRequestMethod();
>>> +}
>
>
>> The above is OK. If you want to polish it further to avoid calling
>> virtual methods from constructors (which is bad style), make these
>> changes:
>
>> 1. Initialize all members in each constructor as needed, without
>> calling clear().
>
>> 2. Make Parser::clear() pure, without a body.
>
>> 3. In RequestParser::clear(), just do:
>
>> *this = RequestParser();
>
>
> I'm never sure during this sequence if complex types like SBuf need an
> explicit call to their constructor at step (1) to free old
> smart-pointer values or not.
>
> Trying it this way.
>
>
>> For the proposed commit message, please add an brief explanation
>> for "why we are doing this", in addition to the already provided
>> "what we are doing" summary.
>
>> And please make sure the commit message explicitly says that we
>> are adding a few potential performance regressions with this
>> change.
>
> Is that a +1 considering these are mostly cosmetic?
>
> Amos
>
> _______________________________________________
> squid-dev mailing list
> squid-dev at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-dev
>
-------------- next part --------------
=== modified file 'configure.ac'
--- configure.ac 2014-10-13 14:40:35 +0000
+++ configure.ac 2014-10-16 12:13:18 +0000
@@ -3766,40 +3766,41 @@
src/base/Makefile
src/acl/Makefile
src/clients/Makefile
src/servers/Makefile
src/fs/Makefile
src/repl/Makefile
src/auth/Makefile
src/auth/basic/Makefile
src/auth/digest/Makefile
src/auth/negotiate/Makefile
src/auth/ntlm/Makefile
src/adaptation/Makefile
src/adaptation/icap/Makefile
src/adaptation/ecap/Makefile
src/comm/Makefile
src/esi/Makefile
src/eui/Makefile
src/format/Makefile
src/helper/Makefile
src/http/Makefile
+ src/http/one/Makefile
src/icmp/Makefile
src/ident/Makefile
src/ip/Makefile
src/log/Makefile
src/ipc/Makefile
src/ssl/Makefile
src/mgr/Makefile
src/parser/Makefile
src/snmp/Makefile
contrib/Makefile
icons/Makefile
errors/Makefile
test-suite/Makefile
doc/Makefile
doc/manuals/Makefile
helpers/Makefile
helpers/basic_auth/Makefile
helpers/basic_auth/DB/Makefile
helpers/basic_auth/fake/Makefile
helpers/basic_auth/getpwnam/Makefile
=== modified file 'src/AccessLogEntry.h'
--- src/AccessLogEntry.h 2014-09-29 05:13:17 +0000
+++ src/AccessLogEntry.h 2014-10-05 10:43:33 +0000
@@ -1,38 +1,38 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
#ifndef SQUID_HTTPACCESSLOGENTRY_H
#define SQUID_HTTPACCESSLOGENTRY_H
#include "anyp/PortCfg.h"
#include "base/RefCount.h"
#include "comm/Connection.h"
#include "HierarchyLogEntry.h"
#include "http/ProtocolVersion.h"
#include "HttpHeader.h"
-#include "HttpRequestMethod.h"
+#include "http/RequestMethod.h"
#include "icp_opcode.h"
#include "ip/Address.h"
#include "LogTags.h"
#include "MessageSizes.h"
#include "Notes.h"
#if ICAP_CLIENT
#include "adaptation/icap/Elements.h"
#endif
#if USE_OPENSSL
#include "ssl/gadgets.h"
#endif
/* forward decls */
class HttpReply;
class HttpRequest;
class CustomLog;
class AccessLogEntry: public RefCountable
{
=== modified file 'src/HttpHeader.cc'
--- src/HttpHeader.cc 2014-09-13 13:59:43 +0000
+++ src/HttpHeader.cc 2014-09-14 12:23:03 +0000
@@ -564,188 +564,189 @@
/* deny bad guys (ok to check for HDR_OTHER) here */
if (denied_mask && CBIT_TEST(*denied_mask, e->id))
continue;
debugs(55, 7, "Updating header '" << HeadersAttrs[e->id].name << "' in cached entry");
addEntry(e->clone());
}
}
/* just handy in parsing: resets and returns false */
int
HttpHeader::reset()
{
clean();
return 0;
}
int
-HttpHeader::parse(const char *header_start, const char *header_end)
+HttpHeader::parse(const char *header_start, size_t hdrLen)
{
const char *field_ptr = header_start;
+ const char *header_end = header_start + hdrLen; // XXX: remove
HttpHeaderEntry *e, *e2;
int warnOnError = (Config.onoff.relaxed_header_parser <= 0 ? DBG_IMPORTANT : 2);
PROF_start(HttpHeaderParse);
assert(header_start && header_end);
- debugs(55, 7, "parsing hdr: (" << this << ")" << std::endl << getStringPrefix(header_start, header_end));
+ debugs(55, 7, "parsing hdr: (" << this << ")" << std::endl << getStringPrefix(header_start, hdrLen));
++ HttpHeaderStats[owner].parsedCount;
char *nulpos;
- if ((nulpos = (char*)memchr(header_start, '\0', header_end - header_start))) {
+ if ((nulpos = (char*)memchr(header_start, '\0', hdrLen))) {
debugs(55, DBG_IMPORTANT, "WARNING: HTTP header contains NULL characters {" <<
- getStringPrefix(header_start, nulpos) << "}\nNULL\n{" << getStringPrefix(nulpos+1, header_end));
+ getStringPrefix(header_start, nulpos-header_start) << "}\nNULL\n{" << getStringPrefix(nulpos+1, hdrLen-(nulpos-header_start)-1));
PROF_stop(HttpHeaderParse);
return reset();
}
/* common format headers are "<name>:[ws]<value>" lines delimited by <CRLF>.
* continuation lines start with a (single) space or tab */
while (field_ptr < header_end) {
const char *field_start = field_ptr;
const char *field_end;
do {
const char *this_line = field_ptr;
field_ptr = (const char *)memchr(field_ptr, '\n', header_end - field_ptr);
if (!field_ptr) {
// missing <LF>
PROF_stop(HttpHeaderParse);
return reset();
}
field_end = field_ptr;
++field_ptr; /* Move to next line */
if (field_end > this_line && field_end[-1] == '\r') {
--field_end; /* Ignore CR LF */
if (owner == hoRequest && field_end > this_line) {
bool cr_only = true;
for (const char *p = this_line; p < field_end && cr_only; ++p) {
if (*p != '\r')
cr_only = false;
}
if (cr_only) {
debugs(55, DBG_IMPORTANT, "SECURITY WARNING: Rejecting HTTP request with a CR+ "
"header field to prevent request smuggling attacks: {" <<
- getStringPrefix(header_start, header_end) << "}");
+ getStringPrefix(header_start, hdrLen) << "}");
PROF_stop(HttpHeaderParse);
return reset();
}
}
}
/* Barf on stray CR characters */
if (memchr(this_line, '\r', field_end - this_line)) {
debugs(55, warnOnError, "WARNING: suspicious CR characters in HTTP header {" <<
- getStringPrefix(field_start, field_end) << "}");
+ getStringPrefix(field_start, field_end-field_start) << "}");
if (Config.onoff.relaxed_header_parser) {
char *p = (char *) this_line; /* XXX Warning! This destroys original header content and violates specifications somewhat */
while ((p = (char *)memchr(p, '\r', field_end - p)) != NULL) {
*p = ' ';
++p;
}
} else {
PROF_stop(HttpHeaderParse);
return reset();
}
}
if (this_line + 1 == field_end && this_line > field_start) {
debugs(55, warnOnError, "WARNING: Blank continuation line in HTTP header {" <<
- getStringPrefix(header_start, header_end) << "}");
+ getStringPrefix(header_start, hdrLen) << "}");
PROF_stop(HttpHeaderParse);
return reset();
}
} while (field_ptr < header_end && (*field_ptr == ' ' || *field_ptr == '\t'));
if (field_start == field_end) {
if (field_ptr < header_end) {
debugs(55, warnOnError, "WARNING: unparseable HTTP header field near {" <<
- getStringPrefix(field_start, header_end) << "}");
+ getStringPrefix(field_start, hdrLen-(field_start-header_start)) << "}");
PROF_stop(HttpHeaderParse);
return reset();
}
break; /* terminating blank line */
}
if ((e = HttpHeaderEntry::parse(field_start, field_end)) == NULL) {
debugs(55, warnOnError, "WARNING: unparseable HTTP header field {" <<
- getStringPrefix(field_start, field_end) << "}");
- debugs(55, warnOnError, " in {" << getStringPrefix(header_start, header_end) << "}");
+ getStringPrefix(field_start, field_end-field_start) << "}");
+ debugs(55, warnOnError, " in {" << getStringPrefix(header_start, hdrLen) << "}");
if (Config.onoff.relaxed_header_parser)
continue;
PROF_stop(HttpHeaderParse);
return reset();
}
if (e->id == HDR_CONTENT_LENGTH && (e2 = findEntry(e->id)) != NULL) {
if (e->value != e2->value) {
int64_t l1, l2;
debugs(55, warnOnError, "WARNING: found two conflicting content-length headers in {" <<
- getStringPrefix(header_start, header_end) << "}");
+ getStringPrefix(header_start, hdrLen) << "}");
if (!Config.onoff.relaxed_header_parser) {
delete e;
PROF_stop(HttpHeaderParse);
return reset();
}
if (!httpHeaderParseOffset(e->value.termedBuf(), &l1)) {
debugs(55, DBG_IMPORTANT, "WARNING: Unparseable content-length '" << e->value << "'");
delete e;
continue;
} else if (!httpHeaderParseOffset(e2->value.termedBuf(), &l2)) {
debugs(55, DBG_IMPORTANT, "WARNING: Unparseable content-length '" << e2->value << "'");
delById(e2->id);
} else if (l1 > l2) {
delById(e2->id);
} else {
delete e;
continue;
}
} else {
debugs(55, warnOnError, "NOTICE: found double content-length header");
delete e;
if (Config.onoff.relaxed_header_parser)
continue;
PROF_stop(HttpHeaderParse);
return reset();
}
}
if (e->id == HDR_OTHER && stringHasWhitespace(e->name.termedBuf())) {
debugs(55, warnOnError, "WARNING: found whitespace in HTTP header name {" <<
- getStringPrefix(field_start, field_end) << "}");
+ getStringPrefix(field_start, field_end-field_start) << "}");
if (!Config.onoff.relaxed_header_parser) {
delete e;
PROF_stop(HttpHeaderParse);
return reset();
}
}
addEntry(e);
}
if (chunked()) {
// RFC 2616 section 4.4: ignore Content-Length with Transfer-Encoding
delById(HDR_CONTENT_LENGTH);
}
PROF_stop(HttpHeaderParse);
return 1; /* even if no fields where found, it is a valid header */
}
@@ -1614,52 +1615,52 @@
const char *name_end = (const char *)memchr(field_start, ':', field_end - field_start);
int name_len = name_end ? name_end - field_start :0;
const char *value_start = field_start + name_len + 1; /* skip ':' */
/* note: value_end == field_end */
++ HeaderEntryParsedCount;
/* do we have a valid field name within this field? */
if (!name_len || name_end > field_end)
return NULL;
if (name_len > 65534) {
/* String must be LESS THAN 64K and it adds a terminating NULL */
debugs(55, DBG_IMPORTANT, "WARNING: ignoring header name of " << name_len << " bytes");
return NULL;
}
if (Config.onoff.relaxed_header_parser && xisspace(field_start[name_len - 1])) {
debugs(55, Config.onoff.relaxed_header_parser <= 0 ? 1 : 2,
- "NOTICE: Whitespace after header name in '" << getStringPrefix(field_start, field_end) << "'");
+ "NOTICE: Whitespace after header name in '" << getStringPrefix(field_start, field_end-field_start) << "'");
while (name_len > 0 && xisspace(field_start[name_len - 1]))
--name_len;
if (!name_len)
return NULL;
}
/* now we know we can parse it */
- debugs(55, 9, "parsing HttpHeaderEntry: near '" << getStringPrefix(field_start, field_end) << "'");
+ debugs(55, 9, "parsing HttpHeaderEntry: near '" << getStringPrefix(field_start, field_end-field_start) << "'");
/* is it a "known" field? */
http_hdr_type id = httpHeaderIdByName(field_start, name_len, Headers, HDR_ENUM_END);
String name;
String value;
if (id < 0)
id = HDR_OTHER;
assert_eid(id);
/* set field name */
if (id == HDR_OTHER)
name.limitInit(field_start, name_len);
else
name = Headers[id].name;
/* trim field value */
=== modified file 'src/HttpHeader.h'
--- src/HttpHeader.h 2014-09-13 13:59:43 +0000
+++ src/HttpHeader.h 2014-09-14 12:23:03 +0000
@@ -1,158 +1,48 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
#ifndef SQUID_HTTPHEADER_H
#define SQUID_HTTPHEADER_H
+#include "http/RegisteredHeaders.h"
/* because we pass a spec by value */
#include "HttpHeaderMask.h"
#include "MemPool.h"
#include "SquidString.h"
#include <vector>
/* class forward declarations */
class HttpHdrCc;
class HttpHdrContRange;
class HttpHdrRange;
class HttpHdrSc;
class Packer;
class StoreEntry;
class SBuf;
-/* constant attributes of http header fields */
-
-/// recognized or "known" header fields; and the RFC which defines them (or not)
-/// http://www.iana.org/assignments/message-headers/message-headers.xhtml
-typedef enum {
- HDR_BAD_HDR = -1,
- HDR_ACCEPT = 0, /**< RFC 7231 */
- HDR_ACCEPT_CHARSET, /**< RFC 7231 */
- HDR_ACCEPT_ENCODING, /**< RFC 7231 */
- /*HDR_ACCEPT_FEATURES,*/ /* RFC 2295 */
- HDR_ACCEPT_LANGUAGE, /**< RFC 7231 */
- HDR_ACCEPT_RANGES, /**< RFC 7233 */
- HDR_AGE, /**< RFC 7234 */
- HDR_ALLOW, /**< RFC 7231 */
- HDR_AUTHENTICATION_INFO, /**< RFC 2617 */
- HDR_AUTHORIZATION, /**< RFC 7235, 4559 */
- HDR_CACHE_CONTROL, /**< RFC 7234 */
- HDR_CONNECTION, /**< RFC 7230 */
- HDR_CONTENT_BASE, /**< obsoleted RFC 2068 */
- HDR_CONTENT_DISPOSITION, /**< RFC 2183, 6266 */
- HDR_CONTENT_ENCODING, /**< RFC 7231 */
- HDR_CONTENT_LANGUAGE, /**< RFC 7231 */
- HDR_CONTENT_LENGTH, /**< RFC 7230 */
- HDR_CONTENT_LOCATION, /**< RFC 7231 */
- HDR_CONTENT_MD5, /**< deprecated, RFC 2616 */
- HDR_CONTENT_RANGE, /**< RFC 7233 */
- HDR_CONTENT_TYPE, /**< RFC 7231 */
- HDR_COOKIE, /**< RFC 6265 header we may need to erase */
- HDR_COOKIE2, /**< obsolete RFC 2965 header we may need to erase */
- HDR_DATE, /**< RFC 7231 */
- /*HDR_DAV,*/ /* RFC 2518 */
- /*HDR_DEPTH,*/ /* RFC 2518 */
- /*HDR_DERIVED_FROM,*/ /* deprecated RFC 2068 */
- /*HDR_DESTINATION,*/ /* RFC 2518 */
- HDR_ETAG, /**< RFC 7232 */
- HDR_EXPECT, /**< RFC 7231 */
- HDR_EXPIRES, /**< RFC 7234 */
- HDR_FORWARDED, /**< RFC 7239 */
- HDR_FROM, /**< RFC 7231 */
- HDR_HOST, /**< RFC 7230 */
- HDR_HTTP2_SETTINGS, /**< HTTP/2.0 upgrade header. see draft-ietf-httpbis-http2-13 */
- /*HDR_IF,*/ /* RFC 2518 */
- HDR_IF_MATCH, /**< RFC 7232 */
- HDR_IF_MODIFIED_SINCE, /**< RFC 7232 */
- HDR_IF_NONE_MATCH, /**< RFC 7232 */
- HDR_IF_RANGE, /**< RFC 7233 */
- HDR_IF_UNMODIFIED_SINCE, /**< RFC 7232 */
- HDR_KEEP_ALIVE, /**< obsoleted RFC 2068 header we may need to erase */
- HDR_KEY, /**< experimental RFC Draft draft-fielding-http-key-02 */
- HDR_LAST_MODIFIED, /**< RFC 7232 */
- HDR_LINK, /**< RFC 5988 */
- HDR_LOCATION, /**< RFC 7231 */
- /*HDR_LOCK_TOKEN,*/ /* RFC 2518 */
- HDR_MAX_FORWARDS, /**< RFC 7231 */
- HDR_MIME_VERSION, /**< RFC 2045, 7231 */
- HDR_NEGOTIATE, /**< experimental RFC 2295. Why only this one from 2295? */
- /*HDR_OVERWRITE,*/ /* RFC 2518 */
- HDR_ORIGIN, /* CORS Draft specification (see http://www.w3.org/TR/cors/) */
- HDR_PRAGMA, /**< RFC 7234 */
- HDR_PROXY_AUTHENTICATE, /**< RFC 7235 */
- HDR_PROXY_AUTHENTICATION_INFO, /**< RFC 2617 */
- HDR_PROXY_AUTHORIZATION, /**< RFC 7235 */
- HDR_PROXY_CONNECTION, /**< obsolete Netscape header we may need to erase. */
- HDR_PROXY_SUPPORT, /**< RFC 4559 */
- HDR_PUBLIC, /**< RFC 2068 */
- HDR_RANGE, /**< RFC 7233 */
- HDR_REFERER, /**< RFC 7231 */
- HDR_REQUEST_RANGE, /**< some clients use this, sigh */
- HDR_RETRY_AFTER, /**< RFC 7231 */
- HDR_SERVER, /**< RFC 7231 */
- HDR_SET_COOKIE, /**< RFC 6265 header we may need to erase */
- HDR_SET_COOKIE2, /**< obsoleted RFC 2965 header we may need to erase */
- /*HDR_STATUS_URI,*/ /* RFC 2518 */
- /*HDR_TCN,*/ /* experimental RFC 2295 */
- HDR_TE, /**< RFC 7230 */
- /*HDR_TIMEOUT,*/ /* RFC 2518 */
- HDR_TITLE, /* obsolete draft suggested header */
- HDR_TRAILER, /**< RFC 7230 */
- HDR_TRANSFER_ENCODING, /**< RFC 7230 */
- HDR_TRANSLATE, /**< IIS custom header we may need to erase */
- HDR_UNLESS_MODIFIED_SINCE, /**< IIS custom header we may need to erase */
- HDR_UPGRADE, /**< RFC 7230 */
- HDR_USER_AGENT, /**< RFC 7231 */
- /*HDR_VARIANT_VARY,*/ /* experimental RFC 2295 */
- HDR_VARY, /**< RFC 7231 */
- HDR_VIA, /**< RFC 7230 */
- HDR_WARNING, /**< RFC 7234 */
- HDR_WWW_AUTHENTICATE, /**< RFC 7235, 4559 */
- HDR_X_CACHE, /**< Squid custom header */
- HDR_X_CACHE_LOOKUP, /**< Squid custom header. temporary hack that became de-facto. TODO remove */
- HDR_X_FORWARDED_FOR, /**< obsolete Squid custom header, RFC 7239 */
- HDR_X_REQUEST_URI, /**< Squid custom header appended if ADD_X_REQUEST_URI is defined */
- HDR_X_SQUID_ERROR, /**< Squid custom header on generated error responses */
-#if X_ACCELERATOR_VARY
- HDR_X_ACCELERATOR_VARY, /**< obsolete Squid custom header. */
-#endif
-#if USE_ADAPTATION
- HDR_X_NEXT_SERVICES, /**< Squid custom ICAP header */
-#endif
- HDR_SURROGATE_CAPABILITY, /**< Edge Side Includes (ESI) header */
- HDR_SURROGATE_CONTROL, /**< Edge Side Includes (ESI) header */
- HDR_FRONT_END_HTTPS, /**< MS Exchange custom header we may have to add */
- HDR_FTP_COMMAND, /**< Internal header for FTP command */
- HDR_FTP_ARGUMENTS, /**< Internal header for FTP command arguments */
- HDR_FTP_PRE, /**< Internal header containing leading FTP control response lines */
- HDR_FTP_STATUS, /**< Internal header for FTP reply status */
- HDR_FTP_REASON, /**< Internal header for FTP reply reason */
- HDR_OTHER, /**< internal tag value for "unknown" headers */
- HDR_ENUM_END
-} http_hdr_type;
-
/** possible types for http header fields */
typedef enum {
ftInvalid = HDR_ENUM_END, /**< to catch nasty errors with hdr_id<->fld_type clashes */
ftInt,
ftInt64,
ftStr,
ftDate_1123,
ftETag,
ftPCc,
ftPContRange,
ftPRange,
ftPSc,
ftDate_1123_or_ETag
} field_type;
/** Possible owners of http header */
typedef enum {
hoNone =0,
#if USE_HTCP
hoHtcpReply,
@@ -202,41 +92,41 @@
class ETag;
class TimeOrTag;
class HttpHeader
{
public:
HttpHeader();
explicit HttpHeader(const http_hdr_owner_type owner);
HttpHeader(const HttpHeader &other);
~HttpHeader();
HttpHeader &operator =(const HttpHeader &other);
/* Interface functions */
void clean();
void append(const HttpHeader * src);
void update (HttpHeader const *fresh, HttpHeaderMask const *denied_mask);
void compact();
int reset();
- int parse(const char *header_start, const char *header_end);
+ int parse(const char *header_start, size_t len);
void packInto(Packer * p, bool mask_sensitive_info=false) const;
HttpHeaderEntry *getEntry(HttpHeaderPos * pos) const;
HttpHeaderEntry *findEntry(http_hdr_type id) const;
int delByName(const char *name);
int delById(http_hdr_type id);
void delAt(HttpHeaderPos pos, int &headers_deleted);
void refreshMask();
void addEntry(HttpHeaderEntry * e);
void insertEntry(HttpHeaderEntry * e);
String getList(http_hdr_type id) const;
bool getList(http_hdr_type id, String *s) const;
String getStrOrList(http_hdr_type id) const;
String getByName(const char *name) const;
/// sets value and returns true iff a [possibly empty] named field is there
bool getByNameIfPresent(const char *name, String &value) const;
String getByNameListMember(const char *name, const char *member, const char separator) const;
String getListMember(http_hdr_type id, const char *member, const char separator) const;
int has(http_hdr_type id) const;
void putInt(http_hdr_type id, int number);
void putInt64(http_hdr_type id, int64_t number);
=== modified file 'src/HttpHeaderTools.cc'
--- src/HttpHeaderTools.cc 2014-09-13 13:59:43 +0000
+++ src/HttpHeaderTools.cc 2014-09-14 12:23:03 +0000
@@ -144,46 +144,45 @@
#if USE_HTTP_VIOLATIONS
if (hdr->has(HDR_PROXY_CONNECTION))
list = hdr->getList(HDR_PROXY_CONNECTION);
else
#endif
if (hdr->has(HDR_CONNECTION))
list = hdr->getList(HDR_CONNECTION);
else
return 0;
res = strListIsMember(&list, directive, ',');
list.clean();
return res;
}
/** handy to printf prefixes of potentially very long buffers */
const char *
-getStringPrefix(const char *str, const char *end)
+getStringPrefix(const char *str, size_t sz)
{
#define SHORT_PREFIX_SIZE 512
LOCAL_ARRAY(char, buf, SHORT_PREFIX_SIZE);
- const int sz = 1 + (end ? end - str : strlen(str));
- xstrncpy(buf, str, (sz > SHORT_PREFIX_SIZE) ? SHORT_PREFIX_SIZE : sz);
+ xstrncpy(buf, str, (sz+1 > SHORT_PREFIX_SIZE) ? SHORT_PREFIX_SIZE : sz);
return buf;
}
/**
* parses an int field, complains if soemthing went wrong, returns true on
* success
*/
int
httpHeaderParseInt(const char *start, int *value)
{
assert(value);
*value = atoi(start);
if (!*value && !xisdigit(*start)) {
debugs(66, 2, "failed to parse an int header field near '" << start << "'");
return 0;
}
return 1;
}
=== modified file 'src/HttpHeaderTools.h'
--- src/HttpHeaderTools.h 2014-09-13 13:59:43 +0000
+++ src/HttpHeaderTools.h 2014-09-14 12:23:03 +0000
@@ -107,25 +107,25 @@
Format::Format *valueFormat;
/// internal ID for "known" headers or HDR_OTHER
http_hdr_type fieldId;
/// whether fieldValue may contain macros
bool quoted;
};
int httpHeaderParseOffset(const char *start, int64_t * off);
HttpHeaderFieldInfo *httpHeaderBuildFieldsInfo(const HttpHeaderFieldAttrs * attrs, int count);
void httpHeaderDestroyFieldsInfo(HttpHeaderFieldInfo * info, int count);
http_hdr_type httpHeaderIdByName(const char *name, size_t name_len, const HttpHeaderFieldInfo * attrs, int end);
http_hdr_type httpHeaderIdByNameDef(const char *name, int name_len);
const char *httpHeaderNameById(int id);
int httpHeaderHasConnDir(const HttpHeader * hdr, const char *directive);
int httpHeaderParseInt(const char *start, int *val);
void httpHeaderPutStrf(HttpHeader * hdr, http_hdr_type id, const char *fmt,...) PRINTF_FORMAT_ARG3;
-const char *getStringPrefix(const char *str, const char *end);
+const char *getStringPrefix(const char *str, size_t len);
void httpHdrMangleList(HttpHeader *, HttpRequest *, int req_or_rep);
#endif
=== modified file 'src/HttpMsg.cc'
--- src/HttpMsg.cc 2014-09-13 13:59:43 +0000
+++ src/HttpMsg.cc 2014-10-11 10:21:16 +0000
@@ -18,41 +18,41 @@
#include "SquidConfig.h"
HttpMsg::HttpMsg(http_hdr_owner_type owner): header(owner),
cache_control(NULL), hdr_sz(0), content_length(0),
pstate(psReadyToParseStartLine)
{}
HttpMsg::~HttpMsg()
{
assert(!body_pipe);
}
HttpMsgParseState &operator++ (HttpMsgParseState &aState)
{
int tmp = (int)aState;
aState = (HttpMsgParseState)(++tmp);
return aState;
}
/* find end of headers */
-int
+static int
httpMsgIsolateHeaders(const char **parse_start, int l, const char **blk_start, const char **blk_end)
{
/*
* parse_start points to the first line of HTTP message *headers*,
* not including the request or status lines
*/
size_t end = headersEnd(*parse_start, l);
int nnl;
if (end) {
*blk_start = *parse_start;
*blk_end = *parse_start + end - 1;
/*
* leave blk_end pointing to the first character after the
* first newline which terminates the headers
*/
assert(**blk_end == '\n');
while (*(*blk_end - 1) == '\r')
--(*blk_end);
@@ -238,48 +238,49 @@
PROF_stop(HttpMsg_httpMsgParseStep);
return httpMsgParseError();
}
*parse_end_ptr = parse_start;
hdr_sz = *parse_end_ptr - buf;
parse_len = parse_len - hdr_sz;
++pstate;
}
/*
* XXX This code uses parse_start; but if we're incrementally parsing then
* this code might not actually be given parse_start at the right spot (just
* after headers.) Grr.
*/
if (pstate == psReadyToParseHeaders) {
if (!httpMsgIsolateHeaders(&parse_start, parse_len, &blk_start, &blk_end)) {
if (atEnd) {
- blk_start = parse_start, blk_end = blk_start + strlen(blk_start);
+ blk_start = parse_start;
+ blk_end = blk_start + strlen(blk_start);
} else {
PROF_stop(HttpMsg_httpMsgParseStep);
return 0;
}
}
- if (!header.parse(blk_start, blk_end)) {
+ if (!header.parse(blk_start, blk_end-blk_start)) {
PROF_stop(HttpMsg_httpMsgParseStep);
return httpMsgParseError();
}
hdrCacheInit();
*parse_end_ptr = parse_start;
hdr_sz = *parse_end_ptr - buf;
++pstate;
}
PROF_stop(HttpMsg_httpMsgParseStep);
return 1;
}
/* handy: resets and returns -1 */
int
HttpMsg::httpMsgParseError()
=== modified file 'src/HttpMsg.h'
--- src/HttpMsg.h 2014-09-13 13:59:43 +0000
+++ src/HttpMsg.h 2014-09-14 12:23:03 +0000
@@ -1,37 +1,37 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
#ifndef SQUID_HTTPMSG_H
#define SQUID_HTTPMSG_H
#include "base/Lock.h"
#include "BodyPipe.h"
+#include "http/forward.h"
#include "http/ProtocolVersion.h"
#include "http/StatusCode.h"
#include "HttpHeader.h"
-#include "HttpRequestMethod.h"
/// common parts of HttpRequest and HttpReply
class HttpMsg : public RefCountable
{
public:
typedef RefCount<HttpMsg> Pointer;
HttpMsg(http_hdr_owner_type owner);
virtual ~HttpMsg();
virtual void reset() = 0; // will have body when http*Clean()s are gone
void packInto(Packer * p, bool full_uri) const;
///< produce a message copy, except for a few connection-specific settings
virtual HttpMsg *clone() const = 0; ///< \todo rename: not a true copy?
/// [re]sets Content-Length header and cached value
void setContentLength(int64_t clen);
@@ -81,26 +81,24 @@
virtual bool inheritProperties(const HttpMsg *aMsg) = 0;
protected:
/**
* Validate the message start line is syntactically correct.
* Set HTTP error status according to problems found.
*
* \retval true Status line has no serious problems.
* \retval false Status line has a serious problem. Correct response is indicated by error.
*/
virtual bool sanityCheckStartLine(MemBuf *buf, const size_t hdr_len, Http::StatusCode *error) = 0;
virtual void packFirstLineInto(Packer * p, bool full_uri) const = 0;
virtual bool parseFirstLine(const char *blk_start, const char *blk_end) = 0;
virtual void hdrCacheInit();
};
-int httpMsgIsolateHeaders(const char **parse_start, int len, const char **blk_start, const char **blk_end);
-
#define HTTPMSGUNLOCK(a) if (a) { if ((a)->unlock() == 0) delete (a); (a)=NULL; }
#define HTTPMSGLOCK(a) (a)->lock()
#endif /* SQUID_HTTPMSG_H */
=== modified file 'src/HttpRequest.cc'
--- src/HttpRequest.cc 2014-09-13 13:59:43 +0000
+++ src/HttpRequest.cc 2014-10-15 07:12:58 +0000
@@ -1,40 +1,41 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
/* DEBUG: section 73 HTTP Request */
#include "squid.h"
#include "AccessLogEntry.h"
#include "acl/AclSizeLimit.h"
#include "acl/FilledChecklist.h"
#include "client_side.h"
#include "DnsLookupDetails.h"
#include "err_detail_type.h"
#include "globals.h"
#include "gopher.h"
#include "http.h"
+#include "http/one/RequestParser.h"
#include "HttpHdrCc.h"
#include "HttpHeaderRange.h"
#include "HttpRequest.h"
#include "log/Config.h"
#include "MemBuf.h"
#include "SquidConfig.h"
#include "Store.h"
#include "URL.h"
#if USE_AUTH
#include "auth/UserRequest.h"
#endif
#if ICAP_CLIENT
#include "adaptation/icap/icap_log.h"
#endif
HttpRequest::HttpRequest() :
HttpMsg(hoRequest)
{
init();
@@ -262,106 +263,112 @@
/**
* Checks the first line of an HTTP request is valid
* currently just checks the request method is present.
*
* NP: Other errors are left for detection later in the parse.
*/
bool
HttpRequest::sanityCheckStartLine(MemBuf *buf, const size_t hdr_len, Http::StatusCode *error)
{
// content is long enough to possibly hold a reply
// 2 being magic size of a 1-byte request method plus space delimiter
if ( buf->contentSize() < 2 ) {
// this is ony a real error if the headers apparently complete.
if (hdr_len > 0) {
debugs(58, 3, HERE << "Too large request header (" << hdr_len << " bytes)");
*error = Http::scInvalidHeader;
}
return false;
}
- /* See if the request buffer starts with a known HTTP request method. */
- if (HttpRequestMethod(buf->content(),NULL) == Http::METHOD_NONE) {
+ /* See if the request buffer starts with a non-whitespace HTTP request 'method'. */
+ HttpRequestMethod m;
+ m.HttpRequestMethodXXX(buf->content());
+ if (m == Http::METHOD_NONE) {
debugs(73, 3, "HttpRequest::sanityCheckStartLine: did not find HTTP request method");
*error = Http::scInvalidHeader;
return false;
}
return true;
}
bool
HttpRequest::parseFirstLine(const char *start, const char *end)
{
- const char *t = start + strcspn(start, w_space);
- method = HttpRequestMethod(start, t);
+ method.HttpRequestMethodXXX(start);
if (method == Http::METHOD_NONE)
return false;
- start = t + strspn(t, w_space);
+ // XXX: performance regression, strcspn() over the method bytes a second time.
+ // cheaper than allocate+copy+deallocate cycle to SBuf convert a piece of start.
+ const char *t = start + strcspn(start, w_space);
+
+ start = t + strspn(t, w_space); // skip w_space after method
const char *ver = findTrailingHTTPVersion(start, end);
if (ver) {
end = ver - 1;
while (xisspace(*end)) // find prev non-space
--end;
++end; // back to space
if (2 != sscanf(ver + 5, "%d.%d", &http_ver.major, &http_ver.minor)) {
debugs(73, DBG_IMPORTANT, "parseRequestLine: Invalid HTTP identifier.");
return false;
}
} else {
http_ver.major = 0;
http_ver.minor = 9;
}
if (end < start) // missing URI
return false;
char save = *end;
* (char *) end = '\0'; // temp terminate URI, XXX dangerous?
HttpRequest *tmp = urlParse(method, (char *) start, this);
* (char *) end = save;
if (NULL == tmp)
return false;
return true;
}
-int
-HttpRequest::parseHeader(const char *parse_start, int len)
+bool
+HttpRequest::parseHeader(Http1::RequestParser &hp)
{
- const char *blk_start, *blk_end;
-
- if (!httpMsgIsolateHeaders(&parse_start, len, &blk_start, &blk_end))
- return 0;
+ // HTTP/1 message contains "zero or more header fields"
+ // zero does not need parsing
+ if (!hp.headerBlockSize())
+ return true;
- int result = header.parse(blk_start, blk_end);
+ // XXX: c_str() reallocates. performance regression.
+ const bool result = header.parse(hp.mimeHeader().c_str(), hp.headerBlockSize());
if (result)
hdrCacheInit();
return result;
}
/* swaps out request using httpRequestPack */
void
HttpRequest::swapOut(StoreEntry * e)
{
Packer p;
assert(e);
packerToStoreInit(&p, e);
pack(&p);
packerClean(&p);
}
/* packs request-line and headers, appends <crlf> terminator */
void
=== modified file 'src/HttpRequest.h'
--- src/HttpRequest.h 2014-09-13 13:59:43 +0000
+++ src/HttpRequest.h 2014-09-14 12:23:03 +0000
@@ -1,37 +1,37 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
#ifndef SQUID_HTTPREQUEST_H
#define SQUID_HTTPREQUEST_H
#include "base/CbcPointer.h"
#include "Debug.h"
#include "err_type.h"
#include "HierarchyLogEntry.h"
#include "HttpMsg.h"
-#include "HttpRequestMethod.h"
+#include "http/RequestMethod.h"
#include "Notes.h"
#include "RequestFlags.h"
#include "URL.h"
#if USE_AUTH
#include "auth/UserRequest.h"
#endif
#if USE_ADAPTATION
#include "adaptation/History.h"
#endif
#if ICAP_CLIENT
#include "adaptation/icap/History.h"
#endif
#if USE_SQUID_EUI
#include "eui/Eui48.h"
#include "eui/Eui64.h"
#endif
class ConnStateData;
@@ -192,41 +192,41 @@
String extacl_user; /* User name returned by extacl lookup */
String extacl_passwd; /* Password returned by extacl lookup */
String extacl_log; /* String to be used for access.log purposes */
String extacl_message; /* String to be used for error page purposes */
#if FOLLOW_X_FORWARDED_FOR
String x_forwarded_for_iterator; /* XXX a list of IP addresses */
#endif /* FOLLOW_X_FORWARDED_FOR */
/// A strong etag of the cached entry. Used for refreshing that entry.
String etag;
public:
bool multipartRangeRequest() const;
bool parseFirstLine(const char *start, const char *end);
- int parseHeader(const char *parse_start, int len);
+ bool parseHeader(Http1::RequestParser &hp); // TODO move this function to the parser
virtual bool expectingBody(const HttpRequestMethod& unused, int64_t&) const;
bool bodyNibbled() const; // the request has a [partially] consumed body
int prefixLen();
void swapOut(StoreEntry * e);
void pack(Packer * p);
static void httpRequestPack(void *obj, Packer *p);
static HttpRequest * CreateFromUrlAndMethod(char * url, const HttpRequestMethod& method);
static HttpRequest * CreateFromUrl(char * url);
ConnStateData *pinnedConnection();
/**
=== modified file 'src/Makefile.am'
--- src/Makefile.am 2014-10-14 13:57:05 +0000
+++ src/Makefile.am 2014-10-16 12:13:18 +0000
@@ -386,50 +386,46 @@
HttpHdrRange.cc \
HttpHdrSc.cc \
HttpHdrSc.h \
HttpHdrScTarget.cc \
HttpHdrScTarget.h \
HttpHdrContRange.cc \
HttpHdrContRange.h \
HttpHeaderStat.h \
HttpHeader.h \
HttpHeader.cc \
HttpHeaderMask.h \
HttpHeaderRange.h \
HttpHeaderFieldInfo.h \
HttpHeaderTools.h \
HttpHeaderTools.cc \
HttpBody.h \
HttpBody.cc \
HttpControlMsg.h \
HttpMsg.cc \
HttpMsg.h \
- HttpParser.cc \
- HttpParser.h \
HttpReply.cc \
HttpReply.h \
RequestFlags.h \
RequestFlags.cc \
HttpRequest.cc \
HttpRequest.h \
- HttpRequestMethod.cc \
- HttpRequestMethod.h \
ICP.h \
icp_opcode.h \
icp_v2.cc \
icp_v3.cc \
int.h \
int.cc \
internal.h \
internal.cc \
$(IPC_SOURCE) \
ipcache.cc \
ipcache.h \
$(LEAKFINDERSOURCE) \
SquidList.h \
SquidList.cc \
LogTags.h \
lookup_t.h \
main.cc \
MasterXaction.cc \
MasterXaction.h \
Mem.h \
@@ -598,69 +594,69 @@
icp_opcode.cc \
LogTags.cc \
lookup_t.cc \
repl_modules.cc \
swap_log_op.cc
CLEANFILES += $(BUILT_SOURCES)
nodist_squid_SOURCES = \
$(DISKIO_GEN_SOURCE) \
$(BUILT_SOURCES)
squid_LDADD = \
$(AUTH_ACL_LIBS) \
ident/libident.la \
acl/libacls.la \
acl/libstate.la \
$(AUTH_LIBS) \
$(DISK_LIBS) \
acl/libapi.la \
+ clients/libclients.la \
+ servers/libservers.la \
+ ftp/libftp.la \
+ helper/libhelper.la \
+ http/libsquid-http.la \
+ parser/libsquid-parser.la \
base/libbase.la \
libsquid.la \
ip/libip.la \
fs/libfs.la \
$(SSL_LIBS) \
ipc/libipc.la \
mgr/libmgr.la \
anyp/libanyp.la \
comm/libcomm.la \
eui/libeui.la \
- helper/libhelper.la \
- http/libsquid-http.la \
icmp/libicmp.la icmp/libicmp-core.la \
log/liblog.la \
format/libformat.la \
- clients/libclients.la \
- servers/libservers.la \
- ftp/libftp.la \
$(XTRA_OBJS) \
$(DISK_LINKOBJS) \
$(REPL_OBJS) \
$(DISK_OS_LIBS) \
$(NETTLELIB) \
$(CRYPTLIB) \
$(REGEXLIB) \
$(ADAPTATION_LIBS) \
$(ESI_LIBS) \
$(SNMP_LIBS) \
- parser/libsquid-parser.la \
$(top_builddir)/lib/libmisccontainers.la \
$(top_builddir)/lib/libmiscencoding.la \
$(top_builddir)/lib/libmiscutil.la \
$(SSLLIB) \
$(EPOLL_LIBS) \
$(MINGW_LIBS) \
$(KRB5LIBS) \
$(COMPAT_LIB) \
$(XTRA_LIBS)
squid_DEPENDENCIES = \
$(DISK_LIBS) \
$(DISK_LINKOBJS) \
$(REPL_OBJS) \
$(ADAPTATION_LIBS) \
$(ESI_LOCAL_LIBS) \
$(SSL_LIBS) \
$(AUTH_ACL_LIBS) \
ident/libident.la \
acl/libacls.la \
eui/libeui.la \
@@ -726,41 +722,41 @@
debug.cc \
int.h \
int.cc \
Mem.h \
mem.cc \
MemBuf.cc \
MemBuf.h \
Parsing.h \
store_key_md5.h \
store_key_md5.cc \
tests/stub_StoreMeta.cc \
StoreMetaUnpacker.cc \
String.cc \
SquidNew.cc \
tests/stub_time.cc \
ufsdump.cc \
dlink.h \
dlink.cc \
helper/ChildConfig.h \
tests/stub_HelperChildConfig.cc \
- HttpRequestMethod.cc \
+ http/RequestMethod.cc \
RemovalPolicy.cc \
$(WIN32_SOURCE) \
fd.h \
tests/stub_fd.cc
ufsdump_LDADD = \
ident/libident.la \
acl/libacls.la \
eui/libeui.la \
acl/libstate.la \
acl/libapi.la \
base/libbase.la \
libsquid.la \
ip/libip.la \
fs/libfs.la \
ipc/libipc.la \
mgr/libmgr.la \
$(XTRA_OBJS) \
$(REPL_OBJS) \
$(NETTLELIB) \
$(CRYPTLIB) \
@@ -1054,41 +1050,41 @@
test_tools.cc: $(top_srcdir)/test-suite/test_tools.cc
cp $(top_srcdir)/test-suite/test_tools.cc .
# stock tools for unit tests - library independent versions of dlink_list
# etc.
# globals.cc is needed by test_tools.cc.
# Neither of these should be disted from here.
TESTSOURCES= \
tests/STUB.h \
test_tools.cc \
globals.cc
check_PROGRAMS+=\
tests/testBoilerplate \
tests/testCacheManager \
tests/testDiskIO \
tests/testEvent \
tests/testEventLoop \
tests/test_http_range \
- tests/testHttpParser \
+ tests/testHttp1Parser \
tests/testHttpReply \
tests/testHttpRequest \
tests/testStore \
tests/testString \
tests/testURL \
tests/testSBuf \
tests/testSBufList \
tests/testConfigParser \
tests/testStatHist
if HAVE_FS_ROCK
check_PROGRAMS += tests/testRock
endif
if HAVE_FS_UFS
check_PROGRAMS += tests/testUfs
endif
## NP: required to run the above list. check_PROGRAMS only builds the binaries...
TESTS += $(check_PROGRAMS)
@@ -1192,40 +1188,41 @@
tests/stub_StatHist.cc \
repl_modules.h \
tests/stub_store.cc \
tests/stub_store_stats.cc \
tools.h \
tests/stub_tools.cc \
tests/stub_HttpRequest.cc \
tests/testHttpReply.cc \
tests/testHttpReply.h \
tests/testMain.cc \
tests/stub_time.cc \
url.cc \
wordlist.h \
wordlist.cc
nodist_tests_testHttpReply_SOURCES=\
$(TESTSOURCES)
tests_testHttpReply_LDFLAGS = $(LIBADD_DL)
tests_testHttpReply_LDADD=\
CommCalls.o \
http/libsquid-http.la \
+ parser/libsquid-parser.la \
acl/libacls.la \
acl/libapi.la \
acl/libstate.la \
anyp/libanyp.la \
ip/libip.la \
base/libbase.la \
ipc/libipc.la \
$(top_builddir)/lib/libmisccontainers.la \
$(top_builddir)/lib/libmiscencoding.la \
$(top_builddir)/lib/libmiscutil.la \
$(SQUID_CPPUNIT_LIBS) \
$(SQUID_CPPUNIT_LA) \
$(NETTLELIB) \
$(SSLLIB) \
$(COMPAT_LIB) \
$(XTRA_LIBS)
tests_testHttpReply_DEPENDENCIES= $(SQUID_CPPUNIT_LA)
tests_testACLMaxUserIP_SOURCES= \
cbdata.cc \
@@ -1239,41 +1236,40 @@
event.cc \
fatal.h \
tests/stub_fatal.cc \
FileMap.h \
filemap.cc \
HttpBody.cc \
HttpHeader.h \
HttpHeader.cc \
HttpHeaderFieldInfo.h \
HttpHeaderTools.h \
HttpHeaderTools.cc \
HttpHdrContRange.cc \
HttpHdrRange.cc \
HttpHeaderFieldStat.h \
HttpHdrCc.h \
HttpHdrCc.cc \
HttpHdrCc.cci \
HttpHdrSc.cc \
HttpHdrScTarget.cc \
HttpMsg.cc \
- HttpRequestMethod.cc \
int.h \
int.cc \
MasterXaction.cc \
MasterXaction.h \
Notes.cc \
Notes.h \
SquidList.h \
SquidList.cc \
mem_node.cc \
Packer.cc \
Parsing.cc \
SquidMath.cc \
StatCounters.cc \
StatCounters.h \
StatHist.h \
StrList.h \
StrList.cc \
tests/stub_StatHist.cc \
stmem.cc \
$(SBUF_SOURCE) \
@@ -1328,40 +1324,41 @@
tests/stub_tools.cc \
tests/stub_cache_manager.cc \
tests/stub_UdsOp.cc \
tests/testACLMaxUserIP.cc \
tests/testACLMaxUserIP.h \
tests/testMain.cc \
tests/stub_time.cc \
url.cc \
URL.h \
Mem.h \
tests/stub_mem.cc \
MemBuf.cc \
wordlist.h \
wordlist.cc
nodist_tests_testACLMaxUserIP_SOURCES= \
$(TESTSOURCES)
tests_testACLMaxUserIP_LDADD= \
libsquid.la \
helper/libhelper.la \
http/libsquid-http.la \
+ parser/libsquid-parser.la \
$(AUTH_ACL_LIBS) \
ident/libident.la \
acl/libacls.la \
eui/libeui.la \
acl/libstate.la \
acl/libapi.la \
anyp/libanyp.la \
base/libbase.la \
ip/libip.la \
ipc/libipc.la \
mgr/libmgr.la \
$(top_builddir)/lib/libmisccontainers.la \
$(top_builddir)/lib/libmiscencoding.la \
$(top_builddir)/lib/libmiscutil.la \
$(DISK_OS_LIBS) \
$(NETTLELIB) \
$(REGEXLIB) \
$(SQUID_CPPUNIT_LIBS) \
$(SSLLIB) \
$(COMPAT_LIB) \
@@ -1379,46 +1376,43 @@
tests/stub_debug.cc \
tests/stub_time.cc
nodist_tests_testBoilerplate_SOURCES = \
tests/stub_cbdata.cc \
tests/stub_MemBuf.cc \
$(TESTSOURCES)
tests_testBoilerplate_LDADD= \
$(SQUID_CPPUNIT_LIBS) \
$(SSLLIB) \
base/libbase.la \
$(COMPAT_LIB) \
$(XTRA_LIBS)
tests_testBoilerplate_LDFLAGS = $(LIBADD_DL)
tests_testBoilerplate_DEPENDENCIES = \
$(SQUID_CPPUNIT_LA)
## Tests of the CacheManager module.
tests_testCacheManager_SOURCES = \
AccessLogEntry.cc \
debug.cc \
- HttpParser.cc \
- HttpParser.h \
RequestFlags.h \
RequestFlags.cc \
HttpRequest.cc \
- HttpRequestMethod.cc \
Mem.h \
tests/stub_mem.cc \
String.cc \
tests/testCacheManager.cc \
tests/testCacheManager.h \
tests/testMain.cc \
tests/stub_main_cc.cc \
tests/stub_ipc_Forwarder.cc \
tests/stub_store_stats.cc \
tests/stub_EventLoop.cc \
time.cc \
BodyPipe.cc \
cache_manager.cc \
cache_cf.h \
AuthReg.h \
YesNoNone.h \
YesNoNone.cc \
RefreshPattern.h \
cache_cf.cc \
CacheDigest.h \
@@ -1581,48 +1575,48 @@
$(UNLINKDSOURCE) \
url.cc \
urn.h \
urn.cc \
wccp2.h \
tests/stub_wccp2.cc \
whois.h \
tests/stub_whois.cc \
FadingCounter.cc \
$(WIN32_SOURCE) \
wordlist.h \
wordlist.cc
nodist_tests_testCacheManager_SOURCES = \
$(BUILT_SOURCES) \
$(DISKIO_GEN_SOURCE)
# comm.cc only requires comm/libcomm.la until fdc_table is dead.
tests_testCacheManager_LDADD = \
libsquid.la \
clients/libclients.la \
servers/libservers.la \
+ ftp/libftp.la \
helper/libhelper.la \
http/libsquid-http.la \
- ftp/libftp.la \
+ parser/libsquid-parser.la \
ident/libident.la \
acl/libacls.la \
acl/libstate.la \
acl/libapi.la \
- parser/libsquid-parser.la \
base/libbase.la \
ip/libip.la \
fs/libfs.la \
comm/libcomm.la \
eui/libeui.la \
icmp/libicmp.la icmp/libicmp-core.la \
log/liblog.la \
format/libformat.la \
$(REPL_OBJS) \
$(DISK_LIBS) \
$(DISK_OS_LIBS) \
$(ADAPTATION_LIBS) \
$(ESI_LIBS) \
$(SSL_LIBS) \
anyp/libanyp.la \
ipc/libipc.la \
mgr/libmgr.la \
$(SNMP_LIBS) \
$(top_builddir)/lib/libmisccontainers.la \
$(top_builddir)/lib/libmiscencoding.la \
@@ -1664,41 +1658,40 @@
fde.cc \
FileMap.h \
filemap.cc \
HttpBody.h \
HttpBody.cc \
HttpHeaderFieldStat.h \
HttpHdrCc.h \
HttpHdrCc.cc \
HttpHdrCc.cci \
HttpHdrContRange.cc \
HttpHdrSc.cc \
HttpHdrScTarget.cc \
HttpHdrRange.cc \
HttpHeaderFieldInfo.h \
HttpHeaderTools.h \
HttpHeaderTools.cc \
HttpHeader.h \
HttpHeader.cc \
HttpMsg.cc \
HttpReply.cc \
- HttpRequestMethod.cc \
int.h \
int.cc \
SquidList.h \
SquidList.cc \
MasterXaction.cc \
MasterXaction.h \
MemBuf.cc \
MemObject.cc \
mem_node.cc \
Mem.h \
tests/stub_mem.cc \
Notes.h \
Notes.cc \
Packer.cc \
Parsing.cc \
refresh.h \
refresh.cc \
RemovalPolicy.cc \
RequestFlags.h \
RequestFlags.cc \
@@ -1771,40 +1764,41 @@
tests/testMain.cc \
tests/testStoreSupport.cc \
tests/testStoreSupport.h \
tests/stub_time.cc \
$(UNLINKDSOURCE) \
url.cc \
$(WIN32_SOURCE) \
wordlist.h \
wordlist.cc \
tools.h \
tests/stub_tools.cc
nodist_tests_testDiskIO_SOURCES= \
$(TESTSOURCES) \
$(DISKIO_GEN_SOURCE) \
SquidMath.cc \
SquidMath.h \
swap_log_op.cc
tests_testDiskIO_LDADD = \
libsquid.la \
http/libsquid-http.la \
+ parser/libsquid-parser.la \
SquidConfig.o \
CommCalls.o \
DnsLookupDetails.o \
ident/libident.la \
acl/libacls.la \
acl/libstate.la \
comm/libcomm.la \
ip/libip.la \
fs/libfs.la \
ipc/libipc.la \
$(REPL_OBJS) \
$(DISK_LIBS) \
$(DISK_OS_LIBS) \
acl/libapi.la \
anyp/libanyp.la \
mgr/libmgr.la \
$(SSL_LIBS) \
ipc/libipc.la \
base/libbase.la \
$(top_builddir)/lib/libmisccontainers.la \
@@ -1887,49 +1881,46 @@
helper.cc \
hier_code.h \
$(HTCPSOURCE) \
http.cc \
HttpBody.h \
HttpBody.cc \
HttpHeader.h \
HttpHeader.cc \
HttpHeaderFieldInfo.h \
HttpHeaderTools.h \
HttpHeaderTools.cc \
HttpHeaderFieldStat.h \
HttpHdrCc.h \
HttpHdrCc.cc \
HttpHdrCc.cci \
HttpHdrContRange.cc \
HttpHdrRange.cc \
HttpHdrSc.cc \
HttpHdrScTarget.cc \
HttpMsg.cc \
- HttpParser.cc \
- HttpParser.h \
HttpReply.cc \
PeerPoolMgr.h \
PeerPoolMgr.cc \
RequestFlags.h \
RequestFlags.cc \
HttpRequest.cc \
- HttpRequestMethod.cc \
icp_v2.cc \
icp_v3.cc \
$(IPC_SOURCE) \
ipcache.cc \
int.h \
int.cc \
internal.h \
internal.cc \
SquidList.h \
SquidList.cc \
MasterXaction.cc \
MasterXaction.h \
Mem.h \
tests/stub_mem.cc \
mem_node.cc \
MemBuf.cc \
MemObject.cc \
mime.h \
mime.cc \
mime_header.h \
@@ -2013,48 +2004,48 @@
tests/stub_tunnel.cc \
MemStore.cc \
$(UNLINKDSOURCE) \
url.cc \
urn.h \
urn.cc \
wccp2.h \
tests/stub_wccp2.cc \
whois.h \
tests/stub_whois.cc \
$(WIN32_SOURCE) \
wordlist.h \
wordlist.cc
nodist_tests_testEvent_SOURCES = \
$(BUILT_SOURCES) \
$(DISKIO_GEN_SOURCE)
tests_testEvent_LDADD = \
libsquid.la \
clients/libclients.la \
servers/libservers.la \
+ ftp/libftp.la \
helper/libhelper.la \
http/libsquid-http.la \
- ftp/libftp.la \
+ parser/libsquid-parser.la \
ident/libident.la \
acl/libacls.la \
acl/libstate.la \
acl/libapi.la \
- parser/libsquid-parser.la \
base/libbase.la \
ip/libip.la \
fs/libfs.la \
anyp/libanyp.la \
icmp/libicmp.la icmp/libicmp-core.la \
comm/libcomm.la \
log/liblog.la \
format/libformat.la \
$(REPL_OBJS) \
$(ADAPTATION_LIBS) \
$(ESI_LIBS) \
$(SSL_LIBS) \
$(top_builddir)/lib/libmisccontainers.la \
$(top_builddir)/lib/libmiscencoding.la \
$(top_builddir)/lib/libmiscutil.la \
$(DISK_LIBS) \
$(DISK_OS_LIBS) \
ipc/libipc.la \
mgr/libmgr.la \
$(SNMP_LIBS) \
@@ -2135,49 +2126,46 @@
helper.cc \
hier_code.h \
$(HTCPSOURCE) \
http.cc \
HttpBody.h \
HttpBody.cc \
HttpHeader.h \
HttpHeader.cc \
HttpHeaderFieldInfo.h \
HttpHeaderTools.h \
HttpHeaderTools.cc \
HttpHeaderFieldStat.h \
HttpHdrCc.h \
HttpHdrCc.cc \
HttpHdrCc.cci \
HttpHdrContRange.cc \
HttpHdrRange.cc \
HttpHdrSc.cc \
HttpHdrScTarget.cc \
HttpMsg.cc \
- HttpParser.cc \
- HttpParser.h \
HttpReply.cc \
PeerPoolMgr.h \
PeerPoolMgr.cc \
RequestFlags.h \
RequestFlags.cc \
HttpRequest.cc \
- HttpRequestMethod.cc \
icp_v2.cc \
icp_v3.cc \
$(IPC_SOURCE) \
ipcache.cc \
int.h \
int.cc \
internal.h \
internal.cc \
SquidList.h \
SquidList.cc \
MasterXaction.cc \
MasterXaction.h \
MemBuf.cc \
MemObject.cc \
Mem.h \
tests/stub_mem.cc \
mem_node.cc \
mime.h \
mime.cc \
mime_header.h \
@@ -2261,48 +2249,48 @@
tests/stub_tunnel.cc \
MemStore.cc \
$(UNLINKDSOURCE) \
url.cc \
urn.h \
urn.cc \
wccp2.h \
tests/stub_wccp2.cc \
whois.h \
tests/stub_whois.cc \
$(WIN32_SOURCE) \
wordlist.h \
wordlist.cc
nodist_tests_testEventLoop_SOURCES = \
$(BUILT_SOURCES) \
$(DISKIO_GEN_SOURCE)
tests_testEventLoop_LDADD = \
libsquid.la \
clients/libclients.la \
servers/libservers.la \
+ ftp/libftp.la \
helper/libhelper.la \
http/libsquid-http.la \
- ftp/libftp.la \
+ parser/libsquid-parser.la \
ident/libident.la \
acl/libacls.la \
acl/libstate.la \
acl/libapi.la \
- parser/libsquid-parser.la \
base/libbase.la \
ip/libip.la \
fs/libfs.la \
anyp/libanyp.la \
icmp/libicmp.la icmp/libicmp-core.la \
comm/libcomm.la \
log/liblog.la \
format/libformat.la \
$(REPL_OBJS) \
$(ADAPTATION_LIBS) \
$(ESI_LIBS) \
$(SSL_LIBS) \
$(top_builddir)/lib/libmisccontainers.la \
$(top_builddir)/lib/libmiscencoding.la \
$(top_builddir)/lib/libmiscutil.la \
$(DISK_LIBS) \
$(DISK_OS_LIBS) \
ipc/libipc.la \
mgr/libmgr.la \
$(SNMP_LIBS) \
@@ -2379,49 +2367,46 @@
helper.cc \
hier_code.h \
$(HTCPSOURCE) \
http.cc \
HttpBody.h \
HttpBody.cc \
HttpHeaderFieldStat.h \
HttpHdrCc.h \
HttpHdrCc.cc \
HttpHdrCc.cci \
HttpHdrContRange.cc \
HttpHdrRange.cc \
HttpHdrSc.cc \
HttpHdrScTarget.cc \
HttpHeader.h \
HttpHeader.cc \
HttpHeaderFieldInfo.h \
HttpHeaderTools.h \
HttpHeaderTools.cc \
HttpMsg.cc \
- HttpParser.cc \
- HttpParser.h \
HttpReply.cc \
PeerPoolMgr.h \
PeerPoolMgr.cc \
RequestFlags.h \
RequestFlags.cc \
HttpRequest.cc \
- HttpRequestMethod.cc \
icp_v2.cc \
icp_v3.cc \
int.h \
int.cc \
internal.h \
internal.cc \
$(IPC_SOURCE) \
ipcache.cc \
SquidList.h \
SquidList.cc \
MasterXaction.cc \
MasterXaction.h \
MemBuf.cc \
MemObject.cc \
Mem.h \
tests/stub_mem.cc \
mem_node.cc \
mime.h \
mime.cc \
mime_header.h \
@@ -2502,136 +2487,136 @@
tools.cc \
tests/stub_tunnel.cc \
$(UNLINKDSOURCE) \
url.cc \
urn.h \
urn.cc \
wccp2.h \
tests/stub_wccp2.cc \
whois.h \
tests/stub_whois.cc \
$(WIN32_SOURCE) \
wordlist.h \
wordlist.cc
nodist_tests_test_http_range_SOURCES = \
$(BUILT_SOURCES) \
$(DISKIO_GEN_SOURCE)
tests_test_http_range_LDADD = \
libsquid.la \
clients/libclients.la \
servers/libservers.la \
+ ftp/libftp.la \
helper/libhelper.la \
http/libsquid-http.la \
- ftp/libftp.la \
+ parser/libsquid-parser.la \
ident/libident.la \
acl/libacls.la \
acl/libstate.la \
acl/libapi.la \
parser/libsquid-parser.la \
ip/libip.la \
fs/libfs.la \
anyp/libanyp.la \
icmp/libicmp.la icmp/libicmp-core.la \
comm/libcomm.la \
log/liblog.la \
format/libformat.la \
$(REPL_OBJS) \
$(DISK_LIBS) \
$(DISK_OS_LIBS) \
$(ADAPTATION_LIBS) \
$(ESI_LIBS) \
$(SSL_LIBS) \
ipc/libipc.la \
base/libbase.la \
mgr/libmgr.la \
$(SNMP_LIBS) \
$(top_builddir)/lib/libmisccontainers.la \
$(top_builddir)/lib/libmiscencoding.la \
$(top_builddir)/lib/libmiscutil.la \
$(NETTLELIB) \
$(REGEXLIB) \
$(SQUID_CPPUNIT_LIBS) \
$(SQUID_CPPUNIT_LA) \
$(SSLLIB) \
$(KRB5LIBS) \
$(COMPAT_LIB) \
$(XTRA_LIBS)
tests_test_http_range_LDFLAGS = $(LIBADD_DL)
tests_test_http_range_DEPENDENCIES = \
$(SQUID_CPPUNIT_LA)
-tests_testHttpParser_SOURCES = \
+tests_testHttp1Parser_SOURCES = \
Debug.h \
- HttpParser.cc \
- HttpParser.h \
MemBuf.cc \
MemBuf.h \
tests/stub_MemObject.cc \
Mem.h \
tests/stub_mem.cc \
+ mime_header.cc \
+ mime_header.h \
String.cc \
cache_cf.h \
YesNoNone.h \
$(SBUF_SOURCE) \
tests/stub_SBufDetailedStats.cc \
tests/stub_cache_cf.cc \
tests/stub_cache_manager.cc \
tests/stub_comm.cc \
tests/stub_cbdata.cc \
tests/stub_debug.cc \
tests/stub_event.cc \
tests/stub_HelperChildConfig.cc \
tests/stub_stmem.cc \
tests/stub_store.cc \
tests/stub_store_stats.cc \
tools.h \
tests/stub_tools.cc \
- tests/testHttpParser.cc \
- tests/testHttpParser.h \
+ tests/testHttp1Parser.cc \
+ tests/testHttp1Parser.h \
tests/testMain.cc \
tests/stub_time.cc \
wordlist.h \
wordlist.cc
-nodist_tests_testHttpParser_SOURCES = \
+nodist_tests_testHttp1Parser_SOURCES = \
$(TESTSOURCES)
-tests_testHttpParser_LDADD= \
+tests_testHttp1Parser_LDADD= \
http/libsquid-http.la \
+ parser/libsquid-parser.la \
+ anyp/libanyp.la \
SquidConfig.o \
base/libbase.la \
ip/libip.la \
$(top_builddir)/lib/libmiscutil.la \
$(SQUID_CPPUNIT_LIBS) \
$(COMPAT_LIB) \
$(XTRA_LIBS)
-tests_testHttpParser_LDFLAGS = $(LIBADD_DL)
-tests_testHttpParser_DEPENDENCIES = \
+tests_testHttp1Parser_LDFLAGS = $(LIBADD_DL)
+tests_testHttp1Parser_DEPENDENCIES = \
$(SQUID_CPPUNIT_LA)
## Tests of the HttpRequest module.
tests_testHttpRequest_SOURCES = \
AccessLogEntry.cc \
- HttpParser.cc \
- HttpParser.h \
RequestFlags.h \
RequestFlags.cc \
HttpRequest.cc \
- HttpRequestMethod.cc \
Mem.h \
tests/stub_mem.cc \
String.cc \
tests/testHttpRequest.h \
tests/testHttpRequest.cc \
tests/testHttpRequestMethod.h \
tests/testHttpRequestMethod.cc \
tests/testMain.cc \
tests/stub_DiskIOModule.cc \
tests/stub_libauth.cc \
tests/stub_main_cc.cc \
tests/stub_ipc_Forwarder.cc \
tests/stub_libeui.cc \
tests/stub_store_stats.cc \
tests/stub_EventLoop.cc \
time.cc \
BodyPipe.cc \
cache_manager.cc \
cache_cf.h \
AuthReg.h \
@@ -2794,58 +2779,59 @@
MemStore.cc \
url.cc \
urn.h \
urn.cc \
wccp2.h \
tests/stub_wccp2.cc \
whois.h \
tests/stub_whois.cc \
FadingCounter.cc \
$(WIN32_SOURCE) \
wordlist.h \
wordlist.cc
nodist_tests_testHttpRequest_SOURCES = \
$(BUILT_SOURCES)
tests_testHttpRequest_LDADD = \
libsquid.la \
clients/libclients.la \
servers/libservers.la \
helper/libhelper.la \
ftp/libftp.la \
+ http/libsquid-http.la \
ident/libident.la \
acl/libacls.la \
acl/libstate.la \
acl/libapi.la \
parser/libsquid-parser.la \
ip/libip.la \
fs/libfs.la \
$(SSL_LIBS) \
ipc/libipc.la \
+ parser/libsquid-parser.la \
base/libbase.la \
mgr/libmgr.la \
anyp/libanyp.la \
$(SNMP_LIBS) \
icmp/libicmp.la icmp/libicmp-core.la \
comm/libcomm.la \
log/liblog.la \
format/libformat.la \
- http/libsquid-http.la \
$(REPL_OBJS) \
$(ADAPTATION_LIBS) \
$(ESI_LIBS) \
$(top_builddir)/lib/libmisccontainers.la \
$(top_builddir)/lib/libmiscencoding.la \
$(top_builddir)/lib/libmiscutil.la \
$(DISK_OS_LIBS) \
$(NETTLELIB) \
$(REGEXLIB) \
$(SQUID_CPPUNIT_LIBS) \
$(SQUID_CPPUNIT_LA) \
$(SSLLIB) \
$(KRB5LIBS) \
$(COMPAT_LIB) \
$(XTRA_LIBS)
tests_testHttpRequest_LDFLAGS = $(LIBADD_DL)
tests_testHttpRequest_DEPENDENCIES = \
$(REPL_OBJS) \
$(SQUID_CPPUNIT_LA)
@@ -2868,41 +2854,40 @@
event.cc \
EventLoop.cc \
fatal.h \
tests/stub_fatal.cc \
FileMap.h \
filemap.cc \
HttpHeaderFieldStat.h \
HttpHdrCc.h \
HttpHdrCc.cc \
HttpHdrCc.cci \
HttpHdrContRange.cc \
HttpHdrRange.cc \
HttpHdrSc.cc \
HttpHdrScTarget.cc \
HttpHeaderFieldInfo.h \
HttpHeaderTools.h \
HttpHeaderTools.cc \
HttpHeader.h \
HttpHeader.cc \
HttpMsg.cc \
- HttpRequestMethod.cc \
RequestFlags.cc \
RequestFlags.h \
int.h \
int.cc \
SquidList.h \
SquidList.cc \
MasterXaction.cc \
MasterXaction.h \
Mem.h \
tests/stub_mem.cc \
mem_node.cc \
MemBuf.cc \
MemObject.cc \
Notes.h \
Notes.cc \
Packer.cc \
Parsing.cc \
RemovalPolicy.cc \
refresh.h \
refresh.cc \
@@ -2981,40 +2966,41 @@
tests/testStoreHashIndex.cc \
tests/testStoreHashIndex.h \
tests/testStoreSupport.cc \
tests/testStoreSupport.h \
tests/TestSwapDir.cc \
tests/TestSwapDir.h \
tests/stub_time.cc \
url.cc \
wordlist.h \
wordlist.cc
nodist_tests_testStore_SOURCES= \
$(TESTSOURCES) \
SquidMath.cc \
SquidMath.h \
swap_log_op.cc
tests_testStore_LDADD= \
libsquid.la \
http/libsquid-http.la \
+ parser/libsquid-parser.la \
ident/libident.la \
acl/libacls.la \
acl/libstate.la \
acl/libapi.la \
base/libbase.la \
ip/libip.la \
fs/libfs.la \
mgr/libmgr.la \
ipc/libipc.la \
anyp/libanyp.la \
$(top_builddir)/lib/libmisccontainers.la \
$(top_builddir)/lib/libmiscencoding.la \
$(top_builddir)/lib/libmiscutil.la \
$(NETTLELIB) \
$(REGEXLIB) \
$(SQUID_CPPUNIT_LIBS) \
$(SSLLIB) \
CommCalls.o \
DnsLookupDetails.o \
$(COMPAT_LIB) \
@@ -3136,41 +3122,40 @@
StoreIOState.cc \
StoreMetaUnpacker.cc \
$(STOREMETA_SOURCE) \
StoreFileSystem.cc \
store_io.cc \
store_swapout.cc \
store_swapmeta.cc \
$(UNLINKDSOURCE) \
$(WIN32_SOURCE) \
event.cc \
$(DELAY_POOL_SOURCE) \
CacheDigest.h \
tests/stub_CacheDigest.cc \
ConfigParser.cc \
EventLoop.cc \
HttpMsg.cc \
RemovalPolicy.cc \
store_dir.cc \
repl_modules.h \
store.cc \
- HttpRequestMethod.cc \
store_key_md5.h \
store_key_md5.cc \
Parsing.cc \
ConfigOption.cc \
SwapDir.cc \
tests/stub_acl.cc \
cache_cf.h \
YesNoNone.h \
tests/stub_cache_cf.cc \
tests/stub_helper.cc \
cbdata.cc \
$(SBUF_SOURCE) \
SBufDetailedStats.h \
tests/stub_SBufDetailedStats.cc \
String.cc \
tests/stub_debug.cc \
tests/stub_client_side_request.cc \
tests/stub_http.cc \
tests/stub_libauth.cc \
mem_node.cc \
@@ -3210,40 +3195,41 @@
refresh.h \
refresh.cc \
tests/stub_store_client.cc \
tools.h \
tests/stub_tools.cc \
tests/testStoreSupport.cc \
tests/testStoreSupport.h \
time.cc \
wordlist.h \
wordlist.cc \
$(DISKIO_SOURCE)
nodist_tests_testUfs_SOURCES = \
$(TESTSOURCES) \
$(DISKIO_GEN_SOURCE) \
SquidMath.cc \
SquidMath.h \
swap_log_op.cc
tests_testUfs_LDADD = \
http/libsquid-http.la \
+ parser/libsquid-parser.la \
CommCalls.o \
DnsLookupDetails.o \
ident/libident.la \
acl/libacls.la \
acl/libstate.la \
acl/libapi.la \
libsquid.la \
ip/libip.la \
fs/libfs.la \
mgr/libmgr.la \
$(REPL_OBJS) \
acl/libacls.la \
$(DISK_LIBS) \
$(DISK_OS_LIBS) \
acl/libapi.la \
$(SSL_LIBS) \
ipc/libipc.la \
comm/libcomm.la \
anyp/libanyp.la \
base/libbase.la \
@@ -3294,41 +3280,40 @@
fd.cc \
fde.h \
fde.cc \
FileMap.h \
filemap.cc \
HttpHeaderFieldStat.h \
HttpBody.h \
HttpBody.cc \
HttpHdrCc.cc \
HttpHdrContRange.cc \
HttpHdrRange.cc \
HttpHdrSc.cc \
HttpHdrScTarget.cc \
HttpHeader.h \
HttpHeader.cc \
HttpHeaderFieldInfo.h \
HttpHeaderTools.h \
HttpHeaderTools.cc \
HttpMsg.cc \
HttpReply.cc \
- HttpRequestMethod.cc \
int.h \
int.cc \
SquidList.h \
SquidList.cc \
MasterXaction.cc \
MasterXaction.h \
Mem.h \
mem.cc \
MemBuf.cc \
MemObject.cc \
mem_node.cc \
Notes.h \
Notes.cc \
Packer.cc \
Parsing.cc \
RemovalPolicy.cc \
RequestFlags.cc \
RequestFlags.h \
StatCounters.h \
StatCounters.cc \
@@ -3395,40 +3380,41 @@
store_rebuild.h \
tests/stub_store_rebuild.cc \
tests/stub_store_stats.cc \
tools.h \
tests/stub_tools.cc \
time.cc \
url.cc \
wordlist.h \
wordlist.cc \
$(DELAY_POOL_SOURCE) \
$(DISKIO_SOURCE) \
$(UNLINKDSOURCE)
nodist_tests_testRock_SOURCES = \
$(DISKIO_GEN_SOURCE) \
swap_log_op.cc \
SquidMath.cc \
SquidMath.h \
$(TESTSOURCES)
tests_testRock_LDADD = \
http/libsquid-http.la \
+ parser/libsquid-parser.la \
libsquid.la \
comm/libcomm.la \
ip/libip.la \
fs/libfs.la \
$(COMMON_LIBS) \
$(REPL_OBJS) \
$(DISK_LIBS) \
$(DISK_OS_LIBS) \
acl/libacls.la \
acl/libapi.la \
acl/libstate.la \
anyp/libanyp.la \
eui/libeui.la \
$(SSL_LIBS) \
ipc/libipc.la \
base/libbase.la \
$(top_builddir)/lib/libmisccontainers.la \
$(top_builddir)/lib/libmiscencoding.la \
$(top_builddir)/lib/libmiscutil.la \
$(NETTLELIB) \
@@ -3503,49 +3489,46 @@
helper.cc \
hier_code.h \
$(HTCPSOURCE) \
http.cc \
HttpBody.h \
HttpBody.cc \
HttpHeaderFieldStat.h \
HttpHdrCc.h \
HttpHdrCc.cc \
HttpHdrCc.cci \
HttpHdrContRange.cc \
HttpHdrRange.cc \
HttpHdrSc.cc \
HttpHdrScTarget.cc \
HttpHeader.h \
HttpHeader.cc \
HttpHeaderFieldInfo.h \
HttpHeaderTools.h \
HttpHeaderTools.cc \
HttpMsg.cc \
- HttpParser.cc \
- HttpParser.h \
HttpReply.cc \
PeerPoolMgr.h \
PeerPoolMgr.cc \
RequestFlags.h \
RequestFlags.cc \
HttpRequest.cc \
- HttpRequestMethod.cc \
icp_v2.cc \
icp_v3.cc \
$(IPC_SOURCE) \
ipcache.cc \
int.h \
int.cc \
internal.h \
internal.cc \
SquidList.h \
SquidList.cc \
MasterXaction.cc \
MasterXaction.h \
multicast.h \
multicast.cc \
Mem.h \
tests/stub_mem.cc \
mem_node.cc \
MemBuf.cc \
MemObject.cc \
mime.h \
@@ -3634,47 +3617,47 @@
url.cc \
urn.h \
urn.cc \
wccp2.h \
tests/stub_wccp2.cc \
whois.h \
tests/stub_whois.cc \
FadingCounter.cc \
$(WIN32_SOURCE) \
wordlist.h \
wordlist.cc
nodist_tests_testURL_SOURCES = \
$(BUILT_SOURCES)
tests_testURL_LDADD = \
libsquid.la \
clients/libclients.la \
servers/libservers.la \
helper/libhelper.la \
http/libsquid-http.la \
ftp/libftp.la \
+ parser/libsquid-parser.la \
anyp/libanyp.la \
ident/libident.la \
acl/libacls.la \
eui/libeui.la \
acl/libstate.la \
acl/libapi.la \
- parser/libsquid-parser.la \
base/libbase.la \
ip/libip.la \
fs/libfs.la \
$(SSL_LIBS) \
ipc/libipc.la \
mgr/libmgr.la \
$(SNMP_LIBS) \
icmp/libicmp.la icmp/libicmp-core.la \
comm/libcomm.la \
log/liblog.la \
$(DISK_OS_LIBS) \
format/libformat.la \
$(REGEXLIB) \
$(REPL_OBJS) \
$(ADAPTATION_LIBS) \
$(ESI_LIBS) \
$(top_builddir)/lib/libmisccontainers.la \
$(top_builddir)/lib/libmiscencoding.la \
$(top_builddir)/lib/libmiscutil.la \
$(NETTLELIB) \
=== modified file 'src/MemObject.h'
--- src/MemObject.h 2014-09-13 13:59:43 +0000
+++ src/MemObject.h 2014-09-14 12:23:03 +0000
@@ -1,34 +1,34 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
#ifndef SQUID_MEMOBJECT_H
#define SQUID_MEMOBJECT_H
#include "CommRead.h"
#include "dlink.h"
-#include "HttpRequestMethod.h"
+#include "http/RequestMethod.h"
#include "RemovalPolicy.h"
#include "stmem.h"
#include "StoreIOBuffer.h"
#include "StoreIOState.h"
#if USE_DELAY_POOLS
#include "DelayId.h"
#endif
typedef void STMCB (void *data, StoreIOBuffer wroteBuffer);
class store_client;
class HttpRequest;
class HttpReply;
class MemObject
{
public:
static size_t inUseCount();
=== modified file 'src/Store.h'
--- src/Store.h 2014-09-13 13:59:43 +0000
+++ src/Store.h 2014-09-14 12:23:03 +0000
@@ -1,42 +1,43 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
#ifndef SQUID_STORE_H
#define SQUID_STORE_H
/**
\defgroup StoreAPI Store API
\ingroup FileSystems
*/
#include "base/RefCount.h"
#include "comm/forward.h"
#include "CommRead.h"
#include "hash.h"
+#include "http/forward.h"
+#include "http/RequestMethod.h"
#include "HttpReply.h"
-#include "HttpRequestMethod.h"
#include "MemObject.h"
#include "Range.h"
#include "RemovalPolicy.h"
#include "StoreIOBuffer.h"
#include "StoreStats.h"
#if USE_SQUID_ESI
#include "esi/Element.h"
#endif
#include <ostream>
class AsyncCall;
class HttpRequest;
class Packer;
class RequestFlags;
class StoreClient;
class StoreSearch;
class SwapDir;
=== modified file 'src/acl/Method.h'
--- src/acl/Method.h 2014-09-13 13:59:43 +0000
+++ src/acl/Method.h 2014-09-14 12:23:03 +0000
@@ -1,34 +1,34 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
#ifndef SQUID_ACLMETHOD_H
#define SQUID_ACLMETHOD_H
#include "acl/Strategised.h"
#include "acl/Strategy.h"
-#include "HttpRequestMethod.h"
+#include "http/RequestMethod.h"
/// \ingroup ACLAPI
class ACLMethodStrategy : public ACLStrategy<HttpRequestMethod>
{
public:
virtual int match (ACLData<MatchType> * &, ACLFilledChecklist *, ACLFlags &);
virtual bool requiresRequest() const {return true;}
static ACLMethodStrategy *Instance();
/**
* Not implemented to prevent copies of the instance.
\par
* Not private to prevent brain dead g+++ warnings about
* private constructors with no friends
*/
ACLMethodStrategy(ACLMethodStrategy const &);
private:
=== modified file 'src/acl/MethodData.cc'
--- src/acl/MethodData.cc 2014-09-13 13:59:43 +0000
+++ src/acl/MethodData.cc 2014-10-15 06:21:21 +0000
@@ -1,35 +1,35 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
/* DEBUG: section 28 Access Control */
#include "squid.h"
#include "acl/Checklist.h"
#include "acl/MethodData.h"
#include "cache_cf.h"
-#include "HttpRequestMethod.h"
+#include "http/RequestMethod.h"
int ACLMethodData::ThePurgeCount = 0;
ACLMethodData::ACLMethodData() : values (NULL)
{}
ACLMethodData::ACLMethodData(ACLMethodData const &old) : values (NULL)
{
assert (!old.values);
}
ACLMethodData::~ACLMethodData()
{
if (values)
delete values;
}
/// todo make this a pass-by-reference now that HTTPRequestMethods a full class?
bool
ACLMethodData::match(HttpRequestMethod toFind)
@@ -48,40 +48,42 @@
{
SBufList sl;
CbDataList<HttpRequestMethod> *data = values;
while (data != NULL) {
sl.push_back(data->element.image());
data = data->next;
}
return sl;
}
void
ACLMethodData::parse()
{
CbDataList<HttpRequestMethod> **Tail;
char *t = NULL;
for (Tail = &values; *Tail; Tail = &((*Tail)->next));
while ((t = strtokFile())) {
- CbDataList<HttpRequestMethod> *q = new CbDataList<HttpRequestMethod> (HttpRequestMethod(t, NULL));
+ HttpRequestMethod m;
+ m.HttpRequestMethodXXX(t);
+ CbDataList<HttpRequestMethod> *q = new CbDataList<HttpRequestMethod>(m);
if (q->element == Http::METHOD_PURGE)
++ThePurgeCount; // configuration code wants to know
*(Tail) = q;
Tail = &q->next;
}
}
bool
ACLMethodData::empty() const
{
return values == NULL;
}
ACLData<HttpRequestMethod> *
ACLMethodData::clone() const
{
assert (!values);
return new ACLMethodData(*this);
}
=== modified file 'src/acl/MethodData.h'
--- src/acl/MethodData.h 2014-09-13 13:59:43 +0000
+++ src/acl/MethodData.h 2014-09-14 12:23:03 +0000
@@ -1,35 +1,35 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
#ifndef SQUID_ACLMETHODDATA_H
#define SQUID_ACLMETHODDATA_H
#include "acl/Acl.h"
#include "acl/Data.h"
#include "CbDataList.h"
-#include "HttpRequestMethod.h"
+#include "http/RequestMethod.h"
/// \ingroup ACLAPI
class ACLMethodData : public ACLData<HttpRequestMethod>
{
public:
MEMPROXY_CLASS(ACLMethodData);
ACLMethodData();
ACLMethodData(ACLMethodData const &);
ACLMethodData &operator= (ACLMethodData const &);
virtual ~ACLMethodData();
bool match(HttpRequestMethod);
virtual SBufList dump() const;
void parse();
bool empty() const;
virtual ACLData<HttpRequestMethod> *clone() const;
CbDataList<HttpRequestMethod> *values;
=== modified file 'src/cache_cf.cc'
--- src/cache_cf.cc 2014-09-13 13:59:43 +0000
+++ src/cache_cf.cc 2014-09-14 12:23:03 +0000
@@ -16,41 +16,40 @@
#include "acl/AclSizeLimit.h"
#include "acl/Gadgets.h"
#include "acl/MethodData.h"
#include "acl/Tree.h"
#include "anyp/PortCfg.h"
#include "anyp/UriScheme.h"
#include "AuthReg.h"
#include "base/RunnersRegistry.h"
#include "cache_cf.h"
#include "CachePeer.h"
#include "CachePeerDomainList.h"
#include "ConfigParser.h"
#include "CpuAffinityMap.h"
#include "DiskIO/DiskIOModule.h"
#include "eui/Config.h"
#include "ExternalACL.h"
#include "format/Format.h"
#include "ftp/Elements.h"
#include "globals.h"
#include "HttpHeaderTools.h"
-#include "HttpRequestMethod.h"
#include "ident/Config.h"
#include "ip/Intercept.h"
#include "ip/QosConfig.h"
#include "ip/tools.h"
#include "ipc/Kids.h"
#include "log/Config.h"
#include "log/CustomLog.h"
#include "Mem.h"
#include "MemBuf.h"
#include "mgr/ActionPasswordList.h"
#include "mgr/Registration.h"
#include "neighbors.h"
#include "NeighborTypeDomainList.h"
#include "Parsing.h"
#include "pconn.h"
#include "PeerDigest.h"
#include "PeerPoolMgr.h"
#include "RefreshPattern.h"
#include "rfc1738.h"
#include "SBufList.h"
=== modified file 'src/client_side.cc'
--- src/client_side.cc 2014-10-07 14:11:12 +0000
+++ src/client_side.cc 2014-10-15 14:07:41 +0000
@@ -69,40 +69,41 @@
#include "client_side_reply.h"
#include "client_side_request.h"
#include "ClientRequestContext.h"
#include "clientStream.h"
#include "comm.h"
#include "comm/Connection.h"
#include "comm/Loops.h"
#include "comm/Read.h"
#include "comm/TcpAcceptor.h"
#include "comm/Write.h"
#include "CommCalls.h"
#include "errorpage.h"
#include "fd.h"
#include "fde.h"
#include "fqdncache.h"
#include "FwdState.h"
#include "globals.h"
#include "helper.h"
#include "helper/Reply.h"
#include "http.h"
+#include "http/one/RequestParser.h"
#include "HttpHdrContRange.h"
#include "HttpHeaderTools.h"
#include "HttpReply.h"
#include "HttpRequest.h"
#include "ident/Config.h"
#include "ident/Ident.h"
#include "internal.h"
#include "ipc/FdNotes.h"
#include "ipc/StartListening.h"
#include "log/access_log.h"
#include "Mem.h"
#include "MemBuf.h"
#include "MemObject.h"
#include "mime_header.h"
#include "parser/Tokenizer.h"
#include "profiler/Profiler.h"
#include "rfc1738.h"
#include "servers/forward.h"
#include "SquidConfig.h"
#include "SquidTime.h"
@@ -181,41 +182,40 @@
#endif
static CTCB clientLifetimeTimeout;
#if USE_IDENT
static IDCB clientIdentDone;
#endif
static int clientIsContentLengthValid(HttpRequest * r);
static int clientIsRequestBodyTooLargeForPolicy(int64_t bodyLength);
static void clientUpdateStatHistCounters(LogTags logType, int svc_time);
static void clientUpdateStatCounters(LogTags logType);
static void clientUpdateHierCounters(HierarchyLogEntry *);
static bool clientPingHasFinished(ping_data const *aPing);
void prepareLogWithRequestDetails(HttpRequest *, AccessLogEntry::Pointer &);
#ifndef PURIFY
static bool connIsUsable(ConnStateData * conn);
#endif
static void ClientSocketContextPushDeferredIfNeeded(ClientSocketContext::Pointer deferredRequest, ConnStateData * conn);
static void clientUpdateSocketStats(LogTags logType, size_t size);
char *skipLeadingSpace(char *aString);
-static void connNoteUseOfBuffer(ConnStateData* conn, size_t byteCount);
clientStreamNode *
ClientSocketContext::getTail() const
{
if (http->client_stream.tail)
return (clientStreamNode *)http->client_stream.tail->data;
return NULL;
}
clientStreamNode *
ClientSocketContext::getClientReplyContext() const
{
return (clientStreamNode *)http->client_stream.tail->prev->data;
}
ConnStateData *
ClientSocketContext::getConn() const
{
return http->getConn();
@@ -1979,359 +1979,307 @@
char *tmp_uri = static_cast<char*>(xmalloc(strlen(uri) + 1));
char *q = tmp_uri;
t = uri;
while (*t) {
if (!xisspace(*t)) {
*q = *t;
++q;
}
++t;
}
*q = '\0';
http->log_uri = xstrndup(rfc1738_escape_unescaped(tmp_uri), MAX_URL);
xfree(tmp_uri);
}
break;
}
}
}
static void
-prepareAcceleratedURL(ConnStateData * conn, ClientHttpRequest *http, char *url, const char *req_hdr)
+prepareAcceleratedURL(ConnStateData * conn, ClientHttpRequest *http, const Http1::RequestParserPointer &hp)
{
int vhost = conn->port->vhost;
int vport = conn->port->vport;
- char *host;
- char ipbuf[MAX_IPSTRLEN];
+ static char ipbuf[MAX_IPSTRLEN];
http->flags.accel = true;
/* BUG: Squid cannot deal with '*' URLs (RFC2616 5.1.2) */
- if (strncasecmp(url, "cache_object://", 15) == 0)
+ static const SBuf cache_object("cache_object://");
+ if (hp->requestUri().startsWith(cache_object))
return; /* already in good shape */
- if (*url != '/') {
+ // XXX: re-use proper URL parser for this
+ SBuf url = hp->requestUri(); // use full provided URI if we abort
+ do { // use a loop so we can break out of it
+ ::Parser::Tokenizer tok(url);
+ if (tok.skip('/')) // origin-form URL already.
+ break;
+
if (conn->port->vhost)
return; /* already in good shape */
- /* else we need to ignore the host name */
- url = strstr(url, "//");
+ // skip the URI scheme
+ static const CharacterSet uriScheme = CharacterSet("URI-scheme","+-.") + CharacterSet::ALPHA + CharacterSet::DIGIT;
+ static const SBuf uriSchemeEnd("://");
+ if (!tok.skipAll(uriScheme) || !tok.skip(uriSchemeEnd))
+ break;
-#if SHOULD_REJECT_UNKNOWN_URLS
+ // skip the authority segment
+ // RFC 3986 complex nested ABNF for "authority" boils down to this:
+ static const CharacterSet authority = CharacterSet("authority","-._~%:@[]!$&'()*+,;=") +
+ CharacterSet::HEXDIG + CharacterSet::ALPHA + CharacterSet::DIGIT;
+ if (!tok.skipAll(authority))
+ break;
- if (!url) {
- hp->request_parse_status = Http::scBadRequest;
- return conn->abortRequestParsing("error:invalid-request");
- }
-#endif
+ static const SBuf slashUri("/");
+ const SBuf t = tok.remaining();
+ if (t.isEmpty())
+ url = slashUri;
+ else if (t[0]=='/') // looks like path
+ url = t;
+ else if (t[0]=='?' || t[0]=='#') { // looks like query or fragment. fix '/'
+ url = slashUri;
+ url.append(t);
+ } // else do nothing. invalid path
- if (url)
- url = strchr(url + 2, '/');
+ } while(false);
- if (!url)
- url = (char *) "/";
+#if SHOULD_REJECT_UNKNOWN_URLS
+ // reject URI which are not well-formed even after the processing above
+ if (url.isEmpty() || url[0] != '/') {
+ hp->request_parse_status = Http::scBadRequest;
+ return conn->abortRequestParsing("error:invalid-request");
}
+#endif
if (vport < 0)
vport = http->getConn()->clientConnection->local.port();
const bool switchedToHttps = conn->switchedToHttps();
const bool tryHostHeader = vhost || switchedToHttps;
- if (tryHostHeader && (host = mime_get_header(req_hdr, "Host")) != NULL) {
+ char *host = NULL;
+ if (tryHostHeader && (host = hp->getHeaderField("Host"))) {
debugs(33, 5, "ACCEL VHOST REWRITE: vhost=" << host << " + vport=" << vport);
char thost[256];
if (vport > 0) {
thost[0] = '\0';
char *t = NULL;
if (host[strlen(host)] != ']' && (t = strrchr(host,':')) != NULL) {
strncpy(thost, host, (t-host));
snprintf(thost+(t-host), sizeof(thost)-(t-host), ":%d", vport);
host = thost;
} else if (!t) {
snprintf(thost, sizeof(thost), "%s:%d",host, vport);
host = thost;
}
} // else nothing to alter port-wise.
- int url_sz = strlen(url) + 32 + Config.appendDomainLen +
- strlen(host);
+ const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen + strlen(host);
http->uri = (char *)xcalloc(url_sz, 1);
const char *protocol = switchedToHttps ?
"https" : AnyP::UriScheme(conn->port->transport.protocol).c_str();
- snprintf(http->uri, url_sz, "%s://%s%s", protocol, host, url);
- debugs(33, 5, "ACCEL VHOST REWRITE: '" << http->uri << "'");
+ snprintf(http->uri, url_sz, "%s://%s" SQUIDSBUFPH, protocol, host, SQUIDSBUFPRINT(url));
+ debugs(33, 5, "ACCEL VHOST REWRITE: " << http->uri);
} else if (conn->port->defaultsite /* && !vhost */) {
debugs(33, 5, "ACCEL DEFAULTSITE REWRITE: defaultsite=" << conn->port->defaultsite << " + vport=" << vport);
- int url_sz = strlen(url) + 32 + Config.appendDomainLen +
+ const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen +
strlen(conn->port->defaultsite);
http->uri = (char *)xcalloc(url_sz, 1);
char vportStr[32];
vportStr[0] = '\0';
if (vport > 0) {
snprintf(vportStr, sizeof(vportStr),":%d",vport);
}
- snprintf(http->uri, url_sz, "%s://%s%s%s",
- AnyP::UriScheme(conn->port->transport.protocol).c_str(), conn->port->defaultsite, vportStr, url);
- debugs(33, 5, "ACCEL DEFAULTSITE REWRITE: '" << http->uri <<"'");
+ snprintf(http->uri, url_sz, "%s://%s%s" SQUIDSBUFPH,
+ AnyP::UriScheme(conn->port->transport.protocol).c_str(), conn->port->defaultsite, vportStr, SQUIDSBUFPRINT(url));
+ debugs(33, 5, "ACCEL DEFAULTSITE REWRITE: " << http->uri);
} else if (vport > 0 /* && (!vhost || no Host:) */) {
debugs(33, 5, "ACCEL VPORT REWRITE: *_port IP + vport=" << vport);
/* Put the local socket IP address as the hostname, with whatever vport we found */
- int url_sz = strlen(url) + 32 + Config.appendDomainLen;
+ const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen;
http->uri = (char *)xcalloc(url_sz, 1);
http->getConn()->clientConnection->local.toHostStr(ipbuf,MAX_IPSTRLEN);
- snprintf(http->uri, url_sz, "%s://%s:%d%s",
+ snprintf(http->uri, url_sz, "%s://%s:%d" SQUIDSBUFPH,
AnyP::UriScheme(conn->port->transport.protocol).c_str(),
- ipbuf, vport, url);
- debugs(33, 5, "ACCEL VPORT REWRITE: '" << http->uri << "'");
+ ipbuf, vport, SQUIDSBUFPRINT(url));
+ debugs(33, 5, "ACCEL VPORT REWRITE: " << http->uri);
}
}
static void
-prepareTransparentURL(ConnStateData * conn, ClientHttpRequest *http, char *url, const char *req_hdr)
+prepareTransparentURL(ConnStateData * conn, ClientHttpRequest *http, const Http1::RequestParserPointer &hp)
{
- char *host;
- char ipbuf[MAX_IPSTRLEN];
-
- if (*url != '/')
+ // TODO Must() on URI !empty when the parser supports throw. For now avoid assert().
+ if (!hp->requestUri().isEmpty() && hp->requestUri()[0] != '/')
return; /* already in good shape */
/* BUG: Squid cannot deal with '*' URLs (RFC2616 5.1.2) */
- if ((host = mime_get_header(req_hdr, "Host")) != NULL) {
- int url_sz = strlen(url) + 32 + Config.appendDomainLen +
+ if (const char *host = hp->getHeaderField("Host")) {
+ const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen +
strlen(host);
http->uri = (char *)xcalloc(url_sz, 1);
- snprintf(http->uri, url_sz, "%s://%s%s", AnyP::UriScheme(conn->port->transport.protocol).c_str(), host, url);
- debugs(33, 5, "TRANSPARENT HOST REWRITE: '" << http->uri <<"'");
+ snprintf(http->uri, url_sz, "%s://%s" SQUIDSBUFPH,
+ AnyP::UriScheme(conn->port->transport.protocol).c_str(), host, SQUIDSBUFPRINT(hp->requestUri()));
+ debugs(33, 5, "TRANSPARENT HOST REWRITE: " << http->uri);
} else {
/* Put the local socket IP address as the hostname. */
- int url_sz = strlen(url) + 32 + Config.appendDomainLen;
+ const int url_sz = hp->requestUri().length() + 32 + Config.appendDomainLen;
http->uri = (char *)xcalloc(url_sz, 1);
+ static char ipbuf[MAX_IPSTRLEN];
http->getConn()->clientConnection->local.toHostStr(ipbuf,MAX_IPSTRLEN);
- snprintf(http->uri, url_sz, "%s://%s:%d%s",
+ snprintf(http->uri, url_sz, "%s://%s:%d" SQUIDSBUFPH,
AnyP::UriScheme(http->getConn()->port->transport.protocol).c_str(),
- ipbuf, http->getConn()->clientConnection->local.port(), url);
- debugs(33, 5, "TRANSPARENT REWRITE: '" << http->uri << "'");
+ ipbuf, http->getConn()->clientConnection->local.port(), SQUIDSBUFPRINT(hp->requestUri()));
+ debugs(33, 5, "TRANSPARENT REWRITE: " << http->uri);
}
}
/** Parse an HTTP request
*
* \note Sets result->flags.parsed_ok to 0 if failed to parse the request,
* to 1 if the request was correctly parsed.
* \param[in] csd a ConnStateData. The caller must make sure it is not null
- * \param[in] hp an HttpParser
+ * \param[in] hp an Http1::RequestParser
* \param[out] mehtod_p will be set as a side-effect of the parsing.
* Pointed-to value will be set to Http::METHOD_NONE in case of
* parsing failure
* \param[out] http_ver will be set as a side-effect of the parsing
* \return NULL on incomplete requests,
* a ClientSocketContext structure on success or failure.
*/
ClientSocketContext *
-parseHttpRequest(ConnStateData *csd, HttpParser *hp, HttpRequestMethod * method_p, Http::ProtocolVersion *http_ver)
+parseHttpRequest(ConnStateData *csd, const Http1::RequestParserPointer &hp)
{
- char *req_hdr = NULL;
- char *end;
- size_t req_sz;
- ClientHttpRequest *http;
- ClientSocketContext *result;
- StoreIOBuffer tempBuffer;
- int r;
-
- /* pre-set these values to make aborting simpler */
- *method_p = Http::METHOD_NONE;
-
- /* NP: don't be tempted to move this down or remove again.
- * It's the only DDoS protection old-String has against long URL */
- if ( hp->bufsiz <= 0) {
- debugs(33, 5, "Incomplete request, waiting for end of request line");
- return NULL;
- } else if ( (size_t)hp->bufsiz >= Config.maxRequestHeaderSize && headersEnd(hp->buf, Config.maxRequestHeaderSize) == 0) {
- debugs(33, 5, "parseHttpRequest: Too large request");
- hp->request_parse_status = Http::scHeaderTooLarge;
- return csd->abortRequestParsing("error:request-too-large");
- }
-
- /* Attempt to parse the first line; this'll define the method, url, version and header begin */
- r = HttpParserParseReqLine(hp);
-
- if (r == 0) {
- debugs(33, 5, "Incomplete request, waiting for end of request line");
- return NULL;
- }
-
- if (r == -1) {
- return csd->abortRequestParsing("error:invalid-request");
- }
+ /* Attempt to parse the first line; this will define where the method, url, version and header begin */
+ {
+ const bool parsedOk = hp->parse(csd->in.buf);
- /* Request line is valid here .. */
- *http_ver = Http::ProtocolVersion(hp->req.v_maj, hp->req.v_min);
+ // sync the buffers after parsing.
+ csd->in.buf = hp->remaining();
- /* This call scans the entire request, not just the headers */
- if (hp->req.v_maj > 0) {
- if ((req_sz = headersEnd(hp->buf, hp->bufsiz)) == 0) {
- debugs(33, 5, "Incomplete request, waiting for end of headers");
+ if (hp->needsMoreData()) {
+ debugs(33, 5, "Incomplete request, waiting for end of request line");
return NULL;
}
- } else {
- debugs(33, 3, "parseHttpRequest: Missing HTTP identifier");
- req_sz = HttpParserReqSz(hp);
- }
-
- /* We know the whole request is in hp->buf now */
-
- assert(req_sz <= (size_t) hp->bufsiz);
- /* Will the following be true with HTTP/0.9 requests? probably not .. */
- /* So the rest of the code will need to deal with '0'-byte headers (ie, none, so don't try parsing em) */
- assert(req_sz > 0);
+ if (!parsedOk) {
+ if (hp->request_parse_status == Http::scRequestHeaderFieldsTooLarge || hp->request_parse_status == Http::scUriTooLong)
+ return csd->abortRequestParsing("error:request-too-large");
- hp->hdr_end = req_sz - 1;
-
- hp->hdr_start = hp->req.end + 1;
-
- /* Enforce max_request_size */
- if (req_sz >= Config.maxRequestHeaderSize) {
- debugs(33, 5, "parseHttpRequest: Too large request");
- hp->request_parse_status = Http::scHeaderTooLarge;
- return csd->abortRequestParsing("error:request-too-large");
+ return csd->abortRequestParsing("error:invalid-request");
+ }
}
- /* Set method_p */
- *method_p = HttpRequestMethod(&hp->buf[hp->req.m_start], &hp->buf[hp->req.m_end]+1);
+ /* We know the whole request is in parser now */
+ debugs(11, 2, "HTTP Client " << csd->clientConnection);
+ debugs(11, 2, "HTTP Client REQUEST:\n---------\n" <<
+ hp->method() << " " << hp->requestUri() << " " << hp->messageProtocol() << "\n" <<
+ hp->mimeHeader() <<
+ "\n----------");
/* deny CONNECT via accelerated ports */
- if (*method_p == Http::METHOD_CONNECT && csd->port != NULL && csd->port->flags.accelSurrogate) {
+ if (hp->method() == Http::METHOD_CONNECT && csd->port != NULL && csd->port->flags.accelSurrogate) {
debugs(33, DBG_IMPORTANT, "WARNING: CONNECT method received on " << csd->port->transport.protocol << " Accelerator port " << csd->port->s.port());
- /* XXX need a way to say "this many character length string" */
- debugs(33, DBG_IMPORTANT, "WARNING: for request: " << hp->buf);
+ debugs(33, DBG_IMPORTANT, "WARNING: for request: " << hp->method() << " " << hp->requestUri() << " " << hp->messageProtocol());
hp->request_parse_status = Http::scMethodNotAllowed;
return csd->abortRequestParsing("error:method-not-allowed");
}
- if (*method_p == Http::METHOD_NONE) {
- /* XXX need a way to say "this many character length string" */
- debugs(33, DBG_IMPORTANT, "clientParseRequestMethod: Unsupported method in request '" << hp->buf << "'");
+ if (hp->method() == Http::METHOD_NONE) {
+ debugs(33, DBG_IMPORTANT, "WARNING: Unsupported method: " << hp->method() << " " << hp->requestUri() << " " << hp->messageProtocol());
hp->request_parse_status = Http::scMethodNotAllowed;
return csd->abortRequestParsing("error:unsupported-request-method");
}
- /*
- * Process headers after request line
- * TODO: Use httpRequestParse here.
- */
- /* XXX this code should be modified to take a const char * later! */
- req_hdr = (char *) hp->buf + hp->req.end + 1;
-
- debugs(33, 3, "parseHttpRequest: req_hdr = {" << req_hdr << "}");
-
- end = (char *) hp->buf + hp->hdr_end;
-
- debugs(33, 3, "parseHttpRequest: end = {" << end << "}");
-
- debugs(33, 3, "parseHttpRequest: prefix_sz = " <<
- (int) HttpParserRequestLen(hp) << ", req_line_sz = " <<
- HttpParserReqSz(hp));
+ // Process headers after request line
+ debugs(33, 3, "complete request received. " <<
+ "prefix_sz = " << hp->messageHeaderSize() <<
+ ", request-line-size=" << hp->firstLineSize() <<
+ ", mime-header-size=" << hp->headerBlockSize() <<
+ ", mime header block:\n" << hp->mimeHeader() << "\n----------");
/* Ok, all headers are received */
- http = new ClientHttpRequest(csd);
+ ClientHttpRequest *http = new ClientHttpRequest(csd);
- http->req_sz = HttpParserRequestLen(hp);
- result = new ClientSocketContext(csd->clientConnection, http);
+ http->req_sz = hp->messageHeaderSize();
+ ClientSocketContext *result = new ClientSocketContext(csd->clientConnection, http);
+
+ StoreIOBuffer tempBuffer;
tempBuffer.data = result->reqbuf;
tempBuffer.length = HTTP_REQBUF_SZ;
ClientStreamData newServer = new clientReplyContext(http);
ClientStreamData newClient = result;
clientStreamInit(&http->client_stream, clientGetMoreData, clientReplyDetach,
clientReplyStatus, newServer, clientSocketRecipient,
clientSocketDetach, newClient, tempBuffer);
- debugs(33, 5, "parseHttpRequest: Request Header is\n" <<(hp->buf) + hp->hdr_start);
-
/* set url */
- /*
- * XXX this should eventually not use a malloc'ed buffer; the transformation code
- * below needs to be modified to not expect a mutable nul-terminated string.
- */
- char *url = (char *)xmalloc(hp->req.u_end - hp->req.u_start + 16);
-
- memcpy(url, hp->buf + hp->req.u_start, hp->req.u_end - hp->req.u_start + 1);
-
- url[hp->req.u_end - hp->req.u_start + 1] = '\0';
-
-#if THIS_VIOLATES_HTTP_SPECS_ON_URL_TRANSFORMATION
-
- if ((t = strchr(url, '#'))) /* remove HTML anchors */
- *t = '\0';
-
-#endif
+ // XXX: c_str() does re-allocate but here replaces explicit malloc/free.
+ // when internalCheck() accepts SBuf removing this will be a net gain for performance.
+ SBuf tmp(hp->requestUri());
+ const char *url = tmp.c_str();
debugs(33,5, HERE << "repare absolute URL from " <<
(csd->transparent()?"intercept":(csd->port->flags.accelSurrogate ? "accel":"")));
/* Rewrite the URL in transparent or accelerator mode */
/* NP: there are several cases to traverse here:
* - standard mode (forward proxy)
* - transparent mode (TPROXY)
* - transparent mode with failures
* - intercept mode (NAT)
* - intercept mode with failures
* - accelerator mode (reverse proxy)
* - internal URL
* - mixed combos of the above with internal URL
* - remote interception with PROXY protocol
* - remote reverse-proxy with PROXY protocol
*/
if (csd->transparent()) {
/* intercept or transparent mode, properly working with no failures */
- prepareTransparentURL(csd, http, url, req_hdr);
+ prepareTransparentURL(csd, http, hp);
} else if (internalCheck(url)) {
/* internal URL mode */
/* prepend our name & port */
http->uri = xstrdup(internalLocalUri(NULL, url));
// We just re-wrote the URL. Must replace the Host: header.
// But have not parsed there yet!! flag for local-only handling.
http->flags.internal = true;
} else if (csd->port->flags.accelSurrogate || csd->switchedToHttps()) {
/* accelerator mode */
- prepareAcceleratedURL(csd, http, url, req_hdr);
+ prepareAcceleratedURL(csd, http, hp);
}
if (!http->uri) {
/* No special rewrites have been applied above, use the
* requested url. may be rewritten later, so make extra room */
- int url_sz = strlen(url) + Config.appendDomainLen + 5;
+ int url_sz = hp->requestUri().length() + Config.appendDomainLen + 5;
http->uri = (char *)xcalloc(url_sz, 1);
strcpy(http->uri, url);
}
- debugs(33, 5, "parseHttpRequest: Complete request received");
-
- // XXX: crop this dump at the end of headers. No need for extras
- debugs(11, 2, "HTTP Client " << csd->clientConnection);
- debugs(11, 2, "HTTP Client REQUEST:\n---------\n" << (hp->buf) + hp->req.m_start << "\n----------");
-
result->flags.parsed_ok = 1;
- xfree(url);
return result;
}
bool
ConnStateData::In::maybeMakeSpaceAvailable()
{
if (buf.spaceSize() < 2) {
const SBuf::size_type haveCapacity = buf.length() + buf.spaceSize();
if (haveCapacity >= Config.maxRequestBufferSize) {
debugs(33, 4, "request buffer full: client_request_buffer_max_size=" << Config.maxRequestBufferSize);
return false;
}
if (haveCapacity == 0) {
// haveCapacity is based on the SBuf visible window of the MemBlob buffer, which may fill up.
// at which point bump the buffer back to default. This allocates a new MemBlob with any un-parsed bytes.
buf.reserveCapacity(CLIENT_REQ_BUF_SZ);
} else {
const SBuf::size_type wantCapacity = min(static_cast<SBuf::size_type>(Config.maxRequestBufferSize), haveCapacity*2);
buf.reserveCapacity(wantCapacity);
}
@@ -2373,47 +2321,40 @@
return 1;
} else if (!Config.onoff.half_closed_clients) {
/* admin doesn't want to support half-closed client sockets */
debugs(33, 3, HERE << clientConnection << " aborted (half_closed_clients disabled)");
notifyAllContexts(0); // no specific error implies abort
return 1;
}
}
return 0;
}
void
ConnStateData::consumeInput(const size_t byteCount)
{
assert(byteCount > 0 && byteCount <= in.buf.length());
in.buf.consume(byteCount);
debugs(33, 5, "in.buf has " << in.buf.length() << " unused bytes");
}
-// TODO: Remove when renaming ConnStateData
-void
-connNoteUseOfBuffer(ConnStateData* conn, size_t byteCount)
-{
- conn->consumeInput(byteCount);
-}
-
void
ConnStateData::clientAfterReadingRequests()
{
// Were we expecting to read more request body from half-closed connection?
if (mayNeedToReadMoreBody() && commIsHalfClosed(clientConnection->fd)) {
debugs(33, 3, HERE << "truncated body: closing half-closed " << clientConnection);
clientConnection->close();
return;
}
if (flags.readMore)
readSomeData();
}
void
ConnStateData::quitAfterError(HttpRequest *request)
{
// From HTTP p.o.v., we do not have to close after every error detected
// at the client-side, but many such errors do require closure and the
// client-side code is bad at handling errors so we play it safe.
@@ -2499,149 +2440,149 @@
}
repContext->setReplyToError(request->method, err);
assert(context->http->out.offset == 0);
context->pullData();
return true;
}
}
}
return false;
}
#endif // USE_OPENSSL
static void
clientProcessRequestFinished(ConnStateData *conn, const HttpRequest::Pointer &request)
{
/*
* DPW 2007-05-18
* Moved the TCP_RESET feature from clientReplyContext::sendMoreData
* to here because calling comm_reset_close() causes http to
- * be freed and the above connNoteUseOfBuffer() would hit an
- * assertion, not to mention that we were accessing freed memory.
+ * be freed before accessing.
*/
if (request != NULL && request->flags.resetTcp && Comm::IsConnOpen(conn->clientConnection)) {
debugs(33, 3, HERE << "Sending TCP RST on " << conn->clientConnection);
conn->flags.readMore = false;
comm_reset_close(conn->clientConnection);
}
}
void
-clientProcessRequest(ConnStateData *conn, HttpParser *hp, ClientSocketContext *context, const HttpRequestMethod& method, Http::ProtocolVersion http_ver)
+clientProcessRequest(ConnStateData *conn, const Http1::RequestParserPointer &hp, ClientSocketContext *context)
{
ClientHttpRequest *http = context->http;
HttpRequest::Pointer request;
- bool notedUseOfBuffer = false;
bool chunked = false;
bool mustReplyToOptions = false;
bool unsupportedTe = false;
bool expectBody = false;
// temporary hack to avoid splitting this huge function with sensitive code
const bool isFtp = !hp;
if (isFtp) {
// In FTP, case, we already have the request parsed and checked, so we
// only need to go through the final body/conn setup to doCallouts().
assert(http->request);
request = http->request;
- notedUseOfBuffer = true;
} else {
if (context->flags.parsed_ok == 0) {
clientStreamNode *node = context->getClientReplyContext();
- debugs(33, 2, "clientProcessRequest: Invalid Request");
+ debugs(33, 2, "Invalid Request");
conn->quitAfterError(NULL);
// setLogUri should called before repContext->setReplyToError
- setLogUri(http, http->uri, true);
+ setLogUri(http, http->uri, true);
clientReplyContext *repContext = dynamic_cast<clientReplyContext *>(node->data.getRaw());
- assert (repContext);
+ assert(repContext);
+
+ // determine which error page templates to use for specific parsing errors
+ err_type errPage = ERR_INVALID_REQ;
switch (hp->request_parse_status) {
- case Http::scHeaderTooLarge:
- repContext->setReplyToError(ERR_TOO_BIG, Http::scBadRequest, method, http->uri, conn->clientConnection->remote, NULL, conn->in.buf.c_str(), NULL);
+ case Http::scRequestHeaderFieldsTooLarge:
+ // fall through to next case
+ case Http::scUriTooLong:
+ errPage = ERR_TOO_BIG;
break;
case Http::scMethodNotAllowed:
- repContext->setReplyToError(ERR_UNSUP_REQ, Http::scMethodNotAllowed, method, http->uri,
- conn->clientConnection->remote, NULL, conn->in.buf.c_str(), NULL);
+ errPage = ERR_UNSUP_REQ;
+ break;
+ case Http::scHttpVersionNotSupported:
+ errPage = ERR_UNSUP_HTTPVERSION;
break;
default:
- repContext->setReplyToError(ERR_INVALID_REQ, hp->request_parse_status, method, http->uri,
- conn->clientConnection->remote, NULL, conn->in.buf.c_str(), NULL);
+ // use default ERR_INVALID_REQ set above.
+ break;
}
+ repContext->setReplyToError(errPage, hp->request_parse_status, hp->method(), http->uri,
+ conn->clientConnection->remote, NULL, conn->in.buf.c_str(), NULL);
assert(context->http->out.offset == 0);
context->pullData();
- connNoteUseOfBuffer(conn, http->req_sz);
return;
}
- if ((request = HttpRequest::CreateFromUrlAndMethod(http->uri, method)) == NULL) {
+ if ((request = HttpRequest::CreateFromUrlAndMethod(http->uri, hp->method())) == NULL) {
clientStreamNode *node = context->getClientReplyContext();
debugs(33, 5, "Invalid URL: " << http->uri);
conn->quitAfterError(request.getRaw());
// setLogUri should called before repContext->setReplyToError
- setLogUri(http, http->uri, true);
+ setLogUri(http, http->uri, true);
clientReplyContext *repContext = dynamic_cast<clientReplyContext *>(node->data.getRaw());
- assert (repContext);
- repContext->setReplyToError(ERR_INVALID_URL, Http::scBadRequest, method, http->uri, conn->clientConnection->remote, NULL, NULL, NULL);
+ assert(repContext);
+ repContext->setReplyToError(ERR_INVALID_URL, Http::scBadRequest, hp->method(), http->uri, conn->clientConnection->remote, NULL, NULL, NULL);
assert(context->http->out.offset == 0);
context->pullData();
- connNoteUseOfBuffer(conn, http->req_sz);
return;
}
/* RFC 2616 section 10.5.6 : handle unsupported HTTP major versions cleanly. */
/* We currently only support 0.9, 1.0, 1.1 properly */
/* TODO: move HTTP-specific processing into servers/HttpServer and such */
- if ( (http_ver.major == 0 && http_ver.minor != 9) ||
- (http_ver.major > 1) ) {
+ if ( (hp->messageProtocol().major == 0 && hp->messageProtocol().minor != 9) ||
+ (hp->messageProtocol().major > 1) ) {
clientStreamNode *node = context->getClientReplyContext();
- debugs(33, 5, "Unsupported HTTP version discovered. :\n" << HttpParserHdrBuf(hp));
+ debugs(33, 5, "Unsupported HTTP version discovered. :\n" << hp->messageProtocol());
conn->quitAfterError(request.getRaw());
// setLogUri should called before repContext->setReplyToError
setLogUri(http, http->uri, true);
clientReplyContext *repContext = dynamic_cast<clientReplyContext *>(node->data.getRaw());
assert (repContext);
- repContext->setReplyToError(ERR_UNSUP_HTTPVERSION, Http::scHttpVersionNotSupported, method, http->uri,
- conn->clientConnection->remote, NULL, HttpParserHdrBuf(hp), NULL);
+ repContext->setReplyToError(ERR_UNSUP_HTTPVERSION, Http::scHttpVersionNotSupported, hp->method(), http->uri,
+ conn->clientConnection->remote, NULL, NULL, NULL);
assert(context->http->out.offset == 0);
context->pullData();
- connNoteUseOfBuffer(conn, http->req_sz);
clientProcessRequestFinished(conn, request);
return;
}
/* compile headers */
- /* we should skip request line! */
- /* XXX should actually know the damned buffer size here */
- if (http_ver.major >= 1 && !request->parseHeader(HttpParserHdrBuf(hp), HttpParserHdrSz(hp))) {
+ if (hp->messageProtocol().major >= 1 && !request->parseHeader(*hp)) {
clientStreamNode *node = context->getClientReplyContext();
- debugs(33, 5, "Failed to parse request headers:\n" << HttpParserHdrBuf(hp));
+ debugs(33, 5, "Failed to parse request headers:\n" << hp->mimeHeader());
conn->quitAfterError(request.getRaw());
// setLogUri should called before repContext->setReplyToError
- setLogUri(http, http->uri, true);
+ setLogUri(http, http->uri, true);
clientReplyContext *repContext = dynamic_cast<clientReplyContext *>(node->data.getRaw());
- assert (repContext);
- repContext->setReplyToError(ERR_INVALID_REQ, Http::scBadRequest, method, http->uri, conn->clientConnection->remote, NULL, NULL, NULL);
+ assert(repContext);
+ repContext->setReplyToError(ERR_INVALID_REQ, Http::scBadRequest, hp->method(), http->uri, conn->clientConnection->remote, NULL, NULL, NULL);
assert(context->http->out.offset == 0);
context->pullData();
- connNoteUseOfBuffer(conn, http->req_sz);
clientProcessRequestFinished(conn, request);
return;
}
}
// Some blobs below are still HTTP-specific, but we would have to rewrite
// this entire function to remove them from the FTP code path. Connection
// setup and body_pipe preparation blobs are needed for FTP.
request->clientConnectionManager = conn;
request->flags.accelerated = http->flags.accel;
request->flags.sslBumped=conn->switchedToHttps();
request->flags.ignoreCc = conn->port->ignore_cc;
// TODO: decouple http->flags.accel from request->flags.sslBumped
request->flags.noDirect = (request->flags.accelerated && !request->flags.sslBumped) ?
!conn->port->allow_direct : 0;
#if USE_AUTH
if (request->flags.sslBumped) {
if (conn->getAuth() != NULL)
@@ -2680,191 +2621,172 @@
request->port = getMyPort();
http->flags.internal = true;
} else
debugs(33, 2, "internal URL found: " << request->url.getScheme() << "://" << request->GetHost() <<
':' << request->port << " (not this proxy)");
}
if (http->flags.internal)
request->login[0] = '\0';
request->flags.internal = http->flags.internal;
setLogUri (http, urlCanonicalClean(request.getRaw()));
request->client_addr = conn->clientConnection->remote; // XXX: remove reuest->client_addr member.
#if FOLLOW_X_FORWARDED_FOR
// indirect client gets stored here because it is an HTTP header result (from X-Forwarded-For:)
// not a details about teh TCP connection itself
request->indirect_client_addr = conn->clientConnection->remote;
#endif /* FOLLOW_X_FORWARDED_FOR */
request->my_addr = conn->clientConnection->local;
request->myportname = conn->port->name;
- request->http_ver = http_ver;
+
+ if (!isFtp) {
+ // XXX: for non-HTTP messages instantiate a different HttpMsg child type
+ // for now Squid only supports HTTP requests
+ const AnyP::ProtocolVersion &http_ver = hp->messageProtocol();
+ assert(request->http_ver.protocol == http_ver.protocol);
+ request->http_ver.major = http_ver.major;
+ request->http_ver.minor = http_ver.minor;
+ }
// Link this HttpRequest to ConnStateData relatively early so the following complex handling can use it
// TODO: this effectively obsoletes a lot of conn->FOO copying. That needs cleaning up later.
request->clientConnectionManager = conn;
if (request->header.chunked()) {
chunked = true;
} else if (request->header.has(HDR_TRANSFER_ENCODING)) {
const String te = request->header.getList(HDR_TRANSFER_ENCODING);
// HTTP/1.1 requires chunking to be the last encoding if there is one
unsupportedTe = te.size() && te != "identity";
} // else implied identity coding
- mustReplyToOptions = (method == Http::METHOD_OPTIONS) &&
+ mustReplyToOptions = (request->method == Http::METHOD_OPTIONS) &&
(request->header.getInt64(HDR_MAX_FORWARDS) == 0);
if (!urlCheckRequest(request.getRaw()) || mustReplyToOptions || unsupportedTe) {
clientStreamNode *node = context->getClientReplyContext();
conn->quitAfterError(request.getRaw());
clientReplyContext *repContext = dynamic_cast<clientReplyContext *>(node->data.getRaw());
assert (repContext);
repContext->setReplyToError(ERR_UNSUP_REQ, Http::scNotImplemented, request->method, NULL,
conn->clientConnection->remote, request.getRaw(), NULL, NULL);
assert(context->http->out.offset == 0);
context->pullData();
- connNoteUseOfBuffer(conn, http->req_sz);
clientProcessRequestFinished(conn, request);
return;
}
if (!chunked && !clientIsContentLengthValid(request.getRaw())) {
clientStreamNode *node = context->getClientReplyContext();
clientReplyContext *repContext = dynamic_cast<clientReplyContext *>(node->data.getRaw());
assert (repContext);
conn->quitAfterError(request.getRaw());
repContext->setReplyToError(ERR_INVALID_REQ,
Http::scLengthRequired, request->method, NULL,
conn->clientConnection->remote, request.getRaw(), NULL, NULL);
assert(context->http->out.offset == 0);
context->pullData();
- connNoteUseOfBuffer(conn, http->req_sz);
clientProcessRequestFinished(conn, request);
return;
}
if (request->header.has(HDR_EXPECT)) {
const String expect = request->header.getList(HDR_EXPECT);
const bool supportedExpect = (expect.caseCmp("100-continue") == 0);
if (!supportedExpect) {
clientStreamNode *node = context->getClientReplyContext();
clientReplyContext *repContext = dynamic_cast<clientReplyContext *>(node->data.getRaw());
assert (repContext);
conn->quitAfterError(request.getRaw());
repContext->setReplyToError(ERR_INVALID_REQ, Http::scExpectationFailed, request->method, http->uri,
conn->clientConnection->remote, request.getRaw(), NULL, NULL);
assert(context->http->out.offset == 0);
context->pullData();
- connNoteUseOfBuffer(conn, http->req_sz);
clientProcessRequestFinished(conn, request);
return;
}
}
if (!isFtp) {
http->request = request.getRaw();
HTTPMSGLOCK(http->request);
}
clientSetKeepaliveFlag(http);
// Let tunneling code be fully responsible for CONNECT requests
if (http->request->method == Http::METHOD_CONNECT) {
context->mayUseConnection(true);
conn->flags.readMore = false;
-
- // consume header early so that tunnel gets just the body
- connNoteUseOfBuffer(conn, http->req_sz);
- notedUseOfBuffer = true;
}
#if USE_OPENSSL
if (conn->switchedToHttps() && conn->serveDelayedError(context)) {
- if (!notedUseOfBuffer)
- connNoteUseOfBuffer(conn, http->req_sz);
clientProcessRequestFinished(conn, request);
return;
}
#endif
/* Do we expect a request-body? */
expectBody = chunked || request->content_length > 0;
if (!context->mayUseConnection() && expectBody) {
request->body_pipe = conn->expectRequestBody(
chunked ? -1 : request->content_length);
- if (!isFtp) {
- // consume header early so that body pipe gets just the body
- connNoteUseOfBuffer(conn, http->req_sz);
- notedUseOfBuffer = true;
- }
-
/* Is it too large? */
if (!chunked && // if chunked, we will check as we accumulate
clientIsRequestBodyTooLargeForPolicy(request->content_length)) {
clientStreamNode *node = context->getClientReplyContext();
clientReplyContext *repContext = dynamic_cast<clientReplyContext *>(node->data.getRaw());
assert (repContext);
conn->quitAfterError(request.getRaw());
repContext->setReplyToError(ERR_TOO_BIG,
Http::scPayloadTooLarge, Http::METHOD_NONE, NULL,
conn->clientConnection->remote, http->request, NULL, NULL);
assert(context->http->out.offset == 0);
context->pullData();
clientProcessRequestFinished(conn, request);
return;
}
if (!isFtp) {
// We may stop producing, comm_close, and/or call setReplyToError()
// below, so quit on errors to avoid http->doCallouts()
if (!conn->handleRequestBodyData()) {
clientProcessRequestFinished(conn, request);
return;
}
if (!request->body_pipe->productionEnded()) {
debugs(33, 5, "need more request body");
context->mayUseConnection(true);
assert(conn->flags.readMore);
}
}
}
http->calloutContext = new ClientRequestContext(http);
http->doCallouts();
- if (!notedUseOfBuffer)
- connNoteUseOfBuffer(conn, http->req_sz);
-
clientProcessRequestFinished(conn, request);
}
-static void
-connStripBufferWhitespace (ConnStateData * conn)
-{
- // XXX: kill this whole function.
- while (!conn->in.buf.isEmpty() && xisspace(conn->in.buf.at(0))) {
- conn->in.buf.consume(1);
- }
-}
-
int
ConnStateData::pipelinePrefetchMax() const
{
return Config.pipeline_max_prefetch;
}
/**
* Limit the number of concurrent requests.
* \return true when there are available position(s) in the pipeline queue for another request.
* \return false when the pipeline queue is full or disabled.
*/
bool
ConnStateData::concurrentRequestQueueFilled() const
{
const int existingRequestCount = getConcurrentRequestCount();
// default to the configured pipeline size.
// add 1 because the head of pipeline is counted in concurrent requests and not prefetch queue
const int concurrentRequestLimit = pipelinePrefetchMax() + 1;
@@ -3141,64 +3063,63 @@
fqdncache_gethostbyaddr(clientConnection->remote, FQDN_LOOKUP_IF_MISS);
return true;
}
/**
* Attempt to parse one or more requests from the input buffer.
* Returns true after completing parsing of at least one request [header]. That
* includes cases where parsing ended with an error (e.g., a huge request).
*/
bool
ConnStateData::clientParseRequests()
{
bool parsed_req = false;
debugs(33, 5, HERE << clientConnection << ": attempting to parse");
// Loop while we have read bytes that are not needed for producing the body
// On errors, bodyPipe may become nil, but readMore will be cleared
while (!in.buf.isEmpty() && !bodyPipe && flags.readMore) {
- connStripBufferWhitespace(this);
/* Don't try to parse if the buffer is empty */
if (in.buf.isEmpty())
break;
/* Limit the number of concurrent requests */
if (concurrentRequestQueueFilled())
break;
// try to parse the PROXY protocol header magic bytes
if (needProxyProtocolHeader_ && !parseProxyProtocolHeader())
break;
- Http::ProtocolVersion http_ver;
- if (ClientSocketContext *context = parseOneRequest(http_ver)) {
+ if (ClientSocketContext *context = parseOneRequest()) {
debugs(33, 5, clientConnection << ": done parsing a request");
+
AsyncCall::Pointer timeoutCall = commCbCall(5, 4, "clientLifetimeTimeout",
CommTimeoutCbPtrFun(clientLifetimeTimeout, context->http));
commSetConnTimeout(clientConnection, Config.Timeout.lifetime, timeoutCall);
context->registerWithConn();
- processParsedRequest(context, http_ver);
+ processParsedRequest(context);
parsed_req = true; // XXX: do we really need to parse everything right NOW ?
if (context->mayUseConnection()) {
debugs(33, 3, HERE << "Not parsing new requests, as this request may need the connection");
break;
}
} else {
debugs(33, 5, clientConnection << ": not enough request data: " <<
in.buf.length() << " < " << Config.maxRequestHeaderSize);
Must(in.buf.length() < Config.maxRequestHeaderSize);
break;
}
}
/* XXX where to 'finish' the parsing pass? */
return parsed_req;
}
void
@@ -3326,41 +3247,41 @@
{
assert(bodyPipe != NULL);
size_t putSize = 0;
if (in.bodyParser) { // chunked encoding
if (const err_type error = handleChunkedRequestBody(putSize)) {
abortChunkedRequestBody(error);
return false;
}
} else { // identity encoding
debugs(33,5, HERE << "handling plain request body for " << clientConnection);
putSize = bodyPipe->putMoreData(in.buf.c_str(), in.buf.length());
if (!bodyPipe->mayNeedMoreData()) {
// BodyPipe will clear us automagically when we produced everything
bodyPipe = NULL;
}
}
if (putSize > 0)
- connNoteUseOfBuffer(this, putSize);
+ consumeInput(putSize);
if (!bodyPipe) {
debugs(33,5, HERE << "produced entire request body for " << clientConnection);
if (const char *reason = stoppedSending()) {
/* we've finished reading like good clients,
* now do the close that initiateClose initiated.
*/
debugs(33, 3, HERE << "closing for earlier sending error: " << reason);
clientConnection->close();
return false;
}
}
return true;
}
/// parses available chunked encoded body bytes, checks size, returns errors
err_type
ConnStateData::handleChunkedRequestBody(size_t &putSize)
=== modified file 'src/client_side.h'
--- src/client_side.h 2014-09-24 14:07:55 +0000
+++ src/client_side.h 2014-09-27 09:53:31 +0000
@@ -1,38 +1,38 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
/* DEBUG: section 33 Client-side Routines */
#ifndef SQUID_CLIENTSIDE_H
#define SQUID_CLIENTSIDE_H
#include "clientStreamForward.h"
#include "comm.h"
#include "helper/forward.h"
#include "HttpControlMsg.h"
-#include "HttpParser.h"
+#include "http/forward.h"
#include "ipc/FdNotes.h"
#include "SBuf.h"
#if USE_AUTH
#include "auth/UserRequest.h"
#endif
#if USE_OPENSSL
#include "ssl/support.h"
#endif
class ConnStateData;
class ClientHttpRequest;
class clientStreamNode;
class ChunkedCodingParser;
namespace AnyP
{
class PortCfg;
} // namespace Anyp
/**
* Badly named.
@@ -392,75 +392,78 @@
/// remove no longer needed leading bytes from the input buffer
void consumeInput(const size_t byteCount);
/* TODO: Make the methods below (at least) non-public when possible. */
/// stop parsing the request and create context for relaying error info
ClientSocketContext *abortRequestParsing(const char *const errUri);
protected:
void startDechunkingRequest();
void finishDechunkingRequest(bool withSuccess);
void abortChunkedRequestBody(const err_type error);
err_type handleChunkedRequestBody(size_t &putSize);
void startPinnedConnectionMonitoring();
void clientPinnedConnectionRead(const CommIoCbParams &io);
/// parse input buffer prefix into a single transfer protocol request
/// return NULL to request more header bytes (after checking any limits)
/// use abortRequestParsing() to handle parsing errors w/o creating request
- virtual ClientSocketContext *parseOneRequest(Http::ProtocolVersion &ver) = 0;
+ virtual ClientSocketContext *parseOneRequest() = 0;
/// start processing a freshly parsed request
- virtual void processParsedRequest(ClientSocketContext *context, const Http::ProtocolVersion &ver) = 0;
+ virtual void processParsedRequest(ClientSocketContext *context) = 0;
/// returning N allows a pipeline of 1+N requests (see pipeline_prefetch)
virtual int pipelinePrefetchMax() const;
/// timeout to use when waiting for the next request
virtual time_t idleTimeout() const = 0;
BodyPipe::Pointer bodyPipe; ///< set when we are reading request body
private:
int connFinishedWithConn(int size);
void clientAfterReadingRequests();
bool concurrentRequestQueueFilled() const;
void pinNewConnection(const Comm::ConnectionPointer &pinServer, HttpRequest *request, CachePeer *aPeer, bool auth);
/* PROXY protocol functionality */
bool proxyProtocolValidateClient();
bool parseProxyProtocolHeader();
bool parseProxy1p0();
bool parseProxy2p0();
bool proxyProtocolError(const char *reason);
/// whether PROXY protocol header is still expected
bool needProxyProtocolHeader_;
#if USE_AUTH
/// some user details that can be used to perform authentication on this connection
Auth::UserRequest::Pointer auth_;
#endif
+ /// the parser state for current HTTP/1.x input buffer processing
+ Http1::RequestParserPointer parser_;
+
#if USE_OPENSSL
bool switchedToHttps_;
/// The SSL server host name appears in CONNECT request or the server ip address for the intercepted requests
String sslConnectHostOrIp; ///< The SSL server host name as passed in the CONNECT request
String sslCommonName; ///< CN name for SSL certificate generation
String sslBumpCertKey; ///< Key to use to store/retrieve generated certificate
/// HTTPS server cert. fetching state for bump-ssl-server-first
Ssl::ServerBump *sslServerBump;
Ssl::CertSignAlgorithm signAlgorithm; ///< The signing algorithm to use
#endif
/// the reason why we no longer write the response or nil
const char *stoppedSending_;
/// the reason why we no longer read the request or nil
const char *stoppedReceiving_;
AsyncCall::Pointer reader; ///< set when we are reading
SBuf connectionTag_; ///< clt_conn_tag=Tag annotation for client connection
@@ -473,25 +476,25 @@
int varyEvaluateMatch(StoreEntry * entry, HttpRequest * req);
/// accept requests to a given port and inform subCall about them
void clientStartListeningOn(AnyP::PortCfgPointer &port, const RefCount< CommCbFunPtrCallT<CommAcceptCbPtrFun> > &subCall, const Ipc::FdNoteId noteId);
void clientOpenListenSockets(void);
void clientConnectionsClose(void);
void httpRequestFree(void *);
/// decide whether to expect multiple requests on the corresponding connection
void clientSetKeepaliveFlag(ClientHttpRequest *http);
/* misplaced declaratrions of Stream callbacks provided/used by client side */
SQUIDCEXTERN CSR clientGetMoreData;
SQUIDCEXTERN CSS clientReplyStatus;
SQUIDCEXTERN CSD clientReplyDetach;
CSCB clientSocketRecipient;
CSD clientSocketDetach;
/* TODO: Move to HttpServer. Warning: Move requires large code nonchanges! */
-ClientSocketContext *parseHttpRequest(ConnStateData *, HttpParser *, HttpRequestMethod *, Http::ProtocolVersion *);
-void clientProcessRequest(ConnStateData *conn, HttpParser *hp, ClientSocketContext *context, const HttpRequestMethod& method, Http::ProtocolVersion http_ver);
-void clientPostHttpsAccept(ConnStateData *connState);
+ClientSocketContext *parseHttpRequest(ConnStateData *, const Http1::RequestParserPointer &);
+void clientProcessRequest(ConnStateData *, const Http1::RequestParserPointer &, ClientSocketContext *);
+void clientPostHttpsAccept(ConnStateData *);
#endif /* SQUID_CLIENTSIDE_H */
=== modified file 'src/htcp.cc'
--- src/htcp.cc 2014-09-29 05:13:17 +0000
+++ src/htcp.cc 2014-10-15 06:31:27 +0000
@@ -132,55 +132,61 @@
Countstr key_name;
Countstr signature;
};
class htcpSpecifier : public StoreClient
{
public:
MEMPROXY_CLASS(htcpSpecifier);
void created (StoreEntry *newEntry);
void checkHit();
void checkedHit(StoreEntry *e);
void setFrom(Ip::Address &from);
void setDataHeader(htcpDataHeader *);
const char *method;
char *uri;
char *version;
char *req_hdrs;
+ size_t reqHdrsSz; ///< size of the req_hdrs content
HttpRequest *request;
private:
HttpRequest *checkHitRequest;
Ip::Address from; // was a ptr. return to such IFF needed. otherwise copy should do.
htcpDataHeader *dhdr;
};
MEMPROXY_CLASS_INLINE(htcpSpecifier);
struct _htcpDetail {
char *resp_hdrs;
+ size_t respHdrsSz;
+
char *entity_hdrs;
+ size_t entityHdrsSz;
+
char *cache_hdrs;
+ size_t cacheHdrsSz;
};
struct _htcpStuff {
int op;
int rr;
int f1;
int response;
int reason;
uint32_t msg_id;
htcpSpecifier S;
htcpDetail D;
};
enum {
HTCP_NOP,
HTCP_TST,
HTCP_MON,
HTCP_SET,
HTCP_CLR,
HTCP_END
@@ -216,41 +222,41 @@
};
static void htcpIncomingConnectionOpened(const Comm::ConnectionPointer &conn, int errNo);
static uint32_t msg_id_counter = 0;
static Comm::ConnectionPointer htcpOutgoingConn = NULL;
static Comm::ConnectionPointer htcpIncomingConn = NULL;
#define N_QUERIED_KEYS 8192
static uint32_t queried_id[N_QUERIED_KEYS];
static cache_key queried_keys[N_QUERIED_KEYS][SQUID_MD5_DIGEST_LENGTH];
static Ip::Address queried_addr[N_QUERIED_KEYS];
static MemAllocator *htcpDetailPool = NULL;
static int old_squid_format = 0;
static ssize_t htcpBuildPacket(char *buf, size_t buflen, htcpStuff * stuff);
static htcpSpecifier *htcpUnpackSpecifier(char *buf, int sz);
static htcpDetail *htcpUnpackDetail(char *buf, int sz);
static ssize_t htcpBuildAuth(char *buf, size_t buflen);
-static ssize_t htcpBuildCountstr(char *buf, size_t buflen, const char *s);
+static ssize_t htcpBuildCountstr(char *buf, size_t buflen, const char *s, size_t len);
static ssize_t htcpBuildData(char *buf, size_t buflen, htcpStuff * stuff);
static ssize_t htcpBuildDetail(char *buf, size_t buflen, htcpStuff * stuff);
static ssize_t htcpBuildOpData(char *buf, size_t buflen, htcpStuff * stuff);
static ssize_t htcpBuildSpecifier(char *buf, size_t buflen, htcpStuff * stuff);
static ssize_t htcpBuildTstOpData(char *buf, size_t buflen, htcpStuff * stuff);
static void htcpFreeSpecifier(htcpSpecifier * s);
static void htcpFreeDetail(htcpDetail * s);
static void htcpHandleMsg(char *buf, int sz, Ip::Address &from);
static void htcpLogHtcp(Ip::Address &, int, LogTags, const char *);
static void htcpHandleMon(htcpDataHeader *, char *buf, int sz, Ip::Address &from);
static void htcpHandleNop(htcpDataHeader *, char *buf, int sz, Ip::Address &from);
static void htcpHandleSet(htcpDataHeader *, char *buf, int sz, Ip::Address &from);
static void htcpHandleTst(htcpDataHeader *, char *buf, int sz, Ip::Address &from);
static void htcpRecv(int fd, void *data);
@@ -290,133 +296,126 @@
/*
* STUFF FOR SENDING HTCP MESSAGES
*/
static ssize_t
htcpBuildAuth(char *buf, size_t buflen)
{
htcpAuthHeader auth;
size_t copy_sz = 0;
assert(2 == sizeof(uint16_t));
auth.length = htons(2);
copy_sz += 2;
if (buflen < copy_sz)
return -1;
memcpy(buf, &auth, copy_sz);
return copy_sz;
}
static ssize_t
-htcpBuildCountstr(char *buf, size_t buflen, const char *s)
+htcpBuildCountstr(char *buf, size_t buflen, const char *s, size_t len)
{
- uint16_t length;
- size_t len;
int off = 0;
if (buflen - off < 2)
return -1;
- if (s)
- len = strlen(s);
- else
- len = 0;
-
debugs(31, 3, "htcpBuildCountstr: LENGTH = " << len);
debugs(31, 3, "htcpBuildCountstr: TEXT = {" << (s ? s : "<NULL>") << "}");
- length = htons((uint16_t) len);
+ uint16_t length = htons((uint16_t) len);
memcpy(buf + off, &length, 2);
off += 2;
if (buflen - off < len)
return -1;
if (len)
memcpy(buf + off, s, len);
off += len;
return off;
}
static ssize_t
htcpBuildSpecifier(char *buf, size_t buflen, htcpStuff * stuff)
{
ssize_t off = 0;
ssize_t s;
- s = htcpBuildCountstr(buf + off, buflen - off, stuff->S.method);
+ s = htcpBuildCountstr(buf + off, buflen - off, stuff->S.method, (stuff->S.method?strlen(stuff->S.method):0));
if (s < 0)
return s;
off += s;
- s = htcpBuildCountstr(buf + off, buflen - off, stuff->S.uri);
+ s = htcpBuildCountstr(buf + off, buflen - off, stuff->S.uri, (stuff->S.uri?strlen(stuff->S.uri):0));
if (s < 0)
return s;
off += s;
- s = htcpBuildCountstr(buf + off, buflen - off, stuff->S.version);
+ s = htcpBuildCountstr(buf + off, buflen - off, stuff->S.version, (stuff->S.version?strlen(stuff->S.version):0));
if (s < 0)
return s;
off += s;
- s = htcpBuildCountstr(buf + off, buflen - off, stuff->S.req_hdrs);
+ s = htcpBuildCountstr(buf + off, buflen - off, stuff->S.req_hdrs, stuff->S.reqHdrsSz);
if (s < 0)
return s;
off += s;
debugs(31, 3, "htcpBuildSpecifier: size " << off);
return off;
}
static ssize_t
htcpBuildDetail(char *buf, size_t buflen, htcpStuff * stuff)
{
ssize_t off = 0;
ssize_t s;
- s = htcpBuildCountstr(buf + off, buflen - off, stuff->D.resp_hdrs);
+ s = htcpBuildCountstr(buf + off, buflen - off, stuff->D.resp_hdrs, stuff->D.respHdrsSz);
if (s < 0)
return s;
off += s;
- s = htcpBuildCountstr(buf + off, buflen - off, stuff->D.entity_hdrs);
+ s = htcpBuildCountstr(buf + off, buflen - off, stuff->D.entity_hdrs, stuff->D.entityHdrsSz);
if (s < 0)
return s;
off += s;
- s = htcpBuildCountstr(buf + off, buflen - off, stuff->D.cache_hdrs);
+ s = htcpBuildCountstr(buf + off, buflen - off, stuff->D.cache_hdrs, stuff->D.cacheHdrsSz);
if (s < 0)
return s;
off += s;
return off;
}
static ssize_t
htcpBuildTstOpData(char *buf, size_t buflen, htcpStuff * stuff)
{
switch (stuff->rr) {
case RR_REQUEST:
debugs(31, 3, "htcpBuildTstOpData: RR_REQUEST");
return htcpBuildSpecifier(buf, buflen, stuff);
case RR_RESPONSE:
debugs(31, 3, "htcpBuildTstOpData: RR_RESPONSE");
@@ -611,40 +610,42 @@
}
static void
htcpFreeSpecifier(htcpSpecifier * s)
{
HTTPMSGUNLOCK(s->request);
delete s;
}
static void
htcpFreeDetail(htcpDetail * d)
{
htcpDetailPool->freeOne(d);
}
/*
* Unpack an HTCP SPECIFIER in place
* This will overwrite any following AUTH block
*/
+// XXX: this needs to be turned into an Htcp1::Parser inheriting from Http1::RequestParser
+// but with different first-line and block unpacking logic.
static htcpSpecifier *
htcpUnpackSpecifier(char *buf, int sz)
{
htcpSpecifier *s = new htcpSpecifier;
HttpRequestMethod method;
/* Find length of METHOD */
uint16_t l = ntohs(*(uint16_t *) buf);
sz -= 2;
buf += 2;
if (l > sz) {
debugs(31, 3, "htcpUnpackSpecifier: failed to unpack METHOD");
htcpFreeSpecifier(s);
return NULL;
}
/* Set METHOD */
s->method = buf;
buf += l;
@@ -692,135 +693,131 @@
debugs(31, 6, "htcpUnpackSpecifier: VERSION (" << l << "/" << sz << ") '" << s->version << "'");
/* Find length of REQ-HDRS */
l = ntohs(*(uint16_t *) buf);
sz -= 2;
if (l > sz) {
debugs(31, 3, "htcpUnpackSpecifier: failed to unpack REQ-HDRS");
htcpFreeSpecifier(s);
return NULL;
}
/* Add terminating null to URI */
*buf = '\0';
buf += 2;
/* Set REQ-HDRS */
s->req_hdrs = buf;
buf += l;
sz -= l;
+ s->reqHdrsSz = l;
debugs(31, 6, "htcpUnpackSpecifier: REQ-HDRS (" << l << "/" << sz << ") '" << s->req_hdrs << "'");
debugs(31, 3, "htcpUnpackSpecifier: " << sz << " bytes left");
/*
* Add terminating null to REQ-HDRS. This is possible because we allocated
* an extra byte when we received the packet. This will overwrite any following
* AUTH block.
*/
*buf = '\0';
- /*
- * Parse the request
- */
- method = HttpRequestMethod(s->method, NULL);
+ // Parse the request
+ method.HttpRequestMethodXXX(s->method);
s->request = HttpRequest::CreateFromUrlAndMethod(s->uri, method == Http::METHOD_NONE ? HttpRequestMethod(Http::METHOD_GET) : method);
if (s->request)
HTTPMSGLOCK(s->request);
return s;
}
/*
* Unpack an HTCP DETAIL in place
* This will overwrite any following AUTH block
*/
static htcpDetail *
htcpUnpackDetail(char *buf, int sz)
{
htcpDetail *d = static_cast<htcpDetail *>(htcpDetailPool->alloc());
/* Find length of RESP-HDRS */
uint16_t l = ntohs(*(uint16_t *) buf);
sz -= 2;
buf += 2;
if (l > sz) {
debugs(31, 3, "htcpUnpackDetail: failed to unpack RESP_HDRS");
htcpFreeDetail(d);
return NULL;
}
/* Set RESP-HDRS */
d->resp_hdrs = buf;
-
buf += l;
-
+ d->respHdrsSz = l;
sz -= l;
/* Find length of ENTITY-HDRS */
l = ntohs(*(uint16_t *) buf);
sz -= 2;
if (l > sz) {
debugs(31, 3, "htcpUnpackDetail: failed to unpack ENTITY_HDRS");
htcpFreeDetail(d);
return NULL;
}
/* Add terminating null to RESP-HDRS */
*buf = '\0';
/* Set ENTITY-HDRS */
buf += 2;
d->entity_hdrs = buf;
-
buf += l;
-
+ d->entityHdrsSz = l;
sz -= l;
/* Find length of CACHE-HDRS */
l = ntohs(*(uint16_t *) buf);
sz -= 2;
if (l > sz) {
debugs(31, 3, "htcpUnpackDetail: failed to unpack CACHE_HDRS");
htcpFreeDetail(d);
return NULL;
}
/* Add terminating null to ENTITY-HDRS */
*buf = '\0';
/* Set CACHE-HDRS */
buf += 2;
d->cache_hdrs = buf;
-
buf += l;
-
+ d->cacheHdrsSz = l;
sz -= l;
debugs(31, 3, "htcpUnpackDetail: " << sz << " bytes left");
/*
* Add terminating null to CACHE-HDRS. This is possible because we allocated
* an extra byte when we received the packet. This will overwrite any following
* AUTH block.
*/
*buf = '\0';
return d;
}
static bool
htcpAccessAllowed(acl_access * acl, htcpSpecifier * s, Ip::Address &from)
{
/* default deny if no access list present */
if (!acl)
return false;
@@ -838,95 +835,102 @@
static char pkt[8192];
HttpHeader hdr(hoHtcpReply);
MemBuf mb;
Packer p;
ssize_t pktlen;
memset(&stuff, '\0', sizeof(stuff));
stuff.op = HTCP_TST;
stuff.rr = RR_RESPONSE;
stuff.f1 = 0;
stuff.response = e ? 0 : 1;
debugs(31, 3, "htcpTstReply: response = " << stuff.response);
stuff.msg_id = dhdr->msg_id;
if (spec) {
mb.init();
packerToMemInit(&p, &mb);
stuff.S.method = spec->method;
stuff.S.uri = spec->uri;
stuff.S.version = spec->version;
stuff.S.req_hdrs = spec->req_hdrs;
+ stuff.S.reqHdrsSz = spec->reqHdrsSz;
if (e)
hdr.putInt(HDR_AGE, (e->timestamp <= squid_curtime ? (squid_curtime - e->timestamp) : 0) );
else
hdr.putInt(HDR_AGE, 0);
hdr.packInto(&p);
stuff.D.resp_hdrs = xstrdup(mb.buf);
+ stuff.D.respHdrsSz = mb.contentSize();
debugs(31, 3, "htcpTstReply: resp_hdrs = {" << stuff.D.resp_hdrs << "}");
mb.reset();
hdr.reset();
if (e && e->expires > -1)
hdr.putTime(HDR_EXPIRES, e->expires);
if (e && e->lastmod > -1)
hdr.putTime(HDR_LAST_MODIFIED, e->lastmod);
hdr.packInto(&p);
stuff.D.entity_hdrs = xstrdup(mb.buf);
+ stuff.D.entityHdrsSz = mb.contentSize();
debugs(31, 3, "htcpTstReply: entity_hdrs = {" << stuff.D.entity_hdrs << "}");
mb.reset();
hdr.reset();
#if USE_ICMP
if (char *host = urlHostname(spec->uri)) {
int rtt = 0;
int hops = 0;
int samp = 0;
netdbHostData(host, &samp, &rtt, &hops);
if (rtt || hops) {
char cto_buf[128];
snprintf(cto_buf, 128, "%s %d %f %d",
host, samp, 0.001 * rtt, hops);
hdr.putExt("Cache-to-Origin", cto_buf);
}
}
#endif /* USE_ICMP */
hdr.packInto(&p);
stuff.D.cache_hdrs = xstrdup(mb.buf);
+ stuff.D.cacheHdrsSz = mb.contentSize();
debugs(31, 3, "htcpTstReply: cache_hdrs = {" << stuff.D.cache_hdrs << "}");
mb.clean();
hdr.clean();
packerClean(&p);
}
pktlen = htcpBuildPacket(pkt, sizeof(pkt), &stuff);
safe_free(stuff.D.resp_hdrs);
+ stuff.D.respHdrsSz = 0;
safe_free(stuff.D.entity_hdrs);
+ stuff.D.entityHdrsSz = 0;
safe_free(stuff.D.cache_hdrs);
+ stuff.D.cacheHdrsSz = 0;
if (!pktlen) {
debugs(31, 3, "htcpTstReply: htcpBuildPacket() failed");
return;
}
htcpSend(pkt, (int) pktlen, from);
}
static void
htcpClrReply(htcpDataHeader * dhdr, int purgeSucceeded, Ip::Address &from)
{
htcpStuff stuff;
static char pkt[8192];
ssize_t pktlen;
/* If dhdr->F1 == 0, no response desired */
if (dhdr->F1 == 0)
@@ -949,52 +953,49 @@
pktlen = htcpBuildPacket(pkt, sizeof(pkt), &stuff);
if (pktlen == 0) {
debugs(31, 3, "htcpClrReply: htcpBuildPacket() failed");
return;
}
htcpSend(pkt, (int) pktlen, from);
}
static void
htcpHandleNop(htcpDataHeader * hdr, char *buf, int sz, Ip::Address &from)
{
debugs(31, 3, "htcpHandleNop: Unimplemented");
}
void
htcpSpecifier::checkHit()
{
- char *blk_end;
checkHitRequest = request;
if (NULL == checkHitRequest) {
debugs(31, 3, "htcpCheckHit: NO; failed to parse URL");
checkedHit(NullStoreEntry::getInstance());
return;
}
- blk_end = req_hdrs + strlen(req_hdrs);
-
- if (!checkHitRequest->header.parse(req_hdrs, blk_end)) {
+ if (!checkHitRequest->header.parse(req_hdrs, reqHdrsSz)) {
debugs(31, 3, "htcpCheckHit: NO; failed to parse request headers");
delete checkHitRequest;
checkHitRequest = NULL;
checkedHit(NullStoreEntry::getInstance());
return;
}
StoreEntry::getPublicByRequest(this, checkHitRequest);
}
void
htcpSpecifier::created (StoreEntry *e)
{
StoreEntry *hit=NULL;
assert (e);
if (e->isNull()) {
debugs(31, 3, "htcpCheckHit: NO; public object not found");
} else if (!e->validToSend()) {
debugs(31, 3, "htcpCheckHit: NO; entry not valid to send" );
@@ -1002,53 +1003,50 @@
debugs(31, 3, "htcpCheckHit: NO; cached response is stale");
} else {
debugs(31, 3, "htcpCheckHit: YES!?");
hit = e;
}
checkedHit (hit);
}
static void
htcpClrStoreEntry(StoreEntry * e)
{
debugs(31, 4, "htcpClrStoreEntry: Clearing store for entry: " << e->url() );
e->releaseRequest();
}
static int
htcpClrStore(const htcpSpecifier * s)
{
HttpRequest *request = s->request;
- char *blk_end;
StoreEntry *e = NULL;
int released = 0;
if (request == NULL) {
debugs(31, 3, "htcpClrStore: failed to parse URL");
return -1;
}
/* Parse request headers */
- blk_end = s->req_hdrs + strlen(s->req_hdrs);
-
- if (!request->header.parse(s->req_hdrs, blk_end)) {
+ if (!request->header.parse(s->req_hdrs, s->reqHdrsSz)) {
debugs(31, 2, "htcpClrStore: failed to parse request headers");
return -1;
}
/* Lookup matching entries. This matches both GET and HEAD */
while ((e = storeGetPublicByRequest(request)) != NULL) {
if (e != NULL) {
htcpClrStoreEntry(e);
++released;
}
}
if (released) {
debugs(31, 4, "htcpClrStore: Cleared " << released << " matching entries");
return 1;
} else {
debugs(31, 4, "htcpClrStore: No matching entry found");
return 0;
}
}
@@ -1109,47 +1107,47 @@
debugs(31, 2, "htcpHandleTstResponse: error condition, F1/MO == 1");
return;
}
htcpReply.msg_id = hdr->msg_id;
debugs(31, 3, "htcpHandleTstResponse: msg_id = " << htcpReply.msg_id);
htcpReply.hit = hdr->response ? 0 : 1;
if (hdr->F1) {
debugs(31, 3, "htcpHandleTstResponse: MISS");
} else {
debugs(31, 3, "htcpHandleTstResponse: HIT");
d = htcpUnpackDetail(buf, sz);
if (d == NULL) {
debugs(31, 3, "htcpHandleTstResponse: bad DETAIL");
return;
}
if ((t = d->resp_hdrs))
- htcpReply.hdr.parse(t, t + strlen(t));
+ htcpReply.hdr.parse(t, d->respHdrsSz);
if ((t = d->entity_hdrs))
- htcpReply.hdr.parse(t, t + strlen(t));
+ htcpReply.hdr.parse(t, d->entityHdrsSz);
if ((t = d->cache_hdrs))
- htcpReply.hdr.parse(t, t + strlen(t));
+ htcpReply.hdr.parse(t, d->cacheHdrsSz);
}
debugs(31, 3, "htcpHandleTstResponse: key (" << key << ") " << storeKeyText(key));
neighborsHtcpReply(key, &htcpReply, from);
htcpReply.hdr.clean();
if (d)
htcpFreeDetail(d);
}
static void
htcpHandleTstRequest(htcpDataHeader * dhdr, char *buf, int sz, Ip::Address &from)
{
/* buf should be a SPECIFIER */
htcpSpecifier *s;
if (sz == 0) {
debugs(31, 3, "htcpHandleTst: nothing to do");
return;
}
=== modified file 'src/htcp.h'
--- src/htcp.h 2014-09-13 13:59:43 +0000
+++ src/htcp.h 2014-09-14 12:23:03 +0000
@@ -1,39 +1,37 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
#ifndef SQUID_HTCP_H
#define SQUID_HTCP_H
#if USE_HTCP
#include "HttpHeader.h"
-#include "HttpRequestMethod.h"
+#include "http/forward.h"
#include "ip/forward.h"
-class HttpRequest;
-
/// \ingroup ServerProtocolHTCP
class HtcpReplyData
{
public:
HtcpReplyData();
int hit;
HttpHeader hdr;
uint32_t msg_id;
double version;
struct cto_t {
/* cache-to-origin */
double rtt;
int samp;
int hops;
} cto;
};
/// \ingroup ServerProtocolHTCP
=== modified file 'src/http/Makefile.am'
--- src/http/Makefile.am 2014-09-13 13:59:43 +0000
+++ src/http/Makefile.am 2014-09-14 12:23:03 +0000
@@ -1,26 +1,37 @@
## Copyright (C) 1996-2014 The Squid Software Foundation and contributors
##
## Squid software is distributed under GPLv2+ license and includes
## contributions from numerous individuals and organizations.
## Please see the COPYING and CONTRIBUTORS files for details.
##
include $(top_srcdir)/src/Common.am
include $(top_srcdir)/src/TestHeaders.am
+AUTOMAKE_OPTIONS = subdir-objects
+
+SUBDIRS = one
+DIST_SUBDIRS = one
+
noinst_LTLIBRARIES = libsquid-http.la
libsquid_http_la_SOURCES = \
+ forward.h \
MethodType.cc \
MethodType.h \
ProtocolVersion.h \
+ RegisteredHeaders.h \
+ RequestMethod.cc \
+ RequestMethod.h \
StatusCode.cc \
StatusCode.h \
StatusLine.cc \
StatusLine.h
+libsquid_http_la_LIBADD= one/libhttp1.la
+
MethodType.cc: MethodType.h $(top_srcdir)/src/mk-string-arrays.awk
($(AWK) -f $(top_srcdir)/src/mk-string-arrays.awk sbuf=1 < $(srcdir)/MethodType.h | \
sed -e 's%METHOD_%%' -e 's%_C%-C%' >$@) || ($(RM) -f $@ && exit 1)
CLEANFILES += MethodType.cc
=== added file 'src/http/RegisteredHeaders.h'
--- src/http/RegisteredHeaders.h 1970-01-01 00:00:00 +0000
+++ src/http/RegisteredHeaders.h 2014-08-19 12:14:45 +0000
@@ -0,0 +1,113 @@
+#ifndef SQUID_HTTP_REGISTEREDHEADERS_H
+#define SQUID_HTTP_REGISTEREDHEADERS_H
+
+/// recognized or "known" header fields; and the RFC which defines them (or not)
+/// http://www.iana.org/assignments/message-headers/message-headers.xhtml
+typedef enum {
+ HDR_BAD_HDR = -1,
+ HDR_ACCEPT = 0, /**< RFC 7231 */
+ HDR_ACCEPT_CHARSET, /**< RFC 7231 */
+ HDR_ACCEPT_ENCODING, /**< RFC 7231 */
+ /*HDR_ACCEPT_FEATURES,*/ /* RFC 2295 */
+ HDR_ACCEPT_LANGUAGE, /**< RFC 7231 */
+ HDR_ACCEPT_RANGES, /**< RFC 7233 */
+ HDR_AGE, /**< RFC 7234 */
+ HDR_ALLOW, /**< RFC 7231 */
+ HDR_AUTHENTICATION_INFO, /**< RFC 2617 */
+ HDR_AUTHORIZATION, /**< RFC 7235, 4559 */
+ HDR_CACHE_CONTROL, /**< RFC 7234 */
+ HDR_CONNECTION, /**< RFC 7230 */
+ HDR_CONTENT_BASE, /**< obsoleted RFC 2068 */
+ HDR_CONTENT_DISPOSITION, /**< RFC 2183, 6266 */
+ HDR_CONTENT_ENCODING, /**< RFC 7231 */
+ HDR_CONTENT_LANGUAGE, /**< RFC 7231 */
+ HDR_CONTENT_LENGTH, /**< RFC 7230 */
+ HDR_CONTENT_LOCATION, /**< RFC 7231 */
+ HDR_CONTENT_MD5, /**< deprecated, RFC 2616 */
+ HDR_CONTENT_RANGE, /**< RFC 7233 */
+ HDR_CONTENT_TYPE, /**< RFC 7231 */
+ HDR_COOKIE, /**< RFC 6265 header we may need to erase */
+ HDR_COOKIE2, /**< obsolete RFC 2965 header we may need to erase */
+ HDR_DATE, /**< RFC 7231 */
+ /*HDR_DAV,*/ /* RFC 2518 */
+ /*HDR_DEPTH,*/ /* RFC 2518 */
+ /*HDR_DERIVED_FROM,*/ /* deprecated RFC 2068 */
+ /*HDR_DESTINATION,*/ /* RFC 2518 */
+ HDR_ETAG, /**< RFC 7232 */
+ HDR_EXPECT, /**< RFC 7231 */
+ HDR_EXPIRES, /**< RFC 7234 */
+ HDR_FORWARDED, /**< RFC 7239 */
+ HDR_FROM, /**< RFC 7231 */
+ HDR_HOST, /**< RFC 7230 */
+ HDR_HTTP2_SETTINGS, /**< HTTP/2.0 upgrade header. see draft-ietf-httpbis-http2-13 */
+ /*HDR_IF,*/ /* RFC 2518 */
+ HDR_IF_MATCH, /**< RFC 7232 */
+ HDR_IF_MODIFIED_SINCE, /**< RFC 7232 */
+ HDR_IF_NONE_MATCH, /**< RFC 7232 */
+ HDR_IF_RANGE, /**< RFC 7233 */
+ HDR_IF_UNMODIFIED_SINCE, /**< RFC 7232 */
+ HDR_KEEP_ALIVE, /**< obsoleted RFC 2068 header we may need to erase */
+ HDR_KEY, /**< experimental RFC Draft draft-fielding-http-key-02 */
+ HDR_LAST_MODIFIED, /**< RFC 7232 */
+ HDR_LINK, /**< RFC 5988 */
+ HDR_LOCATION, /**< RFC 7231 */
+ /*HDR_LOCK_TOKEN,*/ /* RFC 2518 */
+ HDR_MAX_FORWARDS, /**< RFC 7231 */
+ HDR_MIME_VERSION, /**< RFC 2045, 7231 */
+ HDR_NEGOTIATE, /**< experimental RFC 2295. Why only this one from 2295? */
+ /*HDR_OVERWRITE,*/ /* RFC 2518 */
+ HDR_ORIGIN, /* CORS Draft specification (see http://www.w3.org/TR/cors/) */
+ HDR_PRAGMA, /**< RFC 7234 */
+ HDR_PROXY_AUTHENTICATE, /**< RFC 7235 */
+ HDR_PROXY_AUTHENTICATION_INFO, /**< RFC 2617 */
+ HDR_PROXY_AUTHORIZATION, /**< RFC 7235 */
+ HDR_PROXY_CONNECTION, /**< obsolete Netscape header we may need to erase. */
+ HDR_PROXY_SUPPORT, /**< RFC 4559 */
+ HDR_PUBLIC, /**< RFC 2068 */
+ HDR_RANGE, /**< RFC 7233 */
+ HDR_REFERER, /**< RFC 7231 */
+ HDR_REQUEST_RANGE, /**< some clients use this, sigh */
+ HDR_RETRY_AFTER, /**< RFC 7231 */
+ HDR_SERVER, /**< RFC 7231 */
+ HDR_SET_COOKIE, /**< RFC 6265 header we may need to erase */
+ HDR_SET_COOKIE2, /**< obsoleted RFC 2965 header we may need to erase */
+ /*HDR_STATUS_URI,*/ /* RFC 2518 */
+ /*HDR_TCN,*/ /* experimental RFC 2295 */
+ HDR_TE, /**< RFC 7230 */
+ /*HDR_TIMEOUT,*/ /* RFC 2518 */
+ HDR_TITLE, /* obsolete draft suggested header */
+ HDR_TRAILER, /**< RFC 7230 */
+ HDR_TRANSFER_ENCODING, /**< RFC 7230 */
+ HDR_TRANSLATE, /**< IIS custom header we may need to erase */
+ HDR_UNLESS_MODIFIED_SINCE, /**< IIS custom header we may need to erase */
+ HDR_UPGRADE, /**< RFC 7230 */
+ HDR_USER_AGENT, /**< RFC 7231 */
+ /*HDR_VARIANT_VARY,*/ /* experimental RFC 2295 */
+ HDR_VARY, /**< RFC 7231 */
+ HDR_VIA, /**< RFC 7230 */
+ HDR_WARNING, /**< RFC 7234 */
+ HDR_WWW_AUTHENTICATE, /**< RFC 7235, 4559 */
+ HDR_X_CACHE, /**< Squid custom header */
+ HDR_X_CACHE_LOOKUP, /**< Squid custom header. temporary hack that became de-facto. TODO remove */
+ HDR_X_FORWARDED_FOR, /**< obsolete Squid custom header, RFC 7239 */
+ HDR_X_REQUEST_URI, /**< Squid custom header appended if ADD_X_REQUEST_URI is defined */
+ HDR_X_SQUID_ERROR, /**< Squid custom header on generated error responses */
+#if X_ACCELERATOR_VARY
+ HDR_X_ACCELERATOR_VARY, /**< obsolete Squid custom header. */
+#endif
+#if USE_ADAPTATION
+ HDR_X_NEXT_SERVICES, /**< Squid custom ICAP header */
+#endif
+ HDR_SURROGATE_CAPABILITY, /**< Edge Side Includes (ESI) header */
+ HDR_SURROGATE_CONTROL, /**< Edge Side Includes (ESI) header */
+ HDR_FRONT_END_HTTPS, /**< MS Exchange custom header we may have to add */
+ HDR_FTP_COMMAND, /**< Internal header for FTP command */
+ HDR_FTP_ARGUMENTS, /**< Internal header for FTP command arguments */
+ HDR_FTP_PRE, /**< Internal header containing leading FTP control response lines */
+ HDR_FTP_STATUS, /**< Internal header for FTP reply status */
+ HDR_FTP_REASON, /**< Internal header for FTP reply reason */
+ HDR_OTHER, /**< internal tag value for "unknown" headers */
+ HDR_ENUM_END
+} http_hdr_type;
+
+#endif /* SQUID_HTTP_REGISTEREDHEADERS_H */
=== renamed file 'src/HttpRequestMethod.cc' => 'src/http/RequestMethod.cc'
--- src/HttpRequestMethod.cc 2014-09-13 13:59:43 +0000
+++ src/http/RequestMethod.cc 2014-10-15 09:48:49 +0000
@@ -1,84 +1,117 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
/* DEBUG: section 73 HTTP Request */
#include "squid.h"
-#include "HttpRequestMethod.h"
+#include "http/RequestMethod.h"
#include "SquidConfig.h"
#include "wordlist.h"
static Http::MethodType &
operator++ (Http::MethodType &aMethod)
{
int tmp = (int)aMethod;
aMethod = (Http::MethodType)(++tmp);
return aMethod;
}
/**
- * Construct a HttpRequestMethod from a NULL terminated string such as "GET"
- * or from a range of chars, * such as "GET" from "GETFOOBARBAZ"
- * (pass in pointer to G and pointer to F.)
+ * Construct a HttpRequestMethod from a C-string such as "GET"
+ * Assumes the string is either nul-terminated or contains whitespace
+ *
+ * \deprecated use SBuf constructor instead
*/
-HttpRequestMethod::HttpRequestMethod(char const *begin, char const *end) : theMethod(Http::METHOD_NONE)
+void
+HttpRequestMethod::HttpRequestMethodXXX(char const *begin)
{
+ // XXX: performance regression due to this method no longer being a constructor
+ // ensure the members are empty/default values before any of the early-return
+ // optimizations can be used.
+ theMethod = Http::METHOD_NONE;
+ theImage.clear();
+
if (begin == NULL)
return;
- /*
- * if e is NULL, b must be NULL terminated and we
- * make e point to the first whitespace character
- * after b.
- */
- if (NULL == end)
- end = begin + strcspn(begin, w_space);
+ char const *end = begin + strcspn(begin, w_space);
if (end == begin)
return;
// TODO: Optimize this linear search.
for (++theMethod; theMethod < Http::METHOD_ENUM_END; ++theMethod) {
// RFC 2616 section 5.1.1 - Method names are case-sensitive
// NP: this is not a HTTP_VIOLATIONS case since there is no MUST/SHOULD involved.
if (0 == image().caseCmp(begin, end-begin)) {
// relaxed parser allows mixed-case and corrects them on output
if (Config.onoff.relaxed_header_parser)
return;
if (0 == image().cmp(begin, end-begin))
return;
}
}
// if method not found and method string is not null then it is other method
theMethod = Http::METHOD_OTHER;
theImage.assign(begin, end-begin);
}
+/**
+ * Construct a HttpRequestMethod from an SBuf string such as "GET"
+ * or from a range of chars such as "FOO" from buffer "GETFOOBARBAZ"
+ *
+ * Assumes the s parameter contains only the characters representing the method name
+ */
+HttpRequestMethod::HttpRequestMethod(const SBuf &s) : theMethod(Http::METHOD_NONE)
+{
+ if (s.isEmpty())
+ return;
+
+ // TODO: Optimize this linear search.
+ for (++theMethod; theMethod < Http::METHOD_ENUM_END; ++theMethod) {
+ // RFC 2616 section 5.1.1 - Method names are case-sensitive
+ // NP: this is not a HTTP_VIOLATIONS case since there is no MUST/SHOULD involved.
+ if (0 == image().caseCmp(s)) {
+
+ // relaxed parser allows mixed-case and corrects them on output
+ if (Config.onoff.relaxed_header_parser)
+ return;
+
+ if (0 == image().cmp(s))
+ return;
+ }
+ }
+
+ // if method not found and method string is not null then it is other method
+ theMethod = Http::METHOD_OTHER;
+ theImage = s;
+}
+
const SBuf &
HttpRequestMethod::image() const
{
static const SBuf methodOther("METHOD_OTHER");
if (Http::METHOD_OTHER != theMethod) {
return Http::MethodType_sb[theMethod];
} else {
if (!theImage.isEmpty()) {
return theImage;
} else {
return methodOther;
}
}
}
bool
HttpRequestMethod::isHttpSafe() const
{
// Only a few methods are defined as safe. All others are "unsafe"
=== renamed file 'src/HttpRequestMethod.h' => 'src/http/RequestMethod.h'
--- src/HttpRequestMethod.h 2014-09-13 13:59:43 +0000
+++ src/http/RequestMethod.h 2014-10-15 04:17:38 +0000
@@ -1,61 +1,53 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
#ifndef SQUID_HTTPREQUESTMETHOD_H
#define SQUID_HTTPREQUESTMETHOD_H
+#include "http/forward.h"
#include "http/MethodType.h"
#include "SBuf.h"
class SquidConfig;
#include <iosfwd>
/**
* This class represents an HTTP Request METHOD
* - i.e. PUT, POST, GET etc.
* It has a runtime extension facility to allow it to
* efficiently support new methods
*/
-class HttpRequestMethod
+class HttpRequestMethod : public RefCountable
{
-
public:
-// static void Configure(SquidConfig &Config);
-
HttpRequestMethod() : theMethod(Http::METHOD_NONE), theImage() {}
-
HttpRequestMethod(Http::MethodType const aMethod) : theMethod(aMethod), theImage() {}
+ explicit HttpRequestMethod(const SBuf &);
- /**
- \param begin string to convert to request method.
- \param end end of the method string (relative to begin). Use NULL if this is unknown.
- *
- \note DO NOT give end a default (ie NULL). That will cause silent char* conversion clashes.
- */
- HttpRequestMethod(char const * begin, char const * end);
+ void HttpRequestMethodXXX(char const *); // deprecated old c-string to SBuf converter.
HttpRequestMethod & operator = (const HttpRequestMethod& aMethod) {
theMethod = aMethod.theMethod;
theImage = aMethod.theImage;
return *this;
}
HttpRequestMethod & operator = (Http::MethodType const aMethod) {
theMethod = aMethod;
theImage.clear();
return *this;
}
bool operator == (Http::MethodType const & aMethod) const { return theMethod == aMethod; }
bool operator == (HttpRequestMethod const & aMethod) const {
return theMethod == aMethod.theMethod &&
(theMethod != Http::METHOD_OTHER || theImage == aMethod.theImage);
}
bool operator != (Http::MethodType const & aMethod) const { return theMethod != aMethod; }
=== added file 'src/http/forward.h'
--- src/http/forward.h 1970-01-01 00:00:00 +0000
+++ src/http/forward.h 2014-05-20 07:45:55 +0000
@@ -0,0 +1,16 @@
+#ifndef SQUID_SRC_HTTP_FORWARD_H
+#define SQUID_SRC_HTTP_FORWARD_H
+
+#include "http/one/forward.h"
+
+// TODO move these classes into Http namespace
+class HttpRequestMethod;
+typedef RefCount<HttpRequestMethod> HttpRequestMethodPointer;
+
+class HttpRequest;
+typedef RefCount<HttpRequest> HttpRequestPointer;
+
+class HttpReply;
+typedef RefCount<HttpReply> HttpReplyPointer;
+
+#endif /* SQUID_SRC_HTTP_FORWARD_H */
=== added directory 'src/http/one'
=== added file 'src/http/one/Makefile.am'
--- src/http/one/Makefile.am 1970-01-01 00:00:00 +0000
+++ src/http/one/Makefile.am 2014-07-30 07:38:25 +0000
@@ -0,0 +1,11 @@
+include $(top_srcdir)/src/Common.am
+include $(top_srcdir)/src/TestHeaders.am
+
+noinst_LTLIBRARIES = libhttp1.la
+
+libhttp1_la_SOURCES = \
+ forward.h \
+ Parser.cc \
+ Parser.h \
+ RequestParser.cc \
+ RequestParser.h
=== renamed file 'src/HttpParser.cc' => 'src/http/one/Parser.cc'
--- src/HttpParser.cc 2014-09-13 13:59:43 +0000
+++ src/http/one/Parser.cc 2014-09-14 12:23:03 +0000
@@ -1,302 +1,80 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
#include "squid.h"
#include "Debug.h"
-#include "HttpParser.h"
-#include "profiler/Profiler.h"
-#include "SquidConfig.h"
+#include "http/one/Parser.h"
+#include "parser/Tokenizer.h"
-void
-HttpParser::clear()
-{
- state = HTTP_PARSE_NONE;
- request_parse_status = Http::scNone;
- buf = NULL;
- bufsiz = 0;
- req.start = req.end = -1;
- hdr_start = hdr_end = -1;
- req.m_start = req.m_end = -1;
- req.u_start = req.u_end = -1;
- req.v_start = req.v_end = -1;
- req.v_maj = req.v_min = 0;
-}
+/// RFC 7230 section 2.6 - 7 magic octets
+const SBuf Http::One::Parser::Http1magic("HTTP/1.");
void
-HttpParser::reset(const char *aBuf, int len)
+Http::One::Parser::clear()
{
- clear(); // empty the state.
- state = HTTP_PARSE_NEW;
- buf = aBuf;
- bufsiz = len;
- debugs(74, 5, HERE << "Request buffer is " << buf);
+ parsingStage_ = HTTP_PARSE_NONE;
+ buf_ = NULL;
+ msgProtocol_ = AnyP::ProtocolVersion();
+ mimeHeaderBlock_.clear();
}
-int
-HttpParser::parseRequestFirstLine()
+// arbitrary maximum-length for headers which can be found by Http1Parser::getHeaderField()
+#define GET_HDR_SZ 1024
+
+// BUG: returns only the first header line with given name,
+// ignores multi-line headers and obs-fold headers
+char *
+Http::One::Parser::getHeaderField(const char *name)
{
- int second_word = -1; // track the suspected URI start
- int first_whitespace = -1, last_whitespace = -1; // track the first and last SP byte
- int line_end = -1; // tracks the last byte BEFORE terminal \r\n or \n sequence
-
- debugs(74, 5, HERE << "parsing possible request: " << buf);
-
- // Single-pass parse: (provided we have the whole line anyways)
-
- req.start = 0;
- if (Config.onoff.relaxed_header_parser) {
- if (Config.onoff.relaxed_header_parser < 0 && buf[req.start] == ' ')
- debugs(74, DBG_IMPORTANT, "WARNING: Invalid HTTP Request: " <<
- "Whitespace bytes received ahead of method. " <<
- "Ignored due to relaxed_header_parser.");
- // Be tolerant of prefix spaces (other bytes are valid method values)
- for (; req.start < bufsiz && buf[req.start] == ' '; ++req.start);
- }
- req.end = -1;
- for (int i = 0; i < bufsiz; ++i) {
- // track first and last whitespace (SP only)
- if (buf[i] == ' ') {
- last_whitespace = i;
- if (first_whitespace < req.start)
- first_whitespace = i;
- }
-
- // track next non-SP/non-HT byte after first_whitespace
- if (second_word < first_whitespace && buf[i] != ' ' && buf[i] != '\t') {
- second_word = i;
- }
-
- // locate line terminator
- if (buf[i] == '\n') {
- req.end = i;
- line_end = i - 1;
- break;
- }
- if (i < bufsiz - 1 && buf[i] == '\r') {
- if (Config.onoff.relaxed_header_parser) {
- if (Config.onoff.relaxed_header_parser < 0 && buf[i + 1] == '\r')
- debugs(74, DBG_IMPORTANT, "WARNING: Invalid HTTP Request: " <<
- "Series of carriage-return bytes received prior to line terminator. " <<
- "Ignored due to relaxed_header_parser.");
-
- // Be tolerant of invalid multiple \r prior to terminal \n
- if (buf[i + 1] == '\n' || buf[i + 1] == '\r')
- line_end = i - 1;
- while (i < bufsiz - 1 && buf[i + 1] == '\r')
- ++i;
-
- if (buf[i + 1] == '\n') {
- req.end = i + 1;
- break;
- }
- } else {
- if (buf[i + 1] == '\n') {
- req.end = i + 1;
- line_end = i - 1;
- break;
- }
- }
-
- // RFC 2616 section 5.1
- // "No CR or LF is allowed except in the final CRLF sequence"
- request_parse_status = Http::scBadRequest;
- return -1;
- }
- }
- if (req.end == -1) {
- debugs(74, 5, "Parser: retval 0: from " << req.start <<
- "->" << req.end << ": needs more data to complete first line.");
- return 0;
- }
+ if (!headerBlockSize() || !name)
+ return NULL;
- // NP: we have now seen EOL, more-data (0) cannot occur.
- // From here on any failure is -1, success is 1
+ LOCAL_ARRAY(char, header, GET_HDR_SZ);
+ const int namelen = name ? strlen(name) : 0;
- // Input Validation:
+ debugs(25, 5, "looking for " << name);
- // Process what we now know about the line structure into field offsets
- // generating HTTP status for any aborts as we go.
+ // while we can find more LF in the SBuf
+ static CharacterSet iso8859Line = CharacterSet("non-LF",'\0','\n'-1) + CharacterSet(NULL, '\n'+1, (unsigned char)0xFF);
+ ::Parser::Tokenizer tok(mimeHeaderBlock_);
+ SBuf p;
+ static const SBuf crlf("\r\n");
- // First non-whitespace = beginning of method
- if (req.start > line_end) {
- request_parse_status = Http::scBadRequest;
- return -1;
- }
- req.m_start = req.start;
+ while (tok.prefix(p, iso8859Line)) {
+ tok.skipOne(CharacterSet::LF); // move tokenizer past the LF
- // First whitespace = end of method
- if (first_whitespace > line_end || first_whitespace < req.start) {
- request_parse_status = Http::scBadRequest; // no method
- return -1;
- }
- req.m_end = first_whitespace - 1;
- if (req.m_end < req.m_start) {
- request_parse_status = Http::scBadRequest; // missing URI?
- return -1;
- }
+ // header lines must start with the name (case insensitive)
+ if (p.substr(0, namelen).caseCmp(name, namelen))
+ continue;
- // First non-whitespace after first SP = beginning of URL+Version
- if (second_word > line_end || second_word < req.start) {
- request_parse_status = Http::scBadRequest; // missing URI
- return -1;
- }
- req.u_start = second_word;
+ // then a COLON
+ if (p[namelen] != ':')
+ continue;
- // RFC 1945: SP and version following URI are optional, marking version 0.9
- // we identify this by the last whitespace being earlier than URI start
- if (last_whitespace < second_word && last_whitespace >= req.start) {
- req.v_maj = 0;
- req.v_min = 9;
- req.u_end = line_end;
- request_parse_status = Http::scOkay; // HTTP/0.9
- return 1;
- } else {
- // otherwise last whitespace is somewhere after end of URI.
- req.u_end = last_whitespace;
- // crop any trailing whitespace in the area we think of as URI
- for (; req.u_end >= req.u_start && xisspace(buf[req.u_end]); --req.u_end);
- }
- if (req.u_end < req.u_start) {
- request_parse_status = Http::scBadRequest; // missing URI
- return -1;
- }
+ // drop any trailing *CR sequence
+ p.trim(crlf, false, true);
- // Last whitespace SP = before start of protocol/version
- if (last_whitespace >= line_end) {
- request_parse_status = Http::scBadRequest; // missing version
- return -1;
- }
- req.v_start = last_whitespace + 1;
- req.v_end = line_end;
+ debugs(25, 5, "checking " << p);
+ p.consume(namelen + 1);
- // We only accept HTTP protocol requests right now.
- // TODO: accept other protocols; RFC 2326 (RTSP protocol) etc
- if ((req.v_end - req.v_start +1) < 5 || strncasecmp(&buf[req.v_start], "HTTP/", 5) != 0) {
-#if USE_HTTP_VIOLATIONS
- // being lax; old parser accepted strange versions
- // there is a LOT of cases which are ambiguous, therefore we cannot use relaxed_header_parser here.
- req.v_maj = 0;
- req.v_min = 9;
- req.u_end = line_end;
- request_parse_status = Http::scOkay; // treat as HTTP/0.9
- return 1;
-#else
- // protocol not supported / implemented.
- request_parse_status = Http::scHttpVersionNotSupported;
- return -1;
-#endif
- }
+ // TODO: optimize SBuf::trim to take CharacterSet directly
+ ::Parser::Tokenizer t(p);
+ t.skipAll(CharacterSet::WSP);
+ p = t.remaining();
- int i = req.v_start + sizeof("HTTP/") -1;
+ // prevent buffer overrun on char header[];
+ p.chop(0, sizeof(header)-1);
- /* next should be 1 or more digits */
- if (!isdigit(buf[i])) {
- request_parse_status = Http::scHttpVersionNotSupported;
- return -1;
- }
- int maj = 0;
- for (; i <= line_end && (isdigit(buf[i])) && maj < 65536; ++i) {
- maj = maj * 10;
- maj = maj + (buf[i]) - '0';
+ // return the header field-value
+ xstrncpy(header, p.rawContent(), p.length());
+ debugs(25, 5, "returning " << header);
+ return header;
}
- // catch too-big values or missing remainders
- if (maj >= 65536 || i > line_end) {
- request_parse_status = Http::scHttpVersionNotSupported;
- return -1;
- }
- req.v_maj = maj;
- /* next should be .; we -have- to have this as we have a whole line.. */
- if (buf[i] != '.') {
- request_parse_status = Http::scHttpVersionNotSupported;
- return -1;
- }
- // catch missing minor part
- if (++i > line_end) {
- request_parse_status = Http::scHttpVersionNotSupported;
- return -1;
- }
- /* next should be one or more digits */
- if (!isdigit(buf[i])) {
- request_parse_status = Http::scHttpVersionNotSupported;
- return -1;
- }
- int min = 0;
- for (; i <= line_end && (isdigit(buf[i])) && min < 65536; ++i) {
- min = min * 10;
- min = min + (buf[i]) - '0';
- }
- // catch too-big values or trailing garbage
- if (min >= 65536 || i < line_end) {
- request_parse_status = Http::scHttpVersionNotSupported;
- return -1;
- }
- req.v_min = min;
-
- /*
- * Rightio - we have all the schtuff. Return true; we've got enough.
- */
- request_parse_status = Http::scOkay;
- return 1;
-}
-
-int
-HttpParserParseReqLine(HttpParser *hmsg)
-{
- PROF_start(HttpParserParseReqLine);
- int retcode = hmsg->parseRequestFirstLine();
- debugs(74, 5, "Parser: retval " << retcode << ": from " << hmsg->req.start <<
- "->" << hmsg->req.end << ": method " << hmsg->req.m_start << "->" <<
- hmsg->req.m_end << "; url " << hmsg->req.u_start << "->" << hmsg->req.u_end <<
- "; version " << hmsg->req.v_start << "->" << hmsg->req.v_end << " (" << hmsg->req.v_maj <<
- "/" << hmsg->req.v_min << ")");
- PROF_stop(HttpParserParseReqLine);
- return retcode;
-}
-
-#if MSGDODEBUG
-/* XXX This should eventually turn into something inlined or #define'd */
-int
-HttpParserReqSz(HttpParser *hp)
-{
- assert(hp->state == HTTP_PARSE_NEW);
- assert(hp->req.start != -1);
- assert(hp->req.end != -1);
- return hp->req.end - hp->req.start + 1;
-}
-
-/*
- * This +1 makes it 'right' but won't make any sense if
- * there's a 0 byte header? This won't happen normally - a valid header
- * is at -least- a blank line (\n, or \r\n.)
- */
-int
-HttpParserHdrSz(HttpParser *hp)
-{
- assert(hp->state == HTTP_PARSE_NEW);
- assert(hp->hdr_start != -1);
- assert(hp->hdr_end != -1);
- return hp->hdr_end - hp->hdr_start + 1;
-}
-
-const char *
-HttpParserHdrBuf(HttpParser *hp)
-{
- assert(hp->state == HTTP_PARSE_NEW);
- assert(hp->hdr_start != -1);
- assert(hp->hdr_end != -1);
- return hp->buf + hp->hdr_start;
+ return NULL;
}
-
-int
-HttpParserRequestLen(HttpParser *hp)
-{
- return hp->hdr_end - hp->req.start + 1;
-}
-#endif
-
=== renamed file 'src/HttpParser.h' => 'src/http/one/Parser.h'
--- src/HttpParser.h 2014-09-13 13:59:43 +0000
+++ src/http/one/Parser.h 2014-10-15 08:18:33 +0000
@@ -1,104 +1,110 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
-#ifndef _SQUID_SRC_HTTPPARSER_H
-#define _SQUID_SRC_HTTPPARSER_H
+#ifndef _SQUID_SRC_HTTP_ONE_PARSER_H
+#define _SQUID_SRC_HTTP_ONE_PARSER_H
-#include "http/StatusCode.h"
+#include "anyp/ProtocolVersion.h"
+#include "http/one/forward.h"
+#include "SBuf.h"
+
+namespace Http {
+namespace One {
// Parser states
-#define HTTP_PARSE_NONE 0 // nothing. completely unset state.
-#define HTTP_PARSE_NEW 1 // initialized, but nothing usefully parsed yet.
+enum ParseState {
+ HTTP_PARSE_NONE, ///< initialized, but nothing usefully parsed yet
+ HTTP_PARSE_FIRST, ///< HTTP/1 message first-line
+ HTTP_PARSE_MIME, ///< HTTP/1 mime-header block
+ HTTP_PARSE_DONE ///< parsed a message header, or reached a terminal syntax error
+};
-/** HTTP protocol parser.
+/** HTTP/1.x protocol parser
*
* Works on a raw character I/O buffer and tokenizes the content into
- * either an error state or, an HTTP procotol request major segments:
- * 1. Request Line (method, URL, protocol, version)
- * 2. Mime header block
+ * the major CRLF delimited segments of an HTTP/1 procotol message:
+ *
+ * \item first-line (request-line / simple-request / status-line)
+ * \item mime-header 0*( header-name ':' SP field-value CRLF)
*/
-class HttpParser
+class Parser : public RefCountable
{
public:
- HttpParser() { clear(); }
+ typedef SBuf::size_type size_type;
- /** Initialize a new parser.
- * Presenting it a buffer to work on and the current length of available
- * data.
- * NOTE: This is *not* the buffer size, just the parse-able data length.
- * The parse routines may be called again later with more data.
- */
- HttpParser(const char *aBuf, int len) { reset(aBuf,len); };
+ Parser() : parsingStage_(HTTP_PARSE_NONE) {}
+ virtual ~Parser() {}
/// Set this parser back to a default state.
/// Will DROP any reference to a buffer (does not free).
- void clear();
+ virtual void clear() = 0;
+
+ /// attempt to parse a message from the buffer
+ /// \retval true if a full message was found and parsed
+ /// \retval false if incomplete, invalid or no message was found
+ virtual bool parse(const SBuf &aBuf) = 0;
+
+ /** Whether the parser is waiting on more data to complete parsing a message.
+ * Use to distinguish between incomplete data and error results
+ * when parse() returns false.
+ */
+ bool needsMoreData() const {return parsingStage_!=HTTP_PARSE_DONE;}
+
+ /// size in bytes of the first line including CRLF terminator
+ virtual size_type firstLineSize() const = 0;
- /// Reset the parser for use on a new buffer.
- void reset(const char *aBuf, int len);
+ /// size in bytes of the message headers including CRLF terminator(s)
+ /// but excluding first-line bytes
+ size_type headerBlockSize() const {return mimeHeaderBlock_.length();}
+
+ /// size in bytes of HTTP message block, includes first-line and mime headers
+ /// excludes any body/entity/payload bytes
+ /// excludes any garbage prefix before the first-line
+ size_type messageHeaderSize() const {return firstLineSize() + headerBlockSize();}
+
+ /// buffer containing HTTP mime headers, excluding message first-line.
+ SBuf mimeHeader() const {return mimeHeaderBlock_;}
+
+ /// the protocol label for this message
+ const AnyP::ProtocolVersion & messageProtocol() const {return msgProtocol_;}
/**
- * Attempt to parse the first line of a new request message.
+ * Scan the mime header block (badly) for a header with teh given name.
*
- * Governed by:
- * RFC 1945 section 5.1
- * RFC 2616 section 5.1
+ * BUG: omits lines when searching for headers with obs-fold or multiple entries.
*
- * Parsing state is stored between calls. However the current implementation
- * begins parsing from scratch on every call.
- * The return value tells you whether the parsing state fields are valid or not.
+ * BUG: limits output to just 1KB when Squid accepts up to 64KB line length.
*
- * \retval -1 an error occurred. request_parse_status indicates HTTP status result.
- * \retval 1 successful parse. member fields contain the request-line items
- * \retval 0 more data is needed to complete the parse
+ * \return A pointer to a field-value of the first matching field-name, or NULL.
*/
- int parseRequestFirstLine();
+ char *getHeaderField(const char *name);
-public:
- uint8_t state;
- const char *buf;
- int bufsiz;
-
- /// Offsets for pieces of the (HTTP request) Request-Line as per RFC 2616
- struct request_offsets {
- int start, end;
- int m_start, m_end; // method
- int u_start, u_end; // url
- int v_start, v_end; // version (full text)
- int v_maj, v_min; // version numerics
- } req;
+ /// the remaining unprocessed section of buffer
+ const SBuf &remaining() const {return buf_;}
- // Offsets for pieces of the MiME Header segment
- int hdr_start, hdr_end;
+protected:
+ /// RFC 7230 section 2.6 - 7 magic octets
+ static const SBuf Http1magic;
- // TODO: Offsets for pieces of the (HTTP reply) Status-Line as per RFC 2616
+ /// bytes remaining to be parsed
+ SBuf buf_;
- /** HTTP status code to be used on the invalid-request error page
- * Http::scNone indicates incomplete parse, Http::scOkay indicates no error.
- */
- Http::StatusCode request_parse_status;
+ /// what stage the parser is currently up to
+ ParseState parsingStage_;
+
+ /// what protocol label has been found in the first line (if any)
+ AnyP::ProtocolVersion msgProtocol_;
+
+ /// buffer holding the mime headers (if any)
+ SBuf mimeHeaderBlock_;
};
-// Legacy functions
-#define HttpParserInit(h,b,l) (h)->reset((b),(l))
-int HttpParserParseReqLine(HttpParser *hp);
-
-#define MSGDODEBUG 0
-#if MSGDODEBUG
-int HttpParserReqSz(HttpParser *);
-int HttpParserHdrSz(HttpParser *);
-const char * HttpParserHdrBuf(HttpParser *);
-int HttpParserRequestLen(HttpParser *hp);
-#else
-#define HttpParserReqSz(hp) ( (hp)->req.end - (hp)->req.start + 1 )
-#define HttpParserHdrSz(hp) ( (hp)->hdr_end - (hp)->hdr_start + 1 )
-#define HttpParserHdrBuf(hp) ( (hp)->buf + (hp)->hdr_start )
-#define HttpParserRequestLen(hp) ( (hp)->hdr_end - (hp)->req.start + 1 )
-#endif
+} // namespace One
+} // namespace Http
-#endif /* _SQUID_SRC_HTTPPARSER_H */
+#endif /* _SQUID_SRC_HTTP_ONE_PARSER_H */
=== added file 'src/http/one/RequestParser.cc'
--- src/http/one/RequestParser.cc 1970-01-01 00:00:00 +0000
+++ src/http/one/RequestParser.cc 2014-10-15 09:32:12 +0000
@@ -0,0 +1,356 @@
+#include "squid.h"
+#include "Debug.h"
+#include "http/one/RequestParser.h"
+#include "http/ProtocolVersion.h"
+#include "mime_header.h"
+#include "profiler/Profiler.h"
+#include "SquidConfig.h"
+
+Http::One::RequestParser::RequestParser() :
+ Parser(),
+ request_parse_status(Http::scNone)
+{
+ req.start = req.end = -1;
+ req.m_start = req.m_end = -1;
+ req.u_start = req.u_end = -1;
+ req.v_start = req.v_end = -1;
+}
+
+/**
+ * Attempt to parse the first line of a new request message.
+ *
+ * Governed by RFC 7230 section 3.5
+ * "
+ * In the interest of robustness, a server that is expecting to receive
+ * and parse a request-line SHOULD ignore at least one empty line (CRLF)
+ * received prior to the request-line.
+ * "
+ *
+ * Parsing state is stored between calls to avoid repeating buffer scans.
+ * If garbage is found the parsing offset is incremented.
+ */
+void
+Http::One::RequestParser::skipGarbageLines()
+{
+ if (Config.onoff.relaxed_header_parser) {
+ if (Config.onoff.relaxed_header_parser < 0 && (buf_[0] == '\r' || buf_[0] == '\n'))
+ debugs(74, DBG_IMPORTANT, "WARNING: Invalid HTTP Request: " <<
+ "CRLF bytes received ahead of request-line. " <<
+ "Ignored due to relaxed_header_parser.");
+ // Be tolerant of prefix empty lines
+ // ie any series of either \n or \r\n with no other characters and no repeated \r
+ while (!buf_.isEmpty() && (buf_[0] == '\n' || (buf_[0] == '\r' && buf_[1] == '\n'))) {
+ buf_.consume(1);
+ }
+ }
+
+ /* XXX: this is a Squid-specific tolerance
+ * it appears never to have been relevant outside out unit-tests
+ * because the ConnStateData parser loop starts with consumeWhitespace()
+ * which absorbs any SP HTAB VTAB CR LF characters.
+ * But unit-tests called the HttpParser method directly without that pruning.
+ */
+#if USE_HTTP_VIOLATIONS
+ if (Config.onoff.relaxed_header_parser) {
+ if (Config.onoff.relaxed_header_parser < 0 && buf_[0] == ' ')
+ debugs(74, DBG_IMPORTANT, "WARNING: Invalid HTTP Request: " <<
+ "Whitespace bytes received ahead of method. " <<
+ "Ignored due to relaxed_header_parser.");
+ // Be tolerant of prefix spaces (other bytes are valid method values)
+ while (!buf_.isEmpty() && buf_[0] == ' ') {
+ buf_.consume(1);
+ }
+ }
+#endif
+}
+
+/**
+ * Attempt to parse the first line of a new request message.
+ *
+ * Governed by:
+ * RFC 1945 section 5.1
+ * RFC 7230 section 3.1 and 3.5
+ *
+ * Parsing state is stored between calls. However the current implementation
+ * begins parsing from scratch on every call.
+ * The return value tells you whether the parsing state fields are valid or not.
+ *
+ * \retval -1 an error occurred. request_parse_status indicates HTTP status result.
+ * \retval 1 successful parse. member fields contain the request-line items
+ * \retval 0 more data is needed to complete the parse
+ */
+int
+Http::One::RequestParser::parseRequestFirstLine()
+{
+ int second_word = -1; // track the suspected URI start
+ int first_whitespace = -1, last_whitespace = -1; // track the first and last SP byte
+ int line_end = -1; // tracks the last byte BEFORE terminal \r\n or \n sequence
+
+ debugs(74, 5, "parsing possible request: buf.length=" << buf_.length());
+ debugs(74, DBG_DATA, buf_);
+
+ // Single-pass parse: (provided we have the whole line anyways)
+
+ req.start = 0;
+ req.end = -1;
+ for (SBuf::size_type i = 0; i < buf_.length(); ++i) {
+ // track first and last whitespace (SP only)
+ if (buf_[i] == ' ') {
+ last_whitespace = i;
+ if (first_whitespace < req.start)
+ first_whitespace = i;
+ }
+
+ // track next non-SP/non-HT byte after first_whitespace
+ if (second_word < first_whitespace && buf_[i] != ' ' && buf_[i] != '\t') {
+ second_word = i;
+ }
+
+ // locate line terminator
+ if (buf_[i] == '\n') {
+ req.end = i;
+ line_end = i - 1;
+ break;
+ }
+ if (i < buf_.length() - 1 && buf_[i] == '\r') {
+ if (Config.onoff.relaxed_header_parser) {
+ if (Config.onoff.relaxed_header_parser < 0 && buf_[i + 1] == '\r')
+ debugs(74, DBG_IMPORTANT, "WARNING: Invalid HTTP Request: " <<
+ "Series of carriage-return bytes received prior to line terminator. " <<
+ "Ignored due to relaxed_header_parser.");
+
+ // Be tolerant of invalid multiple \r prior to terminal \n
+ if (buf_[i + 1] == '\n' || buf_[i + 1] == '\r')
+ line_end = i - 1;
+ while (i < buf_.length() - 1 && buf_[i + 1] == '\r')
+ ++i;
+
+ if (buf_[i + 1] == '\n') {
+ req.end = i + 1;
+ break;
+ }
+ } else {
+ if (buf_[i + 1] == '\n') {
+ req.end = i + 1;
+ line_end = i - 1;
+ break;
+ }
+ }
+
+ // RFC 7230 section 3.1.1 does not prohibit embeded CR like RFC 2616 used to.
+ // However it does explicitly state an exact syntax which omits un-encoded CR
+ // and defines 400 (Bad Request) as the required action when
+ // handed an invalid request-line.
+ request_parse_status = Http::scBadRequest;
+ return -1;
+ }
+ }
+
+ if (req.end == -1) {
+ // DoS protection against long first-line
+ if ((size_t)buf_.length() >= Config.maxRequestHeaderSize) {
+ debugs(33, 5, "Too large request-line");
+ // RFC 7230 section 3.1.1 mandatory 414 response if URL longer than acceptible.
+ request_parse_status = Http::scUriTooLong;
+ return -1;
+ }
+
+ debugs(74, 5, "Parser: retval 0: from " << req.start <<
+ "->" << req.end << ": needs more data to complete first line.");
+ return 0;
+ }
+
+ // NP: we have now seen EOL, more-data (0) cannot occur.
+ // From here on any failure is -1, success is 1
+
+ // Input Validation:
+
+ // DoS protection against long first-line
+ if ((size_t)(req.end-req.start) >= Config.maxRequestHeaderSize) {
+ debugs(33, 5, "Too large request-line");
+ request_parse_status = Http::scUriTooLong;
+ return -1;
+ }
+
+ // Process what we now know about the line structure into field offsets
+ // generating HTTP status for any aborts as we go.
+
+ // First non-whitespace = beginning of method
+ if (req.start > line_end) {
+ request_parse_status = Http::scBadRequest;
+ return -1;
+ }
+ req.m_start = req.start;
+
+ // First whitespace = end of method
+ if (first_whitespace > line_end || first_whitespace < req.start) {
+ request_parse_status = Http::scBadRequest; // no method
+ return -1;
+ }
+ req.m_end = first_whitespace - 1;
+ if (req.m_end < req.m_start) {
+ request_parse_status = Http::scBadRequest; // missing URI?
+ return -1;
+ }
+
+ /* Set method_ */
+ const SBuf tmp = buf_.substr(req.m_start, req.m_end - req.m_start + 1);
+ method_ = HttpRequestMethod(tmp);
+
+ // First non-whitespace after first SP = beginning of URL+Version
+ if (second_word > line_end || second_word < req.start) {
+ request_parse_status = Http::scBadRequest; // missing URI
+ return -1;
+ }
+ req.u_start = second_word;
+
+ // RFC 1945: SP and version following URI are optional, marking version 0.9
+ // we identify this by the last whitespace being earlier than URI start
+ if (last_whitespace < second_word && last_whitespace >= req.start) {
+ msgProtocol_ = Http::ProtocolVersion(0,9);
+ req.u_end = line_end;
+ uri_ = buf_.substr(req.u_start, req.u_end - req.u_start + 1);
+ request_parse_status = Http::scOkay; // HTTP/0.9
+ return 1;
+ } else {
+ // otherwise last whitespace is somewhere after end of URI.
+ req.u_end = last_whitespace;
+ // crop any trailing whitespace in the area we think of as URI
+ for (; req.u_end >= req.u_start && xisspace(buf_[req.u_end]); --req.u_end);
+ }
+ if (req.u_end < req.u_start) {
+ request_parse_status = Http::scBadRequest; // missing URI
+ return -1;
+ }
+ uri_ = buf_.substr(req.u_start, req.u_end - req.u_start + 1);
+
+ // Last whitespace SP = before start of protocol/version
+ if (last_whitespace >= line_end) {
+ request_parse_status = Http::scBadRequest; // missing version
+ return -1;
+ }
+ req.v_start = last_whitespace + 1;
+ req.v_end = line_end;
+
+ /* RFC 7230 section 2.6 : handle unsupported HTTP major versions cleanly. */
+ if ((req.v_end - req.v_start +1) < (int)Http1magic.length() || !buf_.substr(req.v_start, SBuf::npos).startsWith(Http1magic)) {
+ // non-HTTP/1 protocols not supported / implemented.
+ request_parse_status = Http::scHttpVersionNotSupported;
+ return -1;
+ }
+ // NP: magic octets include the protocol name and major version DIGIT.
+ msgProtocol_.protocol = AnyP::PROTO_HTTP;
+ msgProtocol_.major = 1;
+
+ int i = req.v_start + Http1magic.length() -1;
+
+ // catch missing minor part
+ if (++i > line_end) {
+ request_parse_status = Http::scHttpVersionNotSupported;
+ return -1;
+ }
+ /* next should be one or more digits */
+ if (!isdigit(buf_[i])) {
+ request_parse_status = Http::scHttpVersionNotSupported;
+ return -1;
+ }
+ int min = 0;
+ for (; i <= line_end && (isdigit(buf_[i])) && min < 65536; ++i) {
+ min = min * 10;
+ min = min + (buf_[i]) - '0';
+ }
+ // catch too-big values or trailing garbage
+ if (min >= 65536 || i < line_end) {
+ request_parse_status = Http::scHttpVersionNotSupported;
+ return -1;
+ }
+ msgProtocol_.minor = min;
+
+ /*
+ * Rightio - we have all the schtuff. Return true; we've got enough.
+ */
+ request_parse_status = Http::scOkay;
+ return 1;
+}
+
+bool
+Http::One::RequestParser::parse(const SBuf &aBuf)
+{
+ buf_ = aBuf;
+ debugs(74, DBG_DATA, "Parse buf={length=" << aBuf.length() << ", data='" << aBuf << "'}");
+
+ // stage 1: locate the request-line
+ if (parsingStage_ == HTTP_PARSE_NONE) {
+ skipGarbageLines();
+
+ // if we hit something before EOS treat it as a message
+ if (!buf_.isEmpty())
+ parsingStage_ = HTTP_PARSE_FIRST;
+ else
+ return false;
+ }
+
+ // stage 2: parse the request-line
+ if (parsingStage_ == HTTP_PARSE_FIRST) {
+ PROF_start(HttpParserParseReqLine);
+ const int retcode = parseRequestFirstLine();
+
+ // first-line (or a look-alike) found successfully.
+ if (retcode > 0) {
+ buf_.consume(firstLineSize()); // first line bytes including CRLF terminator are now done.
+ parsingStage_ = HTTP_PARSE_MIME;
+ }
+
+ debugs(74, 5, "request-line: retval " << retcode << ": from " << req.start << "->" << req.end <<
+ " line={" << aBuf.length() << ", data='" << aBuf << "'}");
+ debugs(74, 5, "request-line: method " << req.m_start << "->" << req.m_end << " (" << method_ << ")");
+ debugs(74, 5, "request-line: url " << req.u_start << "->" << req.u_end << " (" << uri_ << ")");
+ debugs(74, 5, "request-line: proto " << req.v_start << "->" << req.v_end << " (" << msgProtocol_ << ")");
+ debugs(74, 5, "Parser: bytes processed=" << (aBuf.length()-buf_.length()));
+ PROF_stop(HttpParserParseReqLine);
+
+ // syntax errors already
+ if (retcode < 0) {
+ parsingStage_ = HTTP_PARSE_DONE;
+ return false;
+ }
+ }
+
+ // stage 3: locate the mime header block
+ if (parsingStage_ == HTTP_PARSE_MIME) {
+ // HTTP/1.x request-line is valid and parsing completed.
+ if (msgProtocol_.major == 1) {
+ /* NOTE: HTTP/0.9 requests do not have a mime header block.
+ * So the rest of the code will need to deal with '0'-byte headers
+ * (ie, none, so don't try parsing em)
+ */
+ int64_t mimeHeaderBytes = 0;
+ // XXX: c_str() reallocates. performance regression.
+ if ((mimeHeaderBytes = headersEnd(buf_.c_str(), buf_.length())) == 0) {
+ if (buf_.length()+firstLineSize() >= Config.maxRequestHeaderSize) {
+ debugs(33, 5, "Too large request");
+ request_parse_status = Http::scRequestHeaderFieldsTooLarge;
+ parsingStage_ = HTTP_PARSE_DONE;
+ } else
+ debugs(33, 5, "Incomplete request, waiting for end of headers");
+ return false;
+ }
+ mimeHeaderBlock_ = buf_.consume(mimeHeaderBytes);
+ debugs(74, 5, "mime header (0-" << mimeHeaderBytes << ") {" << mimeHeaderBlock_ << "}");
+
+ } else
+ debugs(33, 3, "Missing HTTP/1.x identifier");
+
+ // NP: we do not do any further stages here yet so go straight to DONE
+ parsingStage_ = HTTP_PARSE_DONE;
+
+ // Squid could handle these headers, but admin does not want to
+ if (messageHeaderSize() >= Config.maxRequestHeaderSize) {
+ debugs(33, 5, "Too large request");
+ request_parse_status = Http::scRequestHeaderFieldsTooLarge;
+ return false;
+ }
+ }
+
+ return !needsMoreData();
+}
=== added file 'src/http/one/RequestParser.h'
--- src/http/one/RequestParser.h 1970-01-01 00:00:00 +0000
+++ src/http/one/RequestParser.h 2014-10-15 08:19:57 +0000
@@ -0,0 +1,65 @@
+#ifndef _SQUID_SRC_HTTP_ONE_REQUESTPARSER_H
+#define _SQUID_SRC_HTTP_ONE_REQUESTPARSER_H
+
+#include "http/one/Parser.h"
+#include "http/RequestMethod.h"
+#include "http/StatusCode.h"
+
+namespace Http {
+namespace One {
+
+/** HTTP/1.x protocol request parser
+ *
+ * Works on a raw character I/O buffer and tokenizes the content into
+ * the major CRLF delimited segments of an HTTP/1 request message:
+ *
+ * \item request-line (method, URL, protocol, version)
+ * \item mime-header (set of RFC2616 syntax header fields)
+ */
+class RequestParser : public Http1::Parser
+{
+public:
+ RequestParser();
+ virtual ~RequestParser() {}
+
+ /* Http::One::Parser API */
+ virtual void clear() {*this = RequestParser();}
+ virtual Http1::Parser::size_type firstLineSize() const {return req.end - req.start + 1;}
+ virtual bool parse(const SBuf &aBuf);
+
+ /// the HTTP method if this is a request message
+ const HttpRequestMethod & method() const {return method_;}
+
+ /// the request-line URI if this is a request message, or an empty string.
+ const SBuf &requestUri() const {return uri_;}
+
+ /** HTTP status code to be used on the invalid-request error page.
+ * Http::scNone indicates incomplete parse,
+ * Http::scOkay indicates no error.
+ */
+ Http::StatusCode request_parse_status;
+
+private:
+ void skipGarbageLines();
+ int parseRequestFirstLine();
+
+ /// Offsets for pieces of the (HTTP request) Request-Line as per RFC 7230 section 3.1.1.
+ /// only valid before and during parse stage HTTP_PARSE_FIRST
+ struct request_offsets {
+ int start, end;
+ int m_start, m_end; // method
+ int u_start, u_end; // url
+ int v_start, v_end; // version (full text)
+ } req;
+
+ /// what request method has been found on the first line
+ HttpRequestMethod method_;
+
+ /// raw copy of the original client reqeust-line URI field
+ SBuf uri_;
+};
+
+} // namespace One
+} // namespace Http
+
+#endif /* _SQUID_SRC_HTTP_ONE_REQUESTPARSER_H */
=== added file 'src/http/one/forward.h'
--- src/http/one/forward.h 1970-01-01 00:00:00 +0000
+++ src/http/one/forward.h 2014-07-30 07:38:25 +0000
@@ -0,0 +1,20 @@
+#ifndef SQUID_SRC_HTTP_ONE_FORWARD_H
+#define SQUID_SRC_HTTP_ONE_FORWARD_H
+
+#include "base/RefCount.h"
+
+namespace Http {
+namespace One {
+
+class Parser;
+typedef RefCount<Http::One::Parser> ParserPointer;
+
+class RequestParser;
+typedef RefCount<Http::One::RequestParser> RequestParserPointer;
+
+} // namespace One
+} // namespace Http
+
+namespace Http1 = Http::One;
+
+#endif /* SQUID_SRC_HTTP_ONE_FORWARD_H */
=== modified file 'src/mgr/ActionParams.cc'
--- src/mgr/ActionParams.cc 2014-09-13 13:59:43 +0000
+++ src/mgr/ActionParams.cc 2014-10-15 06:22:51 +0000
@@ -6,41 +6,41 @@
* Please see the COPYING and CONTRIBUTORS files for details.
*/
/* DEBUG: section 16 Cache Manager API */
#include "squid.h"
#include "base/TextException.h"
#include "ipc/TypedMsgHdr.h"
#include "mgr/ActionParams.h"
Mgr::ActionParams::ActionParams(): httpMethod(Http::METHOD_NONE)
{
}
Mgr::ActionParams::ActionParams(const Ipc::TypedMsgHdr &msg)
{
msg.getString(httpUri);
String method;
msg.getString(method);
- httpMethod = HttpRequestMethod(method.termedBuf(), NULL);
+ httpMethod.HttpRequestMethodXXX(method.termedBuf());
msg.getPod(httpFlags);
msg.getString(httpOrigin);
msg.getString(actionName);
msg.getString(userName);
msg.getString(password);
queryParams.unpack(msg);
}
void
Mgr::ActionParams::pack(Ipc::TypedMsgHdr &msg) const
{
msg.putString(httpUri);
String foo(httpMethod.image().toString());
msg.putString(foo);
msg.putPod(httpFlags);
msg.putString(httpOrigin);
msg.putString(actionName);
=== modified file 'src/mgr/ActionParams.h'
--- src/mgr/ActionParams.h 2014-09-13 13:59:43 +0000
+++ src/mgr/ActionParams.h 2014-09-14 12:23:03 +0000
@@ -1,34 +1,34 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
/* DEBUG: section 16 Cache Manager API */
#ifndef SQUID_MGR_ACTION_PARAMS_H
#define SQUID_MGR_ACTION_PARAMS_H
-#include "HttpRequestMethod.h"
+#include "http/RequestMethod.h"
#include "ipc/forward.h"
#include "mgr/QueryParams.h"
#include "RequestFlags.h"
namespace Mgr
{
/// Cache Manager Action parameters extracted from the user request
class ActionParams
{
public:
ActionParams();
explicit ActionParams(const Ipc::TypedMsgHdr &msg); ///< load from msg
void pack(Ipc::TypedMsgHdr &msg) const; ///< store into msg
public:
/* details of the client HTTP request that caused the action */
String httpUri; ///< HTTP request URI
HttpRequestMethod httpMethod; ///< HTTP request method
=== modified file 'src/mgr/ActionWriter.h'
--- src/mgr/ActionWriter.h 2014-09-13 13:59:43 +0000
+++ src/mgr/ActionWriter.h 2014-09-14 12:23:03 +0000
@@ -1,35 +1,34 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
/* DEBUG: section 16 Cache Manager API */
#ifndef SQUID_MGR_ACTION_WRITER_H
#define SQUID_MGR_ACTION_WRITER_H
#include "comm/forward.h"
-#include "HttpRequestMethod.h"
#include "mgr/StoreToCommWriter.h"
namespace Mgr
{
/// Creates Store entry, fills it using action's fillEntry(), and
/// Comm-writes it using parent StoreToCommWriter.
class ActionWriter: public StoreToCommWriter
{
public:
ActionWriter(const Action::Pointer &anAction, const Comm::ConnectionPointer &conn);
protected:
/* AsyncJob API */
virtual void start();
private:
Action::Pointer action; ///< action that fills the entry
CBDATA_CLASS2(ActionWriter);
=== modified file 'src/mgr/Filler.h'
--- src/mgr/Filler.h 2014-09-13 13:59:43 +0000
+++ src/mgr/Filler.h 2014-09-14 12:23:03 +0000
@@ -1,35 +1,34 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
/* DEBUG: section 16 Cache Manager API */
#ifndef SQUID_MGR_FILLER_H
#define SQUID_MGR_FILLER_H
#include "comm/forward.h"
-#include "HttpRequestMethod.h"
#include "mgr/Action.h"
#include "mgr/StoreToCommWriter.h"
namespace Mgr
{
/// provides Coordinator with a local cache manager response
class Filler: public StoreToCommWriter
{
public:
Filler(const Action::Pointer &anAction, const Comm::ConnectionPointer &conn, unsigned int aRequestId);
protected:
/* AsyncJob API */
virtual void start();
virtual void swanSong();
private:
Action::Pointer action; ///< action that will run() and sendResponse()
unsigned int requestId; ///< the ID of the Request we are responding to
=== modified file 'src/mime.cc'
--- src/mime.cc 2014-09-13 13:59:43 +0000
+++ src/mime.cc 2014-09-14 12:23:03 +0000
@@ -12,42 +12,40 @@
#include "disk.h"
#include "fde.h"
#include "globals.h"
#include "HttpHdrCc.h"
#include "HttpReply.h"
#include "HttpRequest.h"
#include "internal.h"
#include "Mem.h"
#include "MemBuf.h"
#include "MemObject.h"
#include "mime.h"
#include "RequestFlags.h"
#include "SquidConfig.h"
#include "Store.h"
#include "StoreClient.h"
#if HAVE_SYS_STAT_H
#include <sys/stat.h>
#endif
-#define GET_HDR_SZ 1024
-
/* forward declarations */
static void mimeFreeMemory(void);
static char const *mimeGetIcon(const char *fn);
class MimeIcon : public StoreClient
{
public:
explicit MimeIcon(const char *aName);
~MimeIcon();
void setName(char const *);
char const * getName() const;
void load();
void created(StoreEntry *newEntry);
MEMPROXY_CLASS(MimeIcon);
private:
const char *icon_;
char *url_;
};
MEMPROXY_CLASS_INLINE(MimeIcon);
=== modified file 'src/mime_header.cc'
--- src/mime_header.cc 2014-09-13 13:59:43 +0000
+++ src/mime_header.cc 2014-09-14 12:23:03 +0000
@@ -1,116 +1,34 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
/* DEBUG: section 25 MiME Header Parsing */
#include "squid.h"
-
-#define GET_HDR_SZ 1024
#include "Debug.h"
#include "profiler/Profiler.h"
-/*
- * returns a pointer to a field-value of the first matching field-name where
- * field-value matches prefix if any
- */
-char *
-mime_get_header_field(const char *mime, const char *name, const char *prefix)
-{
- LOCAL_ARRAY(char, header, GET_HDR_SZ);
- const char *p = NULL;
- char *q = NULL;
- char got = 0;
- const int namelen = name ? strlen(name) : 0;
- const int preflen = prefix ? strlen(prefix) : 0;
- int l;
-
- if (NULL == mime)
- return NULL;
-
- assert(NULL != name);
-
- debugs(25, 5, "mime_get_header: looking for '" << name << "'");
-
- for (p = mime; *p; p += strcspn(p, "\n\r")) {
- if (strcmp(p, "\r\n\r\n") == 0 || strcmp(p, "\n\n") == 0)
- return NULL;
-
- while (xisspace(*p))
- ++p;
-
- if (strncasecmp(p, name, namelen))
- continue;
-
- if (!xisspace(p[namelen]) && p[namelen] != ':')
- continue;
-
- l = strcspn(p, "\n\r") + 1;
-
- if (l > GET_HDR_SZ)
- l = GET_HDR_SZ;
-
- xstrncpy(header, p, l);
-
- debugs(25, 5, "mime_get_header: checking '" << header << "'");
-
- q = header;
-
- q += namelen;
-
- if (*q == ':') {
- ++q;
- got = 1;
- }
-
- while (xisspace(*q)) {
- ++q;
- got = 1;
- }
-
- if (got && prefix) {
- /* we could process list entries here if we had strcasestr(). */
- /* make sure we did not match a part of another field-value */
- got = !strncasecmp(q, prefix, preflen) && !xisalpha(q[preflen]);
- }
-
- if (got) {
- debugs(25, 5, "mime_get_header: returning '" << q << "'");
- return q;
- }
- }
-
- return NULL;
-}
-
-/* returns a pointer to a field-value of the first matching field-name */
-char *
-mime_get_header(const char *mime, const char *name)
-{
- return mime_get_header_field(mime, name, NULL);
-}
-
size_t
headersEnd(const char *mime, size_t l)
{
size_t e = 0;
int state = 1;
PROF_start(headersEnd);
while (e < l && state < 3) {
switch (state) {
case 0:
if ('\n' == mime[e])
state = 1;
break;
case 1:
if ('\r' == mime[e])
=== modified file 'src/mime_header.h'
--- src/mime_header.h 2014-09-13 13:59:43 +0000
+++ src/mime_header.h 2014-09-14 12:23:03 +0000
@@ -1,18 +1,16 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
/* DEBUG: section 25 MiME Header Parsing */
#ifndef SQUID_MIME_HEADER_H_
#define SQUID_MIME_HEADER_H_
-char *mime_get_header(const char *mime, const char *header);
-char *mime_get_header_field(const char *mime, const char *name, const char *prefix);
size_t headersEnd(const char *, size_t);
#endif /* SQUID_MIME_HEADER_H_ */
=== modified file 'src/servers/FtpServer.cc'
--- src/servers/FtpServer.cc 2014-09-29 05:13:17 +0000
+++ src/servers/FtpServer.cc 2014-10-05 10:43:33 +0000
@@ -7,40 +7,41 @@
*/
/* DEBUG: section 33 Transfer protocol servers */
#include "squid.h"
#include "base/CharacterSet.h"
#include "base/RefCount.h"
#include "base/Subscription.h"
#include "client_side_reply.h"
#include "client_side_request.h"
#include "clientStream.h"
#include "comm/ConnOpener.h"
#include "comm/Read.h"
#include "comm/TcpAcceptor.h"
#include "comm/Write.h"
#include "errorpage.h"
#include "fd.h"
#include "ftp/Elements.h"
#include "ftp/Parsing.h"
#include "globals.h"
+#include "http/one/RequestParser.h"
#include "HttpHdrCc.h"
#include "ip/tools.h"
#include "ipc/FdNotes.h"
#include "parser/Tokenizer.h"
#include "servers/forward.h"
#include "servers/FtpServer.h"
#include "SquidConfig.h"
#include "StatCounters.h"
#include "tools.h"
#include <set>
#include <map>
CBDATA_NAMESPACED_CLASS_INIT(Ftp, Server);
namespace Ftp
{
static void PrintReply(MemBuf &mb, const HttpReply *reply, const char *const prefix = "");
static bool SupportedCommand(const SBuf &name);
static bool CommandHasPathParameter(const SBuf &cmd);
@@ -123,49 +124,48 @@
{
// zero pipelinePrefetchMax() ensures that there is only parsed request
ClientSocketContext::Pointer context = getCurrentContext();
Must(context != NULL);
Must(getConcurrentRequestCount() == 1);
ClientHttpRequest *const http = context->http;
assert(http != NULL);
HttpRequest *const request = http->request;
Must(http->storeEntry() || request);
const bool mayForward = !http->storeEntry() && handleRequest(request);
if (http->storeEntry() != NULL) {
debugs(33, 4, "got an immediate response");
clientSetKeepaliveFlag(http);
context->pullData();
} else if (mayForward) {
debugs(33, 4, "forwarding request to server side");
assert(http->storeEntry() == NULL);
- clientProcessRequest(this, NULL /*parser*/, context.getRaw(),
- request->method, request->http_ver);
+ clientProcessRequest(this, Http1::RequestParserPointer(), context.getRaw());
} else {
debugs(33, 4, "will resume processing later");
}
}
void
-Ftp::Server::processParsedRequest(ClientSocketContext *context, const Http::ProtocolVersion &)
+Ftp::Server::processParsedRequest(ClientSocketContext *context)
{
Must(getConcurrentRequestCount() == 1);
// Process FTP request asynchronously to make sure FTP
// data connection accept callback is fired first.
CallJobHere(33, 4, CbcPointer<Server>(this),
Ftp::Server, doProcessRequest);
}
/// imports more upload data from the data connection
void
Ftp::Server::readUploadData(const CommIoCbParams &io)
{
debugs(33, 5, io.conn << " size " << io.size);
Must(reader != NULL);
reader = NULL;
assert(Comm::IsConnOpen(dataConn));
assert(io.conn->fd == dataConn->fd);
@@ -601,41 +601,41 @@
// no default so that a compiler can check that we have covered all cases
}
ClientSocketContext *context = abortRequestParsing(errUri);
clientStreamNode *node = context->getClientReplyContext();
Must(node);
clientReplyContext *repContext = dynamic_cast<clientReplyContext *>(node->data.getRaw());
Must(repContext);
// We cannot relay FTP scode/reason via HTTP-specific ErrorState.
// TODO: When/if ErrorState can handle native FTP errors, use it instead.
HttpReply *reply = Ftp::HttpReplyWrapper(scode, reason, Http::scBadRequest, -1);
repContext->setReplyToReply(reply);
return context;
}
/// Parses a single FTP request on the control connection.
/// Returns a new ClientSocketContext on valid requests and all errors.
/// Returns NULL on incomplete requests that may still succeed given more data.
ClientSocketContext *
-Ftp::Server::parseOneRequest(Http::ProtocolVersion &ver)
+Ftp::Server::parseOneRequest()
{
flags.readMore = false; // common for all but one case below
// OWS <command> [ RWS <parameter> ] OWS LF
// InlineSpaceChars are isspace(3) or RFC 959 Section 3.1.1.5.2, except
// for the LF character that we must exclude here (but see FullWhiteSpace).
static const char * const InlineSpaceChars = " \f\r\t\v";
static const CharacterSet InlineSpace = CharacterSet("Ftp::Inline", InlineSpaceChars);
static const CharacterSet FullWhiteSpace = (InlineSpace + CharacterSet::LF).rename("Ftp::FWS");
static const CharacterSet CommandChars = FullWhiteSpace.complement("Ftp::Command");
static const CharacterSet TailChars = CharacterSet::LF.complement("Ftp::Tail");
// This set is used to ignore empty commands without allowing an attacker
// to keep us endlessly busy by feeding us whitespace or empty commands.
static const CharacterSet &LeadingSpace = FullWhiteSpace;
SBuf cmd;
SBuf params;
@@ -702,43 +702,42 @@
if (!Ftp::SupportedCommand(cmd))
return earlyError(eekUnsupportedCommand);
const HttpRequestMethod method =
cmd == cmdAppe() || cmd == cmdStor() || cmd == cmdStou() ?
Http::METHOD_PUT : Http::METHOD_GET;
const SBuf *path = (params.length() && CommandHasPathParameter(cmd)) ?
¶ms : NULL;
calcUri(path);
char *newUri = xstrdup(uri.c_str());
HttpRequest *const request = HttpRequest::CreateFromUrlAndMethod(newUri, method);
if (!request) {
debugs(33, 5, "Invalid FTP URL: " << uri);
uri.clear();
safe_free(newUri);
return earlyError(eekInvalidUri);
}
- ver = Http::ProtocolVersion(Ftp::ProtocolVersion().major, Ftp::ProtocolVersion().minor);
request->flags.ftpNative = true;
- request->http_ver = ver;
+ request->http_ver = Http::ProtocolVersion(Ftp::ProtocolVersion().major, Ftp::ProtocolVersion().minor);
// Our fake Request-URIs are not distinctive enough for caching to work
request->flags.cachable = false; // XXX: reset later by maybeCacheable()
request->flags.noCache = true;
request->header.putStr(HDR_FTP_COMMAND, cmd.c_str());
request->header.putStr(HDR_FTP_ARGUMENTS, params.c_str()); // may be ""
if (method == Http::METHOD_PUT) {
request->header.putStr(HDR_EXPECT, "100-continue");
request->header.putStr(HDR_TRANSFER_ENCODING, "chunked");
}
ClientHttpRequest *const http = new ClientHttpRequest(this);
http->request = request;
HTTPMSGLOCK(http->request);
http->req_sz = tok.parsedSize();
http->uri = newUri;
ClientSocketContext *const result =
new ClientSocketContext(clientConnection, http);
=== modified file 'src/servers/FtpServer.h'
--- src/servers/FtpServer.h 2014-09-13 13:59:43 +0000
+++ src/servers/FtpServer.h 2014-09-14 12:23:03 +0000
@@ -58,42 +58,42 @@
// This is a pointer in hope to minimize future changes when MasterState
// becomes a part of MasterXaction. Guaranteed not to be nil.
MasterState::Pointer master; ///< info shared among our FTP client and server jobs
protected:
friend void StartListening();
// errors detected before it is possible to create an HTTP request wrapper
typedef enum {
eekHugeRequest,
eekMissingLogin,
eekMissingUsername,
eekMissingHost,
eekUnsupportedCommand,
eekInvalidUri,
eekMalformedCommand
} EarlyErrorKind;
/* ConnStateData API */
- virtual ClientSocketContext *parseOneRequest(Http::ProtocolVersion &ver);
- virtual void processParsedRequest(ClientSocketContext *context, const Http::ProtocolVersion &ver);
+ virtual ClientSocketContext *parseOneRequest();
+ virtual void processParsedRequest(ClientSocketContext *context);
virtual void notePeerConnection(Comm::ConnectionPointer conn);
virtual void clientPinnedConnectionClosed(const CommCloseCbParams &io);
virtual void handleReply(HttpReply *header, StoreIOBuffer receivedData);
virtual int pipelinePrefetchMax() const;
virtual void writeControlMsgAndCall(ClientSocketContext *context, HttpReply *rep, AsyncCall::Pointer &call);
virtual time_t idleTimeout() const;
/* BodyPipe API */
virtual void noteMoreBodySpaceAvailable(BodyPipe::Pointer);
virtual void noteBodyConsumerAborted(BodyPipe::Pointer ptr);
/* AsyncJob API */
virtual void start();
/* Comm callbacks */
static void AcceptCtrlConnection(const CommAcceptCbParams ¶ms);
void acceptDataConnection(const CommAcceptCbParams ¶ms);
void readUploadData(const CommIoCbParams &io);
void wroteEarlyReply(const CommIoCbParams &io);
void wroteReply(const CommIoCbParams &io);
=== modified file 'src/servers/HttpServer.cc'
--- src/servers/HttpServer.cc 2014-09-13 13:59:43 +0000
+++ src/servers/HttpServer.cc 2014-09-14 12:23:03 +0000
@@ -1,71 +1,72 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
/* DEBUG: section 33 Client-side Routines */
#include "squid.h"
#include "client_side.h"
#include "client_side_request.h"
#include "comm/Write.h"
#include "HttpHeaderTools.h"
+#include "http/one/RequestParser.h"
#include "profiler/Profiler.h"
#include "servers/forward.h"
#include "SquidConfig.h"
namespace Http
{
/// Manages a connection from an HTTP client.
class Server: public ConnStateData
{
public:
Server(const MasterXaction::Pointer &xact, const bool beHttpsServer);
virtual ~Server() {}
void readSomeHttpData();
protected:
/* ConnStateData API */
- virtual ClientSocketContext *parseOneRequest(Http::ProtocolVersion &ver);
- virtual void processParsedRequest(ClientSocketContext *context, const Http::ProtocolVersion &ver);
+ virtual ClientSocketContext *parseOneRequest();
+ virtual void processParsedRequest(ClientSocketContext *context);
virtual void handleReply(HttpReply *rep, StoreIOBuffer receivedData);
virtual void writeControlMsgAndCall(ClientSocketContext *context, HttpReply *rep, AsyncCall::Pointer &call);
virtual time_t idleTimeout() const;
/* BodyPipe API */
virtual void noteMoreBodySpaceAvailable(BodyPipe::Pointer);
virtual void noteBodyConsumerAborted(BodyPipe::Pointer);
/* AsyncJob API */
virtual void start();
private:
void processHttpRequest(ClientSocketContext *const context);
void handleHttpRequestData();
- HttpParser parser_;
+ Http1::RequestParserPointer parser_;
HttpRequestMethod method_; ///< parsed HTTP method
/// temporary hack to avoid creating a true HttpsServer class
const bool isHttpsServer;
CBDATA_CLASS2(Server);
};
} // namespace Http
CBDATA_NAMESPACED_CLASS_INIT(Http, Server);
Http::Server::Server(const MasterXaction::Pointer &xact, bool beHttpsServer):
AsyncJob("Http::Server"),
ConnStateData(xact),
isHttpsServer(beHttpsServer)
{
}
time_t
@@ -92,54 +93,61 @@
AsyncCall::Pointer timeoutCall = JobCallback(33, 5,
TimeoutDialer, this, Http::Server::requestTimeout);
commSetConnTimeout(clientConnection, Config.Timeout.request, timeoutCall);
readSomeData();
}
void
Http::Server::noteMoreBodySpaceAvailable(BodyPipe::Pointer)
{
if (!handleRequestBodyData())
return;
// too late to read more body
if (!isOpen() || stoppedReceiving())
return;
readSomeData();
}
ClientSocketContext *
-Http::Server::parseOneRequest(Http::ProtocolVersion &ver)
+Http::Server::parseOneRequest()
{
- ClientSocketContext *context = NULL;
PROF_start(HttpServer_parseOneRequest);
- HttpParserInit(&parser_, in.buf.c_str(), in.buf.length());
- context = parseHttpRequest(this, &parser_, &method_, &ver);
+
+ // parser is incremental. Generate new parser state if we,
+ // a) dont have one already
+ // b) have completed the previous request parsing already
+ if (!parser_ || !parser_->needsMoreData())
+ parser_ = new Http1::RequestParser();
+
+ /* Process request */
+ ClientSocketContext *context = parseHttpRequest(this, parser_);
+
PROF_stop(HttpServer_parseOneRequest);
return context;
}
void
-Http::Server::processParsedRequest(ClientSocketContext *context, const Http::ProtocolVersion &ver)
+Http::Server::processParsedRequest(ClientSocketContext *context)
{
- clientProcessRequest(this, &parser_, context, method_, ver);
+ clientProcessRequest(this, parser_, context);
}
void
Http::Server::noteBodyConsumerAborted(BodyPipe::Pointer ptr)
{
ConnStateData::noteBodyConsumerAborted(ptr);
stopReceiving("virgin request body consumer aborted"); // closes ASAP
}
void
Http::Server::handleReply(HttpReply *rep, StoreIOBuffer receivedData)
{
// the caller guarantees that we are dealing with the current context only
ClientSocketContext::Pointer context = getCurrentContext();
Must(context != NULL);
const ClientHttpRequest *http = context->http;
Must(http != NULL);
// After sending Transfer-Encoding: chunked (at least), always send
// the last-chunk if there was no error, ignoring responseFinishedOrFailed.
=== renamed file 'src/tests/testHttpParser.cc' => 'src/tests/testHttp1Parser.cc'
--- src/tests/testHttpParser.cc 2014-09-12 23:00:48 +0000
+++ src/tests/testHttp1Parser.cc 2014-10-15 10:14:29 +0000
@@ -1,1108 +1,1442 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
#include "squid.h"
#include <cppunit/TestAssert.h>
-#include "HttpParser.h"
+#define private public
+#define protected public
+
+#include "testHttp1Parser.h"
+#include "http/one/RequestParser.h"
+#include "http/RequestMethod.h"
#include "Mem.h"
#include "MemBuf.h"
#include "SquidConfig.h"
-#include "testHttpParser.h"
+#include "testHttp1Parser.h"
-CPPUNIT_TEST_SUITE_REGISTRATION( testHttpParser );
+CPPUNIT_TEST_SUITE_REGISTRATION( testHttp1Parser );
void
-testHttpParser::globalSetup()
+testHttp1Parser::globalSetup()
{
static bool setup_done = false;
if (setup_done)
return;
Mem::Init();
setup_done = true;
+
+ // default to strict parser. set for loose parsing specifically where behaviour differs.
+ Config.onoff.relaxed_header_parser = 0;
+
+ Config.maxRequestHeaderSize = 1024; // XXX: unit test the RequestParser handling of this limit
+}
+
+struct resultSet {
+ bool parsed;
+ bool needsMore;
+ Http1::ParseState parserState;
+ Http::StatusCode status;
+ int msgStart;
+ int msgEnd;
+ SBuf::size_type suffixSz;
+ int methodStart;
+ int methodEnd;
+ HttpRequestMethod method;
+ int uriStart;
+ int uriEnd;
+ const char *uri;
+ int versionStart;
+ int versionEnd;
+ AnyP::ProtocolVersion version;
+};
+
+static void
+testResults(int line, const SBuf &input, Http1::RequestParser &output, struct resultSet &expect)
+{
+#if WHEN_TEST_DEBUG_IS_NEEDED
+ printf("TEST @%d, in=%u: " SQUIDSBUFPH "\n", line, input.length(), SQUIDSBUFPRINT(input));
+#endif
+
+ CPPUNIT_ASSERT_EQUAL(expect.parsed, output.parse(input));
+ CPPUNIT_ASSERT_EQUAL(expect.needsMore, output.needsMoreData());
+ if (output.needsMoreData())
+ CPPUNIT_ASSERT_EQUAL(expect.parserState, output.parsingStage_);
+ CPPUNIT_ASSERT_EQUAL(expect.status, output.request_parse_status);
+ CPPUNIT_ASSERT_EQUAL(expect.msgStart, output.req.start);
+ CPPUNIT_ASSERT_EQUAL(expect.msgEnd, output.req.end);
+ CPPUNIT_ASSERT_EQUAL(expect.suffixSz, output.buf_.length());
+ CPPUNIT_ASSERT_EQUAL(expect.methodStart, output.req.m_start);
+ CPPUNIT_ASSERT_EQUAL(expect.methodEnd, output.req.m_end);
+ CPPUNIT_ASSERT_EQUAL(expect.method, output.method_);
+ CPPUNIT_ASSERT_EQUAL(expect.uriStart, output.req.u_start);
+ CPPUNIT_ASSERT_EQUAL(expect.uriEnd, output.req.u_end);
+ if (expect.uri != NULL)
+ CPPUNIT_ASSERT_EQUAL(0, output.uri_.cmp(expect.uri));
+ CPPUNIT_ASSERT_EQUAL(expect.versionStart, output.req.v_start);
+ CPPUNIT_ASSERT_EQUAL(expect.versionEnd, output.req.v_end);
+ CPPUNIT_ASSERT_EQUAL(expect.version, output.msgProtocol_);
}
void
-testHttpParser::testParseRequestLineProtocols()
+testHttp1Parser::testParserConstruct()
+{
+ // whether the constructor works
+ {
+ Http1::RequestParser output;
+ CPPUNIT_ASSERT_EQUAL(true, output.needsMoreData());
+ CPPUNIT_ASSERT_EQUAL(Http1::HTTP_PARSE_NONE, output.parsingStage_);
+ CPPUNIT_ASSERT_EQUAL(Http::scNone, output.request_parse_status); // XXX: clear() not being called.
+ CPPUNIT_ASSERT_EQUAL(-1, output.req.start);
+ CPPUNIT_ASSERT_EQUAL(-1, output.req.end);
+ CPPUNIT_ASSERT(output.buf_.isEmpty());
+ CPPUNIT_ASSERT_EQUAL(-1, output.req.m_start);
+ CPPUNIT_ASSERT_EQUAL(-1, output.req.m_end);
+ CPPUNIT_ASSERT_EQUAL(HttpRequestMethod(Http::METHOD_NONE), output.method_);
+ CPPUNIT_ASSERT_EQUAL(-1, output.req.u_start);
+ CPPUNIT_ASSERT_EQUAL(-1, output.req.u_end);
+ CPPUNIT_ASSERT(output.uri_.isEmpty());
+ CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
+ CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
+ CPPUNIT_ASSERT_EQUAL(AnyP::ProtocolVersion(), output.msgProtocol_);
+ }
+
+ // whether new() works
+ {
+ Http1::RequestParser *output = new Http1::RequestParser;
+ CPPUNIT_ASSERT_EQUAL(true, output->needsMoreData());
+ CPPUNIT_ASSERT_EQUAL(Http1::HTTP_PARSE_NONE, output->parsingStage_);
+ CPPUNIT_ASSERT_EQUAL(Http::scNone, output->request_parse_status);
+ CPPUNIT_ASSERT_EQUAL(-1, output->req.start);
+ CPPUNIT_ASSERT_EQUAL(-1, output->req.end);
+ CPPUNIT_ASSERT(output->buf_.isEmpty());
+ CPPUNIT_ASSERT_EQUAL(-1, output->req.m_start);
+ CPPUNIT_ASSERT_EQUAL(-1, output->req.m_end);
+ CPPUNIT_ASSERT_EQUAL(HttpRequestMethod(Http::METHOD_NONE), output->method_);
+ CPPUNIT_ASSERT_EQUAL(-1, output->req.u_start);
+ CPPUNIT_ASSERT_EQUAL(-1, output->req.u_end);
+ CPPUNIT_ASSERT(output->uri_.isEmpty());
+ CPPUNIT_ASSERT_EQUAL(-1, output->req.v_start);
+ CPPUNIT_ASSERT_EQUAL(-1, output->req.v_end);
+ CPPUNIT_ASSERT_EQUAL(AnyP::ProtocolVersion(), output->msgProtocol_);
+ delete output;
+ }
+}
+
+void
+testHttp1Parser::testParseRequestLineProtocols()
{
// ensure MemPools etc exist
globalSetup();
- MemBuf input;
- HttpParser output;
- input.init();
+ SBuf input;
+ Http1::RequestParser output;
// TEST: Do we comply with RFC 1945 section 5.1 ?
// TEST: Do we comply with RFC 2616 section 5.1 ?
// RFC 1945 : HTTP/0.9 simple-request
{
input.append("GET /\r\n", 7);
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0,memcmp("GET /\r\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start], (output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start], (output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(9, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = true,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,0,9)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// RFC 1945 : invalid HTTP/0.9 simple-request (only GET is valid)
-#if 0
+#if WHEN_RFC_COMPLIANT
{
input.append("POST /\r\n", 7);
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0,memcmp("GET /\r\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start], (output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start], (output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(9, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = true,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 3,
+ .method = HttpRequestMethod(Http::METHOD_POST),
+ .uriStart = 5,
+ .uriEnd = 5,
+ .uri = "/",
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
#endif
-
// RFC 1945 and 2616 : HTTP/1.0 request
{
input.append("GET / HTTP/1.0\r\n", 16);
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/1.0\r\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(6, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(13, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.0", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 13,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,0)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// RFC 2616 : HTTP/1.1 request
{
input.append("GET / HTTP/1.1\r\n", 16);
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/1.1\r\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(6, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(13, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 13,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// RFC 2616 : future version full-request
- { input.append("GET / HTTP/1.2\r\n", 16);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/1.2\r\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(6, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(13, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.2", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(2, output.req.v_min);
- input.reset();
+ {
+ input.append("GET / HTTP/1.2\r\n", 16);
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 13,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,2)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
- // RFC 2616 : future version full-request
+ // RFC 7230 : future versions do not use request-line syntax
{
- // XXX: IETF HTTPbis WG has made this two-digits format invalid.
input.append("GET / HTTP/10.12\r\n", 18);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/10.12\r\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(6, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(15, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/10.12", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(10, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(12, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scHttpVersionNotSupported,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 15,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
- // This stage of the parser does not yet accept non-HTTP protocol names.
+ // unknown non-HTTP protocol names
{
- // violations mode treats them as HTTP/0.9 requests!
input.append("GET / FOO/1.0\n", 14);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
-#if USE_HTTP_VIOLATIONS
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(12, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/ FOO/1.0", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(9, output.req.v_min);
-#else
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scHttpVersionNotSupported, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
-#endif
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / FOO/1.0\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(6, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(12, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("FOO/1.0", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scHttpVersionNotSupported,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 12,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// no version
{
input.append("GET / HTTP/\n", 12);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scHttpVersionNotSupported, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(6, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(10, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scHttpVersionNotSupported,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 10,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// no major version
{
input.append("GET / HTTP/.1\n", 14);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scHttpVersionNotSupported, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(6, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(12, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scHttpVersionNotSupported,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 12,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// no version dot
{
input.append("GET / HTTP/11\n", 14);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scHttpVersionNotSupported, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/11\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(6, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(12, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/11", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scHttpVersionNotSupported,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 12,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// negative major version (bug 3062)
{
input.append("GET / HTTP/-999999.1\n", 21);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scHttpVersionNotSupported, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/-999999.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(6, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(19, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/-999999.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scHttpVersionNotSupported,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 19,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// no minor version
{
input.append("GET / HTTP/1.\n", 14);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scHttpVersionNotSupported, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/1.\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(6, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(12, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scHttpVersionNotSupported,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 12,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,0)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// negative major version (bug 3062 corollary)
{
input.append("GET / HTTP/1.-999999\n", 21);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scHttpVersionNotSupported, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/1.-999999\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(6, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(19, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.-999999", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scHttpVersionNotSupported,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 19,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,0)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
}
void
-testHttpParser::testParseRequestLineStrange()
+testHttp1Parser::testParseRequestLineStrange()
{
// ensure MemPools etc exist
globalSetup();
- MemBuf input;
- HttpParser output;
- input.init();
+ SBuf input;
+ Http1::RequestParser output;
// space padded URL
{
input.append("GET / HTTP/1.1\r\n", 21);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/1.1\r\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(5, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(5, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(11, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(18, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 5,
+ .uriEnd = 5,
+ .uri = "/",
+ .versionStart = 11,
+ .versionEnd = 18,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// whitespace inside URI. (nasty but happens)
+ // XXX: depends on tolerant parser...
{
input.append("GET /fo o/ HTTP/1.1\n", 20);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0,memcmp("GET /fo o/ HTTP/1.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(9, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/fo o/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(11, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(18, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 9,
+ .uri = "/fo o/",
+ .versionStart = 11,
+ .versionEnd = 18,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// additional data in buffer
{
input.append("GET / HTTP/1.1\nboo!", 23);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-5, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/1.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end); // strangeness generated by following RFC
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(10, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(17, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-5,
+ .suffixSz = 4, // strlen("boo!")
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 10,
+ .versionEnd = 17,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
}
void
-testHttpParser::testParseRequestLineTerminators()
+testHttp1Parser::testParseRequestLineTerminators()
{
// ensure MemPools etc exist
globalSetup();
- MemBuf input;
- HttpParser output;
- input.init();
+ SBuf input;
+ Http1::RequestParser output;
// alternative EOL sequence: NL-only
{
input.append("GET / HTTP/1.1\n", 15);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/1.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(6, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(13, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 13,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// alternative EOL sequence: double-NL-only
{
input.append("GET / HTTP/1.1\n\n", 16);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-2, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/1.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(6, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(13, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = true,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-2,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 13,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
- // RELAXED alternative EOL sequence: multi-CR-NL
+ // alternative EOL sequence: multi-CR-NL
{
input.append("GET / HTTP/1.1\r\r\r\n", 18);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- Config.onoff.relaxed_header_parser = 1;
// Being tolerant we can ignore and elide these apparently benign CR
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/1.1\r\r\r\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(6, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(13, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_min);
- input.reset();
- }
+ Config.onoff.relaxed_header_parser = 1;
+ struct resultSet expectRelaxed = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 13,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expectRelaxed);
- // STRICT alternative EOL sequence: multi-CR-NL
- {
- input.append("GET / HTTP/1.1\r\r\r\n", 18);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
// strict mode treats these as several bare-CR in the request line which is explicitly invalid.
Config.onoff.relaxed_header_parser = 0;
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scBadRequest, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expectStrict = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scBadRequest,
+ .msgStart = 0,
+ .msgEnd = -1,
+ .suffixSz = input.length(),
+ .methodStart =-1,
+ .methodEnd = -1,
+ .method = HttpRequestMethod(),
+ .uriStart = -1,
+ .uriEnd = -1,
+ .uri = NULL,
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expectStrict);
+ input.clear();
}
// space padded version
{
// RFC 1945 and 2616 specify version is followed by CRLF. No intermediary bytes.
// NP: the terminal whitespace is a special case: invalid for even HTTP/0.9 with no version tag
input.append("GET / HTTP/1.1 \n", 16);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scBadRequest, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/1.1 \n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(13, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/ HTTP/1.1", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
- }
-
- // incomplete line at various positions
- {
- input.append("GET", 3);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(0, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scNone, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
-
- input.append("GET ", 4);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(0, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scNone, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
-
- input.append("GET / HT", 8);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(0, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scNone, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
-
- input.append("GET / HTTP/1.1", 14);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(0, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scNone, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scBadRequest,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 13,
+ .uri = "/ HTTP/1.1",
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
}
void
-testHttpParser::testParseRequestLineMethods()
+testHttp1Parser::testParseRequestLineMethods()
{
// ensure MemPools etc exist
globalSetup();
- MemBuf input;
- HttpParser output;
- input.init();
+ SBuf input;
+ Http1::RequestParser output;
// RFC 2616 : . method
{
input.append(". / HTTP/1.1\n", 13);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp(". / HTTP/1.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp(".", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(2, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(11, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 0,
+ .method = HttpRequestMethod(SBuf(".")),
+ .uriStart = 2,
+ .uriEnd = 2,
+ .uri = "/",
+ .versionStart = 4,
+ .versionEnd = 11,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// OPTIONS with * URL
{
input.append("OPTIONS * HTTP/1.1\n", 19);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("OPTIONS * HTTP/1.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(6, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("OPTIONS", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(8, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(8, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("*", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(10, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(17, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 6,
+ .method = HttpRequestMethod(Http::METHOD_OPTIONS),
+ .uriStart = 8,
+ .uriEnd = 8,
+ .uri = "*",
+ .versionStart = 10,
+ .versionEnd = 17,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// unknown method
{
input.append("HELLOWORLD / HTTP/1.1\n", 22);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HELLOWORLD / HTTP/1.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(9, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HELLOWORLD", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(11, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(11, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(13, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(20, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 9,
+ .method = HttpRequestMethod(SBuf("HELLOWORLD")),
+ .uriStart = 11,
+ .uriEnd = 11,
+ .uri = "/",
+ .versionStart = 13,
+ .versionEnd = 20,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// method-only
{
input.append("A\n", 2);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scBadRequest, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("A\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scBadRequest,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = -1,
+ .method = HttpRequestMethod(),
+ .uriStart = -1,
+ .uriEnd = -1,
+ .uri = NULL,
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
- input.append("GET\n", 4);
- {
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scBadRequest, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ {
+ input.append("GET\n", 4);
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scBadRequest,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = -1,
+ .method = HttpRequestMethod(),
+ .uriStart = -1,
+ .uriEnd = -1,
+ .uri = NULL,
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
- // RELAXED space padded method (in strict mode SP is reserved so invalid as a method byte)
+ // space padded method (in strict mode SP is reserved so invalid as a method byte)
{
input.append(" GET / HTTP/1.1\n", 16);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
+ // RELAXED mode Squid custom tolerance ignores SP
+#if USE_HTTP_VIOLATIONS
Config.onoff.relaxed_header_parser = 1;
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(1, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET / HTTP/1.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(3, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(5, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(5, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(7, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(14, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_min);
- input.reset();
- }
+ struct resultSet expectRelaxed = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0, // garbage collection consumes the SP
+ .msgEnd = (int)input.length()-2,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 13,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expectRelaxed);
+#endif
- // STRICT space padded method (in strict mode SP is reserved so invalid as a method byte)
- {
- input.append(" GET / HTTP/1.1\n", 16);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
+ // STRICT mode obeys RFC syntax
Config.onoff.relaxed_header_parser = 0;
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scBadRequest, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp(" GET / HTTP/1.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expectStrict = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scBadRequest,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = -1,
+ .method = HttpRequestMethod(),
+ .uriStart = -1,
+ .uriEnd = -1,
+ .uri = NULL,
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expectStrict);
+ input.clear();
+ }
+
+ // RFC 2616 defined tolerance: ignore empty line(s) prefix on messages
+#if WHEN_RFC_COMPLIANT
+ {
+ input.append("\r\n\r\n\nGET / HTTP/1.1\r\n", 21);
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 5,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 5,
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 4,
+ .uri = "/",
+ .versionStart = 6,
+ .versionEnd = 13,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
+#endif
// tab padded method (NP: tab is not SP so treated as any other binary)
{
input.append("\tGET / HTTP/1.1\n", 16);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("\tGET / HTTP/1.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(3, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("\tGET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(5, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(5, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(7, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(14, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_min);
- input.reset();
+#if WHEN_RFC_COMPLIANT
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scBadRequest,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = -1,
+ .methodEnd = -1,
+ .method = HttpRequestMethod(),
+ .uriStart = -1,
+ .uriEnd = -1,
+ .uri = NULL,
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+#else // XXX: currently broken
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0, // garbage collection consumes the SP
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 3,
+ .method = HttpRequestMethod(SBuf("\tGET")),
+ .uriStart = 5,
+ .uriEnd = 5,
+ .uri = "/",
+ .versionStart = 7,
+ .versionEnd = 14,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+#endif
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
}
void
-testHttpParser::testParseRequestLineInvalid()
+testHttp1Parser::testParseRequestLineInvalid()
{
// ensure MemPools etc exist
globalSetup();
- MemBuf input;
- HttpParser output;
- input.init();
+ SBuf input;
+ Http1::RequestParser output;
// no method (but in a form which is ambiguous with HTTP/0.9 simple-request)
{
- // XXX: Bug: HTTP/0.9 requires method to be "GET"
+ // XXX: HTTP/0.9 requires method to be "GET"
input.append("/ HTTP/1.0\n", 11);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/ HTTP/1.0\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(2, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(9, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.0", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(9, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = true,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 0,
+ .method = HttpRequestMethod(SBuf("/")),
+ .uriStart = 2,
+ .uriEnd = 9,
+ .uri = "HTTP/1.0",
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,0,9)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
- // RELAXED no method (an invalid format)
+ // no method (an invalid format)
{
input.append(" / HTTP/1.0\n", 12);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- // When tolerantly ignoring SP prefix this case becomes ambiguous with HTTP/0.9 simple-request)
+
+ // XXX: squid custom tolerance consumes initial SP.
Config.onoff.relaxed_header_parser = 1;
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(1, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/ HTTP/1.0\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(1, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(3, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(10, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.0", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(9, output.req.v_min);
- input.reset();
- }
+ struct resultSet expectRelaxed = {
+ .parsed = true,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-2,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 0,
+ .method = HttpRequestMethod(SBuf("/")),
+ .uriStart = 2,
+ .uriEnd = 9,
+ .uri = "HTTP/1.0",
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,0,9)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expectRelaxed);
- // STRICT no method (an invalid format)
- {
- input.append(" / HTTP/1.0\n", 12);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- // When tolerantly ignoring SP prefix this case becomes ambiguous with HTTP/0.9 simple-request)
+ // STRICT detect as invalid
Config.onoff.relaxed_header_parser = 0;
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scBadRequest, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp(" / HTTP/1.0\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+#if WHEN_RFC_COMPLIANT
+ // XXX: except Squid does not
+ struct resultSet expectStrict = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scBadRequest,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = -1,
+ .method = HttpRequestMethod(),
+ .uriStart = -1,
+ .uriEnd = -1,
+ .uri = NULL,
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+#else
+ struct resultSet expectStrict = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scBadRequest,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = -1,
+ .method = HttpRequestMethod(),
+ .uriStart = -1,
+ .uriEnd = -1,
+ .uri = NULL,
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+#endif
+ output.clear();
+ testResults(__LINE__, input, output, expectStrict);
+ input.clear();
}
- // binary code in method (strange but ...)
+ // binary code in method (invalid)
{
input.append("GET\x0B / HTTP/1.1\n", 16);
- //printf("TEST: %d-%d/%d '%.*s'\n", output.req.start, output.req.end, input.contentSize(), 16, input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET\x0B / HTTP/1.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(3, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET\x0B", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(5, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(5, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(7, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(14, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_min);
- input.reset();
+#if WHEN_RFC_COMPLIANT
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scBadRequest,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = -1,
+ .methodEnd = -1,
+ .method = HttpRequestMethod(),
+ .uriStart = -1,
+ .uriEnd = -1,
+ .uri = NULL,
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+#else
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0, // garbage collection consumes the SP
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 3,
+ .method = HttpRequestMethod(SBuf("GET\x0B", 4)),
+ .uriStart = 5,
+ .uriEnd = 5,
+ .uri = "/",
+ .versionStart = 7,
+ .versionEnd = 14,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+#endif
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// CR in method
{
// RFC 2616 sec 5.1 prohibits CR other than in terminator.
input.append("GET\r / HTTP/1.1\r\n", 16);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scBadRequest, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scBadRequest,
+ .msgStart = 0,
+ .msgEnd = -1, // halt at the first \r
+ .suffixSz = input.length(),
+ .methodStart = -1,
+ .methodEnd = -1,
+ .method = HttpRequestMethod(),
+ .uriStart = -1,
+ .uriEnd = -1,
+ .uri = NULL,
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// binary code NUL! in method (strange but ...)
{
input.append("GET\0 / HTTP/1.1\n", 16);
- //printf("TEST: %d-%d/%d '%.*s'\n", output.req.start, output.req.end, input.contentSize(), 16, input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET\0 / HTTP/1.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(3, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET\0", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(5, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(5, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("/", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(7, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(14, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.v_start],(output.req.v_end-output.req.v_start+1)));
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(1, output.req.v_min);
- input.reset();
+#if WHEN_RFC_COMPLIANT
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scBadRequest,
+ .msgStart = 0,
+ .msgEnd = -1, // halt at the \0
+ .suffixSz = input.length(),
+ .methodStart = -1,
+ .methodEnd = -1,
+ .method = HttpRequestMethod(),
+ .uriStart = -1,
+ .uriEnd = -1,
+ .uri = NULL,
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+#else
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_MIME,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 3,
+ .method = HttpRequestMethod(SBuf("GET\0",4)),
+ .uriStart = 5,
+ .uriEnd = 5,
+ .uri = "/",
+ .versionStart = 7,
+ .versionEnd = 14,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1)
+ };
+#endif
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
- // no URL (grammer otherwise correct)
+ // no URL (grammer invalid, ambiguous with RFC 1945 HTTP/0.9 simple-request)
{
input.append("GET HTTP/1.1\n", 14);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET HTTP/1.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(5, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(12, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(9, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = true,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 5,
+ .uriEnd = 12,
+ .uri = "HTTP/1.1",
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,0,9)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// no URL (grammer invalid, ambiguous with RFC 1945 HTTP/0.9 simple-request)
{
input.append("GET HTTP/1.1\n", 13);
- //printf("TEST: '%s'\n",input.content());
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scOkay, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET HTTP/1.1\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(2, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("GET", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(4, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(11, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("HTTP/1.1", &output.buf[output.req.u_start],(output.req.u_end-output.req.u_start+1)));
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(9, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = true,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scOkay,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = 0,
+ .methodStart = 0,
+ .methodEnd = 2,
+ .method = HttpRequestMethod(Http::METHOD_GET),
+ .uriStart = 4,
+ .uriEnd = 11,
+ .uri = "HTTP/1.1",
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,0,9)
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// binary line
{
input.append("\xB\xC\xE\xF\n", 5);
- //printf("TEST: binary-line\n");
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scBadRequest, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("\xB\xC\xE\xF\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scBadRequest,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = -1,
+ .method = HttpRequestMethod(),
+ .uriStart = -1,
+ .uriEnd = -1,
+ .uri = NULL,
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// mixed whitespace line
{
// We accept non-space binary bytes for method so first \t shows up as that
// but remaining space and tabs are skipped searching for URI-start
input.append("\t \t \t\n", 6);
- //printf("TEST: mixed whitespace\n");
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scBadRequest, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL((int)input.contentSize()-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("\t \t \t\n", &output.buf[output.req.start],(output.req.end-output.req.start+1)));
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(0, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(0, memcmp("\t", &output.buf[output.req.m_start],(output.req.m_end-output.req.m_start+1)));
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scBadRequest,
+ .msgStart = 0,
+ .msgEnd = (int)input.length()-1,
+ .suffixSz = input.length(),
+ .methodStart = 0,
+ .methodEnd = 0,
+ .method = HttpRequestMethod(SBuf("\t")),
+ .uriStart = -1,
+ .uriEnd = -1,
+ .uri = NULL,
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
}
// mixed whitespace line with CR middle
{
// CR aborts on sight, so even initial \t method is not marked as above
// (not when parsing clean with whole line available anyway)
input.append("\t \r \n", 6);
- //printf("TEST: mixed whitespace with CR\n");
- output.reset(input.content(), input.contentSize());
- CPPUNIT_ASSERT_EQUAL(-1, HttpParserParseReqLine(&output));
- CPPUNIT_ASSERT_EQUAL(Http::scBadRequest, output.request_parse_status);
- CPPUNIT_ASSERT_EQUAL(0, output.req.start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.m_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.u_end);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_start);
- CPPUNIT_ASSERT_EQUAL(-1, output.req.v_end);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_maj);
- CPPUNIT_ASSERT_EQUAL(0, output.req.v_min);
- input.reset();
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = false,
+ .parserState = Http1::HTTP_PARSE_DONE,
+ .status = Http::scBadRequest,
+ .msgStart = 0,
+ .msgEnd = -1, // halt on the \r
+ .suffixSz = input.length(),
+ .methodStart = -1,
+ .methodEnd = -1,
+ .method = HttpRequestMethod(),
+ .uriStart = -1,
+ .uriEnd = -1,
+ .uri = NULL,
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+ output.clear();
+ testResults(__LINE__, input, output, expect);
+ input.clear();
+ }
+}
+
+void
+testHttp1Parser::testDripFeed()
+{
+ // Simulate a client drip-feeding Squid a few bytes at a time.
+ // extend the size of the buffer from 0 bytes to full request length
+ // calling the parser repeatedly as visible data grows.
+
+ SBuf data;
+ data.append(" ", 12);
+ SBuf::size_type garbageEnd = data.length();
+ data.append("GET http://example.com/ HTTP/1.1\r\n", 34);
+ SBuf::size_type reqLineEnd = data.length() - 1;
+ data.append("Host: example.com\r\n\r\n", 21);
+ SBuf::size_type mimeEnd = data.length() - 1;
+ data.append("...", 3); // trailer to catch mime EOS errors.
+
+ SBuf ioBuf; // begins empty
+ Http1::RequestParser hp;
+
+ // only relaxed parser accepts the garbage whitespace
+ Config.onoff.relaxed_header_parser = 1;
+
+ // state of things we expect right now
+ struct resultSet expect = {
+ .parsed = false,
+ .needsMore = true,
+ .parserState = Http1::HTTP_PARSE_NONE,
+ .status = Http::scNone,
+ .msgStart = -1,
+ .msgEnd = -1,
+ .suffixSz = 0,
+ .methodStart = -1,
+ .methodEnd = -1,
+ .method = HttpRequestMethod(),
+ .uriStart = -1,
+ .uriEnd = -1,
+ .uri = NULL,
+ .versionStart = -1,
+ .versionEnd = -1,
+ .version = AnyP::ProtocolVersion()
+ };
+
+ Config.maxRequestHeaderSize = 1024; // large enough to hold the test data.
+
+ for (SBuf::size_type pos = 0; pos <= data.length(); ++pos) {
+
+ // simulate reading one more byte
+ ioBuf.append(data.substr(pos,1));
+
+ // when the garbage is passed we expect to start seeing first-line bytes
+ if (pos == garbageEnd) {
+ expect.parserState = Http1::HTTP_PARSE_FIRST;
+ expect.msgStart = 0;
+ }
+
+ // all points after garbage start to see accumulated bytes looking for end of current section
+ if (pos >= garbageEnd)
+ expect.suffixSz = ioBuf.length();
+
+ // at end of request line expect to see method, URI, version details
+ // and switch to seeking Mime header section
+ if (pos == reqLineEnd) {
+ expect.parserState = Http1::HTTP_PARSE_MIME;
+ expect.suffixSz = 0;
+ expect.msgEnd = reqLineEnd-garbageEnd;
+ expect.status = Http::scOkay;
+ expect.methodStart = 0;
+ expect.methodEnd = 2;
+ expect.method = HttpRequestMethod(Http::METHOD_GET);
+ expect.uriStart = 4;
+ expect.uriEnd = 22;
+ expect.uri = "http://example.com/";
+ expect.versionStart = 24;
+ expect.versionEnd = 31;
+ expect.version = AnyP::ProtocolVersion(AnyP::PROTO_HTTP,1,1);
+ }
+
+ // one mime header is done we are expectign a new request
+ // parse results say true and initial data is all gone from the buffer
+ if (pos == mimeEnd) {
+ expect.parsed = true;
+ expect.needsMore = false;
+ expect.suffixSz = 0;
+ }
+
+ testResults(__LINE__, ioBuf, hp, expect);
+
+ // sync the buffers like Squid does
+ ioBuf = hp.remaining();
+
+ // Squid stops using the parser once it has parsed the first message.
+ if (!hp.needsMoreData())
+ break;
}
}
=== renamed file 'src/tests/testHttpParser.h' => 'src/tests/testHttp1Parser.h'
--- src/tests/testHttpParser.h 2014-09-12 23:00:48 +0000
+++ src/tests/testHttp1Parser.h 2014-10-11 19:23:50 +0000
@@ -1,35 +1,41 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
-#ifndef SQUID_SRC_TESTS_TESTHTTPPARSER_H
-#define SQUID_SRC_TESTS_TESTHTTPPARSER_H
+#ifndef SQUID_SRC_TESTS_TESTHTTP1PARSER_H
+#define SQUID_SRC_TESTS_TESTHTTP1PARSER_H
#include <cppunit/extensions/HelperMacros.h>
-class testHttpParser : public CPPUNIT_NS::TestFixture
+class testHttp1Parser : public CPPUNIT_NS::TestFixture
{
- CPPUNIT_TEST_SUITE( testHttpParser );
+ CPPUNIT_TEST_SUITE( testHttp1Parser );
+ CPPUNIT_TEST( testParserConstruct );
CPPUNIT_TEST( testParseRequestLineTerminators );
CPPUNIT_TEST( testParseRequestLineMethods );
CPPUNIT_TEST( testParseRequestLineProtocols );
CPPUNIT_TEST( testParseRequestLineStrange );
CPPUNIT_TEST( testParseRequestLineInvalid );
+ CPPUNIT_TEST( testDripFeed );
CPPUNIT_TEST_SUITE_END();
protected:
void globalSetup(); // MemPools init etc.
+ void testParserConstruct(); // whether the constructor works
+
// request-line unit tests
void testParseRequestLineTerminators(); // terminator detection correct
void testParseRequestLineMethods(); // methoid detection correct
void testParseRequestLineProtocols(); // protocol tokens handled correctly
void testParseRequestLineStrange(); // strange but valid lines accepted
void testParseRequestLineInvalid(); // rejection of invalid lines happens
+
+ void testDripFeed(); // test incremental parse works
};
#endif
=== modified file 'src/tests/testHttpRequestMethod.cc'
--- src/tests/testHttpRequestMethod.cc 2014-09-12 23:00:48 +0000
+++ src/tests/testHttpRequestMethod.cc 2014-10-15 13:39:59 +0000
@@ -1,67 +1,92 @@
/*
* Copyright (C) 1996-2014 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
#include "squid.h"
#include <cppunit/TestAssert.h>
-#include "HttpRequestMethod.h"
+#include "http/RequestMethod.h"
#include "Mem.h"
#include "SquidConfig.h"
#include "testHttpRequestMethod.h"
#include <sstream>
CPPUNIT_TEST_SUITE_REGISTRATION( testHttpRequestMethod );
/*
* We should be able to make an HttpRequestMethod straight from a string.
*/
void
testHttpRequestMethod::testConstructCharStart()
{
+ // string in SBuf
+
/* parse an empty string -> Http::METHOD_NONE */
- CPPUNIT_ASSERT(HttpRequestMethod(NULL,NULL) == Http::METHOD_NONE);
+ CPPUNIT_ASSERT(HttpRequestMethod(SBuf()) == Http::METHOD_NONE);
+
+ /* parsing a literal should work */
+ CPPUNIT_ASSERT(HttpRequestMethod(SBuf("GET")) == Http::METHOD_GET);
+ CPPUNIT_ASSERT(HttpRequestMethod(SBuf("QWERTY")) == Http::METHOD_OTHER);
+
+ // string in char*
+
+ /* parse an empty string -> Http::METHOD_NONE */
+ HttpRequestMethod a;
+ a.HttpRequestMethodXXX(NULL);
+ CPPUNIT_ASSERT(a == Http::METHOD_NONE);
+
/* parsing a literal should work */
- CPPUNIT_ASSERT(HttpRequestMethod("GET", NULL) == Http::METHOD_GET);
- CPPUNIT_ASSERT(HttpRequestMethod("QWERTY", NULL) == Http::METHOD_OTHER);
+ HttpRequestMethod b;
+ b.HttpRequestMethodXXX("GET");
+ CPPUNIT_ASSERT(b == Http::METHOD_GET);
+ CPPUNIT_ASSERT_EQUAL(SBuf("GET"), b.image());
+ HttpRequestMethod c;
+ c.HttpRequestMethodXXX("QWERTY");
+ CPPUNIT_ASSERT(c == Http::METHOD_OTHER);
+ CPPUNIT_ASSERT_EQUAL(SBuf("QWERTY"), c.image());
+
+ // parsing error should not leave stale results
+ b.HttpRequestMethodXXX(NULL);
+ CPPUNIT_ASSERT(b == Http::METHOD_NONE);
+ CPPUNIT_ASSERT_EQUAL(SBuf("NONE"), b.image());
}
/*
- * We can also parse precise ranges of characters
+ * We can also parse precise ranges of characters with SBuf
*/
void
testHttpRequestMethod::testConstructCharStartEnd()
{
char const * buffer;
/* parse an empty string -> Http::METHOD_NONE */
- CPPUNIT_ASSERT(HttpRequestMethod(NULL, NULL) == Http::METHOD_NONE);
+ CPPUNIT_ASSERT(HttpRequestMethod(SBuf()) == Http::METHOD_NONE);
/* parsing a literal should work */
- CPPUNIT_ASSERT(HttpRequestMethod("GET", NULL) == Http::METHOD_GET);
+ CPPUNIT_ASSERT(HttpRequestMethod(SBuf("GET")) == Http::METHOD_GET);
/* parsing with an explicit end should work */
buffer = "POSTPLUS";
- CPPUNIT_ASSERT(HttpRequestMethod(buffer, buffer + 4) == Http::METHOD_POST);
+ CPPUNIT_ASSERT(HttpRequestMethod(SBuf(buffer, 4)) == Http::METHOD_POST);
}
/*
* we should be able to assign a Http::MethodType to a HttpRequestMethod
*/
void
testHttpRequestMethod::testAssignFrommethod_t()
{
HttpRequestMethod method;
method = Http::METHOD_NONE;
CPPUNIT_ASSERT_EQUAL(HttpRequestMethod(Http::METHOD_NONE), method);
method = Http::METHOD_POST;
CPPUNIT_ASSERT_EQUAL(HttpRequestMethod(Http::METHOD_POST), method);
}
/*
* a default constructed HttpRequestMethod is == Http::METHOD_NONE
*/
void
testHttpRequestMethod::testDefaultConstructor()
@@ -73,74 +98,74 @@
/*
* we should be able to construct a HttpRequestMethod from a Http::MethodType
*/
void
testHttpRequestMethod::testConstructmethod_t()
{
CPPUNIT_ASSERT_EQUAL(HttpRequestMethod(Http::METHOD_NONE), HttpRequestMethod(Http::METHOD_NONE));
CPPUNIT_ASSERT_EQUAL(HttpRequestMethod(Http::METHOD_POST), HttpRequestMethod(Http::METHOD_POST));
CPPUNIT_ASSERT(HttpRequestMethod(Http::METHOD_NONE) != HttpRequestMethod(Http::METHOD_POST));
}
/*
* we should be able to get a char const * version of the method.
*/
void
testHttpRequestMethod::testImage()
{
// relaxed RFC-compliance parse HTTP methods are upgraded to correct case
Config.onoff.relaxed_header_parser = 1;
- CPPUNIT_ASSERT_EQUAL(SBuf("POST"), HttpRequestMethod("POST",NULL).image());
- CPPUNIT_ASSERT_EQUAL(SBuf("POST"), HttpRequestMethod("pOsT",NULL).image());
- CPPUNIT_ASSERT_EQUAL(SBuf("POST"), HttpRequestMethod("post",NULL).image());
+ CPPUNIT_ASSERT_EQUAL(SBuf("POST"), HttpRequestMethod(SBuf("POST")).image());
+ CPPUNIT_ASSERT_EQUAL(SBuf("POST"), HttpRequestMethod(SBuf("pOsT")).image());
+ CPPUNIT_ASSERT_EQUAL(SBuf("POST"), HttpRequestMethod(SBuf("post")).image());
// strict RFC-compliance parse HTTP methods are case sensitive
Config.onoff.relaxed_header_parser = 0;
- CPPUNIT_ASSERT_EQUAL(SBuf("POST"), HttpRequestMethod("POST",NULL).image());
- CPPUNIT_ASSERT_EQUAL(SBuf("pOsT"), HttpRequestMethod("pOsT",NULL).image());
- CPPUNIT_ASSERT_EQUAL(SBuf("post"), HttpRequestMethod("post",NULL).image());
+ CPPUNIT_ASSERT_EQUAL(SBuf("POST"), HttpRequestMethod(SBuf("POST")).image());
+ CPPUNIT_ASSERT_EQUAL(SBuf("pOsT"), HttpRequestMethod(SBuf("pOsT")).image());
+ CPPUNIT_ASSERT_EQUAL(SBuf("post"), HttpRequestMethod(SBuf("post")).image());
}
/*
* an HttpRequestMethod should be comparable to a Http::MethodType without false
* matches
*/
void
testHttpRequestMethod::testEqualmethod_t()
{
CPPUNIT_ASSERT(HttpRequestMethod(Http::METHOD_NONE) == Http::METHOD_NONE);
CPPUNIT_ASSERT(not (HttpRequestMethod(Http::METHOD_POST) == Http::METHOD_GET));
CPPUNIT_ASSERT(HttpRequestMethod(Http::METHOD_GET) == Http::METHOD_GET);
CPPUNIT_ASSERT(not (HttpRequestMethod(Http::METHOD_TRACE) == Http::METHOD_SEARCH));
}
/*
* an HttpRequestMethod should testable for inequality without fail maatches
*/
void
testHttpRequestMethod::testNotEqualmethod_t()
{
CPPUNIT_ASSERT(HttpRequestMethod(Http::METHOD_NONE) != Http::METHOD_GET);
CPPUNIT_ASSERT(not (HttpRequestMethod(Http::METHOD_POST) != Http::METHOD_POST));
CPPUNIT_ASSERT(HttpRequestMethod(Http::METHOD_GET) != Http::METHOD_NONE);
CPPUNIT_ASSERT(not (HttpRequestMethod(Http::METHOD_SEARCH) != Http::METHOD_SEARCH));
}
/*
* we should be able to send it to a stream and get the normalised version
*/
void
testHttpRequestMethod::testStream()
{
// relaxed RFC-compliance parse HTTP methods are upgraded to correct case
Config.onoff.relaxed_header_parser = 1;
std::ostringstream buffer;
- buffer << HttpRequestMethod("get", NULL);
+ buffer << HttpRequestMethod(SBuf("get"));
CPPUNIT_ASSERT_EQUAL(String("GET"), String(buffer.str().c_str()));
// strict RFC-compliance parse HTTP methods are case sensitive
Config.onoff.relaxed_header_parser = 0;
std::ostringstream buffer2;
- buffer2 << HttpRequestMethod("get", NULL);
+ buffer2 << HttpRequestMethod(SBuf("get"));
CPPUNIT_ASSERT_EQUAL(String("get"), String(buffer2.str().c_str()));
}
More information about the squid-dev
mailing list