[squid-users] Squid and multipart form decode
Ryan Le
ryanlele264 at gmail.com
Wed Jul 29 16:47:40 UTC 2020
I do apologize I do not have logs of that specific file but I have an
example from the same site and doing the same post.
> Even though it looks like TeChunkedParser is getting all the
> additional headers
>TeChunkedParser has nothing to do with multipart/form-data bodies.
>TeChunkedParser parses chunked encoding, and even then it is applied to
>remove _transfer_ encoding, not to interpret the actual resource content
>inside the chunks.
I do see it in two locations.
2020/07/26 23:11:12.921 kid6| 74,9| TeChunkedParser.cc(45) parse: Parse
buf={length=3667, data='e47
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="action"
frm_submit_dropzone
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="field_id"
8
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="form_id"
5
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="nonce"
6bb20c0bd7
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="file8"; filename="file.zip"
Content-Type: application/x-zip-compressed
As well as the following location
2020/07/26 23:11:12.921 kid6| 58,9| HttpMsg.cc(198) parse: HttpMsg::parse
success (689 bytes) near 'POST http://bbbb.com/post HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0)
Gecko/20100101 Firefox/78.0
Accept: application/json
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cache-Control: no-cache
X-Requested-With: XMLHttpRequest
Content-Type: multipart/form-data;
boundary=---------------------------351645264024548376901231954897
Content-Length: 3655
Origin: http://bbbbbb.com
Referer: http://bbbbb.com.com/http-post/
Cookie: _ga=GA1.2.1194289608.1595640198; _gid=GA1.2.252592555.1595804428;
_gat_gtag_UA_47458108_3=1
Host: bbbbbbb.com
Via: ICAP/1.0 (C-ICAP/0.5.6 ICAP service )
e47
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="action"
frm_submit_dropzone
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="field_id"
8
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="form_id"
5
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="nonce"
6bb20c0bd7
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="file8"; filename="file.zip"
Content-Type: application/x-zip-compressed
>I am not sure, but it looks like you have pasted a part of an ICAP
>message. TeChunkedParser is used to parse chunked transfer encoding used
>for a part of the ICAP message body. Beyond decoding those chunks, it is
>all opaque data to Squid.
Thanks for that information.
>To avoid misunderstanding, in your pasted example, the contents of the
>first chunk starts with these two lines:
> -----------------------------328901485836611227811186534509
> Content-Disposition: form-data; name="action"
>It does _not_ start with the "Content-Disposition:..." line or the
>"frm_submit_dropzone" line.
> I can't seem to create ACL or output them using
> logformat. I was trying to request these headers with
> req_mime_type/resp_mime_type.
>If by "them" you mean MIME headers inside multipart parts, then Squid
>does not see them and does not operate on them. The insides of each
>chunk is opaque data to Squid.
> and also had log_mime_hdrs on and then in
> logformat just had all.
>You should be able to log the HTTP request header values using %>h or
>%>ha. You will not be able to log or match any message body snippets,
>including things like MIME Content-Disposition values. Squid does not
>look inside the body of the POSTed resource.
I will test with the two examples given and see what they return.
>If you need further help, you may want to clarify what you are trying to
>achieve. You said "send multipart form data to another service". Are you
>trying to _route_ request messages based on multipart form _contents_?
What I am ultimately trying to accomplish is to see the best way to get
more detail and have an action on sites
that are posting using multipart/form-data as the Content-Type header. This
is mainly to separate action taken on an actual form being submitted
versus a file being submitted or as you stated route request messages based
on the content.
Whether that be logformat with headers and passing the headers to a custom
external service or within squid itself.
On Wed, Jul 29, 2020 at 12:16 PM Alex Rousskov <
rousskov at measurement-factory.com> wrote:
> On 7/29/20 11:38 AM, Ryan Le wrote:
> > Even though it looks like TeChunkedParser is getting all the
> > additional headers
>
> TeChunkedParser has nothing to do with multipart/form-data bodies.
> TeChunkedParser parses chunked encoding, and even then it is applied to
> remove _transfer_ encoding, not to interpret the actual resource content
> inside the chunks.
>
> I am not sure, but it looks like you have pasted a part of an ICAP
> message. TeChunkedParser is used to parse chunked transfer encoding used
> for a part of the ICAP message body. Beyond decoding those chunks, it is
> all opaque data to Squid.
>
> To avoid misunderstanding, in your pasted example, the contents of the
> first chunk starts with these two lines:
>
> > -----------------------------328901485836611227811186534509
> > Content-Disposition: form-data; name="action"
>
> It does _not_ start with the "Content-Disposition:..." line or the
> "frm_submit_dropzone" line.
>
>
> > I can't seem to create ACL or output them using
> > logformat. I was trying to request these headers with
> > req_mime_type/resp_mime_type.
>
> If by "them" you mean MIME headers inside multipart parts, then Squid
> does not see them and does not operate on them. The insides of each
> chunk is opaque data to Squid.
>
>
> > and alos had log_mime_hdrs on and then in
> > logformat just had all.
>
> You should be able to log the HTTP request header values using %>h or
> %>ha. You will not be able to log or match any message body snippets,
> including things like MIME Content-Disposition values. Squid does not
> look inside the body of the POSTed resource.
>
>
> If you need further help, you may want to clarify what you are trying to
> achieve. You said "send multipart form data to another service". Are you
> trying to _route_ request messages based on multipart form _contents_?
>
>
> HTH,
>
> Alex.
>
>
> > On Thu, Jul 23, 2020 at 11:46 AM Ryan Le wrote:
> >
> > Thanks,
> >
> > I have been looking at the squid debug and can see that it is
> > getting the multipart.
> >
> > POST http://bbbbbb.com
> > User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0)
> > Gecko/20100101 Firefox/78.0
> > Accept: application/json
> > Accept-Language: en-US,en;q=0.5
> > Accept-Encoding: gzip, deflate
> > Referer: http://bbbbb.com
> > Cache-Control: no-cache
> > X-Requested-With: XMLHttpRequest
> > Content-Type: multipart/form-data;
> > boundary=---------------------------328901485836611227811186534509
> > Content-Length: 1245
> > Origin: http://bbbbb.com
> > Cookie: cookie
> > Host: bbbbbbb.com <http://bbbbbbb.com>
> > Via: ICAP/1.0
> >
> > 4dd
> > -----------------------------328901485836611227811186534509
> > Content-Disposition: form-data; name="action"
> >
> > frm_submit_dropzone
> > -----------------------------328901485836611227811186534509
> > Content-Disposition: form-data; name="field_id"
> >
> > 8
> > -----------------------------328901485836611227811186534509
> > Content-Disposition: form-data; name="form_id"
> >
> > 5
> > -----------------------------328901485836611227811186534509
> > Content-Disposition: form-data; name="nonce"
> >
> > e1aca92777
> > -----------------------------328901485836611227811186534509
> > Content-Disposition: form-data; name="file8";
> filename="translate.zip"
> > Content-Type: application/x-zip-compressed
> >
> > On Thu, Jul 23, 2020 at 11:16 AM Alex Rousskov
> > <rousskov at measurement-factory.com
> > <mailto:rousskov at measurement-factory.com>> wrote:
> >
> > On 7/23/20 9:22 AM, Ryan Le wrote:
> > > I have been trying to configure squid to decode and send
> > multipart form
> > > data to another service. Is there an acl or build parameter
> > needed for
> > > multipart form data support?
> >
> > No, there is no need to allow any specific Content-Type,
> including
> > multipart. Squid does not know anything about
> > multipart/form-data. If a
> > multipart/form-data message is well-formed from HTTP point of
> > view, then
> > Squid will process it as any other message, including passing it
> to
> > ICAP/eCAP (where configured).
> >
> > Cheers,
> >
> > Alex.
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20200729/64679e12/attachment-0001.htm>
More information about the squid-users
mailing list