[squid-users] Squid and multipart form decode

Ryan Le ryanlele264 at gmail.com
Wed Jul 29 16:47:40 UTC 2020


I do apologize I do not have logs of that specific file but I have an
example from the same site and doing the same post.

> Even though it looks like TeChunkedParser is getting all the
> additional headers

>TeChunkedParser has nothing to do with multipart/form-data bodies.
>TeChunkedParser parses chunked encoding, and even then it is applied to
>remove _transfer_ encoding, not to interpret the actual resource content
>inside the chunks.

I do see it in two locations.

2020/07/26 23:11:12.921 kid6| 74,9| TeChunkedParser.cc(45) parse: Parse
buf={length=3667, data='e47
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="action"

frm_submit_dropzone
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="field_id"

8
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="form_id"

5
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="nonce"

6bb20c0bd7
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="file8"; filename="file.zip"
Content-Type: application/x-zip-compressed

As well as the following location

2020/07/26 23:11:12.921 kid6| 58,9| HttpMsg.cc(198) parse: HttpMsg::parse
success (689 bytes) near 'POST http://bbbb.com/post HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0)
Gecko/20100101 Firefox/78.0
Accept: application/json
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cache-Control: no-cache
X-Requested-With: XMLHttpRequest
Content-Type: multipart/form-data;
boundary=---------------------------351645264024548376901231954897
Content-Length: 3655
Origin: http://bbbbbb.com
Referer: http://bbbbb.com.com/http-post/
Cookie: _ga=GA1.2.1194289608.1595640198; _gid=GA1.2.252592555.1595804428;
_gat_gtag_UA_47458108_3=1
Host: bbbbbbb.com
Via: ICAP/1.0 (C-ICAP/0.5.6 ICAP service )

e47
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="action"

frm_submit_dropzone
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="field_id"

8
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="form_id"

5
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="nonce"

6bb20c0bd7
-----------------------------351645264024548376901231954897
Content-Disposition: form-data; name="file8"; filename="file.zip"
Content-Type: application/x-zip-compressed


>I am not sure, but it looks like you have pasted a part of an ICAP
>message. TeChunkedParser is used to parse chunked transfer encoding used
>for a part of the ICAP message body. Beyond decoding those chunks, it is
>all opaque data to Squid.

Thanks for that information.


>To avoid misunderstanding, in your pasted example, the contents of the
>first chunk starts with these two lines:

> -----------------------------328901485836611227811186534509
> Content-Disposition: form-data; name="action"

>It does _not_ start with the "Content-Disposition:..." line or the
>"frm_submit_dropzone" line.


> I can't seem to create ACL or output them using
> logformat. I was trying to request these headers with
> req_mime_type/resp_mime_type.

>If by "them" you mean MIME headers inside multipart parts, then Squid
>does not see them and does not operate on them. The insides of each
>chunk is opaque data to Squid.


> and also had log_mime_hdrs on and then in
> logformat just had all.

>You should be able to log the HTTP request header values using %>h or
>%>ha. You will not be able to log or match any message body snippets,
>including things like MIME Content-Disposition values. Squid does not
>look inside the body of the POSTed resource.

I will test with the two examples given and see what they return.


>If you need further help, you may want to clarify what you are trying to
>achieve. You said "send multipart form data to another service". Are you
>trying to _route_ request messages based on multipart form _contents_?

What I am ultimately trying to accomplish is to see the best way to get
more detail and have an action on sites
that are posting using multipart/form-data as the Content-Type header. This
is mainly to separate action taken on an actual form being submitted
versus a file being submitted or as you stated route request messages based
on the content.
Whether that be logformat with headers and passing the headers to a custom
external service or within squid itself.

On Wed, Jul 29, 2020 at 12:16 PM Alex Rousskov <
rousskov at measurement-factory.com> wrote:

> On 7/29/20 11:38 AM, Ryan Le wrote:
> > Even though it looks like TeChunkedParser is getting all the
> > additional headers
>
> TeChunkedParser has nothing to do with multipart/form-data bodies.
> TeChunkedParser parses chunked encoding, and even then it is applied to
> remove _transfer_ encoding, not to interpret the actual resource content
> inside the chunks.
>
> I am not sure, but it looks like you have pasted a part of an ICAP
> message. TeChunkedParser is used to parse chunked transfer encoding used
> for a part of the ICAP message body. Beyond decoding those chunks, it is
> all opaque data to Squid.
>
> To avoid misunderstanding, in your pasted example, the contents of the
> first chunk starts with these two lines:
>
> > -----------------------------328901485836611227811186534509
> > Content-Disposition: form-data; name="action"
>
> It does _not_ start with the "Content-Disposition:..." line or the
> "frm_submit_dropzone" line.
>
>
> > I can't seem to create ACL or output them using
> > logformat. I was trying to request these headers with
> > req_mime_type/resp_mime_type.
>
> If by "them" you mean MIME headers inside multipart parts, then Squid
> does not see them and does not operate on them. The insides of each
> chunk is opaque data to Squid.
>
>
> > and alos had log_mime_hdrs on and then in
> > logformat just had all.
>
> You should be able to log the HTTP request header values using %>h or
> %>ha. You will not be able to log or match any message body snippets,
> including things like MIME Content-Disposition values. Squid does not
> look inside the body of the POSTed resource.
>
>
> If you need further help, you may want to clarify what you are trying to
> achieve. You said "send multipart form data to another service". Are you
> trying to _route_ request messages based on multipart form _contents_?
>
>
> HTH,
>
> Alex.
>
>
> > On Thu, Jul 23, 2020 at 11:46 AM Ryan Le wrote:
> >
> >     Thanks,
> >
> >     I have been looking at the squid debug and can see that it is
> >     getting the multipart.
> >
> >     POST http://bbbbbb.com
> >     User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0)
> >     Gecko/20100101 Firefox/78.0
> >     Accept: application/json
> >     Accept-Language: en-US,en;q=0.5
> >     Accept-Encoding: gzip, deflate
> >     Referer: http://bbbbb.com
> >     Cache-Control: no-cache
> >     X-Requested-With: XMLHttpRequest
> >     Content-Type: multipart/form-data;
> >     boundary=---------------------------328901485836611227811186534509
> >     Content-Length: 1245
> >     Origin: http://bbbbb.com
> >     Cookie: cookie
> >     Host: bbbbbbb.com <http://bbbbbbb.com>
> >     Via: ICAP/1.0
> >
> >     4dd
> >     -----------------------------328901485836611227811186534509
> >     Content-Disposition: form-data; name="action"
> >
> >     frm_submit_dropzone
> >     -----------------------------328901485836611227811186534509
> >     Content-Disposition: form-data; name="field_id"
> >
> >     8
> >     -----------------------------328901485836611227811186534509
> >     Content-Disposition: form-data; name="form_id"
> >
> >     5
> >     -----------------------------328901485836611227811186534509
> >     Content-Disposition: form-data; name="nonce"
> >
> >     e1aca92777
> >     -----------------------------328901485836611227811186534509
> >     Content-Disposition: form-data; name="file8";
> filename="translate.zip"
> >     Content-Type: application/x-zip-compressed
> >
> >     On Thu, Jul 23, 2020 at 11:16 AM Alex Rousskov
> >     <rousskov at measurement-factory.com
> >     <mailto:rousskov at measurement-factory.com>> wrote:
> >
> >         On 7/23/20 9:22 AM, Ryan Le wrote:
> >         > I have been trying to configure squid to decode and send
> >         multipart form
> >         > data to another service. Is there an acl or build parameter
> >         needed for
> >         > multipart form data support?
> >
> >         No, there is no need to allow any specific Content-Type,
> including
> >         multipart. Squid does not know anything about
> >         multipart/form-data. If a
> >         multipart/form-data message is well-formed from HTTP point of
> >         view, then
> >         Squid will process it as any other message, including passing it
> to
> >         ICAP/eCAP (where configured).
> >
> >         Cheers,
> >
> >         Alex.
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20200729/64679e12/attachment-0001.htm>


More information about the squid-users mailing list