[squid-users] "helperHandleRead: unexpected read" for helper function

[Alex Rousskov]

On 5/7/21 10:22 PM, Justin Michael Schwartzbeck wrote:

> So I have written an external acl helper

> while(1) {
>     string category, hostname;
>     cin >> category >> hostname;
>     
>     // Perform REST API


> I have it all configured in my squid.conf:
> external_acl_type host_category_helper %ACL %DST
> /usr/local/bin/squidhelpers/host_category_helper
> acl searchengines external host_category_helper

> I am seeing a million of the following lines:
> 2021/05/08 02:04:57| helperHandleRead: unexpected read from
> host_category_helper #Hlpr6, 4 bytes 'ERR

AFAICT, your helper does not validate its input in any way. Until you
fix that critical design flaw, you may face surprises like this even if
things appear to work in your tests. Check that each line contains the
number of fields you expect (which is 2 AFAICT) _and_ nothing else.

I suspect I know where the immediate bug is, but I think it is best if I
keep those details to myself as a motivation for you to add input
validation :-).


BTW, you should also handle input EOF, which is how Squid gracefully
terminates helpers IIRC.


> I saw also some snippet about concurrency being an issue, ...
> But this doesn't make sense to me since only one helper appears to be
> running:

In this context, concurrency is about concurrent transactions (a.k.a
requests or queries) sent to a single helper instance, not about
concurrent helper instances.

Documentation is rather vague on this point, but, AFAICT, external
helpers are assumed to lack concurrency support by default, so this
should not be an issue for your specific helper configuration. Once you
add validation, you will know for sure.

Said that, your helper does not support and silently breaks concurrent
requests.


> I would have assumed that each helper (there are 5
> of them) would have their own stdin/stdout and be writing to/from it
> rather than everyone at once. 

Your assumption is correct.


> you are supposed to print ERR when there is not a match, correct?

Correct.

On invalid input, respond with BH if you think your helper can recover
from that bad input. Respond with BH _and_ quit the helper otherwise.


HTH,

Alex.