[squid-users] LEGACY_SERVER_CONNECT, ALLOW_UNSAFE_LEGACY_RENEGOTIATION does not work - SSL bump, OpenSSL 3

Alex Rousskov rousskov at measurement-factory.com
Wed Dec 28 17:27:43 UTC 2022 

Hi Amish,

     TLS options are used on _both_ sides, in various cases, but there 
are still too many unknowns, and I cannot quickly answer all of your 
questions at once. Sorry. I can only guide you one step at a time.

* If you are sure that SSL_CTX_set_options() is not called when talking 
to the origin server in your setup, then there is no need to hard-code 
those options. Instead, we need to figure out why Squid does not call 
SSL_CTX_set_options(). The best next step is to configure your Squid 
with the following ssl_bump rules and check whether 
SSL_CTX_set_options() is called for the Squid-server connection in that 
case. Sharing access.log records to double check that the origin server 
was contacted may be a good idea.

     ssl_bump stare all
     ssl_bump bump all

* If you are not sure whether SSL_CTX_set_options() is called when 
talking to the origin server in your setup, then hard-coding those 
options may still be the best next step. If it fixes the problem, then 
we will be able to resolve a few variables without the test in the first 
bullet.


HTH,

Alex.


On 12/28/22 11:49, Amish wrote:
> Hi Alex,
> 
> On 28/12/22 21:31, Alex Rousskov wrote:
>> Hi Amish,
>>
>>     Squid parsing code is tricky. tls_outgoing_options parsing code is 
>> triply so. Even its authors misinterpret it!
>>
>> I assume you have removed multiple tls_outgoing_options directives 
>> from your configuration before testing. If you have not, please merge 
>> those directives into one and retest. You should still see multiple 
>> parsing paths, in part, due to (unfortunate) Security::PeerOptions 
>> implementation and, in part, due to Squid parsing default options 
>> before Squid parses your actual configuration files.
>>
> Yes I had combined all tls_outgoing_options in to a single directive 
> (but in multiple lines ending with a backslash \). It can be seen in the 
> grep command in my previous email.
> 
> But wait, I have found something.
> 
> And I have a doubt that parsedOptions work only for client to squid side 
> of the connection and does not work for squid to server side of the 
> connection.
> 
> What I did is changed my "http(s)_port" directives to include 
> options=0x4. These directives control options for the client to squid 
> side of TLS connection.
> 
> $ grep 'ssl-bump' /etc/squid/squid.conf
> http_port 8080 ssl-bump generate-host-certificates=on 
> tls-cert=/etc/squid/ssl_cert/squid.pem 
> tls-dh=/etc/squid/ssl_cert/dhparam.pem options=0x4
> https_port 8081 intercept ssl-bump generate-host-certificates=on 
> tls-cert=/etc/squid/ssl_cert/squid.pem 
> tls-dh=/etc/squid/ssl_cert/dhparam.pem options=0x4
> 
> Now lets reproduce the issue:
> $ curl --no-progress-meter -kx 127.0.0.1:8080 https://www.jio.com |grep 
> 'TLS\|SSL'
> <pre>[No Error] (TLS code: 
> SQUID_TLS_ERR_CONNECT+TLS_LIB_ERR=A000152+TLS_IO_ERR=1)</pre>
> <p>Failed to establish a secure connection: error:0A000152:SSL 
> routines::unsafe legacy renegotiation disabled</p>
> 
> Lets see what cache.log has to say:
> 
> $ tail -f /var/log/squid/cache.log  |grep -i 'openssl\|parsed'
> 2022/12/28 21:52:56.532 kid1| 83,5| PeerOptions.cc(447) parseOptions: 
> INFO: TLS parsedOptions(1)=4
> 2022/12/28 21:52:56.532 kid1| 83,5| PeerOptions.cc(643) 
> updateContextOptions: set OpenSSL options for context=0x559075043e30, 
> parsedOptions=4
> 2022/12/28 21:52:56.532 kid1| 83,5| PeerOptions.cc(645) 
> updateContextOptions: get OpenSSL options for context=0x559075043e30, 
> getOptions=1179652
> 
> Bingo!! As we can see, parsedOptions is now set to 4!
> 
> And it is also confirmed by ssl_get_options() - 1179652  = 0x120004 (4 
> means the option is accepted by OpenSSL too)
> 
> But this also means that updateContextOptions() is called ONLY FOR 
> client to squid side (curl to squid) and it is not called for squid to 
> server (jio.com in this case) side.
> 
> The reason being that we do not see updateContextOptions() being called 
> twice for a request. But only once. And that is why request still fails 
> with the negotiation error.
> 
> So where exactly is the call for squid to server side being made?
> 
>> If merging directives does not deliver custom options to 
>> SSL_CTX_set_options(), then let's attack this from the other end: 
>> Supply the right options to each SSL_CTX_set_options() call:
>>
>>     const Security::ParsedOptions forcedParsedOptions = 0x4 | 0x40000;
>>     SSL_CTX_set_options(ctx.get(), forcedParsedOptions);
>>
>> Does the above temporary hack fix the problem in your test?
> 
> 
> I will try this tomorrow (its late night here).
> 
> But I think I will have to set this somewhere else and NOT in 
> PeerOptions.cc. Because above code appears to be for client to squid side.
> 
> And I need to add forcedParsedOptions to the code which connects squid 
> to server. But where?
> 
> Thank you,
> 
> Amish.
> 
> 
>> Thank you,
>>
>> Alex.
>>
>> On 12/28/22 02:32, Amish wrote:
>>> Hi Alex,
>>>
>>> Thanks again for your reply.
>>>
>>> To find answers to your questions, I added few debugs() lines to 
>>> PeerOptions.cc.
>>>
>>> The diff file (patch) is attached.
>>>
>>> It prints parsedOptions and options retrieved from SSL context and 
>>> session objects at several stages.
>>>
>>> Here is tls_outgoing_options setting:
>>>
>>> $ grep tls_outgoing_options /etc/squid/squid.conf
>>> tls_outgoing_options \
>>>      cafile=/etc/ssl/cert.pem \
>>> cipher=ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS \
>>>      options=0x4
>>>
>>>
>>> Here is what squid logs on reload i.e. on parsing the squid.conf
>>>
>>> $ systemctl reload squid
>>> $ tail -f /var/log/squid/cache.log  |grep -i 'openssl\|parsed'
>>> 2022/12/28 12:19:30.596 kid1| 83,5| PeerOptions.cc(547) parseOptions: 
>>> INFO: TLS parsedOptions(3)=0
>>> 2022/12/28 12:19:30.598 kid1| 83,5| PeerOptions.cc(547) parseOptions: 
>>> INFO: TLS parsedOptions(3)=0
>>> 2022/12/28 12:19:30.723 kid1| 83,5| PeerOptions.cc(547) parseOptions: 
>>> INFO: TLS parsedOptions(3)=4
>>> 2022/12/28 12:19:30.729 kid1| 83,5| PeerOptions.cc(547) parseOptions: 
>>> INFO: TLS parsedOptions(3)=0
>>> 2022/12/28 12:19:30.729 kid1| 83,5| PeerOptions.cc(547) parseOptions: 
>>> INFO: TLS parsedOptions(3)=0
>>> 2022/12/28 12:19:32.147 kid1| 83,5| PeerOptions.cc(447) parseOptions: 
>>> INFO: TLS parsedOptions(1)=0
>>> 2022/12/28 12:19:32.147 kid1| 83,5| PeerOptions.cc(547) parseOptions: 
>>> INFO: TLS parsedOptions(3)=0
>>> 2022/12/28 12:19:33.524 kid1| 83,5| PeerOptions.cc(447) parseOptions: 
>>> INFO: TLS parsedOptions(1)=0
>>> 2022/12/28 12:19:33.532 kid1| 83,5| PeerOptions.cc(547) parseOptions: 
>>> INFO: TLS parsedOptions(3)=0
>>> 2022/12/28 12:19:33.695 kid1| 83,5| PeerOptions.cc(447) parseOptions: 
>>> INFO: TLS parsedOptions(1)=0
>>> 2022/12/28 12:19:33.695 kid1| 83,5| PeerOptions.cc(643) 
>>> updateContextOptions: set OpenSSL options for context=0x562a5387fe30, 
>>> parsedOptions=0
>>> 2022/12/28 12:19:33.695 kid1| 83,5| PeerOptions.cc(645) 
>>> updateContextOptions: get OpenSSL options for context=0x562a5387fe30, 
>>> getOptions=1179648
>>
>>> 2022/12/28 12:19:33.708 kid1| 83,5| PeerOptions.cc(447) parseOptions: 
>>> INFO: TLS parsedOptions(1)=0
>>> 2022/12/28 12:19:33.708 kid1| 83,5| PeerOptions.cc(643) 
>>> updateContextOptions: set OpenSSL options for context=0x562a53e6e740, 
>>> parsedOptions=0
>>> 2022/12/28 12:19:33.708 kid1| 83,5| PeerOptions.cc(645) 
>>> updateContextOptions: get OpenSSL options for context=0x562a53e6e740, 
>>> getOptions=1179648
>>>
>>> It seems that squid parses the options multiple times and only once 
>>> it gets the value as 4. Rest are parsed as 0.
>>>
>>> The value of 1179648 (0x120000) corresponds to SSL_OP_NO_COMPRESSION 
>>> (0x20000) and SSL_OP_ENABLE_MIDDLEBOX_COMPAT. (0x100000)
>>>
>>>
>>> Now lets reproduce the issue:
>>>
>>> $ curl --no-progress-meter -kx 127.0.0.1:8080 https://www.jio.com 
>>> |grep 'TLS\|SSL'
>>> <pre>[No Error] (TLS code: 
>>> SQUID_TLS_ERR_CONNECT+TLS_LIB_ERR=A000152+TLS_IO_ERR=1)</pre>
>>> <p>Failed to establish a secure connection: error:0A000152:SSL 
>>> routines::unsafe legacy renegotiation disabled</p>
>>>
>>> So, as we can see, we are still not able to access the site.
>>>
>>> Lets see what cache.log has to say.
>>>
>>> $ tail -f /var/log/squid/cache.log  |grep -i 'openssl\|parsed'
>>> 2022/12/28 12:31:09.971 kid1| 83,5| PeerOptions.cc(447) parseOptions: 
>>> INFO: TLS parsedOptions(1)=0
>>> 2022/12/28 12:31:09.971 kid1| 83,5| PeerOptions.cc(643) 
>>> updateContextOptions: set OpenSSL options for context=0x562a53eb14e0, 
>>> parsedOptions=0
>>> 2022/12/28 12:31:09.971 kid1| 83,5| PeerOptions.cc(645) 
>>> updateContextOptions: get OpenSSL options for context=0x562a53eb14e0, 
>>> getOptions=1179648
>>>
>>> Strangely parsedOptions was zero and not 4!
>>>
>>> Now we can answer your questions as below.
>>>
>>>
>>> On 27/12/22 21:52, Alex Rousskov wrote:
>>>> On 12/27/22 10:42, Amish wrote:
>>>>> On 26/12/22 21:31, Alex Rousskov wrote:
>>>>>> tls_outgoing_options options=0x4,0x40000
>>>>
>>>>> With numeric hex values, I do not see the ERROR on stderr.
>>>>
>>>>> But it still does not seem to be working as expected. Squid still 
>>>>> does not open the page and gives same legacy negotiation error.
>>>>
>>>> There are still many unknowns (from my point of view), including:
>>>>
>>>> 1. Does OpenSSL accept the above options? You ask that question below.
>>>
>>> Google search shows some projects using OpenSSL v3 where there is 
>>> mention to use above option when a similar error occurred to them.
>>>
>>> But in our case, its clear that squid does not pass value 4 to SSL 
>>> context, hence we do not know yet if OpenSSL accepts above options.
>>>
>>>
>>>> 2. Does Squid indeed stare at the server, as expected?
>>>> 3. Does Squid apply the accepted options when staring at the server?
>>>
>>> A comment for parseOptions() in PeerOptions.cc states this:
>>>
>>> /**
>>>   * Pre-parse TLS options= parameter to be applied when the TLS 
>>> objects created.
>>>   * Options must not used in the case of peek or stare bump mode.
>>>   */
>>> void Security::PeerOptions::parseOptions()
>>>
>>> So it appears that TLS options is NOT used for peek as well as stare. 
>>> But why? I am not sure.
>>>
>>> How do I make squid use it for 'stare' atleast?
>>>
>>>
>>>> 4. Why does TLS negotiation fail despite those options applied,
>>>>    especially since it succeeds with using openssl s_client
>>>
>>> It possibly fails because options are not applied by squid.
>>>
>>> So, where do I check next on why parsedOptions is still set to 0 and 
>>> why its not used for 'stare'?
>>>
>>> Thanks and regards,
>>>
>>> Amish.
>>>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

More information about the squid-users mailing list