[squid-users] Connection Timeouts

Tue Jun 5 09:53:51 UTC 2018

Sorry the auto words correction changed every single "systemd" to "system" in the body of my email.
I like auto correction but: really???
Ho, I get it.. it's a manual system so I need to add Systemd to the dictionary.

I hope this makes more sense to the body of the email.

Eliezer

----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer at ngtech.co.il

-----Original Message-----
From: squid-users <squid-users-bounces at lists.squid-cache.org> On Behalf Of Eliezer Croitoru
Sent: Tuesday, June 5, 2018 12:42
To: 'Cheadle, Edward' <Edward.Cheadle at cambiahealth.com>
Cc: squid-users at lists.squid-cache.org
Subject: Re: [squid-users] Connection Timeouts

Hey Edward,

I have just seen the AWS Linux container and it seems that they do not use system but they do have updates.
I do not know where did you downloaded the el6 3.5.27 package but their official current release is:
3.5.20-10.34.amzn1

Their squid -v output:
bash-4.2# squid -v
Squid Cache: Version 3.5.20
Service Name: squid
configure options:  '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--target=x86_64-amazon-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--disable-strict-error-checking' '--exec_prefix=/usr' '--libexecdir=/usr/lib64/squid' '--localstatedir=/var' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--with-logdir=$(localstatedir)/log/squid' '--with-pidfile=$(localstatedir)/run/squid.pid' '--disable-dependency-tracking' '--enable-eui' '--enable-follow-x-forwarded-for' '--enable-auth' '--enable-auth-basic=DB,LDAP,MSNT-multi-domain,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB,SMB_LM,getpwnam' '--enable-auth-ntlm=smb_lm,fake' '--enable-auth-digest=file,LDAP,eDirectory' '--enable-auth-negotiate=kerberos' '--enable-external-acl-helpers=file_userip,LDAP_group,unix_group,time_quota,session,wbinfo_group' '--enable-cache-digests' '--enable-cachemgr-hostname=localhost' '--enable-delay-pools' '--enable-epoll' '--enable-ident-lookups' '--enable-linux-netfilter' '--enable-removal-policies=heap,lru' '--enable-snmp' '--enable-ssl-crtd' '--enable-storeio=aufs,diskd,rock,ufs' '--enable-wccpv2' '--enable-esi' '--enable-ecap' '--with-aio' '--with-default-user=squid' '--with-dl' '--with-openssl' '--with-pthreads' '--disable-arch-native' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'target_alias=x86_64-amazon-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fpie' 'CXXFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fpie' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'

so it's basically a RHEL el6 *based* OS which also have support for ssl-bump and is actually el7 without all the system benefits....

I can try to port their current SRPM 3.5.20 to my version and since they do have 4.14 kernel I do believe it's worth the effort.
I have added it to my list of tasks...

And related to timeouts:
http://www.squid-cache.org/Versions/v3/3.5/cfgman/

TIMEOUTS
 -----------------------------------------------------------------------------

    forward_timeout
    connect_timeout
    peer_connect_timeout
    read_timeout
    write_timeout
    request_timeout
    client_idle_pconn_timeout
    ftp_client_idle_timeout
    client_lifetime
    half_closed_clients
    server_idle_pconn_timeout
    ident_timeout
    shutdown_lifetime

is probably the section their support wanted you to see.

But I really do not see if there is any need for such a change.

Also I do not know what AWS FW\NAT connection limits are so there should be taken into account when calculating what might be causing any issues.

Eliezer

----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer at ngtech.co.il

-----Original Message-----
From: Cheadle, Edward <Edward.Cheadle at cambiahealth.com>
Sent: Monday, June 4, 2018 23:07
To: Eliezer Croitoru <eliezer at ngtech.co.il>
Cc: squid-users at lists.squid-cache.org
Subject: Re: [squid-users] Connection Timeouts

Eliezer, you are absolutely right.  I got in a hurry and forgot the basics such as version numbers and all the other details.

The version currently on our squid server is: squid-3.5.27-1.el6.x86_64.rpm We are running AWS Linux:  Amazon Linux AMI 2018.03.0

We are a health care company.  We are using squid proxy to control what the servers in an account can connect to on the internet.  AWS looked at an issue we had with code deploy and they said connections were timing out because the default connection timeout is 1 min, and suggested we change the timeout to 5 min.  It issue has to do with Codedeploy.  Since AWS services are on the internet, I was thinking if we could set an overall timeout, and then one for services that are known to take more time, I thought it would be a way keep the length of the timeout down for most things and free up resources for the majority of tasks.

My concern, as stated below is that connections will take a while to timeout and it will put more pressure on the number of file descriptors we use.  We ran into an issue with the number of file descriptors used, but figured it out and we are fine, but increasing the timeout to 5 min set off a warning flag in my mind, not having a lot of experience with squid. I am not even sure it is an issue, but I thought I try to make sure before we ran into production issues.

The reason for including the link, is that it was the first one I found and in the description they mentioned the ability to set timeouts on a site/domain-specific basis, but in the info that followed and in subsequent searches, I did not see how it was done, so the failure to find information on the subject led me to join the list.  

In looking at the docs, there are a  number of other timeouts, so I obviously have some homework to do.

Thanks for the quick response.

On 6/4/18, 12:31 PM, "Eliezer Croitoru" <eliezer at ngtech.co.il> wrote:

    Hey Edward,

    First congrats!.
    I hope we can help you to figure out the relevant details.

    I am not sure why you have spoken to AWS teams about Squid-Cache, may I ask what OS are you using in AWS?
    Also what version of Squid are you using?
    The timeout settings are "critical" indeed but depends on what you are using and doing with Squid-Cache.
    Despite to the fact that https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.visolve.com%2Fsquid%2Fsquid30%2Ftimeout&data=02%7C01%7CEdward.Cheadle%40cambiahealth.com%7C8be888b30a484f0d8b4f08d5ca49570f%7Ce964274919d44f7fb4df802b2b75a809%7C0%7C0%7C636637338708424102&sdata=SpOxewYBxY1Y7qeK7fk5cEF0pWN2l%2B4UOM6IclHVrbw%3D&reserved=0 Is in a way still a lead it's not "up-to-date"

    Please note that without understanding what issues have you been facing and the purpose of the Squid-Cache instance(s?) there is no way to even guess what might fit your needs.

    Eliezer

    ----
    https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fngtech.co.il%2Flmgtfy%2F&data=02%7C01%7CEdward.Cheadle%40cambiahealth.com%7C8be888b30a484f0d8b4f08d5ca49570f%7Ce964274919d44f7fb4df802b2b75a809%7C0%7C0%7C636637338708424102&sdata=Mpu0Ottn255qQxnsXGT%2F%2ByR432Yz9%2FckeKTuVpZ6aUM%3D&reserved=0
    Linux System Administrator
    Mobile: +972-5-28704261
    Email: eliezer at ngtech.co.il

    From: squid-users <squid-users-bounces at lists.squid-cache.org> On Behalf Of Cheadle, Edward
    Sent: Monday, June 4, 2018 21:06
    To: squid-users at lists.squid-cache.org
    Subject: [squid-users] Connection Timeouts

    We had a person leave and I got selected to update and maintain our squid proxy.   We are talking to AWS and they told us that we needed to change the connection_timeout value from the default to 5 min.

    We have people stress testing out installation and I was concerned that if connection timeouts are too long we may see congestion.

    Should I be worried that connection timeouts will use up file descriptors at a higher rate?

    And what might be the options?

    Doing and internet search I found a web page at https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.visolve.com%2Fsquid%2Fsquid30%2Ftimeout.php&data=02%7C01%7CEdward.Cheadle%40cambiahealth.com%7C8be888b30a484f0d8b4f08d5ca49570f%7Ce964274919d44f7fb4df802b2b75a809%7C0%7C0%7C636637338708424102&sdata=FSq%2FnnFycwsbQaw8xRMzHkBWFY4Iw5F8KeJtdd1hRyc%3D&reserved=0 and in the TIMEOUT description I read

    “TIMEOUT
    Timeout parameters in Squid can be based on overall connection timeouts, peer-specific timeouts, site/domain-specific timeouts, request-specific timeouts etc. Proper setting of timeout values is critical to optimal Squid performance. Relevant parameters for timeout settings are listed”

    Is it possible to narrow the connection timeout to a specific site?  I looked at the website information, squid documentation and did an internet search.

    I did not see anything that narrowed the timeout to a specific timeout.

    I am trying to set connection timeouts to AWS sites, but keep connection timeouts to the default, because it is working well.

    IMPORTANT NOTICE: This communication, including any attachment, contains information that may be confidential or privileged, and is intended solely for the entity or individual to whom it is addressed. If you are not the intended recipient, you should delete this message and are hereby notified that any disclosure, copying, or distribution of this message is strictly prohibited. Nothing in this email, including any attachment, is intended to be a legally binding signature.

    Ensure a sustainable future - only print when necessary.

_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users