[squid-dev] Moved PID file management from Coordinator to Master

[Tsantilas Christos]

On 01/12/2015 07:22 PM, Amos Jeffries wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 12/01/2015 6:02 a.m., Tsantilas Christos wrote:
>> Hi all, this patch moves pid file managment from coordinator
>> process to master process.
>>
>> This move is the first step necessary to avoid the following race
>> condition among PID file deletion and shared segment
>> creation/destruction in SMP Squid:
>>
>> O1) The old Squid Coordinator removes its PID file and quits. N1)
>> The system script notices Coordinator death and starts the new
>> Squid. N2) Shared segments are created by the new Master process.
>> O2) Shared segments are removed by the old Master process. N3) New
>> worker/disker processes fail due to missing segments.
>>
>
> The Coordinator needs to continue coordinating activities over the SMP
> sockets until the workers are all shutdown and SMP sockets closed,
> only then should it do O2 and O1 (in that order).
>
> The planned behaviour for worker shutdown is to:
>   W1) early client FD closures into the beginning of the
> shutdown_timeout period
>   W2) on each client closure or connection going idle, close it
>   W3) at end of shutdown_timeout OR last client disconnect, release all
> resources.
>
> In that design the AsyncEngine still runs right up until the queue
> completes draining. Using SMP sockets to inform Coordinator about
> clean shutdown at the end.

My sense is that the exit status can provide the same functionality and 
also is easier to be implemented.
If the worker aborted early by a segfault then I am doubt that it will 
be able to send a message to an SMP socket.

> The Master process has no way to know if the workers are exiting early
> with no clients, or aborting on worker-specific shutdown_timeout
> values. But the coordinator can receive a terminated message from them
> over SMP sockets.

We can use exit status.

>
>
>> TODO: The second step (not a part of this change) is to delete
>> shared memory segments before PID file is deleted (all in the
>> Master process after this change).
>>
>> Now the Master process receives signals and is responsible for
>> forwarding them to the kids.
>
> The command line control process also used manually for the -k options
>   to send signals also thinks of itself as Master.
>
> How does this new closing of SMP sockets interact with that other
> meaning of Master process?

The master process is the simplest squid process. I believe that it is 
the best process for doing the cleanup.

>
>
>
>>
>> Please for more informations read the patch preamble.
>>
>> This is a Measurement Factory project
>>
>>
>> Some extra notes/ideas --------------------------
>>
>> 1) Multiple shutdown signals received by squid
>>
>> In current squid when coordinator received a shutdown signal, then
>> replaced shutdown signal handlers with the default handlers. This
>> is has as result when a second shutdown signal received then the
>> coordinator process died immediately, without forwarding shutdown
>> signal to kids. The shutdown of the other kids are finished as
>> normal.
>>
>> This patch when master process receives a shutdown signal forward
>> it to kids and master process is ready to receive a second shutdown
>> signal. When a second shutdown signal received to master and this
>> forwarded to kids then the kids died immediately.
>
> Plan was to pass the signal to workers again where they kick off their
> own shutdown_timeout event handlers immediately instead of hard
> killing workers.

So do you believe that the workers should not restore default  handlers 
for shutdown signals. Am I correct?
It is easy to be implemented, it already implemented for 
"kill-parent-hack" where the master process is constrained to send 
multiple kill signals to kids.


>
> FWIW: Ubuntu Gentoo, and RHEL people are enjoying their patches that
> just ignore the repeated signals.
>
>
>>
>> 2) The system admin shows a blocked kid (infinity loop or not
>> responding). He kill with the hand.
>>
>> Current squid does not restart the kids killed by a TERM or KILL
>> signal (squid considers it as normal kid shutdown). This patch does
>> not change this behaviour. The admin is still able to kill with a
>> "kill -11" and in this case the kid will restarted.
>>
>> My opinion is that squid should restart kids in these cases. Should
>> not restart a kid only when a shutdown requested from system admin,
>> or when the kids dying very fast (hopeless()==true ).
>
> TERM and KILL received by the workers often *are* signals sent by the
> system admin, or scripts on their behalf. That may decrease in
> popularity though when we fix the normal shutdown process issues. For
> a while longer we have to take the current reality.

ok.


>
>
>
>
> In related topics, I have been trying to figure out a --foreground
> command line option that operates like -N but does not disable SMP,
> just makes Coordinator == Master. But understanding the SMP
> complexities have been blocking me so far.  Are you able and
> interested in taking that forward?

I do not know :-)
I must ask Alex for this.

Regards,
     Christos


>
> Amos
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.22 (MingW32)
>
> iQEcBAEBAgAGBQJUtALXAAoJELJo5wb/XPRjwKsIAMPzuOaxvC7WpBHOpQZpG1IZ
> 1tgbtosaJu3JweE7At729HLL34mR+YagaJbTz4xF6c2mkpLxxYioT6IzSxKc6YCD
> mYJr8WU8uuJVI662u7w+3UyLVLI+c3vIwrw8d8NDZaKyAkOIn//Xks9YIG7h+xse
> ooK/AAhMaADiS5S1FqY9OM3Q5Pn0nI3R91EpzGIeL1U5bG+43GYiOic3YSKgxSzq
> 8Q3YemiLj7ex00ZBtCbQ955bB8Zz1Q9I8hWgXdAFHgQKrjNmjdUDHqEg5M6E33zf
> Gwpr6M3bO1gbtp7ize9vX7YxIlUjK6TUsbOFPlt9QJYEzzVxoqcgzy0lavVEiXE=
> =5Qeg
> -----END PGP SIGNATURE-----
> _______________________________________________
> squid-dev mailing list
> squid-dev at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-dev
>