[squid-dev] Moved PID file management from Coordinator to Master

[Amos Jeffries]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/01/2015 6:02 a.m., Tsantilas Christos wrote:
> Hi all, this patch moves pid file managment from coordinator
> process to master process.
> 
> This move is the first step necessary to avoid the following race 
> condition among PID file deletion and shared segment 
> creation/destruction in SMP Squid:
> 
> O1) The old Squid Coordinator removes its PID file and quits. N1)
> The system script notices Coordinator death and starts the new
> Squid. N2) Shared segments are created by the new Master process. 
> O2) Shared segments are removed by the old Master process. N3) New
> worker/disker processes fail due to missing segments.
> 

The Coordinator needs to continue coordinating activities over the SMP
sockets until the workers are all shutdown and SMP sockets closed,
only then should it do O2 and O1 (in that order).

The planned behaviour for worker shutdown is to:
 W1) early client FD closures into the beginning of the
shutdown_timeout period
 W2) on each client closure or connection going idle, close it
 W3) at end of shutdown_timeout OR last client disconnect, release all
resources.

In that design the AsyncEngine still runs right up until the queue
completes draining. Using SMP sockets to inform Coordinator about
clean shutdown at the end.
The Master process has no way to know if the workers are exiting early
with no clients, or aborting on worker-specific shutdown_timeout
values. But the coordinator can receive a terminated message from them
over SMP sockets.


> TODO: The second step (not a part of this change) is to delete
> shared memory segments before PID file is deleted (all in the
> Master process after this change).
> 
> Now the Master process receives signals and is responsible for 
> forwarding them to the kids.

The command line control process also used manually for the -k options
 to send signals also thinks of itself as Master.

How does this new closing of SMP sockets interact with that other
meaning of Master process?



> 
> Please for more informations read the patch preamble.
> 
> This is a Measurement Factory project
> 
> 
> Some extra notes/ideas --------------------------
> 
> 1) Multiple shutdown signals received by squid
> 
> In current squid when coordinator received a shutdown signal, then 
> replaced shutdown signal handlers with the default handlers. This
> is has as result when a second shutdown signal received then the
> coordinator process died immediately, without forwarding shutdown
> signal to kids. The shutdown of the other kids are finished as
> normal.
> 
> This patch when master process receives a shutdown signal forward
> it to kids and master process is ready to receive a second shutdown
> signal. When a second shutdown signal received to master and this
> forwarded to kids then the kids died immediately.

Plan was to pass the signal to workers again where they kick off their
own shutdown_timeout event handlers immediately instead of hard
killing workers.

FWIW: Ubuntu Gentoo, and RHEL people are enjoying their patches that
just ignore the repeated signals.


> 
> 2) The system admin shows a blocked kid (infinity loop or not 
> responding). He kill with the hand.
> 
> Current squid does not restart the kids killed by a TERM or KILL 
> signal (squid considers it as normal kid shutdown). This patch does
> not change this behaviour. The admin is still able to kill with a
> "kill -11" and in this case the kid will restarted.
> 
> My opinion is that squid should restart kids in these cases. Should
> not restart a kid only when a shutdown requested from system admin,
> or when the kids dying very fast (hopeless()==true ).

TERM and KILL received by the workers often *are* signals sent by the
system admin, or scripts on their behalf. That may decrease in
popularity though when we fix the normal shutdown process issues. For
a while longer we have to take the current reality.




In related topics, I have been trying to figure out a --foreground
command line option that operates like -N but does not disable SMP,
just makes Coordinator == Master. But understanding the SMP
complexities have been blocking me so far.  Are you able and
interested in taking that forward?

Amos
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)

iQEcBAEBAgAGBQJUtALXAAoJELJo5wb/XPRjwKsIAMPzuOaxvC7WpBHOpQZpG1IZ
1tgbtosaJu3JweE7At729HLL34mR+YagaJbTz4xF6c2mkpLxxYioT6IzSxKc6YCD
mYJr8WU8uuJVI662u7w+3UyLVLI+c3vIwrw8d8NDZaKyAkOIn//Xks9YIG7h+xse
ooK/AAhMaADiS5S1FqY9OM3Q5Pn0nI3R91EpzGIeL1U5bG+43GYiOic3YSKgxSzq
8Q3YemiLj7ex00ZBtCbQ955bB8Zz1Q9I8hWgXdAFHgQKrjNmjdUDHqEg5M6E33zf
Gwpr6M3bO1gbtp7ize9vX7YxIlUjK6TUsbOFPlt9QJYEzzVxoqcgzy0lavVEiXE=
=5Qeg
-----END PGP SIGNATURE-----