[systemd-devel] safe and fast shutdown/reboot

Discussion:

prashantkumar dhotre

2018-03-16 03:43:01 UTC

Hi
I see that default reboot/systemctl reboot command issues SIGTERM to my
apps and hence it is doing graceful stop of apps and this may take some
time and hence shutdown time may be little longer.

I am looking for safe and fastest shutdown/reboot method.

a) It is OK if my apps are stopped ungracefully during shutdown .(app
should not start automatically again after they are killed/stopped during
shutdown)

b) file system and such system level stuff needs to be cleanly shut down

1) From my research, I see that 'systemctl reboot --force' is the one I can
use.
I understand that this command sends SIGKILL to my apps.
So this satisfies both (a) and (b) and hence this command should be used to
reboot faster.
Could you please confirm ?
If this is not right method, please comment on which method to use.

2) Also is there a way to limit SIGKILL to only my apps when I do ' 'systemctl
reboot --force'
so that rest of the system level services still get stopped gracefully

3) If 'systemctl reboot --force' is correct command to use in my case,
then during shutdown , will my apps get restarted
due to 'Restart'/'StartLimitBurst'/'StartLimitInterval' settings in service
file ? I dont want my apps to get restarted if they are stopped/killed
during system shutdown

Thanks

Zeal Jagannatha

2018-03-16 03:53:02 UTC

Permalink

I think it would be better for the services you define to specifically
define their own `KillSignal` so you can control how they shutdown.
https://www.freedesktop.org/software/systemd/man/systemd.kill.html#KillSignal=

It's may not be safe for all the services on the machine to be shut down
with SIGKILL, so you should avoid using '--force' unless you know that
everything running on the system is safe to shutdown with SIGKILL.

On Thu, Mar 15, 2018 at 8:43 PM prashantkumar dhotre <

Post by prashantkumar dhotre
Hi
I see that default reboot/systemctl reboot command issues SIGTERM to my
apps and hence it is doing graceful stop of apps and this may take some
time and hence shutdown time may be little longer.
I am looking for safe and fastest shutdown/reboot method.
a) It is OK if my apps are stopped ungracefully during shutdown .(app
should not start automatically again after they are killed/stopped during
shutdown)
b) file system and such system level stuff needs to be cleanly shut down
1) From my research, I see that 'systemctl reboot --force' is the one I
can use.
I understand that this command sends SIGKILL to my apps.
So this satisfies both (a) and (b) and hence this command should be used
to reboot faster.
Could you please confirm ?
If this is not right method, please comment on which method to use.
2) Also is there a way to limit SIGKILL to only my apps when I do ' 'systemctl
reboot --force'
so that rest of the system level services still get stopped gracefully
3) If 'systemctl reboot --force' is correct command to use in my case,
then during shutdown , will my apps get restarted
due to 'Restart'/'StartLimitBurst'/'StartLimitInterval' settings in
service file ? I dont want my apps to get restarted if they are
stopped/killed during system shutdown
Thanks
_______________________________________________
systemd-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

prashantkumar dhotre

2018-03-16 03:57:05 UTC

Permalink

Thanks but I want to sigkill my services only during system shutdown and
not on normal service stop 'systemctl myservice stop'.
so I can not use ' KillSignal' setting. Is there any other way ?

Post by Zeal Jagannatha
I think it would be better for the services you define to specifically
define their own `KillSignal` so you can control how they shutdown.
https://www.freedesktop.org/software/systemd/man/systemd.
kill.html#KillSignal=
It's may not be safe for all the services on the machine to be shut down
with SIGKILL, so you should avoid using '--force' unless you know that
everything running on the system is safe to shutdown with SIGKILL.
On Thu, Mar 15, 2018 at 8:43 PM prashantkumar dhotre <

Zeal Jagannatha

2018-03-16 04:14:05 UTC

Permalink

Hmm. This is probably not ideal, but you could hook a 'Type=oneshot'
service into shutdown.target which runs 'systemctl kill {your
service}.service'.

I'm not sure if there's a simpler way to do this using targets.

On Thu, Mar 15, 2018 at 8:57 PM prashantkumar dhotre <

Post by prashantkumar dhotre
Thanks but I want to sigkill my services only during system shutdown and
not on normal service stop 'systemctl myservice stop'.
so I can not use ' KillSignal' setting. Is there any other way ?

Post by Zeal Jagannatha
I think it would be better for the services you define to specifically
define their own `KillSignal` so you can control how they shutdown.
https://www.freedesktop.org/software/systemd/man/systemd.kill.html#KillSignal=
It's may not be safe for all the services on the machine to be shut down
with SIGKILL, so you should avoid using '--force' unless you know that
everything running on the system is safe to shutdown with SIGKILL.
On Thu, Mar 15, 2018 at 8:43 PM prashantkumar dhotre <

aleivag

2018-03-16 05:09:32 UTC

Permalink

One other thing that may work, is that you could implement a ExecStop
action in your service unit, that checks if the system is been shutting
down (by checking status of {shutdown,reboot,halt}.target [or maybe also
the runlevel may work?]), and kill the unit if there is one of those
operations.

Alvaro Leiva

Post by Zeal Jagannatha
Hmm. This is probably not ideal, but you could hook a 'Type=oneshot'
service into shutdown.target which runs 'systemctl kill {your
service}.service'.
I'm not sure if there's a simpler way to do this using targets.
On Thu, Mar 15, 2018 at 8:57 PM prashantkumar dhotre <

Post by prashantkumar dhotre
Hi
I see that default reboot/systemctl reboot command issues SIGTERM to my
apps and hence it is doing graceful stop of apps and this may take some
time and hence shutdown time may be little longer.
I am looking for safe and fastest shutdown/reboot method.
a) It is OK if my apps are stopped ungracefully during shutdown .(app
should not start automatically again after they are killed/stopped during
shutdown)
b) file system and such system level stuff needs to be cleanly shut down
1) From my research, I see that 'systemctl reboot --force' is the one I
can use.
I understand that this command sends SIGKILL to my apps.
So this satisfies both (a) and (b) and hence this command should be
used to reboot faster.
Could you please confirm ?
If this is not right method, please comment on which method to use.
2) Also is there a way to limit SIGKILL to only my apps when I do ' 'systemctl
reboot --force'
so that rest of the system level services still get stopped gracefully
3) If 'systemctl reboot --force' is correct command to use in my
case, then during shutdown , will my apps get restarted
due to 'Restart'/'StartLimitBurst'/'StartLimitInterval' settings in
service file ? I dont want my apps to get restarted if they are
stopped/killed during system shutdown
Thanks
_______________________________________________
systemd-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

_______________________________________________
systemd-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Andrei Borzenkov

2018-03-16 05:36:43 UTC

Permalink

Post by aleivag
One other thing that may work, is that you could implement a ExecStop
action in your service unit, that checks if the system is been shutting
down (by checking status of {shutdown,reboot,halt}.target

Won't work. Status changes only when job for a unit completes and jobs
are executed in order of dependencies. Actually, jobs are *queued* in
order of dependencies so nothing would indicate that you are going to
shutdown until it is too late (i.e. all normal services are stopped).

Post by aleivag
[or maybe also
the runlevel may work?]),

Logically runlevel is not changed until *after* new runlevel has been
reached. Practically systemd does not update runlevel during shutdown at
all.

Post by aleivag
and kill the unit if there is one of those
operations.
Alvaro Leiva

Post by prashantkumar dhotre
Hi
I see that default reboot/systemctl reboot command issues SIGTERM to my
apps and hence it is doing graceful stop of apps and this may take some
time and hence shutdown time may be little longer.
I am looking for safe and fastest shutdown/reboot method.
a) It is OK if my apps are stopped ungracefully during shutdown .(app
should not start automatically again after they are killed/stopped during
shutdown)
b) file system and such system level stuff needs to be cleanly shut down
1) From my research, I see that 'systemctl reboot --force' is the one I
can use.
I understand that this command sends SIGKILL to my apps.
So this satisfies both (a) and (b) and hence this command should be
used to reboot faster.
Could you please confirm ?
If this is not right method, please comment on which method to use.
2) Also is there a way to limit SIGKILL to only my apps when I do ' 'systemctl
reboot --force'
so that rest of the system level services still get stopped gracefully
3) If 'systemctl reboot --force' is correct command to use in my
case, then during shutdown , will my apps get restarted
due to 'Restart'/'StartLimitBurst'/'StartLimitInterval' settings in
service file ? I dont want my apps to get restarted if they are
stopped/killed during system shutdown
Thanks
_______________________________________________
systemd-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

_______________________________________________
systemd-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

aleivag

2018-03-16 07:19:46 UTC

Permalink

Post by Andrei Borzenkov
Won't work. Status changes only when job for a unit completes and jobs
are executed in order of dependencies. Actually, jobs are *queued* in
order of dependencies so nothing would indicate that you are going to
shutdown until it is too late (i.e. all normal services are stopped).

yes, but those units have started, so during a `systemctl reboot` you can
execute

[~] systemctl list-jobs shutdown.target reboot.target
JOB UNIT TYPE STATE
1972 reboot.target start waiting
1974 shutdown.target start waiting

2 jobs listed.

and get if shutdown has started

the other thing that may help you know if you are in shutdown mode is
execute `systemctl is-system-running` and then check if returns `stopping`,
during a shutdown is suppose to return something like that. and i think it
does this by checking is shutdown.target has started (
https://github.com/systemd/systemd/blob/7a30dfeb18d09940a844389e06b25ca2bca5e093/src/core/manager.c#L3833-L3836
)

Logically runlevel is not changed until *after* new runlevel has been

Post by Andrei Borzenkov
reached. Practically systemd does not update runlevel during shutdown at
all.

yeap, you are right here, i was wrong :D

Andrei Borzenkov

2018-03-16 07:49:54 UTC

Permalink

Post by aleivag

Yes, you are right. I'm sure I have seen cases when jobs were not
present in other cases so I assumed it will be here as well.

Post by aleivag
the other thing that may help you know if you are in shutdown mode is
execute `systemctl is-system-running` and then check if returns `stopping`,
during a shutdown is suppose to return something like that. and i think it
does this by checking is shutdown.target has started (
https://github.com/systemd/systemd/blob/7a30dfeb18d09940a844389e06b25ca2bca5e093/src/core/manager.c#L3833-L3836
)

Yes, could be more reliable.

Post by aleivag
Logically runlevel is not changed until *after* new runlevel has been

Post by Andrei Borzenkov
reached. Practically systemd does not update runlevel during shutdown at
all.

yeap, you are right here, i was wrong :D

prashantkumar dhotre

2018-03-16 10:36:15 UTC

Permalink

Thanks
I understand that
'systemctl isolate other.target' will stop all present services and start
services of other.target

My thinking is :
I can put all my apps in my.target
and my.target can start after multi-user.target.

When shutting down , somewhere first i will do 'systemctl iso
multi-user.target' so that all my services in my.target will stop.
but my understanding is this will stop my services with SIGTERM
but i want them to be killed with SIGKILL.
Is there a way to SIGKILL my services in my.target when i do 'systemctl
iso multi-user.target' (something equivalent to concept of ' systemctl iso
multi-user.target --signal= SIGKILL ' ) ?

Regards

Post by Andrei Borzenkov

Post by aleivag

Yes, you are right. I'm sure I have seen cases when jobs were not
present in other cases so I assumed it will be here as well.

Post by aleivag
the other thing that may help you know if you are in shutdown mode is
execute `systemctl is-system-running` and then check if returns

`stopping`,

Post by aleivag
during a shutdown is suppose to return something like that. and i think

Post by aleivag
does this by checking is shutdown.target has started (
https://github.com/systemd/systemd/blob/7a30dfeb18d09940a844389e06b25c

a2bca5e093/src/core/manager.c#L3833-L3836

Post by aleivag
)

Yes, could be more reliable.

Post by aleivag
Logically runlevel is not changed until *after* new runlevel has been

Post by Andrei Borzenkov
reached. Practically systemd does not update runlevel during shutdown at
all.

yeap, you are right here, i was wrong :D

_______________________________________________
systemd-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Reindl Harald

2018-03-16 11:07:54 UTC

Permalink

Post by prashantkumar dhotre
Thanks
I understand that
'systemctl isolate other.target' will stop all present services and
start services of other.target
I can put all my apps in my.target
and my.target can start after multi-user.target.

that are all dirty workarounds and even if you manage to get it done
expect it to be fragile because nobody else is running anything this way

what is so special about your services that they

a) take that long for a clean stop
b) you don't care at all at shutdown but at normal stop

Post by prashantkumar dhotre
When shutting down , somewhere first i will do 'systemctl iso
multi-user.target' so that all my services in my.target will stop.
but my understanding is this will stop my services with SIGTERM
but i want them to be killed with SIGKILL.
Is there a way to SIGKILL my services in my.target when i do 'systemctl
iso multi-user.target' (something equivalent to concept of ' systemctl
iso multi-user.target --signal= SIGKILL ' ) ?

Post by aleivag

Yes, you are right. I'm sure I have seen cases when jobs were not
present in other cases so I assumed it will be here as well.

<https://github.com/systemd/systemd/blob/7a30dfeb18d09940a844389e06b25ca2bca5e093/src/core/manager.c#L3833-L3836>

Post by aleivag
)

Yes, could be more reliable.

Post by aleivag
Logically runlevel is not changed until *after* new runlevel has been

Post by Andrei Borzenkov
reached. Practically systemd does not update runlevel during shutdown at
all.

yeap, you are right here, i was wrong :D

Lennart Poettering

2018-03-20 11:14:25 UTC

Permalink

Nope there is not. We do not distinguish between stopping for shutdown
and other reasons (and we can't really, as we merge stop requests if
multiple are queued at the same time, and hence there are regular
cases where more than one reason applies)

Lennart

--
Lennart Poettering, Red Hat

Lennart Poettering

2018-03-20 11:12:44 UTC

Permalink

That is correct. But note that it will also SIGKILL everything else on
the system, including the journal for example, and that means the
journal files will remain in in a dirty state each time (which
journald will deal with, but is not pretty).

Post by prashantkumar dhotre
If this is not right method, please comment on which method to use.
2) Also is there a way to limit SIGKILL to only my apps when I do ' 'systemctl
reboot --force'
so that rest of the system level services still get stopped
gracefully

No, for that (as mentioned elsewhere), use KillSignal=SIGKILL or so.

Post by prashantkumar dhotre
3) If 'systemctl reboot --force' is correct command to use in my case,
then during shutdown , will my apps get restarted
due to 'Restart'/'StartLimitBurst'/'StartLimitInterval' settings in service
file ? I dont want my apps to get restarted if they are stopped/killed
during system shutdown

No. "systemctl reboot --force" means that PID 1 gets replaced by the
"systemd-shutdown" process right away, which has no notion of
services, and will just go on a killing spree and SIGKILL/umount
everything that is left.

Lennart

--
Lennart Poettering, Red Hat