[systemd-devel] feature request: dlopen

Post by Luke Kenneth Casson Leighton
so i'm not going to "protest" - i'm going to try a different approach.
i'd like you to look at this list of debian packages that are
http://lkcl.net/reports/removing_systemd_from_debian/list_of_libsystemd0_dependent_packages.txt
it's an enormous list, comprising some 15% of the packages present in
debian today. it includes apache2-dev, bluetooth / bluez, the gimp,
php, *all* of the video players (xine, mplayer, vlc), *all* of the
games and 3D packages that use SDL or pulseaudio (which is most of
them), *all* of the major desktop environments including xfce4, lxde,
Gnome and KDE/Plasma, cups-daemon, one of the anti-virus daemons, most
of the music software packages and services, most of the VoIP clients,
the android SDK, the eclipse IDE, OpenJDK 7, erlang, Apache
Tomcat..... i'm just absolutely blown away by the extent of the
dependencies.

How exactly did you create such a dependancy list? On my system
(non-debian but fully systemd-only), I picked two random packages from
your list, gimp and cheese:
$ ldd /usr/bin/cheese | grep system
$ ldd /usr/bin/gimp | grep system

Hm, no dependencies there.

Perhaps you might wish to just file a Debian bug so that they fix their
build systems? Arch Linux doesn't have this kind of requirement, so
something must be odd here...

Also, this is a distro choice to make, the Debian developers seem pretty
happy with systemd and how it is being loaded in their systems, so
perhaps you might wish to ask them about this hard-requirement. Other
distros do not have such a hard-requirement at this time, perhaps you
might wish to try one of them if you do not want to use systemd?

thanks,

greg k-h

Michael Biebl

2015-02-17 21:35:07 UTC

Post by Greg KH

How exactly did you create such a dependancy list? On my system
(non-debian but fully systemd-only), I picked two random packages from
$ ldd /usr/bin/cheese | grep system
$ ldd /usr/bin/gimp | grep system
Hm, no dependencies there.
Perhaps you might wish to just file a Debian bug so that they fix their
build systems? Arch Linux doesn't have this kind of requirement, so
something must be odd here...

There is no such dependency in Debian either [1].
Luke simply has no idea what he is talking about.
It would be great if Luke did some basic research and educate himself
and not spread such misinformation.

[1] https://packages.debian.org/sid/gimp

--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

Michael Biebl

2015-02-17 22:14:22 UTC

Post by Michael Biebl

Post by Greg KH

Let me clarify this this misinformation and make it absolutely clear:
There are less then 100 packages in the Debian archive which depend on
libsystemd0

--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

Michael Biebl

2015-02-17 22:20:16 UTC

Post by Michael Biebl

[...]

Post by Michael Biebl
There are less then 100 packages in the Debian archive which depend on
libsystemd0

67 binary packages, on an up-to-date sid/amd64 system, to be precise.
The number of source package is well below 40.

--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

Luke Kenneth Casson Leighton

2015-02-22 22:42:18 UTC

Post by Michael Biebl
There is no such dependency in Debian either [1].
Luke simply has no idea what he is talking about.
It would be great if Luke did some basic research and educate himself
and not spread such misinformation.

michael,

greg's approach is something that i'm going to ask you to learn from:
it was exemplary. he trusted that i had done the research, questioned
how i had arrived at the conclusion that i had, indicated implicitly
that he wanted to verify the research himself, and presented me with
an opportunity to provide him with the method by which he could do
that, when i had *deliberately* chosen, as a way to reduce the length
of an already-long post, to not pollute it with unnecessary commands.
commands which, conceivably, could be interpreted as trying to teach
highly experienced programmers how to suck eggs, and given the level
of competence of the people on this list i decided it would be a good
idea to trust in people's expertise implicitly.

in complete contrast, what you are saying - some of this is actually
explicit rather than implicit - is that i cannot be trusted, i am
incapable of doing basic research, i am illiterate, undereducated,
incompetent, deluded, unable to assess my own mental state and degree
of expertise, unable to learn, and, perhaps worst of all, willing to
waste other people's time by even talking to them.

would that be an accurate assessment of what you wished to convey?

if that is the case, i have to ask: what on earth are your motives for
making such views so plainly clear on this list? what do you hope to
achieve by saying what you did, in the way that you said it?

can i leave you to think about that?

l.

Luke Kenneth Casson Leighton

2015-02-22 22:32:40 UTC

Post by Greg KH

How exactly did you create such a dependancy list?

apt-rdepends -r libsystemd0 | a bit of awk-style manual magic | sort | uniq

someone else did a visual-style version of the same thing... i forget
exactly where i saw it - they removed thousands of packages, limiting
it to only about 400, otherwise it would be far too big.

Post by Greg KH
Perhaps you might wish to just file a Debian bug so that they fix their
build systems? Arch Linux doesn't have this kind of requirement, so
something must be odd here...

https://wiki.archlinux.org/index.php/systemd
they moved to systemd, wholesale.

converting to systemd has become a very rapid, widespread and
wholesale decision that has completely locked out sysvinit, openrc,
*everything*.

the reason why is because there's no precedent for using dlopen, it's
all compile-time hard dependencies in cups, libpulseaudio, libsdl and
a hundred other immediate dependencies that you can check with
apt-cache rdepends libsystemd0.

the only major (i say major, they're pretty low down on the list
these days) GNU/Linux distro left which anyone has heard of not using
systemd is slackware. and that's most likely - this is speculation
btw - because the basis of slackware is "minimal maintainer work". as
systemd hasn't stabilised, they are probably leaving it.

gentoo are the only team working to provide alternative
infrastructure. *all* other major GNU/Linux distros have converted to
systemd and completely abandoned all other options.

debian does provide systemd-shim, but because of the immediate
hard-compile-time dependencies, you are still left with libsystemd0.

Post by Greg KH
Also, this is a distro choice to make, the Debian developers seem pretty
happy with systemd and how it is being loaded in their systems, so
perhaps you might wish to ask them about this hard-requirement.
Other distros do not have such a hard-requirement at this time,

yeah, they do.

Post by Greg KH
perhaps you
might wish to try one of them if you do not want to use systemd?

i manage a dozen systems for clients, many of them remotely, with
absolutely no console access except a computer-illiterate individual
at the keyboard. i just tried upgrading a system tonight, adding in
the angband.pl nosystemd repository and upgrading it to jessie.

it completely failed to come back on the VPN network, and because it
is remote i have absolutely no idea why.

this is *real* bad stuff, greg. i now have to spend hours, tomorrow,
walking my computer-illiterate friend through the process of
recovering his computer.

i cannot possibly take the risk of doing such risky upgrades on
systems where my clients are paying money.

i cannot possibly even _remotely_ contemplate doing a *remote*
conversion of a debian system on which i have five lxc virtual
machines for clients. it would be completely insane to do so, as i
would not only need to convert the base machine but also the five
client machines all at the same time.

if the base host machine fails to come up, it costs me a fortune in
KVM access time at the hosting company.

during the time in which that base machine fails to come up, i have 5
clients all screaming at me "why is my machine not up?"

then when it is finally up i would have to convert them all over to
the replacement OS as well.

that's _days_ of work, greg, none of which may be done in a
transitional manner (i moved away from xen two years ago because after
four years of operation it was proving to be too unstable long-term)

and i am just one systems administrator. there will be others just
like me - world-wide - who aren't brave enough to speak to you guys
directly, who will be going "oh crap..." and who recognise that they
are being press-ganged into a major decision with absolutely no choice
in the matter.

even if i was to accept the downtime on my business, unlike many
others in this situation i can't even convert my laptop to FreeBSD
because it has hardware that is too modern and unsupported for
FreeBSD.

l.

Tom Gundersen

2015-02-17 22:03:37 UTC

Hi Luke,

On Tue, Feb 17, 2015 at 9:24 PM, Luke Kenneth Casson Leighton

Post by Luke Kenneth Casson Leighton
i note that there was announcement recently that the systemd team
'listens to users', so i am taking you at your word on that.

I believe we are listening a lot. That does not necessarily mean that
everyone will get all that they ask for though...

Post by Luke Kenneth Casson Leighton
i'd like you to look at this list of debian packages that are
http://lkcl.net/reports/removing_systemd_from_debian/list_of_libsystemd0_dependent_packages.txt
it's an enormous list, comprising some 15% of the packages present in
debian today. it includes apache2-dev, bluetooth / bluez, the gimp,
php, *all* of the video players (xine, mplayer, vlc), *all* of the
games and 3D packages that use SDL or pulseaudio (which is most of
them), *all* of the major desktop environments including xfce4, lxde,
Gnome and KDE/Plasma, cups-daemon, one of the anti-virus daemons, most
of the music software packages and services, most of the VoIP clients,
the android SDK, the eclipse IDE, OpenJDK 7, erlang, Apache
Tomcat..... i'm just absolutely blown away by the extent of the
dependencies.
oh - and php as well. what the heck php5 is doing with a hard
dependency on libsystemd0 i cannot imagine.
now, because those are hard compile-time dependencies, the only
possibility for the average debian user who chooses *not* to have
libsystemd0 present on their system is for them to simply... not use
*anything* on that list of over four and a half THOUSAND packages.
i think the most important question to ask you at this point is: as a
team, were you aware of the extent to which libsystemd0 has become a
hard compile-time dependency on so many critical software packages in
use today?

Speaking only for myself: no I was not aware of that. Moreover, I find
it completely unbelievable. Admittedly I do not use Debian, and did
not check how they do their packaging, but there is just no reason at
all for most of those packages to link against libsystemd.

Post by Luke Kenneth Casson Leighton
and the second question, which is just as important, is: does this
shock or alarm you enough to appreciate why people find the rapid
introduction of libsystemd0 to be so objectionable?

Again, speaking only for myself: no. (Again with the caveat that I
don't actually believe in your statements, but for the sake of
argument...) we are providing software that we hope people find
useful. That includes a C library that people may link against if they
so wish. If a lot of third-party developers chose to do that, that is
firstly a sign that we are successful in creating useful software and
secondly completely out of our control. If you don't want people to
use our APIs, you should address the third-party developers who do. I
really don't see what there is for us to do here.

If you find the existence of libsystemd on your machine objectionable
(note that it does not necessarily entail actually using systemd as
PID1), then a more useful way to spend your time would be to post
patches to make sure the existence of libsystemd does not adversely
affect the software you are running (these patches probably need to go
to the projects linking against systemd). In most cases that I'm aware
of (e.g. the code I have contributed to third-party packages), the
usage of libsystemd is a noop on systems where systemd is not running,
so I really don't see the problem here.

Post by Luke Kenneth Casson Leighton
to the trouble of digging into why libsystemd0 is found to be so
objectionable. my take on the matter is that the technical arguments
- benefits or otherwise - of systemd and its alternatives - is
completely irrelevant. over time people *will* develop alternatives
(and are already doing so: mdev, eudev, uselessd, openrc and many
more).

Side note: these things are not alternatives to libsystemd (they are
alternatives to udev and/or systemd as PID1).

Post by Luke Kenneth Casson Leighton
no, i feel that it really does have nothing to do with the technical
benefits of the available options: what people are finding completely
objectionable is that they have *no good choices*. it's "use systemd
or go away" - and unfortunately almost without exception (slackware
and FreeBSD being two notable ones) that "piss off" attitude is being
replicated across *the entire GNU/Linux Distro world*. the situation
is completely unprecedented and without parallel in the short history
of software libre (and that's something that, honestly, i find to be
really shocking, hence why i am contacting you).

I disagree with most of that, but importantly you are missing a point:
many (most?) packages linking against libsystemd can already at
runtime fall back to alternatives. There is really no need for dlopen
for this. The only cost you have is to have the (possibly unused)
libsystemd library around on your system. It will be far from the only
unused library you have, so this really seems like a tempest in a
teapot.

Post by Luke Kenneth Casson Leighton
overall, they feel that they're being forced into the use of something
that they feel has not been properly thought through, is still under
development, is increasing in scope in a way that alarms them due to
there being no other choices, causes them some huge inconvenience that
they'd rather have a bit more time to consider but they are *not being
given that chance*, and much more.

I encourage you and anyone else who fear the lack of alternatives to
work on creating alternatives. This is really the wrong audience
though. Unsurprisingly, the systemd developers are probably the people
least interested in creating systemd alternatives...

Post by Luke Kenneth Casson Leighton
i have to tell you: i even heard, on slashdot, that microsoft is now

I'm sorry, but that is not the way someone who wishes to be taken
seriously should start a sentence...

Best of luck!

Tom

Tobias Hunger

2015-02-18 09:10:37 UTC

Hi Luke,

I am mostly a lurker on the systemd mailing list, so my opinion does
not carry weight in this community.

On Tue, Feb 17, 2015 at 9:24 PM, Luke Kenneth Casson Leighton
<***@lkcl.net> wrote:> so i'm not going to "protest" - i'm going to
try a different approach.

Post by Luke Kenneth Casson Leighton
i'd like you to look at this list of debian packages that are
http://lkcl.net/reports/removing_systemd_from_debian/list_of_libsystemd0_dependent_packages.txt

I understood most of these dependencies to be indirect: Packages that
depend on other packages that in turn depend on libsystemd. Is that
correct?

How many and which packages depend *directly* on libsystemd? Are the
numbers by other people replying to this list correct, namely that the
direct dependencies are < 100 packages?

Post by Luke Kenneth Casson Leighton
i think the most important question to ask you at this point is: as a
team, were you aware of the extent to which libsystemd0 has become a
hard compile-time dependency on so many critical software packages in
use today?

My understanding is that libsystemd is a dependency of some packages
down low in the stack that is then (re-)used by other packages.

And for those few packages it makes sense to depend on libsystemd:
Those tend to provide services that do benefit from systemd features
like socket activation. So not having this dependency does seriously
hurt the systemd users.

On the other hand the library is tiny and basically falls back to
being a no-op in the case where systemd is not PID1, so it does not
hurt non-systemd systems to have this library in any way.

Post by Luke Kenneth Casson Leighton
we see that a debian developer has created unofficial packages
that, if installed, provide replacements for key strategic packages
entirely recompiled *without* systemd and without libsystemd0 being
present.

Good for them. I see very little value in replacing a ~150KiB library
that does nothing for the users these packages are targeting, but
everybody is free to spend their time however they want.

Post by Luke Kenneth Casson Leighton
moving on: in what adam wrote (rather hot-headedly, initially), he
goes on to mention that it would be perfectly reasonable to replicate
the effects of how he removed libsystemd0, in a way that would be far
dynamic library loading.

Libsystemd's job is basically to provide exactly what you ask for: A
wrapper around systemd functionality, that fails gracefully in case
systemd is not used.

That wrapper is nicely packaged up into a library so that upstream
projects do not need to keep reimplementing the same dlopen, error
handling, etc. over and over again. Your proposal is to ask every
upstream project to add that same snippet of code? How about putting
that into a library for easier reuse: Maybe libsystemdwrapper. That
can then be wrapped in another wrapper when somebody freaks out about
"everything is linking to libsystemdwrapper".

Maybe just renaming libsystemd would suffice? I am sure hardly nobody
would object to having a tiny "libyzy" on their system:-)

Post by Luke Kenneth Casson Leighton
so can i leave it with you to consider whether the current situation
is tolerable or not?

Again: I can in no way speak for the systemd project. But from where I
stand the systemd project went out of their way to provide you with
exactly what you are asking for in a way that is easy to reuse by
upstream projects. That is libsystemd. Apparently you find that
solution objectionable, but I do not understand why.

I would personally like to see a "libinitd" which brings the socket
activation features that is provided to daemons as part of libsystemd
to other init systems (that can support those). That would make it so
much easier for upstreams to support more than one init system. But I
would expect that to be implemented by the teams working on
alternatives to systemd or by distributions centered around other init
systems.

Post by Luke Kenneth Casson Leighton
i am one of the few people who can cut through all that, who has gone
to the trouble of digging into why libsystemd0 is found to be so
objectionable. my take on the matter is that the technical arguments
- benefits or otherwise - of systemd and its alternatives - is
completely irrelevant. over time people *will* develop alternatives
(and are already doing so: mdev, eudev, uselessd, openrc and many
more).

Sure. I am looking forward to that! I am convinced a bit of
competition and fresh ideas will do systemd a hell of a lot of good:-)

My take is a bit different: I have seen and used lots of init systems
over the years. *Finally* we have one that actually provides some
benefits that developers of unrelated projects actually want to use!
That none of the others ever got to the state is more a testimony for
their failure than for their design -- even if many loud-mouthed
systemd opponents seem to think otherwise.

Please do not ask systemd to be less useful, please ask other projects
to implement better (for whichever metric of "better" you want to
apply) solutions to the problems those developers face.

I do not see how the systemd project or anybody else can change that
at this point. My experience is that many people out there are beyond
a rational debate at this point. And I explicitly want to include
people from both sides of the fence in this statement.

I am afraid we will have to sit this out.

Post by Luke Kenneth Casson Leighton
i have to tell you: i even heard, on slashdot, that microsoft is now
using - to significant success - the situation surrounding systemd as
a sales pitch to have GNU/linux systems successfully replaced with
windows servers.

Isen't it amazing what kind of stories some anonymous cowards make up
over at slashdot? There are some gems of creativity in some of those
systemd flamefests.

Best Regards,
Tobias

Luke Kenneth Casson Leighton

2015-02-22 23:58:25 UTC

Post by Tom Gundersen
Hi Luke,
I am mostly a lurker on the systemd mailing list, so my opinion does
not carry weight in this community.
On Tue, Feb 17, 2015 at 9:24 PM, Luke Kenneth Casson Leighton
try a different approach.

Post by Luke Kenneth Casson Leighton
i'd like you to look at this list of debian packages that are
http://lkcl.net/reports/removing_systemd_from_debian/list_of_libsystemd0_dependent_packages.txt

I understood most of these dependencies to be indirect: Packages that
depend on other packages that in turn depend on libsystemd. Is that
correct?

that's right. so, what that means is that the actual number of
packages which would need to be converted to dynamic loading is
actually very small (about 100), and the remaining 4,483 would be fine
(not need any work done on them at all).

Post by Tom Gundersen
On the other hand the library is tiny and basically falls back to
being a no-op in the case where systemd is not PID1, so it does not
hurt non-systemd systems to have this library in any way.

...except that its introduction (usually --with-libsystemd) in those
100 (or so) packages has been done in a mutually-exclusive,
hard-compile-time switch that *excludes* the possibility of dynamic
(runtime) decision-making.

Good for them. I see very little value in replacing a ~150KiB library
that does nothing for the users these packages are targeting, but
everybody is free to spend their time however they want.

they made it possible to remove systemd entirely. they had to
recompile cups, stunnel, udisks2, upower, util-linux and many more...
here's the list:

http://angband.pl/debian/dists/nosystemd/main/binary-amd64/Packages

but here's the problem: because there is no dynamic run-time
decisions possible here, people are forced to adding that (unofficial)
repo and to risk their systems in the process. reverting is also just
as risky.

but, the worst thing is that because it's not an "official" repo, any
corporation that has for example a support contract is *prevented and
prohibited* from using the angband.pl repo because it would violate
their support contract.

Libsystemd's job is basically to provide exactly what you ask for: A
wrapper around systemd functionality, that fails gracefully in case
systemd is not used.

honestly, i find it hard to argue with that. i think i know what the
problem is. the problem is, i believe, that the detection is not
user-controllable. question: does libsystemd have a config-file
option that it reads, where if "systemd = disabled" is present, for
example, it is effectively equivalent to not having systemd installed?

i'm going with the flow here, btw, even though it actually partially
undermines the case that i'm endeavouring to get across to the team.

Post by Tom Gundersen
That wrapper is nicely packaged up into a library so that upstream
projects do not need to keep reimplementing the same dlopen, error
handling, etc. over and over again. Your proposal is to ask every
upstream project to add that same snippet of code?

yes. in a dynamic way that includes a config file option which
allows run-time disabling of systemd.

Post by Tom Gundersen
How about putting
that into a library for easier reuse: Maybe libsystemdwrapper. That
can then be wrapped in another wrapper when somebody freaks out about
"everything is linking to libsystemdwrapper".

haha yehh, it would look like that wouldn't it. except that at the
first opportunity where it is configurable via a plain text file to
disable the chain-of-loading-loaders, the purpose would have been
achieved, so there would be no need for another loader-of-loaders.

ok i'm going with the silliness, here, but you know what i mean.

Post by Tom Gundersen
Maybe just renaming libsystemd would suffice? I am sure hardly nobody
would object to having a tiny "libyzy" on their system:-)

:) in all seriousness, any kind of modification outside of what's
available from the distros is... it can't be done. really. this is
too low-level. one mistake renaming a library without knowing the
consequences and people would end up with unbootable systems. and
would be in violation of their support contracts. etc. etc.

Post by Luke Kenneth Casson Leighton
so can i leave it with you to consider whether the current situation
is tolerable or not?

in many ways it doesn't matter that i (or anyone else) finds the
solution objectionable. what should matter - to you - is that i (or
anyone else) wishes to make that choice [we might have a solution
above btw as you can see].

the analogy here is libselinux1. libselinux1 (which i've worked
with) can be disabled at boot time, even with a kernel option. it can
even be re-enabled at runtime (if you made the correct option at boot
time).

... does libsystemd0 have that same user-configurable option? or is
it implicit by way of whether systemd is PID 1 or not?

.... i think... really, although technically what i am asking
*appears* to be "achieved in an equivalent fashion", they're not
entirely the same.

bear with me whist i think this through, ok?

my thoughts go like this: virtually every GNU/Linux distro has gone
for a mutually-exclusive all-or-nothing choice: systemd or out. even
Debian has had to provide systemd-shim to quotes stop the whiners
quotes. it has "Breaks: systemd < 209" in the package file. they
*did not* notice that systemd may be easily disabled by not running as
PID 1.

so, whole-sale, applications are *abandoning* the alternatives - in
droves. there are discussions on the FreeBSD mailing lists about what
the hell they're going to do, because the code that *used* to be in
KDE5, for example, is being *ripped out*. it's not even being
maintained as an alternative compile-time option.

so whilst it appears, on the face of it, that one might simply "not
run systemd as PID 1" and it would then be possible dynamically at
runtime for all the applications to go "oh, we're running sysvinit not
systemd, we should do XYZ rather than contact logind over d-bus"...

... in reality there is a massive and alarming near-total abandonment
of all the alternatives - all the incremental work done over the past
20 years - it's *all going*, making systemd the sole exclusive option
pretty much right across the board.

that really really should alarm you quite dramatically.

Post by Tom Gundersen
I would personally like to see a "libinitd" which brings the socket
activation features that is provided to daemons as part of libsystemd
to other init systems (that can support those). That would make it so
much easier for upstreams to support more than one init system. But I
would expect that to be implemented by the teams working on
alternatives to systemd or by distributions centered around other init
systems.

i would _love_ to see a situation where there is more dialog and
collaboration between the developers of systemd and the maintainers of
the alternatives.

and that i think, guys, is part of the problem. the development of
systemd is progressing at such a fast pace, with no dialog with any of
the peers (developers of openrc, maintainers of sysvinit, depinit -
currently me - no-one) that there are people - really experienced
people -really freaking out right now, and they don't fully understand
why.

they know that something's wrong, but are having a real hard time
putting their finger on it. that's led to some er... charitably we
can call them debates ... that have resulted in really senior
dedicated people who have spend in almost all cases over a _decade_ of
their lives on debian... quitting their role and involvement. and the
software libre community as a whole is seriously diminished by the
loss of their input.

Sure. I am looking forward to that! I am convinced a bit of
competition and fresh ideas will do systemd a hell of a lot of good:-)

the problem is, systemd conversion is so near-exclusive and so rapid
that there won't *be* any competition.

the "competition" needs to have a user-base on which to progress and
be developed. even a small break in continuity would mean, instantly,
bit-rot that would make it incredibly hard to reinstate the
alternatives.

that should have the systemd team very very alarmed. that nightmare
scenario where you will *have* no major competition is happening RIGHT
NOW.

gentoo are about the only (major-ish) GNU/Linux distro that's even
remotely contemplating providing alternatives, and the only reason
they may consider doing so is down to the unique nature of their user
and developer base: users are happy to do near-complete or complete
recompiles of the entire code-base (!).

centos: going with the flow of fedora.
debian / ubuntu: converted. abandoned everything but systemd. going
with the flow.
slackware: minimising workload (staying on the other side of the fence).
puppy / mint: going with the flow of ubuntu.

yehhh.. we could get into that. we could get into various technical
analyses of the benefits of each, and in fact many _many_ people have
done exactly that... i've read as many of them as i can stand [if i am
honest about it :) ]

... and i feel that all of the analyses miss the point, which is,
primarily, that the rapid [and accidental] exlusionary adoption of
systemd right across the board is destroying - almost wholesale - the
alternatives.

with no userbase, they won't *have* user or developer mindshare by
which continued work can be justified to consider including them in
any major distribution.

Post by Tom Gundersen
Please do not ask systemd to be less useful, please ask other projects
to implement better (for whichever metric of "better" you want to
apply) solutions to the problems those developers face.

i'm not asking systemd to be "less useful", i'm asking the team to
take stock of the situation, to be aware that there isn't going to be
anything *other* than systemd for anything but the most obscure
(mostly source-compiled) GNU/Linux distributions such as openwrt,
openembedded, gentoo and so on.

which leaves anyone who feels that the way that they use and maintain
their current GNU/Linux distro (be it desktop or server) is better off
not having systemd *despite* the advantages of systemd, being
completely and utterly screwed.

and... betrayed, if i am absolutely honest.

I do not see how the systemd project or anybody else can change that
at this point.

well, apart from saying "really, you should have listened more
carefully before rushing ahead" - a statement which doesn't help fix
the problem *now* but may help others in the future (reading this on
archives) to avoid a similar situation (one might hope), i feel that
it really is down to everyone to work out how to make sure that other
init systems can be included - at runtime - before it really _is_ too
late.

Post by Tom Gundersen
My experience is that many people out there are beyond
a rational debate at this point.

they've only become irrational because they felt like they were being
ignored or betrayed. and this really is quite a complex subject that
many perfectly rational people have been... simply unable to
comprehend [i'm good at analysing cross-project dependencies and
issues, and even i am having difficulty getting to the bottom of what
the hell is really going on here].

the other issue i feel is that these are technically-minded people,
usually who have a charter that says they have to prioritise and
respect *technical* decisions (the Apache Software Foundation Charter
is one), and there's simply no precedent - on such a wholesale
distro-wide scale, by which they can guage what to do.

so they try their best but honestly it's so complex, and so out of
their experience, that, sadly, they get into severe arguments and we
end up with good people leaving the projects that they love, no longer
wishing to speak to people that they have worked with in some cases
for nearly 2 decades of their lives.

i have to say it but this is by far and above _the_ worst situation
that the software libre community has ever faced. all other bust-ups
resulting in forks or new code being created are nothing compared to
this, because it's an "all-at-once" wholesale conversion that's
streaked across all the major distros.

Post by Tom Gundersen
And I explicitly want to include
people from both sides of the fence in this statement.

... yeah. i've read enough to know that's very true. the key here,
after talking to a lot of people, i think is that this really isn't a
"technical merit" issue, it's a strategic one across *all* the
GNU/Linux distros.

... do you know of any one person who is authoritatively making
strategic decisions across all of the GNU/Linux distros? ...
rhetorical question btw... :)

Post by Tom Gundersen
I am afraid we will have to sit this out.

sitting this one out is only going to make matters worse. you're on the clock.

Isen't it amazing what kind of stories some anonymous cowards make up
over at slashdot? There are some gems of creativity in some of those
systemd flamefests.

amazingly the one i started still has, a week later, a few comments
being added (750 and climbing). it also differs in that the comments
*are* quite rational and insightful if they're marked as "insightful"
by moderators. even the flame-fest individuals served a really useful
purpose of allowing saner minds to make sensible contributions through
contrast.

l.

Zbigniew Jędrzejewski-Szmek

2015-02-23 00:52:06 UTC

Post by Luke Kenneth Casson Leighton
i'd like you to look at this list of debian packages that are
http://lkcl.net/reports/removing_systemd_from_debian/list_of_libsystemd0_dependent_packages.txt

I understood most of these dependencies to be indirect: Packages that
depend on other packages that in turn depend on libsystemd. Is that
correct?

I think this is the crux of the matter. Please accept the fact that
this compile time switch does not preclude runtime decision making at
all. When not running under systemd, calls into libsystemd degrade
into silent noops. So they only "cost" that is an extra unused 600kb
library, which is completely insignificant.

Zbyszek

Cameron Norman

2015-02-23 01:25:11 UTC

Post by Zbigniew JÄdrzejewski-Szmek

On Sun, Feb 22, 2015 at 4:52 PM, Zbigniew Jędrzejewski-Szmek

Post by Luke Kenneth Casson Leighton
i'd like you to look at this list of debian packages that are
http://lkcl.net/reports/removing_systemd_from_debian/list_of_libsystemd0_dependent_packages.txt

I understood most of these dependencies to be indirect: Packages that
depend on other packages that in turn depend on libsystemd. Is that
correct?

And when it is significant you are usually in situation where you are
compiling your own packages and can remove the systemd compile time
option.

--
Cameron Norman

Luke Kenneth Casson Leighton

2015-02-23 01:42:44 UTC

Post by Zbigniew JÄdrzejewski-Szmek

On Mon, Feb 23, 2015 at 1:25 AM, Cameron Norman

Post by Cameron Norman

Post by Luke Kenneth Casson Leighton
...except that its introduction (usually --with-libsystemd) in those
100 (or so) packages has been done in a mutually-exclusive,
hard-compile-time switch that *excludes* the possibility of dynamic
(runtime) decision-making.

And when it is significant you are usually in situation where you are
compiling your own packages and can remove the systemd compile time
option.

*thinking this scenario through*.... yes. typically the outlier OSes
and build environments (the embedded ones). openwrt, buildroot,
openembedded, and so on.

emdebian is an interesting corner-case, there, which is probably
going to be adversely affected. emdebian is unique in that it isn't a
pure from-source embedded OS, it's a half-way house between
precompiled (best of debian) and a base on which you are encouraged to
cross-compile further packages.

unlike other scenarios (all of which are from-scratch complete source
recompiles, usually cross-compiled), emdebian _is_ going to be caught
between a rock and a hard place on that one.

l.

Luke Kenneth Casson Leighton

2015-02-23 02:08:26 UTC

Post by Zbigniew JÄdrzejewski-Szmek

On Mon, Feb 23, 2015 at 12:52 AM, Zbigniew Jędrzejewski-Szmek

the problem, zbigniew, is that the intended use of this "silent noop"
feature - to make it *possible* to have an alternative PID1 - *hasn't
happened*. any upstream software developer who has added in support
for systemd has done so by *ripping out* the former alternative code.
not a single upstream system that i've seen has done *any* kind of
run-time detection *at all*. it's all compile-time.

aside from getting the message across to upstream developers about
doing runtime detection, i feel that what you guys really need to do
is to set up conferences with everyone, to talk - urgently - about
ways to ensure that the alternative systems which the wholesale
adoption of systemd has excluded may be reinstated as *runtime*
choices (not compile-time). that may mean discussing a set of APIs
that end up being DL'd (like PAM is, right now), it may mean doing
something similar to apache loadable modules, or something like the
infrastructure behind python objects, i don't know, but you *need to
talk*.

and you know what? if you were to say, "we're genuinely concerned
that the alternatives to systemd are being locked out, let's talk,
urgently. we really mean it", i think you'd find that people would
respond positively.

the situation now is one where people believe that systemd is being
heavily promoted to the exclusion of all other PID1 alternatives,
developed with a focus on fedora / redhat to the exclusion of all
other distros, developed for desktop systems *only* to the exclusion
of servers and embedded systems... it's no wonder that there's a lot
of upset people in the software libre community!

i know it sounds weird to go backwards, but the situation is -
amongst other not very nice things - that the GNU/Linux world now has
a new monoculture attack vector at the PID1 level.... in code that's
being *actively developed and extended, dramatically*.

it was bad enough that shellshock meant that sysvinit was vulnerable
right across the board of every GNU/Linux distribution. but at least
sysvinit is old enough to have had significant security audit and
review over the decades, such that when shellshock was fixed, that was
it, problem solved [until the next vulnerability...]

contrast that to the situation now: with systemd being so actively
developed and then deployed across literally every major GNU/Linux
distro without exception, the possibility for serious vulnerabilities
having a disproportionately large adverse effect is much higher than i
feel it should be.

l.

Martin Pitt

2015-02-23 06:17:24 UTC

Post by Luke Kenneth Casson Leighton
the problem, zbigniew, is that the intended use of this "silent noop"
feature - to make it *possible* to have an alternative PID1 - *hasn't
happened*.

It sure has. Debian supports systemd, SysV init, and to a lesser
degree OpenRC and upstart.

Post by Luke Kenneth Casson Leighton
any upstream software developer who has added in support for systemd
has done so by *ripping out* the former alternative code. not a
single upstream system that i've seen has done *any* kind of
run-time detection *at all*. it's all compile-time.

libsystemd does that very run-time detection, and upstream software
usually falls back to the "normal" way of opening sockets if socket
activation via libsystemd fails (either because you aren't running
systemd or the service isn't socket activated).

You didn't follow, or see the results of the big Debian systemd debate
at all, did you? :-)

Pretty please do some actual research about the situtation first, and
keep apart libsystemd (harmless, works with any init system) from
systemd (the init system, pid 1, services around it, etc).

Martin

--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

Luke Kenneth Casson Leighton

2015-02-23 12:41:12 UTC

Post by Martin Pitt

Post by Luke Kenneth Casson Leighton
the problem, zbigniew, is that the intended use of this "silent noop"
feature - to make it *possible* to have an alternative PID1 - *hasn't
happened*.

It sure has. Debian supports systemd, SysV init, and to a lesser
degree OpenRC and upstart.

see below, last paragraph.

Post by Martin Pitt

see below, last paragraph.

Post by Martin Pitt

You didn't follow, or see the results of the big Debian systemd debate
at all, did you? :-)

the one where people were violently rude to each other? of course not.

Post by Martin Pitt
Pretty please do some actual research about the situtation first, and
keep apart libsystemd (harmless, works with any init system) from
systemd (the init system, pid 1, services around it, etc).

martin, this is very condescending. please don't do that again. it
also misses the larger picture.

what the distros do is fait-accomplit driven by the decisions of the
upstream developers. what the upstream developers do is
fait-accomplit driven by the decisions of their dependencies.
everyone has been working in near-isolation (which is normally fine
and absolutely nothing wrong with that), except that in this one very
specific unique instance it's resulted - *entirely by accident* the
abandonment of the alternatives to systemd. if what you said was
genuinely true - that there is choice *within* the distros,
http://angband.pl/debian/, TriOS, and devuan would not exist.

l.

Reindl Harald

2015-02-23 12:48:09 UTC

Post by Luke Kenneth Casson Leighton
also misses the larger picture.
what the distros do is fait-accomplit driven by the decisions of the
upstream developers. what the upstream developers do is
fait-accomplit driven by the decisions of their dependencies.
everyone has been working in near-isolation (which is normally fine
and absolutely nothing wrong with that), except that in this one very
specific unique instance it's resulted - *entirely by accident* the
abandonment of the alternatives to systemd. if what you said was
genuinely true - that there is choice *within* the distros,
http://angband.pl/debian/, TriOS, and devuan would not exist.

the larger picture is that every project writes it's code and choses to
rely on code of other projects as dependency - that's it - not more and
not less

if there would be a big need and a large base of people to keep and
maintain systemd free distributions it just would happen and if we here
*anything* about "devuan" in two years is more than questionable

what you are missing at "your big picture" is based on mis-information
or how did you come to the conclusion in this thread that systemd is not
for servers (frankly tell that my servers) while a ton of options making
sense more on servers than on workstations or that it is not for
embedded devices while http://0pointer.net/blog/projects/stateless.html
is especially useful for exactly that type of setups

Lennart Poettering

2015-02-23 12:56:32 UTC

On Mon, 23.02.15 12:41, Luke Kenneth Casson Leighton (***@lkcl.net) wrote:

Luke, you are now on moderation.

I am sorry, but I don't find what you are writing particularly useful
or new. You keep repeating the same untruths, and I'd prefer if we
could again focus on technical discussions on the mailing list
instead. Please move this discussion elsewhere.

Thanks,

Lennart

--
Lennart Poettering, Red Hat

Cristian Rodríguez

2015-02-23 06:33:33 UTC

Post by Luke Kenneth Casson Leighton
the problem, zbigniew, is that the intended use of this "silent noop"
feature - to make it *possible* to have an alternative PID1 - *hasn't
happened*. any upstream software developer who has added in support
for systemd has done so by *ripping out* the former alternative code.
not a single upstream system that i've seen has done *any* kind of
run-time detection *at all*. it's all compile-time.

This is because software is written mostly by sane people who has at
least a clue about what they are doing and talking, they are not doing
what you wish, because what you are proposing is batshit insane.

Ha! that's a funny one.. why should we do that? the burden on doing that
is on the people that want this theorical alternatives.

that may mean discussing a set of APIs

Post by Luke Kenneth Casson Leighton
that end up being DL'd (like PAM is, right now),

PAM is not dlopen'ed.. pam *modules* are.. and PAM is not something to
cite as an example how to do things *today* in 2015..

Post by Luke Kenneth Casson Leighton
the situation now is one where people believe that systemd is being
heavily promoted to the exclusion of all other PID1 alternatives,
developed with a focus on fedora / redhat to the exclusion of all
other distros, developed for desktop systems *only* to the exclusion
of servers and embedded systems... it's no wonder that there's a lot
of upset people in the software libre community!

You dropped your tinfoil hat now..

Post by Luke Kenneth Casson Leighton
i know it sounds weird to go backwards, but the situation is -
amongst other not very nice things - that the GNU/Linux world now has
a new monoculture attack vector at the PID1 level.... in code that's
being *actively developed and extended, dramatically*.

Please go and learn how and particulary *why* things work a certain way
before telling people how to do it, in fact don't tell.. .post patches
or experiment yourself.

You can dlopen systemd libraries at your own risk, if you know exactly
what you are doing and why it will work..in most cases it will end in a
terrible mess that we will get the blame for it.. I just wrote a patch
to disallow dlopen of libsystemd alltogether..I hope it won't be needed
because I still trust developers not to be that misguided.

Tomasz Torcz

2015-02-23 10:08:40 UTC

Luke, please be careful with quantificators like ”any upstream developer”.
I invite you to check sd_notify() and/or socket activation that I did* in
various projects: rrd_cached, iscsid, transmissionbt, owfs, uptimed, tor.
In each case there is no degradation when systemd is not used, previous
way of doing things is preserved. Enhancement patches (Type=notify, watchdog)
are NOOP when systemd is not used. Some of the above project don't even
lin with libsystemd.

* in case of Tor and transsmission I've either inspired the change or expanded
it over original patches; for others I prepared a patch which was merged
by upstream

--
Tomasz Torcz ,,If you try to upissue this patchset I shall be seeking
xmpp: ***@chrome.pl an IP-routable hand grenade.'' -- Andrew Morton (LKML)

Tobias Hunger

2015-02-23 11:40:23 UTC

Hi Luke,

On Mon, Feb 23, 2015 at 3:08 AM, Luke Kenneth Casson Leighton

You are starting to mix things: Libsystemd does work the way it was
intended: They make it possible to add socket activation to daemons
and make those report back to systemd. That part is nicely
encapsulated.

There are unrelated things like systemd-logind, which are higher up in
the stack. Those are indeed harder to replace. Part of the problem is
that there are no fully functionally equivalent pieces of software to
replace the systemd stuff with. ConsoleKit and Co. do have severe
limitations, that (apparently) can not be fixed without resorting to
cgroups.

Post by Luke Kenneth Casson Leighton
aside from getting the message across to upstream developers about
doing runtime detection, i feel that what you guys really need to do
is to set up conferences with everyone, to talk - urgently - about
ways to ensure that the alternative systems which the wholesale
adoption of systemd has excluded may be reinstated as *runtime*
choices (not compile-time). that may mean discussing a set of APIs
that end up being DL'd (like PAM is, right now), it may mean doing
something similar to apache loadable modules, or something like the
infrastructure behind python objects, i don't know, but you *need to
talk*.

I do expect developers to search for information and not wait for "the
systemd project" (whoever that is) to feed it to them.

E.g. I would expect people that want to do a distribution without
systemd will actually find out what systemd does first, so that they
can do informed decisions on how to replace the individual parts. The
documentation of systemd is -- at least in my experience -- pretty
good in most areas. And where it is lacking people tend to act when
somebody pokes them about it and actually improve on it.

What do you want to be able to reinstate as runtime choices? There is
a lot of stuff in the systemd repository. Those parts are at different
levels of the stack and with varying levels of dependency on other
parts.

Your request is so generic that it is impossible to answer.

That is a complete mis-representation of the project. Unfortunately I
doubt there is anything the systemd project can do to rectify that
impression at this point: Many people are past rational arguments at
this point.

I do not understand this point. Systemd PID1 is pretty small, the rest
of systemd runs with severely limited privileges. In fact most
services are more tied down than their traditional counterparts ever
were.

Post by Luke Kenneth Casson Leighton
it was bad enough that shellshock meant that sysvinit was vulnerable
right across the board of every GNU/Linux distribution. but at least
sysvinit is old enough to have had significant security audit and
review over the decades, such that when shellshock was fixed, that was
it, problem solved [until the next vulnerability...]

Shellshock was lurking in the code for decades, evading all the
security audits and reviews done on bash in all that time. Xorg has
had similar long-lived bugs.

Sysv-rc also depends on a lot of external code. The shell, core
utilities, awk and whatnot. I doubt all the whole mess ever was
audited for security in its total.

Post by Luke Kenneth Casson Leighton
contrast that to the situation now: with systemd being so actively
developed and then deployed across literally every major GNU/Linux
distro without exception, the possibility for serious vulnerabilities
having a disproportionately large adverse effect is much higher than i
feel it should be.

The same is true for the Linux kernel, and I am more than happy about
that. Bugs do get found in actively developed code, simply because
people actually look at that code!

Maybe we should all run more BSDs? That would also reduce the reliance
on systemd:-) Two birds with one stone.

Best Regards,
Tobias

Tobias Hunger

2015-02-23 11:00:30 UTC

Hello Luke,

On Mon, Feb 23, 2015 at 12:58 AM, Luke Kenneth Casson Leighton

Post by Tobias Hunger
I understood most of these dependencies to be indirect: Packages that
depend on other packages that in turn depend on libsystemd. Is that
correct?

Good. At least all sides agree on the facts:-)

Post by Luke Kenneth Casson Leighton
they made it possible to remove systemd entirely. they had to
recompile cups, stunnel, udisks2, upower, util-linux and many more...
http://angband.pl/debian/dists/nosystemd/main/binary-amd64/Packages
but here's the problem: because there is no dynamic run-time
decisions possible here, people are forced to adding that (unofficial)
repo and to risk their systems in the process. reverting is also just
as risky.

Libsystemd provides the code that does the runtime detection. The
alternative is to copy and paste the code contained in that library
into those 100 binaries directly, with all the negative consequences.

Post by Tobias Hunger
Libsystemd's job is basically to provide exactly what you ask for: A
wrapper around systemd functionality, that fails gracefully in case
systemd is not used.

Libsystemd is fully user control-able: Run systemd as PID1 and
libsystemd does something for you, run anything else as PID1 and it
turns into a noop. Not running systemd as PID1 does state "I do not
want to use systemd" pretty clearly, doesn't it? I see no need to
repeat that statement of intent in some config file.

Again: That seems to be exactly the behavior you are asking for.

Post by Luke Kenneth Casson Leighton
in many ways it doesn't matter that i (or anyone else) finds the
solution objectionable. what should matter - to you - is that i (or
anyone else) wishes to make that choice [we might have a solution
above btw as you can see].

I am afraid you lost me.

You want to not use systemd: Fine. You want to not have any code from
the systemd repository on your system? Fine, too. You want to use a
distribution that does not include systemd code? Fine, I can get that,
I am lazy, too.

But where is the relation between "your distribution does not provide
you with packages without any systemd code that you want" and
systemd-the-project?

Distributions do include a lot of packages with dependencies that make
sense for part of the user base only, provided it does not hurt the
other users too much. This is exactly such a case: Users of systemd
get to use socket activation and whatnot, users that do not use
systemd get a library onto their system that eats a bit of disk space
and RAM and does nothing otherwise.

How does this particular decision of your distribution cause you so
much anguish, while you are ok with packages dragging in other
dependencies you do not strictly need? Why is this a matter of choice
here and not elsewhere?

Post by Luke Kenneth Casson Leighton
.... i think... really, although technically what i am asking
*appears* to be "achieved in an equivalent fashion", they're not
entirely the same.
bear with me whist i think this through, ok?
my thoughts go like this: virtually every GNU/Linux distro has gone
for a mutually-exclusive all-or-nothing choice: systemd or out. even
Debian has had to provide systemd-shim to quotes stop the whiners
quotes. it has "Breaks: systemd < 209" in the package file. they
*did not* notice that systemd may be easily disabled by not running as
PID 1.

So far we talked about libsystemd as a dependency to core packages in Debian.

You are now widening the scope of this discussion. Is that your intention?

You are heading of into systemd-logind land at this point. This is a
very different discussion, as it does effect users and not-users in a
very different way than libsystemd does.

I would be looking forward to having that with you, but we should keep
that in a separate thread and do not mix the topics.

Post by Tobias Hunger
I would personally like to see a "libinitd" which brings the socket
activation features that is provided to daemons as part of libsystemd
to other init systems (that can support those). That would make it so
much easier for upstreams to support more than one init system. But I
would expect that to be implemented by the teams working on
alternatives to systemd or by distributions centered around other init
systems.

i would _love_ to see a situation where there is more dialog and
collaboration between the developers of systemd and the maintainers of
the alternatives.

So would I, but I actually think the people that are willing to
discuss to engage in such a discussion are already doing so.

And before a libinit is possible other init systems might need to
provide features like socket activation, etc. first so that there is
something one can unify in one interface.

Post by Luke Kenneth Casson Leighton
and that i think, guys, is part of the problem. the development of
systemd is progressing at such a fast pace, with no dialog with any of
the peers (developers of openrc, maintainers of sysvinit, depinit -
currently me - no-one) that there are people - really experienced
people -really freaking out right now, and they don't fully understand
why.

Yes, systemd is progressing at a fast pace. But I personally do feel
well informed. There is a *lot* of discussion going on with all kinds
of people. There is "dialog with any of the peers", right here in this
list. There is more going on in person-to-person meetings, at
conferences and whatnot.

Communication is a two-way street. You can not force people to "listen
to the gospel of systemd", you can only reach out and offer to discuss
with anybody willing to engage in such a discussion.

Post by Luke Kenneth Casson Leighton
they know that something's wrong, but are having a real hard time
putting their finger on it. that's led to some er... charitably we
can call them debates ... that have resulted in really senior
dedicated people who have spend in almost all cases over a _decade_ of
their lives on debian... quitting their role and involvement. and the
software libre community as a whole is seriously diminished by the
loss of their input.

This is yet another matter. Let's please keep that separate from libsystemd.

Post by Tobias Hunger
Sure. I am looking forward to that! I am convinced a bit of
competition and fresh ideas will do systemd a hell of a lot of good:-)

the problem is, systemd conversion is so near-exclusive and so rapid
that there won't *be* any competition.

Oh, there is: Most of it admittedly outside of Linux.

Post by Luke Kenneth Casson Leighton
the "competition" needs to have a user-base on which to progress and
be developed. even a small break in continuity would mean, instantly,
bit-rot that would make it incredibly hard to reinstate the
alternatives.

I am convinced that you do can get a user-base for anything that is
not completely crazy:-) And even if it is completely crazy you will
find some fans. Behold the power of the internet:-)

Challengers need to provide value for developers as well as users. So
far all fail on the developers front (at least as far as I can tell):
They must at least provide an idea how to address the points that
systemd and its surrounding infrastructure tackles. Most projects I
looked into are still stuck in a state of denial ("No, systemd does
not address any problems" or "No, that thing systemd tackles is not a
problem").

Post by Luke Kenneth Casson Leighton
that should have the systemd team very very alarmed. that nightmare
scenario where you will *have* no major competition is happening RIGHT
NOW.

I do not see the situation that dire: There are quite a few systems
outside Linux left that can be used as inspiration.

Post by Luke Kenneth Casson Leighton
... and i feel that all of the analyses miss the point, which is,
primarily, that the rapid [and accidental] exlusionary adoption of
systemd right across the board is destroying - almost wholesale - the
alternatives.
with no userbase, they won't *have* user or developer mindshare by
which continued work can be justified to consider including them in
any major distribution.

I am not sure whether that is a bad thing or just something we did not
have before.

Everybody that cares to work on drivers or kernel schedulers does
converge in the linux kernel area. Why is it bad that everybody that
cares about the Linux plumbing layer converges into the systemd space?

Having one master repository for the Linux kernel has not stopped it
from integrating new ideas all the time. As long as the same is true
for systemd I am fine with that.

Yes, this is not what we used to do, but what we used to do was not
perfect either, so I am open to give this approach a try.

Post by Luke Kenneth Casson Leighton
which leaves anyone who feels that the way that they use and maintain
their current GNU/Linux distro (be it desktop or server) is better off
not having systemd *despite* the advantages of systemd, being
completely and utterly screwed.
and... betrayed, if i am absolutely honest.

I am sad that you feel this way, but am somewhat unsure what I can do
about this.

Post by Luke Kenneth Casson Leighton
well, apart from saying "really, you should have listened more
carefully before rushing ahead" - a statement which doesn't help fix
the problem *now* but may help others in the future (reading this on
archives) to avoid a similar situation (one might hope), i feel that
it really is down to everyone to work out how to make sure that other
init systems can be included - at runtime - before it really _is_ too
late.

Do you see any concrete points where the systemd project could have
handled the situation differently?

All they did was to provide some code that the developers of the
project thought to be useful. Other developers agreed with them and
adopted their code. That is how open source projects work. Why is this
one different?

Post by Tobias Hunger
My experience is that many people out there are beyond
a rational debate at this point.

This is getting way outside my social competence comfort zone:-) I
have to skip at this point.

Post by Tobias Hunger
I am afraid we will have to sit this out.

sitting this one out is only going to make matters worse. you're on the clock.

It did work when this horrible, complex abomination called sysv-rc was
introduced to replace the nice, simple and easy to understand BSD
style init script we had before:-)

Ah, the good old times. That was a huge controversy, too -- at least
for the much smaller community we were back then.

Best Regards,
Tobias

Lennart Poettering

2015-02-18 10:00:47 UTC

Post by Luke Kenneth Casson Leighton
i note that there was announcement recently that the systemd team
'listens to users', so i am taking you at your word on that.

Hmm, I am not aware of such an "announcement". I generally listen
though, but I don't always agree. I particularly don't listen to badly
researched conspiracy theories.

Well, I doubt this is correct, and even if it was, I am not sure what
systemd upstream has to do with it. Convince the upstream developers
in question not to link against systemd's libraries, or convince the
distros not to package it like that. We just offer a library, which
apparently is interesting to people to use, but whether they use it is
really not up to us.

Also, what precisely is the problem with having a dependency on that
library? If it's pulled in and nothing else of systemd, where
precisely is your problem? It's just code lying around, and there is
tons of stuff like that.

If you think that code we wrote is really evil shit, that should never
touch your system, then I figure it's too late. The folks involved in
systemd touched a lot more of your system, outside of the systemd
umbrella, our code is in much of the stack, in the desktop and in the
kernel. If you want to avoid using the code we wrote you'll have a
really hard time.

Post by Luke Kenneth Casson Leighton
as i am partly writing to a public audience who may not be so
knowledgeable, please excuse me for describing that the advantages,
are, as you know, that a pre-compiled package may, at runtime, detect
what is available to it and use it. it may even be configured via a
config option to disable the use of that functionality at runtime even
if the dynamic library is present. you know this.

dlopen() is something the programs *using* a library must make use of,
it's not up to us, the providers of the library.

Post by Luke Kenneth Casson Leighton
i am one of the few people who can cut through all that, who has gone
to the trouble of digging into why libsystemd0 is found to be so
objectionable.

Good for you!

Good one!

Lennart

--
Lennart Poettering, Red Hat

Luke Kenneth Casson Leighton

2015-02-23 01:37:08 UTC

On Wed, Feb 18, 2015 at 10:00 AM, Lennart Poettering

Post by Lennart Poettering

Post by Luke Kenneth Casson Leighton
i note that there was announcement recently that the systemd team
'listens to users', so i am taking you at your word on that.

Hmm, I am not aware of such an "announcement".

somewhere there was a blog post i read of yours where (probably
garbled) i got the impression that that was the case. a possible
explanation is that i generally perceive "blogs" to be
"announcements".

Post by Lennart Poettering
I generally listen though, but I don't always agree.

that's good to hear

Post by Lennart Poettering
I particularly don't listen to badly researched conspiracy theories.

good! i trust you're happy to discern the difference between such
and an attempt to understand - from a cross-project perspective - what
the heck is going on here.

i'm applying my reverse-engineering hat, here (normally reserved for
assembly code or network systems) to the situation surrounding
systemd's... rejection, shall we say. as i mentioned to tobias's post
a few minutes ago, the situation - the bad feeling that's got
sysadmins and experienced unix users worried and has made enemies out
of decades-established acquaintances in high profile GNU/Linux distros
- is completely without precedent in the history of free software.

i don't have all the answers, but i _can_ tell that something's
drastically wrong.

Post by Lennart Poettering

Well, I doubt this is correct,

lennaaaart, smacked wrist! :) greg's answer (asking me to provide
the information on how that file was created) is the exemplary one.
apologies that i wasn't keeping up-to-date with the incoming messages,
otherwise i would potentially have provided you with the commands to
replicate the above list, earlier, so there would be no doubt.

ohh ok i'll work out the commands rather than do it by hand like i
did for that list last week, here it is:
***@bigmac:~$ apt-rdepends -r libsystemd0 | grep "Reverse Depends" |
cut -f2 -d: | cut -f2 -d' ' | sort | uniq | wc
Reading package lists... Done
Building dependency tree
Reading state information... Done
4583 4583 70416

the other one (which provides the list of *immediate* dependencies),
is "apt-cache rdepends libsystemd0"

Post by Lennart Poettering
and even if it was, I am not sure what
systemd upstream has to do with it.

i went over this in the [now that i re-read it, very long...
sorry...] reply to tobias.

Post by Lennart Poettering
Convince the upstream developers
in question not to link against systemd's libraries, or convince the
distros not to package it like that.

well, you could provide hints in the documentation (and force them to
be read by deliberately changing the API)

and you could provide an example by leading the conversion. for
example, i understand that you're the developer behind portaudio?
that would be a good place to start, showing people how to begin the
process of dlopen()ing libsystemd0, documenting it well so that it was
easy to follow.

Post by Lennart Poettering
We just offer a library, which
apparently is interesting to people to use, but whether they use it is
really not up to us.

well... they've used it, alright. so much so that there isn't a
single major GNU/Linux distro left, nor a major GNU/Linux desktop
environment, that may be used *without* systemd or a
systemd-compatible component. xfce4, lxde, kde, gnome - they're all
now critically and exclusively dependent on systemd. clarification,
there: i'm aware for example that KDE5 now uses the dbus logind API,
so the FreeBSD team are considering writing a replacement logind
rather than fishing through the old commits that removed the previous
code.

and that's the problem in a nutshell - the chain if you like of
decisions that have got us to this very bad situation:

* distros are forced to follow suit on upstream decisions, without
consulting what any other distros do
* distros are (with the exception of gentoo, freebsd and macports as
they are all source-based) forced to hard-code the best possible
compile-time options that are *available* to them
* the upstream decisions on a per-app basis have been made "in
isolation" without a proper across-the-board cross-project analysis.
* the development of systemd has been done in the perfectly
reasonable way of using fedora as the proving-ground... in isolation
from other distros.

end result: systemd is now *the* de-facto PID 1 across all major
GNU/Linux distros in the space of something like under a year, with
all other work almost completely locked out and being forced to be
"compatible" with systemd in some way (via replicating the API in some
way, by forking older versions of systemd, writing
function-for-function API-compatible equivalent d-bus services which
is *really* hard to get right, and so on).

Post by Lennart Poettering
Also, what precisely is the problem with having a dependency on that
library? If it's pulled in and nothing else of systemd, where
precisely is your problem? It's just code lying around, and there is
tons of stuff like that.

unfortunately, on all the major GNU/Linux distros, the correlation
between "libsystemd0 being present" and "every other PID1 management
system being excluded from consideration" is near 100%.

the detection tobias mentioned that it may have been *intended* that
libsystemd0 provide, of remaining dormant if systemd is not detected
as PID1 should signal to the upstream developers that they should
*permit* alternative PID1s to run, but in reality the upstream
developers have simply ripped out all alternative code - not even
provided a compile-time switch to support the previous code. the KDE5
developer recently removed the code that used console-kit (etc. etc.
whatever it was) and replaced it with a d-bus call to logind, for
example.

Post by Lennart Poettering
If you think that code we wrote is really evil shit, that should never
touch your system, then I figure it's too late. The folks involved in
systemd touched a lot more of your system, outside of the systemd
umbrella, our code is in much of the stack, in the desktop and in the
kernel. If you want to avoid using the code we wrote you'll have a
really hard time.

there are people who are working actively on doing exactly that.
they're prepared to do what it takes. personally, i'm gearing up to
become the maintainer of depinit (i just got it compiling on 64-bit,
last week).

the reasons why (their motivations for doing so) are i believe
irrelevant. they have the right to consider doing that, and as tobias
points out, they should be encouraged for providing competition from
which the systemd team could learn. collaboration at its best.

and *importantly*, provide a backup choice in case something goes
wrong with systemd. hit-by-bus scenarios, monoculture development
leading to ecosystem-wide malware exploits across multiple GNU/Linux
distros, that sort of thing...

... except the rapid and wholesale exclusive adoption of systemd
across the horizontal (isolated) layer of upstream and the vertical
(isolated) layer of the distros *isn't allowing for any competition to
grow*.

regarding the "evil shit" innuendo, amusing as it is, that i feel is
a red herring. i'm aware for example of several independent technical
analysis of the code quality of systemd have encouraged others (the
author of uselessd being one example) to make it much more
cross-platform portable (he took version 209 i believe as the starting
point). that's a good thing: the end result is that the design that
you went to a lot of trouble to put together now has a [partial]
implementation that is of very high portability quality so that it
could even conceivably be installed on FreeBSD or god help us all
Windows or ReactOS... but this is all irrelevant.

everyone - the GNOME team, the KDE team, the cups developers - they
all work (very very hard, and very competently, it has to be
emphasised) in isolation. the distro maintainers have been presented
with the fait-accomplit decisions of the upstreams. _they_ work in
isolation (again, being extremely competent in their dedication to
their chosen tasks).

and suddenly... without anyone noticing... there's no choice left but
to use systemd.

only in the far corners of the software libre community - the outlier
GNU/Linux distros and the BSDs, GNU/Hurds, Minixes and real-time OSes,
in the source-code-based OSes such as gentoo and the embedded ones
such as openembedded, openwrt and buildroot - has systemd *not* become
the exclusive de-facto PID1, where these distros are currently facing
the really hard choice of having to stop following upstream, stick
with older code temporarily whilst they scramble (e.g. with gentoo)
under enormous pressure of their users and of security fixes
potentially having to be back-ported or worse put on hold, to develop
alternatives under less-than-ideal circumstances applications such as
eudev and uselessd.

the wholesale near-exclusive adoption of systemd _really_ has left a
lot of people with some very bad choices, and a hell of a lot of work.
as the people responsible for developing systemd, i feel that it's up
to you to do what you can to minimise the amount of effort that other
people are being put to, as well as to help reduce the enormous
tension building up right across the entire software libre world, as
best you can.

Post by Lennart Poettering

dlopen() is something the programs *using* a library must make use of,
it's not up to us, the providers of the library.

there are ways in which you could provide guidelines, examples, and
even deliberately change the API as an [otherwise purposeless] one-off
(the list of apt-cache rdepends immediate dependencies shows appx 100
packages, each with maintainers, so it's not like 1 person would be
required to do all 100 upstream conversions).

i recall andrew tridgell describing to me in 2000 the best way to
design libraries, i'm sure you're aware of it. bless him he thought i
was a such a complete novice, but i'd been reverse-engineering NT
kernels and had encountered exactly the principle he went to all the
trouble to tell me about. i didn't speak up, because i actually
wanted to hear what he had to say, in case it contained anything i
didn't already know... ah.. i am so silly sometimes, not correcting
people's impressions... ehh never mind... where was i :)

the method was basically to use a vector table [struct containing
higher-order-functions], publishing *only one* actual function (one
publicly exported symbol), and that function took *another* vector
table containing a list of all the functions - *all* of the functions
- that the library would ever need. it was *not permitted* to use any
other functions.

so the [one] exported function would return a dynamically-allocated
instance of the vector table, and the contract was that it would never
call any function other than the ones it was supplied with. the best
implemented libraries would allow you to independently call that [one]
exported function multiple times, with a different parameter
containing a *completely different* set of functions that that
instance should use, and the two "instances" should, under no
circumstances, interfere with each other.

deployment of this technique basically means that there are guarantees
about isolation between different components, and its use in the NT
kernel means that code reviews, changes to APIs and so on are all much
easier.

if a library is designed in this way, then dlopen() need only be
called once: to obtain that all-important one publicly exported
function that returns a struct that points to all available functions.

btw as you probably know (but i will outline for the benefit of people
reading this through archives in the future, hope you more experienced
people don't mind me doing this) you don't *have* to pass in a vector
table containing the functions that the library should [exclusively]
use, and in fact the majority of libraries that are out there *don't*
follow this practice... it just means that there's less guarantees,
and code analysis (unintended side-effects of modifying or improving
the lbirary) is much harder.... but that tends to be the norm.

if by some amazing chance libsystemd0 is *already* designed along
these lines, then i will be very embarrassed and will go have a good
laugh for a bit and get back to you, sharpish / sheepish or both.

if not, then you have an opportunity to make it clear to the upstream
developers the *real* intent behind why libsystemd does its PID1
detection, namely that the applications should be making *run-time*
decisions about what PID1 init system to be using, by preparing some
documentation, converting the code over thus forcing all upstream to
convert to the (recommended) method of dlopen()ing, all at once.

or not. or something else (better than the above idea). and you _do_
have the choice to allow this situation to perpetuate... along with
the hate-mail, the fighting, the vitriol, the burn-out and worse...

anyway, if you got this far, i must apologise for such in-depth
responses. i do that... then i go away, never to return until another
exceptionally-complex cross-project issue arises. once i know that
either the situation is truly hopeless and would be made worse by me
being in the story, or that i've done my bit by making the
consequences clear and you're happy but firmly made it clear that no
action is going to be taken *despite* the warnings i've given, or i
know that, as the competent team that you are, you have listened and
are planning on taking action, i'll go away happy.

i leave it up to you to decide which that should be.

l.

Cristian Rodríguez

2015-02-23 06:56:39 UTC

Post by Luke Kenneth Casson Leighton
well, you could provide hints in the documentation (and force them to
be read by deliberately changing the API)

Wow.. so what you want is even nuttier than I thought..

Post by Luke Kenneth Casson Leighton
that would be a good place to start, showing people how to begin the
process of dlopen()ing libsystemd0, documenting it well so that it was
easy to follow.

No, again. using dlopen with libsystemd is crazy, you will only turn
software even more complex and ugly than already is. and with this
little gem:

-- a/Makefile.am
+++ b/Makefile.am
@@ -3046,7 +3046,7 @@ libsystemd_la_CFLAGS = \
$(libsystemd_journal_internal_la_CFLAGS)

libsystemd_la_LDFLAGS = \
- $(AM_LDFLAGS) \
+ $(AM_LDFLAGS) -Wl,-z,nodlopen \

we can stop the madness from becoming reality :-)

Post by Luke Kenneth Casson Leighton
* distros are forced to follow suit on upstream decisions, without
consulting what any other distros do

No, distros are welcome to come here and influence the direction of the
project and its components, before reality imposes itself. those who do
the work, make decisions.

Lennart Poettering

2015-02-23 12:11:47 UTC

Post by Lennart Poettering
Convince the upstream developers
in question not to link against systemd's libraries, or convince the
distros not to package it like that.

well, you could provide hints in the documentation (and force them to
be read by deliberately changing the API)

To make this clear: I think dlopen() is a really bad idea. It
complicates code, hides dependencies, fucks up API versioning and I am
very sure I will not push people to use it.

Post by Luke Kenneth Casson Leighton
and you could provide an example by leading the conversion. for
example, i understand that you're the developer behind portaudio?

This is non-sense. I have nothing to do with portaudio.

Post by Luke Kenneth Casson Leighton
that would be a good place to start, showing people how to begin the
process of dlopen()ing libsystemd0, documenting it well so that it was
easy to follow.

Post by Lennart Poettering
We just offer a library, which
apparently is interesting to people to use, but whether they use it is
really not up to us.

This is non-sense.

Post by Luke Kenneth Casson Leighton
* distros are forced to follow suit on upstream decisions, without
consulting what any other distros do

Well, distros don't write software. Upstream projects do. The ones who
write the software make the decisions which APIs the software
use. It's that simple.

If you want to make decisions on how software is written, then write
software. Or pay people to write software. But that's really it.

Post by Luke Kenneth Casson Leighton
* distros are (with the exception of gentoo, freebsd and macports as
they are all source-based) forced to hard-code the best possible
compile-time options that are *available* to them

This is non-sense. Nobody is "forced" by anyone here.

Post by Luke Kenneth Casson Leighton
* the upstream decisions on a per-app basis have been made "in
isolation" without a proper across-the-board cross-project analysis.

Yeah, it's how open source works. The developers can do what they
want. And you have the liberity to fork their stuff if you are
unhappy.

I encourage you to do so, and stop annoying people who actually do the
work.

Post by Luke Kenneth Casson Leighton
* the development of systemd has been done in the perfectly
reasonable way of using fedora as the proving-ground... in isolation
from other distros.

This is non-sense.

When we started Kay was working for Suse not RH, and we probably
adopted more Debianisms in systemd, than -isms from any other distro.

Post by Luke Kenneth Casson Leighton
end result: systemd is now *the* de-facto PID 1 across all major
GNU/Linux distros in the space of something like under a year, with

This is non-sense.

Nah, systemd is 5 years old now. It took five years to get most
distros onboard.

Post by Luke Kenneth Casson Leighton
unfortunately, on all the major GNU/Linux distros, the correlation
between "libsystemd0 being present" and "every other PID1 management
system being excluded from consideration" is near 100%.

This is non-sense.

Post by Luke Kenneth Casson Leighton
the detection tobias mentioned that it may have been *intended* that
libsystemd0 provide, of remaining dormant if systemd is not detected
as PID1 should signal to the upstream developers that they should
*permit* alternative PID1s to run, but in reality the upstream
developers have simply ripped out all alternative code - not even
provided a compile-time switch to support the previous code. the KDE5
developer recently removed the code that used console-kit (etc. etc.
whatever it was) and replaced it with a d-bus call to logind, for
example.

This is non-sense.

When I added logind support to gdm for example I carefully made sure
to keep the ConsoleKit codepaths in place, so that gdm could work
against either, and the choice could be made at runtime. I did that
precisely to allow people like you to stick to CK.

Post by Luke Kenneth Casson Leighton
and *importantly*, provide a backup choice in case something goes
wrong with systemd. hit-by-bus scenarios, monoculture development
leading to ecosystem-wide malware exploits across multiple GNU/Linux
distros, that sort of thing...

This is non-sense. You have no idea how Open Source works. Since the
source is open somebody else can and will take over if I disappear.

Post by Luke Kenneth Casson Leighton
... except the rapid and wholesale exclusive adoption of systemd
across the horizontal (isolated) layer of upstream and the vertical
(isolated) layer of the distros *isn't allowing for any competition to
grow*.

Well, our project apparently was a lot more convincing than all other
options, hence distros adopted it.

If you think you can put together something better, please do, make it
convincing, and people will adopt it instead.

Post by Luke Kenneth Casson Leighton
regarding the "evil shit" innuendo, amusing as it is, that i feel is

It's not an "innuendo". It's pretty explicit.

Post by Luke Kenneth Casson Leighton
a red herring. i'm aware for example of several independent technical
analysis of the code quality of systemd have encouraged others (the
author of uselessd being one example) to make it much more
cross-platform portable (he took version 209 i believe as the starting
point). that's a good thing: the end result is that the design that
you went to a lot of trouble to put together now has a [partial]
implementation that is of very high portability quality so that it
could even conceivably be installed on FreeBSD or god help us all
Windows or ReactOS... but this is all irrelevant.

You obviously have no idea about developing.

Post by Luke Kenneth Casson Leighton
everyone - the GNOME team, the KDE team, the cups developers - they
all work (very very hard, and very competently, it has to be
emphasised) in isolation. the distro maintainers have been presented
with the fait-accomplit decisions of the upstreams. _they_ work in
isolation (again, being extremely competent in their dedication to
their chosen tasks).

I think you work in isolation.

I am certainly not. I partake in more conferences, talk to more people all
day, and take part in more email threads than most.

Post by Luke Kenneth Casson Leighton
and suddenly... without anyone noticing... there's no choice left but
to use systemd.

Well, 5 years in IT is certainly "suddenly". It's a long long time.

You have written an awful amount of non-sense on this mailing list in
the past days. I think it would be appreciated by everybody on this
list if you did this elsewhere instead,

thanks,

Lennart

--
Lennart Poettering, Red Hat

Shawn Landden

2015-02-23 04:35:05 UTC