[systemd-devel] [packaging] split of systemd package

[Lukáš Nykrýn]

Hi,

During systemd.conf we have discussed some recommendation for
downstreams, how they could split systemd to subpackages, so lets
continue that discussion here.

Personally I don't think it makes sense to split the package to get a
smaller core package. Most of our binaries are just few KBs. Only
exception here is /usr/lib/udev/hwdb.d which, on fedora rawhide, has
about 5,2 MB (15% of the whole package).

Other aspect would be minimizing external dependencies. I have made a
list of libraries and which binaries pulls them in [1] and from that
point of view it would make sense to put follow binaries to subpackage:
systemd-pull
systemd-journal-gatewayd
systemd-journal-remote
systemd-journal-upload
systemd-firstboot
systemd-networkd

So I suggest following scheme

systemd
systemd-libs
systemd-devel
systemd-journal-remote (so gatewayd,remote,upload)
systemd-networkd (maybe also with resolved?)
systemd-machine (machined,nspawn,importd)
systemd-firstboot (firstboot,sysusers?,factory stuff?)
systemd-hwdb


Regards

Lukas


[1] https://gist.github.com/lnykryn/bd5de7d9ed39986d5147

[Jóhann B. Guðmundsson]

Post by Lukáš Nykrýn
Hi,
During systemd.conf we have discussed some recommendation for
downstreams, how they could split systemd to subpackages, so lets
continue that discussion here.

I thought the conscious was not recommending downstream to split systemd
into subpackages?

JBG

[Michal Sekletar]

On Wed, Nov 11, 2015 at 11:52 AM, Jóhann B. Guðmundsson

Post by Jóhann B. Guðmundsson
I thought the conscious was not recommending downstream to split systemd
into subpackages?

This decision was recently (at systemd.conf) reevaluated :)

Michal

[Jóhann B. Guðmundsson]

Post by Michal Sekletar
On Wed, Nov 11, 2015 at 11:52 AM, Jóhann B. Guðmundsson

Post by Jóhann B. Guðmundsson
I thought the conscious was not recommending downstream to split systemd
into subpackages?

This decision was recently (at systemd.conf) reevaluated :)

Not everybody can attend conference due to wide variety of reasons so
making decisions there without the input of those in the community that
could not attend is disrespectful.

Anyway what was the cause for the reevaluation in other words why did
people change their mind in this regard?

JBG

[Lukáš Nykrýn]

Post by Jóhann B. Guðmundsson
I thought the conscious was not recommending downstream to split systemd
into subpackages?

I think the previous discussion was more about if we should split core
components of systemd like systemd-logind, which still should stay in
the main package.

And most of distributions split their packages already, so it would be
nice to have some general recommendations.

Lukas

[Lennart Poettering]

Post by Lukáš Nykrýn
Hi,
During systemd.conf we have discussed some recommendation for
downstreams, how they could split systemd to subpackages, so lets
continue that discussion here.
Personally I don't think it makes sense to split the package to get a
smaller core package. Most of our binaries are just few KBs. Only
exception here is /usr/lib/udev/hwdb.d which, on fedora rawhide, has
about 5,2 MB (15% of the whole package).
Other aspect would be minimizing external dependencies. I have made a
list of libraries and which binaries pulls them in [1] and from that
systemd-pull
systemd-journal-gatewayd
systemd-journal-remote
systemd-journal-upload
systemd-firstboot
systemd-networkd
So I suggest following scheme
systemd
systemd-libs
systemd-devel
systemd-journal-remote (so gatewayd,remote,upload)

Makes sense.

Post by Lukáš Nykrýn
systemd-networkd (maybe also with resolved?)

I'd probably leave this in the main RPM, after all it doesn't take
possession of any interfaces by default, but makes sure
libsystemd-network returns useful stuff.

But if you do split it out, I'd call the package without the "d" suffix.

What's the effective size of this RPM on x86-64?

Post by Lukáš Nykrýn
systemd-machine (machined,nspawn,importd)

I'd call this "systemd-nspawn.rpm", really... The name of the daemon
is irrelevant.

Post by Lukáš Nykrýn
systemd-firstboot (firstboot,sysusers?,factory stuff?)

I'd really not bother with this stuff. This should be in the base, and
it is tiny. Plese leave this in the main package.

sysusers is definitely something we should make a Fedora default, that
is used distro wide, as it makes user registration portable, and is
also what Atomic wants.

THe factory code is in systemctl and PID 1, it makes no sense to split
out, there's nothing really you could split out.

Post by Lukáš Nykrýn
systemd-hwdb

This makes sense to split out.

From my perspective the only things that make sense to split out is
nspawn/machinectl/machined/importd/pull into "systemd-nspawn.rpm", the
hwdb into "systemd-hwdb.rpm" and "systemd-journal-remote.rpm" for the
journal remoting things.

Lennart

[Lukáš Nykrýn]

Post by Lennart Poettering

Post by Lukáš Nykrýn
systemd-networkd (maybe also with resolved?)

I'd probably leave this in the main RPM, after all it doesn't take
possession of any interfaces by default, but makes sure
libsystemd-network returns useful stuff.
But if you do split it out, I'd call the package without the "d" suffix.
What's the effective size of this RPM on x86-64?

421K systemd-networkd-219-19.el7.x86_64.rpm
394K systemd-resolved-219-19.el7.x86_64.rpm

Post by Lennart Poettering

Post by Lukáš Nykrýn
systemd-machine (machined,nspawn,importd)

I'd call this "systemd-nspawn.rpm", really... The name of the daemon
is irrelevant.

Post by Lukáš Nykrýn
systemd-firstboot (firstboot,sysusers?,factory stuff?)

I'd really not bother with this stuff. This should be in the base, and
it is tiny. Plese leave this in the main package.

The only reason was that it pulls in libcrypt.

Lukas

[Lukáš Nykrýn]

Post by Lukáš Nykrýn

Post by Lennart Poettering

Post by Lukáš Nykrýn
systemd-firstboot (firstboot,sysusers?,factory stuff?)

I'd really not bother with this stuff. This should be in the
base,
and
it is tiny. Plese leave this in the main package.

The only reason was that it pulls in libcrypt.

libcrypt is provided by glibc, which is always installed, so
splitting
this out does not lead to any savings.
Zbyszek

Yep I was probably wrong here. My reasoning was, that we have some
database of used crypto functionality in packages in rhel and we don't
use firstboot there, so for me it would be nice to drop this
dependency. But this is my specific usecase and we should push our
firstboot in upstream more.

Lukas

[Lennart Poettering]

Post by Lukáš Nykrýn

Post by Lennart Poettering

Post by Lukáš Nykrýn
systemd-machine (machined,nspawn,importd)

I'd call this "systemd-nspawn.rpm", really... The name of the daemon
is irrelevant.

Post by Lukáš Nykrýn
systemd-firstboot (firstboot,sysusers?,factory stuff?)

I'd really not bother with this stuff. This should be in the base, and
it is tiny. Plese leave this in the main package.

The only reason was that it pulls in libcrypt.

Hmm?

$ rpm -qf /usr/lib64/libcrypt.so.1
glibc-2.22-3.fc23.x86_64

I am sorry, but I doubt we'll ever be able to get rid of the glibc
dependency of systemd. ;-)

Lennart

[Martin Pitt]

Hello all,

in case it's useful, this is how we split them in Debian.

However, is this even a topic for upstream, apart from giving
recommendations? I. e. do you actually consider putting this kind of
split into the upstream build system à la "make install-<component>"?

Post by Lukáš Nykrýn
Personally I don't think it makes sense to split the package to get a
smaller core package. Most of our binaries are just few KBs.

They are actually fairly big, 100 kB to 1.5 MB for systemd itself. I
think the main reason for that is that they all statically link
libsystemd instead of dynamically linking to libsystemd.so.0. Is there
a particular reason for that?

Post by Lukáš Nykrýn
Other aspect would be minimizing external dependencies.

That is one important reason why we split them in Debian; e. g. the
machined/nspawn/importd group pulls in a number of rather heavy
dependencies. udev (including hwdb) is something which you don't need
in containers, so we split that out too.

Another reason is to make it easy to enable/disable a particular
feature (e. g. libnss-myhostname).

And then of course the infamous "need to run with sysvinit/upstart",
which other distros don't need to be concerned about :-)

Post by Lukáš Nykrýn
So I suggest following scheme
systemd

check

Post by Lukáš Nykrýn
systemd-libs
systemd-devel

They are called a bit differently for distro policy, upgrade safety,
consistency, and multi-arch support reasons; we need separate binary
packages for every lib*.so. But in spirit, "check".

Post by Lukáš Nykrýn
systemd-journal-remote (so gatewayd,remote,upload)

Check, we have exactly this package name.

Post by Lukáš Nykrýn
systemd-networkd (maybe also with resolved?)

We currently keep that in the "systemd" package as splitting it out
now is a bit of an upgrade pain, but we discussed doing this.

Post by Lukáš Nykrýn
systemd-machine (machined,nspawn,importd)

We call that package "systemd-container", but it has exactly those, so
"check".

Post by Lukáš Nykrýn
systemd-firstboot (firstboot,sysusers?,factory stuff?)

We don't have a separate package for that.

Post by Lukáš Nykrýn
systemd-hwdb

We split out the entire udev, including hwdb. This nicely reduces the
footprint in containers and also allows us to use udev with
sysvinit/upstart.

Martin

[Lennart Poettering]

Post by Martin Pitt

Post by Lukáš Nykrýn
systemd-firstboot (firstboot,sysusers?,factory stuff?)

I wonder if this is worth the trouble. The binaries are currently
fairly big, but do they bring in any external dependencies?
Maybe we should instead link them dynamically to libsystemd.so?
This would save some space...

This won't work. We generally try to avoid static variables, but we
have some, for example in log.c to store the log level and
target. Now, you could compile log.c into both libsystemd.so and the
main programs (which is basically what we do), and then make the main
program link against libsystemd.so (which we do not do). In such a
case you'd have two copies of log.c (and thus the static vars) in the
resulting process, and if we set the log level of one, this won't
affect the log level of the other. This is clearly problematic. The
other option of course is to declare all internal APIs exported .so
symbols, but that would mean to commit to a stable API for them (which
is completely out of the question), or to bump the soname on each
release (which is not an option either).

We hence link the internal code into all binaries, and rely on the
linker and compiler to strip as much of that as possible again.

Post by Martin Pitt
We don't have a separate package for that.

Post by Lukáš Nykrýn
systemd-hwdb

We split out the entire udev, including hwdb. This nicely reduces the
footprint in containers and also allows us to use udev with
sysvinit/upstart.

Yeah, that makes a big impact. The only thing that is not clear is
whether systemd-udevd should be part of this package (a), or part of
the main package (b). You do (a), but (b) may make sense to run udevd
without the hardware database. I don't think this is useful except in
very rare circumstances. Someone brought up the case of an embedded
device with a custom db... I don't think this is very convincing,
because in such case you wouldn't ship the text hwdb at all, just
/etc/udev/hwdb.bin.

THere's no point in shipping the non-binary version of the hwdb. The
hwdb isn't a cache, it's a compiled version of the hwdb, and you don't
the sources around for this.

Lennart

[Karel Zak]

The other option of course is to declare all internal APIs exported
.so symbols, but that would mean to commit to a stable API for them
(which is completely out of the question), or to bump the soname on
each release (which is not an option either).

You don't have to change soname, but all you need it use symbols
versioning with package (or build) specific version for private-API.
This method uses libvirt.so where is large number of private but
exported symbols.

https://github.com/libvirt/libvirt/blob/master/src/Makefile.am#L2031

so something like:

...
LIBSYSTEMD_227 {
global:
sd_bus_default_flush_close;
sd_bus_path_decode_many;
sd_bus_path_encode_many;
sd_listen_fds_with_names;
} LIBSYSTEMD_226;


LIBSYSTEMD_PRIVATE_$(VERSION) {
global:
funcA;
funcB;
};

where $(VERSION) is always different, then you can be sure that people
won't be able to link against the symbols and mix libsystemd with
systemd binaries from different versions.

Karel

[Martin Pitt]

Won't you need it for udevadm hwdb --update, after you add a new
hwdb.d/ file? Or can we now have multiple compiled dbs, one shipped by
the package and one built dynamically by hwdb --update?

Well, if you do add those locally. But that's not a typical usecase really.

I meant that the udev package isn't the only thing shipping hdwb.d/
files -- at least libmtp, libgphoto, and media-player-info ship some
as well, and cause the hdwb to be rebuilt on package
installation/removal.

Martin

[Simon McVittie]

Post by Lennart Poettering
The
other option of course is to declare all internal APIs exported .so
symbols, but that would mean to commit to a stable API for them (which
is completely out of the question), or to bump the soname on each
release (which is not an option either).

Have you considered doing something similar to what recent dbus versions
do for libdbus?

* symbols starting dbus_ (except for a couple of historical accidents
that start with dbus_internal_do_not_use_) are explicitly stable ABI

* symbols starting _dbus_ are explicitly not stable ABI, and are only
used (outside libdbus) by dbus-daemon and other utilities in the dbus
source package

* symbols starting dbus_ are versioned (GNU symbol-versioning)
as LIBDBUS_1_3 (as long as the shared library is libdbus-1.so.3)

* symbols starting _dbus_ are versioned as LIBDBUS_PRIVATE_1.10.2
which deliberately changes with each new version

This only applies to our equivalent of systemd's src/basic (the parts
that are needed by both the libdbus public API and the utilities). Our
equivalent of src/shared still ends up in a convenience library that is
statically linked into each utility that needs it.

It's a little easier for systemd to rely on this than it is for dbus to
do the same, because dbus is portable (to Windows, non-GNU Linux, and
non-Linux Unix like *BSD and Solaris), whereas systemd only targets
GNU/Linux and can assume that GNU symbol versioning is present in libc.

S

[Zbigniew Jędrzejewski-Szmek]

Post by Martin Pitt
Hello all,
in case it's useful, this is how we split them in Debian.
However, is this even a topic for upstream, apart from giving
recommendations? I. e. do you actually consider putting this kind of
split into the upstream build system à la "make install-<component>"?

Post by Lukáš Nykrýn
Personally I don't think it makes sense to split the package to get a
smaller core package. Most of our binaries are just few KBs.

They are actually fairly big, 100 kB to 1.5 MB for systemd itself. I
think the main reason for that is that they all statically link
libsystemd instead of dynamically linking to libsystemd.so.0. Is there
a particular reason for that?

Post by Lukáš Nykrýn
Other aspect would be minimizing external dependencies.

That is one important reason why we split them in Debian; e. g. the
machined/nspawn/importd group pulls in a number of rather heavy
dependencies. udev (including hwdb) is something which you don't need
in containers, so we split that out too.
Another reason is to make it easy to enable/disable a particular
feature (e. g. libnss-myhostname).
And then of course the infamous "need to run with sysvinit/upstart",
which other distros don't need to be concerned about :-)

Post by Lukáš Nykrýn
So I suggest following scheme
systemd

check

Post by Lukáš Nykrýn
systemd-libs
systemd-devel

They are called a bit differently for distro policy, upgrade safety,
consistency, and multi-arch support reasons; we need separate binary
packages for every lib*.so. But in spirit, "check".

Post by Lukáš Nykrýn
systemd-journal-remote (so gatewayd,remote,upload)

Check, we have exactly this package name.

Post by Lukáš Nykrýn
systemd-networkd (maybe also with resolved?)

We currently keep that in the "systemd" package as splitting it out
now is a bit of an upgrade pain, but we discussed doing this.

Post by Lukáš Nykrýn
systemd-machine (machined,nspawn,importd)

We call that package "systemd-container", but it has exactly those, so
"check".

Post by Lukáš Nykrýn
systemd-firstboot (firstboot,sysusers?,factory stuff?)

We don't have a separate package for that.

Post by Lukáš Nykrýn
systemd-hwdb

We split out the entire udev, including hwdb. This nicely reduces the
footprint in containers and also allows us to use udev with
sysvinit/upstart.

Yeah, after removing a bunch of stuff, hwdb stuff becomes even more
pronounced.

I prepared a package for rawhide with [1,2] the following subpackages:
systemd-journal-remote (remote, upload, gatewayd)
systemd-container (nspawn, machinectl, machined, importd, pull, var-lib-machines.mount)
systemd-udev (udevd, udevadm, udev rules, hwdb).

The first two are uncontroversial (systemd-journal-remote already existed
as systemd-journal-gateway for a long time).
The last is somewhat controversial: while people seem to agree about the split,
it's not necessary clear whether udevd should be in the subpackage or not.
I went with "yes", to see how that works out. I think this makes more
sense this way, but maybe not.

[1] http://in.waw.pl/git/fedora-systemd/
[2] https://copr.fedoraproject.org/coprs/zbyszek/systemd/build/138959/

[Zbigniew Jędrzejewski-Szmek]

Post by Zbigniew Jędrzejewski-Szmek

Post by Martin Pitt
Hello all,
in case it's useful, this is how we split them in Debian.
However, is this even a topic for upstream, apart from giving
recommendations? I. e. do you actually consider putting this kind of
split into the upstream build system à la "make install-<component>"?

Post by Lukáš Nykrýn
Personally I don't think it makes sense to split the package to get a
smaller core package. Most of our binaries are just few KBs.

They are actually fairly big, 100 kB to 1.5 MB for systemd itself. I
think the main reason for that is that they all statically link
libsystemd instead of dynamically linking to libsystemd.so.0. Is there
a particular reason for that?

Post by Lukáš Nykrýn
Other aspect would be minimizing external dependencies.

That is one important reason why we split them in Debian; e. g. the
machined/nspawn/importd group pulls in a number of rather heavy
dependencies. udev (including hwdb) is something which you don't need
in containers, so we split that out too.
Another reason is to make it easy to enable/disable a particular
feature (e. g. libnss-myhostname).
And then of course the infamous "need to run with sysvinit/upstart",
which other distros don't need to be concerned about :-)

Post by Lukáš Nykrýn
So I suggest following scheme
systemd

check

Post by Lukáš Nykrýn
systemd-libs
systemd-devel

They are called a bit differently for distro policy, upgrade safety,
consistency, and multi-arch support reasons; we need separate binary
packages for every lib*.so. But in spirit, "check".

Post by Lukáš Nykrýn
systemd-journal-remote (so gatewayd,remote,upload)

Check, we have exactly this package name.

Post by Lukáš Nykrýn
systemd-networkd (maybe also with resolved?)

We currently keep that in the "systemd" package as splitting it out
now is a bit of an upgrade pain, but we discussed doing this.

Post by Lukáš Nykrýn
systemd-machine (machined,nspawn,importd)

We call that package "systemd-container", but it has exactly those, so
"check".

Post by Lukáš Nykrýn
systemd-firstboot (firstboot,sysusers?,factory stuff?)

We don't have a separate package for that.

Post by Lukáš Nykrýn
systemd-hwdb

We split out the entire udev, including hwdb. This nicely reduces the
footprint in containers and also allows us to use udev with
sysvinit/upstart.

Yeah, after removing a bunch of stuff, hwdb stuff becomes even more
pronounced.
systemd-journal-remote (remote, upload, gatewayd)
systemd-container (nspawn, machinectl, machined, importd, pull, var-lib-machines.mount)
systemd-udev (udevd, udevadm, udev rules, hwdb).
The first two are uncontroversial (systemd-journal-remote already existed
as systemd-journal-gateway for a long time).
The last is somewhat controversial: while people seem to agree about the split,
it's not necessary clear whether udevd should be in the subpackage or not.
I went with "yes", to see how that works out. I think this makes more
sense this way, but maybe not.
[1] http://in.waw.pl/git/fedora-systemd/
[2] https://copr.fedoraproject.org/coprs/zbyszek/systemd/build/138959/

Installed size of systemd-udev is 6.5MB, systemd-container is 3.5MB,
systemd is 19MB, so the gain is modest. We also lose some dependencies.

Zbyszek

[Martin Pitt]

However, the gain through the extra dependencies is nontrivial: On a
minimal system, installing systemd-container pulls in some 20 extra
packages (libldap, sasl2-modules, libkrb5, libssh, ca-certificates
etc.)

ldap, sasl, kerberos, libssh and the certificates? That sounds very wrong.

systemd-container depends on libcurl3-gnutls, which is the thing we
desperately want to keep out of a minimal install as it has this huge
list of dependencies.

Martin

[Lukáš Nykrýn]

Post by Zbigniew Jędrzejewski-Szmek
I prepared a package for rawhide with [1,2] the following
systemd-journal-remote (remote, upload, gatewayd)
systemd-container (nspawn, machinectl, machined, importd, pull, var
-lib-machines.mount)
systemd-udev (udevd, udevadm, udev rules, hwdb).
The first two are uncontroversial (systemd-journal-remote already existed
as systemd-journal-gateway for a long time).
The last is somewhat controversial: while people seem to agree about the split,
it's not necessary clear whether udevd should be in the subpackage or not.
I went with "yes", to see how that works out. I think this makes more
sense this way, but maybe not.

I would like to see the hwdb in its own package. I can imagine a use
-case where user wants to use udev, but wants to provide its own
smaller hwdb (or hwdb.bin).


Lukas

[Lennart Poettering]

This post might be inappropriate. Click to display it.

[Reindl Harald]

Why not systemd-devel?

Because these aren't development related discussion

this list was multiple times statet also as users-list by Lennart
himself, just use Google to find that statement in the archive

I dont see to what relevance this being user list or development list is
here

interesting - so what did you try to tell the world with "because these
aren't development related discussion"?

you are just pissed off because you where not present at the conference
and nobody asked you before talk aboput something - that's it - period

[Jóhann B. Guðmundsson]

Post by Reindl Harald

Why not systemd-devel?

Because these aren't development related discussion

this list was multiple times statet also as users-list by Lennart
himself, just use Google to find that statement in the archive

I dont see to what relevance this being user list or development list is
here

interesting - so what did you try to tell the world with "because
these aren't development related discussion"?

Last time I checked downstream distributions packaging problems is not
upstream development discussions.

Post by Reindl Harald
you are just pissed off because you where not present at the
conference and nobody asked you before talk aboput something - that's
it - period

I have you know I chose rather to pay for a ticket to be shared with
individuals that could not afford it rather then participate in a
conference where the community was being charge for reflecting on itself.

It's an ethical thing feel free to contact Nils if you need confirmation
of that.

JBG

[Reindl Harald]

Post by Jóhann B. Guðmundsson

Post by Reindl Harald

Because these aren't development related discussion

this list was multiple times statet also as users-list by Lennart
himself, just use Google to find that statement in the archive

I dont see to what relevance this being user list or development list is
here

interesting - so what did you try to tell the world with "because
these aren't development related discussion"?

Last time I checked downstream distributions packaging problems is not
upstream development discussions.

what did you not understnd in that tis list is *not only for usptream
development*?

and frankly if that "go away, you are downstream" attitude would get
down a little bit *anybody* would benefit

Post by Jóhann B. Guðmundsson

Post by Reindl Harald
you are just pissed off because you where not present at the
conference and nobody asked you before talk aboput something - that's
it - period

I have you know I chose rather to pay for a ticket to be shared with
individuals that could not afford it rather then participate in a
conference where the community was being charge for reflecting on itself.

oh my god....

[Richard Maw]

Post by Martin Pitt

Post by Lukáš Nykrýn
Personally I don't think it makes sense to split the package to get a
smaller core package. Most of our binaries are just few KBs.

They are actually fairly big, 100 kB to 1.5 MB for systemd itself. I
think the main reason for that is that they all statically link
libsystemd instead of dynamically linking to libsystemd.so.0. Is there
a particular reason for that?

This came up at the conference, I left with the impression that it was because
there were global variables determining the log levels, and that sharing that
would cause problems.

I think it came up in the questions section of Lennart's initial talk.

[Lennart Poettering]

Why not systemd-devel?

Because these aren't development related discussion and there is a need for
separated collaborated git repository to prevent duplication of downstream
work etc.

We only have one ML, and I'd rather have more inter-distro threads on
this ML than more threads on whether inter-distro threads belong
here. Thanks ;-)

Lennart

[Michael Biebl]

2015-11-11 21:21 GMT+01:00 Jóhann B. Guðmundsson <***@gmail.com>:
[snip]

To coordinate and oversee and collectively share work done between
distribution integrating the relevant components in their distribution.

And now you started an unrelated meta-discussion. Please do that in a
separate thread and don't hijack this one.

Thank you.

Michael

[Jóhann B. Guðmundsson]

Post by Michael Biebl
[snip]

To coordinate and oversee and collectively share work done between
distribution integrating the relevant components in their distribution.

And now you started an unrelated meta-discussion. Please do that in a
separate thread and don't hijack this one.

Please discuss this downstream since downstream packaging is an
downstream matter.

JBG

[Reindl Harald]

Post by Jóhann B. Guðmundsson

Post by Michael Biebl
[snip]

To coordinate and oversee and collectively share work done between
distribution integrating the relevant components in their distribution.

And now you started an unrelated meta-discussion. Please do that in a
separate thread and don't hijack this one.

Please discuss this downstream since downstream packaging is an
downstream matter

and *you* decide that?

get the facts - without any downstream upstream would be meaningless and
that's a matter of respect not only facts

[Colin Guthrie]

This post might be inappropriate. Click to display it.

[poma]

Post by Colin Guthrie

Post by Martin Pitt

Post by Lukáš Nykrýn
systemd-machine (machined,nspawn,importd)

We call that package "systemd-container", but it has exactly those, so
"check".

I think we (Fedora) should follow this, for inter-distro consistency.

I prefer that name to systemd-nspawn. As Lennart's original comment on
the systemd-machine package name suggestion was "the name of the daemon
doesn't matter", I'd argue that the name of the binary also doesn't
matter too much! After all, the "nspawn" itself doesn't mean anything
unless you know what nspawn is, and if you know what it is, then you
know what a container is, so the name systemd-container makes sense there.
So +1 from me for that name as a general recommendation.
Col

man 1 systemd-nspawn
"... In many ways it is similar to chroot(1)..."

Everyone knows what 'chroot' is, so "systemd-chroot" makes sense there, also.
+1

[Lennart Poettering]

Post by Martin Pitt
Hello all,
in case it's useful, this is how we split them in Debian.
However, is this even a topic for upstream, apart from giving
recommendations? I. e. do you actually consider putting this kind of
split into the upstream build system à la "make
install-<component>"?

Nope, we won't do that.

Post by Martin Pitt

Post by Lukáš Nykrýn
Personally I don't think it makes sense to split the package to get a
smaller core package. Most of our binaries are just few KBs.

They are actually fairly big, 100 kB to 1.5 MB for systemd itself. I
think the main reason for that is that they all statically link
libsystemd instead of dynamically linking to libsystemd.so.0. Is there
a particular reason for that?

Yes, there is. If we'd share the common code via a proper .so then
we'd have to commit to a stable API for that, or bump the soname on
every single release. We really don't want to do either. The internal
stuff is stupposed to be internal, and not an API.

Post by Martin Pitt
Another reason is to make it easy to enable/disable a particular
feature (e. g. libnss-myhostname).

I don't see why one would ever disable this feature... I doubt this
makes senseto split out really.

Post by Martin Pitt

Post by Lukáš Nykrýn
systemd-networkd (maybe also with resolved?)

We currently keep that in the "systemd" package as splitting it out
now is a bit of an upgrade pain, but we discussed doing this.

As mentioned I wouldn't split this out either.

Lennart

[Martin Pitt]

Post by Lennart Poettering

Post by Martin Pitt
Another reason is to make it easy to enable/disable a particular
feature (e. g. libnss-myhostname).

I don't see why one would ever disable this feature... I doubt this
makes senseto split out really.

We have to do that because it's a shared library and needs to be
multi-arch compatible (i. e. you might have more than one architecture
installed at the same time); so this was a bad example wrt. feature
enablement indeed. libnss-mymachines and libnss-resolved might be
better ones (but we don't have a choice to not split them out anyway).

Post by Lennart Poettering

Post by Martin Pitt

Post by Lukáš Nykrýn
systemd-networkd (maybe also with resolved?)

We currently keep that in the "systemd" package as splitting it out
now is a bit of an upgrade pain, but we discussed doing this.

As mentioned I wouldn't split this out either.

I'm reluctant to do it too. At the moment there's little reason
because we still have the iptables support disabled. Both because the
iptables vs. nftables question isn't decided yet, and also because it
would mean another set of heavy dependencies in the default install
for a relatively little used feature. libnftnl is much smaller, so
if/once we switch to that, this is much less of a concern.

Martin

[Reindl Harald]

Post by Lennart Poettering

Post by Martin Pitt
Hello all,
in case it's useful, this is how we split them in Debian.
However, is this even a topic for upstream, apart from giving
recommendations? I. e. do you actually consider putting this kind of
split into the upstream build system à la "make
install-<component>"?

Nope, we won't do that.

Post by Martin Pitt

Post by Lukáš Nykrýn
Personally I don't think it makes sense to split the package to get a
smaller core package. Most of our binaries are just few KBs.

They are actually fairly big, 100 kB to 1.5 MB for systemd itself. I
think the main reason for that is that they all statically link
libsystemd instead of dynamically linking to libsystemd.so.0. Is there
a particular reason for that?

Yes, there is. If we'd share the common code via a proper .so then
we'd have to commit to a stable API for that, or bump the soname on
every single release. We really don't want to do either. The internal
stuff is stupposed to be internal, and not an API

that is no valid reasoning, the following postfix libs are *not* a
stable API and *not* supported to be used from 3rd party code, hence
they are not directly in /usr/lib64/

just because a internal .so exists don't make it to a API

postfix /usr/lib64/postfix/libpostfix-dns.so
postfix /usr/lib64/postfix/libpostfix-global.so
postfix /usr/lib64/postfix/libpostfix-master.so
postfix /usr/lib64/postfix/libpostfix-tls.so
postfix /usr/lib64/postfix/libpostfix-util.so

[poma]

Post by Martin Pitt
Hello all,
in case it's useful, this is how we split them in Debian.
However, is this even a topic for upstream, apart from giving
recommendations? I. e. do you actually consider putting this kind of
split into the upstream build system à la "make install-<component>"?

Post by Lukáš Nykrýn
Personally I don't think it makes sense to split the package to get a
smaller core package. Most of our binaries are just few KBs.

They are actually fairly big, 100 kB to 1.5 MB for systemd itself. I
think the main reason for that is that they all statically link
libsystemd instead of dynamically linking to libsystemd.so.0. Is there
a particular reason for that?

Post by Lukáš Nykrýn
Other aspect would be minimizing external dependencies.

That is one important reason why we split them in Debian; e. g. the
machined/nspawn/importd group pulls in a number of rather heavy
dependencies. udev (including hwdb) is something which you don't need
in containers, so we split that out too.
Another reason is to make it easy to enable/disable a particular
feature (e. g. libnss-myhostname).
And then of course the infamous "need to run with sysvinit/upstart",
which other distros don't need to be concerned about :-)

Post by Lukáš Nykrýn
So I suggest following scheme
systemd

check

Post by Lukáš Nykrýn
systemd-libs
systemd-devel

They are called a bit differently for distro policy, upgrade safety,
consistency, and multi-arch support reasons; we need separate binary
packages for every lib*.so. But in spirit, "check".

Post by Lukáš Nykrýn
systemd-journal-remote (so gatewayd,remote,upload)

Check, we have exactly this package name.

"systemd-check"
+1

Post by Martin Pitt

Post by Lukáš Nykrýn
systemd-networkd (maybe also with resolved?)

We currently keep that in the "systemd" package as splitting it out
now is a bit of an upgrade pain, but we discussed doing this.

Post by Lukáš Nykrýn
systemd-machine (machined,nspawn,importd)

We call that package "systemd-container", but it has exactly those, so
"check".

Post by Lukáš Nykrýn
systemd-firstboot (firstboot,sysusers?,factory stuff?)

We don't have a separate package for that.

Post by Lukáš Nykrýn
systemd-hwdb

We split out the entire udev, including hwdb. This nicely reduces the
footprint in containers and also allows us to use udev with
sysvinit/upstart.
Martin

[Jóhann B. Guðmundsson]

Post by Martin Pitt
However, is this even a topic for upstream,

I would argue not.

I would argue that this is a downstream collaboration matter in which a)
the split should be the same regardless of distribution and the sub
components should be split in same manner across all distribution so it
does not matter if you are running Arch/Debian/Ubuntu/Fedora etc. the
documentation and required actions are the same for the consumer.

JBG

[Michael Chapman]

Post by Lukáš Nykrýn
Hi,
During systemd.conf we have discussed some recommendation for
downstreams, how they could split systemd to subpackages, so lets
continue that discussion here.
Personally I don't think it makes sense to split the package to get a
smaller core package. Most of our binaries are just few KBs. Only
exception here is /usr/lib/udev/hwdb.d which, on fedora rawhide, has
about 5,2 MB (15% of the whole package).
Other aspect would be minimizing external dependencies. I have made a
list of libraries and which binaries pulls them in [1] and from that
systemd-pull
systemd-journal-gatewayd
systemd-journal-remote
systemd-journal-upload
systemd-firstboot
systemd-networkd

Hi Lukáš,

It seems like you're just looking at binaries at the moment, but I think
some care needs to be taken with config files too.

One gotcha I discovered in having networkd split out, and specifically in
having 99-default.link in a subpackage, is that it can change the way
predictable interface naming works, whether or not the networkd daemon is
managing network interfaces. Udev's net_setup_link builtin consults the
*.link files directly to determine the interface naming policy.

We have to make sure the mere presence or absence of an otherwise-unused
subpackage on a system doesn't change the system's behaviour too
dramatically.

- Michael

[Lennart Poettering]

Post by Lukáš Nykrýn
Hi,
During systemd.conf we have discussed some recommendation for
downstreams, how they could split systemd to subpackages, so lets
continue that discussion here.
Personally I don't think it makes sense to split the package to get a
smaller core package. Most of our binaries are just few KBs. Only
exception here is /usr/lib/udev/hwdb.d which, on fedora rawhide, has
about 5,2 MB (15% of the whole package).
Other aspect would be minimizing external dependencies. I have made a
list of libraries and which binaries pulls them in [1] and from that
systemd-pull
systemd-journal-gatewayd
systemd-journal-remote
systemd-journal-upload
systemd-firstboot
systemd-networkd

Hi Lukáš,
It seems like you're just looking at binaries at the moment, but I think
some care needs to be taken with config files too.
One gotcha I discovered in having networkd split out, and specifically in
having 99-default.link in a subpackage, is that it can change the way
predictable interface naming works, whether or not the networkd daemon is
managing network interfaces. Udev's net_setup_link builtin consults the
*.link files directly to determine the interface naming policy.
We have to make sure the mere presence or absence of an otherwise-unused
subpackage on a system doesn't change the system's behaviour too
dramatically.

The .link files belong to udev, not networkd. It's that simple.

Lennart

[Lukáš Nykrýn]

Hi,
implementing the split in Fedora deserves a Changes page,
because of the need to coordinate with other components of the
https://fedoraproject.org/wiki/Changes/systemd_package_split
All the details are described on the Change page. If anything
is missing/unclear, please let me know.

Hi,
I'm not currently near my laptop so I can't test it, but how are the
virtualization tools working without machined installed?

Lukas

[Zbigniew Jędrzejewski-Szmek]

Post by Lukáš Nykrýn

Hi,
implementing the split in Fedora deserves a Changes page,
because of the need to coordinate with other components of the
https://fedoraproject.org/wiki/Changes/systemd_package_split
All the details are described on the Change page. If anything
is missing/unclear, please let me know.

Hi,
I'm not currently near my laptop so I can't test it, but how are the
virtualization tools working without machined installed?

Do you mean virsh and similar? virsh should function, but some
functionality would not be available [1]. Not sure about
golang-github-coreos-go-systemd or lxc. I think that those packages
should get an explicit dependency on -container. I'm not aware of
other users.

Zbyszek

[1] http://sources.debian.net/src/libvirt/1.2.21-1/src/util/virsystemd.c/#L248