[systemd-devel] 220 tarball erroneously ships keyboard-keys-from-name.gperf

[Martin Pitt]

Hello,

while packaging 220, I got the audit related build failure that was
already reported [1], but also another one:

| In file included from ../src/udev/udev-builtin-keyboard.c:29:0:
| ./src/udev/keyboard-keys-from-name.h: In function 'keyboard_lookup_key':
| ./src/udev/keyboard-keys-from-name.h:253:21: error: 'KEY_NUMERIC_A' undeclared (first use in this function)
| {"numeric_a", KEY_NUMERIC_A},
| ^
| ./src/udev/keyboard-keys-from-name.h:253:21: note: each undeclared identifier is reported only once for each function it appears in
| ./src/udev/keyboard-keys-from-name.h:275:21: error: 'KEY_NUMERIC_C' undeclared (first use in this function)
| {"numeric_c", KEY_NUMERIC_C},
| ^
| ./src/udev/keyboard-keys-from-name.h:347:21: error: 'KEY_NUMERIC_D' undeclared (first use in this function)
| {"numeric_d", KEY_NUMERIC_D},
| ^
| ./src/udev/keyboard-keys-from-name.h:420:21: error: 'KEY_NUMERIC_B' undeclared (first use in this function)
| {"numeric_b", KEY_NUMERIC_B},
| ^
| ./src/udev/keyboard-keys-from-name.h:464:26: error: 'KEY_ROTATE_DISPLAY' undeclared (first use in this function)
| {"rotate_display", KEY_ROTATE_DISPLAY},

This is because unlike 219 and earlier, the 220 tarball ships a
pregenerated src/udev/keyboard-keys-from-name.gperf. In Debian sid,
the above constants are not defined (in the kernel headers, I
presume). I think src/udev/keyboard-keys-from-name.gperf is supposed
to be built during package build? I see that somewhere between 219 and
220 this happened in Makefile.am:

-CLEANFILES += \
- src/udev/keyboard-keys-from-name.gperf \
- src/udev/keyboard-keys.txt \
- src/udev/net/link-config-gperf.c

but apparently this wasn't put back someplace else?

Thanks,

Martin

[1] http://lists.freedesktop.org/archives/systemd-devel/2015-May/032148.html

[Zbigniew Jędrzejewski-Szmek]

Post by Martin Pitt
Hello,
while packaging 220, I got the audit related build failure that was
| ./src/udev/keyboard-keys-from-name.h:253:21: error: 'KEY_NUMERIC_A' undeclared (first use in this function)
| {"numeric_a", KEY_NUMERIC_A},
| ^
| ./src/udev/keyboard-keys-from-name.h:253:21: note: each undeclared identifier is reported only once for each function it appears in
| ./src/udev/keyboard-keys-from-name.h:275:21: error: 'KEY_NUMERIC_C' undeclared (first use in this function)
| {"numeric_c", KEY_NUMERIC_C},
| ^
| ./src/udev/keyboard-keys-from-name.h:347:21: error: 'KEY_NUMERIC_D' undeclared (first use in this function)
| {"numeric_d", KEY_NUMERIC_D},
| ^
| ./src/udev/keyboard-keys-from-name.h:420:21: error: 'KEY_NUMERIC_B' undeclared (first use in this function)
| {"numeric_b", KEY_NUMERIC_B},
| ^
| ./src/udev/keyboard-keys-from-name.h:464:26: error: 'KEY_ROTATE_DISPLAY' undeclared (first use in this function)
| {"rotate_display", KEY_ROTATE_DISPLAY},
This is because unlike 219 and earlier, the 220 tarball ships a
pregenerated src/udev/keyboard-keys-from-name.gperf. In Debian sid,
the above constants are not defined (in the kernel headers, I
presume). I think src/udev/keyboard-keys-from-name.gperf is supposed
to be built during package build? I see that somewhere between 219 and
-CLEANFILES += \
- src/udev/keyboard-keys-from-name.gperf \
- src/udev/keyboard-keys.txt \
- src/udev/net/link-config-gperf.c
but apparently this wasn't put back someplace else?

This was replaced by uniform rules for all gperf files.
Those generic rules probably need fixing.

Zbyszek

[Lennart Poettering]

Post by Martin Pitt
Hello,
while packaging 220, I got the audit related build failure that was

I think this has been fixed now with Tom's commit
ee3c31bf69746c5afc764c3d0337feec1bf25f0e contributed my Marc-Antoine.

Please verify.

Lennart

Post by Martin Pitt
| ./src/udev/keyboard-keys-from-name.h:253:21: error: 'KEY_NUMERIC_A' undeclared (first use in this function)
| {"numeric_a", KEY_NUMERIC_A},
| ^
| ./src/udev/keyboard-keys-from-name.h:253:21: note: each undeclared identifier is reported only once for each function it appears in
| ./src/udev/keyboard-keys-from-name.h:275:21: error: 'KEY_NUMERIC_C' undeclared (first use in this function)
| {"numeric_c", KEY_NUMERIC_C},
| ^
| ./src/udev/keyboard-keys-from-name.h:347:21: error: 'KEY_NUMERIC_D' undeclared (first use in this function)
| {"numeric_d", KEY_NUMERIC_D},
| ^
| ./src/udev/keyboard-keys-from-name.h:420:21: error: 'KEY_NUMERIC_B' undeclared (first use in this function)
| {"numeric_b", KEY_NUMERIC_B},
| ^
| ./src/udev/keyboard-keys-from-name.h:464:26: error: 'KEY_ROTATE_DISPLAY' undeclared (first use in this function)
| {"rotate_display", KEY_ROTATE_DISPLAY},
This is because unlike 219 and earlier, the 220 tarball ships a
pregenerated src/udev/keyboard-keys-from-name.gperf. In Debian sid,
the above constants are not defined (in the kernel headers, I
presume). I think src/udev/keyboard-keys-from-name.gperf is supposed
to be built during package build? I see that somewhere between 219 and
-CLEANFILES += \
- src/udev/keyboard-keys-from-name.gperf \
- src/udev/keyboard-keys.txt \
- src/udev/net/link-config-gperf.c
but apparently this wasn't put back someplace else?
Thanks,
Martin
[1] http://lists.freedesktop.org/archives/systemd-devel/2015-May/032148.html
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
_______________________________________________
systemd-devel mailing list
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Lennart

[Martin Pitt]

Post by Lennart Poettering
I think this has been fixed now with Tom's commit
ee3c31bf69746c5afc764c3d0337feec1bf25f0e contributed my Marc-Antoine.

Confirmed this morning. Sorry, I should have followed up to the ML
immediately. Thanks Tom!

Martin

[Dimitri John Ledkov]

Post by Martin Pitt

Post by Lennart Poettering
I think this has been fixed now with Tom's commit
ee3c31bf69746c5afc764c3d0337feec1bf25f0e contributed my Marc-Antoine.

Confirmed this morning. Sorry, I should have followed up to the ML
immediately. Thanks Tom!

Can stable-v220 branch be started with these please?

Or will there be a v220.1 release shortly with releasy fix-ups?

[Lennart Poettering]

Post by Dimitri John Ledkov
Or will there be a v220.1 release shortly with releasy fix-ups?

Well, we don't do point releases in systemd.

In systemd git we already have now the change that makes sd-bus and
sd-event public APIs, hence v221 is going to be more than just
bugfixes.

Also, next week I'll be travelling and I doubt I will be able to
finish a new release by then. Maybe in the middle of June we can get
out a new release.

Sorry,

Lennart

[Reindl Harald]

Post by Lennart Poettering

Post by Dimitri John Ledkov
Or will there be a v220.1 release shortly with releasy fix-ups?

Well, we don't do point releases in systemd.

and why?

just "because don't do" is not enough

Post by Lennart Poettering
In systemd git we already have now the change that makes sd-bus and
sd-event public APIs, hence v221 is going to be more than just
bugfixes.

that don't fix 220

Post by Lennart Poettering
Also, next week I'll be travelling and I doubt I will be able to
finish a new release by then. Maybe in the middle of June we can get
out a new release

which shows that a important project like systemd playing on the same
level as the linux kernel *just needs* point releases, there is a reason
that the kernel has point-releases and not only for the latest and
greatest version, systemd as well as the kernel are not a random webbrowser

[Lennart Poettering]

Post by Reindl Harald

Post by Lennart Poettering

Post by Dimitri John Ledkov
Or will there be a v220.1 release shortly with releasy fix-ups?

Well, we don't do point releases in systemd.

and why?
just "because don't do" is not enough

it's just a label after all and I like my bikesheds green without
points.

Post by Reindl Harald

Post by Lennart Poettering
In systemd git we already have now the change that makes sd-bus and
sd-event public APIs, hence v221 is going to be more than just
bugfixes.

that don't fix 220

Post by Lennart Poettering
Also, next week I'll be travelling and I doubt I will be able to
finish a new release by then. Maybe in the middle of June we can get
out a new release

which shows that a important project like systemd playing on the same level
as the linux kernel *just needs* point releases, there is a reason that the
kernel has point-releases and not only for the latest and greatest version,
systemd as well as the kernel are not a random webbrowser

There's a stable git tree maintained by Zbigniew, Lukas and friends,
that carries backports for the stable releases. Most vendors should
consider following that repo I figure.

I try to help the backporting business by tagging the commits I
personally consider backport-worthy with "git notes" and the
"Backport:" field.

Lennart

[Reindl Harald]

Post by Lennart Poettering

Post by Reindl Harald
which shows that a important project like systemd playing on the same level
as the linux kernel *just needs* point releases, there is a reason that the
kernel has point-releases and not only for the latest and greatest version,
systemd as well as the kernel are not a random webbrowser

There's a stable git tree maintained by Zbigniew, Lukas and friends,
that carries backports for the stable releases. Most vendors should
consider following that repo I figure.
I try to help the backporting business by tagging the commits I
personally consider backport-worthy with "git notes" and the
"Backport:" field

it just works poor

* https://bugzilla.redhat.com/show_bug.cgi?id=1185278
never got fixed in F20 but is fixed in Fedora 21
* https://bugzilla.redhat.com/show_bug.cgi?id=1185277 open for months

and in general a bunch of git commits don't replace a upstream changelog
of a minor release where as user you never have any idea what your
current installed package really is

with a 220.1 and a usptream download of 220.1 containing a changelog
things are entirely different (besides that the version numbers for a
project existing just a few years are strange to say it nicely)

[Michael Biebl]

Post by Reindl Harald

Post by Lennart Poettering
There's a stable git tree maintained by Zbigniew, Lukas and friends,
that carries backports for the stable releases. Most vendors should
consider following that repo I figure.
I try to help the backporting business by tagging the commits I
personally consider backport-worthy with "git notes" and the
"Backport:" field

it just works poor
* https://bugzilla.redhat.com/show_bug.cgi?id=1185278
never got fixed in F20 but is fixed in Fedora 21
* https://bugzilla.redhat.com/show_bug.cgi?id=1185277 open for months
and in general a bunch of git commits don't replace a upstream changelog of
a minor release where as user you never have any idea what your current
installed package really is
with a 220.1 and a usptream download of 220.1 containing a changelog things
are entirely different (besides that the version numbers for a project
existing just a few years are strange to say it nicely)

Maybe we(distro maintainers included) could work together with
Zbigniew and cut releases from the stable-branches?

[Lennart Poettering]

Post by Reindl Harald

Post by Lennart Poettering

Post by Reindl Harald
which shows that a important project like systemd playing on the same level
as the linux kernel *just needs* point releases, there is a reason that the
kernel has point-releases and not only for the latest and greatest version,
systemd as well as the kernel are not a random webbrowser

There's a stable git tree maintained by Zbigniew, Lukas and friends,
that carries backports for the stable releases. Most vendors should
consider following that repo I figure.
I try to help the backporting business by tagging the commits I
personally consider backport-worthy with "git notes" and the
"Backport:" field

it just works poor
* https://bugzilla.redhat.com/show_bug.cgi?id=1185278
never got fixed in F20 but is fixed in Fedora 21

Well, it's not really a bug. We require util-linux to be around. If
you make it unavailable then you get to to keep the pieces. It's that
simple.

Post by Reindl Harald
* https://bugzilla.redhat.com/show_bug.cgi?id=1185277 open for
months

I wasn't aware there was a reproducer for this issue now. Good to
know.

Post by Reindl Harald
and in general a bunch of git commits don't replace a upstream changelog of
a minor release where as user you never have any idea what your current
installed package really is.

Nah, this is not how this works. We simply do not provide turn-key
ready versions of systemd. systemd is not an app, it needs to be
integrated by the distributions, and they will have to adapt it to
their needs. Hence: if you want minor release information check the
RPM changelog, but don't expect that from us. We are not a product, we
are just a building block others may build a product from, and thus
need to polish it.

Post by Reindl Harald
with a 220.1 and a usptream download of 220.1 containing a changelog things
are entirely different (besides that the version numbers for a project
existing just a few years are strange to say it nicely)

Well, I certainly have enough things to do. We have trouble keeping up
with he amount of work that we already get, and I have no intention to
make things even more complex by maintaining multiple branches for
you, if the distros already do this anyway.

And again, there's already the stable tree maintained by Zbigniew and
friends. It comes with a change history "git log" and it's constantly
updated. Zbigniew doesn't tag releases in it, but if that's what you
need you can just tag it yourself and count each commit up that is
applied based on a stable branch...

Lennart

[Martin Pitt]

Post by Lennart Poettering
Well, I certainly have enough things to do. We have trouble keeping up
with he amount of work that we already get, and I have no intention to
make things even more complex by maintaining multiple branches for
you, if the distros already do this anyway.

Honestly, git format-patch (or even more convenient proper packaging
tools like Debian's git-buildpackages "pq") make it a trivial
operation to get one or several upstream fixes, it's not more work
(usually less) than packaging an entire new upstream release.

Bugs that affect the tarball are a tad harder, but adding

rm -f src/journal/audit_type-to-name.h src/udev/keyboard-keys-from-name.gperf

to the build script until we get 221 isn't rocket science either.

So I agree that maintaining more branches and upstream releases would
cause a lot of low-benefit work. I'd much rather get some distro-side
testing before we do an upstream release (see my other reply).

Martin

[Reindl Harald]

Post by Lennart Poettering

Post by Reindl Harald

Post by Lennart Poettering

Post by Reindl Harald
which shows that a important project like systemd playing on the same level
as the linux kernel *just needs* point releases, there is a reason that the
kernel has point-releases and not only for the latest and greatest version,
systemd as well as the kernel are not a random webbrowser

There's a stable git tree maintained by Zbigniew, Lukas and friends,
that carries backports for the stable releases. Most vendors should
consider following that repo I figure.
I try to help the backporting business by tagging the commits I
personally consider backport-worthy with "git notes" and the
"Backport:" field

it just works poor
* https://bugzilla.redhat.com/show_bug.cgi?id=1185278
never got fixed in F20 but is fixed in Fedora 21

Well, it's not really a bug. We require util-linux to be around. If
you make it unavailable then you get to to keep the pieces. It's that
simple

sftp-chroot worked fine until Fedora 20 and started to work again with
Fedora 21 so please don't pretend anything else than systemd is (or was)
responsible to fail proper shutdown and termination of the recently by
systemd invented user-session processes

[Lennart Poettering]

Post by Reindl Harald

Post by Lennart Poettering

Post by Reindl Harald

Post by Lennart Poettering

Post by Reindl Harald
which shows that a important project like systemd playing on the same level
as the linux kernel *just needs* point releases, there is a reason that the
kernel has point-releases and not only for the latest and greatest version,
systemd as well as the kernel are not a random webbrowser

There's a stable git tree maintained by Zbigniew, Lukas and friends,
that carries backports for the stable releases. Most vendors should
consider following that repo I figure.
I try to help the backporting business by tagging the commits I
personally consider backport-worthy with "git notes" and the
"Backport:" field

it just works poor
* https://bugzilla.redhat.com/show_bug.cgi?id=1185278
never got fixed in F20 but is fixed in Fedora 21

Well, it's not really a bug. We require util-linux to be around. If
you make it unavailable then you get to to keep the pieces. It's that
simple

sftp-chroot worked fine until Fedora 20 and started to work again with
Fedora 21 so please don't pretend anything else than systemd is (or was)
responsible to fail proper shutdown and termination of the recently by
systemd invented user-session processes

Well, we still require /usr/bin/kill from util-linux to be around. We
did not fix your "bug"... Maybe this doesn't trip up anymore, but we
didn't specifically "fix" anything...

Lennart

[Dave Reisner]

Post by Lennart Poettering

Post by Dimitri John Ledkov
Or will there be a v220.1 release shortly with releasy fix-ups?

Well, we don't do point releases in systemd.
In systemd git we already have now the change that makes sd-bus and
sd-event public APIs, hence v221 is going to be more than just
bugfixes.

Is git revert not an option? Does someone depend on this already, making
a rollback impossible (and why would you allow that)? You say that
versions are "just a label", but a project which adopts semver would be
able to create a branch in git and produce a dot release with backported
bugfixes. Surely you're familiar with this concept -- your colleague
Karel does this routinely with util-linux.

The fact of the matter is, the last 2 releases of systemd have been
brown bag releases. Neither 219 nor 220 are useable as the tarballs are
provided. v220 doesn't even build in a large number of configurations.
v218 wasn't quite as bad, but landed 18 patches in Arch's packaging
before v219 was released. For the average Arch package, this is unheard
of, let alone being a *necessity* in order to make software useable.

Please, let's figure out a way to lower the amount of work that gets
sharded out and duplicated by downstream packagers.

Post by Lennart Poettering
Also, next week I'll be travelling and I doubt I will be able to
finish a new release by then. Maybe in the middle of June we can get
out a new release.
Sorry,

I'm sorry to hear this too. Please find someone else who's paid to work
on systemd and teach them how to package releases. The current state of
affairs absolutely sucks for downstreams. Currently, packaging systemd
takes more of my time than all of my other packaging responsibilities
combined.

At LPC in New Orleans, you asked me how you could make systemd easier to
work with on the downstream side. I told you that I wanted more frequent
releases. You seemed amenable to this, but it really never happened.
I'm asking again: can we have more frequent releases? Can you take the
steps to actually make this happen and not gate releases on your own
personal availability?

d

[Michael Biebl]

This post might be inappropriate. Click to display it.

[Lennart Poettering]

Post by Michael Biebl

Post by Dave Reisner
The fact of the matter is, the last 2 releases of systemd have been
brown bag releases. Neither 219 nor 220 are useable as the tarballs are
provided. v220 doesn't even build in a large number of configurations.
v218 wasn't quite as bad, but landed 18 patches in Arch's packaging
before v219 was released. For the average Arch package, this is unheard
of, let alone being a *necessity* in order to make software useable.
Please, let's figure out a way to lower the amount of work that gets
sharded out and duplicated by downstream packagers.

Glad to know that I'm not the only one who feels this way. Let me
<mbiebl_> zbyszek: looks like we want a v221 release soon
<mbiebl_> fwiw, I think our QA for doing releases sucks
<zbyszek> mbiebl_: I'm working on the selinux bug currently...
<zbyszek> mbiebl_: I think the releases are possibly the point of
highest instability
<mbiebl_> a bit more official planning and giving people time to test
would be a great start
<zbyszek> I suggested doing a week of cooling off before a release
before, but somehow that didn't stick.
<mbiebl_> just telling people, when the release is planned, would help
<mbiebl_> right now, that feels really amateurish
<zbyszek> mbiebl_: yeah, that would help. But without a rule that only
bugfix and cosmetic or documentation changes go in, it's hard to test
stuff properly.
<mbiebl_> absolutely, just saying that atm we don't even have that
<mbiebl_> i.e. we know when a release is planned
<mbiebl_> that's imo the absolute minimum
<mbiebl_> the result is broken dist tarballs
<mbiebl_> and distros having to ship 50+ patches until it get's usable
<mbiebl_> that sucks bigs time

Well, but let's not forget that a major part of the issues popping up
actually were committed weeks ago. Things like the broken gperf
generated bits or the missing EFI dirs were in git since a loooong
time.

It would be great if downstream could help us with testing many of the
build combinations, at any time of the cycle, not just when it comes
to a release. Sure, I can notify a week before I intend to do a
release, but there's no reason to wait that long for testing! Upstream
git would ideally stay bootable and in a releasable state
all the time. And if it isn't, then this is something to fix
right-away, and not just when I sent out a notification..

We have a lot of build combinations, and I and the rest of the core
team only tend to test the upstream default config plus the Fedora
config. What broke are configs that are different than thes that could
have been caught weeks ago. We *rely* on downstreams to test these,
simply because the combinations explode...

Lennart

[Martin Pitt]

Post by Lennart Poettering
Well, but let's not forget that a major part of the issues popping up
actually were committed weeks ago.

Actually, no. As I said, on May 11 most everything was working just
fine, the udev regressions landed very late. The path_is_mount_point()
regression landed much earlier, but is much less visible (and now
there are tests cases with the patch I sent).

Post by Lennart Poettering
Things like the broken gperf generated bits or the missing EFI dirs
were in git since a loooong time.
It would be great if downstream could help us with testing many of the
build combinations, at any time of the cycle, not just when it comes
to a release.

Agreed. The "broken tarball" issues are not visible when building from
git (which is what I'm doing all the time). We didn't have that kind
of issues with 219 or 218 (or at least only negligible ones), but 220
taught us all that we need to test "make dist" builds more often.

So we have two indepenent things to fix:

* Regularly test "make dist", as nobody does that during regular
development.

Alternative: Stop shipping "make dist" tarballs altogether and just
tar the tagged git snapshot. Given the amount of patching that most
distros do, we pretty much all run autoreconf anyway (including
Fedora), so not having the pre-generated autoconfiscation and
pre-built manpages etc. in the tarball isn't actually that much of
a deal.

* Impose a release freeze period with announcing an impending
release, distros do a deep testing (this is a day's work for me, so
can't happen that often). During that time (which really shouldn't
be more than a few days) we really should avoid any commit which
isn't an important bug fix, especially not large refactorings or
new features.

Thanks,

Martin

[Dimitri John Ledkov]

Post by Martin Pitt

Post by Lennart Poettering
Well, but let's not forget that a major part of the issues popping up
actually were committed weeks ago.

Actually, no. As I said, on May 11 most everything was working just
fine, the udev regressions landed very late. The path_is_mount_point()
regression landed much earlier, but is much less visible (and now
there are tests cases with the patch I sent).

Post by Lennart Poettering
Things like the broken gperf generated bits or the missing EFI dirs
were in git since a loooong time.
It would be great if downstream could help us with testing many of the
build combinations, at any time of the cycle, not just when it comes
to a release.

Agreed. The "broken tarball" issues are not visible when building from
git (which is what I'm doing all the time). We didn't have that kind
of issues with 219 or 218 (or at least only negligible ones), but 220
taught us all that we need to test "make dist" builds more often.
* Regularly test "make dist", as nobody does that during regular
development.
Alternative: Stop shipping "make dist" tarballs altogether and just
tar the tagged git snapshot. Given the amount of patching that most
distros do, we pretty much all run autoreconf anyway (including
Fedora), so not having the pre-generated autoconfiscation and
pre-built manpages etc. in the tarball isn't actually that much of
a deal.

+1 for dropping make dist support... git archive is really ought to be
the release tarball.

Post by Martin Pitt
* Impose a release freeze period with announcing an impending
release, distros do a deep testing (this is a day's work for me, so
can't happen that often). During that time (which really shouldn't
be more than a few days) we really should avoid any commit which
isn't an important bug fix, especially not large refactorings or
new features.
Thanks,
Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
_______________________________________________
systemd-devel mailing list
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

[Lennart Poettering]

Post by Martin Pitt

Post by Lennart Poettering
Well, but let's not forget that a major part of the issues popping up
actually were committed weeks ago.

Actually, no. As I said, on May 11 most everything was working just
fine, the udev regressions landed very late. The path_is_mount_point()
regression landed much earlier, but is much less visible (and now
there are tests cases with the patch I sent).

Sure, some of the udev breakage is more recent. But a good chunk of
the other stuff (gperf, EFI, path_is_mount_point()) is much older, and
some of it is easily visible...

Post by Martin Pitt
Agreed. The "broken tarball" issues are not visible when building from
git (which is what I'm doing all the time). We didn't have that kind
of issues with 219 or 218 (or at least only negligible ones), but 220
taught us all that we need to test "make dist" builds more often.
* Regularly test "make dist", as nobody does that during regular
development.

Well, no.

I use "make distcheck" regularly during regular development, and
that's how I generate the final tarball.

However, "make distcheck" generates a tarball off the 220 git tree
just fine, if you had all the options enabled on a modern Fedora, like
I do.

Again, this is about testing options and combinations that are
not regularly covered by the core systemd developers.

My release procedure involves not only doing "make distcheck", but
also then building the result in Fedora's koji, which then also involves
another "make check" run, as part of the RPM build process, on all
architectures Fedora supports. Only after that I tag a new
release.

Post by Martin Pitt
Alternative: Stop shipping "make dist" tarballs altogether and just
tar the tagged git snapshot. Given the amount of patching that most
distros do, we pretty much all run autoreconf anyway (including
Fedora), so not having the pre-generated autoconfiscation and
pre-built manpages etc. in the tarball isn't actually that much of
a deal.

Well, while I sympathize with the idea, this is still not how most
current distros work...

Also, "make dist" worked fine as mentioned, hence altering this part
of the release process will not improve anything, but instead make
things even more sloppy...

Post by Martin Pitt
* Impose a release freeze period with announcing an impending
release, distros do a deep testing (this is a day's work for me, so
can't happen that often). During that time (which really shouldn't
be more than a few days) we really should avoid any commit which
isn't an important bug fix, especially not large refactorings or
new features.

How about this: please run automated tests if you have them in regular
intervals, always tracking git. And a few days before a release I'll
notify the mailing list.

I really don't want to drown development in "deep freeze" phases, that
are then alternated with "crazy merge time" phases. Instead, I'd much
rather increase the release frequency, keep a steady stream of changes,
and ask downstreams for more help with testing and keeping git
constantly in a releasable state.

Lennart

[Martin Pitt]

Hey Lennart,

Post by Lennart Poettering

Post by Martin Pitt
* Regularly test "make dist", as nobody does that during regular
development.

Well, no.

You just said before that we (distros) need to check tarball
generation/build more often. So I'm a bit confused by the "no" (but
see below).

Post by Lennart Poettering
I use "make distcheck" regularly during regular development, and
that's how I generate the final tarball.

True, the auto-generated header bug wouldn't have helped if I were to
do it on e. g. Debian sid and then build packages from that very
tarball, as it stemmed from being autogenerated from different kernel
includes etc.

So in that case the only thing that would have discovered this was
distros testing a tarball that you generated. So, back to "RC tarball,
plz test" a few days before you want to release?

Post by Lennart Poettering

Post by Martin Pitt
Alternative: Stop shipping "make dist" tarballs altogether and just
tar the tagged git snapshot. Given the amount of patching that most
distros do, we pretty much all run autoreconf anyway (including
Fedora), so not having the pre-generated autoconfiscation and
pre-built manpages etc. in the tarball isn't actually that much of
a deal.

Well, while I sympathize with the idea, this is still not how most
current distros work...

OK. It was just a thought for completeness.

Post by Lennart Poettering
How about this: please run automated tests if you have them in regular
intervals, always tracking git. And a few days before a release I'll
notify the mailing list.

Sounds good. I actually did this twice (not sure if you saw my IRC
pings about that) I just didn't do it at the "right time". I think
this would have helped to find most issues indeed (except for the
keys-from-name.gperf thing, but meh -- happens seldomly enough), so
let's try this for 221!

Post by Lennart Poettering
I really don't want to drown development in "deep freeze" phases, that
are then alternated with "crazy merge time" phases. Instead, I'd much
rather increase the release frequency, keep a steady stream of changes,
and ask downstreams for more help with testing and keeping git
constantly in a releasable state.

Also sounds good. With getting rid of a few large patches (like the
chkconfig → update-rc. ones, or the man page paths), doing CI more
often on my "distro" side will also become a much less manual process.

Thanks for having this post-mortem discussion, this is useful!

Martin

[Lennart Poettering]

Post by Martin Pitt
Hey Lennart,

Post by Lennart Poettering

Post by Martin Pitt
* Regularly test "make dist", as nobody does that during regular
development.

Well, no.

You just said before that we (distros) need to check tarball
generation/build more often. So I'm a bit confused by the "no" (but
see below).

The "no" was mostly about the part "nobody does that during regular
development". I actually do.

Post by Martin Pitt

Post by Lennart Poettering
I use "make distcheck" regularly during regular development, and
that's how I generate the final tarball.

True, the auto-generated header bug wouldn't have helped if I were to
do it on e. g. Debian sid and then build packages from that very
tarball, as it stemmed from being autogenerated from different kernel
includes etc.
So in that case the only thing that would have discovered this was
distros testing a tarball that you generated. So, back to "RC
tarball, plz test" a few days before you want to release?

Well, the other option is to simply accept that some bugs are not
easily testable for...

I mean, we can of course build tar balls on all distros and then build
them on all others, but this explodes the test matrix for only limited
benefit...

Post by Martin Pitt

Post by Lennart Poettering
How about this: please run automated tests if you have them in regular
intervals, always tracking git. And a few days before a release I'll
notify the mailing list.

Sounds good. I actually did this twice (not sure if you saw my IRC
pings about that) I just didn't do it at the "right time". I think
this would have helped to find most issues indeed (except for the
keys-from-name.gperf thing, but meh -- happens seldomly enough), so
let's try this for 221!

Any chance to automate that in a cron-job on some server?

Ideally we'd really have a Jenkins instance for that, like David
Strauss' one, but testing the .deb packages Ubuntu (or Debian)
builds. If Ubuntu could provide that this would be very welcome, and
of course it would be something I#d check before I do releases.

Lennart

[Martin Pitt]

Post by Lennart Poettering

Post by Martin Pitt
Sounds good. I actually did this twice (not sure if you saw my IRC
pings about that) I just didn't do it at the "right time". I think
this would have helped to find most issues indeed (except for the
keys-from-name.gperf thing, but meh -- happens seldomly enough), so
let's try this for 221!

Any chance to automate that in a cron-job on some server?

Yes, that has been on my wishlist for quite a while. We have
a fairly comprehensive set of integration tests now which give systemd
quite some beating; the main thing that's causing throuble is that
they won't completely succeed without porting patches, but there's
certainly some heuristics/leeway there (e. g. find the ones which are
required and ignore failure to apply the others).

Post by Lennart Poettering
Ideally we'd really have a Jenkins instance for that, like David
Strauss' one, but testing the .deb packages Ubuntu (or Debian)
builds.

*nod*

Martin

[Lennart Poettering]

Post by Martin Pitt

Post by Lennart Poettering

Post by Martin Pitt
Sounds good. I actually did this twice (not sure if you saw my IRC
pings about that) I just didn't do it at the "right time". I think
this would have helped to find most issues indeed (except for the
keys-from-name.gperf thing, but meh -- happens seldomly enough), so
let's try this for 221!

Any chance to automate that in a cron-job on some server?

Yes, that has been on my wishlist for quite a while. We have
a fairly comprehensive set of integration tests now which give systemd
quite some beating; the main thing that's causing throuble is that
they won't completely succeed without porting patches, but there's
certainly some heuristics/leeway there (e. g. find the ones which are
required and ignore failure to apply the others).

And of course: try to reduce them. As the discussion of the last days
has shown I am sure that quite a number of them can be merged upstream
or solved in a different way.

Lennart

[Marc-Antoine Perennou]

Post by Lennart Poettering
I use "make distcheck" regularly during regular development, and
that's how I generate the final tarball.

What about generating a preview tarball before tagging, and letting us
downstreams a couple of days to test it before you do the actual
tagging and put it in the official location?

Regards,
Marc-Antoine

[Lennart Poettering]

Post by Marc-Antoine Perennou

Post by Lennart Poettering
I use "make distcheck" regularly during regular development, and
that's how I generate the final tarball.

What about generating a preview tarball before tagging, and letting us
downstreams a couple of days to test it before you do the actual
tagging and put it in the official location?

I'd really prefer if downstreams would do this continously, and not
just close to the time of release.

Again, we should try to keep things in a releasable state in git,
every single day if possible.

Any chance you can automate testing like this for your distro? And of
course, hook this up with some but that notifies us about breakages,
maybe via IRC or so...

Lennart

[Lennart Poettering]

Post by Dave Reisner

Post by Lennart Poettering

Post by Dimitri John Ledkov
Or will there be a v220.1 release shortly with releasy fix-ups?

Well, we don't do point releases in systemd.
In systemd git we already have now the change that makes sd-bus and
sd-event public APIs, hence v221 is going to be more than just
bugfixes.

Is git revert not an option? Does someone depend on this already, making
a rollback impossible (and why would you allow that)? You say that
versions are "just a label", but a project which adopts semver would be
able to create a branch in git and produce a dot release with backported
bugfixes. Surely you're familiar with this concept -- your colleague
Karel does this routinely with util-linux.

I have no idea what you are talking of. I don't know what "semver" is
supposed to be? Fedora knows no package by that name. A typo?

Post by Dave Reisner
The fact of the matter is, the last 2 releases of systemd have been
brown bag releases. Neither 219 nor 220 are useable as the tarballs are
provided. v220 doesn't even build in a large number of configurations.
v218 wasn't quite as bad, but landed 18 patches in Arch's packaging
before v219 was released. For the average Arch package, this is unheard
of, let alone being a *necessity* in order to make software useable.

Well, they do work fine in the Fedora build, which is what I
test. It's a pretty comprehensive test. And the release also does work
fine on FEdora. How I know that, because I run the newest systemd
versions exclusively myself, and I do watch some bugzillas.

I can only encourage you to test git more frequently on Arch, and not
wait for the release to do this.

As mentioned before, the gperf and EFI issues you are apparently
referring to have been around since a few weeks. They do not appear
with the config options we use upstream or the ones of Fedora. We
*rely* on downstream testing this, otherwise this will *never* work.

Most software does not have as many options we have. Apparently we
have issues we keeping all combinations of the options working. There
are two solutions to this problem:

a) more testing of the combinations of options used by downstream. For
this we'd have to rely on downstream support though.

b) simply remove the options. i.e. remove configure switches, and be
more aggressive with requiring current versions of libraries,
kernel headers, kernels, and so on.

I am open for b), but I guess many people would certainly prefer us
not doing that.

If we'd follow b) then everybody would run things in the same
combination, and ew test this from upstream we can be sufficiently
sure that it will work for everybody else, too.

But for a) we'd need support from projects like Ubuntu or Arch: for
example, a Jenkins instance or so that builds systemd git continouesly
for those systems and boots them.

We currently have Davi Strauss' Jenkins instance, and I am very
thankful for that, but this tests only one specific configuration of
things. If you want ArchLinux' configuration to be tested regularly,
then this means that ArchLinux would have to provide something in the
area, we can never do that from upstream.

Post by Dave Reisner

Post by Lennart Poettering
Also, next week I'll be travelling and I doubt I will be able to
finish a new release by then. Maybe in the middle of June we can get
out a new release.

I'm sorry to hear this too. Please find someone else who's paid to work
on systemd and teach them how to package releases. The current state of
affairs absolutely sucks for downstreams. Currently, packaging systemd
takes more of my time than all of my other packaging responsibilities
combined.

Well, I see noone volunteering unfortunately.

Like most Open Source project we are understaffed. I wished it wasn't
that way, but that's how it is.

Post by Dave Reisner
At LPC in New Orleans, you asked me how you could make systemd easier to
work with on the downstream side. I told you that I wanted more frequent
releases. You seemed amenable to this, but it really never happened.
I'm asking again: can we have more frequent releases? Can you take the
steps to actually make this happen and not gate releases on your own
personal availability?

Oh yes, I'd like 3 week cycles, too. And I'd like to pass release
management to somebody else, because I am unhappy with this too, and
the ridiculous amount of work this brings for me.

However, this simply falls short due to limited manpower. We need
a skilled full-time person for this, and I don't see that growing on
trees unfortunately.

Lennart

[Martin Pitt]

Hello all,

Post by Dimitri John Ledkov
Or will there be a v220.1 release shortly with releasy fix-ups?

Can we perhaps flip that around? I did a make dist/port our
patches/build packackages/run our tests round on May 11, but that was
before most of the recent "hiccups" landed in master. I'd rather love
to do that once I know that a release is around the corner, but as we
don't currently have "release is imminent" warnings none of the
packagers can do this as a pre-release exercise.

So Lennart: WDYT about announcing your intent to do a stable release
on the ML, and then us packagers can do a "make dist", test an actual
tarball, and thus find all these little tarball errors, udev
regressions and what not before the official release is cut?

At least for me, the biggest part of updating to a new release is to
rebase our ~ 50 patches; doing that against a near-release tarball
will be the same amount of work; running automatic tests is negligible
human effort, and running manual tests doesn't take that long for me
(that's the point which might vastly differ for other packagers -- but
then again, testing pre-release tarballs means that not much will
change any more until the real release, so the second time testing
will be faster).

Martin

[Lennart Poettering]

Post by Martin Pitt
Hello all,

Post by Dimitri John Ledkov
Or will there be a v220.1 release shortly with releasy fix-ups?

Can we perhaps flip that around? I did a make dist/port our
patches/build packackages/run our tests round on May 11, but that was
before most of the recent "hiccups" landed in master. I'd rather love
to do that once I know that a release is around the corner, but as we
don't currently have "release is imminent" warnings none of the
packagers can do this as a pre-release exercise.
So Lennart: WDYT about announcing your intent to do a stable release
on the ML, and then us packagers can do a "make dist", test an actual
tarball, and thus find all these little tarball errors, udev
regressions and what not before the official release is cut?

Well, I had been trying to stabilize things since 3 weeks before the
release, I did mention it on IRC back then. I didn't announce this on
the ML though, but I figure I could do that too, for the next cycle...

I tend to ping some people before the release, who I know have
independent test suites or care for last-minute fixes...

Post by Martin Pitt
At least for me, the biggest part of updating to a new release is to
rebase our ~ 50 patches; doing that against a near-release tarball
will be the same amount of work; running automatic tests is negligible
human effort, and running manual tests doesn't take that long for me
(that's the point which might vastly differ for other packagers -- but
then again, testing pre-release tarballs means that not much will
change any more until the real release, so the second time testing
will be faster).

50 non-backport patches? That sounds like a lot...

Lennart

[Martin Pitt]

Post by Lennart Poettering
Well, I had been trying to stabilize things since 3 weeks before the
release

Ah, that's roughly when I did my last "build packages from upstream
trunk" test, and indeed it looked fairly well back then.

Post by Lennart Poettering
I didn't announce this on the ML though, but I figure I could do
that too, for the next cycle...

That would be nice. Let's give it a try and see how it goes? From your
end it's just an extra mail when you are ready to release (and maybe
giving packagers two days or so), and from our end it's just moving
all the heavy-lifting around a bit.

Post by Lennart Poettering
50 non-backport patches? That sounds like a lot...

It's not that bad, most of them are trivial, like tweaking tmpfiles.d
or some extra historic udev rules. We have quite a bunch of patches to
get rid of Fedora/RedHat-isms and replace them with Debianisms, and
then there's a lot for having a non-merged /usr (which Debian didn't
do yet). Aside from two or three which keep breaking most of them just
tag along, but there's always enough noise to break completely
automatically building packages from git master. Anyway, this is
completely tangential.. :-)

Thanks,

Martin

[Lennart Poettering]

Post by Martin Pitt

Post by Lennart Poettering
50 non-backport patches? That sounds like a lot...

It's not that bad, most of them are trivial, like tweaking tmpfiles.d
or some extra historic udev rules. We have quite a bunch of patches to
get rid of Fedora/RedHat-isms and replace them with Debianisms, and
then there's a lot for having a non-merged /usr (which Debian didn't
do yet). Aside from two or three which keep breaking most of them just
tag along, but there's always enough noise to break completely
automatically building packages from git master. Anyway, this is
completely tangential.. :-)

But note that upstream is supposed to be non-redhat centric, and is
supposed to support non-merged /usr. If it's really about that, I#d be
inetersted to know what these patches are about, and maybe we can move
some upstream.

Lennart

[Martin Pitt]

Hey Lennart,

Post by Lennart Poettering
But note that upstream is supposed to be non-redhat centric, and is
supposed to support non-merged /usr. If it's really about that, I#d be
inetersted to know what these patches are about, and maybe we can move
some upstream.

There's a bunch of different config files, such as /etc/locale.conf
and /etc/X11/xorg.conf.d/00-keyboard.conf (Fedora) vs.
/etc/default/locale and /etc/default/keyboard (Debian) (Debian uses
just one keyboard layout for both console and X), but these got
removed upstream to avoid some #ifdeffery [1]. Not much to worry
about.

More intrusive is the removal of chkconfig and addition of
update-rc.d:
http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/Make-systemctl-enable-disable-call-update-rc.d-for-s.patch?h=experimental
http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/systemctl-don-t-skip-native-units-when-enabling-disa.patch?h=experimental

but again moving these upstream would just mean more #ifdeffery.

Then there's the (now completely separate) fsckd which you also
already rejected:
http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/fsckd-daemon-for-inter-fsckd-communication.patch?h=experimental-220
I'd be happy to put that back and maintain it upstream, perhaps with a
--configure option. But it rebases well as the changes to existing
files are minimal.

Then we have some stupid stuff like
http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/buildsys-Don-t-default-to-gold-as-the-linker.patch?h=experimental-220
which got rejected, but are nevertheless still necessary on distros
(and not only for us, see responses to [2])

The single one which I'd *really* like to get rid of is this 91 kB
patch beast:
http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/Fix-paths-in-man-pages.patch?h=experimental-220
This fixes the manpages to give correct paths for non-merged /usr. I
think the correct upstream fix would be to replace the hardcoded
"/usr/lib/systemd/" bits with XML entities like &systemunitdir; or
perhaps more generically &rootlibdir; (even with --enable-split-usr
and --with-rootprefix=/ some bits still land in /usr/, like
/usr/lib/systemd/user{,-generators} and /usr/lib/systemd/catalog/. If
you'd accept a patch for that, I'll work on that.

That's just a random sample; I'm happy to go through our patches [3] in
more details if you want. The manpage one is the one that causes more
work to update than all other patches together, so getting rid of that
would be quite an improvement!

Martin


[1] http://cgit.freedesktop.org/systemd/systemd/commit/?id=46a2911bf
[2] http://lists.freedesktop.org/archives/systemd-devel/2015-February/028544.html
[3] http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches?h=experimental-220

[Lennart Poettering]

Post by Martin Pitt
Hey Lennart,

Post by Lennart Poettering
But note that upstream is supposed to be non-redhat centric, and is
supposed to support non-merged /usr. If it's really about that, I#d be
inetersted to know what these patches are about, and maybe we can move
some upstream.

There's a bunch of different config files, such as /etc/locale.conf
and /etc/X11/xorg.conf.d/00-keyboard.conf (Fedora) vs.
/etc/default/locale and /etc/default/keyboard (Debian) (Debian uses
just one keyboard layout for both console and X), but these got
removed upstream to avoid some #ifdeffery [1]. Not much to worry
about.

/etc/locale.conf is not a fedoraism. It has not existed on Fedora
before we introduced it. It's a systemd'ism if you so will: we looked
at where the distros configured these things and came to the
conclusion that all them were weird and nothing we wanted to
support. We hence unified this in a new file. Except apparently that
DEbian wasn't willing to migrate :-(

I really don't see how the xorg.conf.d drop-in should be
fedora-specific. That's an upstream X11 thing, and we just picked a
name for the file and that's it.

Are you saying Debian patched out supported for /etc/X11/xorg.conf.d?

Or does it have that dir at a different place? if so: why? but we
could certainly make that path configurable with a configure switch...

Post by Martin Pitt
More intrusive is the removal of chkconfig and addition of
http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/Make-systemctl-enable-disable-call-update-rc.d-for-s.patch?h=experimental
http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/systemctl-don-t-skip-native-units-when-enabling-disa.patch?h=experimental

I offered to merge a patch that adds update-rc.d support side-by-side
with chkconfig support for this upstream, many times.

That said, I think even better would be to maybe make the support for
this generic in systemctl: instead of explicitly invoking chkconfig or
update-rcd, maybe we can just make systemctl invoke some fixed binary
/usr/lib/systemd/systemd-sysv-compat or so with a fixed set of
parameters. The distros could then make that a tool (maybe just a
shell script) that invokes chkconfig or update-rc.d This would then
allow us to remove any chkconfig-specific code from systemd, and would
allow all distros to plug-in the tool of their choice without having
to patch upstream. What do you think?

Post by Martin Pitt
Then there's the (now completely separate) fsckd which you also
http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/fsckd-daemon-for-inter-fsckd-communication.patch?h=experimental-220
I'd be happy to put that back and maintain it upstream, perhaps with a
--configure option. But it rebases well as the changes to existing
files are minimal.

If at all I think this should better be added to ply natively.

Post by Martin Pitt
Then we have some stupid stuff like
http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/buildsys-Don-t-default-to-gold-as-the-linker.patch?h=experimental-220
which got rejected, but are nevertheless still necessary on distros
(and not only for us, see responses to [2])

Well, there's no need to carry a patch for that, all you need is add a
configure cmdline param, no?

Also, this is really just working around issues with gold, and
craziness in the suse build system. I am pretty sure bugs should be
fixed wherever they are, and if that's not possible, then a configure
parameter is certainly OK too.

Post by Martin Pitt
The single one which I'd *really* like to get rid of is this 91 kB
http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/Fix-paths-in-man-pages.patch?h=experimental-220
This fixes the manpages to give correct paths for non-merged /usr. I
think the correct upstream fix would be to replace the hardcoded
"/usr/lib/systemd/" bits with XML entities like &systemunitdir; or
perhaps more generically &rootlibdir; (even with --enable-split-usr
and --with-rootprefix=/ some bits still land in /usr/, like
/usr/lib/systemd/user{,-generators} and /usr/lib/systemd/catalog/. If
you'd accept a patch for that, I'll work on that.

Not enthusiastic about the idea. But the XML magic is mostly
Zbigniew's territory (as this long got more complex than my XML-fu can
still grasp ;-)).

Lennart

[Martin Pitt]

This post might be inappropriate. Click to display it.

[Lennart Poettering]

Post by Martin Pitt

Post by Lennart Poettering
I really don't see how the xorg.conf.d drop-in should be
fedora-specific. That's an upstream X11 thing, and we just picked a
name for the file and that's it.
Are you saying Debian patched out supported for /etc/X11/xorg.conf.d?

No, that of course works if it's present, but Debian wants to
configure the keyboard layout for the console and other consumers too,
and parsing Xorg config in kbd/console-setup would be rather weird. So
Debian uses Xorg's udev support by default, which is essentially
KERNEL=="event*", ENV{ID_INPUT_KEY}=="?*",
IMPORT{file}="/etc/default/keyboard"

udev is should not be used for configuration. That's simply
wrong.

Post by Martin Pitt

Post by Lennart Poettering
That said, I think even better would be to maybe make the support for
this generic in systemctl: instead of explicitly invoking chkconfig or
update-rcd, maybe we can just make systemctl invoke some fixed binary
/usr/lib/systemd/systemd-sysv-compat or so with a fixed set of
parameters. The distros could then make that a tool (maybe just a
shell script) that invokes chkconfig or update-rc.d This would then
allow us to remove any chkconfig-specific code from systemd, and would
allow all distros to plug-in the tool of their choice without having
to patch upstream. What do you think?

That sounds great. If chkconfig and update-r.cd are/were the only two
contenders then #ifdef sounds easier, but I don't know about other
distros like e. g. Gentoo.

Well, chkconfig is the only implementation we support natively right
now. A new compat interface like I proposed would only have to replace
this hookup and all would be good.

Post by Martin Pitt

Post by Lennart Poettering

Post by Martin Pitt
http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/Fix-paths-in-man-pages.patch?h=experimental-220

Not enthusiastic about the idea. But the XML magic is mostly
Zbigniew's territory (as this long got more complex than my XML-fu can
still grasp ;-)).

Heh; I had a quick look and I'm not sure how/where to define new
entities. I'll have a closer look when I'm bored (so, maybe in 5 years
or so :-) ).

Daniel is interested in picking this up.

Lennart

[Michael Biebl]

Post by Lennart Poettering

Post by Martin Pitt

Post by Lennart Poettering

Post by Martin Pitt
http://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/patches/Fix-paths-in-man-pages.patch?h=experimental-220

Not enthusiastic about the idea. But the XML magic is mostly
Zbigniew's territory (as this long got more complex than my XML-fu can
still grasp ;-)).

Heh; I had a quick look and I'm not sure how/where to define new
entities. I'll have a closer look when I'm bored (so, maybe in 5 years
or so :-) ).

Daniel is interested in picking this up.

Zbyszek was sympathetic to this idea, see
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=717491#10

[Alexander E. Patrakov]

Post by Martin Pitt

Post by Lennart Poettering
That said, I think even better would be to maybe make the support for
this generic in systemctl: instead of explicitly invoking chkconfig or
update-rcd, maybe we can just make systemctl invoke some fixed binary
/usr/lib/systemd/systemd-sysv-compat or so with a fixed set of
parameters. The distros could then make that a tool (maybe just a
shell script) that invokes chkconfig or update-rc.d This would then
allow us to remove any chkconfig-specific code from systemd, and would
allow all distros to plug-in the tool of their choice without having
to patch upstream. What do you think?

That sounds great. If chkconfig and update-r.cd are/were the only two
contenders then #ifdef sounds easier, but I don't know about other
distros like e. g. Gentoo.

With my Gentoo user hat on: Gentoo needs no support here. Their
initscripts are based on OpenRC, which is not SysV-compatible enough for
the "fall back to the initscript" logic to make sense at all (e.g.,
their initscripts have #!/sbin/runscript in the beginning, have no LSB
header, and produce an error message if run in the system booted by
systemd). In fact, they explicitly disable SysV compatibility in systemd:

# disable sysv compatibility
--with-sysvinit-path=
--with-sysvrcnd-path=

[Martin Pitt]

Post by Lennart Poettering
That said, I think even better would be to maybe make the support for
this generic in systemctl: instead of explicitly invoking chkconfig or
update-rcd, maybe we can just make systemctl invoke some fixed binary
/usr/lib/systemd/systemd-sysv-compat or so with a fixed set of
parameters. The distros could then make that a tool (maybe just a
shell script) that invokes chkconfig or update-rc.d This would then
allow us to remove any chkconfig-specific code from systemd, and would
allow all distros to plug-in the tool of their choice without having
to patch upstream. What do you think?

I just stumbled over

http://refspecs.linuxbase.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/initsrcinstrm.html

which specifies pretty much what we talked about above:

/usr/lib/lsb/install_initd /etc/init.d/example.com-coffeed
/usr/lib/lsb/remove_initd /etc/init.d/example.com-coffeed

So we could make systemctl just call this if it's available, and
otherwise do nothing for init.d scripts.

I'll cook a patch for this.

Martin

[Lennart Poettering]

Post by Martin Pitt

Post by Lennart Poettering
That said, I think even better would be to maybe make the support for
this generic in systemctl: instead of explicitly invoking chkconfig or
update-rcd, maybe we can just make systemctl invoke some fixed binary
/usr/lib/systemd/systemd-sysv-compat or so with a fixed set of
parameters. The distros could then make that a tool (maybe just a
shell script) that invokes chkconfig or update-rc.d This would then
allow us to remove any chkconfig-specific code from systemd, and would
allow all distros to plug-in the tool of their choice without having
to patch upstream. What do you think?

I just stumbled over
http://refspecs.linuxbase.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/initsrcinstrm.html
/usr/lib/lsb/install_initd /etc/init.d/example.com-coffeed
/usr/lib/lsb/remove_initd /etc/init.d/example.com-coffeed
So we could make systemctl just call this if it's available, and
otherwise do nothing for init.d scripts.

Sounds OK to use something like this, that already exists.

However, we actually need not only enabling/disabling, but also
"is-enabled" support, and idea on that?

Also, I'd like to keep Lukas Nykryn in the loop on this, our
initscripts maintainer.

Lukas, any opinion on this? If we remove chkconfig support from
systemctl and replace it with generic install_initd/remove_initd
support, would you be willing to add the two necessary scripts to
Fedora's initscripts package?

Lennart

[Martin Pitt]

Hey Lennart,

Post by Lennart Poettering

Post by Martin Pitt
/usr/lib/lsb/install_initd /etc/init.d/example.com-coffeed
/usr/lib/lsb/remove_initd /etc/init.d/example.com-coffeed
So we could make systemctl just call this if it's available, and
otherwise do nothing for init.d scripts.

Sounds OK to use something like this, that already exists.
However, we actually need not only enabling/disabling, but also
"is-enabled" support, and idea on that?

My current version of the patch keeps the chkconfig implementation for
now; I suppose we don't want to needlessly enforce a lockstep
situation where you can't use systemd git on Fedora until these
scripts exist.

LSB does not define an interface for checking whether an init.d script
is enabled, and e. g. Debian's update-rc.d does not currently either
(https://bugs.debian.org/705254).

We certainly know whether an init.d script is enabled, as we check
exactly that in the sysv-generator (and if it's disabled we don't
create a .service for it). However, right now the systemctl is-enabled
command will just give you a "not supported with sysvinit" error with
--disable-chkconfig.

Post by Lennart Poettering
Also, I'd like to keep Lukas Nykryn in the loop on this, our
initscripts maintainer.

Did you mean to CC: him?

Thanks,

Martin

[Lennart Poettering]

Post by Martin Pitt
Hey Lennart,

Post by Lennart Poettering

Post by Martin Pitt
/usr/lib/lsb/install_initd /etc/init.d/example.com-coffeed
/usr/lib/lsb/remove_initd /etc/init.d/example.com-coffeed
So we could make systemctl just call this if it's available, and
otherwise do nothing for init.d scripts.

Sounds OK to use something like this, that already exists.
However, we actually need not only enabling/disabling, but also
"is-enabled" support, and idea on that?

My current version of the patch keeps the chkconfig implementation for
now; I suppose we don't want to needlessly enforce a lockstep
situation where you can't use systemd git on Fedora until these
scripts exist.

We barely have any init scripts left, this isn't really a big issue
hence. I think it's no problem to require an update in lockstep for
this for Fedora.

Post by Martin Pitt
LSB does not define an interface for checking whether an init.d script
is enabled, and e. g. Debian's update-rc.d does not currently either
(https://bugs.debian.org/705254).
We certainly know whether an init.d script is enabled, as we check
exactly that in the sysv-generator (and if it's disabled we don't
create a .service for it). However, right now the systemctl is-enabled
command will just give you a "not supported with sysvinit" error with
--disable-chkconfig.

I think it should be the hook that generates that error, not systemctl...

Post by Martin Pitt

Post by Lennart Poettering
Also, I'd like to keep Lukas Nykryn in the loop on this, our
initscripts maintainer.

Did you mean to CC: him?

That was my intention, and I could bet I did add him.

Lukas, I added you now, can you have a look at branch of the thread
please?

Lennart

[Lukáš Nykrýn]

Post by Lennart Poettering

Post by Martin Pitt
Hey Lennart,

Post by Lennart Poettering

Post by Martin Pitt
/usr/lib/lsb/install_initd /etc/init.d/example.com-coffeed
/usr/lib/lsb/remove_initd /etc/init.d/example.com-coffeed
So we could make systemctl just call this if it's available, and
otherwise do nothing for init.d scripts.

Sounds OK to use something like this, that already exists.
However, we actually need not only enabling/disabling, but also
"is-enabled" support, and idea on that?

My current version of the patch keeps the chkconfig implementation for
now; I suppose we don't want to needlessly enforce a lockstep
situation where you can't use systemd git on Fedora until these
scripts exist.

We barely have any init scripts left, this isn't really a big issue
hence. I think it's no problem to require an update in lockstep for
this for Fedora.

Post by Martin Pitt
LSB does not define an interface for checking whether an init.d script
is enabled, and e. g. Debian's update-rc.d does not currently either
(https://bugs.debian.org/705254).
We certainly know whether an init.d script is enabled, as we check
exactly that in the sysv-generator (and if it's disabled we don't
create a .service for it). However, right now the systemctl is-enabled
command will just give you a "not supported with sysvinit" error with
--disable-chkconfig.

I think it should be the hook that generates that error, not systemctl...

Post by Martin Pitt

Post by Lennart Poettering
Also, I'd like to keep Lukas Nykryn in the loop on this, our
initscripts maintainer.

Did you mean to CC: him?

That was my intention, and I could bet I did add him.
Lukas, I added you now, can you have a look at branch of the thread
please?
Lennart

We already have /usr/lib/lsb/install_initd in fedora and rhel, it is in
redhat-lsb-core, but basically it is just a symlink to chkconfig.

But I would be careful about it. The main problem is, that according to
LSB install_initd requires that lsb dependencies are satisfied and
refuses to install an initscript if they are not.

Lukas

[Martin Pitt]

Hey Lukáš,

Post by Lukáš Nykrýn
We already have /usr/lib/lsb/install_initd in fedora and rhel, it is in
redhat-lsb-core, but basically it is just a symlink to chkconfig.

Ah, that's not technically correct, as
/usr/lib/lsb/{install,remove}_initd have a different CLI.

But anyway, this is moot. We won't call those from systemd after all,
but instead introduce /usr/lib/systemd/systemd-sysv-install.

Thanks!

Martin

[Lukáš Nykrýn]

Post by Martin Pitt
Hey Lukáš,

Post by Lukáš Nykrýn
We already have /usr/lib/lsb/install_initd in fedora and rhel, it is in
redhat-lsb-core, but basically it is just a symlink to chkconfig.

Ah, that's not technically correct, as
/usr/lib/lsb/{install,remove}_initd have a different CLI.

In that case our chkconfig behaves differently:
https://git.fedorahosted.org/cgit/chkconfig.git/tree/chkconfig.c#n683

Post by Martin Pitt
But anyway, this is moot. We won't call those from systemd after all,
but instead introduce /usr/lib/systemd/systemd-sysv-install.

Yep it probably make sense, in fedora we can do it as yet another alias
for chkconfig.

Post by Martin Pitt
Thanks!
Martin

Lukas