Proposed update to the python policy

[Josselin Mouette]

Hi,

I think it's time to update the python policy with the progress that has
been made in how we build python packages. The proposed diff is
attached. In summary it includes:
* the deprecation of the "current" keyword;
* making Provides: meaningful in the case of inter-module
dependencies, as discussed at Debconf;
* fixes to the erroneous python-support section.

Comments anyone?

[Josselin Mouette]

Post by Josselin Mouette
I think it's time to update the python policy with the progress that has
been made in how we build python packages. The proposed diff is
* the deprecation of the "current" keyword;
* making Provides: meaningful in the case of inter-module
dependencies, as discussed at Debconf;
* fixes to the erroneous python-support section.

* and deprecation of dh_python, of course.

[Piotr Ożarowski]

[Josselin Mouette, 21.03.2007]

Post by Josselin Mouette
* the deprecation of the "current" keyword;

"current" keyword is deprecated? Why? I'm using it a lot and I like
it...

[Pierre Habouzit]

Post by Piotr Ożarowski
[Josselin Mouette, 21.03.2007]

Post by Josselin Mouette
* the deprecation of the "current" keyword;

"current" keyword is deprecated? Why? I'm using it a lot and I like
it...

What are you using it for exactly ? I mean, please give an example,
with an actual package, that would be okay. Because I'm 100% sure that
current is (1) not a good idea (2) not needed in fact.

[Piotr Ożarowski]

[Pierre Habouzit, 21.03.2007]

Post by Pierre Habouzit

Post by Piotr Ożarowski
"current" keyword is deprecated? Why? I'm using it a lot and I like
it...

What are you using it for exactly ? I mean, please give an example,
with an actual package, that would be okay. Because I'm 100% sure that
current is (1) not a good idea (2) not needed in fact.

it's useful for Python applications that need specific Python version.

f.e. if current Python version is 2.4 and my app. will work only with
python2.5 and above, I can Build-depend on python-dev (>= 2.5) | python2.5-dev
and set XS-Python-Version: to "current, >=2.5"

example packages: emma, pypar2, gaupol, griffith


BTW: I like rest of your diffs (I don't know much about "Python-Depends"
control field, though)

PS Is it the right time to think about merging python-{central,support}
or is it to early for that?

[Pierre Habouzit]

Post by Piotr Ożarowski
[Pierre Habouzit, 21.03.2007]

Post by Pierre Habouzit

Post by Piotr Ożarowski
"current" keyword is deprecated? Why? I'm using it a lot and I like
it...

What are you using it for exactly ? I mean, please give an example,
with an actual package, that would be okay. Because I'm 100% sure that
current is (1) not a good idea (2) not needed in fact.

it's useful for Python applications that need specific Python version.
f.e. if current Python version is 2.4 and my app. will work only with
python2.5 and above, I can Build-depend on python-dev (>= 2.5) | python2.5-dev
and set XS-Python-Version: to "current, >=2.5"
example packages: emma, pypar2, gaupol, griffith

could you explain me in which part 'current' is helping you here ? I
missed to understand what asking for:

XS-Python-Version: >= 2.5

would haven't achieved. In fact, what I understand from current, is
that it designs python2.4 for now, so how is that accurate for your
example ?

I mean, try to set:

XS-Python-Version: >= 2.5

and please tell me what it changed in the package you built with that.
I'm curious. Btw I don't think you answered the question properly, as
you didn't explained the think current achieves for you. And honnestly,
it's not a trick question, I mean it, what is its purpose for you. I
don't see its usefulness, but I may miss a thing :)

Post by Piotr Ożarowski
BTW: I like rest of your diffs (I don't know much about "Python-Depends"
control field, though)
PS Is it the right time to think about merging python-{central,support}
or is it to early for that?

It's only thinking for now, and I think it's really the moment to
think about further plans for lenny obviously.

[Piotr Ożarowski]

[Pierre Habouzit, 21.03.2007]

Post by Pierre Habouzit

Post by Piotr Ożarowski
it's useful for Python applications that need specific Python version.
f.e. if current Python version is 2.4 and my app. will work only with
python2.5 and above, I can Build-depend on python-dev (>= 2.5) | python2.5-dev
and set XS-Python-Version: to "current, >=2.5"
example packages: emma, pypar2, gaupol, griffith

could you explain me in which part 'current' is helping you here ? I

I could swear that with this keyword pycentral will set hashbang to
python2.5 (with python2.4 set as default) but apparently it doesn't (I
just tested it with emma), it just sets Depends: to "python (>=2.5) |
python2.5", just like pysupport do with "2.5-" in debian/pyversions
file.

is it a bug in both pycentral and pysupport?

Anyway, if it does not change hashbang then OK, it's deprecated.

Post by Pierre Habouzit

Post by Piotr Ożarowski
PS Is it the right time to think about merging python-{central,support}
or is it to early for that?

It's only thinking for now, and I think it's really the moment to
think about further plans for lenny obviously.

How about a vote about which one should be used in Lenny? I mean, both
systems are doing basically the same thing now. I'm using python-central
in most of my packages (I use python-support in some co-maintained
packages) - I moved to pycentral because pysupport was not handling
Python extensions at the time I needed it. It does handle it now so if
pysupport wins the vote I will be able to easily move to the new default
system.

BTW: I would really love to see pyversions file content moved back to
debian/control file, so that using grep-dctrl would be a lot easier.

[Pierre Habouzit]

Post by Piotr Ożarowski
[Pierre Habouzit, 21.03.2007]

Post by Pierre Habouzit

Post by Piotr Ożarowski
it's useful for Python applications that need specific Python version.
f.e. if current Python version is 2.4 and my app. will work only with
python2.5 and above, I can Build-depend on python-dev (>= 2.5) | python2.5-dev
and set XS-Python-Version: to "current, >=2.5"
example packages: emma, pypar2, gaupol, griffith

could you explain me in which part 'current' is helping you here ? I

I could swear that with this keyword pycentral will set hashbang to
python2.5 (with python2.4 set as default) but apparently it doesn't (I
just tested it with emma), it just sets Depends: to "python (>=2.5) |
python2.5", just like pysupport do with "2.5-" in debian/pyversions
file.
is it a bug in both pycentral and pysupport?

No it was my point: current does nothing. In fact, for the very
example you propose it serves no end at all :)

Post by Piotr Ożarowski

Post by Pierre Habouzit

Post by Piotr Ożarowski
PS Is it the right time to think about merging python-{central,support}
or is it to early for that?

It's only thinking for now, and I think it's really the moment to
think about further plans for lenny obviously.

How about a vote about which one should be used in Lenny? I mean, both

vote sucks, I'd really prefer to see a real solution be drafted. IMHO
the vote should be the last resort.

[Tristan Seligmann]

Post by Pierre Habouzit

Post by Piotr Ożarowski
it's useful for Python applications that need specific Python version.
f.e. if current Python version is 2.4 and my app. will work only with
python2.5 and above, I can Build-depend on python-dev (>= 2.5) | python2.5-dev
and set XS-Python-Version: to "current, >=2.5"
example packages: emma, pypar2, gaupol, griffith

could you explain me in which part 'current' is helping you here ? I
XS-Python-Version: >= 2.5

Doesn't this indicate that the package should be built for all versions
2.5 and up, rather than a single version?

Post by Pierre Habouzit
and please tell me what it changed in the package you built with that.
I'm curious. Btw I don't think you answered the question properly, as
you didn't explained the think current achieves for you. And honnestly,
it's not a trick question, I mean it, what is its purpose for you. I
don't see its usefulness, but I may miss a thing :)

As I understood it, "current" indicates that the package should only be
built for one version of python, the version that is currently the
default version in Debian.

[Piotr Ożarowski]

[Tristan Seligmann, 22.03.2007]

Post by Tristan Seligmann

Post by Pierre Habouzit

Post by Piotr Ożarowski
it's useful for Python applications that need specific Python version.
f.e. if current Python version is 2.4 and my app. will work only with
python2.5 and above, I can Build-depend on python-dev (>= 2.5) | python2.5-dev
and set XS-Python-Version: to "current, >=2.5"
example packages: emma, pypar2, gaupol, griffith

could you explain me in which part 'current' is helping you here ? I
XS-Python-Version: >= 2.5

Doesn't this indicate that the package should be built for all versions
2.5 and up, rather than a single version?

yes, but since package is depending only on python-dev (and not python-all-dev),
python-<system> should assume "current" by default (and add it to XB-Python-Version
so that there will be no problems with recompilation of pyc files when 2.6
will become default)

Post by Tristan Seligmann

Post by Pierre Habouzit
and please tell me what it changed in the package you built with that.
I'm curious. Btw I don't think you answered the question properly, as
you didn't explained the think current achieves for you. And honnestly,
it's not a trick question, I mean it, what is its purpose for you. I
don't see its usefulness, but I may miss a thing :)

As I understood it, "current" indicates that the package should only be
built for one version of python, the version that is currently the
default version in Debian.

not necessary default (see "current, >=2.X" where 2.X is greater than default)
but for single version only, yes. I understand it this way, but
apparently I don't understand "current", though.

I think compiling (private or not) modules provided by {emma,gaupol,grifffith,any
other Python application} for all supported Python versions is just a
waste of space.

[Pierre Habouzit]

Post by Piotr Ożarowski
[Tristan Seligmann, 22.03.2007]

Post by Tristan Seligmann

Post by Pierre Habouzit

Post by Piotr Ożarowski
it's useful for Python applications that need specific Python version.
f.e. if current Python version is 2.4 and my app. will work only with
python2.5 and above, I can Build-depend on python-dev (>= 2.5) | python2.5-dev
and set XS-Python-Version: to "current, >=2.5"
example packages: emma, pypar2, gaupol, griffith

could you explain me in which part 'current' is helping you here ? I
XS-Python-Version: >= 2.5

Doesn't this indicate that the package should be built for all versions
2.5 and up, rather than a single version?

yes, but since package is depending only on python-dev (and not python-all-dev),
python-<system> should assume "current" by default (and add it to XB-Python-Version
so that there will be no problems with recompilation of pyc files when 2.6
will become default)

exactly, putting current is just yet-another-place where the
maintainers declares that he will only prepare the package for "current"
python. And you're right, python-(all-?)-dev is a already here to give a
hint to the dh_tool about your intention. Excellent point.

Post by Piotr Ożarowski

Post by Tristan Seligmann

Post by Pierre Habouzit
and please tell me what it changed in the package you built with that.
I'm curious. Btw I don't think you answered the question properly, as
you didn't explained the think current achieves for you. And honnestly,
it's not a trick question, I mean it, what is its purpose for you. I
don't see its usefulness, but I may miss a thing :)

As I understood it, "current" indicates that the package should only be
built for one version of python, the version that is currently the
default version in Debian.

not necessary default (see "current, >=2.X" where 2.X is greater than default)
but for single version only, yes. I understand it this way, but
apparently I don't understand "current", though.
I think compiling (private or not) modules provided by {emma,gaupol,grifffith,any
other Python application} for all supported Python versions is just a
waste of space.

For private modules, it's not possible to compile _or_ byte-compile it
for many python versions, because a private module lives in a non
versionned directory, hence you can't have it built for two python
version at a time. There is two cases though:
* either it's a pure python private module, and afaict, the case is
handled by dh_py* tools correctly, and nothing is needed from the
maintainer side.
* or it's a binary module, and then binNMU is mandatory.


When it's a public module, then well, I agree with Steve, it depends
on the popularity of the package. See zope e.g., it's always a python
version behind for many reasons, and people programming for zope will
want to be able to use any popular python module around. Not building
your module for them is a bad idea, so again here the case is twofold:
* either it's a pure python public module, and then, dh_py* tools will
handle an optimal byte compilation by themselves. Let them work like
that, it's exactly what is needed: if byte compiles only for python
versions that are installed on the user machine. There is absolutely
no waste here.
* either it's a binary module, and yes, then it has to be built by the
buildd, hence we can have some sort of waste, and the "do you have a
lot of rdepends or not" is the criterion to know if you should
provide modules for every supported version, or only default.

Just consider that when you are built for many python at the same
time, you ease the python default change a lot.

So there is pros and cons, and I now (with Steves remarks) believe
that this choice is indeed up to the maintainer. So as a matter of a
conclusion, we just need a way to make "us" aware that the maintainer
chosed one or the other.

I like your python-dev vs. python-all-dev remark, I really believe the
dh_py* helpers can use that when there is public binary modules involved
to know what the maintainer decided. For private modules, the task is
easy, and I know dh_pysupport knows how to detect them already, it would
just have to flag them in the Packages.gz somehow in order to help the
RMs.


Of course, that still leaves one corner case:
* there is only one python version supported (current scenario btw, we
only have python2.4) ;
* the package B-D upon python-all-dev ;
* the maintainer only builds for default.

This is the sole case when the dh_py* tools cannot verify if the
maintainer did correct stuff, but that's definitely a bug in the
package. Moreover we can add a lintian check for that: a package that
build-dep onto python-all-dev and do not use $(pyversions -s) has many
chances to be buggy.

I believe pieces are sticking together nicely.

Oh and for your python2.5 only modules, a simple way to have a
painless binNMU _and_ backport friendly transition would be:

debian/control: Build-Depends: python-dev (>= 2.5) | python2.5-dev
debian/pyversions: 2.5- // or the equivalent XS-P-V: >= 2.5
debian/rules:
# pseudo code
if python2.5 is >= $(shell pyversions -d); then
PYTHON = python
else
PYTHON = python2.5
fi
# use $PYTHON everywhere in debian/rules

for now pyversions does not helps to write debian/rules, but I think
it should have a way to answer the question "does my asked
Python-Version also supports current", what he now does through current.
So _YEAH_ thanks to you, I know _where_ current is used for real. cute:
it's used to write the previous ! and indeed, you were correct to ask
for ">= 2.5, current" after all, if then in your debian/rules you did
sth like that:

PYTHON:=$(if $(shell pyversions -r),python,python2.5)


aaaaaaaall right, I think I get it all correctly now. Time to think
about something less confusing :)

[Josselin Mouette]

Post by Pierre Habouzit
exactly, putting current is just yet-another-place where the
maintainers declares that he will only prepare the package for "current"
python. And you're right, python-(all-?)-dev is a already here to give a
hint to the dh_tool about your intention. Excellent point.

Just nitpicking: the dh_ tool doesn't need to know that, as it can guess
it from what was previously built. This is a hint for the release
managers (to know which packages need a binNMU), and could be the base
for a script automating the build process.

[Pierre Habouzit]

Post by Josselin Mouette

Post by Pierre Habouzit
exactly, putting current is just yet-another-place where the
maintainers declares that he will only prepare the package for "current"
python. And you're right, python-(all-?)-dev is a already here to give a
hint to the dh_tool about your intention. Excellent point.

Just nitpicking: the dh_ tool doesn't need to know that, as it can guess
it from what was previously built. This is a hint for the release
managers (to know which packages need a binNMU), and could be the base
for a script automating the build process.

It's not necessarily true: when there is only one python supported,
you can't tell the difference between a package that only supports the
default python, or any of them :)

And the maintainer can change his mind too, so looking in the past
packages looks brittle.

[Josselin Mouette]

Post by Pierre Habouzit

Post by Josselin Mouette
Just nitpicking: the dh_ tool doesn't need to know that, as it can guess
it from what was previously built. This is a hint for the release
managers (to know which packages need a binNMU), and could be the base
for a script automating the build process.

It's not necessarily true: when there is only one python supported,
you can't tell the difference between a package that only supports the
default python, or any of them :)

This is not a problem for the dh_ tool, as the resulting binary package
will be exactly the same in both cases.

This is indeed a problem for the release managers, as they need a way to
disambiguate between these two cases. In this case, "current" is a hint,
a declaration that has to be matched by what's in debian/rules, but it's
not a proof the package is built this way.

If we really need such a hint, I'd prefer to see it in another place
than the field containing version information. However we might as well
use the python-dev / python-all-dev build-dependencies to obtain the
same information.

I don't think we are missing any case, but we should probably write some
binNMU detection script for use by the release managers, summarizing all
of these thoughts.

[Piotr Ożarowski]

[Josselin Mouette, 22.03.2007]

Post by Josselin Mouette

Post by Pierre Habouzit

Post by Josselin Mouette
Just nitpicking: the dh_ tool doesn't need to know that, as it can guess
it from what was previously built. This is a hint for the release
managers (to know which packages need a binNMU), and could be the base
for a script automating the build process.

It's not necessarily true: when there is only one python supported,
you can't tell the difference between a package that only supports the
default python, or any of them :)

This is not a problem for the dh_ tool, as the resulting binary package
will be exactly the same in both cases.

simplejson package is build only for python2.4, because only this version
is fully supported in Debian now. I (as a package maintainer) want it to be
(byte-)compiled for all supported Python versions. It's arch:all so when
new (2.5, 2.6, ...) Python version will be added to the supported ones or user
will install unsupported Python version (that still matches versions specified
in package's Python-Version header) I want python-<system> to byte-compile it
for all installed Python versions that my package can work with.

I think it's dh_tool task to set Python-Version field correctly (by
parsing build dependencies and XS-P-V / pyversions) so that
python-<system> and RMs will not have problems with telling (grep-dctrl):

* which packages will need binNMU when list of supported Python version
will change (public python extensions)
* which packages will need binNMU when default Python version will
change (shebang issue, private python extensions)
* which packages will only need byte-compilation for newly added Python
versions (or removal of old .pyc files)
* which packages will only need recompilation of .pyc files (private modules,
public modules used only by Python applications ["current" keyword])

If dh_tool will set Python-Version header in binary package and it's
(Python-Version's) content will be standardised as well as the location
where python-<system> should look for .py/.so files, I think we can just
set Depend: to python-system where python-system is provided by
python-central or python-support (no need to install both systems)

Post by Josselin Mouette
This is indeed a problem for the release managers, as they need a way to
disambiguate between these two cases. In this case, "current" is a hint,
a declaration that has to be matched by what's in debian/rules, but it's
not a proof the package is built this way.
If we really need such a hint, I'd prefer to see it in another place
than the field containing version information. However we might as well
use the python-dev / python-all-dev build-dependencies to obtain the
same information.

python-dev / python-all-dev issue should be (IMHO, of course) used just
at the build time to set Python-Version correctly. python-system should
not depend on informations from source packages (it simply doesn't have
access to them). Python-Version (or whatever we will call it) should
contain informations about:

* which Python versions this package can work with
* is this package meant to be compiled for all allowed Python versions or
just for a single (default or nearest available) version
* and maybe: does this package has a shebang to handle

If the second, python-system can check [1] if it can/should change shebangs
automatically (if package is arch:all and default Python version meets
requirements)

[1] at user's machine (when new version of "python" package will be
installed)

Post by Josselin Mouette
I don't think we are missing any case, but we should probably write some
binNMU detection script for use by the release managers, summarizing all
of these thoughts.

It should be really easy if all Python-related packages will have
Python-Version field.

[Pierre Habouzit]

Post by Piotr Ożarowski
[Josselin Mouette, 22.03.2007]

Post by Josselin Mouette

Post by Pierre Habouzit

Post by Josselin Mouette
Just nitpicking: the dh_ tool doesn't need to know that, as it can guess
it from what was previously built. This is a hint for the release
managers (to know which packages need a binNMU), and could be the base
for a script automating the build process.

It's not necessarily true: when there is only one python supported,
you can't tell the difference between a package that only supports the
default python, or any of them :)

This is not a problem for the dh_ tool, as the resulting binary package
will be exactly the same in both cases.

simplejson package is build only for python2.4, because only this version

That's a pure python module, that scope of the policy is well
understood, and works without anybody's help thanks to the dh_tools.
There is no human help needed for that, at all.

Post by Piotr Ożarowski

Post by Josselin Mouette
This is indeed a problem for the release managers, as they need a way to
disambiguate between these two cases. In this case, "current" is a hint,
a declaration that has to be matched by what's in debian/rules, but it's
not a proof the package is built this way.
If we really need such a hint, I'd prefer to see it in another place
than the field containing version information. However we might as well
use the python-dev / python-all-dev build-dependencies to obtain the
same information.

python-dev / python-all-dev issue should be (IMHO, of course) used just
at the build time to set Python-Version correctly. python-system should
not depend on informations from source packages (it simply doesn't have
access to them). Python-Version (or whatever we will call it) should
* which Python versions this package can work with

that's not doable. The PV field or its equivalent debian/pyversions
has to be filled. There is no way for the build system to know if this
or that extension is compatible with this or that python version. This
is a declaration that the maintainer has to do.

This information is only useful for the dh_tool though, so I don't see
the need to make it mandatory. It's up to the maintainer to choose what
fits his needs. What has to be mandatory is the annotations that other
python packages needs (the Python-Provides proposal), and what the RM
need (and here PV is only part of the things the RM need).

Post by Piotr Ożarowski
* is this package meant to be compiled for all allowed Python versions or
just for a single (default or nearest available) version
* and maybe: does this package has a shebang to handle

that is the maintainer's job and maybe the dh_tool one to deal with
that.

Post by Piotr Ożarowski
If the second, python-system can check [1] if it can/should change shebangs
automatically (if package is arch:all and default Python version meets
requirements)

No he should not. At least not automatically. In fact, when the
package is pure python, that already works automatically.

I don't really think you grasp the issues we were discussing. The
"issues" you raise are already dealt with and correctly understood.
Though the policy is really badly written if it wasn't already obvious
to you (and I do thing it is, badly written I mean).

Post by Piotr Ożarowski

Post by Josselin Mouette
I don't think we are missing any case, but we should probably write some
binNMU detection script for use by the release managers, summarizing all
of these thoughts.

It should be really easy if all Python-related packages will have
Python-Version field.

No, honnestly I don't think you get it correctly, but I may be
mistaken. Though I think one of my mail already explains the whole thing
pretty clearly.

[Piotr Ożarowski]

[Pierre Habouzit, 22.03.2007]

Post by Pierre Habouzit

Post by Piotr Ożarowski
[Josselin Mouette, 22.03.2007]

Post by Josselin Mouette

Post by Pierre Habouzit

Post by Josselin Mouette
Just nitpicking: the dh_ tool doesn't need to know that, as it can guess
it from what was previously built. This is a hint for the release
managers (to know which packages need a binNMU), and could be the base
for a script automating the build process.

It's not necessarily true: when there is only one python supported,
you can't tell the difference between a package that only supports the
default python, or any of them :)

This is not a problem for the dh_ tool, as the resulting binary package
will be exactly the same in both cases.

simplejson package is build only for python2.4, because only this version

That's a pure python module, that scope of the policy is well
understood, and works without anybody's help thanks to the dh_tools.
There is no human help needed for that, at all.

I wasn't in Mexico, I didn't write the policy but (I think) I still know
the basics of the new policy. I'm maintaining pure Python modules [1],
Python extensions [2], Python applications with public modules [3],
Python applications with private modules [4] and I try to help other
Debian Python Modules Team members. I just want to say that current
tools are not perfect [5] to me and I think we could make them better
(and I'm using both, though I prefer python-central for now; heck, I
have even sent (accepted) patches to CDBS, but still prefer pure
debhelper)

[1] chardet, elixir, jinja, mako, paste, pastedeploy, pastescript,
pastewebkit, pudge, pygments, routes, simplejson, sqlalchemy
[2] pyenchant, pywavelets
[3] gaupol
[4] griffith, pypar2
[5] f.e. both systems are not handling old .pyc files (#385775, #409390)

Post by Pierre Habouzit

Post by Piotr Ożarowski

Post by Josselin Mouette
This is indeed a problem for the release managers, as they need a way to
disambiguate between these two cases. In this case, "current" is a hint,
a declaration that has to be matched by what's in debian/rules, but it's
not a proof the package is built this way.
If we really need such a hint, I'd prefer to see it in another place
than the field containing version information. However we might as well
use the python-dev / python-all-dev build-dependencies to obtain the
same information.

python-dev / python-all-dev issue should be (IMHO, of course) used just
at the build time to set Python-Version correctly. python-system should
not depend on informations from source packages (it simply doesn't have
access to them). Python-Version (or whatever we will call it) should
* which Python versions this package can work with

that's not doable. The PV field or its equivalent debian/pyversions
has to be filled. There is no way for the build system to know if this
or that extension is compatible with this or that python version. This
is a declaration that the maintainer has to do.

I'm just saying that there's no need to put "current" (better name would
be "single") in XS-P-V but dh_tool should put it in *XB-P-V* when
needed.

I still think it's maintainer's job to specify (XS-P-V field) all
compatible Python versions for the package.

Post by Pierre Habouzit
This information is only useful for the dh_tool though, so I don't see
the need to make it mandatory. It's up to the maintainer to choose what
fits his needs. What has to be mandatory is the annotations that other
python packages needs (the Python-Provides proposal), and what the RM
need (and here PV is only part of the things the RM need).

another example: when python2.3 was default and pytho2.4 "only" a
supported one, I've packaged gaupol. Gaupol is Python application with
public modules (i.e. it has files in "site-packages" directory) and it
requires at least python2.4. All I needed to do was:
* set Build-depends: to python-dev (>=2.4) | python2.4-dev
* set XS-P-V to "current, >=2.4"
and pycentral did change the hashbang and didn't compile modules for 2.3
(and I mean on user's machines as well as during the build time). Now I
just want to be sure that pycentral will not compile it for python2.5
when it will be added to the supported Python-versions. Without
"current" in XB-Python-Version I can't be sure of that.

Post by Pierre Habouzit

Post by Piotr Ożarowski
* is this package meant to be compiled for all allowed Python versions or
just for a single (default or nearest available) version
* and maybe: does this package has a shebang to handle

that is the maintainer's job and maybe the dh_tool one to deal with
that.

ok. I just thought it would be great if I could avoid another binNMU
just to change hashbang when default Python version changes (to change,
lets say "python2.4" to "python" - like I did with gaupol months ago)

Post by Pierre Habouzit

Post by Piotr Ożarowski
If the second, python-system can check [1] if it can/should change shebangs
automatically (if package is arch:all and default Python version meets
requirements)

No he should not. At least not automatically. In fact, when the
package is pure python, that already works automatically.

At build time, yes. I was talking about runtime.

Post by Pierre Habouzit
I don't really think you grasp the issues we were discussing. The
"issues" you raise are already dealt with and correctly understood.
Though the policy is really badly written if it wasn't already obvious
to you (and I do thing it is, badly written I mean).

I know 90% of what I wrote is already implemented in pycentral *or*
pysupport. I just wanted to say what I want from python-system before it
could be provided by python-{central,support} (I want them to be merged
at some time).

Post by Pierre Habouzit

Post by Piotr Ożarowski

Post by Josselin Mouette
I don't think we are missing any case, but we should probably write some
binNMU detection script for use by the release managers, summarizing all
of these thoughts.

It should be really easy if all Python-related packages will have
Python-Version field.

No, honnestly I don't think you get it correctly, but I may be
mistaken. Though I think one of my mail already explains the whole thing
pretty clearly.

If Python-Version will have all data I mentioned in my previous mail
(and Python-Version is a field in *binary* package generated
automatically from XS-P-V and Build dependencies during the build
process), I still think it would be easy to detect all packages that
need binNMU.

[Steve Langasek]

Post by Piotr Ożarowski
[Tristan Seligmann, 22.03.2007]

Post by Tristan Seligmann

Post by Pierre Habouzit

Post by Piotr Ożarowski
it's useful for Python applications that need specific Python version.
f.e. if current Python version is 2.4 and my app. will work only with
python2.5 and above, I can Build-depend on python-dev (>= 2.5) | python2.5-dev
and set XS-Python-Version: to "current, >=2.5"
example packages: emma, pypar2, gaupol, griffith

could you explain me in which part 'current' is helping you here ? I
XS-Python-Version: >= 2.5

Doesn't this indicate that the package should be built for all versions
2.5 and up, rather than a single version?

yes, but since package is depending only on python-dev (and not python-all-dev),
python-<system> should assume "current" by default (and add it to XB-Python-Version
so that there will be no problems with recompilation of pyc files when 2.6
will become default)

hmm, three things:

- relying on Build-Depends to indicate whether a package builds "current" or
"all" doesn't seem to leave a way to differentiate between packages that
follow the new policy and really /are/ binNMUable, from those that don't
follow the new policy but obviously still need to b-d on python-*dev.
- Build-Depends information is only in the Sources file, not in Packages;
detecting packages that need binNMUs requires trawling the Packages file,
it would be nice if it didn't require correlating both Packages and
Sources
- having a package's build rules behavior vary in response to the contents
of the build-depends is unprecedented and, IMHO, a very bad idea.
Basically, this would eliminate a very important check that the maintainer
hasn't made a mistake along the line -- it's far better to get a build
failure in such a case than to get a misbuilt package.

Post by Piotr Ożarowski

Post by Tristan Seligmann
As I understood it, "current" indicates that the package should only be
built for one version of python, the version that is currently the
default version in Debian.

not necessary default (see "current, >=2.X" where 2.X is greater than default)
but for single version only, yes. I understand it this way, but
apparently I don't understand "current", though.

I don't think it was intended that "current, >= 2.X" be used to
*successfully* build packages when 2.X is greater than the currently
available python-dev.

[Piotr Ożarowski]

[Steve Langasek, 22.03.2007]

Post by Steve Langasek

Post by Piotr Ożarowski
yes, but since package is depending only on python-dev (and not python-all-dev),
python-<system> should assume "current" by default (and add it to XB-Python-Version
so that there will be no problems with recompilation of pyc files when 2.6
will become default)

- relying on Build-Depends to indicate whether a package builds "current" or
"all" doesn't seem to leave a way to differentiate between packages that
follow the new policy and really /are/ binNMUable, from those that don't
follow the new policy but obviously still need to b-d on python-*dev.
- Build-Depends information is only in the Sources file, not in Packages;
detecting packages that need binNMUs requires trawling the Packages file,
it would be nice if it didn't require correlating both Packages and
Sources

Build-Depends is used only at build time. Python-Versions field is in binary
package.

Post by Steve Langasek
- having a package's build rules behavior vary in response to the contents
of the build-depends is unprecedented and, IMHO, a very bad idea.
Basically, this would eliminate a very important check that the maintainer
hasn't made a mistake along the line -- it's far better to get a build
failure in such a case than to get a misbuilt package.

so lets keep "current" in XS-P-V (and not only in XB-P-V)

BTW: I'm building my python modules for *all* supported Python
version at build time, even for arch:all packages - just to make sure
they compile fine.

Post by Steve Langasek

Post by Piotr Ożarowski

Post by Tristan Seligmann
As I understood it, "current" indicates that the package should only be
built for one version of python, the version that is currently the
default version in Debian.

not necessary default (see "current, >=2.X" where 2.X is greater than default)
but for single version only, yes. I understand it this way, but
apparently I don't understand "current", though.

I don't think it was intended that "current, >= 2.X" be used to
*successfully* build packages when 2.X is greater than the currently
available python-dev.

if python-dev (>=2.X) | python2.X-dev is not available during build
process, it will just fail and maintainer will not be able to upload
such package, am I right?

BTW: "current, >=2.4" helped me a lot with packaging gaupol when
python2.3 was default

[Steve Langasek]

Post by Piotr Ożarowski

Post by Steve Langasek
- relying on Build-Depends to indicate whether a package builds "current" or
"all" doesn't seem to leave a way to differentiate between packages that
follow the new policy and really /are/ binNMUable, from those that don't
follow the new policy but obviously still need to b-d on python-*dev.
- Build-Depends information is only in the Sources file, not in Packages;
detecting packages that need binNMUs requires trawling the Packages file,
it would be nice if it didn't require correlating both Packages and
Sources

Build-Depends is used only at build time. Python-Versions field is in binary
package.

Sorry, what's your point? You seem to be repeating what I've said.

Post by Piotr Ożarowski

Post by Steve Langasek

Post by Piotr Ożarowski

Post by Tristan Seligmann
As I understood it, "current" indicates that the package should only be
built for one version of python, the version that is currently the
default version in Debian.

not necessary default (see "current, >=2.X" where 2.X is greater than default)
but for single version only, yes. I understand it this way, but
apparently I don't understand "current", though.

I don't think it was intended that "current, >= 2.X" be used to
*successfully* build packages when 2.X is greater than the currently
available python-dev.

if python-dev (>=2.X) | python2.X-dev is not available during build
process, it will just fail and maintainer will not be able to upload
such package, am I right?

Ugh, it should fail *regardless* of the existence of python2.X-dev. Why
would you ever call it "current" if it's building for something that *isn't*
the current version of python? A package should only be called "python-foo"
if it's built for "python"; if it's built for python2.X explicitly, the
package name needs to reflect that, which means manual changes are needed to
update it for a new python version. That's out of scope for 'current'.

Post by Piotr Ożarowski
BTW: "current, >=2.4" helped me a lot with packaging gaupol when
python2.3 was default

Which is not an arch: any package, so is irrelevant to binNMU support.

Oh, and if gaupol really needs python 2.4 or better, then the package's
current dependencies are wrong...

[Piotr Ożarowski]

[Steve Langasek, 23.03.2007]

Post by Steve Langasek

Post by Piotr Ożarowski

Post by Steve Langasek
- relying on Build-Depends to indicate whether a package builds "current" or
"all" doesn't seem to leave a way to differentiate between packages that
follow the new policy and really /are/ binNMUable, from those that don't
follow the new policy but obviously still need to b-d on python-*dev.
- Build-Depends information is only in the Sources file, not in Packages;
detecting packages that need binNMUs requires trawling the Packages file,
it would be nice if it didn't require correlating both Packages and
Sources

Build-Depends is used only at build time. Python-Versions field is in binary
package.

Sorry, what's your point? You seem to be repeating what I've said.

My point is Python-Version can contain "current" in XB-P-V even if it's
not in XS-P-V. It was just an proposal (for people that don't like
redundant data)

Post by Steve Langasek

Post by Piotr Ożarowski

Post by Steve Langasek

Post by Piotr Ożarowski

Post by Tristan Seligmann
As I understood it, "current" indicates that the package should only be
built for one version of python, the version that is currently the
default version in Debian.

not necessary default (see "current, >=2.X" where 2.X is greater than default)
but for single version only, yes. I understand it this way, but
apparently I don't understand "current", though.

I don't think it was intended that "current, >= 2.X" be used to
*successfully* build packages when 2.X is greater than the currently
available python-dev.

if python-dev (>=2.X) | python2.X-dev is not available during build
process, it will just fail and maintainer will not be able to upload
such package, am I right?

Ugh, it should fail *regardless* of the existence of python2.X-dev. Why
would you ever call it "current" if it's building for something that *isn't*
the current version of python? A package should only be called "python-foo"
if it's built for "python"; if it's built for python2.X explicitly, the
package name needs to reflect that, which means manual changes are needed to
update it for a new python version. That's out of scope for 'current'.

when I write "current" I mean "single" (I didn't choose name for this
keyword)

If package is build for a single Python version and default Python
version is not supported by this package, hashbang has to be set
correctly and modules it provides (byte-)compiled (at build time *and*
during the install/default python version change)

Post by Steve Langasek

Post by Piotr Ożarowski
BTW: "current, >=2.4" helped me a lot with packaging gaupol when
python2.3 was default

Which is not an arch: any package, so is irrelevant to binNMU support.

I couldn't set "python" in hashbang (as I said before: gaupol will not work
with python2.3). Package was build when python2.3 was default so
hashbang was set to python2.4. Now when python2.3 was removed from
Debian, package needed binNMU (due to wrong hashbang) even if it's
arch:all.

Post by Steve Langasek
Oh, and if gaupol really needs python 2.4 or better, then the package's
current dependencies are wrong...

python2.4 is default now so there's no need to add extra dependencies

[Pierre Habouzit]

Post by Piotr Ożarowski
[Steve Langasek, 23.03.2007]

Post by Steve Langasek

Post by Piotr Ożarowski

Post by Steve Langasek
- relying on Build-Depends to indicate whether a package builds "current" or
"all" doesn't seem to leave a way to differentiate between packages that
follow the new policy and really /are/ binNMUable, from those that don't
follow the new policy but obviously still need to b-d on python-*dev.
- Build-Depends information is only in the Sources file, not in Packages;
detecting packages that need binNMUs requires trawling the Packages file,
it would be nice if it didn't require correlating both Packages and
Sources

Build-Depends is used only at build time. Python-Versions field is in binary
package.

Sorry, what's your point? You seem to be repeating what I've said.

My point is Python-Version can contain "current" in XB-P-V even if it's
not in XS-P-V. It was just an proposal (for people that don't like
redundant data)

current in X?-P-V sucks a lot because X?-P-V explains which python
version the package supports, whereas current is not about that but
about the kind of packaging ways it has. This information should never
have been folded in the same field, I only recently got what it really
meant because of that. It's more than confusing.

Post by Piotr Ożarowski

Post by Steve Langasek
Ugh, it should fail *regardless* of the existence of python2.X-dev. Why
would you ever call it "current" if it's building for something that *isn't*
the current version of python? A package should only be called "python-foo"
if it's built for "python"; if it's built for python2.X explicitly, the
package name needs to reflect that, which means manual changes are needed to
update it for a new python version. That's out of scope for 'current'.

when I write "current" I mean "single" (I didn't choose name for this
keyword)
If package is build for a single Python version and default Python
version is not supported by this package, hashbang has to be set
correctly and modules it provides (byte-)compiled (at build time *and*
during the install/default python version change)

IMHO that's better suited to say that to the dh_tool. Like a :

dh_py{support,central} --single-version

That would indeed flag the package in the Packages accordingly (but
XB-P-V is not the place IMHO).

But yes, this information really has to "leak" somehow as the RM needs
it.



As of your python-dev (>> 2.5) | python2.5-dev be warned that it does
not works for an arch:any package: sbuild always take the first
alternative, so an arch:any package won't build with that hack. It only
works for arch:all packages as pbuilder/debuild/...whatever are more
clever about that.

So there is no way to deal with packages that only support "future"
python versions yet. Though I expect it to concern only a few packages
that will need a sourcefull upload to migrate to a scheme where they
support the policy.

Your B-D + XS-P-V hack only works with brittle side effects, and for
arch:all packages, that are not subject to binNMUs anyway (at least not
untill we have arch:all buildd's).

[Piotr Ożarowski]

[Pierre Habouzit, 23.03.2007]

Post by Pierre Habouzit
current in X?-P-V sucks a lot because X?-P-V explains which python
version the package supports, whereas current is not about that but
about the kind of packaging ways it has. This information should never
have been folded in the same field, I only recently got what it really
meant because of that. It's more than confusing.

OK

X?-Python-Version: informations about supported Python versions ("2.3-", ">2.3",
">=2.4, <=2.7", etc.)
XB-Python-Type: "multiple" (compile for all installed [and supported by
the package] Python versions) or "single" (only for one Python version)

That looks good to me

Post by Pierre Habouzit
dh_py{support,central} --single-version

another solution and it looks good as well. If it will set package type correctly
so that python-system (the one installed on target machine, not the one installed
on build machine) will byte-compile only for Python versions I want, then it's OK
with me. I'm able to control Python versions used during build time, if
I will be able to control it on target machine as well (through
XB-Python-Version and XB-Python-Type) than it's all what I want.

[Josselin Mouette]

Post by Piotr Ożarowski
XB-Python-Type: "multiple" (compile for all installed [and supported by
the package] Python versions) or "single" (only for one Python version)
That looks good to me

And how do you ensure that this matches what's actually done inside
debian/rules?

[Pierre Habouzit]

Post by Josselin Mouette

Post by Piotr Ożarowski
XB-Python-Type: "multiple" (compile for all installed [and supported by
the package] Python versions) or "single" (only for one Python version)
That looks good to me

And how do you ensure that this matches what's actually done inside
debian/rules?

XB is a binary package header. It's up to the dh_tool responsibility
to check that if the maintainer puts "single" and that there is multiple
python obviously supported it's wrong, and it should make the package
FTBFS.

Obviously, the problem is harder when only one python version is
supported at the time, as you cannot made the difference between the
two.

That's why I've not done any proposal yet, because the problem does
not look obvious in the first glance.

[Piotr Ożarowski]

[Pierre Habouzit, 23.03.2007]

Post by Pierre Habouzit

Post by Josselin Mouette

Post by Piotr Ożarowski
XB-Python-Type: "multiple" (compile for all installed [and supported by
the package] Python versions) or "single" (only for one Python version)
That looks good to me

And how do you ensure that this matches what's actually done inside
debian/rules?

and how do we do this now? It works fine with python-central (it just
adds "current" to the XB-P-V)

Post by Pierre Habouzit
XB is a binary package header. It's up to the dh_tool responsibility
to check that if the maintainer puts "single" and that there is multiple
python obviously supported it's wrong, and it should make the package
FTBFS.

FTBFS? Why? No mater for how many Python versions you build your module in
debian/rules, only one set of .py files is installed (in
/usr/share/python-support/ or /usr/share/pycentral/) - that doesn't
apply to .so files, but why should the build process fail? If more than
one .so file is build, only the right one should be installed.

Maintainer have to know what "single" means and why he wants it. It
should not be used with "normal" Python modules (i.e. python-*
packages). If this field is not set, "multiple" should be assumed.

I think it can be detected automatically (f.e. by using mentioned
python-dev vs. python-all-dev dependency or "dh_tool --single-version")
but if you think it's confusing, then setting it by hand shouldn't be a
big problem.

Post by Pierre Habouzit
Obviously, the problem is harder when only one python version is
supported at the time, as you cannot made the difference between the
two.

Sorry, I don't see a problem here. This field cannot be set only by
checking for how many Python versions module was build at the build
time.

Post by Pierre Habouzit
That's why I've not done any proposal yet, because the problem does
not look obvious in the first glance.

I know that it could be hard to understand me (I blame my English skills)
but I think you all understand the problem now and "current" will not
be removed from the policy. My packages are safe for now ;-)

BTW: I was using "XS-Python-Version: 2.4" when "current, >=X.Y" was not
supported. That really wasn't "New policy"-like.

PS I thought it's time to think about merging py{central,support} but I
guess this discussion will end on "current" keyword.

[Pierre Habouzit]

Post by Piotr Ożarowski
[Pierre Habouzit, 23.03.2007]

Post by Pierre Habouzit

Post by Josselin Mouette

Post by Piotr Ożarowski
XB-Python-Type: "multiple" (compile for all installed [and supported by
the package] Python versions) or "single" (only for one Python version)
That looks good to me

And how do you ensure that this matches what's actually done inside
debian/rules?

and how do we do this now? It works fine with python-central (it just
adds "current" to the XB-P-V)

Post by Pierre Habouzit
XB is a binary package header. It's up to the dh_tool responsibility
to check that if the maintainer puts "single" and that there is multiple
python obviously supported it's wrong, and it should make the package
FTBFS.

FTBFS? Why? No mater for how many Python versions you build your module in
debian/rules, only one set of .py files is installed (in
/usr/share/python-support/ or /usr/share/pycentral/) - that doesn't
apply to .so files, but why should the build process fail? If more than
one .so file is build, only the right one should be installed.
Maintainer have to know what "single" means and why he wants it. It
should not be used with "normal" Python modules (i.e. python-*
packages). If this field is not set, "multiple" should be assumed.
I think it can be detected automatically (f.e. by using mentioned
python-dev vs. python-all-dev dependency or "dh_tool --single-version")
but if you think it's confusing, then setting it by hand shouldn't be a
big problem.

Post by Pierre Habouzit
Obviously, the problem is harder when only one python version is
supported at the time, as you cannot made the difference between the
two.

Sorry, I don't see a problem here. This field cannot be set only by
checking for how many Python versions module was build at the build
time.

My point was: if the maintainer asks for a "single" packaging way, and
that the dh_tool finds a "multiple" package or the reverse, it should
complain, loudly. When it detects any kind of inconsistency in fact.
Here was what I tried to explain. Basically we just agree.

Post by Piotr Ożarowski

Post by Pierre Habouzit
That's why I've not done any proposal yet, because the problem does
not look obvious in the first glance.

I know that it could be hard to understand me (I blame my English skills)
but I think you all understand the problem now and "current" will not
be removed from the policy. My packages are safe for now ;-)

Oh I think "current" will be removed from the policy as it is now.
Having it X?-PV is very confusing. THe current from pycentral can
remains where it is in XS-PV but that's an implementation detail from
pycentral, and won't be normative IMHO.

Post by Piotr Ożarowski
PS I thought it's time to think about merging py{central,support} but I
guess this discussion will end on "current" keyword.

No this discussion is about driving the remaining grey areas of the
policy from the needs that the policy has to fulfill rather than the
implementation details of the py* tools, which has been made for too
long.

Current was a big point of disagreement in the past, hence it has been
central in that discussion. It does not mean that we've forgotten the
rest :)

[Steve Langasek]

Post by Piotr Ożarowski

Post by Steve Langasek
Ugh, it should fail *regardless* of the existence of python2.X-dev. Why
would you ever call it "current" if it's building for something that *isn't*
the current version of python? A package should only be called "python-foo"
if it's built for "python"; if it's built for python2.X explicitly, the
package name needs to reflect that, which means manual changes are needed to
update it for a new python version. That's out of scope for 'current'.

when I write "current" I mean "single" (I didn't choose name for this
keyword)

The name 'current' was chosen deliberately, to refer to building for *only*
the current/default version of python.

Post by Piotr Ożarowski
If package is build for a single Python version and default Python
version is not supported by this package, hashbang has to be set
correctly and modules it provides (byte-)compiled (at build time *and*
during the install/default python version change)

Yes, and then that has nothing to do with current.

Post by Piotr Ożarowski

Post by Steve Langasek

Post by Piotr Ożarowski
BTW: "current, >=2.4" helped me a lot with packaging gaupol when
python2.3 was default

Which is not an arch: any package, so is irrelevant to binNMU support.

I couldn't set "python" in hashbang (as I said before: gaupol will not work
with python2.3). Package was build when python2.3 was default so
hashbang was set to python2.4. Now when python2.3 was removed from
Debian, package needed binNMU (due to wrong hashbang) even if it's
arch:all.

No, it needed an *update* when python2.3 was removed. BinNMUs a) are for
arch: any packages only (this is for a reason), and b) not an appropriate
method for changing things like the python hashbang used in a script, which
is something that should be evaluated on a per-package basis.

Post by Piotr Ożarowski

Post by Steve Langasek
Oh, and if gaupol really needs python 2.4 or better, then the package's
current dependencies are wrong...

python2.4 is default now so there's no need to add extra dependencies

Um, no. Your package is supposed to have a versioned dependency on python
(>= 2.4), and it doesn't.

[Piotr Ożarowski]

[Steve Langasek, 24.03.2007]

Post by Steve Langasek

Post by Piotr Ożarowski
I couldn't set "python" in hashbang (as I said before: gaupol will not work
with python2.3). Package was build when python2.3 was default so
hashbang was set to python2.4. Now when python2.3 was removed from
Debian, package needed binNMU (due to wrong hashbang) even if it's
arch:all.

No, it needed an *update* when python2.3 was removed. BinNMUs a) are for
arch: any packages only (this is for a reason), and b) not an appropriate
method for changing things like the python hashbang used in a script, which
is something that should be evaluated on a per-package basis.

you're right, python2.4 is still available in Debian so update is not
required, but please note that all applications with python2.3 in hashbang
will not work now, even if they're arch:all (so new upload is needed).

Post by Steve Langasek

Post by Piotr Ożarowski

Post by Steve Langasek
Oh, and if gaupol really needs python 2.4 or better, then the package's
current dependencies are wrong...

python2.4 is default now so there's no need to add extra dependencies

Um, no. Your package is supposed to have a versioned dependency on python
(>= 2.4), and it doesn't.

I have replaced "2.4" with "current" when python2.4 became default, but
yes - I will change it to "current, >=2.4" with my next upload (package
depends on python (>=2.4) only during build time now)

[Loïc Minier]

Post by Josselin Mouette
I think it's time to update the python policy with the progress that has
been made in how we build python packages. The proposed diff is
* the deprecation of the "current" keyword;
* making Provides: meaningful in the case of inter-module
dependencies, as discussed at Debconf;
* fixes to the erroneous python-support section.

Looks good; this obviously implies that python-central will need to
match the new dependency functionalities.

[Steve Langasek]

Post by Josselin Mouette
I think it's time to update the python policy with the progress that has
been made in how we build python packages. The proposed diff is
* the deprecation of the "current" keyword;

So with current deprecated, what is the solution for a package which wants
to build a single binary extension for the current python version in a
package named python-foo, with no support for other versions of python
returned by pyversions -s?

Post by Josselin Mouette
* making Provides: meaningful in the case of inter-module
dependencies, as discussed at Debconf;

Do the helpers implement this, or is this going to be a policy that packages
can't actually comply with?

[Josselin Mouette]

Post by Steve Langasek

Post by Josselin Mouette
I think it's time to update the python policy with the progress that has
been made in how we build python packages. The proposed diff is
* the deprecation of the "current" keyword;

So with current deprecated, what is the solution for a package which wants
to build a single binary extension for the current python version in a
package named python-foo, with no support for other versions of python
returned by pyversions -s?

If this is a public extension, this goes completely against the spirit
of the policy and should not be allowed. It just means more packages
having to migrate simultaneously with python-defaults.

Post by Steve Langasek

Post by Josselin Mouette
* making Provides: meaningful in the case of inter-module
dependencies, as discussed at Debconf;

Do the helpers implement this, or is this going to be a policy that packages
can't actually comply with?

python-support 0.6 already implements this.

[Steve Langasek]

Post by Josselin Mouette

Post by Steve Langasek

Post by Josselin Mouette
I think it's time to update the python policy with the progress that has
been made in how we build python packages. The proposed diff is
* the deprecation of the "current" keyword;

So with current deprecated, what is the solution for a package which wants
to build a single binary extension for the current python version in a
package named python-foo, with no support for other versions of python
returned by pyversions -s?

If this is a public extension, this goes completely against the spirit
of the policy and should not be allowed. It just means more packages
having to migrate simultaneously with python-defaults.

It's not against the spirit of what was discussed in Mexico, it's just
against the spirit of what you personally think.

[Josselin Mouette]

Post by Steve Langasek

Post by Josselin Mouette
If this is a public extension, this goes completely against the spirit
of the policy and should not be allowed. It just means more packages
having to migrate simultaneously with python-defaults.

It's not against the spirit of what was discussed in Mexico, it's just
against the spirit of what you personally think.

I'm not the one who suddenly declared all packages complying with the
"old" policy RC-buggy. Do I understand that we can drop all the
multi-version stuff now?

[Steve Langasek]

Post by Josselin Mouette

Post by Steve Langasek

Post by Josselin Mouette
If this is a public extension, this goes completely against the spirit
of the policy and should not be allowed. It just means more packages
having to migrate simultaneously with python-defaults.

It's not against the spirit of what was discussed in Mexico, it's just
against the spirit of what you personally think.

I'm not the one who suddenly declared all packages complying with the
"old" policy RC-buggy.

WTF? Neither did the release team.

Post by Josselin Mouette
Do I understand that we can drop all the multi-version stuff now?

Who's "we"?

[Pierre Habouzit]

Post by Steve Langasek

Post by Josselin Mouette

Post by Steve Langasek

Post by Josselin Mouette
If this is a public extension, this goes completely against the spirit
of the policy and should not be allowed. It just means more packages
having to migrate simultaneously with python-defaults.

It's not against the spirit of what was discussed in Mexico, it's just
against the spirit of what you personally think.

I'm not the one who suddenly declared all packages complying with the
"old" policy RC-buggy.

WTF? Neither did the release team.

Post by Josselin Mouette
Do I understand that we can drop all the multi-version stuff now?

Who's "we"?

Please (and it's meant as much to both of you) that part of the thread
is going nowhere useful. We can avoid this easily, shall we ?

[Piotr Ożarowski]

[Steve Langasek, 21.03.2007]

Post by Steve Langasek
So with current deprecated, what is the solution for a package which wants
to build a single binary extension for the current python version in a
package named python-foo, with no support for other versions of python
returned by pyversions -s?

I think depending on python-dev for current only modules/apps and
python-all-dev for the rest should be enough (if both systems will
recognize it correctly, I mean also: "python-dev(>=2.5)|python2.5-dev" )

[Steve Langasek]

Post by Piotr Ożarowski
[Steve Langasek, 21.03.2007]

Post by Steve Langasek
So with current deprecated, what is the solution for a package which wants
to build a single binary extension for the current python version in a
package named python-foo, with no support for other versions of python
returned by pyversions -s?

I think depending on python-dev for current only modules/apps and
python-all-dev for the rest should be enough (if both systems will
recognize it correctly, I mean also: "python-dev(>=2.5)|python2.5-dev" )

No, this has nothing to do with the question of being able to get the
version number of, and build binary extensions for, the current version of
python.

[Piotr Ożarowski]

[Steve Langasek, 21.03.2007]

Post by Steve Langasek

Post by Piotr Ożarowski
I think depending on python-dev for current only modules/apps and
python-all-dev for the rest should be enough (if both systems will
recognize it correctly, I mean also: "python-dev(>=2.5)|python2.5-dev" )

No, this has nothing to do with the question of being able to get the
version number of, and build binary extensions for, the current version of
python.

how about this:
^^^^^^^^^^^^^^^

case 1: emma - python application that installs private module
Build-Depends: python-dev (>= 2.5) | python2.5-dev
XS-Python-Version: >=2.5

Architecture: all

case 2: emma - python application that installs private module (and lets
say it is providing private python extension as well)
Build-Depends: python-dev (>= 2.4) | python2.4-dev
XS-Python-Version: >=2.4

Architecture: any


case 3: sqlalchemy - python module
Build-Depends: python-all-dev
XS-Python-Version: >=2.3

Architecture: all

case 4: pywavelets - python extension
Build-Depends: python-all-dev
XS-Python-Version: >=2.4

Architecture: any


and I expect python-<system> to:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

case 1:
* compile it for current Python version only (no binNMU needed after
default Python version change, just recompile the module on user's
machine)
* set XB-Python-Version to "current, >2.5" # here "current" can't be deprecated,
but this field should be filled automatically (think ${python:Versions})
so maintainer doesn't have to know about "current"
* change hashbang if needed (and remember about it [also in in XB-Python-Version
probably] - what if Python version from versioned hashbang will be
removed from supported Python versions?)


case 2:
* as above, except binNMU is needed after default python version change, no need
to remember hashbang change


case 3:
* build for all supported python versions (that are >=2.3)
* set XB-Python-Version to ">2.3" (or "2.3-")
* no binNMU needed, just recompile after `pyversions -s` will change

case 4:
* as above, binNMU needed
* XB-Python-Version: >=2.4
* Provides: python2.4-wavelets

[Pierre Habouzit]

Post by Piotr Ożarowski
[Steve Langasek, 21.03.2007]

Post by Steve Langasek

Post by Piotr Ożarowski
I think depending on python-dev for current only modules/apps and
python-all-dev for the rest should be enough (if both systems will
recognize it correctly, I mean also: "python-dev(>=2.5)|python2.5-dev" )

No, this has nothing to do with the question of being able to get the
version number of, and build binary extensions for, the current version of
python.

^^^^^^^^^^^^^^^
case 1: emma - python application that installs private module
Build-Depends: python-dev (>= 2.5) | python2.5-dev
XS-Python-Version: >=2.5
Architecture: all
case 2: emma - python application that installs private module (and lets
say it is providing private python extension as well)
Build-Depends: python-dev (>= 2.4) | python2.4-dev
XS-Python-Version: >=2.4
Architecture: any
case 3: sqlalchemy - python module
Build-Depends: python-all-dev
XS-Python-Version: >=2.3
Architecture: all
case 4: pywavelets - python extension
Build-Depends: python-all-dev
XS-Python-Version: >=2.4
Architecture: any
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* compile it for current Python version only (no binNMU needed after
default Python version change, just recompile the module on user's
machine)

for a pure module or a binary one ?

Post by Piotr Ożarowski
* set XB-Python-Version to "current, >2.5" # here "current" can't be deprecated,
but this field should be filled automatically (think ${python:Versions})
so maintainer doesn't have to know about "current"

current, > 2.5 has no sense at all, at least today, as current is
(today) meaning: please build that for python2.4

Post by Piotr Ożarowski
* change hashbang if needed (and remember about it [also in in XB-Python-Version
probably] - what if Python version from versioned hashbang will be
removed from supported Python versions?)

hashbang should always be /usr/bin/python as soon as the default
version is supported. We implicitely assume that as soon as a X.Y python
is supported, next version will. This is safe for 99% of the packages.


But IMHO the new policy doesn't help you if your package doesn't
support the current default python. I mean, there is obviously tricks in
the debian/rules that are possible to render the package binNMUable, but
I don't think the dh_py* tools can help here. But I may be mistaken.

Post by Piotr Ożarowski
* as above, except binNMU is needed after default python version change, no need
to remember hashbang change

A default python change only requires to binNMU packages that have
private binary modules. _or_ packages with binary extensions that were
only built for the old default version and not others.

So for your example, indeed, you have to build only for $(pyversions -s).

Post by Piotr Ożarowski
* build for all supported python versions (that are >=2.3)
* set XB-Python-Version to ">2.3" (or "2.3-")
* no binNMU needed, just recompile after `pyversions -s` will change

no recompilation is needed in that case, even if pyversions -s
changes.

Post by Piotr Ożarowski
* as above, binNMU needed
* XB-Python-Version: >=2.4
* Provides: python2.4-wavelets

no binNMU is needed here either for a default python change. It is
recommended for a python version removal (to avoid to waste space) and
needed for the introduction of a new python (to support the new one).

[Piotr Ożarowski]

[Pierre Habouzit, 22.03.2007]

Post by Pierre Habouzit

Post by Piotr Ożarowski
[Steve Langasek, 21.03.2007]

Post by Steve Langasek

Post by Piotr Ożarowski
I think depending on python-dev for current only modules/apps and
python-all-dev for the rest should be enough (if both systems will
recognize it correctly, I mean also: "python-dev(>=2.5)|python2.5-dev" )

No, this has nothing to do with the question of being able to get the
version number of, and build binary extensions for, the current version of
python.

^^^^^^^^^^^^^^^
case 1: emma - python application that installs private module
Build-Depends: python-dev (>= 2.5) | python2.5-dev
XS-Python-Version: >=2.5
Architecture: all
case 2: emma - python application that installs private module (and lets
say it is providing private python extension as well)
Build-Depends: python-dev (>= 2.4) | python2.4-dev
XS-Python-Version: >=2.4
Architecture: any
case 3: sqlalchemy - python module
Build-Depends: python-all-dev
XS-Python-Version: >=2.3
Architecture: all
case 4: pywavelets - python extension
Build-Depends: python-all-dev
XS-Python-Version: >=2.4
Architecture: any
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* compile it for current Python version only (no binNMU needed after
default Python version change, just recompile the module on user's
machine)

for a pure module or a binary one ?

I mean just recompile .pyc files (for new default Python version), leave
package untouched

Post by Pierre Habouzit

Post by Piotr Ożarowski
* set XB-Python-Version to "current, >2.5" # here "current" can't be deprecated,
but this field should be filled automatically (think ${python:Versions})
so maintainer doesn't have to know about "current"

current, > 2.5 has no sense at all, at least today, as current is
(today) meaning: please build that for python2.4

How will python-<system> know to recompile it just for one version and not
for all supported ones?

Post by Pierre Habouzit

Post by Piotr Ożarowski
* change hashbang if needed (and remember about it [also in in XB-Python-Version
probably] - what if Python version from versioned hashbang will be
removed from supported Python versions?)

hashbang should always be /usr/bin/python as soon as the default
version is supported. We implicitely assume that as soon as a X.Y python
is supported, next version will. This is safe for 99% of the packages.
But IMHO the new policy doesn't help you if your package doesn't
support the current default python. I mean, there is obviously tricks in
the debian/rules that are possible to render the package binNMUable, but
I don't think the dh_py* tools can help here. But I may be mistaken.

What if my application needs python2.X where X = Y+1 and Y is default
one? (I had this problem with gaupol when python2.3 was default, and
"current, >2.4" worked just fine!)

Post by Pierre Habouzit

Post by Piotr Ożarowski
* build for all supported python versions (that are >=2.3)
* set XB-Python-Version to ">2.3" (or "2.3-")
* no binNMU needed, just recompile after `pyversions -s` will change

no recompilation is needed in that case, even if pyversions -s
changes.

I mean recompile pyc files only of course, not the whole package (as I
said: no binNMU )

Post by Pierre Habouzit

Post by Piotr Ożarowski
* as above, binNMU needed
* XB-Python-Version: >=2.4
* Provides: python2.4-wavelets

no binNMU is needed here either for a default python change. It is
recommended for a python version removal (to avoid to waste space) and
needed for the introduction of a new python (to support the new one).

new supported python version is available, package provides .so files
and you say no binNMU is needed?

[Pierre Habouzit]

Post by Piotr Ożarowski
[Pierre Habouzit, 22.03.2007]

Post by Pierre Habouzit

Post by Piotr Ożarowski
* set XB-Python-Version to "current, >2.5" # here "current" can't be deprecated,
but this field should be filled automatically (think ${python:Versions})
so maintainer doesn't have to know about "current"

current, > 2.5 has no sense at all, at least today, as current is
(today) meaning: please build that for python2.4

How will python-<system> know to recompile it just for one version and not
for all supported ones?

Why would you prevent the user to bytecompile your package for every
python version he choose to install ? I see the point to avoid archive
bloat in not building every binary extension. But locally ? Well, that
seems wrong. Really.

Especially since the dh_py* tools will only byte-compile code for
python versions that are supported _and_ installed on the user machine.
For pure python packages "current" does not makes sense.

Post by Piotr Ożarowski

Post by Pierre Habouzit

Post by Piotr Ożarowski
* change hashbang if needed (and remember about it [also in in XB-Python-Version
probably] - what if Python version from versioned hashbang will be
removed from supported Python versions?)

hashbang should always be /usr/bin/python as soon as the default
version is supported. We implicitely assume that as soon as a X.Y python
is supported, next version will. This is safe for 99% of the packages.
But IMHO the new policy doesn't help you if your package doesn't
support the current default python. I mean, there is obviously tricks in
the debian/rules that are possible to render the package binNMUable, but
I don't think the dh_py* tools can help here. But I may be mistaken.

What if my application needs python2.X where X = Y+1 and Y is default
one? (I had this problem with gaupol when python2.3 was default, and
"current, >2.4" worked just fine!)

then you ask for 2.4- and the dh_* tool will deal with things for the
dependencies. It's up to you to fix the hashbangs properly. But again,
dh_py* tools won't help you a lot for packages that do not _at least_
support pyversions -d. You will need to do some extra work.

But IMHO the whole point of that policy is that a move to a new python
(python2.5 here) can be done very fast, before there is too many
packages that only work with python2.5, meaning that there will be few
packages in that case, and that it will only be a transitionnal
situation for them.

Post by Piotr Ożarowski

Post by Pierre Habouzit

Post by Piotr Ożarowski
* as above, binNMU needed
* XB-Python-Version: >=2.4
* Provides: python2.4-wavelets

no binNMU is needed here either for a default python change. It is
recommended for a python version removal (to avoid to waste space) and
needed for the introduction of a new python (to support the new one).

new supported python version is available, package provides .so files
and you say no binNMU is needed?

You definitely seem to have misread me.

[Ben Finney]

Post by Pierre Habouzit

Post by Piotr Ożarowski
How will python-<system> know to recompile it just for one version
and not for all supported ones?

Why would you prevent the user to bytecompile your package for every
python version he choose to install ? I see the point to avoid archive
bloat in not building every binary extension. But locally ? Well, that
seems wrong. Really.

One reason I can think of: The package fails to build on Python
earlier than a particular version, and the user has Python versions
older than that concurrently installed.
--
\ "Don't worry about what anybody else is going to do. The best |
`\ way to predict the future is to invent it." -- Alan Kay |
_o__) |
Ben Finney

[Josselin Mouette]

Post by Ben Finney

Post by Pierre Habouzit
Why would you prevent the user to bytecompile your package for every
python version he choose to install ? I see the point to avoid archive
bloat in not building every binary extension. But locally ? Well, that
seems wrong. Really.

One reason I can think of: The package fails to build on Python
earlier than a particular version, and the user has Python versions
older than that concurrently installed.

This kind of information is already stored in the binary package, with
XB-Python-Version or /usr/share/python-support/$package/.version.

[Steve Langasek]

Post by Pierre Habouzit

Post by Piotr Ożarowski
* set XB-Python-Version to "current, >2.5" # here "current" can't be deprecated,
but this field should be filled automatically (think ${python:Versions})
so maintainer doesn't have to know about "current"

current, > 2.5 has no sense at all, at least today, as current is
(today) meaning: please build that for python2.4

It would mean that such a package is not buildable today in unstable. Why
is that not a useful declaration to have? Just because a build-dependency
isn't satisfiable doesn't make the build-dep itself useless. (E.g., for an
example strictly within Debian, consider a package uploaded to lenny that
had this declaration as a means of preventing broken backports.)

Post by Pierre Habouzit
But IMHO the new policy doesn't help you if your package doesn't
support the current default python. I mean, there is obviously tricks in
the debian/rules that are possible to render the package binNMUable, but
I don't think the dh_py* tools can help here. But I may be mistaken.

I believe that's correct.

Post by Pierre Habouzit

Post by Piotr Ożarowski
* build for all supported python versions (that are >=2.3)
* set XB-Python-Version to ">2.3" (or "2.3-")
* no binNMU needed, just recompile after `pyversions -s` will change

no recompilation is needed in that case, even if pyversions -s
changes.

As you allude later, ideally you would have binNMUs in response to the
pyversions -s change, so that the package is updated for additions/deletions
to the supported list. The binNMU is not *required*, true, but doing such
binNMUS may be a factor in how smooth the upgrade path is between stable
releases.

[Pierre Habouzit]

Post by Steve Langasek

Post by Josselin Mouette
I think it's time to update the python policy with the progress that has
been made in how we build python packages. The proposed diff is
* the deprecation of the "current" keyword;

So with current deprecated, what is the solution for a package which wants
to build a single binary extension for the current python version in a
package named python-foo, with no support for other versions of python
returned by pyversions -s?

I must say I quite agree with Josselin here. What would be the purpose
of a package like this ? Either we comply with the idea behind the
policy, meaning that a package is built for as many supported python
version we can, either we don't.

If we don't, I don't see the purpose of the policy alltogether. If we
do, then current is quite broken IMHO. I mean, we have basically 3 types
of packages (there is more, but for what I will try to expose, we can
fold things onto those three).

* packages written in pure python: those enter in the scope of the
policy as soon as they support (at least) the default python
version. In that case, well, there is nothing special to do, it
spoils nothing (neither build time nor space) to support every
python version available.

* packages with private binary modules: those (with or without
"current") can only be built for one version of python at a time.
For them current is meaningless. In fact it's even confusing,
because when you look at a package that was built with a pythonX.Y
as default, current meant X.Y at that time. If you look it after
having switched to a pythonX'.Y' the "current" you see has another
meaning (and yes pycentral uses it in XB-P-V, and IMHO it's a bug).

* packages with binary public modules. Those are the one that indeed
may spoil a bit of resources (space on the user hard drive, and time
as we basically build the package twice). Though, if we have
concerns about speed or time for _public_ modules (meaning the very
modules that are meant to be used by the programmer right?) we fail
completely to follow the spirit of the "new" (that is not so new
right now ;p) policy. So here current seems if not broken, nor
useless, at least against the idea of the policy.

Is there anything that I miss with those explanations ?

[Steve Langasek]

Post by Pierre Habouzit

Post by Steve Langasek

Post by Josselin Mouette
I think it's time to update the python policy with the progress that has
been made in how we build python packages. The proposed diff is
* the deprecation of the "current" keyword;

So with current deprecated, what is the solution for a package which wants
to build a single binary extension for the current python version in a
package named python-foo, with no support for other versions of python
returned by pyversions -s?

I must say I quite agree with Josselin here. What would be the purpose
of a package like this ? Either we comply with the idea behind the
policy, meaning that a package is built for as many supported python
version we can, either we don't.

No. That may have been the idea behind *Josselin's* policy, but that was
never "the" idea behind the policy that was being proposed and discussed
early last year on this list.

Post by Pierre Habouzit
If we don't, I don't see the purpose of the policy alltogether.

Allowing transitions between default versions of python without package
renames, bypassing NEW, allowing binNMUable transitions, and generally
simplifying the python upgrade path for users across releases.

Supporting multiple binary extensions in a single python-foo package is a
tool that *facilitates* that goal, but it was *never* supposed to be
mandatory. There are extensions with no/few reverse-dependencies and a
small install base, such that supporting multiple versions of python is
useless archive bloat.

Evidently everyone has lost sight of this as a result of Josselin's process
hijacking. Oh well.

Post by Pierre Habouzit
* packages with binary public modules. Those are the one that indeed
may spoil a bit of resources (space on the user hard drive, and time
as we basically build the package twice). Though, if we have
concerns about speed or time for _public_ modules (meaning the very
modules that are meant to be used by the programmer right?) we fail
completely to follow the spirit of the "new" (that is not so new
right now ;p) policy. So here current seems if not broken, nor
useless, at least against the idea of the policy.

No, it's your ideas of what the policy should be that are broken.

[Pierre Habouzit]

Post by Steve Langasek

Post by Pierre Habouzit
If we don't, I don't see the purpose of the policy alltogether.

Allowing transitions between default versions of python without package
renames, bypassing NEW, allowing binNMUable transitions, and generally
simplifying the python upgrade path for users across releases.
Supporting multiple binary extensions in a single python-foo package is a
tool that *facilitates* that goal, but it was *never* supposed to be
mandatory. There are extensions with no/few reverse-dependencies and a
small install base, such that supporting multiple versions of python is
useless archive bloat.
Evidently everyone has lost sight of this

I see. What does "current" has to do with it then ? in the current
state of affairs, nothing prevent the maintainer to only build the
package for $(pyversions -d) only, which would be:
(1) binNMUable
(2) only built for the current python version as per spec of what you
want to achieve.

I think I don't say anything foolish here, and that current was not a
requirement.

Though, I know what you will argue next, it's that you need (as a RM)
to be able to compute the list of packages needing to be queued for
binNMU. I've not evaluated yet if current helps here or not (I don't
think so, but I've nothing to backup that assertion yet, so I wont say
anything until I've a good and minimal solution to propose).

Post by Steve Langasek
as a result of Josselin's process hijacking. Oh well.

Bleh, I think we can avoid that part :) I don't want to point fingers,
but to find solutions. Really.

[Steve Langasek]

Post by Pierre Habouzit

Post by Steve Langasek

Post by Pierre Habouzit
If we don't, I don't see the purpose of the policy alltogether.

Allowing transitions between default versions of python without package
renames, bypassing NEW, allowing binNMUable transitions, and generally
simplifying the python upgrade path for users across releases.
Supporting multiple binary extensions in a single python-foo package is a
tool that *facilitates* that goal, but it was *never* supposed to be
mandatory. There are extensions with no/few reverse-dependencies and a
small install base, such that supporting multiple versions of python is
useless archive bloat.

I see. What does "current" has to do with it then ? in the current
state of affairs, nothing prevent the maintainer to only build the
(1) binNMUable
(2) only built for the current python version as per spec of what you
want to achieve.

In the original proposal, 'current' was the flag to tell the packaging tools
that pyversions -d *should* be used. There is of course nothing that stops
a maintainer from invoking pyversions -d manually; the question is, how will
doing so fit into the python policy, will there be support for automating
this in the helper tools, and will packages that do this (and are therefore
binNMUable for python transitions) be automatically detectable from the
Sources and/or Packages files?

Post by Pierre Habouzit
Though, I know what you will argue next, it's that you need (as a RM)
to be able to compute the list of packages needing to be queued for
binNMU.

Exactly. :)

Post by Pierre Habouzit
I've not evaluated yet if current helps here or not (I don't think so, but
I've nothing to backup that assertion yet, so I wont say anything until
I've a good and minimal solution to propose).

Ok, I look forward to hearing this proposal.

[Pierre Habouzit]

Post by Steve Langasek

Post by Pierre Habouzit

Post by Steve Langasek

Post by Pierre Habouzit
If we don't, I don't see the purpose of the policy alltogether.

Allowing transitions between default versions of python without package
renames, bypassing NEW, allowing binNMUable transitions, and generally
simplifying the python upgrade path for users across releases.
Supporting multiple binary extensions in a single python-foo package is a
tool that *facilitates* that goal, but it was *never* supposed to be
mandatory. There are extensions with no/few reverse-dependencies and a
small install base, such that supporting multiple versions of python is
useless archive bloat.

I see. What does "current" has to do with it then ? in the current
state of affairs, nothing prevent the maintainer to only build the
(1) binNMUable
(2) only built for the current python version as per spec of what you
want to achieve.

In the original proposal, 'current' was the flag to tell the packaging tools
that pyversions -d *should* be used. There is of course nothing that stops
a maintainer from invoking pyversions -d manually;

Okay I see. As a coder, I don't like it then, and I feel reinforced in
my gut dislike of that field. "current" is declarative, and does not
says anything about what the packager really put in his debian/rules.

If he does not writes what has to be to multi-build the package, and
that he does not puts current, well, the package basically will only be
built for current. As you already acknowledge the reverse situation also
holds.

And as a matter of a fact, maintainers do not seem to understand what
current is for, Piotr python2.5-only package is a very good example of
this IMHO.

Post by Steve Langasek
the question is, how will
doing so fit into the python policy, will there be support for automating
this in the helper tools, and will packages that do this (and are therefore
binNMUable for python transitions) be automatically detectable from the
Sources and/or Packages files?

like I say later, I knew it was what really matters for you. And IMHO
it's really more interesting that the dh_py* tools explains what has
really been built. They already do the job to generate the correct
substs for python:Depends and python:Provides, so basically, the code to
detect that exists.

One just has to make it clear in the Packages.gz files. Like I said, I
don't really think Sources.gz are relevant, as it's declarative from the
maintainer PoV.

But like said, I've not yet thought about all the consequences yet,
and I do not know what's needed or not. I've the suspicion that XB-P-V
could indeed be the way to go (even if for packages with modules
Provides: show that information already), but I'm not sure that the
current use of XB-P-V carries all the information we need.

Post by Steve Langasek

Post by Pierre Habouzit
Though, I know what you will argue next, it's that you need (as a RM)
to be able to compute the list of packages needing to be queued for
binNMU.

Exactly. :)

Post by Pierre Habouzit
I've not evaluated yet if current helps here or not (I don't think so, but
I've nothing to backup that assertion yet, so I wont say anything until
I've a good and minimal solution to propose).

Ok, I look forward to hearing this proposal.

Thanks. I'll try to make this proposal driven from our needs, and not
trying to advantage this or that implementation of the dh_py* helpers,
which was the ground of the argument when I joined this (mess ?) half a
year ago. I think with your explanations, I quite understand the
requirements from the user and packager point of view (NEW, number of
packages efficiency, etc...), and I believe the sole other need is the
ease to deal with transitions from the RM PoV and for that it needs to
answer the 3 questions:

* what should be binNMUed for a python version removal (assuming it
was not default btw ;P), that one is easy IMHO: basically nothing
_needs_ to, but:
+ packages with a strong dependency on that pythonX.Y and _no_
python dependency have to be dropped altogether (or likely need
porting and are not binNMUable) ;
+ for space efficiency we could think of binNMUing every package
that has built a public module for that particular python
version. Here, XB-P-V is not needed if we have the Provides that
is mandatory (and that is what Joss proposes, and I think it's
sane anyway).

* what should be done for a default python change: here concerned
packages are:
+ packages with private binary extensions,
+ packages built only for the "current" python version.

* what should be done for a new python: here concerned packages are
_only_ packages that build public binary modules for more than the
"current" python. Here again, XB-P-V does not seems required if the
Provides is, they provide the same information, except for the "I'm
only built for current and if you binNMU me I won't generate a new
package anyway".

So that first quick look just says that what we need is a way to find
out about packages that:
* build private extensions (needed for question (2));
* only build things for the default python version (opposed to those
who build for as many as possible), knowing that I'm only speaking
of packages with public binary modules here.
(needed for 2, and for 3)


*phew*. I don't think I said anything new, I was just trying to sum up
my thoughts.

[Steve Langasek]

Post by Pierre Habouzit

Post by Steve Langasek
In the original proposal, 'current' was the flag to tell the packaging tools
that pyversions -d *should* be used. There is of course nothing that stops
a maintainer from invoking pyversions -d manually;

Okay I see. As a coder, I don't like it then, and I feel reinforced in
my gut dislike of that field. "current" is declarative, and does not
says anything about what the packager really put in his debian/rules.

That's fine. I'm not wedded to the, ah, "current" implementation; my
concern is that it not be gutted from the policy because of concerns about
the implementation, instead of finding a non-buggy solution.

Post by Pierre Habouzit
If he does not writes what has to be to multi-build the package, and
that he does not puts current, well, the package basically will only be
built for current.

And in that case, do all of the helpers correctly calculate the Provides
based on the contents of the binary packages?

Post by Pierre Habouzit
And as a matter of a fact, maintainers do not seem to understand what
current is for, Piotr python2.5-only package is a very good example of
this IMHO.

Well, information surrounding 'current' is certainly muddled at present, but
I don't think that points to a technical flaw.

Post by Pierre Habouzit

Post by Steve Langasek
the question is, how will
doing so fit into the python policy, will there be support for automating
this in the helper tools, and will packages that do this (and are therefore
binNMUable for python transitions) be automatically detectable from the
Sources and/or Packages files?

like I say later, I knew it was what really matters for you. And IMHO
it's really more interesting that the dh_py* tools explains what has
really been built. They already do the job to generate the correct
substs for python:Depends and python:Provides, so basically, the code to
detect that exists.

Yes, so automation of the package builds themselves is addressed, with or
without 'current'.

Post by Pierre Habouzit
One just has to make it clear in the Packages.gz files. Like I said, I
don't really think Sources.gz are relevant, as it's declarative from the
maintainer PoV.

I can't think of any reason it would be a problem to have this information
in Packages only.

Post by Pierre Habouzit
But like said, I've not yet thought about all the consequences yet,
and I do not know what's needed or not. I've the suspicion that XB-P-V
could indeed be the way to go (even if for packages with modules
Provides: show that information already), but I'm not sure that the
current use of XB-P-V carries all the information we need.

AIUI the current use of XB-P-V, as endorsed by the python policy, indicates
what versions of python the package has been built for, which is not all
that relevant for binNMUs; the same information can already be extracted
(though less easily) from the provides/depends. What is needed is a
description of what will happen to the package if a buildd is *attempted*
against a particular version of python.

Post by Pierre Habouzit
Thanks. I'll try to make this proposal driven from our needs, and not
trying to advantage this or that implementation of the dh_py* helpers,
which was the ground of the argument when I joined this (mess ?) half a
year ago. I think with your explanations, I quite understand the
requirements from the user and packager point of view (NEW, number of
packages efficiency, etc...), and I believe the sole other need is the
ease to deal with transitions from the RM PoV and for that it needs to
* what should be binNMUed for a python version removal (assuming it
was not default btw ;P), that one is easy IMHO: basically nothing
+ packages with a strong dependency on that pythonX.Y and _no_
python dependency have to be dropped altogether (or likely need
porting and are not binNMUable) ;
+ for space efficiency we could think of binNMUing every package
that has built a public module for that particular python
version. Here, XB-P-V is not needed if we have the Provides that
is mandatory (and that is what Joss proposes, and I think it's
sane anyway).

In addition to space efficiency, this should be done to ensure that packages
are in a consistent state at release time, so that we don't risk a security
update significantly changing the contents of one of these packages.

Post by Pierre Habouzit
* what should be done for a default python change: here concerned
+ packages with private binary extensions,
+ packages built only for the "current" python version.

Yes, I believe that's correct.

Post by Pierre Habouzit
* what should be done for a new python: here concerned packages are
_only_ packages that build public binary modules for more than the
"current" python. Here again, XB-P-V does not seems required if the
Provides is, they provide the same information, except for the "I'm
only built for current and if you binNMU me I won't generate a new
package anyway".

Right.

Post by Pierre Habouzit
So that first quick look just says that what we need is a way to find
* build private extensions (needed for question (2));
* only build things for the default python version (opposed to those
who build for as many as possible), knowing that I'm only speaking
of packages with public binary modules here.
(needed for 2, and for 3)

Looks right to me.

[Josselin Mouette]

Post by Steve Langasek

Post by Pierre Habouzit
If we don't, I don't see the purpose of the policy alltogether.

Allowing transitions between default versions of python without package
renames, bypassing NEW, allowing binNMUable transitions, and generally
simplifying the python upgrade path for users across releases.

How does the broken "current" semantics help with any of these goals?

Post by Steve Langasek
Supporting multiple binary extensions in a single python-foo package is a
tool that *facilitates* that goal, but it was *never* supposed to be
mandatory. There are extensions with no/few reverse-dependencies and a
small install base, such that supporting multiple versions of python is
useless archive bloat.

This has absolutely nothing to do with what we are discussing. The
decision to build for one or all python versions belongs to the
maintainer, and the build process should reflect it. But by expressing
"current" instead of the real list of supported python versions, you
completely lose track of which python versions are actually supported by
the source package.

Post by Steve Langasek
Evidently everyone has lost sight of this as a result of Josselin's process
hijacking. Oh well.

I'm interested to know which process I have hijacked. I have not
contributed to python policy changes since the "new policy" madness has
started, and I haven't forced anyone to implement things. I've just
implemented a tool that does things *correctly* and helps other
maintainers in this maelstrom of incorrect documentation and blurry
packaging processes.

Or maybe you're saying that just because it's easier to point someone as
the sole responsible for the current fiasco than to find real, working
solutions.

[Steve Langasek]

Post by Josselin Mouette

Post by Steve Langasek

Post by Pierre Habouzit
If we don't, I don't see the purpose of the policy alltogether.

Allowing transitions between default versions of python without package
renames, bypassing NEW, allowing binNMUable transitions, and generally
simplifying the python upgrade path for users across releases.

How does the broken "current" semantics help with any of these goals?

The question I asked was, what is this policy going to do to support the use
case that 'current' was designed to address?

Your answer appears to be 'nothing'.

That's not acceptable.

Deprecating the 'current' keyword without offering an alternative that's at
least as usable for the intended purpose as 'current' is, is a step in the
wrong direction.

Post by Josselin Mouette

Post by Steve Langasek
Evidently everyone has lost sight of this as a result of Josselin's process
hijacking. Oh well.

I'm interested to know which process I have hijacked. I have not
contributed to python policy changes since the "new policy" madness has
started, and I haven't forced anyone to implement things. I've just
implemented a tool that does things *correctly* and helps other
maintainers in this maelstrom of incorrect documentation and blurry
packaging processes.

You implemented a tool that *ignored* some of the use cases that went into
the initial policy, among them the case for 'current'.

[Josselin Mouette]

Post by Steve Langasek
You implemented a tool that *ignored* some of the use cases that went into
the initial policy, among them the case for 'current'.

This is wrong. Python-support doesn't rely on anything else than what
the maintainer chooses to build. The dh_pysupport script is a drop-in
replacement for the old dh_python (except for the multiple python2.X-foo
binary case), and will work whatever versions are built.

I'm not aware of any use case that python-support can't currently
handle. If you find one, please file a bug report.

[Mikhail Gusarov]

Twas brillig at 19:13:19 21.03.2007 UTC-07 when Steve Langasek did gyre and gimble:

SL> You implemented a tool that *ignored* some of the use cases that went into
SL> the initial policy, among them the case for 'current'.

Please, give us a link to the *written* use cases, so we can map them to tools
and to policy. Not everyone can attend the conferences.

[Josselin Mouette]

While the discussion is still ongoing about the "current" keyword, it
seems that everyone agrees with the other changes which are only loosely
related. Can we proceed with these, until we agree on how "current"
should be replaced?

[Piotr Ożarowski]

[Josselin Mouette, 28.03.2007]

Post by Josselin Mouette
While the discussion is still ongoing about the "current" keyword, it
seems that everyone agrees with the other changes which are only loosely
related. Can we proceed with these, until we agree on how "current"
should be replaced?

IMHO, yes

[Steve Langasek]

Post by Josselin Mouette
While the discussion is still ongoing about the "current" keyword, it
seems that everyone agrees with the other changes which are only loosely
related. Can we proceed with these, until we agree on how "current"
should be replaced?

No objections.

[Pierre Habouzit]

Post by Steve Langasek

Post by Josselin Mouette
While the discussion is still ongoing about the "current" keyword, it
seems that everyone agrees with the other changes which are only loosely
related. Can we proceed with these, until we agree on how "current"
should be replaced?

No objections.

None here either.

wrt the "current" thingie, I may have a proposal ready soon, I just
need to polish the details, and look how "hard" it would be to upgrade
the dh_py* tools to them. Well, I've a hard week of paid work ahead, so
I don't expect to have it ready before next week.

[Floris Bruynooghe]

Hi

Post by Pierre Habouzit
wrt the "current" thingie, I may have a proposal ready soon, I just
need to polish the details, and look how "hard" it would be to upgrade
the dh_py* tools to them. Well, I've a hard week of paid work ahead, so
I don't expect to have it ready before next week.

So is there any sort of consensus about "current" then? How should we
use it -- or avoid it? Or did I completely miss the conclusion of
this discussion?

Cheers
Floris

[Pierre Habouzit]

Post by Floris Bruynooghe
Hi

Post by Pierre Habouzit
wrt the "current" thingie, I may have a proposal ready soon, I just
need to polish the details, and look how "hard" it would be to upgrade
the dh_py* tools to them. Well, I've a hard week of paid work ahead, so
I don't expect to have it ready before next week.

So is there any sort of consensus about "current" then? How should we
use it -- or avoid it? Or did I completely miss the conclusion of
this discussion?

Due to real life events (a son) I've not been able to draft the
proposal I want to make. But basically, "current" semantics as is is
quite broken and we are many to concur. I'll *try* to have something
soon, for a fairly blurry definition of soon.