High Level Discussion about the Future of Django

[orokusaki]

This is a bit abstract, but I'd like to bring up this idea, and
firstly let me say that I don't intend to waste the time of the major
contributors (unless you want to join in of course). I mostly want to
get an idea of what some of the contributors/feature proposers out
there are thinking of, in a sort of fly by the seat of your pants way.

Through reading some of the ideas/problems on this group (including my
own) I've noticed that some tend to be A) too far in the future, B)
too abstract, C) (very important) Backwards incompatible, D) (very
important) Too much architecture changes. The discussions tend to turn
from macro to micro very quickly because of some of the existing
constraints.

2 thoughts came to mind:

1) What if every major element could be re-factored for better
extensibility (and perhaps speed as well) without regard for the
backwards compatibility.
2) Imagine the progress that could be made if the existing code base
was able to be re-factored in one week (impossible of course, but
hypothetically speaking), knowing everything that the developers know
now.

I know neither of those is possible at the moment, but take those two
ideas (rules) in mind, and talk about what you'd add / change / make
better / etc.

Perhaps this is way more 2.0 than even 1.3, but I wanted to get a very
early look through foggy goggles into the future so that I can come up
with ideas. I've been bombarding Russell K M with questions, thoughts,
etc that are just very poorly timed with 1.2 Beta and all, and I want
to step back and really prepare for next time.


2 related questions for anyone who cares to answer:

1) Is anything allowed to become non-backwards compatible during a .x
release? (ie, from 1.2 to 1.3 or 1.4)
2) When will 2.0 development begin?

[Gabriel Hurley]

If you haven't already, take a read through the "internals" section of
the docs. It covers a number of the questions you've asked about
deprecation, what can be changed in minor (1.x) releases, etc.

http://docs.djangoproject.com/en/dev/internals/

They were definitely interesting for me.

Being a long-time django user but only recently a contributor, I'd
imagine any truly breaking changes would have to be targeted for 2.0,
which really is just a "someday" right now from all I've gathered.

Also, scattered throughout the wiki section of the docs are numerous
high-level ideas for future releases. They're hard to find, but
they're in there.

Most of the high-level stuff I'd personally like to see has already
been brought up in GSOC proposals and wiki pages, so I can't add much
to that discussion right now. But hey, that's a good thing, right? It
means I like where Django's (probably) headed!

Once 1.2 gets released, I think we'll all be a lot happier, right? (by
the way, congrats to the core devs on knocking out the last ORM ticket
that was open against 1.2 today! That's awesome!)

All the best,

- Gabriel

Post by orokusaki
This is a bit abstract, but I'd like to bring up this idea, and
firstly let me say that I don't intend to waste the time of the major
contributors (unless you want to join in of course). I mostly want to
get an idea of what some of the contributors/feature proposers out
there are thinking of, in a sort of fly by the seat of your pants way.
Through reading some of the ideas/problems on this group (including my
own) I've noticed that some tend to be A) too far in the future, B)
too abstract, C) (very important) Backwards incompatible, D) (very
important) Too much architecture changes. The discussions tend to turn
from macro to micro very quickly because of some of the existing
constraints.
1) What if every major element could be re-factored for better
extensibility (and perhaps speed as well) without regard for the
backwards compatibility.
2) Imagine the progress that could be made if the existing code base
was able to be re-factored in one week (impossible of course, but
hypothetically speaking), knowing everything that the developers know
now.
I know neither of those is possible at the moment, but take those two
ideas (rules) in mind, and talk about what you'd add / change / make
better / etc.
Perhaps this is way more 2.0 than even 1.3, but I wanted to get a very
early look through foggy goggles into the future so that I can come up
with ideas. I've been bombarding Russell K M with questions, thoughts,
etc that are just very poorly timed with 1.2 Beta and all, and I want
to step back and really prepare for next time.
1) Is anything allowed to become non-backwards compatible during a .x
release? (ie, from 1.2 to 1.3 or 1.4)
2) When will 2.0 development begin?

[Russell Keith-Magee]

Post by orokusaki
This is a bit abstract, but I'd like to bring up this idea, and
firstly let me say that I don't intend to waste the time of the major
contributors (unless you want to join in of course). I mostly want to
get an idea of what some of the contributors/feature proposers out
there are thinking of, in a sort of fly by the seat of your pants way.
Through reading some of the ideas/problems on this group (including my
own) I've noticed that some tend to be A) too far in the future, B)
too abstract, C) (very important) Backwards incompatible, D) (very
important) Too much architecture changes. The discussions tend to turn
from macro to micro very quickly because of some of the existing
constraints.

Welcome to working with a large codebase with legacy users :-)

Post by orokusaki
1) What if every major element could be re-factored for better
extensibility (and perhaps speed as well) without regard for the
backwards compatibility.

... and a Pony!

This is about as big a what-if as you can get. It's a simple fact that
we just can't change code like this. Stability matters.

Post by orokusaki
2) Imagine the progress that could be made if the existing code base
was able to be re-factored in one week (impossible of course, but
hypothetically speaking), knowing everything that the developers know
now.

Honestly, the problem isn't refactoring. I'm not going to claim
Django's internals aren't perfect, but there aren't many areas that
need wholesale refactoring. The issue is mostly supporting old code
and design conventions, such as:
* The eccentricities of handling of arguments in template tags like
{% cycle %} and {% url %}
* The handling of the URL name in the {% url %} tag
* The data validation expectations of data validation on
model.save() and modelform.is_valid()

If we could drop backwards compatibility, we could clean up most of
these problems really quickly. However we can't, so we can't.

Post by orokusaki
I know neither of those is possible at the moment, but take those two
ideas (rules) in mind, and talk about what you'd add / change / make
better / etc.

The "Milestone 2.0" tag exists in Trac for logging these ideas.

Post by orokusaki
Perhaps this is way more 2.0 than even 1.3, but I wanted to get a very
early look through foggy goggles into the future so that I can come up
with ideas. I've been bombarding Russell K M with questions, thoughts,
etc that are just very poorly timed with 1.2 Beta and all, and I want
to step back and really prepare for next time.

I have two problems with this sort of discussion:

* The timing. As you note, we're trying to get 1.2 out the door; any
discussion that isn't specifically 1.2 related just distracts from the
release effort. Once 1.2 is done, we'll have plenty of time for
discussions of new design.

* The abstract. It's very easy to get stuck into exceedingly abstract
discussions of what might be, and what could be. Again, these are
mostly distractions from Getting Things Done. Having a conreI'm more
than happy to discuss *specific* proposals (when the time is right);
I'd rather avoid the abstract ones.

Post by orokusaki
1) Is anything allowed to become non-backwards compatible during a .x
release? (ie, from 1.2 to 1.3 or 1.4)

No with and if; yes with a but. :-)

The golden rule is that code written against Django 1.X *must* work in
Django 1.(X+1) without any modifications at all.

However, we will allow changes that are backwards incompatible to be
introduced if:

1) The change is required to solve a major bug or security problem
that cannot be solved any other way (and when I say major bug, I mean
*major*. Demonstrated catastrophic data loss under easily reproducible
conditions, for example).

OR

2) The change can be introduced gradually over multiple releases.

For example, the changes to the way you include admin URLs that were
made in Django 1.1 are backwards incompatible. However, if you're
using the Django 1.0 way under Django 1.1, you get a (silent)
PendingDeprecationWarning. If you use the same code in Django 1.2, you
get a (loud) DeprecationWarning. In Django 1.3, that code will stop
working. This means that users get 2 full releases (approx 18-24
months) in which the changes required are well known and making
themselves known.

Post by orokusaki
2) When will 2.0 development begin?

When it needs to begin :-)

Seriously, we don't have any concrete plans regarding 2.0 at this
point. Django 2.0 is in the same category as Python 3000 was a few
years ago - a notional point in the future at which backwards
compatibility will be broken in order to solve a number of well known
problems, but with no concrete plan until we reach a point where we
think we're ready. One day, one of the core devs will say "So, how
about we start on Django 2.0?", and it will become a concrete target.

If I had to guess, I'd say that moving to official support for Python
3 will be the trigger for starting the Django 2.0 discussion. There
will be more to Django 2.0 than just updating Python support, but if
you're going to break backwards compatibility at a language level, you
might as well take the opportunity to fix all the other niggling bits
at the same time.

However, we can't seriously start talking about Python 3 until:

* all the downstream vendors (DB-API implementations, mod_wsgi, etc)
have viable Python 3 implementations, and

* There is a viable ecosystem of OS vendors that are providing Python
3 environments (and hosting companies supporting these OSes).

For some perspective - even though Python 3.1 is out, dropping support
for Python 2.3 in Django 1.2 is being greeted as controversial in some
circles because RedHat Enterprise Linux 5 is still officially
supported by RedHat, and RHEL5 ships with Python 2.3.

If we dropped support for 1 Python release with each Django release,
we need to have 3 Django release cycles (dropping Python 2.4, 2.5, 2.6
respectively) before we will be in a position to start encouraging
Python 3 adoption. This means we're *at least* 2-3 years off having a
serious Django 2.0 discussion -- and that's assuming we deprecate
Python releases at the rate of 1 per Django version. I won't be the
least bit surprised if it takes longer than that.

Yours,
Russ Magee %-)

[Dennis Kaarsemaker]

Post by Russell Keith-Magee
For some perspective - even though Python 3.1 is out, dropping support
for Python 2.3 in Django 1.2 is being greeted as controversial in some
circles because RedHat Enterprise Linux 5 is still officially
supported by RedHat, and RHEL5 ships with Python 2.3.

Rhel 5 ships with 2.4, rhel 4 shipped with 2.3. I still have to use
django on the latter, so the support for 2.3 being dropped is an issue
for me. Then again, rhel 4 is positively ancient and I really should
move on :-)
--
Dennis K.

They've gone to plaid!

[Jerome Leclanche]

If you're going to use such an ancient version of a distribution, you
are only crippling yourself. As you said yourself, you should move on;
if someone is using Python 2.3, they can use Django 1.1/1.2. If they
want all-new 1.3 features, then updating Python/distro should not be a
roadblock.

This is a common issue in software backwards compatibility, and I'm at
least one to think that just because someone, somewhere still uses an
old version of python, they can't also keep using an old version of
Django.


J. Leclanche / Adys



On Mon, Apr 5, 2010 at 9:55 AM, Dennis Kaarsemaker

Post by Dennis Kaarsemaker

Post by Russell Keith-Magee
For some perspective - even though Python 3.1 is out, dropping support
for Python 2.3 in Django 1.2 is being greeted as controversial in some
circles because RedHat Enterprise Linux 5 is still officially
supported by RedHat, and RHEL5 ships with Python 2.3.

Rhel 5 ships with 2.4, rhel 4 shipped with 2.3. I still have to use
django on the latter, so the support for 2.3 being dropped is an issue
for me. Then again, rhel 4 is positively ancient and I really should
move on :-)
--
Dennis K.
They've gone to plaid!
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Russell Keith-Magee]

Post by Jerome Leclanche
If you're going to use such an ancient version of a distribution, you
are only crippling yourself. As you said yourself, you should move on;
if someone is using Python 2.3, they can use Django 1.1/1.2. If they
want all-new 1.3 features, then updating Python/distro should not be a
roadblock.
This is a common issue in software backwards compatibility, and I'm at
least one to think that just because someone, somewhere still uses an
old version of python, they can't also keep using an old version of
Django.

I agree -- to an extent.

I have no problem with the argument that you can't expect to be able
to walk the leading edge and the trailing edge at the same time.

That said, the problem here is that the users that are most affected
by backwards incompatibilities and version deprecations are the users
with the most economic clout - "enterprise" users. For better or
worse, large organizations have inertia when it comes to adopting core
infrastructure, and it can be difficult to get this core
infrastructure updated. However, large organizations also have the
biggest potential to grow the adoption of a framework like Django and
move it into the mainstream. This is especially important while we're
adding features that are of the most interest to enterprise users,
like multiple database support.

While we don't want to completely hamstring development of Django by
requiring support for ancient Python versions, we also don't want to
limit the adoption of Django by large organizations simply by making
arbitrary demands on core infrastructure, or by breaking backwards
incompatibility of core features.

It's a fine line we have to walk between being an innovative framework
and supporting the users that have helped us get to where we are.

Yours,
Russ Magee %-)

[Jerome Leclanche]

The Right Solution for that is officially supporting Python 2.old in
Django 1.old, and eventually backporting minor features/fixes in
Django 1.old. The tradeoff here depends on what takes the most
development time: Backporting features and fixes, or hacking
compatibility with an old version of the language.

With a solid, automatized regression suite; good version/branch
management (when is Django moving over to git?) and a Generally Good
Codebase, this is never an issue and I don't believe it would be an
issue with Django.

While I know Django 1.0 is still somewhat supported, focusing efforts
on such management also enables developers to tackle another issue
raised earlier here. Upcoming stable releases should NEVER cripple
development of future features and other changes that won't make such
a release. Unfortunately, it's what happened in 1.1 and what is
currently happening.

J. Leclanche / Adys



On Mon, Apr 5, 2010 at 10:48 AM, Russell Keith-Magee

Post by Jerome Leclanche
If you're going to use such an ancient version of a distribution, you
are only crippling yourself. As you said yourself, you should move on;
if someone is using Python 2.3, they can use Django 1.1/1.2. If they
want all-new 1.3 features, then updating Python/distro should not be a
roadblock.
This is a common issue in software backwards compatibility, and I'm at
least one to think that just because someone, somewhere still uses an
old version of python, they can't also keep using an old version of
Django.

I agree --  to an extent.
I have no problem with the argument that you can't expect to be able
to walk the leading edge and the trailing edge at the same time.
That said, the problem here is that the users that are most affected
by backwards incompatibilities and version deprecations are the users
with the most economic clout - "enterprise" users. For better or
worse, large organizations have inertia when it comes to adopting core
infrastructure, and it can be difficult to get this core
infrastructure updated. However, large organizations also have the
biggest potential to grow the adoption of a framework like Django and
move it into the mainstream. This is especially important while we're
adding features that are of the most interest to enterprise users,
like multiple database support.
While we don't want to completely hamstring development of Django by
requiring support for ancient Python versions, we also don't want to
limit the adoption of Django by large organizations simply by making
arbitrary demands on core infrastructure, or by breaking backwards
incompatibility of core features.
It's a fine line we have to walk between being an innovative framework
and supporting the users that have helped us get to where we are.
Yours,
Russ Magee %-)
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Russell Keith-Magee]

Post by Jerome Leclanche
The Right Solution for that is officially supporting Python 2.old in
Django 1.old, and eventually backporting minor features/fixes in
Django 1.old. The tradeoff here depends on what takes the most
development time: Backporting features and fixes, or hacking
compatibility with an old version of the language.

I must be missing something - How exactly does this vary from Django's
current policy?

Django 1.1 supported Python 2.3+ at time of release. It still does.

Django 1.2 will require Python 2.4+. Eventually, we will drop support
for Python 2.4, but Django 1.2 will always support Python 2.4+.

While we're developing Django 1.2, any bugfix is backported to Django
1.1. If that means rewriting or modifying the patch to suit Python
2.3, then that's what we do.

How does this differ from what you call "the Right Solution"?

Post by Jerome Leclanche
With a solid, automatized regression suite;

It's a pity we don't have one of those... oh wait... we do.

Post by Jerome Leclanche
good version/branch
management (when is Django moving over to git?)

Let's get this straight right now. Moving to git will change *exactly
nothing* in Django's development process. There will *always* be an
"official" repository. Commit access to that core repository will be
limited. It doesn't matter whether that repository is git, svn, or hg
- there will always be a single, canonical, exclusive repository for
trunk.

If you want to use Git to help you manage your own source code or your
contributions to Django, you're welcome to do so. There are well
maintained git mirrors that you can use as a starting point. There are
many people in the Django community using this mirror to manage their
own development work. Allow me to assure you that if you have a git
branch that is worth pulling, someone in the Django core will be able
to pull that branch and merge it into SVN trunk without *any*
difficulty. Multi-db, for instance, was almost entirely developed by
Alex Gaynor on his github branch.

And this is completely independent of the bike shed argument of which
DVCS we will adopt. This isn't even remotely a settled argument. By
making an official move to git, we would alienate a large community
that, for better or worse, can't move their production platforms to
git. It also alienates the hg and bzr communities, which is a not
insignificant consideration (for the record, there are also well
maintained hg and bzr mirrors).

Post by Jerome Leclanche
and a Generally Good
Codebase, this is never an issue and I don't believe it would be an
issue with Django.

I'm still not sure what you think the "issue" is. I certainly can't
see how it's related to the original discussion point.

Post by Jerome Leclanche
While I know Django 1.0 is still somewhat supported,

Somewhat? The exact level of support for Django versions is quite well
documented.

Djangp 1.0 is in security support mode. The only thing that will cause
the release or checkin to the 1.0.X branch is the discovery of a
security fault.

Django 1.1 is in support mode. This means that any modification to
trunk that involves a bug fix will be backported to 1.1.

Post by Jerome Leclanche
focusing efforts
on such management also enables developers to tackle another issue
raised earlier here. Upcoming stable releases should NEVER cripple
development of future features and other changes that won't make such
a release. Unfortunately, it's what happened in 1.1 and what is
currently happening.

I presume that you're referring to the regular calls to "wait until
the 1.X development period". There's a world of difference between
saying "Please, can we concentrate on finishing a release", and us
"crippling the development of future features".

Independent development is in no way hampered by this policy. If you
have an itch, you're welcome to scratch it at any time you like.
However, if you expect the core team to volunteer to help you scratch
that itch, then I'm afraid you're going to have a little bit of
consideration for the fact that *your* priorities aren't necessarily
shared by the core team, or by the community as a whole.

We need to put out formal releases. We can't put out formal releases
without killing bugs in features that have been added. We have limited
resources, so for a short period in every development cycle, when
we're trying to get a release out the door, we try to focus on fixing
the problems we already have, rather than working out how to add new
features. If we spend all our resources in the development of new
features, we won't *ever* get a release out the door.

I don't think it's entirely unreasonable that for 2-3 months out of 9,
we ask people to concentrating on fixing what's already been written.
I challenge you to provide any proof that demonstrates that this
policy has "crippled the development of future features".

Yours,
Russ Magee %-)

[Russell Keith-Magee]

On Mon, Apr 5, 2010 at 2:55 PM, Dennis Kaarsemaker

Post by Dennis Kaarsemaker

Post by Russell Keith-Magee
For some perspective - even though Python 3.1 is out, dropping support
for Python 2.3 in Django 1.2 is being greeted as controversial in some
circles because RedHat Enterprise Linux 5 is still officially
supported by RedHat, and RHEL5 ships with Python 2.3.

Rhel 5 ships with 2.4, rhel 4 shipped with 2.3. I still have to use
django on the latter, so the support for 2.3 being dropped is an issue
for me. Then again, rhel 4 is positively ancient and I really should
move on :-)

Sorry - you're completely right - I got my versions confused.

Yours,
Russ Magee %-)

[Graham Dumpleton]

 * all the downstream vendors (DB-API implementations, mod_wsgi, etc)
have viable Python 3 implementations, and

Hmmm, mod_wsgi has had working Python 3.0 support for over a year. I
even did a quick test of Martin's original go at porting Django to
Python 3.0 and at least got the home page working.

I accept though that that is but one required bit and there is a lot
more other important stuff besides it.

:-)

Graham

[Jacob Kaplan-Moss]

This is a bit abstract, but I'd like to bring up this idea, [...]

Well, I'm sorry, but I just don't have time to engage on big abstract
discussions like this, so feel free to write whatever you want, but
don't count on my participation. I'm also going to suggest the other
core Django developers similarly ignore this discussion, though I see
you've managed to get Russ to engage.

I do appreciate your enthusiasm, but you seem to have forgotten a
basic tenant of open source: there's one -- and only one -- way of
getting your way:

Write code.

Show me code, and I'll pay attention to your proposals. Show me big
crazy abstract "what ifs" and I, well, won't.

Jacob

[Jerome Leclanche]

Without trying to defend anyone or anything here... Why ask other
developers to ignore an otherwise healthy discussion?
I believe Russ engaged in the discussion because he's interested; if
not in the idea, at least in discussing it. Not everything has to be
backed up with code...

J. Leclanche / Adys

Post by Jacob Kaplan-Moss

This is a bit abstract, but I'd like to bring up this idea, [...]

Well, I'm sorry, but I just don't have time to engage on big abstract
discussions like this, so feel free to write whatever you want, but
don't count on my participation. I'm also going to suggest the other
core Django developers similarly ignore this discussion, though I see
you've managed to get Russ to engage.
I do appreciate your enthusiasm, but you seem to have forgotten a
basic tenant of open source: there's one -- and only one -- way of
Write code.
Show me code, and I'll pay attention to your proposals. Show me big
crazy abstract "what ifs" and I, well, won't.
Jacob
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Russell Keith-Magee]

Post by Jerome Leclanche
Without trying to defend anyone or anything here... Why ask other
developers to ignore an otherwise healthy discussion?
I believe Russ engaged in the discussion because he's interested; if
not in the idea, at least in discussing it.

Actually, if you read my original post, you'll see that I'm not, and I
said exactly that. I have *zero* interest in abstract discussions,
*especially* when we're trying to get 1.2 out the door.

The bit that I have been engaging with is the discussion of (and
apparent misconceptions around) Django's backwards compatibility
policy, and our policies regarding support for older Python versions.
However, even these discussions have limited interest for me unless
they rapidly converge on a *specific* criticism or problem that
requires rectification.

Yours,
Russ Magee %-)

[Dennis Kaarsemaker]

Post by Russell Keith-Magee
The bit that I have been engaging with is the discussion of (and
apparent misconceptions around) Django's backwards compatibility
policy, and our policies regarding support for older Python versions.

And I appreciate that you have done so, thanks!

Post by Russell Keith-Magee
However, even these discussions have limited interest for me unless
they rapidly converge on a *specific* criticism or problem that
requires rectification.

Not a criticism per se, but I am wondering why the next 1.1.x is
released alongside 1.2 instead of as a release on its own. I've yet
again seen a case of python 2.6.5 breaking django tests, so I would
welcome a new release of 1.1.x a bit sooner than 1.2, if only from a
#django support perspective.
--
Dennis K.

They've gone to plaid!

[Karen Tracey]

On Mon, Apr 5, 2010 at 10:35 AM, Dennis Kaarsemaker

Post by Dennis Kaarsemaker
Not a criticism per se, but I am wondering why the next 1.1.x is
released alongside 1.2 instead of as a release on its own. I've yet
again seen a case of python 2.6.5 breaking django tests, so I would
welcome a new release of 1.1.x a bit sooner than 1.2, if only from a
#django support perspective.

The problem is if we release Django 1.1.2 now, we still need another Django
1.1.X release simultaneously with 1.2 to package in an official release all
remaining bug fixes backported to the 1.1.X branch before 1.2 is finalized.
That would make for two very close together 1.1.X releases, which is
generally not appreciated by users, though I realize it would be appreciated
by early users of Python 2.6.5. Unfortunately the timing of Python 2.6.5 and
Django 1.2 is a bit unfortunate here...for those who are quickly moving to
Python 2.6.5 the best thing to do is run with a checkout of the 1.1.X
branch.

Karen

[Russell Keith-Magee]

On Mon, Apr 5, 2010 at 10:35 PM, Dennis Kaarsemaker

Post by Dennis Kaarsemaker

Post by Russell Keith-Magee
The bit that I have been engaging with is the discussion of (and
apparent misconceptions around) Django's backwards compatibility
policy, and our policies regarding support for older Python versions.

And I appreciate that you have done so, thanks!

Post by Russell Keith-Magee
However, even these discussions have limited interest for me unless
they rapidly converge on a *specific* criticism or problem that
requires rectification.

Not a criticism per se, but I am wondering why the next 1.1.x is
released alongside 1.2 instead of as a release on its own. I've yet
again seen a case of python 2.6.5 breaking django tests, so I would
welcome a new release of 1.1.x a bit sooner than 1.2, if only from a
#django support perspective.

The release of 1.2 will be the point at which 1.1 moves into security
maintenance mode, so it makes sense that we cut both 1.2 and a 1.1.X
release at the same time (to make sure 1.1.X contains as many bug
fixes as possible before it moves into maintenance mode).

So, there will always be *a* 1.1.X release when we release 1.2 - the
only question is whether we release (or at this point, should have
released) an interim 1.1.2, and then make 1.1.3 the 'final' 1.1
release? The problem here is that there hasn't been a single time
where cutting 1.1.2 was both appropriate, and the importance of
cutting an interim release was known.

If you take the 1.1 development cycle as a guide, we cut a 1.0 point
release when we released 1.1-beta1. However, at the time of 1.2-beta1,
the 1.1.X branch didn't actually contain all that much, so there
wasn't much point cutting a 1.1.2 release. If we'd cut a release then,
It wouldn't have included the fix for the Python 2.6.5 problems you
are talking about.

Another useful trigger might have been the point at which we passed
our original RC date (early March). If we'd cut 1.1.2 then, we would
have fixed the 1.1.2 problems - but here, we hit a communication
problem. While I was aware that the Python 2.6.5 had problems with
1.1.1 (I fixed the bug, after all), I wasn't really aware that this
was a problem in practice for a significant body of users until about
a week or so ago.

This isn't a blame thing - I'm just pointing out that if there is a
recurring problem on IRC support (or django-users for that matter),
that doesn't necessarily mean that the core team are also aware of the
problem. And even if we are aware of the problem, it doesn't mean we
are aware of the scope of the problem, and have made plans to address
it.

Django has a huge community, and the core team can't be everywhere all
the time. We really do rely on the active members of our community to
help us identify problems as they emerge. If you're doing your share
of the heavy lifting in #django or django-users (and I know you are,
Dennis, so a big thanks for that), we're certainly interested in
hearing anything that you think will make your life easier. A quick
message to django-dev explaining the problem will sometimes be all it
takes to set the wheels in motion.

So that brings us to the present day -- at this point, we're hopefully
just a week or two away from RC, so there isn't much incentive to turn
the handle and produce a release, just to turn it again in a couple of
weeks for a second release that will only contain minor differences. I
know it's a dangerous to start singing 'tomorrow is only a day away',
but the time we spend cutting a release only serves to delay the final
release of 1.2 a little bit more.

I'll freely admit that despite the major improvements landing in 1.2,
the development cycle itself hasn't been flawless. Hopefully I've been
able to provide some explanation for why things ended up the way they
did.

So - tl;dr:

* Yes, with hindsight, we probably should have cut a 1.1.2 release
* We probably should have cut that release somewhere around the start of March
* We're close enough to 1.2 that we're not going to cut a 1.1.X
release until 1.2-final
* Direct feedback from the #django and django-users trenches might
have avoided this problem
* We'll try to do better next time.

Yours,
Russ Magee %-)

[Dennis Kaarsemaker]

Post by Russell Keith-Magee
I'll freely admit that despite the major improvements landing in 1.2,
the development cycle itself hasn't been flawless. Hopefully I've been
able to provide some explanation for why things ended up the way they
did.

You have, thank you very much.

Post by Russell Keith-Magee
* Yes, with hindsight, we probably should have cut a 1.1.2 release
* We probably should have cut that release somewhere around the start of March

Agreed.

Post by Russell Keith-Magee
* We're close enough to 1.2 that we're not going to cut a 1.1.X
release until 1.2-final

Agreed again.

Post by Russell Keith-Magee
* Direct feedback from the #django and django-users trenches might
have avoided this problem
* We'll try to do better next time.

I will try to do better as well, bringing up often-reported problems on
the -developers mailing list.
--
Dennis K.

They've gone to plaid!

[orokusaki]

@Everyone who has commented here.

I never intended to cause any animosity and I really appreciate
everything that the core team does for us all.

@Jacob

I really do intend to write code. I make money doing non-Django
development but I love Django so much that I spend 25+ hours a week
not getting paid just so I can learn to adopt Django and eventually
replace my income with Django development. I think I might even be
part pony (DNA testing underway). There are just certain elements of
Django that I feel (slightly) holds back a developer of large multi-
tenant architecture, which could be a really big way to get major
enterprises on-board, and this is where I intend to contribute the
most. I just don't have the time (or money after Uncle Sam's 30k this
March) to spend on contributing if I'm not sure my ideas will be
accepted. This is why I want to spend a little time getting the 5,000
foot view so that my ideas are in sync with what users want, what the
core team will accept and what works best with ``from future import
*``.

If I spend 100% more time on preparing to save 50% of my time, I'll be
happy because I've learned more.

@Russell

Thanks for indulging in conversation, if just for the meta thoughts,
this is precisely the sort of macro conversation that helps me
understand the Django process's rationale.

[ShawnMilo]

Post by Jerome Leclanche
Without trying to defend anyone or anything here... Why ask other
developers to ignore an otherwise healthy discussion?
I believe Russ engaged in the discussion because he's interested; if
not in the idea, at least in discussing it. Not everything has to be
backed up with code...
J. Leclanche / Adys

I don't know Jacob, and I'm not a Django contributor of any kind (at least not yet). So this is entirely my own opinion. What I'm suspect the problem is is that, although the discussion is just jabber among some random people on a mailing list, the topic of discussion is ultimately what those random people believe the core contributors ought to be spending their time on -- if not now, then in the future.

So if some people want to engage in this conversation, then that's fine. But there's very little chance of it having any effect on Django unless it:

1. Is inline with the well-established strategies in place for making changes (especially backwards-incompatible changes).
2. Someone is willing to step up and do the work.

It's not reasonable to expect the creators and core committers to ever have the time and energy to do something like what is being discussed here. However, if you have the time and energy and put in some work, you have a foot in the door to start a real conversation regarding getting your changes merged into trunk at some point.

I think that it's good to remember, every once in a while, that Django is not a commercial product. It's all well and good to bitch that Apple's iPad doesn't have a camera, or that Windows 7 has this or that flaw, or that your car's gas mileage sucks. But an all-volunteer open-source project is completely different. Just because certain individuals have stepped forward and shown that they can shoulder the burden of doing regular releases, maintaining old releases for security, and generally being awesome doesn't mean they work for us. It's like us trying to tell the best, smartest, and most active contributors to this mailing list how they could do a better job helping us. It's a disincentive to them, and makes us look like ingrates.

Shawn

[veena]

Thanks for opening this discussion.

Am I only one who see the django improvement process too slow? I mean
refactoring, decoupling and making code more reusable in the time when
we realize that previous design has too much constraints and there
should be better design now.

I know there's django deprecation policy nicely documented
http://docs.djangoproject.com/en/1.1/internals/release-process/#internal-release-deprecation-policy

But what I don't know is how you discover it. Is it described
somewhere in the text or the video from conference? What were the
reasons to have this deprecation policy? Was there any user research?
Research of when the django users upgrade, what are the main problems
of upgrades and how they imagine upgrading should work?

I saw in the discussion in this thread
http://groups.google.cz/group/django-developers/browse_thread/thread/a89935f2f9f9102b
that Russell wrote:

"Put it this way - If part of your migration plan involves every one
user of Django carefully reading the and following the upgrade
instructions, then I want your home phone number so I can give it to
everyone that contacts me personally complaining about their broken
Django installs. And be warned -- I'm in a timezone that guarantees
you will be called at three in the morning. :-)"

I think this problem could be solved by other ways than setting the
strong deprecation policy. (Eg. more visible of release changes, not
providing support for free or get out the "unaware" users out of the
community just by the "faster" upgrade policy).


What I try to say is that I'm little bit afraid that it seems like
improvements of django will take years instead of months. In the time
when Rails 3 is coming it looks like the next killer framework in
terms of easy-to-use and also power and extendability. The django core
devs are doing great job but it's sure there's and still will be a lot
of new use cases for django in the various web applications discovered
by community. I think this could lead to fork the django by some devs
and rapid development of this fork. Maybe then in one moment there
would rise the discussion about merging these two and it will take
year (as in Merb & Rails case). Is this the best way of development
OS?


Vaclav

Post by orokusaki
This is a bit abstract, but I'd like to bring up this idea, and
firstly let me say that I don't intend to waste the time of the major
contributors (unless you want to join in of course). I mostly want to
get an idea of what some of the contributors/feature proposers out
there are thinking of, in a sort of fly by the seat of your pants way.
Through reading some of the ideas/problems on this group (including my
own) I've noticed that some tend to be A) too far in the future, B)
too abstract, C) (very important) Backwards incompatible, D) (very
important) Too much architecture changes. The discussions tend to turn
from macro to micro very quickly because of some of the existing
constraints.
1) What if every major element could be re-factored for better
extensibility (and perhaps speed as well) without regard for the
backwards compatibility.
2) Imagine the progress that could be made if the existing code base
was able to be re-factored in one week (impossible of course, but
hypothetically speaking), knowing everything that the developers know
now.
I know neither of those is possible at the moment, but take those two
ideas (rules) in mind, and talk about what you'd add / change / make
better / etc.
Perhaps this is way more 2.0 than even 1.3, but I wanted to get a very
early look through foggy goggles into the future so that I can come up
with ideas. I've been bombarding Russell K M with questions, thoughts,
etc that are just very poorly timed with 1.2 Beta and all, and I want
to step back and really prepare for next time.
1) Is anything allowed to become non-backwards compatible during a .x
release? (ie, from 1.2 to 1.3 or 1.4)
2) When will 2.0 development begin?

[Russell Keith-Magee]

Post by veena
I know there's django deprecation policy nicely documented
http://docs.djangoproject.com/en/1.1/internals/release-process/#internal-release-deprecation-policy
But what I don't know is how you discover it. Is it described
somewhere in the text or the video from conference? What were the
reasons to have this deprecation policy? Was there any user research?
Research of when the django users upgrade, what are the main problems
of upgrades and how they imagine upgrading should work?

The policy was arrived at after a debate between the core team, based
on how the core team believe a well-behaved project should behave. For
the record, it wasn't much of a debate, either - we were all pretty
much in agreement on the core points from the beginning.

In the opinion of the core, well-behaved projects don't require
massive rewrites (or worse - subtle bug chasing efforts) every time a
new release is made. Developers using a library should have the
confidence to know that when they write code, it will continue to work
for a long time, not just until the core developers have a change of
heart.

I would suggest to you that one of the reasons for Django's success
has been it's policy on backwards compatibility.

Post by veena
What I try to say is that I'm little bit afraid that it seems like
improvements of django will take years instead of months.

...

Post by veena
I think this could lead to fork the django by some devs
and rapid development of this fork

You seem to be suggesting that a fork will somehow magically fix the
speed of Django development. I ask you: who is going to work on this
fork?

Progress on Django may be slower than many would like, but it's not
slow because we're hampered by backwards compatibility. It's because
the core team all have full time jobs, families and friends, and we
contribute to Django in our spare time. If you want to fix the speed
of development, pitch in.

Yours,
Russ Magee %-)

[Kevin Howerton]

"You seem to be suggesting that a fork will somehow magically fix the
speed of Django development. I ask you: who is going to work on this
fork?"

I think a hostile fork is almost a certain outcome if development
continues as it has. Not only is the resistance to make backwards
incompatible changes in future releases a bad policy for the quality
of the framework, but the behavior in trac has a negative effect on
community contributions.

The level of resistance I see to change or outsider code contribution
is an enormous de-motivator for people (like me) to want to make any
contributions in the first place. Why should I contribute a patch to
your flawed architecture if I'm going to be talked down to, ridiculed,
then eventually have the patch rejected because it breaks code in some
edge-use-case?

Personally I believe my time might be better spent developing a fork,
than arguing over clear flaws in architecture decisions that "can't be
changed".

It's a good idea to avoid breaking backwards compatibility in point
releases, but as far as major releases go ... I whole heartedly
encourage it. If keeping backwards compatibility was a good idea, my
macbook pro would have a 5.12" floppy drive, a 3.25" drive, a jazz
drive, it would accept serial and adb plugs.... and maybe have a
display mode to emulate those green crt monitors of yore.

Good software is about removing flaws, improving the architecture, and
learning from past mistakes. If this comes at a price, then pay.

-k

On Wed, Apr 14, 2010 at 9:25 AM, Russell Keith-Magee

Post by Russell Keith-Magee

Post by veena
I know there's django deprecation policy nicely documented
http://docs.djangoproject.com/en/1.1/internals/release-process/#internal-release-deprecation-policy
But what I don't know is how you discover it. Is it described
somewhere in the text or the video from conference? What were the
reasons to have this deprecation policy? Was there any user research?
Research of when the django users upgrade, what are the main problems
of upgrades and how they imagine upgrading should work?

The policy was arrived at after a debate between the core team, based
on how the core team believe a well-behaved project should behave. For
the record, it wasn't much of a debate, either - we were all pretty
much in agreement on the core points from the beginning.
In the opinion of the core, well-behaved projects don't require
massive rewrites (or worse - subtle bug chasing efforts) every time a
new release is made. Developers using a library should have the
confidence to know that when they write code, it will continue to work
for a long time, not just until the core developers have a change of
heart.
I would suggest to you that one of the reasons for Django's success
has been it's policy on backwards compatibility.

Post by veena
What I try to say is that I'm little bit afraid that it seems like
improvements of django will take years instead of months.

...

Post by veena
I think this could lead to fork the django by some devs
and rapid development of this fork

You seem to be suggesting that a fork will somehow magically fix the
speed of Django development. I ask you: who is going to work on this
fork?
Progress on Django may be slower than many would like, but it's not
slow because we're hampered by backwards compatibility. It's because
the core team all have full time jobs, families and friends, and we
contribute to Django in our spare time. If you want to fix the speed
of development, pitch in.
Yours,
Russ Magee %-)
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Jacob Kaplan-Moss]

On Thu, Apr 15, 2010 at 1:57 PM, Kevin Howerton

Post by Kevin Howerton
The level of resistance I see to change or outsider code contribution
is an enormous de-motivator for people (like me) to want to make any
contributions in the first place.  Why should I contribute a patch to
your flawed architecture if I'm going to be talked down to, ridiculed,
then eventually have the patch rejected because it breaks code in some
edge-use-case?

I'm sorry that you've felt ridiculed or talked down to. I can promise
you that wasn't the intent. To satiate my own curiosity, and to help
me not make similar mistakes, could you point me to where that's
happened?

Post by Kevin Howerton
It's a good idea to avoid breaking backwards compatibility in point
releases, but as far as major releases go ... I whole heartedly
encourage it.

That is, in fact, our policy. 1.1 is compatible with 1.0; 1.2 with 1.1; etc.

Post by Kevin Howerton
Personally I believe my time might be better spent developing a fork,
than arguing over clear flaws in architecture decisions that "can't be
changed".

This is open source, and that's your prerogative. If you want to start
a fork, do. I hope you'll consider contributing back to the trunk, but
that's up to you.

For better or worse, we've chosen a development policy that
prioritizes stability, maturity, and longevity. If those aren't your
priorities, then perhaps a fork is the right answer.

Jacob

[Kevin Howerton]

"That is, in fact, our policy. 1.1 is compatible with 1.0; 1.2 with 1.1; etc."

1.1 and 1.2 are by definition not point releases. Point releases
don't introduce new features.

Post by Jacob Kaplan-Moss
On Thu, Apr 15, 2010 at 1:57 PM, Kevin Howerton

Post by Kevin Howerton
The level of resistance I see to change or outsider code contribution
is an enormous de-motivator for people (like me) to want to make any
contributions in the first place.  Why should I contribute a patch to
your flawed architecture if I'm going to be talked down to, ridiculed,
then eventually have the patch rejected because it breaks code in some
edge-use-case?

I'm sorry that you've felt ridiculed or talked down to. I can promise
you that wasn't the intent. To satiate my own curiosity, and to help
me not make similar mistakes, could you point me to where that's
happened?

Post by Kevin Howerton
It's a good idea to avoid breaking backwards compatibility in point
releases, but as far as major releases go ... I whole heartedly
encourage it.

That is, in fact, our policy. 1.1 is compatible with 1.0; 1.2 with 1.1; etc.

Post by Kevin Howerton
Personally I believe my time might be better spent developing a fork,
than arguing over clear flaws in architecture decisions that "can't be
changed".

This is open source, and that's your prerogative. If you want to start
a fork, do. I hope you'll consider contributing back to the trunk, but
that's up to you.
For better or worse, we've chosen a development policy that
prioritizes stability, maturity, and longevity. If those aren't your
priorities, then perhaps a fork is the right answer.
Jacob
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[Mike]

Post by Jacob Kaplan-Moss
For better or worse, we've chosen a development policy that
prioritizes stability, maturity, and longevity. If those aren't your
priorities, then perhaps a fork is the right answer.

Correct me if I'm wrong but I read it as "If you do not like our
policy
then stability, maturity, and longevity aren't your priorities".
With all due respect it is not fair.

Regards,
Mike

[Jacob Kaplan-Moss]

Post by Mike
Correct me if I'm wrong but I read it as "If you do not like our
policy then stability, maturity, and longevity aren't your priorities".
With all due respect it is not fair.

But isn't that exactly what people in this thread are saying? The main
complaint I'm reading here is that folks are frustrated with Django's
backwards-compatibily policy, and that they'd be willing to break
backwards-compatibility in exchange for new features. In other words,
that forward motion and architectural changes are more desirable than
backwards-compatibility.

Or am I reading Kevin, Tom, Skylar, et al. wrong?

I'm not arguing that "stability, maturity, and longevity" are
"correct" priorities, only that, well, those are the ones we've
chosen. I'm not saying it's "wrong" to want more rapid improvement,
only that it's lower on *my* list.

I think you're reading a value judgement where none's intended. We've
simply chosen certain priorities; reasonable people may have different
profiles. Nobody's "right" here; I'm trying to point out what
philosophies are leading to our conservatism.

Jacob

[Tom X. Tobin]

Post by Jacob Kaplan-Moss
I'm not arguing that "stability, maturity, and longevity" are
"correct" priorities, only that, well, those are the ones we've
chosen. I'm not saying it's "wrong" to want more rapid improvement,
only that it's lower on *my* list.

My priorities overlap, but not completely.

Stability is very important to me, and should be important to *anyone*
whose livelihood depends on Django. Stability is a large part of the
reason we *can* run our own Django branch and run production sites
based on it, yet not lose our minds in the process.

Maturity is fairly important; I want solutions that have experience
and judgement behind them. Maturity can become rigidity, though; I
enjoy exploring new ways to solve existing problems, and a new, but
superior, solution isn't — strictly speaking — "mature".

Longevity is where I part ways; I'd much rather have a clean break
than keep working around a wart. I think my overriding principle here
is correctness; I'm perfectly happy to do the work to fix my code if
it means adopting the *right* solution.

None of this means that I think the core development process should
change. (Well, besides my fervent desire that they officially adopted
git — and yes, I do believe it *would* make a difference, centralized
"official" branch and all — but that's a discussion for another time
and a few beers.) Django has been very successful with the current
process, and I'd be very wary of tinkering with the foundations of
that success.

But here's the great part: nothing is stoping anyone from hacking new
paths through the jungle on their own branches. What you *can't* get
— and honestly *shouldn't* get — is automatic recognition that your
branch is somehow officially supported, and all the notions of
stability, maturity, and longevity that go with that recognition. If
you know enough to make significant changes to Django, you also
probably know enough to fix the problems that can crop up due to your
changes — and that's not something we should expect from the average
developer who just wants to Get Work Done.

So ... who has a GitHub account and some neat code to look at? :-)

[sago]

Isn't that what forking is for?

A group of folks feel frustrated about not being able to commit, so
they make their own copy of the source code available. A few months
later they either implode when they realise just how much work it is
going to take to do anything remotely sensible, or they come up with
some bit of nuggety goodness that can be chopped around and
backported.

As long as the forking happens without falling out, that is...

On a completely unrelated note, any plans to move Django to git?

Ian.

Not that I count for jack, but its +1 on API stability and backwards
compatibility from me, btw!

Post by Tom X. Tobin

Post by Jacob Kaplan-Moss
I'm not arguing that "stability, maturity, and longevity" are
"correct" priorities, only that, well, those are the ones we've
chosen. I'm not saying it's "wrong" to want more rapid improvement,
only that it's lower on *my* list.

My priorities overlap, but not completely.
Stability is very important to me, and should be important to *anyone*
whose livelihood depends on Django.  Stability is a large part of the
reason we *can* run our own Django branch and run production sites
based on it, yet not lose our minds in the process.
Maturity is fairly important; I want solutions that have experience
and judgement behind them.  Maturity can become rigidity, though; I
enjoy exploring new ways to solve existing problems, and a new, but
superior, solution isn't — strictly speaking — "mature".
Longevity is where I part ways; I'd much rather have a clean break
than keep working around a wart.  I think my overriding principle here
is correctness; I'm perfectly happy to do the work to fix my code if
it means adopting the *right* solution.
None of this means that I think the core development process should
change.  (Well, besides my fervent desire that they officially adopted
git — and yes, I do believe it *would* make a difference, centralized
"official" branch and all — but that's a discussion for another time
and a few beers.)  Django has been very successful with the current
process, and I'd be very wary of tinkering with the foundations of
that success.
But here's the great part: nothing is stoping anyone from hacking new
paths through the jungle on their own branches.  What you *can't* get
— and honestly *shouldn't* get — is automatic recognition that your
branch is somehow officially supported, and all the notions of
stability, maturity, and longevity that go with that recognition.  If
you know enough to make significant changes to Django, you also
probably know enough to fix the problems that can crop up due to your
changes — and that's not something we should expect from the average
developer who just wants to Get Work Done.
So ... who has a GitHub account and some neat code to look at?  :-)
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group athttp://groups.google.com/group/django-developers?hl=en.

[Russell Keith-Magee]

Post by sago
On a completely unrelated note, any plans to move Django to git?

I answered this exact question earlier in this thread. The answer is
no, because it would make exactly no difference to anything. Search
out the earlier answer for more detail.

Yours,
Russ Magee %-)

[Tom X. Tobin]

On Fri, Apr 16, 2010 at 9:36 PM, Russell Keith-Magee

Post by Russell Keith-Magee

Post by sago
On a completely unrelated note, any plans to move Django to git?

I answered this exact question earlier in this thread. The answer is
no, because it would make exactly no difference to anything. Search
out the earlier answer for more detail.

I *strongly* disagree with "it would make exactly no difference to
anything", and I'd like to address it at some point in the future.
I'll start bugging you guys about it after you're done with the 1.2
push. :p

[Tom X. Tobin]

Post by Tom X. Tobin
But here's the great part: nothing is stoping anyone from hacking new

Argh, the snoot in me just winced at re-reading my post and noticing
that I misspelled "stopping". ::hangs head::

[Taylor Marshall]

Post by Tom X. Tobin
None of this means that I think the core development process should
change.  (Well, besides my fervent desire that they officially adopted
git — and yes, I do believe it *would* make a difference, centralized
"official" branch and all — but that's a discussion for another time
and a few beers.)

You might be able to make a case on the whole DVCS thing in general,
but I'm not sure why any particular flavor is necessarily *the* choice
(i.e. the whole git vs mercurial vs bazaar holy war).

Post by Tom X. Tobin
So ... who has a GitHub account and some neat code to look at?  :-)

There's already a unofficial mirror on GitHub which is maintained by jezdez:

http://github.com/django/django

Doesn't that solve the "make it easier to branch and track upstream
changes" thing?

[James Bennett]

On Fri, Apr 16, 2010 at 4:34 PM, Taylor Marshall
AFAIK there are mirrors on pretty much every DVCS/"social code"
hosting site; bitbucket's got one as well, for example.

[Tom X. Tobin]

On Fri, Apr 16, 2010 at 4:34 PM, Taylor Marshall

Post by Taylor Marshall

Post by Tom X. Tobin
None of this means that I think the core development process should
change.  (Well, besides my fervent desire that they officially adopted
git — and yes, I do believe it *would* make a difference, centralized
"official" branch and all — but that's a discussion for another time
and a few beers.)

You might be able to make a case on the whole DVCS thing in general,
but I'm not sure why any particular flavor is necessarily *the* choice
(i.e. the whole git vs mercurial vs bazaar holy war).

There's a *huge* difference between git and the other DVCSs
(completely different model), but let's save that for another time.
:-)

Post by Taylor Marshall

Post by Tom X. Tobin
So ... who has a GitHub account and some neat code to look at?  :-)

http://github.com/django/django
Doesn't that solve the "make it easier to branch and track upstream
changes" thing?

No, because it doesn't address ultimately pushing back up from the
DVCS-using developers; git makes me loathe submitting patch files to
Trac. But gah, again, I'd rather address the DVCS issue another time.

[Tom X. Tobin]

Post by Mike

Post by Jacob Kaplan-Moss
For better or worse, we've chosen a development policy that
prioritizes stability, maturity, and longevity. If those aren't your
priorities, then perhaps a fork is the right answer.

Correct me if I'm wrong but I read it as "If you do not like our
policy
then stability, maturity, and longevity aren't your priorities".
With all due respect it is not fair.

Logically speaking, P -> Q doesn't imply Q -> P.

[Tom X. Tobin]

On Thu, Apr 15, 2010 at 1:57 PM, Kevin Howerton

Post by Kevin Howerton
"You seem to be suggesting that a fork will somehow magically fix the
speed of Django development. I ask you: who is going to work on this
fork?"
I think a hostile fork is almost a certain outcome if development
continues as it has.  Not only is the resistance to make backwards
incompatible changes in future releases a bad policy for the quality
of the framework, but the behavior in trac has a negative effect on
community contributions.

[...]

Post by Kevin Howerton
Personally I believe my time might be better spent developing a fork,
than arguing over clear flaws in architecture decisions that "can't be
changed".

Django is BSD licensed; no one is going to stop someone from making a
fork if they want to. That no one has done so is, IMHO, a good sign
that the Django codebase and development process is considered solid
by the community.

Solid, of course, can sometimes be boring. :-) There's nothing wrong
with public experimentation; why not push a Django branch up on GitHub
with some features you find interesting? You don't have to "fork" in
order to develop your own branch; we've been maintaining an internal
branch of Django at The Onion for a couple of years now, but we still
track upstream aggressively. Sometimes I wonder if a long-running
experimental playground branch (or branches) on GitHub would be a
healthy way to direct some of the energy and interest in less
conservative changes; as pieces matured there, they could be
considered for the mainstream trunk, and trunk would in turn remain
nice and stable.

[s***@public.gmane.org]

+1 I would love to join in with some liberal, even wild-eyed experimentation on github. I have all sorts of cock-eyed ideas that I would like to play with.

I don't like patches; I don't like svn; I'm not a fan of trac. I do however enjoy the fact that Django trunk leans toward being boring and stable.

Make a hostile fork. If it turns out to be awesome and stable and tested and so on .. then awesome..

Not sure why it has to be "hostile". But, depending on the project, I could definitely see advantages to giving up on some of the backward compatibility overhead.

Cheap branches with git I think could be a great way for community members like myself to get their feet wet without the overhead and constraints of the full contributing process.
sent from my baz foone

-----Original Message-----
From: "Tom X. Tobin" <tomxtobin-e1dn8vUCZwaaMJb+***@public.gmane.org>
Date: Thu, 15 Apr 2010 14:38:01
To: <django-developers-/***@public.gmane.org>
Subject: Re: High Level Discussion about the Future of Django

On Thu, Apr 15, 2010 at 1:57 PM, Kevin Howerton

Post by Kevin Howerton
"You seem to be suggesting that a fork will somehow magically fix the
speed of Django development. I ask you: who is going to work on this
fork?"
I think a hostile fork is almost a certain outcome if development
continues as it has.  Not only is the resistance to make backwards
incompatible changes in future releases a bad policy for the quality
of the framework, but the behavior in trac has a negative effect on
community contributions.

[...]

Post by Kevin Howerton
Personally I believe my time might be better spent developing a fork,
than arguing over clear flaws in architecture decisions that "can't be
changed".

Django is BSD licensed; no one is going to stop someone from making a
fork if they want to. That no one has done so is, IMHO, a good sign
that the Django codebase and development process is considered solid
by the community.

Solid, of course, can sometimes be boring. :-) There's nothing wrong
with public experimentation; why not push a Django branch up on GitHub
with some features you find interesting? You don't have to "fork" in
order to develop your own branch; we've been maintaining an internal
branch of Django at The Onion for a couple of years now, but we still
track upstream aggressively. Sometimes I wonder if a long-running
experimental playground branch (or branches) on GitHub would be a
healthy way to direct some of the energy and interest in less
conservative changes; as pieces matured there, they could be
considered for the mainstream trunk, and trunk would in turn remain
nice and stable.
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To unsubscribe from this group, send email to django-developers+***@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

[tiemonster]

One of the main advantages of Django over other web frameworks is
twofold:

1. Almost anything can be overridden with a custom backend (auth, e-
mail, context processors, middleware, etc.)
2. Custom backends can be plugged in side-by-side with "stock"
backends

What functionality do you feel is holding Django back in particular?
NoSQL support? Cloud architecture? In my opinion, the best course of
action is to write a backend for a desired piece of functionality and
then release it as open source. Just because something doesn't make it
into trunk doesn't mean that it can't be a popular modular extension
to Django in real-world deployments.

As for Python 3.x support, I think the #python welcome message sums it
up: "It's too early to use Python 3.x" :-)

[orokusaki]

When I first started posting things on trac, I put up a request that
took me an hour to create, explaining the justification, as well as
putting the code in there. I didn't know how to make a patch, and I
went about it the wrong way, but regardless of that, I put a lot of
thought into it. Less than 3 minutes after I posted it, it was closed
and resolution set to `wontfix`. It was an extremely minor change that
didn't break any backwards compatibility, but I didn't follow the
right procedures, so wham, gone. I tend to agree with Kevin's
assertion that it's not worth spending time developing when you don't
know if your ideas will be thrown in the trash, but I won't complain
about it because everything these people do for Django is free to us,
and if they weren't hardcore, the framework could have turned into
muck by this point (with the number of feature requests out there).

After a few rounds of doing that, I realized that people like Russell,
who appear to be closed minded at first glance, are really not closed
minded at all, but just overworked by tickets and don't have extra
time to do every single ticket. I believe this is primarily because
for every added feature, tons and tons of code has to be changed /
added to keep everything backwards compatible with code from a year
ago.

I think I speak for a pretty broad user base when I say that folks who
use Django are bleeding edge developers who want cool stuff, and don't
mind paying a little extra to have it. It isn't like IBM and Microsoft
are using Django for huge distributed projects, and upgrading all
their clients to the latest version each week. And, again back to
Kevin's point; if they are upgrading quickly, they are the types that
understand the value of doing so.

If the core team doesn't want to change the backwards compatibility
policy a tiny bit to accommodate faster paced code ( keeping up with
the Jone's and such, such as RoR 3.0 ), it might be worth investing
the time to start a fork. A little competition for the core team
wouldn't hurt them a bit. It would relieve a little of the stress of
being bombarded by tickets for things that are not possible for a
whole year, etc. I would only suggest that if you were to go this
route, to do so in a way that isn't harmful to Django as a whole (e.g.
making empty promises will leave a bitter taste and ruin the name).

Post by Kevin Howerton
"You seem to be suggesting that a fork will somehow magically fix the
speed of Django development. I ask you: who is going to work on this
fork?"
I think a hostile fork is almost a certain outcome if development
continues as it has.  Not only is the resistance to make backwards
incompatible changes in future releases a bad policy for the quality
of the framework, but the behavior in trac has a negative effect on
community contributions.
The level of resistance I see to change or outsider code contribution
is an enormous de-motivator for people (like me) to want to make any
contributions in the first place.  Why should I contribute a patch to
your flawed architecture if I'm going to be talked down to, ridiculed,
then eventually have the patch rejected because it breaks code in some
edge-use-case?
Personally I believe my time might be better spent developing a fork,
than arguing over clear flaws in architecture decisions that "can't be
changed".
It's a good idea to avoid breaking backwards compatibility in point
releases, but as far as major releases go ... I whole heartedly
encourage it.  If keeping backwards compatibility was a good idea, my
macbook pro would have a 5.12" floppy drive, a 3.25" drive, a jazz
drive, it would accept serial and adb plugs.... and maybe have a
display mode to emulate those green crt monitors of yore.
Good software is about removing flaws, improving the architecture, and
learning from past mistakes.  If this comes at a price, then pay.
-k
On Wed, Apr 14, 2010 at 9:25 AM, Russell Keith-Magee

Post by Russell Keith-Magee

Post by veena
I know there's django deprecation policy nicely documented
http://docs.djangoproject.com/en/1.1/internals/release-process/#inter...
But what I don't know is how you discover it. Is it described
somewhere in the text or the video from conference? What were the
reasons to have this deprecation policy? Was there any user research?
Research of when the django users upgrade, what are the main problems
of upgrades and how they imagine upgrading should work?

The policy was arrived at after a debate between the core team, based
on how the core team believe a well-behaved project should behave. For
the record, it wasn't much of a debate, either - we were all pretty
much in agreement on the core points from the beginning.
In the opinion of the core, well-behaved projects don't require
massive rewrites (or worse - subtle bug chasing efforts) every time a
new release is made. Developers using a library should have the
confidence to know that when they write code, it will continue to work
for a long time, not just until the core developers have a change of
heart.
I would suggest to you that one of the reasons for Django's success
has been it's policy on backwards compatibility.

Post by veena
What I try to say is that I'm little bit afraid that it seems like
improvements of django will take years instead of months.

...

Post by veena
I think this could lead to fork the django by some devs
and rapid development of this fork

You seem to be suggesting that a fork will somehow magically fix the
speed of Django development. I ask you: who is going to work on this
fork?
Progress on Django may be slower than many would like, but it's not
slow because we're hampered by backwards compatibility. It's because
the core team all have full time jobs, families and friends, and we
contribute to Django in our spare time. If you want to fix the speed
of development, pitch in.
Yours,
Russ Magee %-)
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group athttp://groups.google.com/group/django-developers?hl=en.

[Tom Evans]

Post by orokusaki
...
I think I speak for a pretty broad user base when I say that folks who
use Django are bleeding edge developers who want cool stuff, and don't
mind paying a little extra to have it. It isn't like IBM and Microsoft
are using Django for huge distributed projects, and upgrading all
their clients to the latest version each week. And, again back to
Kevin's point; if they are upgrading quickly, they are the types that
understand the value of doing so.

I don't work for Microsoft or IBM, but as someone who actually does
run a mission critical service built around django, with 99.99% uptime
requirements (60 mins downtime/year), we seriously appreciate the
stability of django development - it was one of the main pros compared
to other frameworks when we decided to move our web development from
C++ to a dynamic language.

Don't get me wrong, we love new features as much as the next person,
and we're eagerly awaiting the chance to get 1.2-release into testing,
but each new release means about a week of testing, code reviews and
so on. The current balance between new features and stability suits us
just fine - features aren't rushed in, even in contrib, and that has
to be a good thing.

Cheers

Tom

[Russell Keith-Magee]

Post by orokusaki
When I first started posting things on trac, I put up a request that
took me an hour to create, explaining the justification, as well as
putting the code in there. I didn't know how to make a patch, and I
went about it the wrong way, but regardless of that, I put a lot of
thought into it. Less than 3 minutes after I posted it, it was closed
and resolution set to `wontfix`. It was an extremely minor change that
didn't break any backwards compatibility, but I didn't follow the
right procedures, so wham, gone.

I'm going to stop you there. You repeatedly *asserted* that it was
backwards compatible. We (Joseph and I) *repeatedly* told you that we
didn't think it was, and we *repeatedly* told you to take the
discussion to django-dev, and you *repeatedly* didn't, and we
*repeatedly* pointed you at the contribution docs to tell you why we
were saying you should take it to django-dev, and you *repeatedly*
didn't... until you finally did, at which point, we demonstrated why
your idea wasn't backwards compatible, and -- unless I'm mistaken --
you agreed.

Please don't characterize what Joseph and I did as arbitrary or closed minded.

Post by orokusaki
If the core team doesn't want to change the backwards compatibility
policy a tiny bit to accommodate faster paced code ( keeping up with
the Jone's and such, such as RoR 3.0 ),

Django's backwards incompatibility policy doesn't mean *no change*. It
means *no sudden change*. There are *many* examples in Django 1.2
alone of features that are being deprecated or altered in ways that
will ultimately be incompatible. Code written for Django 1.0 won't run
unmodified in Django 1.4 due to changes in admin registration,
messaging, CSRF, and many other features. However, these changes are
introduced as a series of progressive enhancements rather than sudden
changes. This enables use to provide ample active warnings advising
that code changes will be required. It also avoids the need to
introduce "USE_NEW_BEHAVIOR" settings switches that require us to
maintain two implementations in parallel and in perpetuity.

We're open to any proposal to change the status quo - as long as it
can be done gradually and gracefully. If you think hard enough, *most*
problems can be fixed in this way. Those that can't are unfortunate
warts, but that's the price we pay for having a stable platform. As
Jacob said, that's not a claim that Django-style stability is
"correct" - it's just the policy we have adopted based on our
particular priorities.

Yours,
Russ Magee %-)

[orokusaki]

Russell,

This is what I meant by "straw hat" the other day. You took what I
said out of context in a sly attempt at ignoratio elenchi. I made it
clear in the first paragraph that **I started out thinking you were
closed minded**, but then said that **I later realized that you were
just busy**. I was bringing into question the policy of backwards
compatibility, not your personality. It is you who has been engaged in
ad hominem, not I.

This sort of behavior is not conducive to a productive development
environment.

The real points you should be taking from all these conversations:

1) People don't like slow progress, but also want stability. Maybe
it's time to revisit the policy to find a better balance, so that
users aren't always recommended to use the trunk in production
websites.

2) It seems that others are concerned with SVN and want to use a
definative version, e.g. 1.2.2, instead of some r5481849448 number,
and without having to wait for 6 months for the official 1.2 release.

3) People specifically aren't contributing because they know that
every attempt they make will get them exactly nowhere, except close
lined. Perhaps problem 1 could be solved by solving this problem with
a slightly more open policy, instead of closing tickets 5 seconds
after they're open.

4) The attitude projected at developers gives the idea that Django is
for the core team only, and that users are graced with the ability to
use Django. While the contribution is much appreciated, the attitude
is harmful to the core team and to the user base.

Post by Russell Keith-Magee

Post by orokusaki
When I first started posting things on trac, I put up a request that
took me an hour to create, explaining the justification, as well as
putting the code in there. I didn't know how to make a patch, and I
went about it the wrong way, but regardless of that, I put a lot of
thought into it. Less than 3 minutes after I posted it, it was closed
and resolution set to `wontfix`. It was an extremely minor change that
didn't break any backwards compatibility, but I didn't follow the
right procedures, so wham, gone.

I'm going to stop you there. You repeatedly *asserted* that it was
backwards compatible. We (Joseph and I) *repeatedly* told you that we
didn't think it was, and we *repeatedly* told you to take the
discussion to django-dev, and you *repeatedly* didn't, and we
*repeatedly* pointed you at the contribution docs to tell you why we
were saying you should take it to django-dev, and you *repeatedly*
didn't... until you finally did, at which point, we demonstrated why
your idea wasn't backwards compatible, and -- unless I'm mistaken --
you agreed.
Please don't characterize what Joseph and I did as arbitrary or closed minded.

Post by orokusaki
If the core team doesn't want to change the backwards compatibility
policy a tiny bit to accommodate faster paced code ( keeping up with
the Jone's and such, such as RoR 3.0 ),

Django's backwards incompatibility policy doesn't mean *no change*. It
means *no sudden change*. There are *many* examples in Django 1.2
alone of features that are being deprecated or altered in ways that
will ultimately be incompatible. Code written for Django 1.0 won't run
unmodified in Django 1.4 due to changes in admin registration,
messaging, CSRF, and many other features. However, these changes are
introduced as a series of progressive enhancements rather than sudden
changes. This enables use to provide ample active warnings advising
that code changes will be required. It also avoids the need to
introduce "USE_NEW_BEHAVIOR" settings switches that require us to
maintain two implementations in parallel and in perpetuity.
We're open to any proposal to change the status quo - as long as it
can be done gradually and gracefully. If you think hard enough, *most*
problems can be fixed in this way. Those that can't are unfortunate
warts, but that's the price we pay for having a stable platform. As
Jacob said, that's not a claim that Django-style stability is
"correct" - it's just the policy we have adopted based on our
particular priorities.
Yours,
Russ Magee %-)
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group athttp://groups.google.com/group/django-developers?hl=en.

[Nick Phillips]

Post by orokusaki
4) The attitude projected at developers gives the idea that Django is
for the core team only, and that users are graced with the ability to
use Django. While the contribution is much appreciated, the attitude
is harmful to the core team and to the user base.

You might like to consider the fact that this is not at all the
"attitude" I see.

What I see is a group of people working very hard and holding themselves
to very high standards. In such a situation I would expect that I too
would have to meet those standards were I to wish any contribution I
might make to be included in their work ("work" in the sense of "a
work").

That is a challenge I would love to have the time for, but I don't. So
where I find that Django either has issues or fails to meet my needs, I
work around the problems in other ways. I might well file a ticket here
and there, and even a patch if I have one that might be useful, but
"inclusion straight into trunk" is not the only way a patch can be
useful. I'm certainly not going to try to tell them that they should be
doing things my way - and if I did want to, I wouldn't expect to get
anywhere with your current tactic.

This thread is distracting energy that could be better spent, so I'm
going to leave it at that, and would suggest that others do too.


Cheers,


Nick
--
Nick Phillips / [phone] / nick.phillips-I1m3D+***@public.gmane.org
# these statements are my own, not those of the University of Otago

[Russell Keith-Magee]

Post by orokusaki
Russell,
This is what I meant by "straw hat" the other day. You took what I
said out of context in a sly attempt at ignoratio elenchi. I made it
clear in the first paragraph that **I started out thinking you were
closed minded**, but then said that **I later realized that you were
just busy**. I was bringing into question the policy of backwards
compatibility, not your personality. It is you who has been engaged in
ad hominem, not I.

I won't take any argument with the assertion that I'm busy. What I'm
getting frustrated at is that you are using argumentum ad nauseam to
establish that you are somehow the victim of either a capricious or
overworked (or both) core team. Every single decision that you have
been subjected to is, on considered reflection, completely correct and
would be repeated under identical circumstances.

The decisions that Joseph and I made on those tickets were *not* made
in haste, and were *not* made incorrectly. Your proposal *was*
backwards incompatible, and you *repeatedly* refused to follow advice
to take the discussion to django developers. Your indignation at the
fact that it took 3 minutes to reject the ticket ignores the fact that
the people closing the ticket have 18 months of prior experience
discussing the exact problem you apparently solved in an hour.

When you accused Karen of a "straw hat", you completely missed the
point that what she said was, again, completely accurate, and
substantively identical to the argument that I presented (which was
apparently sufficient for some reason).

I have no problem being accused of things that I have actually done. I
take *deep* offense at being accused of things I haven't done.

As for the backwards compatibility policy - as I've said several times
now, backwards compatibility doesn't mean *no change*, it means *no
sudden change*. This is based on a consensus between the core team on
how a software project should behave, and history demonstrates that
*most* problems can be solved within our existing policy framework.

If you want to make a case for a specific change that you think should
be given an exception to this policy, please make your case.

If you've got a suggestion for how we should modify the policy to
allow for a specific type of change, we're open to that too - but
specifics matter. Abstract discussions go nowhere fast. We would need
to see specific suggestions of how your new policy would get us out of
specific architectural holes without violating the core principles we
want to maintain.

Post by orokusaki
1) People don't like slow progress, but also want stability. Maybe
it's time to revisit the policy to find a better balance, so that
users aren't always recommended to use the trunk in production
websites.

Firstly, the advice on the FAQ should be updated. The current advice
there hasn't been updated since pre 1.0 days, when we *did* advise
that people used trunk. These days, you should be using the stable
branch in production (1.1.X, at present) unless you're willing to take
on the burden of being an alpha tester.

As for slow progress: we need to be clear here. When you say slow
progress, are you referring to:
1) The fact that 1.2 is 2 months behind schedule, or
2) The fact that we have a strict policy on backwards incompatible changes?

If it's (1), the reason progress is slow is that the core team -- and
the rest of the development community, for that matter -- are
volunteers. Developing software takes time. The only solution to this
is more resources, and that means we need people to volunteer their
time.

If it's (2), then I simply disagree. I'm only aware of 2 problems in
Django that are serious flaws that can't be fixed in a backwards
compatible way:
* The inconsistent handling of the url name in {% url %}
* The need to manually call full_clean() when saving a model (and the
ModelForm problems that stem from this).

I don't really see either of these as a serious impediment to
progress. They're annoying, yes. If backwards compatibility wasn't an
issue, I'd fix them in a heartbeat. But they are two very small
aspects of Django overall, there are perfectly adequate workarounds
available, and "fixing" them would mean every single Django user in
existence would be forced to do a potentially expensive audit of their
code before an upgrade -- and that's simply not something we're
willing to ask the user base to do.

Post by orokusaki
2) It seems that others are concerned with SVN and want to use a
definative version, e.g. 1.2.2, instead of some r5481849448 number,
and without having to wait for 6 months for the official 1.2 release.

Again - this is something we haven't handled well over the 1.2
lifecycle. In hindsight, we probably should have put out a 1.1.2
release a couple of months ago.

However, the reason we haven't put out 1.2 yet is because it isn't
ready yet. We've added a lot of features, and we're still fixing
problems. If we're going to encourage people to use a stable release,
it actually needs to be stable, so we're going to err on the side of
caution here. Again, if you want the release to happen faster, pitch
in.

Post by orokusaki
3) People specifically aren't contributing because they know that
every attempt they make will get them exactly nowhere, except close
lined. Perhaps problem 1 could be solved by solving this problem with
a slightly more open policy, instead of closing tickets 5 seconds
after they're open.

Again, argumentum ad nauseum. See above.

Post by orokusaki
4) The attitude projected at developers gives the idea that Django is
for the core team only, and that users are graced with the ability to
use Django. While the contribution is much appreciated, the attitude
is harmful to the core team and to the user base.

I am completely floored by this. I simply don't see where this
perception has come from, and it doesn't mesh with the feedback I have
received in this thread, privately in response to this thread, or on
IRC.

Yours,
Russ Magee %-)

[George Sakkis]

Post by Kevin Howerton
The level of resistance I see to change or outsider code contribution
is an enormous de-motivator for people (like me) to want to make any
contributions in the first place.  Why should I contribute a patch to
your flawed architecture if I'm going to be talked down to, ridiculed,
then eventually have the patch rejected because it breaks code in some
edge-use-case?

Good luck pushing backwards incompatible patches when as we speak
there are almost 400 open tickets with patches at accepted [1] and
"ready for checkin" [2] stage. Under these circumstances, backwards
compatibility is almost a red herring; the bigger issue IMO is the
increasing pile of bug fixes and solid, backwards compatible patches
languishing for months or years.

A fork that encouraged and achieved a faster submit-review-accept-
commit lifecycle, even with the same stability, maturity, and
longevity policies, could be a breath of fresh air.

George


[1]
http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&needs_better_patch=!1&needs_tests=!1&needs_docs=!1&has_patch=1&order=priority&stage=Accepted

[2]
http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&needs_better_patch=!1&needs_tests=!1&needs_docs=!1&has_patch=1&order=priority&stage=Ready+for+checkin

[Russell Keith-Magee]

Post by George Sakkis

Post by Kevin Howerton
The level of resistance I see to change or outsider code contribution
is an enormous de-motivator for people (like me) to want to make any
contributions in the first place.  Why should I contribute a patch to
your flawed architecture if I'm going to be talked down to, ridiculed,
then eventually have the patch rejected because it breaks code in some
edge-use-case?

Good luck pushing backwards incompatible patches when as we speak
there are almost 400 open tickets with patches at accepted [1] and
"ready for checkin" [2] stage. Under these circumstances, backwards
compatibility is almost a red herring; the bigger issue IMO is the
increasing pile of bug fixes and solid, backwards compatible patches
languishing for months or years.
A fork that encouraged and achieved a faster submit-review-accept-
commit lifecycle, even with the same stability, maturity, and
longevity policies, could be a breath of fresh air.

As I have said *many* times in the past - I would *love* for someone
to do this. "Fork" is an extreme way of describing it -- I'd prefer to
think of it as a "trunk-ready branch", in the same way that Linus
relies on his lieutenants as a source of vetted patches for Linux
trunk -- but I have no problem with the basic idea.

If such a branch were to exist, and the person (or people) maintaining
the branch (or branches) maintained the same levels of quality that
Django's trunk expects - good architectural style, extensive
documentation, testing, good commit messages, etc - I would use that
branch as a source of material to commit to trunk *in a heartbeat*.
Examining a small number of branches that have been curated by people
I trust would be a much more effective use of my bug-fixing time than
trolling the entirety of Trac.

However, at this point, I would like to tell you a story about four
people named Everybody, Somebody, Anybody, Nobody.

Everybody agreed that a trunk-ready branch was needed.

Somebody should have worked on the trunk-ready branch.

Anybody could have worked on the trunk-ready branch.

However, Nobody actually worked on the trunk-ready branch.

It's *really* easy to stand on the sidelines and say "there are too
many open tickets" or "progress isn't fast enough". It's another thing
entirely to actually put the time and effort into doing the work. It's
yet another thing to *still* be doing that same work 3 months (or, in
my case, 4 years) later.

So - if you want Django to progress faster - pitch in. If anyone wants
to put the effort into maintaining a trunk-ready branch, go right
ahead. Personally, I have the ability to merge branches from both git
and hg, so you can't claim that the lack of an official DVCS is
hampering you. Once you've got something worth merging, post to
django-dev describing what you've got, and we'll take a look.

Yours,
Russ Magee %-)

[Tom X. Tobin]

On Fri, Apr 16, 2010 at 10:10 PM, Russell Keith-Magee

Post by Russell Keith-Magee
However, at this point, I would like to tell you a story about four
people named Everybody,  Somebody, Anybody, Nobody.

This is exactly why I try not to bitch too much about Django's
development process. It's very easy to complain, but it's not quite
so easy to "shut up and show me the code".

[Jerome Leclanche]

Post by Tom X. Tobin
On Fri, Apr 16, 2010 at 10:10 PM, Russell Keith-Magee

Post by Russell Keith-Magee
However, at this point, I would like to tell you a story about four
people named Everybody,  Somebody, Anybody, Nobody.

This is exactly why I try not to bitch too much about Django's
development process.  It's very easy to complain, but it's not quite
so easy to "shut up and show me the code".
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

And it's not supposed to be. Sometimes, all you need is feedback. Here is some!

I personally love to refer to the Bazaar development process as an
extremely healthy project and way to work. Hopefully, you guys can
learn a thing of two out of this.

For one, there is no split between a -users mailing list and a
-developers mailing list. Understand that the Bazaar mailing list is
just as active as django-developers (so less active than -users +
-developers). But it does have one clear benefit. Users don't get
pushed around with "Django-developers is for discussion about blah
blah blah".
Django, even more so than Bazaar, is an application that has
developers as its primary userbase. Using the same mailing list for
"everything" gives developers an unique insight into what users want,
where problems exist and gives users the feeling they are able to
contribute more easily, including with code!

Of course this is just a proposal, but I've seen the "Wrong mailing
list!" warning spat out too easily. Sometimes questions should be
answered by developers. Who else is more able to answer about the
Proper Usage of a specific feature than its own author?

Another health feature of Bazaar is how tight the community is.
Django's community is similarly-sized, yet Bazaar manages to gather
its community, giving it a much friendlier environment. Launchpad
(their bug tracking platform) is being used for much of the off-list
discussions and lets the users easily contact developers on the list,
while trac still looks confusing, has a disgusting login system, and
Thousands More Issues.
(Before anyone throws me a "Shut up and show me the code!", I've been
working on a bug/issue tracker using Django as backend. I'll gladly
accept help from anyone who also wants to work on this!)

Plugin development and discussion is also done on the same list. This
ties the community even closer, while Django's best fallback is
djangosnippets.org. Last time I saw anything like it on this list was
a few days ago with the django template language port to Qt. And what
does the Django Developer say to the guy who releases a Cool Project?
"Wrong mailing list!".
Needless to say (but saying it anyway), this is also the sort of
undeserved hostility people have been talking about in this thread.

Oh and lastly, they don't use svn ;)

2¢

J. Leclanche / Adys

[Tom X. Tobin]

Post by Jerome Leclanche
For one, there is no split between a -users mailing list and a
-developers mailing list. Understand that the Bazaar mailing list is
just as active as django-developers (so less active than -users +
-developers). But it does have one clear benefit. Users don't get
pushed around with "Django-developers is for discussion about blah
blah blah".
Django, even more so than Bazaar, is an application that has
developers as its primary userbase. Using the same mailing list for
"everything" gives developers an unique insight into what users want,
where problems exist and gives users the feeling they are able to
contribute more easily, including with code!

I think this is due to an unfortunate (but common) confusion of
"developers" and "users". In the mailing list sense, "developers"
means developers *of Django itself*, not "developers who use Django";
"users" means "anyone who uses Django", which (obviously) includes
developers in the general sense of the term.

I feel the separation of mailing lists is a must, even if their names
are not necessarily ideal; considering the high volume of posts to
-users, it would be far too easy for posts about developing Django
itself to get lost there.

[George Sakkis]

Post by Tom X. Tobin
On Fri, Apr 16, 2010 at 10:10 PM, Russell Keith-Magee

Post by Russell Keith-Magee
However, at this point, I would like to tell you a story about four
people named Everybody,  Somebody, Anybody, Nobody.

This is exactly why I try not to bitch too much about Django's
development process.  It's very easy to complain, but it's not quite
so easy to "shut up and show me the code".

My point is that unfortunately this is not enough. The 400 languishing
patches have been submitted by people who did exactly that, they "shut
up and showed the code", possibly without ever complaining in this
list. And not only that but their patches (or some percentage of them
at any rate) have been "accepted" or became "ready for checkin" at
some point. How come a developer finds the time to review a patch,
accept it, consider it ready for checkin but not actually commit it ?

Healthy projects don't need a separately maintained fork/branch on
github or bitbucket just to review and apply patches. They open up
their gates and they invite more contributors to the development
process (in a controlled manner of course) so that they can keep up
with the increasing volume of external contributions.

George

[Russell Keith-Magee]

Post by George Sakkis

Post by Tom X. Tobin
On Fri, Apr 16, 2010 at 10:10 PM, Russell Keith-Magee

Post by Russell Keith-Magee
However, at this point, I would like to tell you a story about four
people named Everybody,  Somebody, Anybody, Nobody.

This is exactly why I try not to bitch too much about Django's
development process.  It's very easy to complain, but it's not quite
so easy to "shut up and show me the code".

My point is that unfortunately this is not enough. The 400 languishing
patches have been submitted by people who did exactly that, they "shut
up and showed the code", possibly without ever complaining in this
list. And not only that but their patches (or some percentage of them
at any rate) have been "accepted" or became "ready for checkin" at
some point. How come a developer finds the time to review a patch,
accept it, consider it ready for checkin but not actually commit it ?

For the record, there are 62 tickets marked ready for checkin, not 400
[1]. 29 of those are documentation and translation patches (5 of which
are specifically marked for inclusion in 1.2).

[1] http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&order=priority&stage=Ready+for+checkin

On top of that, the Ready For Checkin status doesn't mean that a
member of the core team has reviewed a patch. It means that someone --
anyone -- thinks the patch is ready for checkin. There's no guarantee
that a Ready For Checkin patch is *actually* ready for checkin. If you
do a survey of the Ready For Checkin patches, you'll find tickets that
don't have test cases, or add features that aren't documented, or make
a significant changes that haven't been discussed on django-dev. If I
were to sit down and work through that list, I guarantee I wouldn't
end up making 62 commits to trunk using the material that has been
provided on those tickets.

I would also point out the folly of looking at raw ticket counts.
Python (the language) has 1078 tickets in the "having patch" status,
and 96 in the "needing review" status. Does this mean that Python is a
project in crisis?

Yes, there is a ticket backlog. Yes, this means there is a lot of work
that needs to be done. What we need is people volunteering to actually
do that work. However, as I've already indicated in this thread, much
of that work could be done without any change in current project
policy - we just needs people to actually do the work.

My dream outcome would to be in the situation where I don't *ever*
have to spend time on Trac trying to work out if the ticket that has
been marked Ready For Checkin is *actually* ready for checkin. Give me
a rich vein of trunk ready tickets that has been reviewed by someone
whose reputation I know and trust, and believe me -- I will use it.

Post by George Sakkis
Healthy projects don't need a separately maintained fork/branch on
github or bitbucket just to review and apply patches. They open up
their gates and they invite more contributors to the development
process (in a controlled manner of course) so that they can keep up
with the increasing volume of external contributions.

The flipside of this is that too many cooks spoil the broth. If we
want to maintain a high quality product, we can't just add a dozen new
developers to the core team.

I would also point out that even in projects that do have large teams
with the commit bit, access to the "core trunk" is generally only made
available to a restricted subset of the entire team. Alternatively,
some sort of code review process is used to ensure that multiple team
members (occasionally, specially blessed team members) check patches
before they are committed. There's a lot more to commit policies in
open source than raw team size.

Yours,
Russ Magee %-)

[Stephen Wolff]

I feel quite sad reading this thread. Good luck completing 1.2. I only wish
I had time and energy to contribute. I suggest the core team ignore the
thread for now if at all possible.

On Apr 17, 5:35 am...

For the record, there are 62 tickets marked ready for checkin, not 400
[1]. 29 of those are documentation and translation patches (5 of which
are specifically marked for inclusion in 1.2).

[1]
http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&order=priority&stage=Ready+for+checkin

On top of that, the Ready For Checkin status doesn't mean that a
member of the core team has reviewed a patch. It means that someone --
anyone -- thinks the patch is ready for checkin. There's no guarantee
that a Ready For Checkin patch is *actually* ready for checkin. If you
do a survey of the Ready For Checkin patches, you'll find tickets that
don't have test cases, or add features that aren't documented, or make
a significant changes that haven't been discussed on django-dev. If I
were to sit down and work through that list, I guarantee I wouldn't
end up making 62 commits to trunk using the material that has been
provided on those tickets.

I would also point out the folly of looking at raw ticket counts.
Python (the language) has 1078 tickets in the "having patch" status,
and 96 in the "needing review" status. Does this mean that Python is a
project in crisis?

Yes, there is a ticket backlog. Yes, this means there is a lot of work
that needs to be done. What we need is people volunteering to actually
do that work. However, as I've already indicated in this thread, much
of that work could be done without any change in current project
policy - we just needs people to actually do the work.

My dream outcome would to be in the situation where I don't *ever*
have to spend time on Trac trying to work out if the ticket that has
been marked Ready For Checkin is *actually* ready for checkin. Give me
a rich vein of trunk ready tickets that has been reviewed by someone
whose reputation I know and trust, and believe me -- I will use it.

Healthy projects don't need a separately maintained fork/branch on
github or bitbucket just to ...

The flipside of this is that too many cooks spoil the broth. If we
want to maintain a high quality product, we can't just add a dozen new
developers to the core team.

I would also point out that even in projects that do have large teams
with the commit bit, access to the "core trunk" is generally only made
available to a restricted subset of the entire team. Alternatively,
some sort of code review process is used to ensure that multiple team
members (occasionally, specially blessed team members) check patches
before they are committed. There's a lot more to commit policies in
open source than raw team size.

Yours,
Russ Magee %-)

[Simone Federici]

The work of the core team is outstanding and I find that the process of
development
is to be taken as an example.

Unfortunately, customers often want features, but we are programmers,
engineers,
and we know who we are and what is our role.

Compatibility is strongly important when choosing a tool.
Otherwise who would have used? only geeks?


Imagine if we can use frameworks that between one release and another
introduce incompatibility. no thanks.

thanks guys
S

Post by Stephen Wolff
I feel quite sad reading this thread. Good luck completing 1.2. I only wish
I had time and energy to contribute. I suggest the core team ignore the
thread for now if at all possible.

On Apr 17, 5:35 am...

For the record, there are 62 tickets marked ready for checkin, not 400
[1]. 29 of those are documentation and translation patches (5 of which
are specifically marked for inclusion in 1.2).
[1]
http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&order=priority&stage=Ready+for+checkin
On top of that, the Ready For Checkin status doesn't mean that a
member of the core team has reviewed a patch. It means that someone --
anyone -- thinks the patch is ready for checkin. There's no guarantee
that a Ready For Checkin patch is *actually* ready for checkin. If you
do a survey of the Ready For Checkin patches, you'll find tickets that
don't have test cases, or add features that aren't documented, or make
a significant changes that haven't been discussed on django-dev. If I
were to sit down and work through that list, I guarantee I wouldn't
end up making 62 commits to trunk using the material that has been
provided on those tickets.
I would also point out the folly of looking at raw ticket counts.
Python (the language) has 1078 tickets in the "having patch" status,
and 96 in the "needing review" status. Does this mean that Python is a
project in crisis?
Yes, there is a ticket backlog. Yes, this means there is a lot of work
that needs to be done. What we need is people volunteering to actually
do that work. However, as I've already indicated in this thread, much
of that work could be done without any change in current project
policy - we just needs people to actually do the work.
My dream outcome would to be in the situation where I don't *ever*
have to spend time on Trac trying to work out if the ticket that has
been marked Ready For Checkin is *actually* ready for checkin. Give me
a rich vein of trunk ready tickets that has been reviewed by someone
whose reputation I know and trust, and believe me -- I will use it.

Healthy projects don't need a separately maintained fork/branch on
github or bitbucket just to ...

The flipside of this is that too many cooks spoil the broth. If we
want to maintain a high quality product, we can't just add a dozen new
developers to the core team.
I would also point out that even in projects that do have large teams
with the commit bit, access to the "core trunk" is generally only made
available to a restricted subset of the entire team. Alternatively,
some sort of code review process is used to ensure that multiple team
members (occasionally, specially blessed team members) check patches
before they are committed. There's a lot more to commit policies in
open source than raw team size.
Yours,
Russ Magee %-)
--
You received this message because you are subscribed to the Google Groups
"Django developers" g...
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group, send email to
.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.

[David Cramer]

I just want to throw my 2 cents into the ring here. I'm not against a
fork, but at the same time I want to see the Django mainline progress.
However, let me tell you my story, and how I've seen the Django
development process over the years.

I started with Django 4 years ago. It was cool, shiny, and let us get
up and running. At the time, were were one of the largest Django
installations. We had needs, and some of those needs were met.
However, many were not. We were the only ones furiously trying to get
the core team to accept our patches, ideas, etc.. Now while I'm not
going to say every idea we had was right for Django, there were in
fact many that were great, and eventually have made it into trunk.
There are also still many (lets take the classic #17 here), that still
haven't made it into trunk, even though people have been even more
aggressively pushing them lately. I honestly can only remember a
single patch that I've committed that has ever been fully integrated
into trunk (select_related refactor).

Now, the development process has changed with Django over the years,
but I will sadly say that I feel it's been for the worst. I've
completely given up on trac, submitting patches, or even
participating. Now while some may not like my aggressive tacts (e.g.
James) that doesn't mean what I've brought to the table hasn't been
needed. For the last year or two all I've seen on the trac whenever I
took up the time to write a patch, or even submit a ticket, was a
closure of "wontfix" for some, honestly, stupid reason. It just isnt
worth my time to submit the same ticket 3 times just so its "correct",
or it fits everyones needs. Tickets are not patches, and they
shouldn't be treated like "if this one isnt accepted, create a new
one".

I think there's a split within the Django core team right now and I
strongly believe that unless you can tirelessly convince a core
developer (no matter how large the following), a feature is not going
to make it into mainline. This to me is a serious issue when you're
talking about an open source, community contributed project. Sure, the
core team does a large amount of the work, but not without help from
the community. I'll take this back to my old analogy, not everyone is
building a blog, and if they are, they can go use WordPress. Many,
many things have gone ignored for far too long. I love Malcom's ORM
refactor, but that was at a standstill for I don't know how long, and
that entire time any patch which was related to improvements to the
ORM was ignored simply stating "we're working on a refactor of the
ORM". This philosophy seems to continue still today.

Just recently there was a post about "High Level Talk About
Django" (or whatever it was called). Now while the thread didn't make
a whole lot of sense in general, it was just an attempt to gather some
ideas, and brainstorm. Immediately it was shut down by the core
developers.

What frustrates me even more is all of this pony talk. If there's one
thing I dislike it's Django's philosophy that "if it can be done 3rd
party, do it", yet even the simplest things, like the template engine,
have better 3rd party implementations (Jinja2). Django still doesn't
have migrations. It still doesn't have dependancies. It's seriously
lacking in many areas which other (albeit lesser) alternatives such as
Rails have made available for far too long. Now while there's great
3rd party apps for things like this (South), and there's a few
mediocre sites to find pieces of code (Django Snippets), this doesn't
solve the problem which is really going on in Django: The community
cant contribute beyond what the core team deems necessary.

For me, I've entirely given up on trying to give back to Django. I've
written enormous amounts of questionable code simply so I didn't have
to patch Django, or even bother dealing w/ the process of Django's
development. Monkey patching, ugly metaclass hacks, you name it.
Anything that's made it easier to avoid this "process", has made it
easier for us to develop. I continue to build these "ponies", but that
doesn't make them any easier to integrate in Django.

All in all, I think some things have been ignored for far too long.
Simple things, again, like migrations, JSON and RESTful utilities, and
even the tools to make development easier (the debug toolbar hasn't
been around that long). Yet so much time is spent on things like
refactoring the admin (while it's useful, in the big picture, its not
flexible, and never can be), the template system, and many other
things which have been done and done again by other people.

Again, this is just my opinion, and I do know that many share it. This
has been one of the largest outcries of people I've seen in a while. I
honestly can't see that I see a fork succeeding, but I would
definitely like to see what can happen to make the "process"
friendlier to people like myself and some of the others who have
posted here. Really, for me, I just don't (nor do I want to) have the
time to keep up on Django's process. If I see a bug, I want to let
people know, as easily as possible. If I think of a must-have feature,
I want it to be shot down for a real reason.

Post by Simone Federici
The work of the core team is outstanding and I find that the process of
development
is to be taken as an example.
Unfortunately, customers often want features, but we are programmers,
engineers,
and we know who we are and what is our role.
Compatibility is strongly important when choosing a tool.
Otherwise who would have used? only geeks?
Imagine if we can use frameworks that between one release and another
introduce incompatibility. no thanks.
thanks guys
S

Post by Stephen Wolff
I feel quite sad reading this thread. Good luck completing 1.2. I only wish
I had time and energy to contribute. I suggest the core team ignore the
thread for now if at all possible.

On Apr 17, 5:35 am...

For the record, there are 62 tickets marked ready for checkin, not 400
[1]. 29 of those are documentation and translation patches (5 of which
are specifically marked for inclusion in 1.2).
[1]
http://code.djangoproject.com/query?status=new&status=assigned&status...
On top of that, the Ready For Checkin status doesn't mean that a
member of the core team has reviewed a patch. It means that someone --
anyone -- thinks the patch is ready for checkin. There's no guarantee
that a Ready For Checkin patch is *actually* ready for checkin. If you
do a survey of the Ready For Checkin patches, you'll find tickets that
don't have test cases, or add features that aren't documented, or make
a significant changes that haven't been discussed on django-dev. If I
were to sit down and work through that list, I guarantee I wouldn't
end up making 62 commits to trunk using the material that has been
provided on those tickets.
I would also point out the folly of looking at raw ticket counts.
Python (the language) has 1078 tickets in the "having patch" status,
and 96 in the "needing review" status. Does this mean that Python is a
project in crisis?
Yes, there is a ticket backlog. Yes, this means there is a lot of work
that needs to be done. What we need is people volunteering to actually
do that work. However, as I've already indicated in this thread, much
of that work could be done without any change in current project
policy - we just needs people to actually do the work.
My dream outcome would to be in the situation where I don't *ever*
have to spend time on Trac trying to work out if the ticket that has
been marked Ready For Checkin is *actually* ready for checkin. Give me
a rich vein of trunk ready tickets that has been reviewed by someone
whose reputation I know and trust, and believe me -- I will use it.

Healthy projects don't need a separately maintained fork/branch on
github or bitbucket just to ...

The flipside of this is that too many cooks spoil the broth. If we
want to maintain a high quality product, we can't just add a dozen new
developers to the core team.
I would also point out that even in projects that do have large teams
with the commit bit, access to the "core trunk" is generally only made
available to a restricted subset of the entire team. Alternatively,
some sort of code review process is used to ensure that multiple team
members (occasionally, specially blessed team members) check patches
before they are committed. There's a lot more to commit policies in
open source than raw team size.
Yours,
Russ Magee %-)
--
You received this message because you are subscribed to the Google Groups
"Django developers" g...
 --
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group, send email to
.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group athttp://groups.google.com/group/django-developers?hl=en.

[orokusaki]

Russell,

I apologize for the apparent argumentum ad nauseam. I am not trying to
be sly. I am just looking for open dialogue about ideas and I feel
like the door is closed and caucus is frowned upon. This is the only
way I feel like I can get any floor time. The tickets I create get
closed quickly, and if I open better, more refined ones, they're
closed as duplicates.

I really am not against, per se, the backwards compatibility
guidelines, and I understand that there would be just as many folks
upset if it were too lax. I just think that it ought to be revisited.
Call this abstract, but that is only because I don't have the
"correct" solution.

The 'full_clean' stuff is the only thing that really got me fired up.
My change was not backward incompatible. ModelForm._post_clean()
currently calls nearly the exact same code that is in Model.clean()
instead of simply calling Model.clean() which would fix the entire
problem in less than "one hour".

If I'm wrong, then explain to me how it breaks backward compatibility
instead of simply erasing the docs. I can't help but feel like you did
this to punish me for not following protocol, since you've not given
my tickets more that 3 minutes. I've spent much more than 18 months
thinking about a lot of things, but I'm always open to suggestion from
eager hellers with a vested interest in helping me solve a problem.

[Russell Keith-Magee]

Post by orokusaki
Russell,
I apologize for the apparent argumentum ad nauseam. I am not trying to
be sly. I am just looking for open dialogue about ideas and I feel
like the door is closed and caucus is frowned upon. This is the only
way I feel like I can get any floor time. The tickets I create get
closed quickly, and if I open better, more refined ones, they're
closed as duplicates.
I really am not against, per se, the backwards compatibility
guidelines, and I understand that there would be just as many folks
upset if it were too lax. I just think that it ought to be revisited.
Call this abstract, but that is only because I don't have the
"correct" solution.
The 'full_clean' stuff is the only thing that really got me fired up.
My change was not backward incompatible. ModelForm._post_clean()
currently calls nearly the exact same code that is in Model.clean()
instead of simply calling Model.clean() which would fix the entire
problem in less than "one hour".
If I'm wrong, then explain to me how it breaks backward compatibility
instead of simply erasing the docs. I can't help but feel like you did
this to punish me for not following protocol, since you've not given
my tickets more that 3 minutes. I've spent much more than 18 months
thinking about a lot of things, but I'm always open to suggestion from
eager hellers with a vested interest in helping me solve a problem.

And if you want that answer, you've been told *many* times what you need to do.

If you choose to think of this as punishment for not following
protocol, that's up to you. I prefer to think of it as not encouraging
developers to engage in practices that we know (from experience) to be
counterproductive to the development process.

Yours,
Russ Magee %-)

[orokusaki]

Russell,

With all respect, you still haven't addressed my main concern: You
told me that it was because of backward compatibility that this simple
change couldn't be put in the trunk. It is backward compatible. If I'm
wrong, it would suffice to have a simple explanation of what it
breaks.

Post by Russell Keith-Magee

Post by orokusaki
Russell,
I apologize for the apparent argumentum ad nauseam. I am not trying to
be sly. I am just looking for open dialogue about ideas and I feel
like the door is closed and caucus is frowned upon. This is the only
way I feel like I can get any floor time. The tickets I create get
closed quickly, and if I open better, more refined ones, they're
closed as duplicates.
I really am not against, per se, the backwards compatibility
guidelines, and I understand that there would be just as many folks
upset if it were too lax. I just think that it ought to be revisited.
Call this abstract, but that is only because I don't have the
"correct" solution.
The 'full_clean' stuff is the only thing that really got me fired up.
My change was not backward incompatible. ModelForm._post_clean()
currently calls nearly the exact same code that is in Model.clean()
instead of simply calling Model.clean() which would fix the entire
problem in less than "one hour".
If I'm wrong, then explain to me how it breaks backward compatibility
instead of simply erasing the docs. I can't help but feel like you did
this to punish me for not following protocol, since you've not given
my tickets more that 3 minutes. I've spent much more than 18 months
thinking about a lot of things, but I'm always open to suggestion from
eager hellers with a vested interest in helping me solve a problem.

And if you want that answer, you've been told *many* times what you need to do.
If you choose to think of this as punishment for not following
protocol, that's up to you. I prefer to think of it as not encouraging
developers to engage in practices that we know (from experience) to be
counterproductive to the development process.
Yours,
Russ Magee %-)
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group athttp://groups.google.com/group/django-developers?hl=en.

[Richard Laager]

Post by orokusaki
With all respect, you still haven't addressed my main concern: You
told me that it was because of backward compatibility that this simple
change couldn't be put in the trunk. It is backward compatible. If I'm
wrong, it would suffice to have a simple explanation of what it
breaks.

I'd like to second this question. orokusaki suggested a couple of things
in ticket #13100, but I'm seconding specifically this comment:
http://code.djangoproject.com/ticket/13100#comment:8

This is a serious bug/missing feature for my company's application and I
don't see an obvious problem, backwards-compatibility or otherwise, with
the suggested fix.

Richard


orokusaki: Instead of copying-and-pasting the function before and after
your change, could you generate a diff (ideally against Django SVN's
HEAD)? That's a more natural format for developers to review and it
makes it easier for them to commit. (For these reasons, it's the normal
way to submit patches in most open source projects.) Making life easy
for the core developers is a key step in getting your ticket addressed
(be it with Django or another project), and it's something Russell
reiterated in the video.

The basic steps would be:
1) svn co http://code.djangoproject.com/svn/django/trunk/ django-trunk
2) Make the change you listed.
3) svn diff > ~/13100-full-clean-dry.diff
4) Upload 13100-full-clean-dry.diff to the ticket.

I could easily do this (and will if it's necessary), but I thought it
better to help you to do it.

[James Bennett]

Post by Richard Laager

Post by orokusaki
With all respect, you still haven't addressed my main concern: You
told me that it was because of backward compatibility that this simple
change couldn't be put in the trunk. It is backward compatible. If I'm
wrong, it would suffice to have a simple explanation of what it
breaks.

I'd like to second this question. orokusaki suggested a couple of things
http://code.djangoproject.com/ticket/13100#comment:8

The difference between how ModelForm works and how Model works is
that, if you're overriding clean() on a ModelForm subclass, you don't
automatically get uniqueness validation -- you have to call up to the
parent clean(), or manually apply the uniqueness validation in your
own clean().

In Django 1.0 and 1.1, this is documented behavior:

http://docs.djangoproject.com/en/1.0/topics/forms/modelforms/#overriding-the-clean-method
http://docs.djangoproject.com/en/1.1/topics/forms/modelforms/#overriding-the-clean-method

As such, changing ModelForm to always behave identically to, or to
always call, Model.full_clean() would have to change documented
behavior. We can't do that in the 1.1 -> 1.2 jump, and for future
consideration trying to force them to behave identically is probably
unworkable (better would be to come up with API that lets you
explicitly control uniqueness validation).

This is why that ticket has been changed to a documentation issue: the
wording of the documentation with respect to ModelForm and model
validation is pretty bad right now, and needs to be cleaned up for the
1.2 release. And this is why for a month now multiple committers have
been saying that the proposed code changes are backwards-incompatible:
ModelForm.clean() and Model.full_clean() *cannot* be made to function
identically right now without changing documented behavior.

And for the record, my own frustration on that ticket boils down to a
simple thing: Joseph pointed out there was a backwards-compatibility
issue, and opted to salvage the most workable solution by changing it
to a documentation issue. The reporter reverted that. Russell chimed
in and pointed out that Joseph was probably right and set the ticket
back to a documentation issue. At that point our intrepid bug reporter
could've gotten all the discussion he wanted by paying attention to
something he'd been told multiple times, and which is clearly pointed
out in the contributing docs we encourage everyone to read as they
dive in: if you don't like the decision a committer made on a ticket,
start a thread here on the dev list to talk about it. Instead he
opened duplicate tickets, ranted in the tracker, insulted people, and
generally turned the whole thing into a big radioactive mess that
nobody wanted to touch with a ten-foot pole.

And with that I'm going to bow out of this thread; Jacob's already
posted a separate message to collect concrete suggestions, and that's
the discussion I plan to pay attention to, since I think this one's
pretty much boiled down to the same people endlessly saying the same
things at each other and expecting different results.

[orokusaki]

Ok, problem solved:

``Model.full_clean()``

def full_clean(self, exclude=None, validate_unique=True):
"""
Calls clean_fields, clean, and validate_unique, on the model,
and raises a ``ValidationError`` for any errors that occured.
"""
errors = {}
if exclude is None:
exclude = []

try:
self.clean_fields(exclude=exclude)
except ValidationError, e:
errors = e.update_error_dict(errors)

# Form.clean() is run even if other validation fails, so do
the
# same with Model.clean() for consistency.
try:
self.clean()
except ValidationError, e:
errors = e.update_error_dict(errors)

# Run unique checks, but only for fields that passed
validation.
for name in errors.keys():
if name != NON_FIELD_ERRORS and name not in exclude:
exclude.append(name)

if validate_unique:
try:
self.validate_unique(exclude=exclude)
except ValidationError, e:
errors = e.update_error_dict(errors)

if errors:
raise ValidationError(errors)


``ModelForm._post_clean()``

def _post_clean(self):
exclude = self._get_validation_exclusions()
opts = self._meta

# Update the model instance with self.cleaned_data.
self.instance = construct_instance(self, self.instance,
opts.fields, opts.exclude)

# Clean the model instance and catch any and all fields.
try:
self.instance.full_clean(exclude=exclude,
validate_unique=self._validate_unique)
except ValidationError, e:
# Add the field errors in and NON_FIELD_ERRORS here (I
don't know the ins and outs of how `ValidationError` works).


This single change allows for so much more to be done in the model
layer instead of repetitive views. It allows you to actually do what
the docs say and use `Model.clean()` to do necessary additions to an
instance before it's saving, and without regards to data integrity,
because **this allows for you to wrap `Model.full_clean()` in a
transaction** and trust that it will always be used. I'm no RoR fan,
but Rails' policy of keeping data related logic inside of models is a
very wise one.

Post by James Bennett

Post by Richard Laager

Post by orokusaki
With all respect, you still haven't addressed my main concern: You
told me that it was because of backward compatibility that this simple
change couldn't be put in the trunk. It is backward compatible. If I'm
wrong, it would suffice to have a simple explanation of what it
breaks.

I'd like to second this question. orokusaki suggested a couple of things
http://code.djangoproject.com/ticket/13100#comment:8

The difference between how ModelForm works and how Model works is
that, if you're overriding clean() on a ModelForm subclass, you don't
automatically get uniqueness validation -- you have to call up to the
parent clean(), or manually apply the uniqueness validation in your
own clean().
http://docs.djangoproject.com/en/1.0/topics/forms/modelforms/#overrid...http://docs.djangoproject.com/en/1.1/topics/forms/modelforms/#overrid...
As such, changing ModelForm to always behave identically to, or to
always call, Model.full_clean() would have to change documented
behavior. We can't do that in the 1.1 -> 1.2 jump, and for future
consideration trying to force them to behave identically is probably
unworkable (better would be to come up with API that lets you
explicitly control uniqueness validation).
This is why that ticket has been changed to a documentation issue: the
wording of the documentation with respect to ModelForm and model
validation is pretty bad right now, and needs to be cleaned up for the
1.2 release. And this is why for a month now multiple committers have
ModelForm.clean() and Model.full_clean() *cannot* be made to function
identically right now without changing documented behavior.
And for the record, my own frustration on that ticket boils down to a
simple thing: Joseph pointed out there was a backwards-compatibility
issue, and opted to salvage the most workable solution by changing it
to a documentation issue. The reporter reverted that. Russell chimed
in and pointed out that Joseph was probably right and set the ticket
back to a documentation issue. At that point our intrepid bug reporter
could've gotten all the discussion he wanted by paying attention to
something he'd been told multiple times, and which is clearly pointed
out in the contributing docs we encourage everyone to read as they
dive in: if you don't like the decision a committer made on a ticket,
start a thread here on the dev list to talk about it. Instead he
opened duplicate tickets, ranted in the tracker, insulted people, and
generally turned the whole thing into a big radioactive mess that
nobody wanted to touch with a ten-foot pole.
And with that I'm going to bow out of this thread; Jacob's already
posted a separate message to collect concrete suggestions, and that's
the discussion I plan to pay attention to, since I think this one's
pretty much boiled down to the same people endlessly saying the same
things at each other and expecting different results.
--
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group athttp://groups.google.com/group/django-developers?hl=en.

[Jacob Kaplan-Moss]

When I apply this patch I get six test failures.

Jacob

[Richard Laager]

In the end, *my* requirement is that I have *some place* to put
validation code that 1) can see the whole model instance, 2) will be run
from the admin interface, and 3) will return nice validation failures to
the user (not throw exceptions that will give the user a 500 error and
send me an email).

A) Is this an unreasonable pony? If so, why?
B) If not, how can I implement this such that it will get accepted?

I'd like to have it in for 1.2 if possible, as the model validation
interfaces are new. Once released, there will be more
backwards-compatibility guarantees to maintain.

Post by James Bennett

Post by Richard Laager

Post by orokusaki
With all respect, you still haven't addressed my main concern: You
told me that it was because of backward compatibility that this simple
change couldn't be put in the trunk. It is backward compatible. If I'm
wrong, it would suffice to have a simple explanation of what it
breaks.

I'd like to second this question. orokusaki suggested a couple of things
http://code.djangoproject.com/ticket/13100#comment:8

The difference between how ModelForm works and how Model works is
that, if you're overriding clean() on a ModelForm subclass, you don't
automatically get uniqueness validation -- you have to call up to the
parent clean(), or manually apply the uniqueness validation in your
own clean().

Thank you for this explanation.

orokusaki noted in ticket #13100:
"The only difference between its implementation and
ModelForm?._post_clean() is the internal check it makes before running
validate_unique()."

So is the actual issue here that naively calling Model.full_clean() will
always run Model.validate_unique(), when the existing
ModelForm._post_clean() code only calls Model.validate_unique() when
self._validate_unique?

If so, Model.full_clean() is new in 1.2. So, could we just add a kwarg
like this (or similar)?
def full_clean(self, exclude=None, validate_unique=True)

Then, it would be modified to only call Model.validate_unique if the
validate_unique argument was True.

Then, you could call Model.full_clean() from ModelForm._post_clean(),
passing self._validate_unique and preserve the same behavior.

Alternatively, could we add another function to Model that allows for
whole-model validation but does not call Model.validate_unique() and
then call that from ModelForm._post_clean() instead of calling
Model.full_clean()? Of course, Model.full_clean() would have to call
this new validation function as well.

Richard

[David Cramer]

Realizing my original statement I was regarding this thread, in this
thread, it's obvious that this has gone completely off track. I might
have to take back everything I thought about this being useful.

If you want to address a SPECIFIC concern, it makes sense to do that
under its own topic. Think of this mailing list like a forum, as,
after all, many of us browse it just like one. When a "thread" happens
to have 12 different topics it loses its value fast.

Post by Richard Laager
In the end, *my* requirement is that I have *some place* to put
validation code that 1) can see the whole model instance, 2) will be run
from the admin interface, and 3) will return nice validation failures to
the user (not throw exceptions that will give the user a 500 error and
send me an email).
A) Is this an unreasonable pony? If so, why?
B) If not, how can I implement this such that it will get accepted?
I'd like to have it in for 1.2 if possible, as the model validation
interfaces are new. Once released, there will be more
backwards-compatibility guarantees to maintain.

Post by James Bennett

Post by Richard Laager

Post by orokusaki
With all respect, you still haven't addressed my main concern: You
told me that it was because of backward compatibility that this simple
change couldn't be put in the trunk. It is backward compatible. If I'm
wrong, it would suffice to have a simple explanation of what it
breaks.

I'd like to second this question. orokusaki suggested a couple of things
http://code.djangoproject.com/ticket/13100#comment:8

The difference between how ModelForm works and how Model works is
that, if you're overriding clean() on a ModelForm subclass, you don't
automatically get uniqueness validation -- you have to call up to the
parent clean(), or manually apply the uniqueness validation in your
own clean().

Thank you for this explanation.
"The only difference between its implementation and
ModelForm?._post_clean() is the internal check it makes before running
validate_unique()."
So is the actual issue here that naively calling Model.full_clean() will
always run Model.validate_unique(), when the existing
ModelForm._post_clean() code only calls Model.validate_unique() when
self._validate_unique?
If so, Model.full_clean() is new in 1.2. So, could we just add a kwarg
like this (or similar)?
        def full_clean(self, exclude=None, validate_unique=True)
Then, it would be modified to only call Model.validate_unique if the
validate_unique argument was True.
Then, you could call Model.full_clean() from ModelForm._post_clean(),
passing self._validate_unique and preserve the same behavior.
Alternatively, could we add another function to Model that allows for
whole-model validation but does not call Model.validate_unique() and
then call that from ModelForm._post_clean() instead of calling
Model.full_clean()? Of course, Model.full_clean() would have to call
this new validation function as well.
Richard
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group athttp://groups.google.com/group/django-developers?hl=en.

[Richard Laager]

Post by Richard Laager
In the end, *my* requirement is that I have *some place* to put
validation code that 1) can see the whole model instance, 2) will be run
from the admin interface, and 3) will return nice validation failures to
the user (not throw exceptions that will give the user a 500 error and
send me an email).

It's looking like this is a case of user error. I think my Django 1.2
checkout is simply too old. As I went to build a test case with HEAD, I
found it works to define clean(), as documented. Sorry for the noise.
I'll come back if I find a real bug.

Richard

[Jacob Kaplan-Moss]

Post by orokusaki
With all respect, you still haven't addressed my main concern: You
told me that it was because of backward compatibility that this simple
change couldn't be put in the trunk. It is backward compatible. If I'm
wrong, it would suffice to have a simple explanation of what it
breaks.

You've been told by three separate developers now that it's not
backwards compatible. It's time for you to trust that we know what
we're talking about and move on.

Jacob

[orokusaki]

Jacob,

With respect, If I simply "trusted" folks, I would be:

1) making exactly 120k less per year, as my previous employers told me
to "trust" them right before they went out of business and fired
everyone
2) a lot less intelligent than I am
3) ignoring the advice of Benjamin Franklin "it is the first
responsibility of every citizen to question authority."
4) committing a fallacy of defective induction.

I'll stick to the broad advice of those who created the country I live
in by convention, and ignore the other reasons.

Post by Jacob Kaplan-Moss

Post by orokusaki
With all respect, you still haven't addressed my main concern: You
told me that it was because of backward compatibility that this simple
change couldn't be put in the trunk. It is backward compatible. If I'm
wrong, it would suffice to have a simple explanation of what it
breaks.

You've been told by three separate developers now that it's not
backwards compatible. It's time for you to trust that we know what
we're talking about and move on.
Jacob
--
You received this message because you are subscribed to the Google Groups "Django developers" group.
For more options, visit this group athttp://groups.google.com/group/django-developers?hl=en.

[Mike Scott]

I agree almost whole-heartedly with the perception that David
portrays. His feelings almost mirror mine. Albeit I haven't submitted
contributions to the django development process I've been involved
with a number of issues and come away with similar feelings viewing
the process.

I love what those who do contribute do, and I whole-heartedly stand
behind the ideals that the core team produce, but I do feel that the
"community" aspect of django is slipping.

Post by David Cramer
I just want to throw my 2 cents into the ring here. I'm not against a
fork, but at the same time I want to see the Django mainline progress.
However, let me tell you my story, and how I've seen the Django
development process over the years.
I started with Django 4 years ago. It was cool, shiny, and let us get
up and running. At the time, were were one of the largest Django
installations. We had needs, and some of those needs were met.
However, many were not. We were the only ones furiously trying to get
the core team to accept our patches, ideas, etc.. Now while I'm not
going to say every idea we had was right for Django, there were in
fact many that were great, and eventually have made it into trunk.
There are also still many (lets take the classic #17 here), that still
haven't made it into trunk, even though people have been even more
aggressively pushing them lately. I honestly can only remember a
single patch that I've committed that has ever been fully integrated
into trunk (select_related refactor).
Now, the development process has changed with Django over the years,
but I will sadly say that I feel it's been for the worst. I've
completely given up on trac, submitting patches, or even
participating. Now while some may not like my aggressive tacts (e.g.
James) that doesn't mean what I've brought to the table hasn't been
needed. For the last year or two all I've seen on the trac whenever I
took up the time to write a patch, or even submit a ticket, was a
closure of "wontfix" for some, honestly, stupid reason. It just isnt
worth my time to submit the same ticket 3 times just so its "correct",
or it fits everyones needs. Tickets are not patches, and they
shouldn't be treated like "if this one isnt accepted, create a new
one".
I think there's a split within the Django core team right now and I
strongly believe that unless you can tirelessly convince a core
developer (no matter how large the following), a feature is not going
to make it into mainline. This to me is a serious issue when you're
talking about an open source, community contributed project. Sure, the
core team does a large amount of the work, but not without help from
the community. I'll take this back to my old analogy, not everyone is
building a blog, and if they are, they can go use WordPress. Many,
many things have gone ignored for far too long. I love Malcom's ORM
refactor, but that was at a standstill for I don't know how long, and
that entire time any patch which was related to improvements to the
ORM was ignored simply stating "we're working on a refactor of the
ORM". This philosophy seems to continue still today.
Just recently there was a post about "High Level Talk About
Django" (or whatever it was called). Now while the thread didn't make
a whole lot of sense in general, it was just an attempt to gather some
ideas, and brainstorm. Immediately it was shut down by the core
developers.
What frustrates me even more is all of this pony talk. If there's one
thing I dislike it's Django's philosophy that "if it can be done 3rd
party, do it", yet even the simplest things, like the template engine,
have better 3rd party implementations (Jinja2). Django still doesn't
have migrations. It still doesn't have dependancies. It's seriously
lacking in many areas which other (albeit lesser) alternatives such as
Rails have made available for far too long. Now while there's great
3rd party apps for things like this (South), and there's a few
mediocre sites to find pieces of code (Django Snippets), this doesn't
solve the problem which is really going on in Django: The community
cant contribute beyond what the core team deems necessary.
For me, I've entirely given up on trying to give back to Django. I've
written enormous amounts of questionable code simply so I didn't have
to patch Django, or even bother dealing w/ the process of Django's
development. Monkey patching, ugly metaclass hacks, you name it.
Anything that's made it easier to avoid this "process", has made it
easier for us to develop. I continue to build these "ponies", but that
doesn't make them any easier to integrate in Django.
All in all, I think some things have been ignored for far too long.
Simple things, again, like migrations, JSON and RESTful utilities, and
even the tools to make development easier (the debug toolbar hasn't
been around that long). Yet so much time is spent on things like
refactoring the admin (while it's useful, in the big picture, its not
flexible, and never can be), the template system, and many other
things which have been done and done again by other people.
Again, this is just my opinion, and I do know that many share it. This
has been one of the largest outcries of people I've seen in a while. I
honestly can't see that I see a fork succeeding, but I would
definitely like to see what can happen to make the "process"
friendlier to people like myself and some of the others who have
posted here. Really, for me, I just don't (nor do I want to) have the
time to keep up on Django's process. If I see a bug, I want to let
people know, as easily as possible. If I think of a must-have feature,
I want it to be shot down for a real reason.

Post by Simone Federici
The work of the core team is outstanding and I find that the process of
development
is to be taken as an example.
Unfortunately, customers often want features, but we are programmers,
engineers,
and we know who we are and what is our role.
Compatibility is strongly important when choosing a tool.
Otherwise who would have used? only geeks?
Imagine if we can use frameworks that between one release and another
introduce incompatibility. no thanks.
thanks guys
S

Post by Stephen Wolff
I feel quite sad reading this thread. Good luck completing 1.2. I only wish
I had time and energy to contribute. I suggest the core team ignore the
thread for now if at all possible.

On Apr 17, 5:35 am...

For the record, there are 62 tickets marked ready for checkin, not 400
[1]. 29 of those are documentation and translation patches (5 of which
are specifically marked for inclusion in 1.2).
[1]
http://code.djangoproject.com/query?status=new&status=assigned&status...
On top of that, the Ready For Checkin status doesn't mean that a
member of the core team has reviewed a patch. It means that someone --
anyone -- thinks the patch is ready for checkin. There's no guarantee
that a Ready For Checkin patch is *actually* ready for checkin. If you
do a survey of the Ready For Checkin patches, you'll find tickets that
don't have test cases, or add features that aren't documented, or make
a significant changes that haven't been discussed on django-dev. If I
were to sit down and work through that list, I guarantee I wouldn't
end up making 62 commits to trunk using the material that has been
provided on those tickets.
I would also point out the folly of looking at raw ticket counts.
Python (the language) has 1078 tickets in the "having patch" status,
and 96 in the "needing review" status. Does this mean that Python is a
project in crisis?
Yes, there is a ticket backlog. Yes, this means there is a lot of work
that needs to be done. What we need is people volunteering to actually
do that work. However, as I've already indicated in this thread, much
of that work could be done without any change in current project
policy - we just needs people to actually do the work.
My dream outcome would to be in the situation where I don't *ever*
have to spend time on Trac trying to work out if the ticket that has
been marked Ready For Checkin is *actually* ready for checkin. Give me
a rich vein of trunk ready tickets that has been reviewed by someone
whose reputation I know and trust, and believe me -- I will use it.

Healthy projects don't need a separately maintained fork/branch on
github or bitbucket just to ...

The flipside of this is that too many cooks spoil the broth. If we
want to maintain a high quality product, we can't just add a dozen new
developers to the core team.
I would also point out that even in projects that do have large teams
with the commit bit, access to the "core trunk" is generally only made
available to a restricted subset of the entire team. Alternatively,
some sort of code review process is used to ensure that multiple team
members (occasionally, specially blessed team members) check patches
before they are committed. There's a lot more to commit policies in
open source than raw team size.
Yours,
Russ Magee %-)
--
You received this message because you are subscribed to the Google Groups
"Django developers" g...
 --
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group, send email to
.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To unsubscribe from this group, send email to

...
read more »

[Russell Keith-Magee]

Post by David Cramer
I just want to throw my 2 cents into the ring here. I'm not against a
fork, but at the same time I want to see the Django mainline progress.
However, let me tell you my story, and how I've seen the Django
development process over the years.

I was going to do a point by point teardown, but then I realized that
I already have, at DjangoCon 2009:

http://djangocon.blip.tv/file/3043562/

The opening is light hearted; the hard details start about 5 minutes
in. By sheer coincidence, I think I addressed almost every one of the
tickets/API areas that you've mentioned in your post, as well as the
general question of why we reject certain ideas, and what you need to
do to get your pony into Django itself.

Yours,
Russ Magee %-)

[Luke Plant]

Post by Russell Keith-Magee
I was going to do a point by point teardown, but then I realized
http://djangocon.blip.tv/file/3043562/
The opening is light hearted; the hard details start about 5
minutes in. By sheer coincidence, I think I addressed almost every
one of the tickets/API areas that you've mentioned in your post,
as well as the general question of why we reject certain ideas,
and what you need to do to get your pony into Django itself.

I've never watched that before, but it's extremely good. Could we
perhaps link it from the page about contributing to Django, or maybe
in the FAQ under "But I’ve reminded you several times and you keep
ignoring my patch!" [1]. The material in the latter actually covers
some of the material in your talk and in this thread, but seeing a
person saying it with concrete examples is very helpful. We can then
point to official documentation when this comes up again.

Luke

[1] http://goo.gl/wF57

[Jerome Leclanche]

Post by Stephen Wolff
I feel quite sad reading this thread. Good luck completing 1.2. I only wish
I had time and energy to contribute. I suggest the core team ignore the
thread for now if at all possible.

Yes, let's ignore the users and all the other major issues - "Maybe if
we don't say anything they will shut up".

Again, with the unnecessary hostility, while core devs are actually
trying to help out and contribute to this thread. What was that for?

J

[George Sakkis]

Post by Russell Keith-Magee
For the record, there are 62 tickets marked ready for checkin, not 400
[1]. 29 of those are documentation and translation patches (5 of which
are specifically marked for inclusion in 1.2).
[1]http://code.djangoproject.com/query?status=new&status=assigned&status...
On top of that, the Ready For Checkin status doesn't mean that a
member of the core team has reviewed a patch. It means that someone --
anyone -- thinks the patch is ready for checkin. There's no guarantee
that a Ready For Checkin patch is *actually* ready for checkin. If you
do a survey of the Ready For Checkin patches, you'll find tickets that
don't have test cases, or add features that aren't documented, or make
a significant changes that haven't been discussed on django-dev. If I
were to sit down and work through that list, I guarantee I wouldn't
end up making 62 commits to trunk using the material that has been
provided on those tickets.

If the tracker fields are not to be trusted as authoritative, why give
public access to them in the first place ? The Python tracker allows
only developers to modify most fields, I guess Trac should have a way
to control access too.

Post by Russell Keith-Magee
I would also point out the folly of looking at raw ticket counts.
Python (the language) has 1078 tickets in the "having patch" status,
and 96 in the "needing review" status. Does this mean that Python is a
project in crisis?

For the record, if you count all tickets with patches, then Django has
992 (they drop to 616 after excluding those that need improvement,
documentation and tests and to 406 when considering only the
"accepted" and "ready for checkin" stage - assuming these numbers mean
anything). That's pretty close to the ticket count of a much larger in
size and complexity project.

Speaking of Python (the language) contribution process, I had the
recent pleasant experience of having a patch of mine accepted for
Python 2.7. It's not a bug fix, it's a new feature and so it could
have easily been ignored, postponed after the release or simply
dismissed as unnecessary but it wasn't; within two weeks since the
original submission and with great responsiveness and feedback from
the core dev that reviewed it, it was committed a few days before the
first beta. Quite a different experience from Django.

Post by Russell Keith-Magee
My dream outcome would to be in the situation where I don't *ever*
have to spend time on Trac trying to work out if the ticket that has
been marked Ready For Checkin is *actually* ready for checkin. Give me
a rich vein of trunk ready tickets that has been reviewed by someone
whose reputation I know and trust, and believe me -- I will use it.

Again, unless there is a good reason for giving public access to all
fields, make them accessible only to trusted members and let the
official tracker become this rich vein of trunk ready tickets. Even if
nothing else changes, we should at least be able to trust the report
counts and have a more accurate view of the project's status.

Post by Russell Keith-Magee

Post by George Sakkis
Healthy projects don't need a separately maintained fork/branch on
github or bitbucket just to review and apply patches. They open up
their gates and they invite more contributors to the development
process (in a controlled manner of course) so that they can keep up
with the increasing volume of external contributions.

The flipside of this is that too many cooks spoil the broth. If we
want to maintain a high quality product, we can't just add a dozen new
developers to the core team.
I would also point out that even in projects that do have large teams
with the commit bit, access to the "core trunk" is generally only made
available to a restricted subset of the entire team. Alternatively,
some sort of code review process is used to ensure that multiple team
members (occasionally, specially blessed team members) check patches
before they are committed. There's a lot more to commit policies in
open source than raw team size.

Agreed, that's why I stressed "in a controlled manner". The question
is what prevents the influx of new skilled and trustworthy blessed
members, the institution of code review policies and everything else
that a large project needs to flourish.

George

[Russell Keith-Magee]

Post by George Sakkis

Post by Russell Keith-Magee
For the record, there are 62 tickets marked ready for checkin, not 400
[1]. 29 of those are documentation and translation patches (5 of which
are specifically marked for inclusion in 1.2).
[1]http://code.djangoproject.com/query?status=new&status=assigned&status...
On top of that, the Ready For Checkin status doesn't mean that a
member of the core team has reviewed a patch. It means that someone --
anyone -- thinks the patch is ready for checkin. There's no guarantee
that a Ready For Checkin patch is *actually* ready for checkin. If you
do a survey of the Ready For Checkin patches, you'll find tickets that
don't have test cases, or add features that aren't documented, or make
a significant changes that haven't been discussed on django-dev. If I
were to sit down and work through that list, I guarantee I wouldn't
end up making 62 commits to trunk using the material that has been
provided on those tickets.

If the tracker fields are not to be trusted as authoritative, why give
public access to them in the first place ? The Python tracker allows
only developers to modify most fields, I guess Trac should have a way
to control access too.

It can, but we've configured it not to. Again, this is territory that
is well covered in django-dev history, but in brief: there is a
tradeoff. By leaving Trac open, it means anyone can contribute, but
the states aren't necessarily accurate. Alternatively, we can lock
down Trac, making the ticket states completely accurate, but it relies
upon trusted individuals dedicating the time to doing official triage
work -- which also means that there is an increased likelyhood that a
patch *won't* get triaged (or won't get triaged quickly), because only
a select few can actually do the work.

So - we need to make a choice. Which is better? Having a Trac that is
100% accurate whenever a judgement is made, but that only has 75% of
tickets triaged? Or a ticket that has 100% of tickets triaged, but is
only 75% accurate? We've opted for the latter, and rely upon crowd
effects (re-reporting of issues, repeated threads on django-users,
etc) to identify critical problems that require attention.

Post by George Sakkis
Speaking of Python (the language) contribution process, I had the
recent pleasant experience of having a patch of mine accepted for
Python 2.7. It's not a bug fix, it's a new feature and so it could
have easily been ignored, postponed after the release or simply
dismissed as unnecessary but it wasn't; within two weeks since the
original submission and with great responsiveness and feedback from
the core dev that reviewed it, it was committed a few days before the
first beta. Quite a different experience from Django.

The plural of anecdote is not data. I'm glad you had a good experience
with Python's ticket tracker. I'm sorry you've had a bad experience
with Django's. However, I could easily point at a number of tickets
that were closed fixed within hours of original reporting because they
caught the attention of a core developer at the right time. I can also
point at tickets in Python's ticket repository that have been
languishing for years.

The reason I know there are long lived tickets in Python is because
I've submitted some of them. In fact, the very first ticket that I
submitted to Python's tracker [1] took 9 months before it even got
triaged. It still hasn't been resolved, and I submitted it in 2006.
This doesn't mean that Python's development process is flawed. It
means that I haven't been actively pursuing a resolution to this
ticket inside the Python community.

[1] http://bugs.python.org/issue1521051

Post by George Sakkis

Post by Russell Keith-Magee

Post by George Sakkis
Healthy projects don't need a separately maintained fork/branch on
github or bitbucket just to review and apply patches. They open up
their gates and they invite more contributors to the development
process (in a controlled manner of course) so that they can keep up
with the increasing volume of external contributions.

The flipside of this is that too many cooks spoil the broth. If we
want to maintain a high quality product, we can't just add a dozen new
developers to the core team.
I would also point out that even in projects that do have large teams
with the commit bit, access to the "core trunk" is generally only made
available to a restricted subset of the entire team. Alternatively,
some sort of code review process is used to ensure that multiple team
members (occasionally, specially blessed team members) check patches
before they are committed. There's a lot more to commit policies in
open source than raw team size.

Agreed, that's why I stressed "in a controlled manner". The question
is what prevents the influx of new skilled and trustworthy blessed
members, the institution of code review policies and everything else
that a large project needs to flourish.

We already have a policy for giving people commit access in a
controlled manner [2], and lots of suggestions on the same page for
what you can do to contribute to prove you're worthy of commit access.
If anyone can provide concrete suggestions of any other ways we can
encourage skilled individuals to become long term contributors worthy
of consideration for commit access, I'm all ears.

[2] http://docs.djangoproject.com/en/1.1/internals/contributing/#commit-access

Yours,
Russ Magee %-)

[Chris]

Post by Russell Keith-Magee
I would also point out the folly of looking at raw ticket counts.
Python (the language) has 1078 tickets in the "having patch" status,
and 96 in the "needing review" status. Does this mean that Python is a
project in crisis?
Yes, there is a ticket backlog. Yes, this means there is a lot of work
that needs to be done.

lol talk about unfortunate paragraph transition

[George Sakkis]

Post by Kevin Howerton
The level of resistance I see to change or outsider code contribution
is an enormous de-motivator for people (like me) to want to make any
contributions in the first place. Why should I contribute a patch to
your flawed architecture if I'm going to be talked down to, ridiculed,
then eventually have the patch rejected because it breaks code in some
edge-use-case?

Good luck pushing backwards incompatible patches when as we speak
there are almost 400 open tickets with patches at accepted [1] and
"ready for checkin" [2] stage. Under these circumstances, backwards
compatibility is almost a red herring; the bigger issue IMO is the
increasing pile of bug fixes and solid, backwards compatible patches
languishing for months or years.

A fork that encouraged and achieved a faster submit-review-accept-
commit lifecycle, even with the same stability, maturity, and
longevity policies, could be a breath of fresh air.

George


[1]
http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&needs_better_patch=!1&needs_tests=!1&needs_docs=!1&has_patch=1&order=priority&stage=Accepted

[2]
http://code.djangoproject.com/query?status=new&status=assigned&status=reopened&needs_better_patch=!1&needs_tests=!1&needs_docs=!1&has_patch=1&order=priority&stage=Ready+for+checkin