Subversion, decentralized version control, and the future.

[Karl Fogel]

I've been wanting to post this for a while, but was waiting for the
dust from Linus Torvald's GIT talk to settle first (for those who
haven't seen it:

Eric
Raymond's recent post thanking the Subversion team gives me the excuse
I needed to finally sit down and write this :-). (Eric's post is at
http://subversion.tigris.org/servlets/ReadMsg?list=dev&msgNo=128106.)

In his talk, Torvalds explained why he thinks decentralized version
control systems (like GIT and Mercurial) are the way of the future,
and why he thinks Subversion got it all wrong. I think that's a
misanalysis, and will describe why below. Unfortunately, Torvalds
also indulged in a childish presentation style that distracted from
his useful technical criticisms of Subversion. Since I'd like to use
some of his arguments as a jumping-off point for thoughts on
Subversion's future, here they are in brief:

* Optimizing merging is as important as optimizing branching (if
not more so).

* Speed matters: when a common operation goes from thirty seconds
to half a second, that changes the whole way you work.

* Having all history locally (or at least as much history as you
need for a given operation) is useful.

* Reducing the thickness of the "commit access" wall is good for
development. Torvalds didn't make this argument terribly well,
so I'll try to restate what I think was his point:

The important question is, who can put changes into the
repository that the project is publishing releases from? This
should not be confused with commit access in the technical sense.
Instead, think of it this way: committing is just a way to
connect changes to other changes, and you shouldn't need my trust
in order to connect your changes to anything you want to connect
them to. The real question is, when and how do I include your
changes in my release? So the issue isn't commit access, it's
having trust networks and convenient methods of change selection.

When I talked to Brian Fitzpatrick about this, he listed three things
as top priorities:

- Faster. Subversion does need to be faster for many ops.
- Offline commits.
- Local branches.

I would add "better merging", but basically agree with Fitz (note that
we're getting much-improved merging in Subversion 1.5).

Now, since decentralized is formally a superset of centralized, one
way to get all those things is just to use a decentralized system.
That doesn't guarantee "faster", of course, but we already know that
Mercurial and GIT have good performance, so no problem there.

But there's another factor...

True decentralized systems are really hard for most people to wrap
their heads around.

Those of us who work in version control, who think about branching and
merging and change algebra and diff3/diff4 algorithms all day, often
forget this. One of Subversion's biggest advantages, and one of the
reasons it's taking over the world, is that it's really easy to
understand. There's a repository; you check stuff out; you modify the
stuff; you check it back in. Comprehensibility is a big part of
survivability: wherever Subversion goes from here, it must not become
so complex that it can't be explained in five minutes or less with no
questions from the audience. (I've witnessed enough explanations of
decentralized systems to know that their learning curve is generally a
bit steeper, though it may well be worth it.)

So we should acquire some of the characteristics of decentralized
systems (SVK will be a useful guide here, as will GIT and Mercurial
and other systems). But, in the words of Eric Gillespie in IRC just
now, we shouldn't morph into an "also-ran DVCS"; the result would be
hard to maintain and impossible to explain.

Sure, centralized VC can be viewed as a particular restricted mode of
decentralized VC. But in practice it won't work out very well to have
a natively decentralized VC that most users configure to be
centralized. For many organizations, including open source projects,
centralization is a feature: you want changes (and branches) to end up
in the master repository sooner rather than later, so they'll be
visible to everyone, so they'll be backed up, so they'll go through
the central hook system, etc. It focuses the community on a shared
object (Ben Collins-Sussman makes this argument in more detail at
http://blog.red-bean.com/sussman/?p=20).

A general tool configured to behave in a specific way is never quite
as natural to use as a tool designed for that specific use in the
first place. In other words, Subversion can -- will have to -- take
on some of the features of decentralized VC systems, but it will never
be as good a decentralized system as they are. By the same token, a
decentralized system can be configured to work like a centralized one,
but will never be as good at it as Subversion is. The trick for us is
to keep the centralization feature without some of the limitations
that have traditionally come with centralization.

Concretely, what does this mean?

One of Subversion's flaws (mea culpa) is that we didn't realize the
usefulness of having symmetrical functionality on the client and
server sides. The working copy should really be a repository, even if
it's not always going to store all the history available on the server
side (with some projects, you really can't, it's too big).

So we're going to need a working copy rewrite. We knew that; in fact
we've talked about rewriting the repository to use something like
Mercurial's revlog format, for various reasons, and about using that
kind of repository for working copies as well.

We also have to be faster. Fortunately, we've pretty much agreed,
IIRC, that we're willing to punt on subdirectory detachability in
working copies in order to get performance improvements.

And now I'm going to hand-wave on a lot of details. I don't mean to
start the Subversion 2.0 design thread now, just to offer some
thoughts on general goals. We don't need to let labels guide our
thinking ("We are a centralized system!" / "We are a decentralized
system!"). We do need to recognize that users are not interested in
becoming version control experts, and we need to pay close attention
to what they actually want, as opposed to what experts might want them
to want.

Case in point: what's the most popular feature added to Subversion
after the 1.0 release? Probably file locking (the ultimate
centralized feature, by the way). Yes, the heavy-duty developers wish
for better merging, and I don't blame them. But from watching users@
and irc.freenode.net/#svn, talking to companies that do Subversion
support, and from doing some Subversion consulting myself, I think
locking was actually a more important feature. (Of course, we have it
already, so that doesn't change anything about Subversion's future,
I'm just making a point about what's important to users.)

Subversion's phenomenal adoption rate (*) isn't due to being the only
game in town. We never were, if you count the proprietary systems,
and we're even less so now that the open source version control world
has become so fertile. The reason Subversion is taking over the world
is because it is tremendously user-focused, and because it provides
well-documented APIs that enable other developers to write software on
top of Subversion. We should copy what we need from the decentralized
systems, but remember that most users don't know or care whether a
system is centralized or decentralized -- their ideal system is one
they don't notice. Let's keep our eye on the ball, so they don't have
to.

-Karl

(*) http://subversion.tigris.org/svn-dav-securityspace-survey.html

[eg]

First of all, thanks to you and all the devs for creating and evolving
Subversion!

Second, thanks for the well thought out and well written analysis.

I have to chime in and agree with one major point you make which kept
getting lost amongst previous diatribes on decentralized (vs
centralized). That being that not everyone needs it.

I am sure there are many large organizations who are spread out over
many locations and time zones who are in desperate need of
decentralization. However, there are many of us who are not in that
demographic, and we just don't need the full decentralized model.
Smallish groups of developers who work in a single location for
example... how many of these exist, but have small voices because they
cannot claim to be a big project? Quite a few I would think.

I know I have looked and asked myself, "what benefit would these other
revision control options provide us", and I haven't really seen anything
that would make me switch yet. Again, this is not a knock against these
decentralized options. More a matter of finding and using the right tool
for the job at hand.

Post by Karl Fogel
Subversion's phenomenal adoption rate (*) isn't due to being the only
game in town. We never were, if you count the proprietary systems,
and we're even less so now that the open source version control world
has become so fertile. The reason Subversion is taking over the world
is because it is tremendously user-focused, and because it provides
well-documented APIs that enable other developers to write software on
top of Subversion.

I would also submit that a constantly updated set of well written
end-user documentation (i.e. the book, etc) has been a huge
factor/benefit... please don't undersell the significance of that.

[Dan Christian]

Would good caching give you 90% of the distributed speed win with 10%
of the effort?

DAV/serf is supposed to allow caching of reads using available apache
modules. Has anyone implemented this? How do you handle the
authentication issues (if not everything is readable by everybody)?

If this works, you could cache at the LAN level and get faster
checkout, history, blame, etc without storing the whole (potentially
large) repository.

-Dan C

P.S. apologies to egoots for taking the post in exactly the opposite
direction of his point.

Post by eg
I am sure there are many large organizations who are spread out over
many locations and time zones who are in desperate need of
decentralization.

[Karl Fogel]

Post by eg

Post by Karl Fogel
has become so fertile. The reason Subversion is taking over the world
is because it is tremendously user-focused, and because it provides
well-documented APIs that enable other developers to write software on
top of Subversion.

I would also submit that a constantly updated set of well written
end-user documentation (i.e. the book, etc) has been a huge
factor/benefit... please don't undersell the significance of that.

Oh, absolutely -- that was part of "user-focused" in my mind.

[Giovanni Bajo]

Post by Karl Fogel
So we're going to need a working copy rewrite. We knew that; in fact

And this time (if you allow me my humble two cent) get it right and cut the
development time down to 1/10th by using a more evolved language than C.
Python comes to mind, but anything would do.

[Justin Erenkrantz]

Post by Giovanni Bajo
And this time (if you allow me my humble two cent) get it right and cut the
development time down to 1/10th by using a more evolved language than C.
Python comes to mind, but anything would do.

One thing I'd hope we avoid with "Subversion 2.0" is second system
syndrome and such a desire smacks of it.

Moving away from C means we'd most likely remove all reasonable
extensibility mechanisms. One of the reasons that I view SVN as a
success is that we made it fairly simple for third-parties to
integrate with SVN - C is by far the best language for that.

Rewriting all of Subversion in Python/C++/Java/Ruby is also *way*
easier said than done. As we discussed at the summit, I don't have a
problem with the higher-level parts - such as the command-line client
- being implemented in Python which wraps around C libraries. But,
libsvn_wc, libsvn_repos, etc. only being accessible via Python? Yuck.
-- justin

[Ben Collins-Sussman]

Post by Justin Erenkrantz
One of the reasons that I view SVN as a
success is that we made it fairly simple for third-parties to
integrate with SVN - C is by far the best language for that.

And yet, despite this fact, teams have gone off and reimplemented
*everything* in Java and C#.

Maybe this is inevitable. Maybe we should just define svn 2.0 as a
specification with a test-suite to certify implementations. ;-)

[Karl Fogel]

Post by Ben Collins-Sussman

Post by Justin Erenkrantz
One of the reasons that I view SVN as a
success is that we made it fairly simple for third-parties to
integrate with SVN - C is by far the best language for that.

And yet, despite this fact, teams have gone off and reimplemented
*everything* in Java and C#.

Yes, and how many third-party tools are built on top of those
implementations?

Post by Ben Collins-Sussman
Maybe this is inevitable. Maybe we should just define svn 2.0 as a
specification with a test-suite to certify implementations. ;-)

No reason not to spec it out, but of course, users can't run a
specification -- we have to actually implement it too.

FWIW, I think C is fine. We've had our libraries (string, IO, etc)
developed for a while now. Full garbage collection would be nice, but
apr pools are pretty easy to work with. The advantages of Python or
Java or C# or insert-your-favorite-GC'd-high-level-language-here are
somewhat overstated. There is a slight development-time cost to using
C, but I think it's outweighed by the advantages Justin listed. Maybe
we should move to C++ (Daniel Berlin thinks so, IIRC).

(Of course, if anyone proposes rewriting in Lisp, I'm +1 :-) ).

-Karl

[Giovanni Bajo]

Post by Karl Fogel

Post by Ben Collins-Sussman
Maybe this is inevitable. Maybe we should just define svn 2.0 as a
specification with a test-suite to certify implementations. ;-)

No reason not to spec it out, but of course, users can't run a
specification -- we have to actually implement it too.
FWIW, I think C is fine. We've had our libraries (string, IO, etc)
developed for a while now. Full garbage collection would be nice, but
apr pools are pretty easy to work with. The advantages of Python or
Java or C# or insert-your-favorite-GC'd-high-level-language-here are
somewhat overstated. There is a slight development-time cost to using
C, but I think it's outweighed by the advantages Justin listed. Maybe
we should move to C++ (Daniel Berlin thinks so, IIRC).

While you find C programmers everywhere, you don't find C programmers
fluent in APR and your own libraries as easily. I myself always had
problems contributing to Subversion core exactly because of this. Just
recompiling everything needed to hack Subversion can take several hours
(and I read in a book somewhere that lowering this barrier can help
getting more contributors -- wait, was that YOUR book? :)).

On the contrary, I was able to start hacking on Mercurial after a few
minutes, with 0 seconds wasted in setting up a build environment.

Don't underestimate this issue. If you're going to rewrite wc, it's a
good time to fix it.

[Karl Fogel]

Post by Giovanni Bajo
While you find C programmers everywhere, you don't find C programmers
fluent in APR and your own libraries as easily. I myself always had
problems contributing to Subversion core exactly because of this. Just
recompiling everything needed to hack Subversion can take several
hours (and I read in a book somewhere that lowering this barrier can
help getting more contributors -- wait, was that YOUR book? :)).
On the contrary, I was able to start hacking on Mercurial after a few
minutes, with 0 seconds wasted in setting up a build environment.
Don't underestimate this issue. If you're going to rewrite wc, it's a
good time to fix it.

Hmmm, that's a good point. I forget the dependency burden, having
taken care of my own set up many years ago (haven't had to think about
it much since then).

In IRC just now, Mark Phippard just said something very wise:

"I maintain there will never be a 2.0 if we do not shut off all talk
about the language it will be written in and instead talk about the
changes that are needed and let the language work itself out"

Language is not the issue, behavior is. Once we know where we're
going, we can have a better conversation about language.

[egoots]

Post by Karl Fogel

Post by Giovanni Bajo
Don't underestimate this issue. If you're going to rewrite wc, it's a
good time to fix it.

Hmmm, that's a good point. I forget the dependency burden, having
taken care of my own set up many years ago (haven't had to think about
it much since then).

That's a double "hear, hear" for doing this under Windows.

[Peter Samuelson]

[Karl Fogel]

I forget the dependency burden, having taken care of my own set up
many years ago (haven't had to think about it much since then).

For any prospective developer reading this who is wondering about
installing dependencies, note that it _can_ be a pretty simple process:

1. Find a recent Debian or Ubuntu installation CD and install the OS.
2. Log in as root and install stuff:

# aptitude install build-essential subversion emacs21
# apt-get build-dep subversion

(The "emacs21" is not strictly necessary, but if you're going to
hack on Subversion, you will need a good text editor.)

This should get you most of the way there. There are details such as
the fact that Subversion uses a rather underhanded way to detect and
use Berkeley DB, which won't work with Debian's apr-util (a trivial
patch is at [0], but I'm not sure if it's suitable for non-Debian use).
Another detail is that the Debian package 'libserf-0-0-dev', which is
needed for building libsvn_ra_serf, is only available in unreleased OS
versions: Debian lenny or Ubuntu gusty/Universe.

[0] svn://svn.debian.org/pkg-subversion/trunk/debian/patches/fix-bdb-version-detection

I mention all this for information, not to contradict what others have
said about setting up a build environment. Certainly it is not this
simple on all OSes.

[David Glasser]

Post by Karl Fogel

Post by Ben Collins-Sussman
And yet, despite this fact, teams have gone off and reimplemented
*everything* in Java and C#.

Yes, and how many third-party tools are built on top of those
implementations?

I can't tell which implication you're trying to make by this
statement, but svnkit.com lists 17 programs using it on their front
page (and the Eclipse plugins are *very* popular; yes, theoretically
they can interface with JavaHL too, but I don't believe most people
bother...)

--dave

[Karl Fogel]

Post by David Glasser
I can't tell which implication you're trying to make by this
statement, but svnkit.com lists 17 programs using it on their front
page (and the Eclipse plugins are *very* popular; yes, theoretically
they can interface with JavaHL too, but I don't believe most people
bother...)

Thanks -- that's useful to know (and opposite to what I was implying,
but perhaps I'm in need of new facts).

[Blair Zajac]

Post by Justin Erenkrantz

Post by Giovanni Bajo
And this time (if you allow me my humble two cent) get it right and cut the
development time down to 1/10th by using a more evolved language than C.
Python comes to mind, but anything would do.

One thing I'd hope we avoid with "Subversion 2.0" is second system
syndrome and such a desire smacks of it.
Moving away from C means we'd most likely remove all reasonable
extensibility mechanisms. One of the reasons that I view SVN as a
success is that we made it fairly simple for third-parties to
integrate with SVN - C is by far the best language for that.
Rewriting all of Subversion in Python/C++/Java/Ruby is also *way*
easier said than done. As we discussed at the summit, I don't have a
problem with the higher-level parts - such as the command-line client
- being implemented in Python which wraps around C libraries. But,
libsvn_wc, libsvn_repos, etc. only being accessible via Python? Yuck.
-- justin

Agreed. I couldn't integrate the Subversion filesystem into a new asset
management system supporting thousands of RPC calls per second if it
wasn't in C and I couldn't link it to Ice or anything else I needed.

Regards,
Blair

[Giovanni Bajo]

Post by Blair Zajac

Post by Justin Erenkrantz

Post by Giovanni Bajo
And this time (if you allow me my humble two cent) get it right and cut the
development time down to 1/10th by using a more evolved language than C.
Python comes to mind, but anything would do.

One thing I'd hope we avoid with "Subversion 2.0" is second system
syndrome and such a desire smacks of it.
Moving away from C means we'd most likely remove all reasonable
extensibility mechanisms. One of the reasons that I view SVN as a
success is that we made it fairly simple for third-parties to
integrate with SVN - C is by far the best language for that.
Rewriting all of Subversion in Python/C++/Java/Ruby is also *way*
easier said than done. As we discussed at the summit, I don't have a
problem with the higher-level parts - such as the command-line client
- being implemented in Python which wraps around C libraries. But,
libsvn_wc, libsvn_repos, etc. only being accessible via Python? Yuck.
-- justin

Agreed. I couldn't integrate the Subversion filesystem into a new asset
management system supporting thousands of RPC calls per second if it
wasn't in C and I couldn't link it to Ice or anything else I needed.

Please don't overstate what I proposed. I'm speaking of wc only (no repos, no
ra, etc.). And only because libsvn_wc is going to be rewritten from scratch
anyway.

Right now, the wc specifications are "private" and third parties are supposed
to access wc only through the libsvn_wc API. I find that mostly inconvenient.
I'd rather the wc specifications to be public and official, and have a
testsuite to certify wc implementations. After that, there's no reason we
should code our wc implementation in C.

[Blair Zajac]

Post by Giovanni Bajo

Post by Blair Zajac

Post by Justin Erenkrantz

Post by Giovanni Bajo
And this time (if you allow me my humble two cent) get it right and cut the
development time down to 1/10th by using a more evolved language than C.
Python comes to mind, but anything would do.

One thing I'd hope we avoid with "Subversion 2.0" is second system
syndrome and such a desire smacks of it.
Moving away from C means we'd most likely remove all reasonable
extensibility mechanisms. One of the reasons that I view SVN as a
success is that we made it fairly simple for third-parties to
integrate with SVN - C is by far the best language for that.
Rewriting all of Subversion in Python/C++/Java/Ruby is also *way*
easier said than done. As we discussed at the summit, I don't have a
problem with the higher-level parts - such as the command-line client
- being implemented in Python which wraps around C libraries. But,
libsvn_wc, libsvn_repos, etc. only being accessible via Python? Yuck.
-- justin

Agreed. I couldn't integrate the Subversion filesystem into a new
asset management system supporting thousands of RPC calls per second
if it wasn't in C and I couldn't link it to Ice or anything else I
needed.

Please don't overstate what I proposed. I'm speaking of wc only (no
repos, no ra, etc.). And only because libsvn_wc is going to be rewritten
from scratch anyway.
Right now, the wc specifications are "private" and third parties are
supposed to access wc only through the libsvn_wc API. I find that mostly
inconvenient. I'd rather the wc specifications to be public and
official, and have a testsuite to certify wc implementations. After
that, there's no reason we should code our wc implementation in C.

Making the wc spec a document instead of just in the implementation would be a
good thing.

However, TortoiseSVN and all the other apps that use the C libsvn_wc API would
have to use bind to Python instead? Or the Subversion team would write
libsvn_wc in Python and somebody else would end up writing a C libsvn_wc,
because somebody is going to do it because it is needed. I would rather see us
do the C version and other people do the non-C versions.

Regards,
Blair

[Giovanni Bajo]

Post by Blair Zajac
Making the wc spec a document instead of just in the implementation
would be a good thing.

OK, we agree here. That's great since it's not been like this till now!

Post by Blair Zajac
However, TortoiseSVN and all the other apps that use the C libsvn_wc API
would have to use bind to Python instead? Or the Subversion team would
write libsvn_wc in Python and somebody else would end up writing a C
libsvn_wc, because somebody is going to do it because it is needed. I
would rather see us do the C version and other people do the non-C
versions.

Of course, if you think in terms of "binding", C has so many advantages that
it's impossible to compare it to anything.

But is a *binding* to libsvn_wc really required? What if we release a command
line application/interface to access the WC functionalities?

In my opinion, svnmerge.py that spawns svn works very well for *almost*
anything and requires less code, less dependencies, etc. I reckon that
svnmerge.py wouldn't have attracted so many contributors if people were
required to install the Python bindings of libsvn and to study what an
"editor" is before starting to hack.

The issues I found with spawning the svn command line client are:

1) Under Windows (thinking Tortoise here), there's a relatively high price to
pay to start a process.
2) svn does not export all the desired functionalities because its interface
is meant for users, not programmers.

I think both these points can be solved by producing a different application,
which speaks through stdin/stdout with a simple API (so that can be started
once and used for multiple commands). This application would be meant for
programmers only (eg: it would default to xml output for commands), and export
all required functionalities without worries about final users.

This way, no bindings are required at all, and you totally decouple the
libsvn_wc implementation language from binding issues.

[Dustin J. Mitchell]

In my opinion, svnmerge.py that spawns svn works very well for *almost* anything and requires less code, less dependencies, etc. I reckon that svnmerge.py wouldn't have attracted
so many contributors if people were required to install the Python bindings of libsvn and to study what an "editor" is before starting to hack.

On the other hand, svnmerge.py could probably handle a lot more "corner
cases" and be a lot faster if it were written against the Python
bindings instead of simply invoking 'svn'.

It's a tradeoff, for sure.

Dustin

[Giovanni Bajo]

Post by Dustin J. Mitchell

In my opinion, svnmerge.py that spawns svn works very well for *almost* anything and requires less code, less dependencies, etc. I reckon that svnmerge.py wouldn't have attracted
so many contributors if people were required to install the Python bindings of libsvn and to study what an "editor" is before starting to hack.

On the other hand, svnmerge.py could probably handle a lot more "corner
cases" and be a lot faster if it were written against the Python
bindings instead of simply invoking 'svn'.

As I said, this is because the "svn" executable as a command line API meant
for final users, not developers of tools. This can be fixed by having a
different application, "svn_cli" (or whatever) which implements a simple
text-based protocol over stdin/stdout; this protocol would expose the whole
API which is now accessible only through bindings.

At that point, the language in which svn_cli is written is irrelevant, and you
need zero setup from your favorite programming language to use it.

[Carsten Breuer]

Hi Giovanni,

Post by Giovanni Bajo
Please don't overstate what I proposed. I'm speaking of wc only (no
repos, no ra, etc.). And only because libsvn_wc is going to be rewritten
from scratch anyway.
Right now, the wc specifications are "private" and third parties are
supposed to access wc only through the libsvn_wc API. I find that mostly
inconvenient. I'd rather the wc specifications to be public and
official, and have a testsuite to certify wc implementations. After
that, there's no reason we should code our wc implementation in C.

I don't see any problem to write wc-api in C and create python
bindings then. Where is the problem?

Best Regards,



Carsten

[B. Smith-Mannschott]

Post by Justin Erenkrantz

Post by Giovanni Bajo
And this time (if you allow me my humble two cent) get it right and cut the
development time down to 1/10th by using a more evolved language than C.
Python comes to mind, but anything would do.

One thing I'd hope we avoid with "Subversion 2.0" is second system
syndrome and such a desire smacks of it.
Moving away from C means we'd most likely remove all reasonable
extensibility mechanisms. One of the reasons that I view SVN as a
success is that we made it fairly simple for third-parties to
integrate with SVN - C is by far the best language for that.
Rewriting all of Subversion in Python/C++/Java/Ruby is also *way*
easier said than done. As we discussed at the summit, I don't have a
problem with the higher-level parts - such as the command-line client
- being implemented in Python which wraps around C libraries. But,
libsvn_wc, libsvn_repos, etc. only being accessible via Python? Yuck.

True. I *do* think it would be worthwhile to consider embedding a
scripting language (Lua and Guile come to mind). This would allow
features that don't need the performance of raw C to be implemented
more concisely and maintainably. This approach has worked out pretty
well for EMACS.

// Ben

[Troy Curtis Jr]

Post by Giovanni Bajo

Post by Karl Fogel
So we're going to need a working copy rewrite. We knew that; in fact

And this time (if you allow me my humble two cent) get it right and cut the
development time down to 1/10th by using a more evolved language than C.
Python comes to mind, but anything would do.
--
Giovanni Bajo
---------------------------------------------------------------------

Yeah but he stated we needed to be faster! C->python certainly
wouldn't facilitate that! :)

Troy

[Giovanni Bajo]

Post by Troy Curtis Jr

Post by Giovanni Bajo

Post by Karl Fogel
So we're going to need a working copy rewrite. We knew that; in fact

And this time (if you allow me my humble two cent) get it right and cut the
development time down to 1/10th by using a more evolved language than C.
Python comes to mind, but anything would do.

Yeah but he stated we needed to be faster! C->python certainly
wouldn't facilitate that! :)

Speed has nothing to do with the language here, it's all about design and
algorithms. Mercurial is many times faster than Subversion in common
operations while being written in Python.

[Daniel Rall]

Post by Giovanni Bajo

Post by Troy Curtis Jr

Post by Giovanni Bajo

Post by Karl Fogel
So we're going to need a working copy rewrite. We knew that; in fact

And this time (if you allow me my humble two cent) get it right and cut the
development time down to 1/10th by using a more evolved language than C.
Python comes to mind, but anything would do.

Yeah but he stated we needed to be faster! C->python certainly
wouldn't facilitate that! :)

Speed has nothing to do with the language here, it's all about design and
algorithms. Mercurial is many times faster than Subversion in common
operations while being written in Python.

I agree with Giovanni's concept here. Last year at the Subversion
Developer Summit at Google, several of us suggested writing at least
the command-line binaries in higher level language (Python in
particular came up repeatedly) for Subversion 2.0.

One thing of note -- not all of Mercurial is written in straight
Python. It has several libraries written in C to handle certain hot
spots (base85, bdiff, and mpatch). Interestingly enough, its revlog
FS backend is written in Python.

[Carsten Breuer]

Post by Giovanni Bajo

Post by Troy Curtis Jr
Yeah but he stated we needed to be faster! C->python certainly
wouldn't facilitate that! :)

Speed has nothing to do with the language here, it's all about design
and algorithms. Mercurial is many times faster than Subversion in common
operations while being written in Python.

Of course you can do things worse in C and good in python.
Nonetheless, python as a scripting language is
of course a (little) bit slower. It's funny how
people compares things that are not comparable ;-).
Isn't php also faster then C <gr&d>.

Just my two cents,



Carsten

[Blair Zajac]

Post by Giovanni Bajo

Post by Troy Curtis Jr

Post by Giovanni Bajo

Post by Karl Fogel
So we're going to need a working copy rewrite. We knew that; in fact

And this time (if you allow me my humble two cent) get it right and cut the
development time down to 1/10th by using a more evolved language than C.
Python comes to mind, but anything would do.

Yeah but he stated we needed to be faster! C->python certainly
wouldn't facilitate that! :)

Speed has nothing to do with the language here, it's all about design
and algorithms. Mercurial is many times faster than Subversion in common
operations while being written in Python.

I would guess that's because the Subversion working copy is loggy. I
don't know about the Mercurial working copy, but does it provide the
same features as the Subversion working copy?

Blair

[Eric S. Raymond]

Post by Giovanni Bajo

Post by Karl Fogel
So we're going to need a working copy rewrite. We knew that; in fact

And this time (if you allow me my humble two cent) get it right and cut the
development time down to 1/10th by using a more evolved language than C.
Python comes to mind, but anything would do.

I concur. It's 2007, which is way too late in the game for you guys to be
spending your time chasing memory-allocation bugs. Mercurial is an
existance proof that Python is fast enough for a production-quality VCS;
learn from it!

[Carsten Breuer]

Hi Eric, hi all,

Post by Eric S. Raymond

Post by Giovanni Bajo

Post by Karl Fogel
So we're going to need a working copy rewrite. We knew that; in fact

And this time (if you allow me my humble two cent) get it right and cut the
development time down to 1/10th by using a more evolved language than C.
Python comes to mind, but anything would do.

I concur. It's 2007, which is way too late in the game for you guys to be
spending your time chasing memory-allocation bugs.

Do we? Memory-allocation bugs are always a problem of discipline and
bad coding style. IMHO a good programmer doesn't need something
like a garbage collection. He knows what he have allocated.
With C++ we can wrap memory allocations in classes that handle memory
allocations correct. So perhaps it is time to switch over to C++.


Best Regards,



Carsten

[Karl Fogel]

If I may make a meta-post:

We're getting into choosing the programming language, before we have
the design, and as many have pointed out, that's backwards. That's
fine, the thread can go that way, I'm not trying to shut it down. But
my main goal in posting was to confirm that the ideas about 2.0 that
we had at the summit last October had solidified into a basic
consensus, which seems to be the case. Now, 1.5 is probably more
urgent... :-)

-Karl

Post by Carsten Breuer
Hi Eric, hi all,

Post by Eric S. Raymond

Post by Giovanni Bajo

Post by Karl Fogel
So we're going to need a working copy rewrite. We knew that; in fact

And this time (if you allow me my humble two cent) get it right and
cut the development time down to 1/10th by using a more evolved
language than C. Python comes to mind, but anything would do.

I concur. It's 2007, which is way too late in the game for you guys
to be spending your time chasing memory-allocation bugs.

Do we? Memory-allocation bugs are always a problem of discipline and
bad coding style. IMHO a good programmer doesn't need something
like a garbage collection. He knows what he have allocated.
With C++ we can wrap memory allocations in classes that handle memory
allocations correct. So perhaps it is time to switch over to C++.
Best Regards,
Carsten
---------------------------------------------------------------------

[Eric S. Raymond]

Post by Carsten Breuer
Do we? Memory-allocation bugs are always a problem of discipline and
bad coding style.

No, they're primarily a bug that arises from not baving proper dynamic memory
management. Don't lecture me about C++ -- I learned better than this
writing LISP back in the 1970s back when C++ was not yet even a gleam in
Bjarne Stroustrup's eye, then I paid two decades' dues writing C because
hardware was not yet inexpensive enough to support LISP-like languages
outside of academia. I can make a pretty good guess at your age by
the way you write, and my guess is I've been clued in about these
issues longer than you've been walking and talking.

I'm coding C++ right now on Battle For Wesnoth. It's an ugly, bloated,
over-complex language -- an octopus made by nailing extra legs onto a
dog -- and it encourages ugly, bloated, over-complex architectures. I
am pretty sure the Subversion architects know better than to go there.

The "discipline" you speak of has become a *waste of time* for anybody
not doing kernel or realtime work. With menory and cycles so cheap, p
programmer hours are best spent on solving real problems, not on low-level
resource bookeeping.

[Karl Fogel]

Post by Eric S. Raymond
No, they're primarily a bug that arises from not baving proper dynamic memory
management. Don't lecture me about C++ -- I learned better than this
writing LISP back in the 1970s back when C++ was not yet even a gleam in
Bjarne Stroustrup's eye, then I paid two decades' dues writing C because
hardware was not yet inexpensive enough to support LISP-like languages
outside of academia. I can make a pretty good guess at your age by
the way you write, and my guess is I've been clued in about these
issues longer than you've been walking and talking.

Uh, ungh? (Does that kind of ad hominem putdown really serve any purpose?)

I think I might be coming around to agree with you about language
choice, but if so, it's not because of arguments like the above.

-Karl

[Eric S. Raymond]

Post by Karl Fogel
Uh, ungh? (Does that kind of ad hominem putdown really serve any purpose?)

Possibly not, but I've been hearing variants of that particular self-assured
nonsense for close to a querter century now and I'm getting rather tired of it.

[Carsten Breuer]

Post by Eric S. Raymond

Post by Carsten Breuer
Do we? Memory-allocation bugs are always a problem of discipline and
bad coding style.

No, they're primarily a bug that arises from not baving proper

dynamic > memory

Post by Eric S. Raymond
management. Don't lecture me about C++ -- I learned better than this
writing LISP back in the 1970s back when C++ was not yet even a gleam
in Bjarne Stroustrup's eye, then ....

LISP...yes i know. That was the funny language that need 30% runtime
for garbage collection ;-) . Good job ;-) . I work also as a developer
for a long time and have seen too many people talking about garbage
collection and doesn't know what they are doing. A fool need a tool.

Post by Eric S. Raymond
I paid two decades' dues writing C because
hardware was not yet inexpensive enough to support LISP-like languages
outside of academia. I can make a pretty good guess at your age by
the way you write, and my guess is I've been clued in about these
issues longer than you've been walking and talking.

Well, you are 8 years older. That is really far away, right ;-) .

Post by Eric S. Raymond
I'm coding C++ right now on Battle For Wesnoth. It's an ugly,
bloated, over-complex language -- an octopus made by nailing extra
legs onto a dog -- and it encourages ugly, bloated, over-complex
architectures.

ROFL...Yes, KDE and QT are really bad projects ;-) .
There is enough at C++ i don't like, But i disagree that
all problems are gone if you use python. Python, with this
identation is also pretty ugly an a lot of problems occurs at runtime.
Things that would never happen on C++.

Post by Eric S. Raymond
The "discipline" you speak of has become a *waste of time* for anybody
not doing kernel or realtime work.

Is discipline ever a waste of time. All i talk about is "think before
implement" and that the "complementarity principle of problems" is also
valid for python.

Post by Eric S. Raymond
With menory and cycles so cheap, p
programmer hours are best spent on solving real problems, not on
low-level resource bookeeping.

Well, C++ classes can encapsulate this pretty well.


Carsten

[Eric S. Raymond]

Eric, are you coming to OSCON this year? I'd love to meet you, if so;
we've been "one node away" for years now, by many routes :-).

Alas, I won't be there this year.

[Karl Fogel]

Post by Eric S. Raymond

Eric, are you coming to OSCON this year? I'd love to meet you, if so;
we've been "one node away" for years now, by many routes :-).

Alas, I won't be there this year.

Pity. Another time, perhaps.

Folks: it's not Eric's fault that he accidentally CC'd dev@ on this
personal mail. My mailreader sets "Mail-followup-to" to include the
list when I reply to mails on ***@. In this case, I hit 'R'eply on
Eric's mail to the list, then made sure the list address was not in a
recipient field, but never removed the "Mail-followup-to" header.

When Eric replied, he probably hit 'F'ollowup instead of 'R'eply, but
considering that the mail he was responding to (mine) was sent only to
him, he had every right to expect that a followup would only go to me.

The reason I set "Mail-followup-to" in the first place is because our
list doesn't set "Reply-to" to the list address, for the reasons
listed in http://www.unicom.com/pw/reply-to-harmful.html. It is an
entirely unexpected result of my hack that a personal mail ended up on
the list -- precisely the situation Chip Rosenthal's document exhorts
us to avoid. I still think we shouldn't set "Reply-to", but, I must
admit, the irony weighs heavy here.

-Karl

[Carsten Breuer]

Hi Giovanni, hi all,

Post by Giovanni Bajo

Post by Karl Fogel
So we're going to need a working copy rewrite. We knew that; in fact

And this time (if you allow me my humble two cent) get it right and cut
the development time down to 1/10th by using a more evolved language
than C. Python comes to mind, but anything would do.

Why not Visual Basic ;-)?

I don't think that python make anything better here.
If the language should be switched, it should be IMHO
C++. Python *can* reduce the development time, because
it come with a bunch of functions. A good C++ lib
do the same.

The goal of C/C++ is that you can port it to
any platform you like. With python, net or
other scripting languages you can only use the
platform where the VM exist.

Did you ever see the python C code?
It's worse ;-).


Best Regards,



Carsten

[Karl Fogel]

Post by Carsten Breuer
The goal of C/C++ is that you can port it to
any platform you like. With python, net or
other scripting languages you can only use the
platform where the VM exist.

Is there any platform that runs Subversion that doesn't run the Python
VM, that people actually use?

[Vincent Lefevre]

Post by Karl Fogel
Is there any platform that runs Subversion that doesn't run the
Python VM, that people actually use?

There is a Subversion client for the Zaurus[*] while Python isn't
installed by default, and installing it (if possible) would take
even more disk space (which is rather limited). Now, I haven't
tried Subversion on my Zaurus though it could be very useful,
because there is no way to strip the local copies of big files
from the working copy.

[*] http://www.oesf.org/forums/index.php?showtopic=14764

[John Peacock]

Post by Karl Fogel
The working copy should really be a repository, even if
it's not always going to store all the history available on the server
side (with some projects, you really can't, it's too big).

Strangely enough, this is precisely how SVK manages it. But, if
Subversion is still going to support moving a subtree out of a WC and
still have it work, literally the working copy may need to be an entire
tree of [loosely coupled] repositories. SVK has a central (local)
repository, so that isn't an issue, but it seems like there has always
be a strong current in Subversion proper against centralized file
storage (especially in "hidden" directories of various platform-specific
species).

John

[Karl Fogel]

Post by John Peacock
Strangely enough, this is precisely how SVK manages it. But, if
Subversion is still going to support moving a subtree out of a WC and
still have it work, literally the working copy may need to be an
entire tree of [loosely coupled] repositories.

That's the "detachable subdirectory" functionality that I think we're
willing to give up; I certainly am, seems like many other devs agree.
So we may stop supporting that -- it's been an expensive feature, and
not many users use it (or even know about it).

Post by John Peacock
SVK has a central
(local) repository, so that isn't an issue, but it seems like there
has always be a strong current in Subversion proper against
centralized file storage (especially in "hidden" directories of
various platform-specific species).

No, I think we've overcome that now, and that people are okay with one
centralized admin area per working copy (that still allows nesting, of
course, just not detachment). That centralized admin area would be a
full-fledged repository.

-Karl

[John Peacock]

Post by Karl Fogel
No, I think we've overcome that now, and that people are okay with one
centralized admin area per working copy (that still allows nesting, of
course, just not detachment). That centralized admin area would be a
full-fledged repository.

So you could (lots of arm waving here) implement lib_wc using [a
possibly stripped down form of] lib_fsfs - one that only keeps 'N'
active revisions and tracks the central repository's revnums in local
properties. And which relies on checksums more heavily to determine
whether a file has been modified (since you can't just stat the
text-base).

Cool!

John

[Stefan Küng]

Post by Karl Fogel
True decentralized systems are really hard for most people to wrap
their heads around.

That's maybe the biggest advantage Subversion has now over those
decentralized systems. There still are thousands of companies which
don't use or sometimes never even heard of version control (maybe hard
to believe for you guys, but it is true - in my area, I think not even
20% of all companies which do some kind of SW development don't use
version control at all). If you try to explain to those why they need
version control, they will easily understand that. But then comes the
time to choose a system, and that's when they simply reject
decentralized systems: they are too far away from their current working
cycle. Subversion on the other hand can be configured and set up so they
don't have to change their working cycle too much. So that is what most
of them choose.

Post by Karl Fogel
One of Subversion's flaws (mea culpa) is that we didn't realize the
usefulness of having symmetrical functionality on the client and
server sides. The working copy should really be a repository, even if
it's not always going to store all the history available on the server
side (with some projects, you really can't, it's too big).

I agree here. The more information is available offline, the better.
Because offline is faster. In TortoiseSVN, Stefan Fuhrmann has
implemented a cache for log messages (my Kudos for that btw) because it
is simply too slow to get them from the repository every time you need
them, especially for bigger projects.

Post by Karl Fogel
We also have to be faster. Fortunately, we've pretty much agreed,
IIRC, that we're willing to punt on subdirectory detachability in
working copies in order to get performance improvements.

That may be one of the biggest improvements where users will see
immediately the performance gain.

Post by Karl Fogel
Subversion's phenomenal adoption rate (*) isn't due to being the only
game in town. We never were, if you count the proprietary systems,
and we're even less so now that the open source version control world
has become so fertile. The reason Subversion is taking over the world
is because it is tremendously user-focused, and because it provides
well-documented APIs that enable other developers to write software on
top of Subversion. We should copy what we need from the decentralized
systems, but remember that most users don't know or care whether a
system is centralized or decentralized -- their ideal system is one
they don't notice. Let's keep our eye on the ball, so they don't have
to.

Also never forget that for a high adoption rate, you need good UI
clients. Yes, command line clients are used too. But most Windows users
simply refuse to work with those. If a company has to choose a version
control system, they not only look at the features and speed, they also
look at what UI clients there are and how they perform. If there are no
UI clients or no good ones, that's a big minus point and it *will*
influence their decision for the version control system.

Another big plus for Subversion is that it was designed to also work on
Windows. That means there are no (or almost none (1)) ugly hacks in
there to make it work on Windows - something most decentralized systems
have (if they even work on Windows at all). Even Mercurial wasn't
designed to work on Windows when it started from what I've seen (2). And
Git, well I think it will never work on Windows in a way that it's
usable (3).
Some people think that Windows is not important and hope it will get
replaced by a free OS anyway. But that won't happen anytime soon. The
reality simply is that most people have to use Windows at work and
therefore need a version control system which works there well.

In this thread, there already were some discussions about which language
to use for Subversion 2.0. Even though I think it's too early to discuss
this because the first step would be to specify the features and
requirements first, I'd like to throw in some important facts which
should be considered when the time to choose the language arrives:
In order to get a lot of UI clients (see above why UI clients are
important), it's not enough to have a language that many people
understand and can work with. It's also important to have a library in a
language that can be used everywhere. Languages which require some kind
of environment they run in (usually languages which have garbage
collection) can't be used for plugins. At least not without unsolvable
incompatibility problems. A good explanation on what happens if you e.g.
write a shell extension plugin in .NET can be found here (4). And I'm
sure the same applies to other high level languages. And of course not
only for shell plugins but for plugins for any other application (say IDEs).
So when the time comes to choose a language, please consider these
problems too.

Stefan


(1)
http://svn.collab.net/viewvc/svn/trunk/subversion/libsvn_subr/io.c?revision=8341&view=markup
(2) http://www.selenic.com/mercurial/wiki/index.cgi/EncodeDecodeFilter
needed for line-ending conversions. Subversion is much better here
because you can decide for every file, not just by file extension
what line endings to use.
(3) http://marc.info/?l=git&m=114131099531633
(4) http://blogs.msdn.com/oldnewthing/archive/2006/12/18/1317290.aspx

[Branko Čibej]

Thank you, Karl! Beautifully put, as usual.

Post by Karl Fogel
We should copy what we need from the decentralized
systems, but remember that most users don't know or care whether a
system is centralized or decentralized -- their ideal system is one
they don't notice.

I'd like to take this one step further:

Version control must become ubiquitous.

In other words: We must aim towards the day when everybody uses version
control all the time, without having to think about it.

Far-fetched?

Consider: Not so long ago, many widely-used computers did not have the
concept of a file system; or if they did, it was not hierarchic. The
argument then was that users didn't need that organizing power, as they
didn't have that much information to organize. (Instead, we invented
interesting labelling schemes to keep the cassette tapes containing our
data marginally organized).

Consider: Not so long ago,, many widely-used computers did not have the
concept of multitasking, or if they did, it was done in a way that
practically required manual context switching. The argument then was
that users didn't need that concurrency, since they could only use one
program at a time.

(And people who did have access to real multitasking OSes with real
filesystems, e.g., RSX-11, TOPS-10, and later VMS and Unix, were often
in danger of bursting their guts for laughing when they heard such
arguments. Did you hear the one about 8-bit registers being good enough
for most applications?)

Post by Karl Fogel
From today's perspective, these anecdotes sound a bit like a museum

guide expounding on the technological revolution brought on by the
palaeolithic hammerstone. And the list of such anecdotes is endless.

These days, version control is generally accepted by a large, or at
least vocal, minority of programmers as "the thing to do." And I stress:
*minority*. Too many programmers still view version control as extra
work that's imposed on them for no visible benefit. (Well, heh, too many
programmers also view software engineering as extra work with no visible
benefit.)

So we knurds all have hammerstones, and some of us even prefer using
them over teeth -- or foreheads -- for cracking our nuts, despite the
mashed fingers that sometimes result from their unwary application.


But what about everyone else? We keep hearing and talking about version
control in the context of software development. Some even prefer the
term "source control," as if program source it the only artefact that
merits versioning. And everyone else can eat their mammoth haunch raw,
because fire is only for shamans. ...

No! Being able to see the history of one's work should be as implicit as
saving said work in a named file that can be manipulated separately from
the application that created it -- or having on one's desk a box with a
dual-core CPU, real-time, true-colour 3D graphics, and a multitasking
OS. And I believe the Subversion community is an ideal nurturing ground
for such a concept, precisely because it is so uncommonly user-centric.
Let us therefore stop trying to take over the world -- and subvert it
instead.

-- Brane


P.S.: The other day, an architect was complaining to me about how hard
and error-prone it is to keep track of different versions of his
projects. So I gave a small demonstration with Subversion and WebDAV
auto-commit. If I may allow myself a bit of an understatement: the man
found the concept of automatic version control quite interesting, and
was intrigued by the circumstance that his multi-kilobuck drafting tools
don't support it. Actually, "slavering" and "raving mad" would be apt
descriptions of his reactions. ...

[Karl Fogel]

Post by Branko Čibej
Version control must become ubiquitous.
In other words: We must aim towards the day when everybody uses version
control all the time, without having to think about it.

Brane, we are so much in agreement here that airport security is
having trouble telling us apart.

On July 24th, I'm talking a bit at the O'Reilly Radar Executive
Briefing (http://conferences.oreillynet.com/pub/w/58/radar.html) about
Subversion and version control in general. When O'Reilly asked me to
suggest some topics, I offered the following:

- Why it's important to be able to search the change history for
Wikipedia entries (and why it's incredibly cumbersome using
today's interface);

- Why the U.S. Congress needs real version control tools (and why
it's our own fault for not providing them);

- Why good version tracking is important in a world where more and
more creativity consists of mixing existing things together.

I think Subversion stands a better chance of achieving this kind of
ubiquity (or being part of something that achieves it) than many
systems.

Post by Branko Čibej
These days, version control is generally accepted by a large, or at
*minority*. Too many programmers still view version control as extra
work that's imposed on them for no visible benefit. (Well, heh, too many
programmers also view software engineering as extra work with no visible
benefit.)
So we knurds all have hammerstones, and some of us even prefer using
them over teeth -- or foreheads -- for cracking our nuts, despite the
mashed fingers that sometimes result from their unwary application.
But what about everyone else? We keep hearing and talking about version
control in the context of software development. Some even prefer the
term "source control," as if program source it the only artefact that
merits versioning. And everyone else can eat their mammoth haunch raw,
because fire is only for shamans. ...

... *better* *merge* *tools* ...

Post by Branko Čibej
No! Being able to see the history of one's work should be as implicit as
saving said work in a named file that can be manipulated separately from
the application that created it -- or having on one's desk a box with a
dual-core CPU, real-time, true-colour 3D graphics, and a multitasking
OS. And I believe the Subversion community is an ideal nurturing ground
for such a concept, precisely because it is so uncommonly user-centric.
Let us therefore stop trying to take over the world -- and subvert it
instead.

Amen!

Post by Branko Čibej
P.S.: The other day, an architect was complaining to me about how hard
and error-prone it is to keep track of different versions of his
projects. So I gave a small demonstration with Subversion and WebDAV
auto-commit. If I may allow myself a bit of an understatement: the man
found the concept of automatic version control quite interesting, and
was intrigued by the circumstance that his multi-kilobuck drafting tools
don't support it. Actually, "slavering" and "raving mad" would be apt
descriptions of his reactions. ...

Yup, I sometimes encounter that reaction too, when explaining
Subversion to non-programmers who have to do document revision control
in their daily work (artists, editors, lawyers, civil servants, etc).

On the other hand, I've lately been watching some version control
newbies -- the volunteer translators of my book -- struggle with
Subversion. Without TortoiseSVN, they'd be completely sunk. As it
is, some of them are just barely treading water. I still have to give
technical support sometimes: for example, they're mostly working in
teams of two or more, and recently one of the teams encountered its
first conflict. It took several emails to explain what was going on
and how to resolve it.

It's been a sobering experience; we've a long way to go, I think.

-Karl

[Graham Leggett]

Post by Branko Čibej
Version control must become ubiquitous.
In other words: We must aim towards the day when everybody uses version
control all the time, without having to think about it.
Far-fetched?

Not at all.

My business is today architected around a subversion repository. All the
accounts, scans of invoices and correspondence, everything is checked
in. If someone needs access to certain documents, they get access to
part of the tree they need via their browser. At year end when accounts
are finalised, the repository is tagged.

Regards,
Graham
--

[Garance A Drosihn]

Post by Karl Fogel
I've been wanting to post this for a while, but was waiting
for the dust from Linus Torvald's GIT talk to settle first

Thanks for starting this thread. I think discussion of some
of these ideas will be helpful to subversion development. I
have moved to subversion for many things, and I'm not eager
to move off of subversion to git or other alternatives, but
there are some features I'd love to have.

Apologies that this is going to be so long-winded, but I'm
trying to build up a case (in my own mind) for exactly what
I would like to see, and why I would like to see it.

Post by Karl Fogel
In his talk, Torvalds explained why he thinks decentralized
version control systems (like GIT and Mercurial) are the way
of the future, ...

While his presentation was rather grating to listen to, I
think he did cover some interesting points.

Post by Karl Fogel
Since I'd like to use some of his arguments as a jumping-off
point for thoughts on Subversion's future, here they are in
* Optimizing merging is as important as optimizing
branching (if not more so).
* Speed matters: ....
* Having all history locally (...) is useful.
* Reducing the thickness of the "commit access" wall is
good for development. Torvalds didn't make this argument
terribly well, so I'll try to restate what I think was
The important question is, who can put changes into the
repository that the project is publishing releases from?

I think you're approaching this from a different direction than
Linus does, and I think *most* projects don't work in the same
way that Linus likes to work. Git is obviously successful for
how Linus does work, and I think the key reason is that Linus
works in a "pull" model of updating, while most of us work in
a "push" model.

Linus has his repository, and he has no intention of giving the
world access to it. However, he's quite willing to pull specific
updates from other people. Strictly speaking, *Linus* is the
only one who can put changes into the repository "the project"
is publishing releases from. This works, because what he does
is let everyone else create *their* repositories, and thus they
have access to their repository -- not his. And then he pulls
changes from their repo into his repo. He pulls what he wants,
and when he feels like pulling it.

In a project like FreeBSD, there is a central repository, and a
few hundred developers have push access into it. Oh, there's
code-review, and you can get bashed around the head if enough
developers are not happy with some commit that you pushed into
the central repo, but each developer is pushing his own updates
into the common repository.

I cannot see Linus giving 400+ people access to his personal
repository, and I cannot see FreeBSD switching to have a "pull"
model for bringing updates into the "official" project repository.

Post by Karl Fogel
When I talked to Brian Fitzpatrick about this, he listed three
- Faster. Subversion does need to be faster for many ops.
- Offline commits.
- Local branches.
I would add "better merging", but basically agree with Fitz
(note that we're getting much-improved merging in Subversion 1.5).

What I think I would like is more than just local branches. I
want branches which are recognized as being totally separate
from the original repository. Disjoint, separately administered.
People who have zero write access to a master repository should
still be able to do track their changes, and track those changes
wrt the official repository.

Also, from what I can tell, what we're getting is much-improved
tracking of merges made within a single repository. That is a
major benefit, of course, but we're still in the mindset of a
working from a single master repository.

Post by Karl Fogel
But there's another factor...
True decentralized systems are really hard for most people to
wrap their heads around.
... One of Subversion's biggest advantages, and one of the
reasons it's taking over the world, is that it's really easy
to understand. There's a repository; you check stuff out;
you modify the stuff; you check it back in.

I think a full-blown distributed change system is harder for
the average person to understand, but I think subversion could
provide a subset of it that would be really easy to explain.

Post by Karl Fogel
[....] For many organizations, including open source projects,
centralization is a feature: you want changes (and branches)
to end up in the master repository sooner rather than later,
so they'll be visible to everyone, so they'll be backed up,
so they'll go through the central hook system, etc. It
focuses the community on a shared object (Ben Collins-Sussman
makes this argument in more detail at
http://blog.red-bean.com/sussman/?p=20).

To quote from that blog:

So while most people say: "isn't it great that I can fork
the whole project without anyone knowing!" My reaction is,
"yikes, why aren't you working with everyone else? why
aren't you asking for commit access?" This is a problem
that is solved socially: projects should actively encourage
side-work of small teams and grant access to private
branches early and often

This overlooks one key point. Even though CVS and subversion
do not support any kind of distributed model, developers *WILL*
do it. They *ALREADY* do it. I have commit access to the
FreeBSD project, for instance, but there are small works-in-
progress that I simply do not want to clutter up the main
repository with. No matter how frilly and wonderful you make
"collaboration" sound, the fact is that it is counter-productive
to put things in the official repository until you have some
idea how it's going to work out. I'm not going to commit
something and have that CVSUP-ed to 100,000 machines across
the planet, if I think I might rip out all those changes
tomorrow just to do it in a completely different manner.

Is it frightening? Well, that's too bad. It is what is in
fact happening, and it's absurd to pretend that it does not.
Or to pretend it go away if your SCM doesn't support it.

It happens that many developers in the FreeBSD project get
around this by having a *second* repository, which is sync'ed
from the official CVS repository and is in perforce. But I
don't feel like learning perforce just for this benefit. (The
developers who use it, use it due other benefits. But for
*me* this would be the only benefit from using the p4-based
repository).

And I *have* access to the FreeBSD repository. What about
someone who doesn't have any access, but wants to work on
some changes before sending in a PR? How are they supposed
to come up with a patch that they are confident in? Should
they send in one PR one day, and then send in another PR the
next day saying "Well, my previous patch was all screwed up,
but how about this one?". This is not a good way to build
credibility.

- - - - - - - - - - - - - - - - -
All of the above rambling is just background for:
- - - - - - - - - - - - - - - - -

Post by Karl Fogel
And now I'm going to hand-wave on a lot of details. I don't
mean to start the Subversion 2.0 design thread now, just to
offer some thoughts on general goals.

I also have to hand-wave on the details of what I want, because
I'm still not completely sure what would work right...

I think what I want is to be able to create my own repository,
and maybe say that the HEAD branch in this new repository is a
read-only copy of HEAD in the master repository. The local
HEAD branch would automatically sync-up with the original.
This obviously presents some technical issues wrt revision
numbers, etc. Perhaps that could be tracked via a second
revprop for the mirrored version of the HEAD branch.

And really I'd like to specify more than one branch.
Something like "Create me a new repository which is a fork
of <OtherRespository>, starting at July 1/2006, and limit
that mirroring to just the HEAD and 6.x branches".

[Karl Fogel]

Post by Garance A Drosihn
I cannot see Linus giving 400+ people access to his personal
repository, and I cannot see FreeBSD switching to have a "pull"
model for bringing updates into the "official" project repository.

I don't see a real distinction between "push" and "pull" here. We
could just as easily say 400 people have the ability to "pull" changes
from their personal repositories into the main FreeBSD repository. It
doesn't matter whether it's 1 or 400 developers doing it, or how many
repositories each developer has write access to.

Post by Garance A Drosihn

Post by Karl Fogel
When I talked to Brian Fitzpatrick about this, he listed three
- Faster. Subversion does need to be faster for many ops.
- Offline commits.
- Local branches.
I would add "better merging", but basically agree with Fitz
(note that we're getting much-improved merging in Subversion 1.5).

What I think I would like is more than just local branches. I
want branches which are recognized as being totally separate
from the original repository. Disjoint, separately administered.
People who have zero write access to a master repository should
still be able to do track their changes, and track those changes
wrt the official repository.

Right, that's what I'm talking about.

Post by Garance A Drosihn
Also, from what I can tell, what we're getting is much-improved
tracking of merges made within a single repository. That is a
major benefit, of course, but we're still in the mindset of a
working from a single master repository.

Well, if you're talking about 1.5's upcoming merge-tracking, that's
because it has to work with a single master repository :-).

Post by Garance A Drosihn
I think a full-blown distributed change system is harder for
the average person to understand, but I think subversion could
provide a subset of it that would be really easy to explain.

Yeah, that's sort of what I think our course should be.

Post by Garance A Drosihn
So while most people say: "isn't it great that I can fork
the whole project without anyone knowing!" My reaction is,
"yikes, why aren't you working with everyone else? why
aren't you asking for commit access?" This is a problem
that is solved socially: projects should actively encourage
side-work of small teams and grant access to private
branches early and often
This overlooks one key point. Even though CVS and subversion
do not support any kind of distributed model, developers *WILL*
do it. They *ALREADY* do it. I have commit access to the
FreeBSD project, for instance, but there are small works-in-
progress that I simply do not want to clutter up the main
repository with. No matter how frilly and wonderful you make
"collaboration" sound, the fact is that it is counter-productive
to put things in the official repository until you have some
idea how it's going to work out. I'm not going to commit
something and have that CVSUP-ed to 100,000 machines across
the planet, if I think I might rip out all those changes
tomorrow just to do it in a completely different manner.
Is it frightening? Well, that's too bad. It is what is in
fact happening, and it's absurd to pretend that it does not.
Or to pretend it go away if your SCM doesn't support it.

I agree with you, always have; I expect Ben does too.

Post by Garance A Drosihn
It happens that many developers in the FreeBSD project get
around this by having a *second* repository, which is sync'ed
from the official CVS repository and is in perforce. But I
don't feel like learning perforce just for this benefit. (The
developers who use it, use it due other benefits. But for
*me* this would be the only benefit from using the p4-based
repository).
And I *have* access to the FreeBSD repository. What about
someone who doesn't have any access, but wants to work on
some changes before sending in a PR? How are they supposed
to come up with a patch that they are confident in? Should
they send in one PR one day, and then send in another PR the
next day saying "Well, my previous patch was all screwed up,
but how about this one?". This is not a good way to build
credibility.

I think you don't realize how much we're already on your page... :-)

Post by Garance A Drosihn
I also have to hand-wave on the details of what I want, because
I'm still not completely sure what would work right...
I think what I want is to be able to create my own repository,
and maybe say that the HEAD branch in this new repository is a
read-only copy of HEAD in the master repository. The local
HEAD branch would automatically sync-up with the original.
This obviously presents some technical issues wrt revision
numbers, etc. Perhaps that could be tracked via a second
revprop for the mirrored version of the HEAD branch.
And really I'd like to specify more than one branch.
Something like "Create me a new repository which is a fork
of <OtherRespository>, starting at July 1/2006, and limit
that mirroring to just the HEAD and 6.x branches".

Yes. What you want is more or less what I was saying. (SVK is
probably the place we should look for guidance first.)

I think you might have heard me to be saying what you expected me to
say, rather than what I intended to say. (IOW, if you reread my mail
starting from the assumption that I have the same goals you do, I
think nothing will stand out to contradict that assumption.)

Best,
-Karl

[Garance A Drosihn]

Post by Garance A Drosihn

Post by Karl Fogel
I would add "better merging", but basically agree with Fitz
(note that we're getting much-improved merging in Subversion 1.5).

Also, from what I can tell, what we're getting is much-improved
tracking of merges made within a single repository. That is a
major benefit, of course, but we're still in the mindset of a
working from a single master repository.

I haven't used git at all, so I don't understand how Linus solves
this problem across disjoint repositories. It kinda sounds like
he makes a sha-1 digest of the lines which make up a given patch,
and then keeps track of that value. That way you can track the
application of a patch across repositories, even if all the lines
in that original patch are replaced by later updates.

[Stefan Sperling]

Post by Garance A Drosihn
And I *have* access to the FreeBSD repository. What about
someone who doesn't have any access, but wants to work on
some changes before sending in a PR? How are they supposed
to come up with a patch that they are confident in?

I used the CVS_LOCAL_BRANCH_NUM feature to work on my wake
on lan patch for FreeBSD (http://stsp.name/wol/).

It's described in development(7). You create a copy of the
FreeBSD CVS repo, create a local branch with a revision number
you make up and commit to the branch, hoping that there will never
be a branch with the same number in the official repo.
CVSup will (in most cases) manage to keep your branch intact
when you sync to the master repo.

I think this feature was hacked into CVS by the FreeBSD people
to facilitate customisation. I don't know how many people actually
make use of it, and in fact I stopped using it after a while out of
lazyness, and now I just maintain my patch against a workspace from
anoncvs. But that only works because the patch is rather small.
And doing this makes it hard for me to track -CURRENT :-/

Post by Garance A Drosihn
Should they send in one PR one day, and then send in another PR the
next day saying "Well, my previous patch was all screwed up,
but how about this one?". This is not a good way to build
credibility.

My patch has been sitting around for more than 2 years now.
A few users have asked for it to go into mainline,
but I have no committer at hand who can nitpick and commit it
for me, even though I've asked several times no one seems
interested :(
This is a social problem though, not a technical one.

</freebsd>

<subversion>

Post by Garance A Drosihn
I think what I want is to be able to create my own repository,
and maybe say that the HEAD branch in this new repository is a
read-only copy of HEAD in the master repository. The local
HEAD branch would automatically sync-up with the original.

That would be nice indeed. I'm also working on the port
of Linux to the Nintendo DS console, and we are using
subversion. For people without commit access the only
option currently is to create patches against anonsvn
and send them to the mailing list. They don't really
learn how to use version control properly doing this,
which is a problem because it means there's an additional
learning curve involved in becoming a committer.

And a lot of patches we get are not well done. Maybe if
people submitting patches had more of an idea of how version
control actually works they would send better patches.
It would be great if subversion could educate people this way.

Post by Garance A Drosihn
This obviously presents some technical issues wrt revision
numbers, etc. Perhaps that could be tracked via a second
revprop for the mirrored version of the HEAD branch.

This would be interesting to implement :)

The company I work for does SCM consulting and is also focusing
on distributed development. A lot of our clients use subversion
at the moment. Getting more distributed development features into
subversion would be great, and I think we'd gladly help designing
and implementing them.

Post by Garance A Drosihn
And really I'd like to specify more than one branch.
Something like "Create me a new repository which is a fork
of <OtherRespository>, starting at July 1/2006, and limit
that mirroring to just the HEAD and 6.x branches".

Since branches are just directories in subversion, selecting
a subset of directories from the repo you are mirroring
should be relatively trivial to implement.

[Garance A Drosihn]

Post by Karl Fogel
We also have to be faster. Fortunately, we've pretty much agreed,
IIRC, that we're willing to punt on subdirectory detachability in
working copies in order to get performance improvements.

What is meant by "subdirectory detachability"? Just curious. I
didn't see it mentioned in the book on subversion I have, and
google doesn't want to help me with a likely reference...

[Karl Fogel]

Post by Garance A Drosihn

Post by Karl Fogel
We also have to be faster. Fortunately, we've pretty much agreed,
IIRC, that we're willing to punt on subdirectory detachability in
working copies in order to get performance improvements.

What is meant by "subdirectory detachability"? Just curious. I
didn't see it mentioned in the book on subversion I have, and
google doesn't want to help me with a likely reference...

$ svn checkout http://svn.collab.net/repos/svn/trunk/ svn-src
$ cd svn-src
$ mv notes www ..
$ cd ..
$ rm -rf svn-src
$ cd notes
$ svn info | grep URL
URL: http://svn.collab.net/repos/svn/trunk/notes
### Yup, it's a working copy. ###
$ cd ../www
$ svn info | grep URL
URL: http://svn.collab.net/repos/svn/trunk/www
### Yup, this is a working copy too. ###
$

[Giovanni Bajo]

Post by Karl Fogel

Post by Garance A Drosihn

Post by Karl Fogel
We also have to be faster. Fortunately, we've pretty much agreed,
IIRC, that we're willing to punt on subdirectory detachability in
working copies in order to get performance improvements.

What is meant by "subdirectory detachability"? Just curious. I
didn't see it mentioned in the book on subversion I have, and
google doesn't want to help me with a likely reference...

$ svn checkout http://svn.collab.net/repos/svn/trunk/ svn-src
$ cd svn-src
$ mv notes www ..
$ cd ..
$ rm -rf svn-src
$ cd notes
$ svn info | grep URL
URL: http://svn.collab.net/repos/svn/trunk/notes
### Yup, it's a working copy. ###
$ cd ../www
$ svn info | grep URL
URL: http://svn.collab.net/repos/svn/trunk/www
### Yup, this is a working copy too. ###
$

I think that part of the reason why this feature might be important in
Subversion is that Subversion insists on *not* having a built-in concept of
"project", "branch" or "tag", or IOW to expose its versioned-filesystem guts
up to the command line client.

I think this is a point that should be revised for 2.0. I understand that the
versioned filesystem gut is important at the repos level (and maybe at the ra
level?). But I think that having it at the user level (command line) it's
creating more confusion and mess than anything else.

Repeating myself, I'm speaking *exclusively* of the UI.

The history also tells us something. CVS had an explicit concept and support
of branches and tags. SVN has it implicit (convention of using "/trunk",
"/branches", etc.). Mercurial/git has it *explicit* again.

By having implicit support for branches/tags, SVN UI is percolated with long
URLs everywhere. To do any non-trivial operation, you must enter the full URL
anytime. For instance, to answer a question like "what's the difference in
this file between this working copy and the last release of the project":

* cvs diff -rRELEASE_12 foo.c
* hg diff -rRELEASE_12 foo.c

you have to type the full URL, *twice*:

* svn diff http://myrepo/project/tags/RELEASE_12/a/b/c/foo.c
http://myrepo/project/trunk/a/b/c/foo.c

Basically, anything that require tagging/branching/merging requires full URLs.
Compare that with the UI of Mercurial or CVS, where you simply do "hg tag
MYNEWTAG" to create a tag, and other simple commands to merge, show
differences, etc.

Notice that this problem isn't fixed even by graphical tools, at the moment.
TortoiseSVN itself is somehow forced to inherit the working copy model and to
exclusively act on the single working copy tree, without having a broader look
at the project, at the branches/tags, and thus providing higher-level commands
for the user.

[Branko Čibej]

Post by Giovanni Bajo

Post by Karl Fogel

Post by Garance A Drosihn

Post by Karl Fogel
We also have to be faster. Fortunately, we've pretty much agreed,
IIRC, that we're willing to punt on subdirectory detachability in
working copies in order to get performance improvements.

What is meant by "subdirectory detachability"? Just curious. I
didn't see it mentioned in the book on subversion I have, and
google doesn't want to help me with a likely reference...

$ svn checkout http://svn.collab.net/repos/svn/trunk/ svn-src
$ cd svn-src
$ mv notes www ..
$ cd ..
$ rm -rf svn-src
$ cd notes
$ svn info | grep URL
URL: http://svn.collab.net/repos/svn/trunk/notes
### Yup, it's a working copy. ###
$ cd ../www
$ svn info | grep URL
URL: http://svn.collab.net/repos/svn/trunk/www
### Yup, this is a working copy too. ###
$

I think that part of the reason why this feature might be important in
Subversion is that Subversion insists on *not* having a built-in
concept of "project", "branch" or "tag", or IOW to expose its
versioned-filesystem guts up to the command line client.

Now that is one thing I'd like to get rid of. It turns out that in real
life, in large projects, conflating the concepts of branches and
directories is a PITA. Of course the /implementation/ of branches and
directories is similar, but the semantics (and therefore user interface)
is different.

-- Brane

[Ph. Marek]

On Mittwoch, 4. Juli 2007, Branko Čibej wrote:
...

Post by Branko Čibej

Post by Giovanni Bajo
I think that part of the reason why this feature might be important in
Subversion is that Subversion insists on *not* having a built-in
concept of "project", "branch" or "tag", or IOW to expose its
versioned-filesystem guts up to the command line client.

Now that is one thing I'd like to get rid of. It turns out that in real
life, in large projects, conflating the concepts of branches and
directories is a PITA. Of course the /implementation/ of branches and
directories is similar, but the semantics (and therefore user interface)
is different.

To this I'd like to mention that the *technical* aspect of "just a directory
structure" is one of the most valuable design choices in subversion (IMHO, at
least).

Please don't change that.


If there are simply a few commands "svn branch", "svn tag" etc., which use
some properties at the repository root to know whether the structure is
/project/{trunk,branches,tags}
or
/{branches,tags,trunk}/project1,
the user interface can be done in any thinkable way.


But please keep the *basic* structure clean and simple.

(Probably I just misunderstood and worry too much, and the structure will
never change anyway.)


Regards,

Phil

[Branko Čibej]

Post by Ph. Marek
...

Post by Branko Čibej

Post by Giovanni Bajo
I think that part of the reason why this feature might be important in
Subversion is that Subversion insists on *not* having a built-in
concept of "project", "branch" or "tag", or IOW to expose its
versioned-filesystem guts up to the command line client.

Now that is one thing I'd like to get rid of. It turns out that in real
life, in large projects, conflating the concepts of branches and
directories is a PITA. Of course the /implementation/ of branches and
directories is similar, but the semantics (and therefore user interface)
is different.

To this I'd like to mention that the *technical* aspect of "just a directory
structure" is one of the most valuable design choices in subversion (IMHO, at
least).

Why? I'm really interested: What, exactly, do you (we) gain by that?
Compared to all the real pain it's causing.

Post by Ph. Marek
Please don't change that.

Obviously you can always map two namespaces into one by adding a level
of indirection.

-- Brane

[Ph. Marek]

Post by Branko Čibej

Post by Ph. Marek
To this I'd like to mention that the *technical* aspect of "just a
directory structure" is one of the most valuable design choices in
subversion (IMHO, at least).

Why? I'm really interested: What, exactly, do you (we) gain by that?

- Extensible
- Versatile
Eg. you can easily do some debian-like naming "experimental",
"unstable", "testing", "stable" and merge in this directions.
- Easily understandable to anyone who knows directories/folders
(At least, if not the "old" understanding of CVS' tags/branches as a
new dimension is present).
I think that this point is *the* point for new users.
- As soon as the information "Make pointer to old data" is given, all
implications follow naturally.

Post by Branko Čibej
Compared to all the real pain it's causing.

What pain does it cause?
Make "tag:A" be a synonym for "$URL/tags/A", and if another layout is present,
require a "svn:layout" property in $URL.

Then use "svn cp . tag:A", or "svn diff tag:1.1 tag:1.2" or whatever.
Similar for branches. Result? No visible difference to CVS (for the user).

(Please take that with a grain of salt. I have surely overlooked some point
which makes that more problematic).

[Of course, that could clash with files named "tag:*". But this problem exists
with directories named "http:", too, no?)

Post by Branko Čibej

Post by Ph. Marek
Please don't change that.

Obviously you can always map two namespaces into one by adding a level
of indirection.

Right, and that's what makes subversion so powerful IMO - the extensibility
(w.r.t. the repository layout).


Naturally there could be ways to do trunk/branches/tags invisible to the
casual user (like NTFS hides its internal files from the user, by just
showing a subtree), which would make no difference for me.

But then there'd be at least the addressing issues again ... so just map the
destination into the path.


Regards,

Phil

[Branko Čibej]

Post by Ph. Marek

Post by Branko Čibej

Post by Ph. Marek
To this I'd like to mention that the *technical* aspect of "just a
directory structure" is one of the most valuable design choices in
subversion (IMHO, at least).

Why? I'm really interested: What, exactly, do you (we) gain by that?

- Extensible
- Versatile
Eg. you can easily do some debian-like naming "experimental",
"unstable", "testing", "stable" and merge in this directions.
- Easily understandable to anyone who knows directories/folders
(At least, if not the "old" understanding of CVS' tags/branches as a
new dimension is present).
I think that this point is *the* point for new users.
- As soon as the information "Make pointer to old data" is given, all
implications follow naturally.

Users new to the concept of version control will have trouble
understanding branches no matter how they're presented in the UI. The
current model may make newbies happy (through they end up with an
inefficient repository), but will give headaches to people who do
understand version control.

Post by Ph. Marek

Post by Branko Čibej
Compared to all the real pain it's causing.

What pain does it cause?
Make "tag:A" be a synonym for "$URL/tags/A", and if another layout is present,
require a "svn:layout" property in $URL.
Then use "svn cp . tag:A", or "svn diff tag:1.1 tag:1.2" or whatever.
Similar for branches. Result? No visible difference to CVS (for the user).

In my experience, user interface tricks like that are just that: tricks.
And they cost implementation time, and are hard to get right. The moment
you allow free-form project structuring, and multiple project (with
different structures) in the same repository ... you get a combinatorial
explosion.

Another problem is that you can't really create a hierarchy of branches
without playing tricks with branch naming. So we enforce a flat
namespace on a concept where hierarchy is useful (remember I'm talking
about large, complex projects here, not relatively trivial ones like
Subversion or GCC or KDE or ... heh). We actually made things worse than
necessary in real life by proposing that infamous trunk/branches/tags
structure -- because trunk is privileged compared to other branches,
users tend to assume there's something special about trunk. I've seen so
many broken repositories where the breakage can be traced directly to
that assumption that it's not even funny.

I might also mention a propensity among the new users that this
conflated namespace allegedly helps to check out whole repositories --
all branches, all tags, everything -- and create branches locally by
literally doing a WC->WC svn copy; or even plain copy + add + commit,
which happily tosses away history. Then you hear complaints about how
slow Subversion is for creating tags and branches, and how much space it
burns on your disk.

So, I submit:

* New users are generally ignorant about branches and tags, and get
confused by the extra cruft directories.
* Knowledgeable users need far more expressive power than our model
actually provides.
* Any loss of flexibility is regained by allowing cross-repository
merging (something that we need anyway if we allow local branches
and off-line commits), and restrict to one "project" per repository.

(Well, Multi-repository "views" in the working copy would be nice, but
when every working copy is a repository, you effectively get those for
free.)


-- Brane

[Ph. Marek]

Post by Branko Čibej
Users new to the concept of version control will have trouble
understanding branches no matter how they're presented in the UI.

Well, I have different experiences, but that's probably because of the small
number (and very likely homogenity) of samples ...

Post by Branko Čibej
The
current model may make newbies happy (through they end up with an
inefficient repository), but will give headaches to people who do
understand version control.

Here I don't understand.
What is inefficient?
Why does it give headaches?

I always found the structure of a subversion repository very nice and elegant.

Post by Branko Čibej

Post by Ph. Marek

Post by Branko Čibej
Compared to all the real pain it's causing.

What pain does it cause?
Make "tag:A" be a synonym for "$URL/tags/A", and if another layout is
present, require a "svn:layout" property in $URL.
Then use "svn cp . tag:A", or "svn diff tag:1.1 tag:1.2" or whatever.
Similar for branches. Result? No visible difference to CVS (for the user).

In my experience, user interface tricks like that are just that: tricks.
And they cost implementation time, and are hard to get right.

90% of life is finding the right tricks, to make life easier.
If there are simple, documented rules how this gets translated, it's easy to
use.

Post by Branko Čibej
The moment
you allow free-form project structuring, and multiple project (with
different structures) in the same repository ... you get a combinatorial
explosion.

Then restrict the "tag" and "branch" commands to normalized structures.

Either people use the suggested directory hierarchy (or one of two examples),
and can use the full power of the tools, or they cook their own cake and have
to provide the ingredients.

Post by Branko Čibej
Another problem is that you can't really create a hierarchy of branches
without playing tricks with branch naming. So we enforce a flat
namespace on a concept where hierarchy is useful

But I thought that's exact the flexibility of subversion???
Just use
trunk/
tags/
branches/
branch-1/
trunk/
tags/
branches/
branch-1.1/
branch-1.2/
branch-2/

etc.

It's easily possible, and I got that idea about 15 seconds after reading your
lines (not that I never heard of hierarchical branches before, but they're
not in my primary thinking) - which I take as a point for subversions'
flexibility :-)

Post by Branko Čibej
(remember I'm talking
about large, complex projects here, not relatively trivial ones like
Subversion or GCC or KDE or ... heh). We actually made things worse than
necessary in real life by proposing that infamous trunk/branches/tags
structure -- because trunk is privileged compared to other branches,
users tend to assume there's something special about trunk.

Hmmm, the users I know don't have that expectation, but there are certainly
some.

Post by Branko Čibej
I've seen so
many broken repositories where the breakage can be traced directly to
that assumption that it's not even funny.

Then don't create a trunk/ but a branches/devel or some such, and it's clear
that it's the same as other branches. trunk is just used more often, and a
shorter name is better Huffman coding.

Post by Branko Čibej
I might also mention a propensity among the new users that this
conflated namespace allegedly helps to check out whole repositories --
all branches, all tags, everything --

Well, you have more experience with users, I believe ... So you'll see a wider
spectrum.

Post by Branko Čibej
and create branches locally by
literally doing a WC->WC svn copy; or even plain copy + add + commit,
which happily tosses away history. Then you hear complaints about how
slow Subversion is for creating tags and branches, and how much space it
burns on your disk.

Then you say "read the documentation, creating branches in the repository",
and slap them behind the ears ;-)
No, I understand your worries.

But mis-use has been done with CVS too ... I remember having seen (and read) a
lot of stories how CVS is "bad", eg. because someone tagged *after every
single commit* to get what subversion has implicit - relations between the
files, to reconstruct some common base line.


There'll always be some users who don't read the documentation, read it and
forget it or do other than suggested because "it's more logical", only to
have problems later ...

That's one of the points where I say "subversion should give enough rope" :-)

Post by Branko Čibej
* New users are generally ignorant about branches and tags, and get
confused by the extra cruft directories.
* Knowledgeable users need far more expressive power than our model
actually provides.
* Any loss of flexibility is regained by allowing cross-repository
merging (something that we need anyway if we allow local branches
and off-line commits), and restrict to one "project" per repository.
(Well, Multi-repository "views" in the working copy would be nice, but
when every working copy is a repository, you effectively get those for
free.)

I don't share your experiences, but I think you know what you're doing.


Regards,

Phil

[Graham Leggett]

Post by Branko Čibej
Why? I'm really interested: What, exactly, do you (we) gain by that?

The concept returns to the original method of version control that people
used before there was version control, and that was to "make a copy of a
the source directory". Tags modelled like directories are easy for new
users to understand.

Post by Branko Čibej
Compared to all the real pain it's causing.

Having introduced this concept during a number of migrations to
subversion, I honestly haven't seen anyone go through any pain.

Regards,
Graham
--

[C. Michael Pilato]

Post by Graham Leggett

Post by Branko Čibej
Why? I'm really interested: What, exactly, do you (we) gain by that?

The concept returns to the original method of version control that people
used before there was version control, and that was to "make a copy of a
the source directory". Tags modelled like directories are easy for new
users to understand.

Post by Branko Čibej
Compared to all the real pain it's causing.

Having introduced this concept during a number of migrations to
subversion, I honestly haven't seen anyone go through any pain.

Yeah, I have to say that in my experience, Subversion's approach to
branching and tagging has only ever been a benefit to them in terms of
reducing a complex concept to a comprehensible one.

I do see seasoned version control users get tripped up, but it's typically
in slinging the specific command lines needed for merges, not in the
"concept comprehension" department.

There are only two realistic complaints that I can recall seeing leveraged
against Subversion's way of doing branches and tags:

1. all that flexibility makes it harder for Subversion and other third-party
tools to recognize and treat a directory copy as a branch. Common questions
like, "What tags exist?" can't be answered unless the answering system first
understands the policy used to denote tags in a given repository ("stuff
immediately under /tags"; "stuff immediately under an immediate subdirectory
of /tags"; ...)

2. in a multi-project-under-single-trunk repository, whether a tag/branch is
created as a copy of a particular project's directory or of the whole trunk
is a matter of policy, and if not consistent, can really screw up people's
attempts to switch a working copy to such a branch.

[David Glasser]

Post by C. Michael Pilato
1. all that flexibility makes it harder for Subversion and other third-party
tools to recognize and treat a directory copy as a branch. Common questions
like, "What tags exist?" can't be answered unless the answering system first
understands the policy used to denote tags in a given repository ("stuff
immediately under /tags"; "stuff immediately under an immediate subdirectory
of /tags"; ...)

... and as an extension of this, it leads directly to the whole "no
project-wide configuration" can of worms.

--dave

[B. Smith-Mannschott]

Post by David Glasser

Post by C. Michael Pilato
1. all that flexibility makes it harder for Subversion and other third-party
tools to recognize and treat a directory copy as a branch. Common questions
like, "What tags exist?" can't be answered unless the answering system first
understands the policy used to denote tags in a given repository ("stuff
immediately under /tags"; "stuff immediately under an immediate subdirectory
of /tags"; ...)

... and as an extension of this, it leads directly to the whole "no
project-wide configuration" can of worms.
--dave

Wouldn't this the sort of thing properties are for? We could use a
property to record the meaning we intend the directory to have. i.e.
it's "role".

e.g. suppose we could say:

svn cp --tag SOURCE DESTINATION
svn cp --branch SOURCE DESTINATION

which would behave just like svn cp, except that it sets a property
svn:role=tag or svn:role=branch.

Perhaps the repository could even keep track of which directories
currently visible in head have an svn:role and provide some efficient
way to list them. Their history could tell us what they are tags or
branches *of*.

If we marked the directories in this way, we wouldn't have to know
policy to browse existing tags. (Only if we wanted to abstract their
creation somehow.)

Just a thought
// ben smith-mannschott

[Steinar Bang]

Post by B. Smith-Mannschott
Wouldn't this the sort of thing properties are for? We could use a
property to record the meaning we intend the directory to have. i.e.
it's "role".

To me, this sounds like a way you should go.

Mark the semantics of a directory that is a branch or a tag
explicitly, instead of by convention, from its location in the
repository.

The main thing that has bugged me with subversion from the start, is
that the semantics of branches or tags have existed only in user's
head, and in GUI tool heuristics, rather than being marked up
explictly in the repository.

[Folker Schamel]

Post by Ph. Marek

Post by Branko Čibej
Compared to all the real pain it's causing.

What pain does it cause?
Make "tag:A" be a synonym for "$URL/tags/A", and if another layout is present,
require a "svn:layout" property in $URL.
Then use "svn cp . tag:A", or "svn diff tag:1.1 tag:1.2" or whatever.
Similar for branches. Result? No visible difference to CVS (for the user).
(Please take that with a grain of salt. I have surely overlooked some point
which makes that more problematic).
[Of course, that could clash with files named "tag:*". But this problem exists
with directories named "http:", too, no?)

I think a shortcut for the repository root including "/" (e.g. "$")
would be absolutely sufficient:

Then use "svn cp . $tags/A", or "svn diff $tags/1.1 $tags/1.2" or whatever.

No need for svn:layout property.

Post by Ph. Marek
1. all that flexibility makes it harder for Subversion and other third-party
tools to recognize and treat a directory copy as a branch. Common questions
like, "What tags exist?" can't be answered unless the answering system first
understands the policy used to denote tags in a given repository ("stuff
immediately under /tags"; "stuff immediately under an immediate subdirectory
of /tags"; ...)

When opening the TortoiseSVN repo-browser,
you see trunk, tags, and branches.
When clicking the tags folder, you see all top-level tags.

And if you really want to optimize away this single mouse click,
then you could implement a new menu item "Open tags/ in repository browser".

BTW, TortoiseSVN handles /tags in a special way at some places.
For example, when trying to commit to /tags/, you get a warning.

Personally I like the tag/branch concept of svn very much,
I think it is much simpler and more intuitive than the CVS mechanism.

Cheers,
Folker

[Branko Čibej]

Post by Folker Schamel
I think a shortcut for the repository root including "/" (e.g. "$")
Then use "svn cp . $tags/A", or "svn diff $tags/1.1 $tags/1.2" or whatever.
No need for svn:layout property.

So you're already assuming that there is one single fixed layout, and no
flexibility. Definitely not something I'd support.

-- Brane

[Folker Schamel]

Post by Branko Čibej

Post by Folker Schamel
I think a shortcut for the repository root including "/" (e.g. "$")
Then use "svn cp . $tags/A", or "svn diff $tags/1.1 $tags/1.2" or whatever.
No need for svn:layout property.

So you're already assuming that there is one single fixed layout, and no
flexibility. Definitely not something I'd support.

You still have all flexibility, for example

"svn cp . $project1/tags/A"
or
"svn diff $project2/tags/1.1 $project37/branches/releases/customer_a/1.2",
or whatever.

Cheers,
Folker

[Marc Haisenko]

Post by Folker Schamel

Post by Branko Čibej

Post by Folker Schamel
I think a shortcut for the repository root including "/" (e.g. "$")
Then use "svn cp . $tags/A", or "svn diff $tags/1.1 $tags/1.2" or whatever.
No need for svn:layout property.

So you're already assuming that there is one single fixed layout, and no
flexibility. Definitely not something I'd support.

You still have all flexibility, for example
"svn cp . $project1/tags/A"
or
"svn diff $project2/tags/1.1 $project37/branches/releases/customer_a/1.2",
or whatever.
Cheers,
Folker

Make that something that won't be evaluated as a variable, like +root+ or just
to give and example, and the idea does indeed have something to think about,
IMHO.
Marc

[C. Michael Pilato]

Post by Branko Čibej

Post by Folker Schamel
I think a shortcut for the repository root including "/" (e.g. "$")
Then use "svn cp . $tags/A", or "svn diff $tags/1.1 $tags/1.2" or whatever.
No need for svn:layout property.

So you're already assuming that there is one single fixed layout, and no
flexibility. Definitely not something I'd support.

Right.

There are applications like Cenqua's FishEye repository browser which allow
you to describe your TTB structure so that the notions of "branch" and "tag"
can be applied to particular directories. (See
http://www.cenqua.com/fisheye/doc/latest/admin/svnsymbolic.html)
But I admit that it is a shame that you pretty much *have* to implement
something like this across all similar applications to get that sort of
functionality.

[Ph. Marek]

Post by Folker Schamel
I think a shortcut for the repository root including "/" (e.g. "$")
Then use "svn cp . $tags/A", or "svn diff $tags/1.1 $tags/1.2" or whatever.

Ok.

Post by Folker Schamel
No need for svn:layout property.

Hmmm... take it a step further, and make a "svn:shortcut" which might list "$"
as "/"? Then set "T" as "$/tags", and you can use T/1.4.0 ...

Post by Folker Schamel

Post by C. Michael Pilato
1. all that flexibility makes it harder for Subversion and other
third-party tools to recognize and treat a directory copy as a branch.
Common questions like, "What tags exist?" can't be answered unless the
answering system first understands the policy used to denote tags in a
given repository ("stuff immediately under /tags"; "stuff immediately
under an immediate subdirectory of /tags"; ...)

... and as an extension of this, it leads directly to the whole "no
project-wide configuration" can of worms.

Then make a "project-wide" configuration! Simply require that the top level
directory checked out *must* include a "svn:layout" or whatever.

I believe that the additional complexity by giving a new *dimension* -- both
in code, and *in explanations* (where Branko already said that it's difficult
to grasp for users!) is not worth the "magic", esp. if you can get it in
easier terms.
(Whether we'd *really* get all the needed functionality, I don't know. I
didn't see a chart "What we have/What is needed" listing what is missing now,
so I can only interpolate).

Post by Folker Schamel
When opening the TortoiseSVN repo-browser,
you see trunk, tags, and branches.
When clicking the tags folder, you see all top-level tags.
And if you really want to optimize away this single mouse click,
then you could implement a new menu item "Open tags/ in repository browser".
BTW, TortoiseSVN handles /tags in a special way at some places.
For example, when trying to commit to /tags/, you get a warning.
Personally I like the tag/branch concept of svn very much,
I think it is much simpler and more intuitive than the CVS mechanism.

+1


Regards,

Phil

[Graham Leggett]

Post by Ph. Marek

Post by Folker Schamel
No need for svn:layout property.

Hmmm... take it a step further, and make a "svn:shortcut" which might list "$"
as "/"? Then set "T" as "$/tags", and you can use T/1.4.0 ...

Looking at this, having such a syntax is fine for highly technical people
for whom seeing $something makes perfect sense, but for other people who
use version control, say to manage documentation, would see $something is
incomprehensible.

What may be useful are two properties called svn:tags and svn:branches.

You would assign the property onto your trunk directory, and it would give
you a relative path to where you might find your tags, or your branches,
eg:

svn:tags ../tags
svn:branches ../branches

You don't necessarily have to put it on the trunk directory. You might do
something like this:

trunk/module
svn:tags ../../tags/module
svn:branches ../../branches/module

Extending this further, you might have an svn:trunk property that tells
you where your trunk lives, and the property might be applied to your tags
directory, or your branches directory.

Regards,
Graham
--

[Nik Clayton]

Post by Folker Schamel
I think a shortcut for the repository root including "/" (e.g. "$")
Then use "svn cp . $tags/A", or "svn diff $tags/1.1 $tags/1.2" or whatever.

<bikeshed colour="blue">

'$' is a bad choice because of the need to quote it in shells.

I like the SVK approach. You have a client-side configuration file that
maps prefixes to URLs. It's like the CVS 'modules' feature, but
client-side rather than server-side.

Instead of using URLs on the command line, you use a path that looks like:

/<prefix>/<rest-of-path>

So with a configuration like this, and assuming that svn(1) supported
this syntax.

'': svn://host/repo/path
svn: http://svn.collab.net/repos/svn/
svnt: http://svn.collab.net/repos/svn/trunk

you could do:

svn checkout //trunk
is svn checkout svn://host/repo/path

or

svn checkout /svn/trunk
is svn checkout http://svn.collab.net/repos/svn/trunk

or

svn checkout /svnt
is svn checkout http://svn.collab.net/repos/svn/trunk

svn copy /svnt /svn/branches/my-first-branch
is svn copy http://svn.collab.net/repos/svn/trunk
http://svn.collab.net/repos/branches/my-first-branch

Any reason why libsvnclient couldn't be modified to support this syntax?

N
</bikeshed>

[Roy Franz]

Post by C. Michael Pilato

Post by Graham Leggett

Post by Branko Čibej
Why? I'm really interested: What, exactly, do you (we) gain by that?

The concept returns to the original method of version control that people
used before there was version control, and that was to "make a copy of a
the source directory". Tags modelled like directories are easy for new
users to understand.

Post by Branko Čibej
Compared to all the real pain it's causing.

Having introduced this concept during a number of migrations to
subversion, I honestly haven't seen anyone go through any pain.

Yeah, I have to say that in my experience, Subversion's approach to
branching and tagging has only ever been a benefit to them in terms of
reducing a complex concept to a comprehensible one.
I do see seasoned version control users get tripped up, but it's typically
in slinging the specific command lines needed for merges, not in the
"concept comprehension" department.
There are only two realistic complaints that I can recall seeing leveraged
1. all that flexibility makes it harder for Subversion and other third-party
tools to recognize and treat a directory copy as a branch. Common questions
like, "What tags exist?" can't be answered unless the answering system first
understands the policy used to denote tags in a given repository ("stuff
immediately under /tags"; "stuff immediately under an immediate subdirectory
of /tags"; ...)
2. in a multi-project-under-single-trunk repository, whether a tag/branch is
created as a copy of a particular project's directory or of the whole trunk
is a matter of policy, and if not consistent, can really screw up people's
attempts to switch a working copy to such a branch.
--
CollabNet <> www.collab.net <> Distributed Development On Demand

These two complaints are very real, and very serious.

I recently led a CVS to SVN migration a repository at work, and
for many developers it was not a positive experience, mostly due to
how branches and tags are (not) handled by SVN. A number of
developers remain unhappy with it after several months. All these
developers 'get' branching and tagging in CVS and SVN, and most
previously used branching in CVS. Our primary reason for moving
to SVN was better branching (CVS is basically unusable) and directory
tracking. Some of us are realizing those benefits, but many
developers just see the regressions.

Some of the big problems are:

* Subversion makes some very useful operations much more difficult than CVS:
- for a given file, provide a list of tags and their corresponding
revisions. (We are interested in where the file changed, so the
repository revision created by the tag is not interesting. If 10 tags
where made from the same version of the file on trunk, they should
have the same revision.) SVN would also need to provide paths for the
revisions due to tags/branches being in the normal filesystem
namespace.)
- for a given file, draw a graph that shows branches and tags.
TKCVS does this very nicely and instantaneously for CVS. TKSVN sort
of does this, but the implementation is a hack, and it only works on
repositories with 'clean' history. Even when it works it is orders of
magnitude slower. (Someone accidentally removed the entire 'tags'
directory from the repository. Doing the simple fix of just copying
it from a previous revision completely broke TKSVN due to log
--stop-on-copy now giving the revision where the copy of the tags
directory was done. We had to do a dump/load to fix TKSVN.) Drawing
these graphs in a robust manner requires processing the complete
history of the repository - even if a program implemented this, it
would be very slow.

The branch/tag handling in SVN is a nice implementation, but it is
lacking in its
handling of the difference in meaning between
directories/tags/branches. This is all left to convention, and other
tools, which means that there is no consistent way of doing things.
These problems seem to have been defined as 'someone else's problem',
so they remain outside the scope of things that should be fixed in
subversion. In this aspect SVN is much more a versioned file system
rather than a version control system. For instance, 'log
--stop-on-copy' is used as a proxy to determine when a branch/tag was
created, but this is not always right (see above about tags dir
deletion.)

I hope that as Subversion continues to mature that these issues will
be addressed.
I think that many users expect a version control system to care about
the concepts of branches and tags, even if the underlying
implementation doesn't care.

Roy

[Karl Fogel]

Post by Roy Franz
I hope that as Subversion continues to mature that these issues will
be addressed.
I think that many users expect a version control system to care about
the concepts of branches and tags, even if the underlying
implementation doesn't care.

Couldn't have said it better. Thanks, Roy, that was a much-needed
description of how our branches look from the user's point of view.

[Danny van Heumen]

Post by Karl Fogel

Post by Roy Franz
I hope that as Subversion continues to mature that these issues will
be addressed.
I think that many users expect a version control system to care about
the concepts of branches and tags, even if the underlying
implementation doesn't care.

Couldn't have said it better. Thanks, Roy, that was a much-needed
description of how our branches look from the user's point of view.

What bothers me most is the fact that, at the moment, 'tags' and
'branches' are "the thing".
(I'm looking at this from a point of view where 'branches' and 'tags'
are just ways of working with existing data for i.e. stabilizing an
application.)

But what if tomorrow some VCS invents something new? I do acknowledge
the fact that there's a great need for a user interface that makes
tagging/branching easier, quicker and shorter (length of the command) to
do. But I don't think it should be something to implement in the main
application.

Danny

[Roy Franz]

Post by Danny van Heumen

Post by Karl Fogel

Post by Roy Franz
I hope that as Subversion continues to mature that these issues will
be addressed.
I think that many users expect a version control system to care about
the concepts of branches and tags, even if the underlying
implementation doesn't care.

Couldn't have said it better. Thanks, Roy, that was a much-needed
description of how our branches look from the user's point of view.

What bothers me most is the fact that, at the moment, 'tags' and
'branches' are "the thing".
(I'm looking at this from a point of view where 'branches' and 'tags'
are just ways of working with existing data for i.e. stabilizing an
application.)
But what if tomorrow some VCS invents something new? I do acknowledge
the fact that there's a great need for a user interface that makes
tagging/branching easier, quicker and shorter (length of the command) to
do. But I don't think it should be something to implement in the main
application.
Danny
---------------------------------------------------------------------

Tags and branches have been widely used in VCS systems for well over a decade,
so I think they are more than a fad. If a better model develops in
the future it can be evaluated at that time. Putting off fixing
current problems because a future invention might make them obsolete
is not good reasoning. (I don't mean to imply that tags/branches are
the be-all/end-all, but they are quite effective for many uses.)

Also, while I am one of the people hoping for improved 'first-class'
treatment of tags/branches in subversion, I don't see that this
necessarily implies a big change to the versioned filesystem that
Subversion implements. For example, it is probably possible to
address most of these tag/branch issues by just keeping additional
metadata about certain operations. The versioned filesystem with cheap
copies that SVN provides is very elegant, but I don't see any reason
that this would need to be compromised to improve in these other
areas.


Roy

[Mark Reibert]

Post by Roy Franz
Also, while I am one of the people hoping for improved 'first-class'
treatment of tags/branches in subversion, I don't see that this

+1, if we are voting!

Like you, I admire the simplicity and elegance of a tag/branch being
just a copy. On the other hand, the fact that these operations are
special only by convention makes administering and controlling them
difficult. For example, in days gone by under CVS we allowed anyone to
place a "private" tag on objects, whereas only repository admins could
create "official" tags (e.g., for a release). Private tags began with
"p_<user_initials>". Official tags were automatically inserted into
Bugzilla as versions of the associated product.

This was extremely easy to setup and enforce with a trigger script under
CVS, since tagging is a sentient operation. With Subversion, as far as I
can tell this requires a pre-commit hook that inspects the target of the
copy and needs to use a set of heuristics to figure out the user's
intent. Moreover, it seems the user could easily subvert the attempted
control by simply choosing a location or name that is not part of the
heuristics.

Just my $0.02 worth ...

Mark

[Folker Schamel]

Post by Mark Reibert

Post by Roy Franz
Also, while I am one of the people hoping for improved 'first-class'
treatment of tags/branches in subversion, I don't see that this

+1, if we are voting!
Like you, I admire the simplicity and elegance of a tag/branch being
just a copy. On the other hand, the fact that these operations are
special only by convention makes administering and controlling them
difficult. For example, in days gone by under CVS we allowed anyone to
place a "private" tag on objects, whereas only repository admins could
create "official" tags (e.g., for a release). Private tags began with
"p_<user_initials>". Official tags were automatically inserted into
Bugzilla as versions of the associated product.
This was extremely easy to setup and enforce with a trigger script under
CVS, since tagging is a sentient operation. With Subversion, as far as I
can tell this requires a pre-commit hook that inspects the target of the
copy and needs to use a set of heuristics to figure out the user's
intent. Moreover, it seems the user could easily subvert the attempted
control by simply choosing a location or name that is not part of the
heuristics.
Just my $0.02 worth ...

What about pre-commit script just like the following pseudo-code?

if commit_path matches /trunk/.* or commit_path matches /branches/.*:
return Access_allowed
if commit_path matches /tags/.*:
if commit_path matches /tags/official/\w* and user == admin:
return Access_allowed
if commit_path matches /tags/private/<user>/\w*:
return Access_allowed
return Access_denied

Cheers,
Folker

[Mark Reibert]

Post by Folker Schamel

Post by Mark Reibert
Like you, I admire the simplicity and elegance of a tag/branch being
just a copy. On the other hand, the fact that these operations are
special only by convention makes administering and controlling them
difficult. For example, in days gone by under CVS we allowed anyone to
place a "private" tag on objects, whereas only repository admins could
create "official" tags (e.g., for a release). Private tags began with
"p_<user_initials>". Official tags were automatically inserted into
Bugzilla as versions of the associated product.
This was extremely easy to setup and enforce with a trigger script under
CVS, since tagging is a sentient operation. With Subversion, as far as I
can tell this requires a pre-commit hook that inspects the target of the
copy and needs to use a set of heuristics to figure out the user's
intent. Moreover, it seems the user could easily subvert the attempted
control by simply choosing a location or name that is not part of the
heuristics.
Just my $0.02 worth ...

What about pre-commit script just like the following pseudo-code?
return Access_allowed
return Access_allowed
return Access_allowed
return Access_denied

Hello Folker,

This approach works only if all the users behave well; that is, place
their tags in /tags. In my experience this is not always the case. The
less experienced users will start copying files all over in /trunk
because they do not understand the convention, whereas the experienced
users may be inclined to start tagging in /trunk just to side-step the
attempted controls.

Thanks,
Mark

[Folker Schamel]

Post by Mark Reibert

Post by Folker Schamel

Post by Mark Reibert
Like you, I admire the simplicity and elegance of a tag/branch being
just a copy. On the other hand, the fact that these operations are
special only by convention makes administering and controlling them
difficult. For example, in days gone by under CVS we allowed anyone to
place a "private" tag on objects, whereas only repository admins could
create "official" tags (e.g., for a release). Private tags began with
"p_<user_initials>". Official tags were automatically inserted into
Bugzilla as versions of the associated product.
This was extremely easy to setup and enforce with a trigger script under
CVS, since tagging is a sentient operation. With Subversion, as far as I
can tell this requires a pre-commit hook that inspects the target of the
copy and needs to use a set of heuristics to figure out the user's
intent. Moreover, it seems the user could easily subvert the attempted
control by simply choosing a location or name that is not part of the
heuristics.
Just my $0.02 worth ...

What about pre-commit script just like the following pseudo-code?
return Access_allowed
return Access_allowed
return Access_allowed
return Access_denied

Hello Folker,
This approach works only if all the users behave well; that is, place
their tags in /tags. In my experience this is not always the case. The
less experienced users will start copying files all over in /trunk
because they do not understand the convention, whereas the experienced
users may be inclined to start tagging in /trunk just to side-step the
attempted controls.
Thanks,
Mark

Well, no tagging mechanism on earth can prevent this.

The only way to prevent this would be using a version control system
which does not support directory copying.
And even then, a user always can emulate private tags
by just making a local disk copy of his working copy.

I think you have a social problem, no technical problem.

Cheers,
Folker

[Mark Reibert]

Post by Folker Schamel

Post by Mark Reibert
This approach works only if all the users behave well; that is, place
their tags in /tags. In my experience this is not always the case. The
less experienced users will start copying files all over in /trunk
because they do not understand the convention, whereas the experienced
users may be inclined to start tagging in /trunk just to side-step the
attempted controls.
Thanks,
Mark

Well, no tagging mechanism on earth can prevent this.
The only way to prevent this would be using a version control system
which does not support directory copying.
And even then, a user always can emulate private tags
by just making a local disk copy of his working copy.
I think you have a social problem, no technical problem.

Of course ... there is certainly a social aspect---and possibly even a
large one---to this problem. As I wrote, there are no problems if all
users behave well!

But I don't want to get too far off track. My point is simply the
technical solution is more complicated because tagging is based purely
on convention. The corollary is it becomes easier for users to avoid the
SCM process one is trying to implement, often times simply because they
do not remember the convention.

The tagging-by-convention model in SVN is manageable in a code-centric
environment where large chunks of the repository (often the entire
trunk) are tagged as a unit, by someone who understands what he is
doing. My frustration stems from attempting to use SVN/TSVN as a
revision tool for general-purpose documentation. Try to explain to your
PHB who is working on a contract nested 8 directories deep in the
repository that he should create the same directory structure in this
other, magical "tags" place and copy his file in there. He will look at
you like you are drunk and just copy "foo.doc" to "foo 7/9/07a.doc".
(Using the stupid U.S. non-sorting m/d/yy date format - and I am an
American so I can write that! ;-). My belief is I would have much better
luck if instead I gave the same PHB a simple command that works "in
place" so he does not have to abstract himself outside of his single,
8-level-deep directory.

Maybe you are right ... I guess it is a social problem!

Enjoy,
Mark

[Folker Schamel]

Post by Mark Reibert

Post by Folker Schamel

Post by Mark Reibert
This approach works only if all the users behave well; that is, place
their tags in /tags. In my experience this is not always the case. The
less experienced users will start copying files all over in /trunk
because they do not understand the convention, whereas the experienced
users may be inclined to start tagging in /trunk just to side-step the
attempted controls.
Thanks,
Mark

Well, no tagging mechanism on earth can prevent this.
The only way to prevent this would be using a version control system
which does not support directory copying.
And even then, a user always can emulate private tags
by just making a local disk copy of his working copy.
I think you have a social problem, no technical problem.

Of course ... there is certainly a social aspect---and possibly even a
large one---to this problem. As I wrote, there are no problems if all
users behave well!
But I don't want to get too far off track. My point is simply the
technical solution is more complicated because tagging is based purely
on convention. The corollary is it becomes easier for users to avoid the
SCM process one is trying to implement, often times simply because they
do not remember the convention.
The tagging-by-convention model in SVN is manageable in a code-centric
environment where large chunks of the repository (often the entire
trunk) are tagged as a unit, by someone who understands what he is
doing. My frustration stems from attempting to use SVN/TSVN as a
revision tool for general-purpose documentation. Try to explain to your
PHB who is working on a contract nested 8 directories deep in the
repository that he should create the same directory structure in this
other, magical "tags" place and copy his file in there. He will look at
you like you are drunk and just copy "foo.doc" to "foo 7/9/07a.doc".
(Using the stupid U.S. non-sorting m/d/yy date format - and I am an
American so I can write that! ;-). My belief is I would have much better
luck if instead I gave the same PHB a simple command that works "in
place" so he does not have to abstract himself outside of his single,
8-level-deep directory.
Maybe you are right ... I guess it is a social problem!

I think I understand your situation.

And even if it is a social problem, I think you have a good point
that sometimes social problems can be improved or even solved
by technology, for example by better UIs.

In your particular case, my first impression is that TSVN
should provide more explicit tagging support.
For example, just a context menu "Create tag".

And maybe some standardized "layout" property really makes sense.

Cheers,
Folker

[Mark Reibert]

Post by Folker Schamel

Post by Mark Reibert
The tagging-by-convention model in SVN is manageable in a code-centric
environment where large chunks of the repository (often the entire
trunk) are tagged as a unit, by someone who understands what he is
doing. My frustration stems from attempting to use SVN/TSVN as a
revision tool for general-purpose documentation. Try to explain to your
PHB who is working on a contract nested 8 directories deep in the
repository that he should create the same directory structure in this
other, magical "tags" place and copy his file in there. He will look at
you like you are drunk and just copy "foo.doc" to "foo 7/9/07a.doc".
(Using the stupid U.S. non-sorting m/d/yy date format - and I am an
American so I can write that! ;-). My belief is I would have much better
luck if instead I gave the same PHB a simple command that works "in
place" so he does not have to abstract himself outside of his single,
8-level-deep directory.
Maybe you are right ... I guess it is a social problem!

I think I understand your situation.
And even if it is a social problem, I think you have a good point
that sometimes social problems can be improved or even solved
by technology, for example by better UIs.
In your particular case, my first impression is that TSVN
should provide more explicit tagging support.
For example, just a context menu "Create tag".
And maybe some standardized "layout" property really makes sense.

TSVN does have a "Branch/tag" option, which is helpful but still
requires the user to type in the full URL for the target of the copy. So
really it is the layout in /tags - and mirroring /trunk when you are not
tagging at a high-level directory - that is cumbersome.

This is not a deal breaker, or course. It is just one problem we did not
have when typing "cvs tag foo bar".

[Vincent Lefevre]

Post by Roy Franz
- for a given file, provide a list of tags and their corresponding
revisions. (We are interested in where the file changed, so the
repository revision created by the tag is not interesting. If 10 tags
where made from the same version of the file on trunk, they should
have the same revision.) SVN would also need to provide paths for the
revisions due to tags/branches being in the normal filesystem
namespace.)

But it is possible to write a tool that does what you want (with some
kind of cache to make it fast). Note that CVS has much more problems
due to the asymmetry between the trunk, branches and tags, and I really
prefer Subversion.

Post by Roy Franz
- for a given file, draw a graph that shows branches and tags.
TKCVS does this very nicely and instantaneously for CVS. TKSVN sort
of does this, but the implementation is a hack, and it only works on
repositories with 'clean' history. Even when it works it is orders of
magnitude slower. (Someone accidentally removed the entire 'tags'
directory from the repository.

Well, why not setting up a hook that would prevent such "invalid"
operations? (BTW, such a hook should be provided with Subversion
as it would be useful for many users.)

Also note that things can be much worse with CVS, due to the fact
that tags are not versioned.

[David Waite]

I believe they are speaking to how .svn files are in every (sub)directory of
a checkout. As an example, if I check out the root of a repository broken
into /branches, /tags, /trunk, etc. I can take the trunk directory, move it
out of the way and delete the rest, and things will still work. I believe
there is also minimal support for detecting that a directory was moved but
not svn mv'd (although in my experience, it is more that you will get
freak-outs than that this will do something which would be identified as
intelligent).
-DW

Post by Garance A Drosihn

Post by Karl Fogel
We also have to be faster. Fortunately, we've pretty much agreed,
IIRC, that we're willing to punt on subdirectory detachability in
working copies in order to get performance improvements.

What is meant by "subdirectory detachability"? Just curious. I
didn't see it mentioned in the book on subversion I have, and
google doesn't want to help me with a likely reference...
--
---------------------------------------------------------------------

[John Szakmeister]

Sorry if this is a repeat, but my first post didn't seem to make it through.

Post by Karl Fogel
I've been wanting to post this for a while, but was waiting for the
dust from Linus Torvald's GIT talk to settle first (for those who
haven't seen it: http://youtu.be/4XpnKHJAok8 Eric
Raymond's recent post thanking the Subversion team gives me the excuse
I needed to finally sit down and write this :-). (Eric's post is at
http://subversion.tigris.org/servlets/ReadMsg?list=dev&msgNo=128106.)
In his talk, Torvalds explained why he thinks decentralized version
control systems (like GIT and Mercurial) are the way of the future,
and why he thinks Subversion got it all wrong. I think that's a
misanalysis, and will describe why below. Unfortunately, Torvalds
also indulged in a childish presentation style that distracted from
his useful technical criticisms of Subversion. Since I'd like to use
some of his arguments as a jumping-off point for thoughts on
* Optimizing merging is as important as optimizing branching (if
not more so).
* Speed matters: when a common operation goes from thirty seconds
to half a second, that changes the whole way you work.
* Having all history locally (or at least as much history as you
need for a given operation) is useful.
* Reducing the thickness of the "commit access" wall is good for
development. Torvalds didn't make this argument terribly well,
The important question is, who can put changes into the
repository that the project is publishing releases from? This
should not be confused with commit access in the technical sense.
Instead, think of it this way: committing is just a way to
connect changes to other changes, and you shouldn't need my trust
in order to connect your changes to anything you want to connect
them to. The real question is, when and how do I include your
changes in my release? So the issue isn't commit access, it's
having trust networks and convenient methods of change selection.
When I talked to Brian Fitzpatrick about this, he listed three things
- Faster. Subversion does need to be faster for many ops.
- Offline commits.
- Local branches.

I'm not sure what "local branches" are... but if it's an opportunity to
try a few things (versioned) locally, and then submit the result to the
repository, then a huge +1 to that.

Post by Karl Fogel
I would add "better merging", but basically agree with Fitz (note that
we're getting much-improved merging in Subversion 1.5).

A big +1 on better merging. We're definitely getting better merge
support in 1.5 (and thanks to all involved for that!). However, as
dberlin pointed out in a separate thread, there are some cases that
would be tough to solve without a changeset DAG.

And while we're at it, the number one complaint that I hear over, and
over again: not being able to query the complete history of a file
efficiently. I get emails and calls from customers and other
individuals all the time about this (I don't do Subversion support, but
my customers know that I'm involved in the community). They desperately
want that ability. Hidden under this, they also want branches and tags
to be first class citizens, and have revision aliases.

[snip]

Post by Karl Fogel
A general tool configured to behave in a specific way is never quite
as natural to use as a tool designed for that specific use in the
first place. In other words, Subversion can -- will have to -- take
on some of the features of decentralized VC systems, but it will never
be as good a decentralized system as they are. By the same token, a
decentralized system can be configured to work like a centralized one,
but will never be as good at it as Subversion is. The trick for us is
to keep the centralization feature without some of the limitations
that have traditionally come with centralization.

Argeed. I'd definitely like to see some features of a decentralized VC
incorporated, but not if it means overly complicating the user interface.

Post by Karl Fogel
Concretely, what does this mean?
One of Subversion's flaws (mea culpa) is that we didn't realize the
usefulness of having symmetrical functionality on the client and
server sides. The working copy should really be a repository, even if
it's not always going to store all the history available on the server
side (with some projects, you really can't, it's too big).

...and we definitely don't want to download an entire repository that
hosts multiple (potentially large) projects.

Post by Karl Fogel
So we're going to need a working copy rewrite. We knew that; in fact
we've talked about rewriting the repository to use something like
Mercurial's revlog format, for various reasons, and about using that
kind of repository for working copies as well.

One big issue that I see here is that the repository was designed not to
forget. So we'll need to really think about how to manage the working
copy area, without it growing to large... and more importantly, without
user intervention.

I've played with SVK, and one of the big disappointments was that when I
was done with a project, I still had all of that projects history on my
computer. I had no way to remove it without affecting other mirrored
projects. :-(

Post by Karl Fogel
We also have to be faster. Fortunately, we've pretty much agreed,
IIRC, that we're willing to punt on subdirectory detachability in
working copies in order to get performance improvements.

+100! I think this has been standing in the way of other things, like
rename tracking, as well.

Post by Karl Fogel
And now I'm going to hand-wave on a lot of details. I don't mean to
start the Subversion 2.0 design thread now, just to offer some
thoughts on general goals. We don't need to let labels guide our
thinking ("We are a centralized system!" / "We are a decentralized
system!"). We do need to recognize that users are not interested in
becoming version control experts, and we need to pay close attention
to what they actually want, as opposed to what experts might want them
to want.
Case in point: what's the most popular feature added to Subversion
after the 1.0 release? Probably file locking (the ultimate
centralized feature, by the way). Yes, the heavy-duty developers wish
and irc.freenode.net/#svn, talking to companies that do Subversion
support, and from doing some Subversion consulting myself, I think
locking was actually a more important feature. (Of course, we have it
already, so that doesn't change anything about Subversion's future,
I'm just making a point about what's important to users.)

Agreed. Linus's standpoint was not to put anything over 10MB into the
repository (IIRC), and he completely neglected to address merge
resolution of binary files in a decentralized environment.
Unfortunately, in the real world, we have to deal with these things.
For instance, we do a lot of hardware work at my company. We need
firmware images from 3rd parties, schematic captures of the design,
binary test inputs, disk images, drawings, documentation, and
specialized tools in the repository. We've got several repositories
10's of gigabytes in size, because of all the binary data surrounding
it. Subversion's locking helped people to prevent wasting time, and
that's extremely important in our fast-paced environment.

Post by Karl Fogel
Subversion's phenomenal adoption rate (*) isn't due to being the only
game in town. We never were, if you count the proprietary systems,
and we're even less so now that the open source version control world
has become so fertile. The reason Subversion is taking over the world
is because it is tremendously user-focused, and because it provides
well-documented APIs that enable other developers to write software on
top of Subversion. We should copy what we need from the decentralized
systems, but remember that most users don't know or care whether a
system is centralized or decentralized -- their ideal system is one
they don't notice. Let's keep our eye on the ball, so they don't have
to.

Agreed. A number of customers that I work with chose Subversion
primarily because it was easy to get up and running compared to other
alternatives. They recognized that the couldn't gain traction in their
organizations unless the tool was easy... and they were right.

-John

[Blair Zajac]

Post by John Szakmeister
One big issue that I see here is that the repository was designed not to
forget. So we'll need to really think about how to manage the working
copy area, without it growing to large... and more importantly, without
user intervention.
I've played with SVK, and one of the big disappointments was that when I
was done with a project, I still had all of that projects history on my
computer. I had no way to remove it without affecting other mirrored
projects. :-(

I set up separate depots for each remote repository I would use, then
you can freely drop them when you need to.

Regards,
Blair

[Steinar Bang]

One thing Raymond didn't touch on was the _reason_ people stayed with
CVS, in spite of all of its flaws: you could trust it not to break
your data. And here I mean break the data by itself, not through
misuse.

And _that_ is where I see subversion as the only free CVS replacement
candidate: subversion can be trusted not to break your data.

I can't say I have the same feeling about the rest. Distributed or
not becomes a secondary issue.

[Karl Fogel]

Post by Steinar Bang
One thing Raymond didn't touch on was the _reason_ people stayed with
CVS, in spite of all of its flaws: you could trust it not to break
your data. And here I mean break the data by itself, not through
misuse.
And _that_ is where I see subversion as the only free CVS replacement
candidate: subversion can be trusted not to break your data.
I can't say I have the same feeling about the rest. Distributed or
not becomes a secondary issue.

I haven't heard of data-loss problems with any of the other systems.
(I do know people who use Mercurial, GIT, and SVK on a regular basis.)

FWIW,
-Karl

[Steinar Bang]

Post by Karl Fogel
I haven't heard of data-loss problems with any of the other systems.
(I do know people who use Mercurial, GIT, and SVK on a regular basis.)

I didn't mean to imply that they did.

But when I read about them, it's all about changesets and distributed
repositories. Not about the infrastructure behind. Not about lots of
tests, and tidy releases.