Discussion:
[fossil-users] "how to use git to lose data"
Stephan Beal
2014-09-01 15:29:41 UTC
Permalink
Okay, more git bashing...

After losing 3 hours' work today from something as simple as "git stash
save" (where git stashed 3000+ untracked/generated files, despite the docs
saying it doesn't do that), then not being able to do "stash apply"
(because "file already exists..." for 3000+ files), and having to construct
the work from screenshots of the diff i was able to grab before deleting
and checking out again (as one so often has to do with git), i went
searching for "data loss in git" and stumbled across this page near the top
of the results:

http://www.cs.cmu.edu/~davide/howto/git_lose.html

What really makes that worth reading is the list of suggestions at the end
of the page. They start out with this little gem:


-

Internalize the concept that git is *designed* to forget things. If you
haven't seen something reach another repository, maybe it didn't. Heck,
even if you *did* see it go somewhere else, maybe it fell out of the
historical record there and then got garbage-collected.


It makes me sick to no end that people accept that so readily, and then go
back for a second helping.

It occurred to me today that in nearly 31 years of using a computer i have,
in total, lost more data to git (while following the instructions!!!) than
any other single piece of software. Also concluded is that git is the only
SCM out there which makes SCM difficult for the simple stuff. Even RCS is
simpler to use. Sure CVS has limits, but respect those limits and it works
just fine. Never lost a line of code in CVS.

...
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
Scott Robison
2014-09-01 21:49:02 UTC
Permalink
Post by Stephan Beal
Okay, more git bashing...
{snipped stuff went here}
It occurred to me today that in nearly 31 years of using a computer i
have, in total, lost more data to git (while following the instructions!!!)
than any other single piece of software. Also concluded is that git is the
only SCM out there which makes SCM difficult for the simple stuff. Even RCS
is simpler to use. Sure CVS has limits, but respect those limits and it
works just fine. Never lost a line of code in CVS.


Interesting. I think I've mentioned my employer is in the midst of
converting from an svn backed model to a git based model, though slowly on
a subproject by subproject basis. I've shared this with the
git-master-chief person at work and followed it up with the following
Post by Stephan Beal
Based on reading {Stephan's message}, what do you agree or disagree with?
It seems to me (after reading this and thinking about version control
systems in a slightly new way for the first time today), git is focused
less on securely keeping track of source code and far more on providing a
toolbox of ways to reorganize code to "simplify" (I use that term loosely)
social interactions between developers / users of git (aka collaboration).
It's not that it can't keep track of source code, but that it considers the
social aspects / reorganization tools to be more important, while at the
same time being quite terse / obtuse in the documentation / usage area.
Post by Stephan Beal
Is that an unfair assessment on my part? I still readily agree that I'm a
git newbie, and even a dvcs neophyte, and the reasons I use fossil have
little to do with its distributed nature (though I'm using it more often
that way as time goes by). Also that for certain project types where large
/ deep hierarchies of collaborators are at work, fossil is probably not an
ideal solution. It certainly wouldn't work in the same way git is used by
the linux kernel team.

I'll be interested to hear back from him what he thinks.

--
Scott Robison
Stephan Beal
2014-09-02 08:12:45 UTC
Permalink
Post by Scott Robison
Post by Stephan Beal
Based on reading {Stephan's message}, what do you agree or disagree with?
FWIW: i am in the small minority of my colleagues who regularly have
problems with git. They seem to be able to do the same things, click the
same buttons, and get their code in and out of where it should be.
Post by Scott Robison
Post by Stephan Beal
It seems to me (after reading this and thinking about version control
systems in a slightly new way for the first time today), git is focused
less on securely keeping track of source code and far more on providing a
toolbox of ways to reorganize code to "simplify" (I use that term loosely)
social interactions between developers / users of git (aka collaboration).
It's not that it can't keep track of source code, but that it considers the
social aspects / reorganization tools to be more important, while at the
same time being quite terse / obtuse in the documentation / usage area.
An interesting response. Hadn't thought of it that way.
Post by Scott Robison
Post by Stephan Beal
Is that an unfair assessment on my part? I still readily agree that I'm
a git newbie, and even a dvcs neophyte, and the reasons I use fossil have
little to do with its distributed nature (though I'm using it more often
that way as time goes by). Also that for certain project types where large
/ deep hierarchies of collaborators are at work, fossil is probably not an
ideal solution. It certainly wouldn't work in the same way git is used by
the linux kernel team.
Agreed completely.
Post by Scott Robison
I'll be interested to hear back from him what he thinks.
Me as well.
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
Ron W
2014-09-02 18:09:54 UTC
Permalink
It certainly wouldn't work in the same way git is used by the linux kernel
team.
Git was originally created by the Linux Kernel team, including Linus. It's
hardly surprising that git would be a better fir for them than Fossil or
any other VCS (distributed or not).
Scott Robison
2014-09-02 18:19:49 UTC
Permalink
Post by Ron W
It certainly wouldn't work in the same way git is used by the linux
kernel team.
Post by Ron W
Git was originally created by the Linux Kernel team, including Linus.
It's hardly surprising that git would be a better fir for them than Fossil
or any other VCS (distributed or not).

That was the point I was going for. Maybe should have made it explicit.
Warren Young
2014-09-02 18:35:56 UTC
Permalink
Post by Scott Robison
the reasons I use fossil
have little to do with its distributed nature (though I'm using it more
often that way as time goes by).
A DVCS can be useful even to a lone developer. Several times since
switching from svn to Fossil, I've spent some of my disconnected travel
time working on a project backed by Fossil, and was able to check in
changes while offline, knowing I can synch later, with every detail of
my change history synched discretely, rather than as a monolithic blob.

When doing the same kind of work in my svn days, I'd end up with these
massive diffs that I had to spend a bunch of time disentangling in order
to get sane checkin comments on each aspect of the change set.

If you have more than one computer connected to a VCS and at least one
is mobile, you should be using a DVCS. Fossil vs Git is a side issue,
when it comes to that.

(This is also why I've been advocating for the uber-patch feature. It
lets outsiders contribute patches to a project they don't have commit
permissions on, without making the one applying the patch do the work of
disentangling many unrelated elements of the patch.)
Ron W
2014-09-02 20:53:07 UTC
Permalink
If you have more than one computer connected to a VCS and at least one is
mobile, you should be using a DVCS. Fossil vs Git is a side issue, when it
comes to that.
I do and I use Fossil (no surprise there, right?) because of the simplicity
of set up.

(This is also why I've been advocating for the uber-patch feature. It lets
outsiders contribute patches to a project they don't have commit
permissions on, without making the one applying the patch do the work of
disentangling many unrelated elements of the patch.)
This could be done using the --incremental feature of fossil export /
import. It's just tricky to use.

My experience with submitting patches (several different projects) has been
(a) each patch must be limited to one fix or enhancement, and (b) should
not result in merge conflicts when the dev applying the patch applies the
patch. (Generally this means pulling the latest, merging, resolving,
building and testing, then pulling and merging again to make sure. Then
create the patch and send it as quickly as possible.)
Warren Young
2014-09-02 21:09:45 UTC
Permalink
Post by Warren Young
(This is also why I've been advocating for the uber-patch feature.
My experience with submitting patches (several different projects) has
been (a) each patch must be limited to one fix or enhancement, and (b)
should not result in merge conflicts when the dev applying the patch
applies the patch.
I've been running an open source project for a decade now, so I can tell
you from experience that a lot of patches come in that do multiple
things. You can often tell that they were built up in stages, each an
independent step worth a separate checkin, but because they didn't have
checkin privileges, they had to submit the whole mess as a single big patch.

I will predict that if Fossil ever *does* get an uber-patch feature,
that I'll still get intermingled hairball patches. Still, at least it
will give the outsiders a *chance* to do the right thing.
Ron W
2014-09-02 22:07:22 UTC
Permalink
Post by Warren Young
I've been running an open source project for a decade now, so I can tell
you from experience that a lot of patches come in that do multiple things.
Apparently, the projects I've submitted patches to have stricter rules.

Now that I think of it, most projects probably can't afford such strict
rules for patches.

Not sure what I would do if I were actually running an open source project.
Warren Young
2014-09-02 23:11:40 UTC
Permalink
Post by Warren Young
I've been running an open source project for a decade now, so I can
tell you from experience that a lot of patches come in that do
multiple things.
Apparently, the projects I've submitted patches to have stricter rules.
Oh, I *have* the rule. That doesn't stop people from violating it,
though. :)
Post by Warren Young
Not sure what I would do if I were actually running an open source project.
1. Be an ogre, and yell at the person to resubmit, then 50% of the time
get no answer, so you have to...

2. Sigh, then dice the blob-o-hackage up into multiple patches, and
apply them.

Then when you get tired of the bad odds on #1, start doing #2 by default.
Stephan Beal
2014-09-03 15:58:08 UTC
Permalink
Post by Ron W
Now that I think of it, most projects probably can't afford such strict
rules for patches.
LOL! Yeah, i think most of us are just happy someone sent a patch. Others
will tell you, "won't accept until spaces are replaced with tabs" (serious
- happened recently), at which point i give up (i don't have the energy for
people who get difficult over invisible glyphs).
Post by Ron W
Not sure what I would do if I were actually running an open source project.
i think it depends on the project. sqlite, for example, has exceedingly
high stability requirements due to its "unusually wide" footprint in terms
of installations and platforms. They are necessarily quite choosy.
Likewise, i am choosier about what changes i make to fossil than those i
throw into my own repos, because people actually use Fossil to get stuff
done. Head over to one of my repos, though, and you'll likely see lots of
commits which "stable" projects wouldn't like to include.
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
John Long
2014-09-02 08:08:41 UTC
Permalink
Post by Stephan Beal
Okay, more git bashing...
Yeah. It's too easy _not_ to do.

Git is just another steaming Linux-centric pile that makes me so thankful
there are people like Dr. Hipp and you and all the fossil guys.

Consider the following points:

1) The world is not Linux
2) See (1) above. There are many operating systems that aren't Linux
3) We want something safe that uses a proven database because source control
is about controlling source, not losing it and not littering it all over
your system.
4) We want something that builds without drama on many platforms using
standards compliant compilers and without requiring gcc extensions or
specific shells like bash or any other proprietary (yes, I said it!) gnu
crap.. Git = MAJOR FAIL.
5) We want something efficient, compact and clean that doesn't depend on
scripting languages or need add-ons to be useful. It shouldn't create
dozens of executables and libraries all over the target system. Because
if it sucks we're going to wanna yank it out cleanly.
6) We don't want to drag a thousand tons of gnu prereqs just to build an
application or its documentation(!) and we don't want to depend on dozens
of prereqs to run it. Heavy = bad, light = good.
7) A source control system should be sensible from the point of view of the
person using it to manage source code. It should not be Linux-centric. It
should not require you to understand its internals to use it effectively
and it must not require you to understand the internals to avoid pitfalls
and gotchas. Tricks = bad, least surprise = good.
8) Source control is not a hobby for normal healthy people. It's not
something to become an expert in for chest-banging purposes. It's a
critical tool that's supposed to stay the hell out of the way and let
you write and keep track of code.

You guys scored a huge win by creating fossil and basing it on sqlite. It's
ingenious, it's simple, it's trustworthy, it's complete, and most of all
it's nothing we _don't_ want in a source control system!

Thank you!

/jl
--
ASCII ribbon campaign ( ) Powered by Lemote Fuloong
against HTML e-mail X Loongson MIPS and OpenBSD
and proprietary / \ http://www.mutt.org
attachments / \ Code Blue or Go Home!
Encrypted email preferred PGP Key 2048R/DA65BC04
Stephan Beal
2014-09-02 08:16:32 UTC
Permalink
Post by John Long
specific shells like bash or any other proprietary (yes, I said it!) gnu
LOL!
Post by John Long
8) Source control is not a hobby for normal healthy people.
Hey! ;)
Post by John Long
You guys scored a huge win by creating fossil and basing it on sqlite.
That was 100% DRH, and i believe Andreas K. was there at the start - the
rest of us came along sometime between then and recently.
Post by John Long
It's
ingenious, it's simple, it's trustworthy, it's complete, and most of all
it's nothing we _don't_ want in a source control system!
Which is why we all joined :).

Thank you!
Thanks for your praise :).
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
Gour
2014-09-02 08:44:05 UTC
Permalink
On Tue, 2 Sep 2014 08:08:41 +0000
Post by John Long
8) Source control is not a hobby for normal healthy people.
It's not something to become an expert in for chest-banging purposes.
It's a critical tool that's supposed to stay the hell out of the way
and let you write and keep track of code.
9) Source control system is not only for keeping the code - here it's
used for very general writings (even non-computer-related). (too)
specific = selfish, universal = broad-minded.


10) Considering 9) (above) it's a proof that those serving God, serve
other people as well.


Sincerely,
Gour
--
As fire is covered by smoke, as a mirror is covered by dust,
or as the embryo is covered by the womb, the living entity is
similarly covered by different degrees of this lust.
Scott Robison
2014-09-02 14:07:48 UTC
Permalink
Post by Gour
9) Source control system is not only for keeping the code - here it's
used for very general writings (even non-computer-related). (too)
specific = selfish, universal = broad-minded.
Interesting you should write this. One of my newest uses for fossil is the
one case in which I'm using it distributed (even though all by myself): My
blog (such as it is). It is not a unique idea at all, but I finally tired
of heavy weight blog platforms and decided I wanted to just keep track of
things in text files. I've started using the pelican static site generator
to keep all my site's source files (restructured text files in a content
tree & config files & etc) as well as the generated files (public tree). I
only maintain the site on my home computer (including generating the public
stuff), but then I commit & sync it to the remote server and update the
live site, making the generated file tree available (and giving me a "live"
backup of all the files).
--
Scott Robison
Gour
2014-09-03 07:12:02 UTC
Permalink
On Tue, 2 Sep 2014 08:07:48 -0600
Post by Scott Robison
Interesting you should write this. One of my newest uses for fossil
is the one case in which I'm using it distributed (even though all by
myself): My blog (such as it is). It is not a unique idea at all, but
I finally tired of heavy weight blog platforms and decided I wanted
to just keep track of things in text files. I've started using the
pelican static site generator to keep all my site's source files
(restructured text files in a content tree & config files & etc) as
well as the generated files (public tree).
Same here - I'm moving to Hugo static site generator written in Go, but
I keep under FOssil stuff like my study notes as well, iow. everything
which has to be safe. :-)


Sincerely,
Gour
--
He is a perfect yogī who, by comparison to his own self,
sees the true equality of all beings, in both their
happiness and their distress, O Arjuna!
Michai Ramakers
2014-09-03 07:32:52 UTC
Permalink
Hello,
Post by Gour
On Tue, 2 Sep 2014 08:07:48 -0600
Post by Scott Robison
Interesting you should write this. One of my newest uses for fossil
is the one case in which I'm using it distributed (even though all by
myself): My blog (such as it is).
...
Same here - I'm moving to Hugo static site generator written in Go, but
I keep under FOssil stuff like my study notes as well, iow. everything
which has to be safe. :-)
fwiw, an alternative use for Fossil here is my diary (agenda? is that
a word?); report-SQL to filter/colour entries according to ETA.

Michai
Miles Fidelman
2014-09-06 15:52:26 UTC
Permalink
Post by Gour
9) Source control system is not only for keeping the code - here it's
used for very general writings (even non-computer-related). (too)
specific = selfish, universal = broad-minded.
Interesting you should write this. One of my newest uses for fossil is
the one case in which I'm using it distributed (even though all by
myself): My blog (such as it is). It is not a unique idea at all, but
I finally tired of heavy weight blog platforms and decided I wanted to
just keep track of things in text files. I've started using the
pelican static site generator to keep all my site's source files
(restructured text files in a content tree & config files & etc) as
well as the generated files (public tree). I only maintain the site on
my home computer (including generating the public stuff), but then I
commit & sync it to the remote server and update the live site, making
the generated file tree available (and giving me a "live" backup of
all the files).
A little late getting back to this, but...

Scott, can you say a little more about your tool chain, end to end from
editing to online? (I've been thinking about doing something similar,
but for maintaining some project documentation and works in progress).

Miles Fidelman
Richard Hipp
2014-09-06 16:03:44 UTC
Permalink
One of my newest uses for fossil is ... My blog
Scott, can you say a little more about your tool chain,...
I'll let Scott answer the specific question. But I just want to remind
Miles of the "Embedded Documentation" feature of Fossil (
http://www.fossil-scm.org/fossil/doc/tip/www/embeddeddoc.wiki) and that the
main Fossil website including all of the on-line documentation is really
just a running instance of Fossil serving the Fossil self-hosting
repository. (All of the main website, except the Download page, that is.)

Because the Fossil website is served from a live Fossil repository, any of
the dozens of Fossil contributors can make a change to documentation by
editing some files, then doing a "fossil commit" of the changes. The
changes get pushed automatically to the main Fossil repo at
http://www.fossil-scm.org/ and are immediately visible to readers. And
within minutes a cron-job will further push those changes out the the
(geographically distributed) backup Fossil websites at
http://www2.fossil-scm.org/ and http://www3.fossil-scm.org/ where the
changes are automatically displayed there as well.

So if you are a committer, updating the Fossil documentation is as simple
as editing the document files on your local machine and then typing "fossil
commit". No logging into servers or taking other actions are necessary to
publish - publication is automatic.
--
D. Richard Hipp
***@sqlite.org
Miles Fidelman
2014-09-06 16:47:03 UTC
Permalink
On Sat, Sep 6, 2014 at 11:52 AM, Miles Fidelman
One of my newest uses for fossil is ... My blog
Scott, can you say a little more about your tool chain,...
I'll let Scott answer the specific question. But I just want to
remind Miles of the "Embedded Documentation" feature of Fossil
(http://www.fossil-scm.org/fossil/doc/tip/www/embeddeddoc.wiki) and
that the main Fossil website including all of the on-line
documentation is really just a running instance of Fossil serving the
Fossil self-hosting repository. (All of the main website, except the
Download page, that is.)So if you are a committer, updating the Fossil
documentation is as simple as editing the document files on your local
machine and then typing "fossil commit". No logging into servers or
taking other actions are necessary to publish - publication is automatic.
--
Thanks Richard. I do really like that feature - though if the
branching/merging features worked for documentation that would be even
better, and even better if the embedded wiki supported branching/merging
within the embedded wiki! That would be a thing of beauty.
Unfortunately, I'm not enough of a code monkey to do that myself.

Meanwhile, for the documentation I'm working on, I'm looking more at
stuff that comes out in DocBook format, with indexing and such. So I'm
thinking more of treating individual pages as something to be edited and
version controlled, with auto assembly into the current live version -
hence my interest in Scott's tool chain.

Thanks,

Miles
Ron W
2014-09-06 19:44:34 UTC
Permalink
Post by Miles Fidelman
Thanks Richard. I do really like that feature - though if the
branching/merging features worked for documentation that would be even
better, and even better if the embedded wiki supported branching/merging
within the embedded wiki! That would be a thing of beauty. Unfortunately,
I'm not enough of a code monkey to do that myself.
The "Embedded Documents" are handled like other project files in your
workspace. You edit them and commit them via the CLI, and can do branching
and merging the same as any other files in your work space. They do get
some special treatment in Fossil's web interface. See
http://fossil-scm.org/index.html/doc/tip/www/embeddeddoc.wiki

This is different from "Fossil Wiki" pages.
Miles Fidelman
2014-09-07 20:30:05 UTC
Permalink
On Sat, Sep 6, 2014 at 12:47 PM, Miles Fidelman
Thanks Richard. I do really like that feature - though if the
branching/merging features worked for documentation that would be
even better, and even better if the embedded wiki supported
branching/merging within the embedded wiki! That would be a thing
of beauty. Unfortunately, I'm not enough of a code monkey to do
that myself.
The "Embedded Documents" are handled like other project files in your
workspace. You edit them and commit them via the CLI, and can do
branching and merging the same as any other files in your work space.
They do get some special treatment in Fossil's web interface. See
http://fossil-scm.org/index.html/doc/tip/www/embeddeddoc.wiki
This is different from "Fossil Wiki" pages.
Yes, but what I'd really like is an embedded wiki, that operates like a
wiki - but with distributed editing, merges, updates, etc. Seems like
the fossil embedded wiki provides half of that, embedded documentation
the other. Sigh...

Oh well. Or am I missing something about the embedded documentation?

Cheers,

Miles
--
In theory, there is no difference between theory and practice.
In practice, there is. .... Yogi Berra
Ron W
2014-09-07 21:03:03 UTC
Permalink
Post by Miles Fidelman
Yes, but what I'd really like is an embedded wiki, that operates like a
wiki - but with distributed editing, merges, updates, etc. Seems like the
fossil embedded wiki provides half of that, embedded documentation the
other. Sigh...
Oh well. Or am I missing something about the embedded documentation?
Right now, Fossil will provide "regular" wiki pages or "embedded" wiki
pages.

"Regular" wiki pages can be edited via the web interface. But, currently,
conflicting edits result in forks, which are viewable in the time line,
just no easy way to do proper branching and merging.

"Embedded" wiki pages act like "regular" wiki pages (since they are wiki
pages). Also, they can be easily branched and merged. But editing them is
done the same way you edit your project source files, externally to Fossil,
then the changes committed. Fossil does provide a way to preview these
pages before committing them.

People have expressed interest in adding support for branching and merging
"regular" wiki pages. The hold up is the how.
Miles Fidelman
2014-09-07 21:49:50 UTC
Permalink
On Sun, Sep 7, 2014 at 4:30 PM, Miles Fidelman
Yes, but what I'd really like is an embedded wiki, that operates
like a wiki - but with distributed editing, merges, updates, etc.
Seems like the fossil embedded wiki provides half of that,
embedded documentation the other. Sigh...
Oh well. Or am I missing something about the embedded documentation?
Right now, Fossil will provide "regular" wiki pages or "embedded" wiki
pages.
"Regular" wiki pages can be edited via the web interface. But,
currently, conflicting edits result in forks, which are viewable in
the time line, just no easy way to do proper branching and merging.
"Embedded" wiki pages act like "regular" wiki pages (since they are
wiki pages). Also, they can be easily branched and merged. But editing
them is done the same way you edit your project source files,
externally to Fossil, then the changes committed. Fossil does provide
a way to preview these pages before committing them.
People have expressed interest in adding support for branching and
merging "regular" wiki pages. The hold up is the how.
Is anybody actually thinking about that, or working on it?

Miles Fidelman
--
In theory, there is no difference between theory and practice.
In practice, there is. .... Yogi Berra
Alaric Snell-Pym
2014-09-08 12:59:56 UTC
Permalink
Post by Richard Hipp
I'll let Scott answer the specific question. But I just want to remind
Miles of the "Embedded Documentation" feature of Fossil
(http://www.fossil-scm.org/fossil/doc/tip/www/embeddeddoc.wiki) and that
the main Fossil website including all of the on-line documentation is
really just a running instance of Fossil serving the Fossil self-hosting
repository. (All of the main website, except the Download page, that is.)
I have a related kind of setup at https://www.kitten-technologies.co.uk/

The top-level site is static HTML (but generated via scripts from master
content in a fossil repo). However, there are a number of fossil repos
for different projects, which are invoked via fossil-as-cgi.

https://www.kitten-technologies.co.uk/project/kitten-technologies is the
project that generates the master site, which includes the cgi thingies!
Eg, "/project/kitten-technologies" itself comes from
https://www.kitten-technologies.co.uk/project/kitten-technologies/artifact/cdeb6789a25bed09afebb6c4f8577a5fc00ad349

And I've hacked the CSS around like crazy to all refer to the central
stuff. Per-repo stuff like the skin gets rolled out from the central
configuration by a script.

It... works! And keeps down the maintenance cost for me of a heap of
fossil repos for my crazy open-source outpourings.

ABS
--
Alaric Snell-Pym
http://www.snell-pym.org.uk/alaric/
Scott Robison
2014-09-06 18:18:07 UTC
Permalink
Post by Miles Fidelman
... One of my newest uses for fossil is the one case in which I'm using
it distributed (even though all by myself): My blog (such as it is). It is
not a unique idea at all, but I finally tired of heavy weight blog
platforms and decided I wanted to just keep track of things in text files.
I've started using the pelican static site generator to keep all my site's
source files (restructured text files in a content tree & config files &
etc) as well as the generated files (public tree). I only maintain the site
on my home computer (including generating the public stuff), but then I
commit & sync it to the remote server and update the live site, making the
generated file tree available (and giving me a "live" backup of all the
files).
A little late getting back to this, but...
Scott, can you say a little more about your tool chain, end to end from
editing to online? (I've been thinking about doing something similar, but
for maintaining some project documentation and works in progress).
It's been a while since I started, so there may be some amounts of "hand
waving" over things I don't remember exactly, but:

1. I installed the pelican static site generator along with its
prerequisites: http://blog.getpelican.com/

2. I had been using a private Wordpress installation, so I exported that
database somehow and used some utility to convert it to restructured text
files.

3. I use Notepad++ on Windows 7 for my text editing, and a command prompt
for organizing files (moving or deleting some older blog posts that I
didn't want in the exported blog for whatever reason).

4. I wrote a few simple scripts in PHP to provide a very basic chisel-esque
management portal on my website; they are used to create repos, change
passwords, and provide a front end to all my repositories. Using that I
created my initial empty repo.

5. I cloned/opened the repo to my Windows 7 machine and added all the
source and generated public files.

6. I synced that back to my server and opened the repo in the web
directory, organized so that my public directory is the only one published.

7. Because I was porting an older blog to this new system, I had a fair
number of media files (mostly images, some audio, a sprinkling of others)
that were not in the Wordpress database, only in the old file system. So as
I cleaned up each of the approximately 100 posts on my Win7 box, I would
move media from the old location of my blog on the server to the open
workspace, add them and commit them server side.

8. Update on Win7 to get the changes.

9. Regenerate the blog, addremove, commit, sync to the server.

10. Update the server side and check out my work, making any needed tweaks
to ensure everything was presented essentially as I wanted it.

11. Repeat steps 7 through 10 for each of the posts.

It's not a very impressive or sophisticated workflow IMO, but it did what I
wanted quite nicely. While the blog itself is also not terribly impressive
and not frequently updated (though I keep meaning to change that!) you can
see it's current state at www.casaderobison.com. I've cleaned up all the
posts and made sure all the media is available. I still have work to do
cleaning up the category/tag system and tweaking the theme to my liking.

I hope that answers the questions and wasn't *too* tedious. :)
--
Scott Robison
Miles Fidelman
2014-09-07 20:25:58 UTC
Permalink
Thanks! Not tedious at all - details are appreciated!
On Sat, Sep 6, 2014 at 9:52 AM, Miles Fidelman
... One of my newest uses for fossil is the one case in which
I'm using it distributed (even though all by myself): My blog
(such as it is). It is not a unique idea at all, but I finally
tired of heavy weight blog platforms and decided I wanted to
just keep track of things in text files. I've started using
the pelican static site generator to keep all my site's source
files (restructured text files in a content tree & config
files & etc) as well as the generated files (public tree). I
only maintain the site on my home computer (including
generating the public stuff), but then I commit & sync it to
the remote server and update the live site, making the
generated file tree available (and giving me a "live" backup
of all the files).
A little late getting back to this, but...
Scott, can you say a little more about your tool chain, end to end
from editing to online? (I've been thinking about doing something
similar, but for maintaining some project documentation and works
in progress).
It's been a while since I started, so there may be some amounts of
1. I installed the pelican static site generator along with its
prerequisites: http://blog.getpelican.com/
2. I had been using a private Wordpress installation, so I exported
that database somehow and used some utility to convert it to
restructured text files.
3. I use Notepad++ on Windows 7 for my text editing, and a command
prompt for organizing files (moving or deleting some older blog posts
that I didn't want in the exported blog for whatever reason).
4. I wrote a few simple scripts in PHP to provide a very basic
chisel-esque management portal on my website; they are used to create
repos, change passwords, and provide a front end to all my
repositories. Using that I created my initial empty repo.
5. I cloned/opened the repo to my Windows 7 machine and added all the
source and generated public files.
6. I synced that back to my server and opened the repo in the web
directory, organized so that my public directory is the only one published.
7. Because I was porting an older blog to this new system, I had a
fair number of media files (mostly images, some audio, a sprinkling of
others) that were not in the Wordpress database, only in the old file
system. So as I cleaned up each of the approximately 100 posts on my
Win7 box, I would move media from the old location of my blog on the
server to the open workspace, add them and commit them server side.
8. Update on Win7 to get the changes.
9. Regenerate the blog, addremove, commit, sync to the server.
10. Update the server side and check out my work, making any needed
tweaks to ensure everything was presented essentially as I wanted it.
11. Repeat steps 7 through 10 for each of the posts.
It's not a very impressive or sophisticated workflow IMO, but it did
what I wanted quite nicely. While the blog itself is also not terribly
impressive and not frequently updated (though I keep meaning to change
that!) you can see it's current state at www.casaderobison.com
<http://www.casaderobison.com>. I've cleaned up all the posts and made
sure all the media is available. I still have work to do cleaning up
the category/tag system and tweaking the theme to my liking.
I hope that answers the questions and wasn't *too* tedious. :)
--
Scott Robison
--
In theory, there is no difference between theory and practice.
In practice, there is. .... Yogi Berra
Dömötör Gulyás
2014-09-02 15:00:55 UTC
Permalink
Post by John Long
7) A source control system should be sensible from the point of view of the
person using it to manage source code. It should not be Linux-centric. It
should not require you to understand its internals to use it effectively
and it must not require you to understand the internals to avoid pitfalls
and gotchas. Tricks = bad, least surprise = good.
This is the main issue I have: git does not follow the principle of
least surprise. I'm sure it *can* do everything, if you know all of
the switches and gotchas. But you don't, even if you think you do.
Apparently many advanced git users have their subset well figured out,
and those never can understand outsiders complaining about how
difficult git can be.
Warren Young
2014-09-02 18:08:22 UTC
Permalink
Post by Dömötör Gulyás
This is the main issue I have: git does not follow the principle of
least surprise. I'm sure it *can* do everything, if you know all of
the switches and gotchas. But you don't, even if you think you do.
Apparently many advanced git users have their subset well figured out,
and those never can understand outsiders complaining about how
difficult git can be.
Git solves the Linux kernel problem, and it solves it well. The thing
is, Linus Torvalds is unique. No one else on the planet has a problem
that big and complex to solve. Why is everyone trying to use a tool
designed to serve his requirements?

For almost everyone else on the planet with a source control problem,
using Git is like using a computer-controlled quilting and embroidering
machine to sew lost buttons back onto a shirt.
Joerg Sonnenberger
2014-09-02 18:38:46 UTC
Permalink
Post by Warren Young
Post by Dömötör Gulyás
This is the main issue I have: git does not follow the principle of
least surprise. I'm sure it *can* do everything, if you know all of
the switches and gotchas. But you don't, even if you think you do.
Apparently many advanced git users have their subset well figured out,
and those never can understand outsiders complaining about how
difficult git can be.
Git solves the Linux kernel problem, and it solves it well. The
thing is, Linus Torvalds is unique. No one else on the planet has a
problem that big and complex to solve. Why is everyone trying to
use a tool designed to serve his requirements?
While I agree on the uniqueness of Torvalds, I don't agree with the
rest. The Linux kernel is *not* that big when compared with many other
projects. There are quite a few other projects that are comparable in
size and history. There is something else which is often ignored,
especially by git advocats. The development model pushed by git is the
development model used by Torvalds and that's pretty unique as well. In
a way, it inverts the normal process used by most projects. A somewhat
sarcastic view would call it organised distrust. Most FOSS projects and
many commercial projects have a moderately flat hierachy. Members have
work areas they commit code in, releases and contributions outside that
area is handled either by senior members or other special subteams.
There rarely is a single point of failure / approval... The Linux model
effectively inverts that model.

Joerg
Warren Young
2014-09-02 20:31:37 UTC
Permalink
Post by Joerg Sonnenberger
Post by Warren Young
Post by Dömötör Gulyás
This is the main issue I have: git does not follow the principle of
least surprise.
Linus Torvalds is unique. No one else on the planet has a
problem that big and complex to solve.
While I agree on the uniqueness of Torvalds, I don't agree with the
rest. The Linux kernel is *not* that big when compared with many other
projects.
Lines of code is not the important measure here.

The uncommon thing about the Linux kernel development effort is that it
is highly distributed, with many merge layers and multiple independent
but communicating major repositories. An outsider wanting to get a
change into the kernel doesn't just email a patch(1) file to
***@linux.com, he has to work it up through these layers. Some
changes sit for months or years in one of the alternate kernel repos
before it makes its way into Linus's git repo.

Such a messy process requires a tool set that can wrangle that mess into
some semblance of coherency.

If you don't have that kind of mess, you don't need those tools.

Other large projects either...

1. ...live largely or entirely within the scope of a single organization
so presumably the check-in hierarchy is either flat or nearly so. (e.g.
the Windows OS)

2. ...have a simple dividing line between those "inside" the project and
those "outside" it. You either have a commit bit or you do not. Those
without must submit patches. (e.g. FreeBSD)

In both cases, commits end up in The Repository, singular, in short order.

Such projects that use a DVCS are likely using it as "master plus
remotes with minor temporary differences" rather than the federation
model of Linux kernel development.
Gour
2014-09-03 09:05:27 UTC
Permalink
On Mon, 1 Sep 2014 17:29:41 +0200
Post by Stephan Beal
It occurred to me today that in nearly 31 years of using a computer i
have, in total, lost more data to git (while following the
instructions!!!) than any other single piece of software. Also
concluded is that git is the only SCM out there which makes SCM
difficult for the simple stuff. Even RCS is simpler to use. Sure CVS
has limits, but respect those limits and it works just fine. Never
lost a line of code in CVS.
Heh, that's called irony - today I noticed the following article:

"Git: The Safety Net for Your Project"

and added comment recommending to use Fossil.
(http://alistapart.com/article/git-the-safety-net-for-your-projects)


Sincerely,
Gour
--
As the ignorant perform their duties with attachment to results,
the learned may similarly act, but without attachment, for the
sake of leading people on the right path.
Nico Williams
2014-09-05 20:03:45 UTC
Permalink
Post by Stephan Beal
Okay, more git bashing...
Seems like a lot of the complaints are the sorts of complaints you
would get about -say- laptops:

- it's easy to forget you left something on your laptop two flights
ago, when you had no connectivity, and forgot to copy it to ...

Maybe mobile devices should behave like Fossil repos, though that
would require automatic conflict resolution to be workable.

I've long thought that we need a decent filesystem with disconnected
operation functionality. And with ZFS and such we know that a Merkle
hash tree version-control-like approach works well for filesystems.

But on the other hand, I like to be in control of what code goes up
and what doesn't. To me the "designed to forget" comments seem like a
stretch.

Nico
--
Stephan Beal
2014-09-06 08:22:13 UTC
Permalink
Post by Nico Williams
To me the "designed to forget" comments seem like a
stretch.
git branch -D name
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
Nico Williams
2014-09-06 23:24:25 UTC
Permalink
Post by Stephan Beal
Post by Nico Williams
To me the "designed to forget" comments seem like a
stretch.
git branch -D name
Eh, filesystems let you delete files. Unlike most filesystems, git lets
you restore your deleted branches (yes, provided you don't gc the repo).
Scott Robison
2014-09-07 00:05:33 UTC
Permalink
Post by Nico Williams
Post by Stephan Beal
git branch -D name
Eh, filesystems let you delete files. Unlike most filesystems, git lets
you restore your deleted branches (yes, provided you don't gc the repo).
Then just use a file system and various command line tools! But seriously,
it's a philosophical difference between those who want to be able to
rewrite their history to what they should have done and those who want to
keep the history around to see what they did. It's not perfect, and
certainly there are arguments for both approaches, but git seems fragile to
some people while fossil seems inflexible to others.
--
Scott Robison
John Long
2014-09-11 15:18:49 UTC
Permalink
Post by Scott Robison
Post by Nico Williams
Post by Stephan Beal
git branch -D name
Eh, filesystems let you delete files. Unlike most filesystems, git lets
you restore your deleted branches (yes, provided you don't gc the repo).
Then just use a file system and various command line tools! But seriously,
it's a philosophical difference between those who want to be able to
rewrite their history to what they should have done and those who want to
keep the history around to see what they did.
I can understand for personal projects people might want stuff to disappear.
And I can understand vandals want to make stuff disappear when they attack a
system. But in a serious, large project there are often requirements for an
audit trail. There is no downside to this given the cheapness of backing
storage. And it can prevent all sorts of problems.

In the places I work the problem tracking system entries are immutable. And
while a lot of those places don't use source control for various reasons
when they do use them the source changes and history of who did what and
when he did it are also immutable.

/jl
--
ASCII ribbon campaign ( ) Powered by Lemote Fuloong
against HTML e-mail X Loongson MIPS and OpenBSD
and proprietary / \ http://www.mutt.org
attachments / \ Code Blue or Go Home!
Encrypted email preferred PGP Key 2048R/DA65BC04
Nico Williams
2014-09-11 16:07:39 UTC
Permalink
Post by John Long
Post by Scott Robison
Post by Nico Williams
Post by Stephan Beal
git branch -D name
Eh, filesystems let you delete files. Unlike most filesystems, git lets
you restore your deleted branches (yes, provided you don't gc the repo).
Then just use a file system and various command line tools! But seriously,
it's a philosophical difference between those who want to be able to
rewrite their history to what they should have done and those who want to
keep the history around to see what they did.
I can understand for personal projects people might want stuff to disappear.
And I can understand vandals want to make stuff disappear when they attack a
system. But in a serious, large project there are often requirements for an
audit trail. There is no downside to this given the cheapness of backing
storage. And it can prevent all sorts of problems.
If you want a secure audit trail then you have to send the audit
records elsewhere (preferably replicate the trails too). And you have
to commit the audit records to their destinations before (or otherwise
atomically as) commits are incorporated into a repo.

Refusing to modify history doesn't make history unmodifiable -- Merkle
hash chains alone only provide integrity protection for the past, but
not if the attacker controls it and the present. An attacker can
always edit the history directly via SQLite3 access to the repo. Of
course, anyone who has fetched the pre-edit history will notice the
re-write, which is why pushing and pulling often is helpful.

Alternatively you can sign commits. Then editing history breaks the
signatures, thus requiring re-signing of edited commits or a signature
by some authority describing what history editing took place and why.
Post by John Long
In the places I work the problem tracking system entries are immutable. And
[...]
Nothing can really be made immutable, but you can detect mutation.
That has nothing to do with whether a VCS permits mutation, and
everything to do with using Merkle hash chaining and replicating
(auditing, if you wish) the chains to secure storage as soon as
possible so as to make mutation detectable. Whether to permit
mutation for all, some, or no branches, is a policy decision to be
made. I don't think one size fits all.

Nico
--
Richard Hipp
2014-09-11 16:18:13 UTC
Permalink
Post by Nico Williams
Nothing can really be made immutable, but you can detect mutation.
No. Version 9491ba7d738528f168657adb43a198238abde19e (the SQLite 3.8.6
release) cannot be modified in any way without changing its hash value,
thus making it something different. (Unless you can mount a pre-image
attack against SHA1 - let's assume that is impossible.)

Object 9491ba7d738528f168657adb43a198238abde19e is immutable because
changing it also changes its name, creating a new object.
--
D. Richard Hipp
***@sqlite.org
Nico Williams
2014-09-11 16:32:28 UTC
Permalink
Post by Richard Hipp
Post by Nico Williams
Nothing can really be made immutable, but you can detect mutation.
No. Version 9491ba7d738528f168657adb43a198238abde19e (the SQLite 3.8.6
release) cannot be modified in any way without changing its hash value, thus
making it something different. (Unless you can mount a pre-image attack
against SHA1 - let's assume that is impossible.)
But the repo containing it can be mutated to, for example, no longer
have that commit.

This can only be detected. It can't be prevented. Merkle hash chains
are only one part of the detection story. Digital signatures and/or
replication are another.
Ron W
2014-09-11 18:12:49 UTC
Permalink
(Unless you can mount a pre-image attack against SHA1 - let's assume that
is impossible.)
FYI, FWIW, SHA1 seems to be deprecated. There was a summary on slashdot.org
about Google planning to change Chrome's default setting for SHA1 warnings
to on in an attempt to encourage websites to switch to a more secure hash
algorithm. (The summary also mentioned the risk to Google in doing this,
claiming that users might be inclined to switch web browsers when their
current browser starts complaining about their favorite web sites.)
Stephan Beal
2014-09-11 16:19:48 UTC
Permalink
Post by Nico Williams
not if the attacker controls it and the present. An attacker can
always edit the history directly via SQLite3 access to the repo.
No, he can't. Well, he can, but he will break the hashes of other records,
so any tamping will be noticed. Specifically, the Z- and R-cards detect any
sort of tampering.

A Z-card is a hash of the contents of a manifest (formal change record). An
R-card is a "mega-hash" of everything referenced by that record. Any change
in any content is going to change both of those, invalidating them.
Post by Nico Williams
Nothing can really be made immutable, but you can detect mutation.
That's splitting hairs, though, as any mutation breaks the data chain.
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
Nico Williams
2014-09-11 16:33:38 UTC
Permalink
Post by Stephan Beal
No, he can't. Well, he can, but he will break the hashes of other records,
so any tamping will be noticed. Specifically, the Z- and R-cards detect any
sort of tampering.
Right. He can. If you've not pushed the commits anywhere else before
the attack, and if you've not signed them, and if no one has pulled
the commits elsewhere before the attack, then you can't detect it.
Stephan Beal
2014-09-11 16:42:43 UTC
Permalink
Post by Stephan Beal
Post by Stephan Beal
No, he can't. Well, he can, but he will break the hashes of other
records,
Post by Stephan Beal
so any tamping will be noticed. Specifically, the Z- and R-cards detect
any
Post by Stephan Beal
sort of tampering.
Right. He can. If you've not pushed the commits anywhere else before
the attack, and if you've not signed them, and if no one has pulled
the commits elsewhere before the attack, then you can't detect it.
i'd have to see it to believe it. Hashing is not the only hurdle there.
He'd have to get past Fossil's internal deltification, as well, which
allows any given artifact to be used as the basis for a delta for any other
(semantically unrelated) artifact (and undoing that "rat's tail" is largely
what makes popping the top-most commit so very difficult).
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
Loading...