Discussion:
[fossil-users] Random thoughts on Fossil v2
Stephan Beal
2013-07-21 10:54:05 UTC
Permalink
Hi, all,

This topic has been tossed around before, but the amount of effort involved
in its undertaking has always kept us from actually doing it...

To help bootstrap the process of figuring out what Fossil v2 might look
like i have started writing down ideas in a public Google Doc:

https://docs.google.com/document/d/12g0s5A2TPX7-y47Nsw235rvsjcuh49TnHfMDB4ASvlo/view

Any of you who have write access to the JSON API docs also have write
access to that one, so feel free to expand/comment/etc. It's just a big
scratchpad, not a formal doc, nor does it provide any indication of what
Fossil's future has in store - it's just ideas regarding what v2 "might"
look like if i were to start working on it today (which, in a way, i am ;).

If you don't have access to that doc, either send me your gmail address and
i'll gladly add you, or post your ideas here and i'll integrate them into
the doc.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Eduardo Morras
2013-07-21 17:12:06 UTC
Permalink
On Sun, 21 Jul 2013 12:54:05 +0200
Stephan Beal <***@googlemail.com> wrote:

> Hi, all,
>
> This topic has been tossed around before, but the amount of effort involved
> in its undertaking has always kept us from actually doing it...
>
> To help bootstrap the process of figuring out what Fossil v2 might look
> like i have started writing down ideas in a public Google Doc:
>
> https://docs.google.com/document/d/12g0s5A2TPX7-y47Nsw235rvsjcuh49TnHfMDB4ASvlo/view
>
> Any of you who have write access to the JSON API docs also have write
> access to that one, so feel free to expand/comment/etc. It's just a big
> scratchpad, not a formal doc, nor does it provide any indication of what
> Fossil's future has in store - it's just ideas regarding what v2 "might"
> look like if i were to start working on it today (which, in a way, i am ;).
>
> If you don't have access to that doc, either send me your gmail address and
> i'll gladly add you, or post your ideas here and i'll integrate them into
> the doc.

I post them here then:

a) Creation of graphs to show statistics.

I'm on it now, writing a minimal png with 32bit ARGB color and a minimal graph lib.

b) Plugins and hooks c modules

I use fossil with a central fossil repository from/to where individuals sync. Some module examples:

a) when a user sync/merge to trunk in central server, it compile/test the code, after receive but before merge, and if compile/test fails reject sync/merge.
b) project management features, global gant graphs, percentage of work done,
c) source code parser, to find where a function, macro, etc... is declared
d) fts of documentation

c) Repository of javascript, th1 code, css

When starting a new repository I import html, js, css, th1 addons and features. If a central repository of those addons exist we, the fossil community, could share those addons, f.ex. code to integrate fossil in drupal/joomla,

d) extend user capabilities

Some superuser can own/lock some files in central repository or at least, in trunk. For example makefiles, binary files, project configuration, etc...

--- ---
Eduardo Morras <***@yahoo.es>
Stephan Beal
2013-07-21 19:15:56 UTC
Permalink
On Sun, Jul 21, 2013 at 7:12 PM, Eduardo Morras <***@yahoo.es> wrote:

> a) Creation of graphs to show statistics.
>
> I'm on it now, writing a minimal png with 32bit ARGB color and a minimal
> graph lib.
>

Added to the list.


> b) Plugins and hooks c modules
>
> I use fossil with a central fossil repository from/to where individuals
> sync. Some module examples:
>

Added to the list.


> a) when a user sync/merge to trunk in central server, it compile/test
> the code, after receive but before merge, and if compile/test fails reject
> sync/merge.
> b) project management features, global gant graphs, percentage of work
> done,
> c) source code parser, to find where a function, macro, etc... is
> declared
> d) fts of documentation
>

If we don't want to write such hooks in C (which might not be practical)
then a a scripting engine is implied. IMO the right choice, given fossil's
history, would be TCL, but any option we choose brings with it long-term
dependencies or maintenance (if we embed a small interpreter like jimtcl or
lua). To me jimtcl would be the obvious candidate, but i was told at some
point that it has too many incompatibilities with standard TCL to be of
interest to the harder-code TCLers.



> d) extend user capabilities
>
> Some superuser can own/lock some files in central repository or at
> least, in trunk. For example makefiles, binary files, project
> configuration, etc...
>

i was thinking about that as well, but IMO the current system works
remarkably well. It effectively offers up to 26 (or 52) roles, which is far
more than projects of fossil's scale need. We could perhaps add a mechanism
to allow clients to add letters to the permissions alphabet. Or toss out
the mechanism and do something new. i think a full-fledged role management
system would be way overkill, but if we're starting from scratch anyway...

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Baruch Burstein
2013-07-21 20:29:19 UTC
Permalink
My wishlist:

1. Full-text search through the file contents and history. Search this list
for "howto `grep' through old revisions" to see some ideas (mine is at the
end of that thread), but some other ideas could work great, too.
2. Partial pulls - I mean that if pulling from a large repository, and the
pull is aborted partway, whatever artifacts (or deltas) were already pulled
and stored in the repo should remain, instead of being rolled back. I think
this wouldn't even be too hard in the current fossil, just a matter of
moving a couple of commit statements around.

On Sun, Jul 21, 2013 at 10:15 PM, Stephan Beal <***@googlemail.com>wrote:

> On Sun, Jul 21, 2013 at 7:12 PM, Eduardo Morras <***@yahoo.es> wrote:
>
>> a) Creation of graphs to show statistics.
>>
>> I'm on it now, writing a minimal png with 32bit ARGB color and a
>> minimal graph lib.
>>
>
> Added to the list.
>
>
>> b) Plugins and hooks c modules
>>
>> I use fossil with a central fossil repository from/to where individuals
>> sync. Some module examples:
>>
>
> Added to the list.
>
>
>> a) when a user sync/merge to trunk in central server, it compile/test
>> the code, after receive but before merge, and if compile/test fails reject
>> sync/merge.
>> b) project management features, global gant graphs, percentage of work
>> done,
>> c) source code parser, to find where a function, macro, etc... is
>> declared
>> d) fts of documentation
>>
>
> If we don't want to write such hooks in C (which might not be practical)
> then a a scripting engine is implied. IMO the right choice, given fossil's
> history, would be TCL, but any option we choose brings with it long-term
> dependencies or maintenance (if we embed a small interpreter like jimtcl or
> lua). To me jimtcl would be the obvious candidate, but i was told at some
> point that it has too many incompatibilities with standard TCL to be of
> interest to the harder-code TCLers.
>
>
>
>> d) extend user capabilities
>>
>> Some superuser can own/lock some files in central repository or at
>> least, in trunk. For example makefiles, binary files, project
>> configuration, etc...
>>
>
> i was thinking about that as well, but IMO the current system works
> remarkably well. It effectively offers up to 26 (or 52) roles, which is far
> more than projects of fossil's scale need. We could perhaps add a mechanism
> to allow clients to add letters to the permissions alphabet. Or toss out
> the mechanism and do something new. i think a full-fledged role management
> system would be way overkill, but if we're starting from scratch anyway...
>
> --
> ----- stephan beal
> http://wanderinghorse.net/home/stephan/
> http://gplus.to/sgbeal
>
> _______________________________________________
> fossil-users mailing list
> fossil-***@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
>


--
˙uʍop-ǝpısdn sı ɹoʇıuoɯ ɹnoʎ 'sıɥʇ pɐǝɹ uɐɔ noʎ ɟı
Eduardo Morras
2013-07-22 10:09:38 UTC
Permalink
On Sun, 21 Jul 2013 21:15:56 +0200
Stephan Beal <***@googlemail.com> wrote:
> > a) when a user sync/merge to trunk in central server, it compile/test
> > the code, after receive but before merge, and if compile/test fails reject
> > sync/merge.
> > b) project management features, global gant graphs, percentage of work
> > done,
> > c) source code parser, to find where a function, macro, etc... is
> > declared
> > d) fts of documentation
> >
>
> If we don't want to write such hooks in C (which might not be practical)
> then a a scripting engine is implied. IMO the right choice, given fossil's
> history, would be TCL, but any option we choose brings with it long-term
> dependencies or maintenance (if we embed a small interpreter like jimtcl or
> lua). To me jimtcl would be the obvious candidate, but i was told at some
> point that it has too many incompatibilities with standard TCL to be of
> interest to the harder-code TCLers.

No, not as hooks, but as plugins. I think that include a scripting engine is great, I'm a lua user, but force to use only one not. Allowing to call external have this advantages:

a) Not tied to one scripting language
b) In smp environments, the script will run on other processor (or even remotely)
c) It can work asyncronous, fossil may not wait the script ends
d) If script fails, fossil will not hang or crash or silently quit
e) Fossil size keeps tiny
f) Scripts can be persistent
g) Update/Bug fix script engine without recompile fossil or wait for a new fossil release

Of course, with security at first.

About hooks, thinking more about it, perhaps something more like a trigger. For example, in our c projects, we have this on the central repositories:

1) a user syncs to central
2) after sync, we must run a script manually that
3) checkout last trunk version
4) compare trunk with trunk
5) if some change is detected, compile with clang static analyzer, it has an option to make an html report
6) add the report directory to reports branch
7) copy the new trunk to old trunk dir and rm everything else
8) close the checkout

It will be great if fossil shoots the script each time trunk is modified,

> > d) extend user capabilities
> >
> > Some superuser can own/lock some files in central repository or at
> > least, in trunk. For example makefiles, binary files, project
> > configuration, etc...
> >
>
> i was thinking about that as well, but IMO the current system works
> remarkably well. It effectively offers up to 26 (or 52) roles, which is far
> more than projects of fossil's scale need. We could perhaps add a mechanism
> to allow clients to add letters to the permissions alphabet. Or toss out
> the mechanism and do something new. i think a full-fledged role management
> system would be way overkill, but if we're starting from scratch anyway...

Perhaps I mixed the concept of user and role. I thought more about the concept of file owner. By default all users owns all files, but some roles, like superuser or project architect, own some files or dirs and only him/her/it can modify them.


--- ---
Eduardo Morras <***@yahoo.es>
Stephan Beal
2013-07-22 10:20:44 UTC
Permalink
On Mon, Jul 22, 2013 at 12:09 PM, Eduardo Morras <***@yahoo.es> wrote:

> ...No, not as hooks, but as plugins. I think that include a scripting
> engine is great, I'm a lua user, but force to use only one not. Allowing to
> call external have this advantages:
>
> a) Not tied to one scripting language
> b) In smp environments, the script will run on other processor (or even
> remotely)
> c) It can work asyncronous, fossil may not wait the script ends
>

For some checks Fossil _must_ wait on the script to end. How can it do a
commit confirmation check if it doesn't wait on the results?



> Of course, with security at first.
>

One of my weak points - i'm very much relying on others to spot any
potential security problems.


> About hooks, thinking more about it, perhaps something more like a
> trigger. For example, in our c projects, we have this on the central
> repositories:
>

To be clear, i often use the words hook/plugin/trigger interchangeably.


> 1) a user syncs to central
> 2) after sync, we must run a script manually that
> 3) checkout last trunk version
> 4) compare trunk with trunk

5) if some change is detected, compile with clang static analyzer, it has
> an option to make an html report
> 6) add the report directory to reports branch
> 7) copy the new trunk to old trunk dir and rm everything else
> 8) close the checkout
>
> It will be great if fossil shoots the script each time trunk is modified,
>

Ideally we'll be able to do that type of thing. We need to define (A) when
various hook points will be triggered, (B) what parameters/data they
receive from fossil, (C) how fossil should deal with various exit/result
codes. e.g. do we just have success/failure, or are there varying degrees
of success/failure? This topic will certainly keep us busy for a while (but
will be very interesting). Once we are far enough along to have a script
binding of some kind we can experiment with this freely before deciding on
concrete semantics.

> i was thinking about that as well, but IMO the current system works
> > remarkably well. It effectively offers up to 26 (or 52) roles, which is
> far
> > more than projects of fossil's scale need. We could perhaps add a
> mechanism
> > to allow clients to add letters to the permissions alphabet. Or toss out
> > the mechanism and do something new. i think a full-fledged role
> management
> > system would be way overkill, but if we're starting from scratch
> anyway...
>
> Perhaps I mixed the concept of user and role. I thought more about the
> concept of file owner. By default all users owns all files, but some roles,
> like superuser or project architect, own some files or dirs and only
> him/her/it can modify them.
>

i think that would be impossible to implement properly in a DVCS (it would
be possible in a centralized SCM). When i clone your repo, i need access to
ALL of the files i cloned. If i cannot check out or modify certain files
just because you "own" them, the repo is useless to me. It means that if a
developer dies (as they occasionally do) then all of his locks are there
forever. This doesn't sound realistic to me. One of the things fossil
promises which other SCMs do not is "forever" - it's data format is
designed to outlive the software, which means that in 200 years someone
could read the fossil data format, import an ancient sqlite3 fossil DB into
his sqlite27 db, and get back to work. Except that now he can't make
changes because the files in the repo belong to another user ;).

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Eduardo Morras
2013-07-22 11:30:17 UTC
Permalink
On Mon, 22 Jul 2013 12:20:44 +0200
Stephan Beal <***@googlemail.com> wrote:

> On Mon, Jul 22, 2013 at 12:09 PM, Eduardo Morras <***@yahoo.es> wrote:
>
> > ...No, not as hooks, but as plugins. I think that include a scripting
> > engine is great, I'm a lua user, but force to use only one not. Allowing to
> > call external have this advantages:
> >
> > a) Not tied to one scripting language
> > b) In smp environments, the script will run on other processor (or even
> > remotely)
> > c) It can work asyncronous, fossil may not wait the script ends
> >
>
> For some checks Fossil _must_ wait on the script to end. How can it do a
> commit confirmation check if it doesn't wait on the results?

Xcb uses an async mechanism, it creates a cookie, sends it and data to the external code, keep working. External code calls xcb, sends the cookie with the answer. Data and cookie can be exchanged through sql table.

> To be clear, i often use the words hook/plugin/trigger interchangeably.

They are different. A hook is capability we add to a function, as a anonymous function pointer. This way we can "hook" the any function to program flow, changing it. A plugin is a module that adds capabilities to program. For example Sqlite3 has plugin mechanism to add external aggreagate calculations. A trigger is code that executes other code when some conditions met.

So the difference is that a hook allows a plugin work when the trigger wants.

> > Perhaps I mixed the concept of user and role. I thought more about the
> > concept of file owner. By default all users owns all files, but some roles,
> > like superuser or project architect, own some files or dirs and only
> > him/her/it can modify them.
> >
>
> i think that would be impossible to implement properly in a DVCS (it would
> be possible in a centralized SCM). When i clone your repo, i need access to
> ALL of the files i cloned. If i cannot check out or modify certain files
> just because you "own" them, the repo is useless to me. It means that if a
> developer dies (as they occasionally do) then all of his locks are there
> forever. This doesn't sound realistic to me. One of the things fossil
> promises which other SCMs do not is "forever" - it's data format is
> designed to outlive the software, which means that in 200 years someone
> could read the fossil data format, import an ancient sqlite3 fossil DB into
> his sqlite27 db, and get back to work. Except that now he can't make
> changes because the files in the repo belong to another user ;).

That's why I said I confused user with role. The role owns the files, not the user. Currently, a user without wiki role capabilities can't modify the wiki files. Similar, a C developer role can modify /src dir but not /documentation or /sql, the DBA role can change /sql but not /src and similar. Role info is already exchanged in pull/push/sync, user info not. So in your scenary, a user can get the role capabilities and access them.

The problem is that a user is admin of his own repository file and can set whatever s/he wants and access everything. A global dvcs role and user data can solve this, not allowing this user make changes in public branches or trunk directories of central repo (or other users repos) if s/he has no privileges to do so.


--- ---
Eduardo Morras <***@yahoo.es>
Stephan Beal
2013-07-22 11:36:37 UTC
Permalink
On Mon, Jul 22, 2013 at 1:30 PM, Eduardo Morras <***@yahoo.es> wrote:

> Xcb uses an async mechanism, it creates a cookie, sends it and data to the
> external code, keep working. External code calls xcb, sends the cookie with
> the answer. Data and cookie can be exchanged through sql table.
>

Then it starts looking like a message queue, and i personally have no
intention of seeing fossil grow into such a creature.


> > To be clear, i often use the words hook/plugin/trigger interchangeably.
>
> They are different. A hook is capability we add to a function, as a
> anonymous function pointer. This way we can "hook" the any function to
> program flow, changing it. A plugin is a module that adds capabilities to
> program. For example Sqlite3 has plugin mechanism to add external
> aggreagate calculations. A trigger is code that executes other code when
> some conditions met.
>

Yes - didn't mean to imply that my (mis)use of the terms was correct ;).



> So the difference is that a hook allows a plugin work when the trigger
> wants.
>

Agreed.


> That's why I said I confused user with role. The role owns the files, not
> the user. Currently, a user without wiki role capabilities can't modify the
> wiki files.
>

Yes he can - he simply needs to clone it. At that point any access rights
fly out the window.



> Similar, a C developer role can modify /src dir but not /documentation or
> /sql, the DBA role can change /sql but not /src and similar. Role info is
> already exchanged in pull/push/sync, user info not. So in your scenary, a
> user can get the role capabilities and access them.
>

So what happens when i (as a wiki editor) make a change to src/foo.c,
commit it to my local copy (because locally i have admin access) and then
try to push? i can't, and my repo then gets (permanently) out of sync. i
don't think this model is possible in a DVCS.


> The problem is that a user is admin of his own repository file and can set
> whatever s/he wants and access everything. A global dvcs role and user data
> can solve this, not allowing this user make changes in public branches or
> trunk directories of central repo (or other users repos) if s/he has no
> privileges to do so.
>

Once i clone it, the central repository has NO say-so in what i can and
cannot check out. It can only control what i check IN, or (more correctly),
what i try to push to the central server. If it will not let me push the
changes i have already committed to my local repo then i have a serious
problem.


--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Gautier DI FOLCO
2013-07-22 11:53:12 UTC
Permalink
2013/7/22 Stephan Beal <***@googlemail.com>

> Then it starts looking like a message queue, and i personally have no
> intention of seeing fossil grow into such a creature.
>

Keep it simple, RPCs can be queued by an external tool.
Eduardo Morras
2013-07-23 11:04:38 UTC
Permalink
On Mon, 22 Jul 2013 13:36:37 +0200
Stephan Beal <***@googlemail.com> wrote:

> On Mon, Jul 22, 2013 at 1:30 PM, Eduardo Morras <***@yahoo.es> wrote:
>
> > Xcb uses an async mechanism, it creates a cookie, sends it and data to the
> > external code, keep working. External code calls xcb, sends the cookie with
> > the answer. Data and cookie can be exchanged through sql table.
> >
>
> Then it starts looking like a message queue, and i personally have no
> intention of seeing fossil grow into such a creature.

And fossil is single thread single process app and can't do any other thing while waiting. I applaud the single thread, I dislike threads too, don't understand why it must be single process.

> > That's why I said I confused user with role. The role owns the files, not
> > the user. Currently, a user without wiki role capabilities can't modify the
> > wiki files.
> >
>
> Yes he can - he simply needs to clone it. At that point any access rights
> fly out the window.
>
> > Similar, a C developer role can modify /src dir but not /documentation or
> > /sql, the DBA role can change /sql but not /src and similar. Role info is
> > already exchanged in pull/push/sync, user info not. So in your scenary, a
> > user can get the role capabilities and access them.
> >
>
> So what happens when i (as a wiki editor) make a change to src/foo.c,
> commit it to my local copy (because locally i have admin access) and then
> try to push? i can't, and my repo then gets (permanently) out of sync. i
> don't think this model is possible in a DVCS.

Yes it is. There's a big difference between Project and Repository. There's the fossil project, only one, but there are lots of repositories and fossil project forks that aren't the main 'Fossil Project'. I don't want to restrict what you do with your repository or your project fork, I want to restrict what you can do to main Project.

> > The problem is that a user is admin of his own repository file and can set
> > whatever s/he wants and access everything. A global dvcs role and user data
> > can solve this, not allowing this user make changes in public branches or
> > trunk directories of central repo (or other users repos) if s/he has no
> > privileges to do so.
> >
>
> Once i clone it, the central repository has NO say-so in what i can and
> cannot check out. It can only control what i check IN, or (more correctly),
> what i try to push to the central server. If it will not let me push the
> changes i have already committed to my local repo then i have a serious
> problem.

No, I sync with fossil main repository each week, but I don't have permission to push my changes. That's because my user (anonymous) don't have the role to do so. If I want to push my changes I must convince someone who has the correct role to do so. I want to restrict that a developer user, with a defined role, can push changes only to certains files/directories of main repository and not others. For example, in Fossil, sqlite3.c, sqlite3.h, copyright file, makefile, etc... should be changed only by user drh, at least in trunk, because he is the project architect. It's true that 'it shouldn't happen if everyone follow the manual and touch the files he owns', but there are sometimes, specially with newbies, that the repository becomes unusable because they touch a file they shouldn't.


--- ---
Eduardo Morras <***@yahoo.es>
Stephan Beal
2013-07-23 11:17:47 UTC
Permalink
On Tue, Jul 23, 2013 at 1:04 PM, Eduardo Morras <***@yahoo.es> wrote:

> Yes it is. There's a big difference between Project and Repository.
> There's the fossil project, only one, but there are lots of repositories
> and fossil project forks that aren't the main 'Fossil Project'. I don't
> want to restrict what you do with your repository or your project fork, I
> want to restrict what you can do to main Project.
>

That's the problem: my clone of your project is a first-class copy of your
project. There is essentially no distinction. If you copy goes up in flames
me can drop in my copy, re-add the users, and we're done. So here's what
happens with user restrictions:

- i clone your repo
- i edit some dir which you have restricted.
- i check it in locally (offline)
- Later i try to push to your repo and it fails. My repo is now in a state
where i can _never_ push my copy to yours ever again because pushing to
yours will fail as long as i have no access (on your side) for that file.


> No, I sync with fossil main repository each week, but I don't have
> permission to push my changes. That's because my user (anonymous) don't
> have the role to do so. If I want to push my changes I must convince
> someone who has the correct role to do so.
>

But for "partial access" ALL pushes from me to you will fail if i have
edited a file to which i cannot push. There is no "partial push" in fossil
- you can push everything or nothing. Fossil, as a piece of software, could
not possibly decide which parts of my commits are "safe" to push, and
Fossil _MUST_ fail if _any_ part of my push/sync fails because it has no
heuristic which can say, "oh, that was just that file under /noaccess/...
which you edited 3 weeks ago and is still not writable for you" (and if it
could decide that, then failing is still the proper response). The only
reasonable action Fossil can take there is to fail. How could it possibly
know which parts are "okay" and which not? How can i possibly ever recover
my repo to a sane state if you refuse to give me write access to changes i
have already made locally (but not cannot commit, leaving my repo in an
unclean/divergent state).


--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Eduardo Morras
2013-07-23 11:54:05 UTC
Permalink
On Tue, 23 Jul 2013 13:17:47 +0200
Stephan Beal <***@googlemail.com> wrote:

> On Tue, Jul 23, 2013 at 1:04 PM, Eduardo Morras <***@yahoo.es> wrote:
>
> > Yes it is. There's a big difference between Project and Repository.
> > There's the fossil project, only one, but there are lots of repositories
> > and fossil project forks that aren't the main 'Fossil Project'. I don't
> > want to restrict what you do with your repository or your project fork, I
> > want to restrict what you can do to main Project.
> >
>
> That's the problem: my clone of your project is a first-class copy of your
> project. There is essentially no distinction. If you copy goes up in flames
> me can drop in my copy, re-add the users, and we're done. So here's what
> happens with user restrictions:
>
> - i clone your repo
> - i edit some dir which you have restricted.
> - i check it in locally (offline)
> - Later i try to push to your repo and it fails. My repo is now in a state
> where i can _never_ push my copy to yours ever again because pushing to
> yours will fail as long as i have no access (on your side) for that file.


I must insist here ;) There's a distinction, your repository is not the official Project repository, mine, the central one, yes. You can fork and create a new project and name it as you wish and change what you want. If you want to merge with the Only and Real True Project, your changes must be approved by the person that has the permission to do that, that's me.


> > No, I sync with fossil main repository each week, but I don't have
> > permission to push my changes. That's because my user (anonymous) don't
> > have the role to do so. If I want to push my changes I must convince
> > someone who has the correct role to do so.
> >
>
> But for "partial access" ALL pushes from me to you will fail if i have
> edited a file to which i cannot push. There is no "partial push" in fossil
> - you can push everything or nothing. Fossil, as a piece of software, could
> not possibly decide which parts of my commits are "safe" to push, and
> Fossil _MUST_ fail if _any_ part of my push/sync fails because it has no
> heuristic which can say, "oh, that was just that file under /noaccess/...
> which you edited 3 weeks ago and is still not writable for you" (and if it
> could decide that, then failing is still the proper response). The only
> reasonable action Fossil can take there is to fail. How could it possibly
> know which parts are "okay" and which not? How can i possibly ever recover
> my repo to a sane state if you refuse to give me write access to changes i
> have already made locally (but not cannot commit, leaving my repo in an
> unclean/divergent state).
>
>
> --
> ----- stephan beal
> http://wanderinghorse.net/home/stephan/
> http://gplus.to/sgbeal


--- ---
Eduardo Morras <***@yahoo.es>
Stephan Beal
2013-07-23 14:22:38 UTC
Permalink
On Tue, Jul 23, 2013 at 1:54 PM, Eduardo Morras <***@yahoo.es> wrote:

> I must insist here ;) There's a distinction, your repository is not the
> official Project repository, mine, the central one, yes.
>

That's the problem - here is no CENTRAL in _D_VCS. As soon as i clone, my
copy IS a master copy for all intents and purposes.

http://en.wikipedia.org/wiki/Distributed_revision_control

So as soon as i clone yours, both of us have the One True Copy (at least as
far as Fossil is concerned).

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Edward Berner
2013-07-24 06:38:10 UTC
Permalink
On 7/23/2013 7:22 AM, Stephan Beal wrote:
> On Tue, Jul 23, 2013 at 1:54 PM, Eduardo Morras <***@yahoo.es
> <mailto:***@yahoo.es>> wrote:
>
> I must insist here ;) There's a distinction, your repository is
> not the official Project repository, mine, the central one, yes.
>
>
> That's the problem - here is no CENTRAL in _D_VCS. As soon as i clone,
> my copy IS a master copy for all intents and purposes.
>
> http://en.wikipedia.org/wiki/Distributed_revision_control
>
> So as soon as i clone yours, both of us have the One True Copy (at
> least as far as Fossil is concerned).
>

Hmmm... actually, as I understand it, if you clone a Fossil repository
you'll have all of the files and their history but you won't have the
entire repository.

Specifically, a repository's local state is not cloned (see 2nd
paragraph of section 1.0 of
http://www.fossil-scm.org/index.html/doc/trunk/www/sync.wiki ). Most of
the local state can be cloned with other commands but you need to go a
bit out of your way (and have extra permissions?) to do it.

It seems fair for those servers of a project which do house the full
repository to be referred to as the central servers or central repositories.

--
Edward Berner
Eduardo Morras
2013-07-23 12:02:27 UTC
Permalink
Opps, sorry, I sent too early.

On Tue, 23 Jul 2013 13:17:47 +0200
Stephan Beal <***@googlemail.com> wrote:

> On Tue, Jul 23, 2013 at 1:04 PM, Eduardo Morras <***@yahoo.es> wrote:
>
> > Yes it is. There's a big difference between Project and Repository.
> > There's the fossil project, only one, but there are lots of repositories
> > and fossil project forks that aren't the main 'Fossil Project'. I don't
> > want to restrict what you do with your repository or your project fork, I
> > want to restrict what you can do to main Project.
> >
>
> That's the problem: my clone of your project is a first-class copy of your
> project. There is essentially no distinction. If you copy goes up in flames
> me can drop in my copy, re-add the users, and we're done. So here's what
> happens with user restrictions:
>
> - i clone your repo
> - i edit some dir which you have restricted.
> - i check it in locally (offline)
> - Later i try to push to your repo and it fails. My repo is now in a state
> where i can _never_ push my copy to yours ever again because pushing to
> yours will fail as long as i have no access (on your side) for that file.


I must insist here ;) There's a distinction, your repository is not the official Project repository, mine, the central one, yes. You can fork and create a new project and name it as you wish and change what you want. If you want to merge with the Only and Real True Project, your changes must be approved by the person that has the permission to do that, that's me.


> > No, I sync with fossil main repository each week, but I don't have
> > permission to push my changes. That's because my user (anonymous) don't
> > have the role to do so. If I want to push my changes I must convince
> > someone who has the correct role to do so.
> >
>
> But for "partial access" ALL pushes from me to you will fail if i have
> edited a file to which i cannot push. There is no "partial push" in fossil
> - you can push everything or nothing. Fossil, as a piece of software, could
> not possibly decide which parts of my commits are "safe" to push, and
> Fossil _MUST_ fail if _any_ part of my push/sync fails because it has no
> heuristic which can say, "oh, that was just that file under /noaccess/...
> which you edited 3 weeks ago and is still not writable for you" (and if it
> could decide that, then failing is still the proper response). The only
> reasonable action Fossil can take there is to fail. How could it possibly
> know which parts are "okay" and which not? How can i possibly ever recover
> my repo to a sane state if you refuse to give me write access to changes i
> have already made locally (but not cannot commit, leaving my repo in an
> unclean/divergent state).

If you don't have access to some files and you modified them, fossil shouldn't fail, fossil must fail the push/sync. You worked and touch things you shouldn't and must correct it. If you need a change in a file you can't touch, ask the user who owns them and convince him/her the changes are needed. If not, his/her work could fail and waste his/her time because someone modified the code. Out of fossil and scms, if you change the db schema without asking or noting the dba, the hell will be a pleasant location to hide ;)

--- ---
Eduardo Morras <***@yahoo.es>
Edward Berner
2013-07-24 06:09:34 UTC
Permalink
On 7/22/2013 4:36 AM, Stephan Beal wrote:
> On Mon, Jul 22, 2013 at 1:30 PM, Eduardo Morras <***@yahoo.es
> <mailto:***@yahoo.es>> wrote:
>
> Similar, a C developer role can modify /src dir but not
> /documentation or /sql, the DBA role can change /sql but not /src
> and similar. Role info is already exchanged in pull/push/sync,
> user info not. So in your scenary, a user can get the role
> capabilities and access them.
>
>
> So what happens when i (as a wiki editor) make a change to src/foo.c,
> commit it to my local copy (because locally i have admin access) and
> then try to push? i can't, and my repo then gets (permanently) out of
> sync. i don't think this model is possible in a DVCS.
>
> The problem is that a user is admin of his own repository file and
> can set whatever s/he wants and access everything. A global dvcs
> role and user data can solve this, not allowing this user make
> changes in public branches or trunk directories of central repo
> (or other users repos) if s/he has no privileges to do so.
>
>
> Once i clone it, the central repository has NO say-so in what i can
> and cannot check out. It can only control what i check IN, or (more
> correctly), what i try to push to the central server. If it will not
> let me push the changes i have already committed to my local repo then
> i have a serious problem.
>

Actually, I think something similar already exists, as shunning. If you
commit an artifact to your local repo that has been shunned at the
central repo you won't be able to push it to the central repo. I didn't
actually try that, though, so I don't know what the user interface
behavior is -- perhaps it appears to sync and silently drops the shunned
artifact.

If the user interface for committing a shunned artifact is or could be
made friendly enough, then perhaps the requested behavior of user-level
commit restrictions could be achieved by generalizing the shunning code
path. Instead of just "deny if artifact is in shunned list", allow
"deny if username is foo and filename is bar".

(I haven't actually looked at the shunning code, though, so it is
entirely possible that the above would not be feasible.)

--
Edward Berner
Joseph R. Justice
2013-07-22 22:52:06 UTC
Permalink
On Sun, Jul 21, 2013 at 1:12 PM, Eduardo Morras <***@yahoo.es> wrote:


> b) Plugins and hooks c modules
>
[...]

> c) source code parser, to find where a function, macro, etc... is
> declared
>

For programs in C and C++ (and presumably eventually other programming
languages), the LLVM Compiler Infrastructure Project (LLVM apparently
formerly stood for "Low Level Virtual Machine" but is now not an acronym,
at http://llvm.org/ ) and its subproject Clang ("a C language family
frontend for LLVM", at http://clang.llvm.org/ ) might be of some interest.
The goal, or at least one of the goals, of LLVM is to create modular and
reuseable components which can be used to build all sorts of things,
including (for instance) source code parsers. It does appear that
implementations (not necessarily the prime or core implementation however)
of many other popular programming languages are also using components of
the LLVM Project.



c) Repository of javascript, th1 code, css
>
> When starting a new repository I import html, js, css, th1 addons and
> features. If a central repository of those addons exist we, the fossil
> community, could share those addons, f.ex. code to integrate fossil in
> drupal/joomla,
>

At first thought, this almost sounds like something spiritually akin to
CPAN ("Comprehensive Perl Archive Network", at http://www.cpan.org/ ) and
its myriad siblings for all sorts of programming languages and tools.



Thanks for giving me a moment of your time by reading this. I hope it's of
some use, interest. Be well.



Joseph
Clark Christensen
2013-07-22 00:01:02 UTC
Permalink
Hi Stephan,

What you propose sounds like the SQLite model where the core is a lib, and the part we SCM users interact with is the CLI (or HTTP). 
Stephan Beal
2013-07-22 08:02:47 UTC
Permalink
Hi, Clark! A brief response from the office, and a longer one tonight when
i get home...

On Mon, Jul 22, 2013 at 2:01 AM, Clark Christensen <***@yahoo.com>wrote:

> Scripting language: I understand the Tcl roots, and I hope you would
> consider Javascript as a target. JS seems more universal these days.
> Maybe the Tcl guys would disagree. I have no idea what that means from an
> implementation standpoint.
>

The problem is the interpreter. i am not aware of a small embedable JS
interpreter. SpiderMonkey/Jaegermonkey are complex and poorly documented.
Google V8 is nice but (A) huge, (B) C++, and (C) they recently made drastic
API changes which invalidated every single v8-using client out there
(breaking 4-5 years of accumulated code of mine, and i'll never have the
time to fix it), which means that very few people outside of Google can
actually use v8 at the moment. JS _would_ also be my first choice, if we
only had a small, well-maintained interpreter.i don't know TCL, but the TCL
and Fossil communities seem to be cosmically bound to one another. There
are relatively few well-established small/embeddable interpreters. Lua,
TCL, ... none others come to mind (which are small).


I use Fossil as much as a straight ticketing system almost as much as I do
> as SCM. OK, maybe that's an exaggeration :-) In my company, doing
> anything through channels takes an act of god. So being able to throw
> together an ad-hoc issue tracker for a project in a matter of an hour is
> valuable. So, in the ticketing subsystem, I'd like to see, in no
> particular order:
> * built-in persistent integer ticket numbers in addition to the SHA1
> ticket/artifact ID. The SHA1 hexdigest fragments are too geeky for
> management during the weekly status meeting.
>

Stable incremental numbers are literally not possible to solve for DCVS
systems, which is why the SHA's (geeky/unwieldy as they are) are used.


> * persistent ticket create time built-into the ticket record. It's tough
> to write aging reports without a create time. I know I can add it after
> the fact. I just don't want to have to.
>

Certainly this data is there? (Maybe not as a concrete, field, though?)


> * ticket event notifications, or a table where they're queued for some
> other process to retrieve/send them.
>

This implies hooks, which basically implies scripting, so it's a few levels
of abstraction away from being possible (but is certainly something to be
looked at closely - this has been a long-requested feature).


> Otherwise, Fossil as-is fulfills my needs pretty well. Though,
> admittedly, my needs are relatively small at about 20 active project
> repos. The projects themselves are relatively small Perl/JS/HTML/CSS apps
> and the shared code to support them.
>
> Thanks for listening to me ramble :-))
>

Please keep it coming - in my experience users provide far better ideas for
new features than developers tend to.

i'll answer in more detail tonight...

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Gautier DI FOLCO
2013-07-22 08:13:51 UTC
Permalink
2013/7/22 Stephan Beal <***@googlemail.com>

> The problem is the interpreter. i am not aware of a small embedable JS
> interpreter. SpiderMonkey/Jaegermonkey are complex and poorly documented.
> Google V8 is nice but (A) huge, (B) C++, and (C) they recently made drastic
> API changes which invalidated every single v8-using client out there
> (breaking 4-5 years of accumulated code of mine, and i'll never have the
> time to fix it), which means that very few people outside of Google can
> actually use v8 at the moment. JS _would_ also be my first choice, if we
> only had a small, well-maintained interpreter.i don't know TCL, but the TCL
> and Fossil communities seem to be cosmically bound to one another. There
> are relatively few well-established small/embeddable interpreters. Lua,
> TCL, ... none others come to mind (which are small).
>

I think you can have a look at lo (http://iolanguage.org/) which is a very
tiny programming language.
If I can make a request : clean the code would be great, especially for
those (like me) which have university C skills.

PS : I know, this phrase is not english, sorry for that.
Laurens Van Houtven
2013-07-22 08:20:22 UTC
Permalink
Hi,

While, ceteris paribus, I'd certainly prefer Python, I definitely
understand that embedding CPython may be more trouble than it's worth. If
you are going to embed *something*, I'd vote for a langauge explicitly
designed with that purpose in mind, say, Lua.

However, if this is to happen, I really do think that Fossil shouldn't have
more than one embedded dynamic language. So, either we do everything with
TH1, or we have something that replaces TH1...

Definitely a huge +1 for hooks, though; I'd be perfectly happy with
something that gets passed to system(3)...

lvh
Stephan Beal
2013-07-22 08:29:14 UTC
Permalink
On Mon, Jul 22, 2013 at 10:20 AM, Laurens Van Houtven <***@lvh.io> wrote:

> Hi,
>
> While, ceteris paribus, I'd certainly prefer Python, I definitely
> understand that embedding CPython may be more trouble than it's worth. If
> you are going to embed *something*, I'd vote for a langauge explicitly
> designed with that purpose in mind, say, Lua.
>

Python is a very popular choice, but AFAIK none of the fossil devs use it
;). There _is_ a choice of intepreters (i've also got my own i'd like to
use) but TCL "would seem to be" the best fit, given fossil's history. LUA
is small, well-established, and portable but also total alien to most
people (myself included). All that said: if the core library is designed
properly then any number of script bindings can be attached to it. (e.g.
sqlite has at least 10 or 12 script bindings out there.)

However, if this is to happen, I really do think that Fossil shouldn't have
> more than one embedded dynamic language. So, either we do everything with
> TH1, or we have something that replaces TH1...
>

i spent lots of time last summer trying to get th1 to do more than it
currently does. The main problem with it is that it cannot report error
location information, so if you have a 20-line script and a typo on line
18, then you get to search through the first 18 lines to try to find the
mistake. th1 quickly becomes unmanageable for larger scripts.

Definitely a huge +1 for hooks, though; I'd be perfectly happy with
> something that gets passed to system(3)...
>

Maybe the shell "should" be the script interface language. But that of
course rules out usage on Windows, which would upset a great number of
people (not me ;). i think a scripting language is our only
realistic/portable option for hooks.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Richard Offer
2013-07-22 09:44:11 UTC
Permalink
On Jul 22, 2013, at 9:29 AM, Stephan Beal <***@googlemail.com> wrote:

>
> Maybe the shell "should" be the script interface language. But that of course rules out usage on Windows, which would upset a great number of people (not me ;). i think a scripting language is our only realistic/portable option for hooks.


What about a web-hook type mechanism? If I want to write my hooks in Python, I implement them inside a simple web-server (i.e. python -m SimpleHttpServer), Likewise for any other interpreted langauge...


I agree its not as nice as a real embedded scripting language - buts its better than picking a scipting langauge that I don't want/like/know :-)


richard.

>
> --
> ----- stephan beal
> http://wanderinghorse.net/home/stephan/
> http://gplus.to/sgbeal
Stephan Beal
2013-07-22 09:49:14 UTC
Permalink
On Mon, Jul 22, 2013 at 11:44 AM, Richard Offer <***@whitequeen.com>wrote:

> What about a web-hook type mechanism? If I want to write my hooks in
> Python, I implement them inside a simple web-server (i.e. python -m
> SimpleHttpServer), Likewise for any other interpreted langauge...
>

Web hooks is a new topic for me - i'll read up on that.


> I agree its not as nice as a real embedded scripting language - buts its
> better than picking a scipting langauge that I don't want/like/know :-)
>

Rest assured, your preferences for a scripting language will play a
significant role in any decision process ;). But if we can avoid tying
ourselves to a single language (and i think we can), then so much the
better.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Gautier DI FOLCO
2013-07-22 09:52:34 UTC
Permalink
2013/7/22 Richard Offer <***@whitequeen.com>

> What about a web-hook type mechanism? If I want to write my hooks in
> Python, I implement them inside a simple web-server (i.e. python -m
> SimpleHttpServer), Likewise for any other interpreted langauge...
>
>
> I agree its not as nice as a real embedded scripting language - buts its
> better than picking a scipting langauge that I don't want/like/know :-)
>
>
An RPC mecanism would be cool.
Stephan Beal
2013-07-22 09:58:57 UTC
Permalink
On Mon, Jul 22, 2013 at 11:52 AM, Gautier DI FOLCO <
***@gmail.com> wrote:

> 2013/7/22 Richard Offer <***@whitequeen.com>
>
>> What about a web-hook type mechanism? If I want to write my hooks in
>> Python, I implement them inside a simple web-server (i.e. python -m
>> SimpleHttpServer), Likewise for any other interpreted langauge...
>>
>>
>> I agree its not as nice as a real embedded scripting language - buts its
>> better than picking a scipting langauge that I don't want/like/know :-)
>>
>>
> An RPC mecanism would be cool.
>

That's an interesting idea. What would you imagine doing with fossil over
RPC?

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Gautier DI FOLCO
2013-07-22 10:04:39 UTC
Permalink
2013/7/22 Stephan Beal <***@googlemail.com>
>
> That's an interesting idea. What would you imagine doing with fossil over
> RPC?
>

For example putting the calls in queues (ZeroMQ, RabbitMQ, etc.) to make
asynchronous and distributed calls, to have a scallable architecture.
Stephan Beal
2013-07-22 10:10:30 UTC
Permalink
On Mon, Jul 22, 2013 at 12:04 PM, Gautier DI FOLCO <
***@gmail.com> wrote:

> 2013/7/22 Stephan Beal <***@googlemail.com>
>>
>> That's an interesting idea. What would you imagine doing with fossil over
>> RPC?
>>
>
> For example putting the calls in queues (ZeroMQ, RabbitMQ, etc.) to make
> asynchronous and distributed calls, to have a scallable architecture.
>

So you envision fossil making RPC calls to other services, correct? The
JSON API is a sort of RPC service. If we will add scriptable triggers then
they could do this sort of thing.
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Laurens Van Houtven
2013-07-22 10:19:53 UTC
Permalink
Hi,


Just my 2c: a JSON hook API a la Github would be fantastic.

Documentation: https://help.github.com/articles/post-receive-hooks

It would also hopefully make it easy to re-use existing services with
fossil, if the spec were sufficiently close :)

cheers
lvh


On Mon, Jul 22, 2013 at 12:10 PM, Stephan Beal <***@googlemail.com>wrote:

> On Mon, Jul 22, 2013 at 12:04 PM, Gautier DI FOLCO <
> ***@gmail.com> wrote:
>
>> 2013/7/22 Stephan Beal <***@googlemail.com>
>>>
>>> That's an interesting idea. What would you imagine doing with fossil
>>> over RPC?
>>>
>>
>> For example putting the calls in queues (ZeroMQ, RabbitMQ, etc.) to make
>> asynchronous and distributed calls, to have a scallable architecture.
>>
>
> So you envision fossil making RPC calls to other services, correct? The
> JSON API is a sort of RPC service. If we will add scriptable triggers then
> they could do this sort of thing.
> --
> ----- stephan beal
> http://wanderinghorse.net/home/stephan/
> http://gplus.to/sgbeal
>
> _______________________________________________
> fossil-users mailing list
> fossil-***@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
>
Jan Nijtmans
2013-07-22 10:28:13 UTC
Permalink
2013/7/22 Laurens Van Houtven <***@lvh.io>:
> Just my 2c: a JSON hook API a la Github would be fantastic.
>
> Documentation: https://help.github.com/articles/post-receive-hooks
>
> It would also hopefully make it easy to re-use existing services with
> fossil, if the spec were sufficiently close :)

A similar think is already in development now, See:
<https://www.fossil-scm.org/index.html/timeline?r=tkt-change-hook>
It is meant for "Ticket" and "Commit" hooks, but more hooks
could be added the same way.

Regards,
Jan Nijtmans
Gautier DI FOLCO
2013-07-22 11:05:24 UTC
Permalink
2013/7/22 Stephan Beal <***@googlemail.com>

> So you envision fossil making RPC calls to other services, correct? The
> JSON API is a sort of RPC service. If we will add scriptable triggers then
> they could do this sort of thing.
>

Yes, it seems to be the simpliest way to do it, to make a hook system "a la
Github" as said Laurens Van Houtven.
Jacek Cała
2013-07-22 14:47:37 UTC
Permalink
Hi all,

My 2 cents below regarding ticket numbering:

2013/7/22 Stephan Beal <***@googlemail.com>

>
> * built-in persistent integer ticket numbers in addition to the SHA1
>> ticket/artifact ID. The SHA1 hexdigest fragments are too geeky for
>> management during the weekly status meeting.
>>
>
> Stable incremental numbers are literally not possible to solve for DCVS
> systems, which is why the SHA's (geeky/unwieldy as they are) are used.
>

I wrote some time ago about this as I think you can have a simple,
relatively easy to grasp ticket numbering in DCVS (not integer though). It
involves three simple steps:
- project create initializes internal repo ticket number with '1',
- project clone adds suffix '.1' to the repo ticket number,
- creating a ticket increases the last number after the last dot or the
only number if it is the primary project repository.

Having this you would end up with a tree of ticket numbers like:
'1', '2',...
'1.1', '1.2', ...
'2.1', '2.2', ...

and it would reflect the tree structure of all repositories of a project.

Cheers,
Jacek
Isaac Jurado
2013-07-22 15:27:23 UTC
Permalink
On Mon, Jul 22, 2013 at 4:47 PM, Jacek Cała <***@gmail.com> wrote:
> Hi all,
>
> My 2 cents below regarding ticket numbering:
>
> 2013/7/22 Stephan Beal <***@googlemail.com>
>>
>>
>>> * built-in persistent integer ticket numbers in addition to the SHA1
>>> ticket/artifact ID. The SHA1 hexdigest fragments are too geeky for
>>> management during the weekly status meeting.
>>
>>
>> Stable incremental numbers are literally not possible to solve for
>> DCVS systems, which is why the SHA's (geeky/unwieldy as they are) are
>> used.
>
> I wrote some time ago about this as I think you can have a simple,
> relatively easy to grasp ticket numbering in DCVS (not integer
> though). It involves three simple steps:
>
> - project create initializes internal repo ticket number with '1',
> - project clone adds suffix '.1' to the repo ticket number,

Like this?

Origin: 1
Clone Origin to A: 1.1
Clone Origin to B: 1.2
Clone A to C: 1.1.1
Clone A to D: 1.1.2
Clone B to E: 1.2.1
Clone B to F: 1.2.2

A creates ticket: 1.1.1
B creates ticket: 1.2.1

Regards.

--
Isaac Jurado

"The noblest pleasure is the joy of understanding"
Leonardo da Vinci
Stephan Beal
2013-07-22 15:34:13 UTC
Permalink
On Mon, Jul 22, 2013 at 4:47 PM, Jacek Cała <***@gmail.com> wrote:

> - project create initializes internal repo ticket number with '1',
> - project clone adds suffix '.1' to the repo ticket number,
>

That would require that cloning change the central repo (because it has to
assign and store the new number for each clone) and would be open to attack
by simply cloning the repo in a loop. If i clone it 10 times (and i've
_certainly_ cloned the main fossil repo at least that many times) then i
end up with 10 different numbers after the dot.

Hundreds, if no thousands, of clones of the core fossil repo and tcl repos
exists. i don't see this approach working for those two trees.

Or am i missing/misunderstanding some detail?

:-?

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Stephan Beal
2013-07-22 15:40:23 UTC
Permalink
On Mon, Jul 22, 2013 at 5:34 PM, Stephan Beal <***@googlemail.com> wrote:

> On Mon, Jul 22, 2013 at 4:47 PM, Jacek Cała <***@gmail.com> wrote:
>
>> - project create initializes internal repo ticket number with '1',
>> - project clone adds suffix '.1' to the repo ticket number,
>>
>
Alternate suggestion: we simply print the SHAs in decimal form ;). They're
not sequential but at least they're human-readable ;).

A lot of people who are _much_ brainier than i have determined that the
sequential numbering problem cannot be solved for DVCS systems, and i tend
to trust their judgement/calculations. As soon as you give up a central
authority, sequential numbering becomes impossible to properly implement.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
j. van den hoff
2013-07-22 15:52:56 UTC
Permalink
On Mon, 22 Jul 2013 17:40:23 +0200, Stephan Beal <***@googlemail.com>=
=

wrote:

> On Mon, Jul 22, 2013 at 5:34 PM, Stephan Beal <***@googlemail.com> =
=

> wrote:
>
>> On Mon, Jul 22, 2013 at 4:47 PM, Jacek Ca=C5=82a <***@gmail.co=
m> =

>> wrote:
>>
>>> - project create initializes internal repo ticket number with '1',
>>> - project clone adds suffix '.1' to the repo ticket number,
>>>
>>
> Alternate suggestion: we simply print the SHAs in decimal form ;). =

> They're
> not sequential but at least they're human-readable ;).
>
> A lot of people who are _much_ brainier than i have determined that th=
e
> sequential numbering problem cannot be solved for DVCS systems, and i =
=

> tend
> to trust their judgement/calculations. As soon as you give up a centra=
l
> authority, sequential numbering becomes impossible to properly impleme=
nt.


that issue has come up before and I believe you are right w.r.t. a =

globally valid sequential numbering being impossible (and translating th=
e =

'topology' of checkins and repos/clones by something like 1.2.3 does not=
=

seem to help much, either, although it sure is possible in principle).

still, I would _much_ prefer to be able to perform all local actions on =
a =

repo using _locally_ unambiguous sequential numbering of all checkins =

(just like `hg' has always been doing it: checkins are denoted by =

something like 10:0cb3d1256b... where the `10' is the locally valid =

incremental index of the checkin which can be used instead of the sha1 =

hash in all commands). one obvious example being inspection of diffs =

between consecutive checkins.

I also have never heard that use of such sequential numbering has caused=
=

confusion for hg users (as long as it is understood not to talk about =

"checkin number 10" across different clones of the project repo).

for the wishlist: this would be 'nice to have'.
as would be a facility to get the complete timeline output without havin=
g =

to guess/enter a sufficiently large number for the `-n' flag....
and of course `grep'.


j.


>


-- =

Using Opera's revolutionary email client: http://www.opera.com/mail/
Stephan Beal
2013-07-22 16:00:09 UTC
Permalink
On Mon, Jul 22, 2013 at 5:52 PM, j. van den hoff
<***@googlemail.com>wrote:

> unambiguous sequential numbering of all checkins (just like `hg' has
> always been doing it: checkins are denoted by something like
> 10:0cb3d1256b... where the `10' is the locally valid incremental index of
> the checkin which can be used instead of the sha1 hash in all commands).
> one obvious example being inspection of diffs between consecutive checkins.
>

We could probably use the local db record IDs for that purpose. They might
be 3-5 digits and not necessarily incremented in steps of one (because
blobs are used for various purposes), but they would be sequential in time
(at least in theory).


> as would be a facility to get the complete timeline output without having
> to guess/enter a sufficiently large number for the `-n' flag....
>

The simplest change would be for it to interpret negative values as
"unlimited", and maybe add a --reverse flag so that --reverse -n -1 would
show only the first checkin?


> and of course `grep'.
>

There is some code for this but it only looks in local files, so it's just
a glorified (and platform-neutral) version of the /usr/bin/grep. In any
case, i've added a section on "search" to the doc.

i'm off of work starting Wednesday and will attempt to set up a skeleton
app where we can start prototyping/experimenting (while making no sort of
commitments). It occurred to me today that the command line arg processing
is app-level, not lib-level, and there's lots of "pre-work" i can do there
without having any concrete details about the library.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Stephan Beal
2013-07-22 16:01:27 UTC
Permalink
On Mon, Jul 22, 2013 at 6:00 PM, Stephan Beal <***@googlemail.com> wrote:

> reverse flag so that --reverse -n -1 would show only the first checkin?
>

Correction: --reverse -n 1
or: ... -n -1 | tail -1

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
j. van den hoff
2013-07-22 16:09:10 UTC
Permalink
On Mon, 22 Jul 2013 18:01:27 +0200, Stephan Beal <***@googlemail.com>
wrote:

> On Mon, Jul 22, 2013 at 6:00 PM, Stephan Beal <***@googlemail.com>
> wrote:
>
>> reverse flag so that --reverse -n -1 would show only the first checkin?

then I would opt for `-n 0' to get the whole time line. but actually I
would prefer a `-u(nlimited)' or `-a(ll)' flag or similar. the usefullness
of --reverse I'm not so sure about, at least I do not miss it right now...

>>
>
> Correction: --reverse -n 1
> or: ... -n -1 | tail -1
>


--
Using Opera's revolutionary email client: http://www.opera.com/mail/
Stephan Beal
2013-07-22 16:12:03 UTC
Permalink
On Mon, Jul 22, 2013 at 6:09 PM, j. van den hoff
<***@googlemail.com>wrote:

> then I would opt for `-n 0' to get the whole time line. but actually I
> would prefer a `-u(nlimited)' or `-a(ll)' flag or similar. the usefullness
> of --reverse I'm not so sure about, at least I do not miss it right now...
>

-n 0 is already interpreted as some default, so that would break anyone
depending on the current semantics. -u is used for USER (maybe not in the
CLI version). i don't think -a is used, but i recently added that to the
'search' command, so there is a precedence for it.


--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
j. van den hoff
2013-07-22 16:18:38 UTC
Permalink
On Mon, 22 Jul 2013 18:12:03 +0200, Stephan Beal <***@googlemail.com>
wrote:

> On Mon, Jul 22, 2013 at 6:09 PM, j. van den hoff
> <***@googlemail.com>wrote:
>
>> then I would opt for `-n 0' to get the whole time line. but actually I
>> would prefer a `-u(nlimited)' or `-a(ll)' flag or similar. the
>> usefullness
>> of --reverse I'm not so sure about, at least I do not miss it right
>> now...
>>
>
> -n 0 is already interpreted as some default, so that would break anyone
> depending on the current semantics. -u is used for USER (maybe not in the
> CLI version). i don't think -a is used, but i recently added that to the
> 'search' command, so there is a precedence for it.


`-n 1' and `-n 0' currently seem to do the same thing: showing the last
checkin. this does not even seem consistent ( -n 0 probably right now
should show nothing?) and the CLI does neither use -u nor -a as `timeline'
options, so this should work, right?

another small observation: fossil help timeline contains the statement:

Options:
-n|--limit N Output the first N changes (default 20)

where the "first" probably should be a "last" or "most recent" I'd say.

>
>


--
Using Opera's revolutionary email client: http://www.opera.com/mail/
Stephan Beal
2013-07-22 16:23:43 UTC
Permalink
On Mon, Jul 22, 2013 at 6:18 PM, j. van den hoff
<***@googlemail.com>wrote:

> Options:
> -n|--limit N Output the first N changes (default 20)
>
> where the "first" probably should be a "last" or "most recent" I'd say.


-n is a bit misleading, actually - that limits the number of _lines_ of
output, not the number of results (i had forgotten that until now). If the
limit would truncate a given commit then it finishes outputting that before
ending:

***@tiny:~/cvs/fossil/cwal/th1ish$ f time -n 2
=== 2013-07-21 ===
18:26:43 [77d1757a8b] *CURRENT* Added Object.toJSONString(). Still missing
overridable toJSON() support. Moved th1ish_value_XXX_part() to
cwal_value_XXX_part(). Added String.applyFormat(). (user: stephan
tags: trunk)

***@tiny:~/cvs/fossil/cwal/th1ish$ f time -n 4
=== 2013-07-21 ===
18:26:43 [77d1757a8b] *CURRENT* Added Object.toJSONString(). Still missing
overridable toJSON() support. Moved th1ish_value_XXX_part() to
cwal_value_XXX_part(). Added String.applyFormat(). (user: stephan
tags: trunk)

(same thing, but if i use -n 5 i get the next entry as well)

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Martin Gagnon
2013-07-22 17:26:32 UTC
Permalink
Le 22 juil. 2013 12:23, "Stephan Beal" <***@googlemail.com> a écrit :
>
> On Mon, Jul 22, 2013 at 6:18 PM, j. van den hoff <
***@googlemail.com> wrote:
>>
>> Options:
>> -n|--limit N Output the first N changes (default 20)
>>
>> where the "first" probably should be a "last" or "most recent" I'd say.
>
>
> -n is a bit misleading, actually - that limits the number of _lines_ of
output, not the number of results (i had forgotten that until now). If the
limit would truncate a given commit then it finishes outputting that before
ending:
>
> ***@tiny:~/cvs/fossil/cwal/th1ish$ f time -n 2
> === 2013-07-21 ===
> 18:26:43 [77d1757a8b] *CURRENT* Added Object.toJSONString(). Still missing
> overridable toJSON() support. Moved th1ish_value_XXX_part() to
> cwal_value_XXX_part(). Added String.applyFormat(). (user: stephan
> tags: trunk)
>
> ***@tiny:~/cvs/fossil/cwal/th1ish$ f time -n 4
> === 2013-07-21 ===
> 18:26:43 [77d1757a8b] *CURRENT* Added Object.toJSONString(). Still missing
> overridable toJSON() support. Moved th1ish_value_XXX_part() to
> cwal_value_XXX_part(). Added String.applyFormat(). (user: stephan
> tags: trunk)
>
> (same thing, but if i use -n 5 i get the next entry as well)

Actually, I've always wonder why the -n option is like this on CLI, and why
it is different from the n= parameter on the webpage counterpart?. May be
not a lot of people use it on the CLI?

--
Martin G.
Stephan Beal
2013-07-22 17:28:55 UTC
Permalink
On Mon, Jul 22, 2013 at 7:26 PM, Martin Gagnon <***@gmail.com> wrote:

> Actually, I've always wonder why the -n option is like this on CLI, and
> why it is different from the n= parameter on the webpage counterpart?. May
> be not a lot of people use it on the CLI?
>

i think we all wonder that ;). It's "historical in nature," and probably
best left as-is for the v1 app.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
David Given
2013-07-22 16:06:43 UTC
Permalink
Stephan Beal wrote:
[...]
> Alternate suggestion: we simply print the SHAs in decimal form ;).
> They're not sequential but at least they're human-readable ;).

A while back I did come up with an astoundingly stupid idea for making
SHAs more readable; fortunately I never got around to implementing it...

The idea is: take the first few bytes of the hash, split it up into 7 or
8 bit chunks, and look each one up in a dictionary. This gives us a
codename for the hash. So, instead of referring to a bug as 639ab1, we
get to call it 'CORRECT HORSE BATTERY', or 'CASE NIGHTMARE GREEN', or
'LUMINOUS PANDA UKELELE', or whatever.

The advantage of this is that the randomly-generated names are vastly
more memorable than the hashes. I'd probably want to limit it to three
words, maybe four if you push it; depending on the size of the codename
dictionary this would probably give about 20 to 30 bits of entropy.
Tweaking the algorithm to prevent duplications would probably also be a
good idea.

Done right this gives a lossless bidirectional mapping from hashes to
codenames. The difficult bit, of course, is coming up with the codename
dictionary (and being able to use the codenames with a straight face).

(I should point out that what3words.com is doing something very similar
for locations, although their codenames seem to be database keys
generated on demand rather than encoding the location directly.)

--
┌───  ───── http://www.cowlark.com ─────
│ "USER'S MANUAL VERSION 1.0: The information presented in this
│ publication has been carefully for reliability." --- anonymous
│ computer hardware manual
Stephan Beal
2013-07-22 16:10:16 UTC
Permalink
On Mon, Jul 22, 2013 at 6:06 PM, David Given <***@cowlark.com> wrote:

> ....SHAs more readable; fortunately I never got around to implementing
> it...
> ...t to call it 'CORRECT HORSE BATTERY', or 'CASE NIGHTMARE GREEN', or
> 'LUMINOUS PANDA UKELELE', or whatever.
>

LOL! Yes, thank goodness you never implemented it ;). XKCD actually had a
comic on that:

http://xkcd.com/936/

he "proves" that CORRECT HORSE BATTERY is actually less secure because it's
more memorable (see, you've already memorized it ;).


> Done right this gives a lossless bidirectional mapping from hashes to
> codenames. The difficult bit, of course, is coming up with the codename
> dictionary (and being able to use the codenames with a straight face).
>

i would certainly like to see a proof of concept, but i can't imagine being
taken seriously when i post a message saying, "guys, can you please review
commit BASIC ORANGE MONKEY and tell me what you think?" ;)

But that WOULD be a funny proof-of-concept plugin for the v2 API.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Jacek Cała
2013-07-22 16:02:53 UTC
Permalink
2013/7/22 Stephan Beal <***@googlemail.com>

> On Mon, Jul 22, 2013 at 4:47 PM, Jacek Cała <***@gmail.com> wrote:
>
>> - project create initializes internal repo ticket number with '1',
>> - project clone adds suffix '.1' to the repo ticket number,
>>
>
> That would require that cloning change the central repo (because it has to
> assign and store the new number for each clone) and would be open to attack
> by simply cloning the repo in a loop. If i clone it 10 times (and i've
> _certainly_ cloned the main fossil repo at least that many times) then i
> end up with 10 different numbers after the dot.
>
> Hundreds, if no thousands, of clones of the core fossil repo and tcl repos
> exists. i don't see this approach working for those two trees.
>
> Or am i missing/misunderstanding some detail?
>

Sorry for not being precise enough. You don't need to change the central
repo. In fact you don't change any repo at all. Only during cloning you
create a new 'seed' ticket number. Each repository has its own, ticket
number prefix, either '1' if it is created or '...n.n.1' if it is cloned.

Together with this 'seed' each repo has it's last_ticket_number variable
(initalized with 1) which is a simple integer added to the seed and
incremented on the 'create ticket' operation. I was to quick to say about
dots only, so the last_ticket_number is added to the seed with, let say, a
dash. Then, a ticket number is constructed as '<seed>-<last_ticket_number>'.

Cloning creates a repository tree:
1
1.1
1.2
1.3
1.2.1
etc.

whereas creating tickets adds an internal ticket number (independent for
each repo) and so you can have:
1-1, 1-2, 1-3
1.1-1, 1.1-2
1.2.1-1, 1.2.1-2,...

Is this clearer?

Cheers,
Jacek
Stephan Beal
2013-07-22 16:07:23 UTC
Permalink
On Mon, Jul 22, 2013 at 6:02 PM, Jacek Cała <***@gmail.com> wrote:

> Sorry for not being precise enough. You don't need to change the central
> repo. In fact you don't change any repo at all. Only during cloning you
> create a new 'seed' ticket number. Each repository has its own, ticket
> number prefix, either '1' if it is created or '...n.n.1' if it is cloned.
>

When you say "repo" do you mean each clone, or each "central" repo?

Together with this 'seed' each repo has it's last_ticket_number variable
> (initalized with 1) which is a simple integer added to the seed and
> incremented on the 'create ticket' operation. I was to quick to say about
> dots only, so the last_ticket_number is added to the seed with, let say, a
> dash. Then, a ticket number is constructed as '<seed>-<last_ticket_number>'.
>

Those would need to be purely local, correct?


> Cloning creates a repository tree:
> 1
> 1.1
> 1.2
> 1.3
> 1.2.1
> etc.
>

In the clone or in the central one?


> whereas creating tickets adds an internal ticket number (independent for
> each repo) and so you can have:
> 1-1, 1-2, 1-3
> 1.1-1, 1.1-2
> 1.2.1-1, 1.2.1-2,...
>
> Is this clearer?
>

i am assuming by "each repo" you mean "each clone" (which is also "each
repo"). If that is the case, i can conceive of this working strictly
locally, but i still don't see how it can possibly scale if those numbers
propagate in any way. If i clone the repo 100 times, do i end up with 1.1
... 1.100 ? What if a malicious person clones my repo in a loop a million
times? If the numbers are local, not a problem, but if they propagate then
that is an attack vector.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Jacek Cała
2013-07-22 16:15:02 UTC
Permalink
2013/7/22 Stephan Beal <***@googlemail.com>

> On Mon, Jul 22, 2013 at 6:02 PM, Jacek Cała <***@gmail.com> wrote:
>
>> Sorry for not being precise enough. You don't need to change the central
>> repo. In fact you don't change any repo at all. Only during cloning you
>> create a new 'seed' ticket number. Each repository has its own, ticket
>> number prefix, either '1' if it is created or '...n.n.1' if it is cloned.
>>
>
> When you say "repo" do you mean each clone, or each "central" repo?
>

Each clone


> Together with this 'seed' each repo has it's last_ticket_number variable
>> (initalized with 1) which is a simple integer added to the seed and
>> incremented on the 'create ticket' operation. I was to quick to say about
>> dots only, so the last_ticket_number is added to the seed with, let say, a
>> dash. Then, a ticket number is constructed as '<seed>-<last_ticket_number>'.
>>
>
> Those would need to be purely local, correct?
>

Indeed, both just local variables, both set on clone operation.


>
>
>> Cloning creates a repository tree:
>> 1
>> 1.1
>> 1.2
>> 1.3
>> 1.2.1
>> etc.
>>
>
> In the clone or in the central one?
>

These are the seeds of each of the clones. '1' is the primary repo created
with 'fossil init', others are clones, e.g. '1.2.1' comes from cloning
'1.2' which is the second clone of '1'.


>
>> whereas creating tickets adds an internal ticket number (independent for
>> each repo) and so you can have:
>> 1-1, 1-2, 1-3
>> 1.1-1, 1.1-2
>> 1.2.1-1, 1.2.1-2,...
>>
>> Is this clearer?
>>
>
> i am assuming by "each repo" you mean "each clone" (which is also "each
> repo"). If that is the case, i can conceive of this working strictly
> locally, but i still don't see how it can possibly scale if those numbers
> propagate in any way. If i clone the repo 100 times, do i end up with 1.1
> ... 1.100 ?
>

Yes, that would only mean that your clone tickets will start with
'1.100-...' but it doesn't affect clone '1' much. All tickets of '1' are in
form of '1-n'


> What if a malicious person clones my repo in a loop a million times? If
> the numbers are local, not a problem, but if they propagate then that is an
> attack vector.
>

The only downside I see is that if a malicious person did that (given you
have enough network bandwith) and you clone your repo next time, your
tickets will start with prefix '1.1000000-'.


> --
> ----- stephan beal
> http://wanderinghorse.net/home/stephan/
> http://gplus.to/sgbeal
>
> _______________________________________________
> fossil-users mailing list
> fossil-***@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
>
Stephan Beal
2013-07-22 16:20:52 UTC
Permalink
On Mon, Jul 22, 2013 at 6:15 PM, Jacek Cała <***@gmail.com> wrote:

>
>> i am assuming by "each repo" you mean "each clone" (which is also "each
>> repo"). If that is the case, i can conceive of this working strictly
>> locally, but i still don't see how it can possibly scale if those numbers
>> propagate in any way. If i clone the repo 100 times, do i end up with 1.1
>> ... 1.100 ?
>>
>
> Yes, that would only mean that your clone tickets will start with
> '1.100-...' but it doesn't affect clone '1' much. All tickets of '1' are in
> form of '1-n'
>

There's the rub. For that to happen, the number needs to be save in the
repo, which requires write access to the repo for anyone who can clone (and
guest can clone from most repos). That alone is a potential hole which more
security-conscious people wouldn't tolerate. If some malicious **** ran
this from 10 shells:

# while true; do rm -f clone.fsl; fossil clone http://.... clone.fsl; done

and let it run for a few days, he's just screwed up my numbering so bad
that i probably won't want to use sequential numbering anymore. It's not
helpful if my numbers are 8 digits long.

i used to think that fossil was immune to the attentions of "those sorts of
people" but i have had my own fossil wikis attacked/replaced by bots (due
to a permissions mishap, not a fossil bug), so i'm weary of anything which
gives anyone any sort of write access which i don't explicitly give them.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Jacek Cała
2013-07-22 16:31:06 UTC
Permalink
2013/7/22 Stephan Beal <***@googlemail.com>

> On Mon, Jul 22, 2013 at 6:15 PM, Jacek Cała <***@gmail.com> wrote:
>
>>
>>> i am assuming by "each repo" you mean "each clone" (which is also "each
>>> repo"). If that is the case, i can conceive of this working strictly
>>> locally, but i still don't see how it can possibly scale if those numbers
>>> propagate in any way. If i clone the repo 100 times, do i end up with 1.1
>>> ... 1.100 ?
>>>
>>
>> Yes, that would only mean that your clone tickets will start with
>> '1.100-...' but it doesn't affect clone '1' much. All tickets of '1' are in
>> form of '1-n'
>>
>
> There's the rub. For that to happen, the number needs to be save in the
> repo, which requires write access to the repo for anyone who can clone (and
> guest can clone from most repos). That alone is a potential hole which more
> security-conscious people wouldn't tolerate. If some malicious **** ran
> this from 10 shells:
>
> # while true; do rm -f clone.fsl; fossil clone http://.... clone.fsl; done
>
> and let it run for a few days, he's just screwed up my numbering so bad
> that i probably won't want to use sequential numbering anymore. It's not
> helpful if my numbers are 8 digits long.
>

Certainly, true... but you can think about a 'superclone' that has the
mentioned feature (with access restrictions), whereas the standard clone
operation would initialize the cloned repo with some random seed not
changing the parent. And even with random seeds you would still have a nice
ticket numbering within your clone.

Generally, I found not having human readable tickets numbers so difficult
that I used to embed some sort of identifiers within the ticket name (not
scalable at all :-(

Cheers,
Jacek
Stephan Beal
2013-07-22 16:45:27 UTC
Permalink
On Mon, Jul 22, 2013 at 6:31 PM, Jacek Cała <***@gmail.com> wrote:

> Generally, I found not having human readable tickets numbers so difficult
> that I used to embed some sort of identifiers within the ticket name (not
> scalable at all :-(
>

i suspect that using the local db record IDs would be a usable
solution/workaround for this. They won't necessarily be stable across
systems/clones, but that's just a property of the nature of DVCS. i'll add
that to the list momentarily.

i've been editing that gdoc so long now, and so many people of been in and
out of it all day, that when i'm typing emails it feels like you all are
watching as i type my emails. :/

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Stephan Beal
2013-07-22 09:00:53 UTC
Permalink
It's remarkably slow at work so far today, so here's the answer i promised
for tonight...

On Mon, Jul 22, 2013 at 2:01 AM, Clark Christensen <***@yahoo.com>wrote:

> Hi Stephan,
>
> What you propose sounds like the SQLite model where the core is a lib, and
Baruch Burstein
2013-07-22 11:58:13 UTC
Permalink
On Mon, Jul 22, 2013 at 12:00 PM, Stephan Beal <***@googlemail.com>wrote:

> * built-in full text search for tickets.
>>
>
> Searching has been one of the most-requested features for fossil lately.
> The main difficulty is that it's not as simple as "select * from xyz where
> field like ..." because artifacts are stored as deltas (which are useless
> for searching purposes). So to search an artifact its body has to be
> recreated from its baseline and deltas. To find out what version of a file
> a given search term was in we have to rebuild _every_ version of that file
> and then remember them long enough to be able to report the search results
> to the user. Take a large file, llke sqlite3.c, try to search through its
> whole history for the word "sqlite" and you'll probably spend the rest of
> the day waiting on fossil to calculate the results. It "might"(???) make
> more sense to provide searching as an extra tool which effectively builds a
> cache of a "deconstructed" repo, containing "expanded" copies of all
> artifacts, and performs searches against them.
>
> The full-text index can also be kept in deltas, just not the same format
as the artifact deltas. For each artifact, you just keep a list of terms
that were added and terms that were removed relative to it's parent
artifact. Or it might make sense to keep these deltas (and artifact deltas
too) in reverse order (delta from child), since later versions are wanted
at a higher frequency than older versions.

>
> --
> ----- stephan beal
> http://wanderinghorse.net/home/stephan/
> http://gplus.to/sgbeal
>
> _______________________________________________
> fossil-users mailing list
> fossil-***@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
>


--
˙uʍop-ǝpısdn sı ɹoʇıuoɯ ɹnoʎ 'sıɥʇ pɐǝɹ uɐɔ noʎ ɟı
Konstantin Khomoutov
2013-07-22 09:21:56 UTC
Permalink
On Sun, 21 Jul 2013 17:01:02 -0700 (PDT)
Clark Christensen <***@yahoo.com> wrote:

[...]
> Scripting language: I understand the Tcl roots, and I hope you would
> consider Javascript as a target.  JS seems more universal these days.
[...]

Please, don't. JS is a wart right from the start -- supposedly the
only popular programming language these days for which a book titled
"JavaScript: the good parts" has been written, which actually devoted
to teaching the prospective developers how to *not* use the
language. Yes, there are hordes of (usually underskilled) web
developers who know JS by necessity, but IMO this is not a *technical*
consideration. All this current hype for JS (node.js included) is a
good display of herd behaviour in human society, something not to be
proud of.
Joseph R. Justice
2013-07-22 23:20:21 UTC
Permalink
On Mon, Jul 22, 2013 at 5:21 AM, Konstantin Khomoutov <
***@users.sourceforge.net> wrote:

> On Sun, 21 Jul 2013 17:01:02 -0700 (PDT) Clark Christensen <
> ***@yahoo.com> wrote:
>


> > Scripting language: I understand the Tcl roots, and I hope you would
> > consider Javascript as a target. JS seems more universal these days.
> [...]
>
> Please, don't. JS is a wart right from the start -- supposedly the
> only popular programming language these days for which a book titled
> "JavaScript: the good parts" has been written, which actually devoted
> to teaching the prospective developers how to *not* use the
> language.


FWIW, there appear to be at least three other "Xyzzy: The Good Parts" books
out there now, all by O'Reilly & Associates, for PHP, Java, and "HTML &
CSS". (Of course, given the history of these particular programming
languages and technologies, the fact of this isn't necessarily doing
Javascript any favors, I admit.)

I do believe there's also a number of more recent programming languages out
there which are supposed to be nicer / better than Javascript but compile
into it, or are supposed to be safer / better subsets of full Javascript,
or whatever. Admittedly, because these *are* more recent languages,
they're less mature and likely more unstable / likely to change and perhaps
even change non-trivially.

All this isn't to disparage your charges against Javascript. I don't
nevessarily disagree with them. I just think that, perhaps, things are a
little more nuanced or grey than a straight, black and white,
uncompromising stance of "Javascript BAD!"

And, remember, JSON (which I do believe people here are attempting to
incorporate into Fossil) stands for "JavaScript Object Notation" (see
http://www.json.org/ ). Now, it's not a programming language in and of
itself, but... *shrug*



Thanks got giving me a moment of your time by reading this. Hope it's of
some use, interest. Be well.



Joseph
Aaron W.Hsu
2013-07-23 01:25:17 UTC
Permalink
I am a relatively new Fossil user, and I have not even really stretched fossil’s muscles yet. However, I have made extensive use of Git and Monotone. I far preferred Monotone to Git, but the social pressures eventually made me give up fighting for Monotone and move to Git. I hope that Fossil can provide a number of the things that I really enjoyed about Monotone without the associated disadvantages.


This brings me to the discussion that I have seen about Scripting languages in DVCS systems. I’m really not sure how I feel about scripting and all of these hooks in various DVCS systems, especially in comparison to the relative plug and play nature of Fossil. In particular, I am thinking of the very plumbing oriented nature of Git, and the very integrated scripting layer, in Lua, for Monotone. For one thing, while I enjoy the dirty convenience of the piping and UNIX style workflow for most work, I’m not as convinced when it comes to version control.


I’ve so far managed to avoid using the feature myself, but I have been on teams where people tried to use this to some effect, and unfortunately, the interfaces always seemed to scale very poorly. That’s not to say that they cannot be made to scale, but it seems to be a little too fragile for my sensibilities. Monotone’s integrated scripting, on the other hand, had another problem. From an user’s perspective, I found that the strict support for a single scripting language made it difficult to integrate Monotone fully into whatever project I was using. While the Lua scripting enabled me to gain a level of sophistication and relative rigor in the process more than what I could get from normal UNIX plumbing, if my project wasn’t in Lua in the first place, I found it breaking my concentration a good deal more than I would have liked.


I feel like there should be a balance when it comes to extensibility. Indeed, I think that a good DVCS ought to never require the user to start extending or programming their VCS in order to use it normally from day to day. The plug-and-play nature of Fossil is a huge advantage to me right now, as it’s so simple to use and get running on any given platform and workflow. It just works, and does not rely on any extension interface, such as a scripting or hook layer to get things done. On the other hand, it seems to me that scripting and a programmatic interface *is* very good when it comes to integrating VCS into a project’s workflow. It’s this last part that I think should be the target of any extensibility. I would argue that, to me, one should not direct the programmatic interface towards end-users, but rather, those who wish to integrate Fossil into a larger framework. In that case, I see the limitation of a single scripting language as a problem. I also think that the ad hoc nature of a “text” based interface like UNIX style plumbing to be a problem. Instead, perhaps there is a way of creating a public interface to Fossil that allows one to reliably extend its functionality without restricting yourself to a single language.


Here’s a simple idea of how this might work. Publish a specific interface in C against which people can write their code. Allow people to generate shared objects that export a specific set of functions for registering or extending functionality of the Fossil executable. Then, these executables can be placed inside of a specific directory, perhaps found through a set of paths in an environment variable. When fossil runs, it can dynamically link in any of the shared objects that it finds there, and these can register additional functionality with the Fossil system. In this way, people can use whatever language they want to use, as long as it has a bi-directional C FFI, which most languages I have seen have. The fossil executable will still be shipped as a single executable, and people would only need a simple C header file to use the extended features, which can be obtained separately. It would have the benefit of a rigorously defined interface with more than textual interfaces for increased reliability of the code, but people could still use shell scripts if that’s what they really wanted to do, they would just have to be dispatched from a shared object.


Just a few thoughts that I have wondered about when it comes to extension interfaces. Most of the time I tend not to use them because they don’t work with my language or they are too ad hoc for me to rely on. I think the above would solve those issues while still keeping Fossil dirt simple for the end user.



--
Aaron W. Hsu | ***@sacrideo.us | http://www.sacrideo.us
Stephan Beal
2013-07-23 08:29:36 UTC
Permalink
On Tue, Jul 23, 2013 at 3:25 AM, Aaron W.Hsu <***@sacrideo.us> wrote:

> I’ve so far managed to avoid using the feature myself, but I have been on
> teams where people tried to use this to some effect, and unfortunately, the
> interfaces always seemed to scale very poorly. That’s not to say that they
> cannot be made to scale, but it seems to be a little too fragile for my
> sensibilities. Monotone’s integrated scripting, on the other hand, had
j. van den hoff
2013-07-23 09:03:09 UTC
Permalink
On Tue, 23 Jul 2013 10:29:36 +0200, Stephan Beal <***@googlemail.com>=
=

wrote:

> On Tue, Jul 23, 2013 at 3:25 AM, Aaron W.Hsu <***@sacrideo.us> wro=
te:
>
>> I=E2=80=99ve so far managed to avoid using the feature myself, but I =
have been =

>> on
>> teams where people tried to use this to some effect, and unfortunatel=
y, =

>> the
>> interfaces always seemed to scale very poorly. That=E2=80=99s not to =
say that =

>> they
>> cannot be made to scale, but it seems to be a little too fragile for =
my
>> sensibilities. Monotone=E2=80=99s integrated scripting, on the other =
hand, had
>> another problem.
Konstantin Khomoutov
2013-07-23 09:29:32 UTC
Permalink
On Tue, 23 Jul 2013 11:03:09 +0200
"j. van den hoff" <***@googlemail.com> wrote:

[...]

> >> While the Lua scripting enabled me to gain a level of
> >> sophistication and relative rigor in the process more than what I
> >> could get from normal UNIX
> >> plumbing, if my project wasn’t in Lua in the first place, I found
> >> it breaking my concentration a good deal more than I would have
> >> liked.
> >
> > That's my impression of lua, though i haven't worked with it (it's
> > too weird, both at the script and C API level).
>
> really? regarding the language (the scripting level) I find lua
> exceptionally well thought out and clear. the syntax is
> intentionally very "boring" -- no surprises here -- and sure much
> easier to swallow than tcl, for instance, for people
> starting from scratch. regarding the C level API I don't have any
> experience (nor an opinion)
> but it claims to be much easier than that of other scripting
> languages. the LaTeX guys at least have decided to
> move into this direction to get a real scripting facility into latex
> http://luatex.org/ which personally
> I feel was a good decision.

[...]

But please don't also miss out a first-hand experience of someone who
implemented a well-visible program centered around Lua: [1], [2].

Personally, I find that minimality (of the runtime) is the only strong
point of Lua. Then it quickly falls apart when you hit its idiocy with
using tables for everything which is ripe with special weird cases.

Lua's stack-based API debunked in [1] also sounds bad to me as I have
decent experience with extending Tcl in C (and embedding it in C), and
I find Tcl's C API to be brilliant as well as its concept of
reference-counted objects. Unfortunately, I have no experience using
Lua from C so I, personally, have no real say here. But still...

1. http://julien.danjou.info/blog/2011/why-not-lua
2. http://julien.danjou.info/blog/2008/rants-about-lua
j. van den hoff
2013-07-23 10:37:39 UTC
Permalink
On Tue, 23 Jul 2013 11:29:32 +0200, Konstantin Khomoutov =

<***@users.sourceforge.net> wrote:

> On Tue, 23 Jul 2013 11:03:09 +0200
> "j. van den hoff" <***@googlemail.com> wrote:
>
> [...]
>
>> >> While the Lua scripting enabled me to gain a level of
>> >> sophistication and relative rigor in the process more than what I
>> >> could get from normal UNIX
>> >> plumbing, if my project wasn=E2=80=99t in Lua in the first place, =
I found
>> >> it breaking my concentration a good deal more than I would have
>> >> liked.
>> >
>> > That's my impression of lua, though i haven't worked with it (it's
>> > too weird, both at the script and C API level).
>>
>> really? regarding the language (the scripting level) I find lua
>> exceptionally well thought out and clear. the syntax is
>> intentionally very "boring" -- no surprises here -- and sure much
>> easier to swallow than tcl, for instance, for people
>> starting from scratch. regarding the C level API I don't have any
>> experience (nor an opinion)
>> but it claims to be much easier than that of other scripting
>> languages. the LaTeX guys at least have decided to
>> move into this direction to get a real scripting facility into latex
>> http://luatex.org/ which personally
>> I feel was a good decision.
>
> [...]
>
> But please don't also miss out a first-hand experience of someone who
> implemented a well-visible program centered around Lua: [1], [2].

I don't know about `awesome' being that "well-visible". in any case the =
=

original `dwm' is much saner and way more stable (or used to be when I =

last looked at `awesome' a year ago or so). otherwise it's just a single=
=

(and sure not majority) view of someone who's personal taste is differen=
t. =

fine with me but sur not sufficient reason for dismissing lua.

> Personally, I find that minimality (of the runtime) is the only strong=

> point of Lua. Then it quickly falls apart when you hit its idiocy wit=
h
> using tables for everything which is ripe with special weird cases.

always talking from the average scripting usage perspective: I don't see=
=

any deficiency (let alone idiocy) here. works well. I don't miss any oth=
er =

data type/container. it's just not fancy, but works all the same. but =

again: I'm not a lua fan boy and ultimately I don't care what the final =
=

choice will be (if there is to be a single choice).

>
> Lua's stack-based API debunked in [1] also sounds bad to me as I have
> decent experience with extending Tcl in C (and embedding it in C), and=

> I find Tcl's C API to be brilliant as well as its concept of
> reference-counted objects. Unfortunately, I have no experience using
> Lua from C so I, personally, have no real say here. But still...

yes, all that might be which would mean implementation (making it work) =
=

might be less fun than it could. more important to me is the question ho=
w =

good the user visible part is working in the end. and here I'm still qui=
te =

sure that people would be more happy with lua than with tcl (if that's t=
he =

alternative), assuming they know neither before starting.

but to reiterate: is it really worth the trouble to add scripting =

capabilities to fossil instead of focusing on more pressing improvements=
=

such as a good grep through old revisions? and also, maybe, just making =
a =

smoother more consistent CLI? fossil sure is the first VCS where I felt =
=

the need to write several small shell scripts (or use others) in order t=
o =

make live easy enough (did not happen to me with svn and mercurial at =

least). and obviously I'm not the only one as the mailing list clearly =

shows...

but i think I will now try to no longer disrupt this thread with such =

defeatism ;-)


>
> 1. http://julien.danjou.info/blog/2011/why-not-lua
> 2. http://julien.danjou.info/blog/2008/rants-about-lua


-- =

Using Opera's revolutionary email client: http://www.opera.com/mail/
Laurens Van Houtven
2013-07-23 10:51:00 UTC
Permalink
Hi,


Not aimed at anyone in particular, but if you are going to suggest a
particular language, can you please point to an interpreter that is easily
embeddable into fossil? That seems to be the problem with at least JS and
Python, and seems to be Lua's strong point. There's no point in discussing
the relative merits of languages if we can't actually reasonably *use that
language*.

cheers
lvh
Stephan Beal
2013-07-23 10:55:41 UTC
Permalink
On Tue, Jul 23, 2013 at 12:51 PM, Laurens Van Houtven <***@lvh.io> wrote:

> Not aimed at anyone in particular, but if you are going to suggest a
> particular language, can you please point to an interpreter that is easily
> embeddable into fossil? That seems to be the problem with at least JS and
> Python, and seems to be Lua's strong point. There's no point in discussing
> the relative merits of languages if we can't actually reasonably *use that
> language*.
>

That's one of the beauties of restructuring fossil as a library: we don't
need to embed any language at all. Instead, they can be built on top of the
library. Yes, we'll want/need one "standard" interpreter for unit test
purposes, but that will "almost certainly" end up being TCL or jimtcl,
simply for reasons of historical momentum.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Laurens Van Houtven
2013-07-23 11:20:21 UTC
Permalink
On Tue, Jul 23, 2013 at 12:55 PM, Stephan Beal <***@googlemail.com>wrote:

> That's one of the beauties of restructuring fossil as a library: we don't
> need to embed any language at all. Instead, they can be built on top of the
> library.
>

Sure thing: but if that happens (which I'm rooting for), arguing about what
language is better is even sillier, since any particular binding existing
doesn't negatively affect any other binding :)

cheers
lvh
Aaron W.Hsu
2013-07-23 17:57:27 UTC
Permalink
Dear Stephan:


With regards to scripting and fossil as a separate library, I actually see these as separate things. Having Fossil as a separate library allows other programs to use fossil capabilities, while any scripting interface, such as a public interface used to build shared objects that are dynamically linked at runtime by the Fossil program, allows Fossil to use the capabilities of other software. These are two different things. You don’t need to separate Fossil as a library in order to get a language agnostic extension interface for Fossil.



Yours truly,


Aaron W. Hsu


--
Aaron W. Hsu | ***@sacrideo.us | http://www.sacrideo.us




From: Stephan Beal
Sent: ‎Tuesday‎, ‎July‎ ‎23‎, ‎2013 ‎6‎:‎55‎ ‎AM
To: Fossil SCM user's discussion


On Tue, Jul 23, 2013 at 12:51 PM, Laurens Van Houtven <***@lvh.io> wrote:






Not aimed at anyone in particular, but if you are going to suggest a particular language, can you please point to an interpreter that is easily embeddable into fossil? That seems to be the problem with at least JS and Python, and seems to be Lua's strong point. There's no point in discussing the relative merits of languages if we can't actually reasonably *use that language*.





That's one of the beauties of restructuring fossil as a library: we don't need to embed any language at all. Instead, they can be built on top of the library. Yes, we'll want/need one "standard" interpreter for unit test purposes, but that will "almost certainly" end up being TCL or jimtcl, simply for reasons of historical momentum.



--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Stephan Beal
2013-07-23 18:20:17 UTC
Permalink
On Tue, Jul 23, 2013 at 7:57 PM, Aaron W.Hsu <***@sacrideo.us> wrote:

> With regards to scripting and fossil as a separate library, I actually see
> these as separate things. Having Fossil as a separate library allows other
> programs to use fossil capabilities, while any scripting interface, such as
> a public interface used to build shared objects that are dynamically linked
> at runtime by the Fossil program, allows Fossil to use the capabilities of
> other software. These are two different things.
>

They are two separate things, but in the case of fossil...


> You don’t need to separate Fossil as a library in order to get a language
> agnostic extension interface for Fossil.
>

You do, though - fossil's core logic and UI are very interwoven and its
error handling mechanism prohibits any usable scripting. For example,
assume the following pseudo-script:

fossil.open("/path/to/my.fsl")
fossil.checkout("my-branch")

if the open() fails (for _any_ reason), fossil exits _fatally_. We do not
get a chance to handle that error and try something else. This makes it
impossible to write a shell-like interface for fossil (that was one of the
first things i tried to extend fossil to do, back in 2008). It also
prohibits any useful scripting unless that scripting language is closely
tied to the fossil internals and can intercept all of the errors (which is
not possible if it uses the internal utility APIs - it would have to
implement all/most of the useful functionality itself). The current
monolithic approach makes reuse next to impossible - the user interaction
and app logic are tightly woven together.

Yes, there has been some confusion about the whole "which scripting
language?" topic. The fact is, once we have a library, anyone can tie any
scripting language to it, so that question simply goes away. Specific
application parts, e.g. the HTML UI will almost certainly select some
scripting language (maybe TH1, maybe TCL, maybe lua) for their parts, but
that doesn't affect others in any way. But there might be more than one
HTML UI, so even then we might end up with multiple choices for scripting
engines. If/when the v2 library starts to become functional i will be
binding it to my own scripting engine, basically as a sanity check for,
"does this API make sense for use with script bindings?" That particular
binding will be for my own use/play, though, and i don't intend to submit
it for inclusion into the core fossil project (because my scripting engine
is a toy).

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Stephan Beal
2013-07-23 18:29:35 UTC
Permalink
On Tue, Jul 23, 2013 at 8:20 PM, Stephan Beal <***@googlemail.com> wrote:

> Yes, there has been some confusion about the whole "which scripting
> language?" topic. The fact is, once we have a library, anyone can tie any
>

amendment... scripting, to me, is important mainly because it provides an
indication of how extensible an application is. If it has one or more
scripting APIs, the chances are very good that it is also suitable for
easily extending from C. So i often say "scripting" when i really mean,
"extensibility."

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Aaron W.Hsu
2013-07-23 18:28:11 UTC
Permalink
Dear Stephan:


Thanks for the response. I think I’m a little confused about what the intended use cases for a scripting language are? When I think of scripting the Fossil SCM I’m envisioning like post and pre-commit hooks, or adding support for a different sort of Wiki format, or integrating a release schedule or engineering methodology (so, perhaps, a change committed which triggers an engineering changes activates and populates a ticket that points to the most recent commit for consideration by another team, such as testing).


I’m not sure how having a separate library would help with this, because the user would still be interacting with the program through the fossil(1) executable. There would not be a separate program that they are running, it’s just that additional things happen in the background when the user starts working with Fossil. In this case, I don’t want to be able to run something like Checkout from a separate program; fossil(1) already lets me do a checkout, I just want to adjust what happens when checking out, or committing, or the like. Yes, I would need programmatic access to do certain things from within Fossil, perhaps. However, in the course of making that access public to the world, that would naturally require some sort of refactoring to ensure that errors and exceptional situations were all dealt with in a way that provides the appropriate levels of safety.


It’s much less interesting to me whether or not we can write separate, external programs that are accessed on their own, which happen to use Fossil as a backend. That’s the case that I see libraries would be good for, but in the previous case I detailed above, I don’t understand how a library would make any use, because the point is for the user to have that code accessed and used during the normal course of using the Fossil executable, not through a separate program.



--
Aaron W. Hsu | ***@sacrideo.us | http://www.sacrideo.us




From: Stephan Beal
Sent: ‎Tuesday‎, ‎July‎ ‎23‎, ‎2013 ‎2‎:‎20‎ ‎PM
To: Fossil SCM user's discussion





You do, though - fossil's core logic and UI are very interwoven and its error handling mechanism prohibits any usable scripting. For example, assume the following pseudo-script:




fossil.open("/path/to/my.fsl")

fossil.checkout("my-branch")




if the open() fails (for _any_ reason), fossil exits _fatally_. We do not get a chance to handle that error and try something else. This makes it impossible to write a shell-like interface for fossil (that was one of the first things i tried to extend fossil to do, back in 2008). It also prohibits any useful scripting unless that scripting language is closely tied to the fossil internals and can intercept all of the errors (which is not possible if it uses the internal utility APIs - it would have to implement all/most of the useful functionality itself). The current monolithic approach makes reuse next to impossible - the user interaction and app logic are tightly woven together.




Yes, there has been some confusion about the whole "which scripting language?" topic. The fact is, once we have a library, anyone can tie any scripting language to it, so that question simply goes away. Specific application parts, e.g. the HTML UI will almost certainly select some scripting language (maybe TH1, maybe TCL, maybe lua) for their parts, but that doesn't affect others in any way. But there might be more than one HTML UI, so even then we might end up with multiple choices for scripting engines. If/when the v2 library starts to become functional i will be binding it to my own scripting engine, basically as a sanity check for, "does this API make sense for use with script bindings?" That particular binding will be for my own use/play, though, and i don't intend to submit it for inclusion into the core fossil project (because my scripting engine is a toy).



--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Stephan Beal
2013-07-23 18:58:34 UTC
Permalink
On Tue, Jul 23, 2013 at 8:28 PM, Aaron W.Hsu <***@sacrideo.us> wrote:

> Thanks for the response. I think I’m a little confused about what the
> intended use cases for a scripting language are?
>

Scriptability, more than anything, is simply a benchmark which says, "this
app is extensible." Personally i'm not so much interested in _how_ people
extend it, just that they _can_. In my experience, adding a scripting API
to a C/C++ library can drastically improve its development by allowing the
devs to write more code (test cases) more quickly, and try out new ideas
more easily than writing them in C.



> When I think of scripting the Fossil SCM I’m envisioning like post and
> pre-commit hooks, or adding support for a different sort of Wiki format, or
> integrating a release schedule or engineering methodology (so, perhaps, a
> change committed which triggers an engineering changes activates and
> populates a ticket that points to the most recent commit for consideration
> by another team, such as testing).
>

Okay, imagine this:

http://fossil.wanderinghorse.net/wikis/cson/?page=cson

that site is actually a fossil repo, but i have completely replaced the UI
with one written in HTML5, CSS, and JavaScript. That is possible because it
uses the JSON API (which is, in effect, a networked library interface for
fossil). Scriptability is very similar, and with it i gain the ability to
write custom UIs like that in a language of my choosing. Special-purpose
apps, e.g. a ticket polling widget, are a good use case for scripting.


> I’m not sure how having a separate library would help with this, because
> the user would still be interacting with the program through the fossil(1)
> executable. There would not be a separate program that they are running,
> it’s just that
>

"The user" is an important term here. "The user" as we currently now know
"the user," yes, will still be interacting with fossil(1). However, if i
have access to a library which allows me to embed version control inside an
arbitrary app, then i am going to use it to do things my apps cannot
currently do. i.e. i want to use fossil(3) outside of fossil(1).

what happens when checking out, or committing, or the like. Yes, I would
> need programmatic access to do certain things from within Fossil, perhaps.
>

Custom reports are another case where scripting simplifies things.


> However, in the course of making that access public to the world, that
> would naturally require some sort of refactoring to ensure that errors and
> exceptional situations were all dealt with in a way that provides the
> appropriate levels of safety.
>

There are no intentions to break how people use fossil(1), in particular no
plans to change how the current version (long may it live) works in any
way. Any "librification" of fossil would/will be done under the umbrella of
"version 2," and i think people expect "some" level of change between v1
and v2 of just about any program. The goal here is not to obviate
fossil(1), per se, just taking it up a step on the evolutionary scale. i
think it goes without saying that even if v2 appeared today, v1 would still
be in widespread use for years to come.

It’s much less interesting to me whether or not we can write separate,
> external programs that are accessed on their own, which happen to use
> Fossil as a backend.
>

But it is to me :-D

That’s the case that I see libraries would be good for, but in the previous
> case I detailed above, I don’t understand how a library would make any use,
> because the point is for the user to have that code accessed and used
> during the normal course of using the Fossil executable, not through a
> separate program.
>

For 98% of users/people there will be zero interest in the underlying
library (they don't need to know it exists). For the other 2% of us,
though, there are all kinds of things we could do with such a tool. Adding
versioning to essentially any data in any program, complete with merging,
diffing, delta compression, etc. AND the stability of an sqlite data store.
That's a dream come true for some of us :).

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Aaron W.Hsu
2013-07-23 19:07:02 UTC
Permalink
Dear Stephan:


Thanks for your explanation. I, of course, agree that there are plenty of interesting benefits that one gains from having a fossil(3) in addition to a fossil(1). On the other hand, my initial email and my comments have nothing to do with a fossil(3) interface. What I’ve been talking about is how fossil, in the future, might choose to extend fossil’s behavior itself, *not* how other programs might have access to fossil capabilities. These are two quite different things. I think it is helpful to distinguish these use cases and the technical solutions that present themselves for these cases. Everything you’ve mentioned in your previous email related directly to how one might have access to fossil within another program. That is certainly useful and desirable, but what I’ve been talking about is how people programmatically alter the behavior of the fossil program, not how they use fossil within other programs. When working in the use case you have mentioned, most people tend to follow the pattern of designing a library; this library tends to be accessible through any language that has good intra-language support. However, in the use case that I am discussing, IMO, people usually choose either to embed a very specific scripting language into the program to allow its behavior to be extended, or they go with some sort of UNIX style plumbing.


The point of my message is to suggest that Fossil might strongly consider not tying itself down to a specific scripting language in order to extend fossil itself (that is, the use case that has nothing to do with accessing fossil’s capabilities from other programs or interfaces), and to not follow others in using the UNIX plumbing approach, but instead to create an equally robust API that allows one to extend the behavior of fossil(1) using any programming language that they want, simply by providing a new shared object that uses the API. This is something completely different than what you’ve been talking about, and it’s something that very few people do when writing extension interfaces. For instance, Emacs is extended using Elisp, and that’s pretty much it. Monotone is in Lua. Blender is in Python. Some software does do a good job of it, and it’s always a welcome change from the status quo. When you have a well designed interface that can be accessed and used by your languages of choice, no matter what they are (reasonably speaking), then you can integrate fossil into your development workflow much easier.


So, again, splitting fossil into a library and front end has no bearing on this particular problem, because the technical question here is not how to use fossil from other programs, but how fossil uses other code (ideally written in any language) within itself. It does no good to have a library if every time you want to extend the behavior of the fossil program you have to write your own version of the fossil front-end.



Yours truly,


Aaron W. Hsu



--
Aaron W. Hsu | ***@sacrideo.us | http://www.sacrideo.us




From: Stephan Beal
Sent: ‎Tuesday‎, ‎July‎ ‎23‎, ‎2013 ‎2‎:‎58‎ ‎PM
To: Fossil SCM user's discussion


On Tue, Jul 23, 2013 at 8:28 PM, Aaron W.Hsu <***@sacrideo.us> wrote:






Thanks for the response. I think I’m a little confused about what the intended use cases for a scripting language are?




Scriptability, more than anything, is simply a benchmark which says, "this app is extensible." Personally i'm not so much interested in _how_ people extend it, just that they _can_. In my experience, adding a scripting API to a C/C++ library can drastically improve its development by allowing the devs to write more code (test cases) more quickly, and try out new ideas more easily than writing them in C.









When I think of scripting the Fossil SCM I’m envisioning like post and pre-commit hooks, or adding support for a different sort of Wiki format, or integrating a release schedule or engineering methodology (so, perhaps, a change committed which triggers an engineering changes activates and populates a ticket that points to the most recent commit for consideration by another team, such as testing).




Okay, imagine this:




http://fossil.wanderinghorse.net/wikis/cson/?page=cson





that site is actually a fossil repo, but i have completely replaced the UI with one written in HTML5, CSS, and JavaScript. That is possible because it uses the JSON API (which is, in effect, a networked library interface for fossil). Scriptability is very similar, and with it i gain the ability to write custom UIs like that in a language of my choosing. Special-purpose apps, e.g. a ticket polling widget, are a good use case for scripting.






I’m not sure how having a separate library would help with this, because the user would still be interacting with the program through the fossil(1) executable. There would not be a separate program that they are running, it’s just that




"The user" is an important term here. "The user" as we currently now know "the user," yes, will still be interacting with fossil(1). However, if i have access to a library which allows me to embed version control inside an arbitrary app, then i am going to use it to do things my apps cannot currently do. i.e. i want to use fossil(3) outside of fossil(1).







what happens when checking out, or committing, or the like. Yes, I would need programmatic access to do certain things from within Fossil, perhaps.




Custom reports are another case where scripting simplifies things.






However, in the course of making that access public to the world, that would naturally require some sort of refactoring to ensure that errors and exceptional situations were all dealt with in a way that provides the appropriate levels of safety.




There are no intentions to break how people use fossil(1), in particular no plans to change how the current version (long may it live) works in any way. Any "librification" of fossil would/will be done under the umbrella of "version 2," and i think people expect "some" level of change between v1 and v2 of just about any program. The goal here is not to obviate fossil(1), per se, just taking it up a step on the evolutionary scale. i think it goes without saying that even if v2 appeared today, v1 would still be in widespread use for years to come.







It’s much less interesting to me whether or not we can write separate, external programs that are accessed on their own, which happen to use Fossil as a backend.




But it is to me :-D







That’s the case that I see libraries would be good for, but in the previous case I detailed above, I don’t understand how a library would make any use, because the point is for the user to have that code accessed and used during the normal course of using the Fossil executable, not through a separate program.




For 98% of users/people there will be zero interest in the underlying library (they don't need to know it exists). For the other 2% of us, though, there are all kinds of things we could do with such a tool. Adding versioning to essentially any data in any program, complete with merging, diffing, delta compression, etc. AND the stability of an sqlite data store. That's a dream come true for some of us :).



--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Stephan Beal
2013-07-23 19:56:05 UTC
Permalink
On Tue, Jul 23, 2013 at 9:07 PM, Aaron W.Hsu <***@sacrideo.us> wrote:

> distinguish these use cases and the technical solutions that present
> themselves for these cases. Everything you’ve mentioned in your previous
> email related directly to how one might have access to fossil within
> another program. That is certainly useful and desirable, but what I’ve been
> talking about is how people programmatically alter the behavior of the
> fossil program, not how they use fossil within other programs.
>

The two are closely related. If the API is sane, it can be used for both
purposes.

When working in the use case you have mentioned, most people tend to follow
> the pattern of designing a library; this library tends to be accessible
> through any language that has good intra-language support. However, in the
> use case that I am discussing, IMO, people usually choose either to embed a
> very specific scripting language into the program to allow its behavior to
> be extended, or they go with some sort of UNIX style plumbing.
>

Fossil has th1 as a language, but it's not scalable to the things i'd like
to do with it. It cannot report error locations, for example, making
debugging problematic.


>
> The point of my message is to suggest that Fossil might strongly consider
> not tying itself down to a specific scripting language in order to extend
> fossil itself (that is, the use case that has nothing to do with accessing
> fossil’s capabilities from other programs or interfaces), and to not follow
> others in using the UNIX plumbing approach, but instead to create an
> equally robust API that allows one to extend the behavior of fossil(1)
> using any programming language that they want, simply by providing a new
> shared object that uses the API.
>

That's essentially my goal with fossil(3). Scriptability would be a natural
side effect a robust/portable/flexible API, and is an important
consideration in the APIs design, but scriptability itself is not the core
goal - it's just bonus points.

Side note: For those who don't know what this fossil(1) and fossil(3)
business is - it's "man(1) notation", referring to sections 1
(applications) and 3 (libraries) of the "man page" documentation on Unix
platforms. Google "man strcmp" and you'll see what i mean. It's not
referring to fossil v1 and v3.



> This is something completely different than what you’ve been talking
> about, and it’s something that very few people do when writing extension
> interfaces.
>

To me they are either the same or very closely related.


> For instance, Emacs is extended using Elisp, and that’s pretty much it.
> Monotone is in Lua. Blender is in Python. Some software does do a good job
> of it, and it’s always a welcome change from the status quo. When you have
> a well designed interface that can be accessed and used by your languages
> of choice, no matter what they are (reasonably speaking), then you can
> integrate fossil into your development workflow much easier.
>

i think you and i are talking about the same things, just from different
angles :).


> So, again, splitting fossil into a library and front end has no bearing on
> this particular problem, because the technical question here is not how to
> use fossil from other programs, but how fossil uses other code (ideally
> written in any language) within itself. It does no good to have a library
> if every time you want to extend the behavior of the fossil program you
> have to write your own version of the fossil front-end.
>

That's the point where we differ. To me the API is the core/central
concept, and everything else just kind of cascades out from there - natural
side effects of having a flexible tool. fossil(1) is just one use case of
fossil(3) for me. fossil(1) _can_ also be implemented as a library with a
thin app shell on top of it, making it a basis for other applications
(which is what i understand you would like to see?). e.g. hypothetically
fossil(1) could provide Wordpress-like features to allow people to plug in
their own site layouts, everything else. It could provide a server-side
scripting language and an AJAX interface. Or maybe it provides slightly
lower-level components which could be used with UI toolkits (e.g. some
abstraction over the timeline data, which a UI app might export directly
into to Excel using ODBC or COM, or whatever the current IPC model is on
Windows). With fossil(3) in place, we can add any number of abstraction
levels to integrate various apps in different ways. Fundamentally i think
we want the exact same things, i'm just seeing the details and eventual use
cases from a different perspective (from the library user level, primarily)
than you.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Aaron W.Hsu
2013-07-23 23:55:31 UTC
Permalink
Dear Stephan:


Here’s my attempt to define and clearly distinguish the various parts of this discussion into a piece that might be useful for talking about these things in the future. In my opinion, there are a few mutually orthogonal things to think about:

The fossil API itself, which is just the abstract interface for speaking about Fossil operations in some fashion, independent of how this API is realized;
The fossil(1) program and interface itself, which is the current interface seen by users, which happens to also contain other servers for hosting other interfaces -- I distinguish here the Fossil interface from the Web interface;
The Web Interface, which is currently launched by various calls to the fossil(1) interface;
The “extension” API, which is to say, the API allowing one to alter the observable behavior of the fossil(1) program; and finally,
The fossil(3) library, which is an instantiated implementation of the fossil API.


I think the important first note is that each of the above five things are technically independent of one another. One need not have a realized fossil API in order to have an extension API; if you do not, you will not have programmatic access to fossil operations when writing extensions, but you might still do interesting things. Likewise, one need not have a literal fossil(3) library to provide access to the fossil API. It is perfectly possible for Fossil to dynamically, at runtime, load user-written shared objects which access this functionality without needed these shared objects to explicitly link at link time against a fossil(3) library. This functionality can therefore be provided by the fossil(1) executable program as well as from a separate shared object against which the fossil(1) program links. On the other hand, not having the fossil(3) library may preclude certain use cases, such as when one wishes to use the fossil operations without ever interfacing or touching the fossil(1) program, which is something that you have provided a number of use cases for.


Once we recognize that these are all technically orthogonal, we can then try to understand how they synergize together. For instance, it makes sense that the fossil API that is accessible through shared objects loaded by fossil(1) at runtime and the API provided by fossil(3) be the same. On the other hand, it doesn’t make much sense for the extension API to be a part of fossil(3).


After all this, we come to the question of what the best implementation of each of the above actually is. Here are a few of those questions that have come up:

Should fossil(3) exist at all?
What is the extension API?
What is the fossil API?


Notice that I’ve explicitly created a difference between the extension API and the fossil API. I think this is not only important, but at the crux of our particular current line of discourse. In particular, you’ve mentioned a number of times that having a fossil API, such as what fossil(3) might provide, sort of automatically entails extensibility and scriptability. I disagree with the usage of those terms here. I think one should distinguish clearly between the extension API and the fossil API. There is no extension API in fossil(3). Namely, you cannot change the way that fossil operations work. Instead, you are expected to have access to these operations as primitives for writing other programs, but by and large you are not altering the behavior of fossil itself. An extension API is not for providing fossil operations to the rest of the programming space, but is instead for the sole purpose of allowing one to alter the behavior of the fossil operations themselves.


Based on your prior messages, I think it is clear that you are talking about the fossil API, and consider that to be the really sticky issue. I have no doubt that the fossil API itself is a very important and tricky issue. However, I want to elevate the extension API question to the fore here. What’s interesting and relevant here is not the fossil API, but the extension API. And here I want to emphasize that the fossil API and the extension API can both reasonably and technically exist *without* ever having to create a shared fossil(3) library. Whether one should do that or not is a different question, and important, but not the question I want to focus on.


Let me focus in on the specific question of the extension API to clarify it, because I think it’s a very important design choice that doesn’t receive enough attention. To lift out this question, let’s assume that we still only have a static fossil executable, and no shared object. Let us further assume that the fossil API already exists, and is implemented in that fossil executable. Now, there is the question of the extension API. How do current systems allow for the extending of their VCS?

Some use specific “hook” points that call external programs to do work;
Some embed a specific scripting language.


I’ve written in my first email in this thread about the various pros and cons of these two approaches. The important thing is that both of these is, essentially, an extension API. In the first case, one might access the fossil API through the fossil(1) program interface, which might provide explicit flags for producing easily parsed textual output. In the second case, the fossil API might usually be embedded as a part of the scripting language’s primitive forms, together with some sort of hooking interface or implicit parameters such as callbacks that might fire at the appropriate times during fossil operations, providing the means of altering the behavior of those operations.


I think both of these approached should be examined skeptically. The first, UNIX plumbing approach is too fragile. The second is too rigid, in that it privileges a single language above others in the system. Instead, I am recommending that the extension API actually be a standard C API that we can create shared objects against. These shared objects might be called plugins, if we wanted to call them that. They can, of course, use whatever language that they want. Then, the fossil(1) program (which, recall, in this scenario, is still just a statically linked binary) can load them dynamically at runtime and alter its behavior accordingly. This is a more robust, but also more flexible solution, and would set fossil apart in terms of extensibility.


in summary, based on the above definitions, I would urge fossil v2 to take serious design consideration over the question not only of a fossil API, but also, and perhaps more so, on the question of the extension API, whether one should exist, and whether or not it should follow traditional practices in this area, as noted above, or whether, instead, it might not distinguish itself with a more flexible, language agnostic, but reliable C-style API. I would also encourage those discussing these topics to clearly distinguish all five of the above design spaces, and not to conflate one with the other, recognizing them as completely orthogonal, though synergistic, to one another.



Yours truly,


Aaron W. Hsu


--
Aaron W. Hsu | ***@sacrideo.us | http://www.sacrideo.us
Stephan Beal
2013-07-24 01:42:40 UTC
Permalink
On Wed, Jul 24, 2013 at 1:55 AM, Aaron W.Hsu <***@sacrideo.us> wrote:

> against a fossil(3) library. This functionality can therefore be provided
> by the fossil(1) executable program as well as from a separate shared
> object against which the fossil(1) program links.
>

Right - that's just a question of how the modules/extensions are linked.
That bit only affects how they are loaded/initialized, but not how they
operate.

Once we recognize that these are all technically orthogonal, we can then
> try to understand how they synergize together. For instance, it makes sense
> that the fossil API that is accessible through shared objects loaded by
> fossil(1) at runtime and the API provided by fossil(3) be the same.
>

i hadn't ever thought explicitly about them being available via fossil(1),
probably because the matter of where the APIs are linked to/from doesn't
change the fundamental design (which is where the work/energy is going at
the moment).


> On the other hand, it doesn’t make much sense for the extension API to be
> a part of fossil(3).
>

Not necessarily. Where exactly those bits will/should/could live isn't a
problem i've considered much yet. i'm at the point where i can instantiate
and destroy a fossil instance and create formatted output to a few
different output channels, but not yet at a point where anything really
interesting is happening, e.g. opening an existing repo DB will be the next
step.


> 1. Should fossil(3) exist at all?
>
>
That is a fair question. Despite my overall enthusiasm and hype regarding
fossil(3), i do have concerns about whether it is truly going to work.
Richard has also expressed skepticism - you're not alone! i do, however,
think that it's an interesting enough problem to be worth trying out. If it
turns out that it's not reasonable or doesn't fit, it'll get tossed aside.
At this point there is _no_ commitment that fossil(3) will/should/must
happen.


>
> 1. What is the extension API?
>
>
(Sorry, gmail is renumbering your entries when i split them.)

i'm not yet that far along, but i expect to have some interesting
discussions on that topic later.


>
> 1.
> 2. What is the fossil API?
>
>
> Notice that I’ve explicitly created a difference between the extension API
> and the fossil API. I think this is not only important, but at the crux of
> our particular current line of discourse. In particular, you’ve mentioned a
> number of times that having a fossil API, such as what fossil(3) might
> provide, sort of automatically entails extensibility and scriptability. I
> disagree with the usage of those terms here. I think one should distinguish
> clearly between the extension API and the fossil API. There is no extension
> API in fossil(3).
>

i hadn't thought about those being separate until your post. i'll have to
ponder why that separation is significant. i haven't yet gotten to the
level of detail where i can just point to the function and say, "that does
or does not fit."



> Namely, you cannot change the way that fossil operations work. Instead,
> you are expected to have access to these operations as primitives for
> writing other programs, but by and large you are not altering the behavior
> of fossil itself.
>

Correct.


> An extension API is not for providing fossil operations to the rest of the
> programming space, but is instead for the sole purpose of allowing one to
> alter the behavior of the fossil operations themselves.
>

Yes, and building off of them to provide bigger or more special-purpose
features. e.g. specialized timeline reports or exports to spreadsheets.



> Based on your prior messages, I think it is clear that you are talking
> about the fossil API, and consider that to be the really sticky issue.
>

Right.


> I have no doubt that the fossil API itself is a very important and tricky
> issue. However, I want to elevate the extension API question to the fore
> here. What’s interesting and relevant here is not the fossil API, but the
> extension API. And here I want to emphasize that the fossil API and the
> extension API can both reasonably and technically exist *without* ever
> having to create a shared fossil(3) library. Whether one should do that or
> not is a different question, and important, but not the question I want to
> focus on.
>

That's an interesting direction. i would at this point argue that until we
know what the core fossil lib really should look like, that we can't say
what will be possible with the extension API. That said, we do need to know
what the extension API _should_ be able to do in order to design the core
API to do it,. So from that angle, the extension API is really the bigger
factor. We've seen, the past couple days, a huge amount of input on what
people want it to do, so i think we have a lot of good data for figuring
out what the core API needs to look like. There's still lots of thinking,
experimenting, and hacking left here, and nothing is ruled out at this
point (except maybe an Oracle back-end - i won't go quite that far ;).


> Let me focus in on the specific question of the extension API to clarify
> it, because I think it’s a very important design choice that doesn’t
> receive enough attention.
>

Not until you came along and pointed out the distinction.


> To lift out this question, let’s assume that we still only have a static
> fossil executable, and no shared object.
>

In fact, i expect that will be the default/majority installation.


> Let us further assume that the fossil API already exists, and is
> implemented in that fossil executable. Now, there is the ...case, the
> fossil API might usually be embedded as a part of the scripting language’s
> primitive forms, together with some sort of hooking interface or implicit
> parameters such as callbacks that might fire at the appropriate times
> during fossil operations, providing the means of altering the behavior of
> those operations.
>

How best to implement those probably depends on the specific behaviours the
triggers want to accomplish. My current thinking is an event model which
client code (==not fossil(3)) registers callbacks with, but this is an area
others have spent much more time contemplating, and there is also much
discussion (and bikeshedding ;) to be done on this topic. If you are not
already on the dev list, i would ask you to subscribe to it (fossil-dev),
as i'd prefer to discuss this level of detail over there - most user list
members have probably filtered out any posts from me by now due to volume
:/.

I think both of these approached should be examined skeptically. The first,
> UNIX plumbing approach is too fragile. The second is too rigid, in that it
> privileges a single language above others in the system. Instead, I am
> recommending that the extension API actually be a standard C API that we
> can create shared objects against. These shared objects might be called
> plugins, if we wanted to call them that. They can, of course, use whatever
> language that they want. Then, the fossil(1) program (which, recall, in
> this scenario, is still just a statically linked binary) can load them
> dynamically at runtime and alter its behavior accordingly. This is a more
> robust, but also more flexible solution, and would set fossil apart in
> terms of extensibility.
>

Agreed on every point. To me the extensibility of fossil(3) is the primary
factor, so long as it's not painfully difficult to use properly (ever
worked with the MySQL C API?). Ease of use will play a larger factor in
fossil(1), however, since ease of use is one of fossil's advantages (IMO).


> in summary, based on the above definitions, I would urge fossil v2 to take
> serious design consideration over the question not only of a fossil API,
> but also, and perhaps more so, on the question of the extension API,
> whether one should exist, and whether or not it should follow traditional
> practices in this area, as noted above, or whether, instead, it might not
> distinguish itself with a more flexible, language agnostic, but reliable
> C-style API.
>

Do you have a concrete suggestion for a 3rd (or 4th) alternative? If so, i
would really like to talk about that further on the dev list. i am not
comfortable with either the plumbing or single-script-language approaches,
and can't imagine implementing either one of them.


> I would also encourage those discussing these topics to clearly
> distinguish all five of the above design spaces, and not to conflate one
> with the other, recognizing them as completely orthogonal, though
> synergistic, to one another.
>

You have given me a great deal to introspect upon in these wee hours of the
morning (3:30). i very, very much appreciate your detailed discourse, and
i'm hoping pick your brain more about your ideas for
events/triggers/plugins/etc.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
B Harder
2013-07-23 23:32:54 UTC
Permalink
I think these things support each other. Specifically, with respect to Tcl
(and likely Guile, Lua), scripting support can come by embedding the Tcl
interpreter inside the app (ie: the standalone fossil executable), by
linking libtcl with fossil, and having fossil call into it appropriately.

Alternatively, one can take a library (ie: hypothetical libfossil), and
build bindings to it which satisfy loading those bindings/library into Tcl.
In this case, a Tcl script (or person at the keyboard) does the driving,
calling into fossil routines appropriately.

The first example would be "Fossil with hooks", while the second would be
"full scripting environment drives the operation, with full access to
fossil routines".

I'm personally more interested in the second. To my senses, it's more
dynamic, and I'd say lends itself to more rapid prototyping/testing,
accelerating the development of the fossil core itself. I think in terms of
Tcl, but this second approach also facilitates other languages
participating more than the first model does. Ie: if Perl wants to
participate, they write bindings against libfossil. If Ruby wants to
participate, they write bindings against libfossil, etc. In the first
model, to support multiple languages, you must necessarily fork fossil and
build out internal support within the fossil executable for Perl, in
another for support for Ruby, in another fork, support for Python, etc.

Does that make sense, or am I missing something?

-bch
On Jul 23, 2013 11:01 AM, "Aaron W.Hsu" <***@sacrideo.us> wrote:

> Dear Stephan:
>
> With regards to scripting and fossil as a separate library, I actually see
> these as separate things. Having Fossil as a separate library allows other
> programs to use fossil capabilities, while any scripting interface, such as
> a public interface used to build shared objects that are dynamically linked
> at runtime by the Fossil program, allows Fossil to use the capabilities of
> other software. These are two different things. You don’t need to separate
> Fossil as a library in order to get a language agnostic extension interface
> for Fossil.
>
>
> Yours truly,
>
> Aaron W. Hsu
>
>
> --
> Aaron W. Hsu | ***@sacrideo.us | http://www.sacrideo.us
>
> *From:* Stephan Beal
> *Sent:* Tuesday, July 23, 2013 6:55 AM
> *To:* Fossil SCM user's discussion
>
> On Tue, Jul 23, 2013 at 12:51 PM, Laurens Van Houtven <***@lvh.io> wrote:
>
>> Not aimed at anyone in particular, but if you are going to suggest a
>> particular language, can you please point to an interpreter that is easily
>> embeddable into fossil? That seems to be the problem with at least JS and
>> Python, and seems to be Lua's strong point. There's no point in discussing
>> the relative merits of languages if we can't actually reasonably *use that
>> language*.
>>
>
> That's one of the beauties of restructuring fossil as a library: we don't
> need to embed any language at all. Instead, they can be built on top of the
> library. Yes, we'll want/need one "standard" interpreter for unit test
> purposes, but that will "almost certainly" end up being TCL or jimtcl,
> simply for reasons of historical momentum.
>
> --
> ----- stephan beal
> http://wanderinghorse.net/home/stephan/
> http://gplus.to/sgbeal
>
> _______________________________________________
> fossil-users mailing list
> fossil-***@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
>
Stephan Beal
2013-07-24 01:16:39 UTC
Permalink
On Wed, Jul 24, 2013 at 1:32 AM, B Harder <***@gmail.com> wrote:

> The first example would be "Fossil with hooks", while the second would be
> "full scripting environment drives the operation, with full access to
> fossil routines".
>
> I'm personally more interested in the second.To my senses, it's more
> dynamic, and I'd say lends itself to more rapid prototyping/testing,
> accelerating the development of the fossil core itself.
>
That might sound counter-intuitive to those who haven't experienced it for
themselves, but i will second it.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Leo Razoumov
2013-07-23 21:42:15 UTC
Permalink
On Tue, Jul 23, 2013 at 5:29 AM, Konstantin Khomoutov
<***@users.sourceforge.net> wrote:
> [...]
>
> But please don't also miss out a first-hand experience of someone who
> implemented a well-visible program centered around Lua: [1], [2].
>
> Personally, I find that minimality (of the runtime) is the only strong
> point of Lua. Then it quickly falls apart when you hit its idiocy with
> using tables for everything which is ripe with special weird cases.
>
> Lua's stack-based API debunked in [1] also sounds bad to me as I have
> decent experience with extending Tcl in C (and embedding it in C), and
> I find Tcl's C API to be brilliant as well as its concept of
> reference-counted objects. Unfortunately, I have no experience using
> Lua from C so I, personally, have no real say here. But still...
>
> 1. http://julien.danjou.info/blog/2011/why-not-lua
> 2. http://julien.danjou.info/blog/2008/rants-about-lua
>

Well, everyone is entitled to her/his opinion. Hence, here is mine.
I have been using Lua for the last 6 years over several projects from small
to moderatly large (as in 30K Lua + 20K in C++) and it has very invigorating
and positive experience. Somewhat repetative nature of stack based C-API
can be substantially simplified by means of helper functions.

Moreover, using LuaJIT with its own FFI facility has been a pure joy.
Added bonus being that LuaJIT is the fastest
JIT compiler for any scripting language out there (beats JS V8 handily).

In any case give Lua or/and LuaJIT a try and form your own opinion.

Just my two cents.
--Leo--
Ron Wilson
2013-07-23 22:12:18 UTC
Permalink
On Tue, Jul 23, 2013 at 4:29 AM, Stephan Beal <***@googlemail.com> wrote:

> On Tue, Jul 23, 2013 at 3:25 AM, Aaron W.Hsu <***@sacrideo.us> wrote:
>
>> When fossil runs, it can dynamically link in any of the shared objects
>> that it finds there, and these can register additional functionality with
>> the Fossil system.
>>
>
> And we're very much on the same page. Note, however, that people really
> like having fossil as a static binary, so that will likely be the "tier 1
> target."
>

Seems to me it would act like a static binary unless requested to load one
or more shared libraries, only calling whatever hook "services" are
registered by the shared libraries. If no libraries are loaded, then the
binary would behave as if it were static (that is, self-contained with no
required dependencies).

As an example, here is some pseudo code:

Hook_XXX( hookData )
{
if (registered(hook_XXX_service))
{
return hook_XXX_service( hookData );
}
return SUCCESS;
}

For a "notification only" hook, it could be like:

Hook_NNN( hookData )
{
if (registered(hook_NNN_service))
{
return spawn_no_wait(hook_NNN_service( hookData ));
}
return SUCCESS;
}

Baring a near complete restructuring, I think this would be the most
flexible way to add extensions to Fossil and, theoretically, could be
implemented in the current Fossil.
Stephan Beal
2013-07-24 01:12:52 UTC
Permalink
On Wed, Jul 24, 2013 at 12:12 AM, Ron Wilson <***@gmail.com> wrote:

> Seems to me it would act like a static binary unless requested to load one
> or more shared libraries, only calling whatever hook "services" are
> registered by the shared libraries. If no libraries are loaded, then the
> binary would behave as if it were static (that is, self-contained with no
> required dependencies).
>

Absolutely. One of my goals is to be able to build modules either
statically compiled in or as DLLs. Basically a classloader which is
ignorant of the linkage of the bits its loading. i've done that in C++
before, but doing it in C will be new for me.

For a "notification only" hook, it could be like:
>

i've been pondering the ideas of events, similar to HTML DOM events, so
you'd register a callback which abstractly looks like:

int callback( Fossil * f, event_type_t eventId, void * eventState )

the eventId would determine the type of state passed to the event. i don't
know if this is reasonable/feasible yet, but i _think_ that would allow us
to easily plug in (from client code) post-commit, pre-commit, etc. events
which the client (i.e. app) then uses to trigger whatever he wants.


> return spawn_no_wait(hook_NNN_service( hookData ));
>

Can we do that one cross-platform? Then again, add-ons don't necessarily
need to be platform-specific.


> Baring a near complete restructuring, I think this would be the most
> flexible way to add extensions to Fossil and, theoretically, could be
> implemented in the current Fossil.
>

i'd like to discuss that further and get your ideas on how
hooks/triggers/etc can/should be done. Can we move this one to the dev
list? (i've spammed the -user list enough this week!)


--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Arnel Legaspi
2013-07-22 10:21:13 UTC
Permalink
On 7/22/2013 4:29 PM, fossil-users-***@lists.fossil-scm.org wrote:
> Date: Mon, 22 Jul 2013 10:02:47 +0200
> From: Stephan Beal <***@googlemail.com>
> To: "Fossil SCM user's discussion" <fossil-***@lists.fossil-scm.org>
> Subject: Re: [fossil-users] Random thoughts on Fossil v2
> Message-ID:
> <CAKd4nAh3BXxcg0pbk5W3+***@mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi, Clark! A brief response from the office, and a longer one tonight when
> i get home...
>
> On Mon, Jul 22, 2013 at 2:01 AM, Clark Christensen <***@yahoo.com>wrote:
>> Scripting language: I understand the Tcl roots, and I hope you would
>> consider Javascript as a target. JS seems more universal these days.
>> Maybe the Tcl guys would disagree. I have no idea what that means from an
>> implementation standpoint.
> The problem is the interpreter. i am not aware of a small embedable JS
> interpreter. SpiderMonkey/Jaegermonkey are complex and poorly documented.
> Google V8 is nice but (A) huge, (B) C++, and (C) they recently made drastic
> API changes which invalidated every single v8-using client out there
> (breaking 4-5 years of accumulated code of mine, and i'll never have the
> time to fix it), which means that very few people outside of Google can
> actually use v8 at the moment. JS _would_ also be my first choice, if we
> only had a small, well-maintained interpreter.i don't know TCL, but the TCL
> and Fossil communities seem to be cosmically bound to one another. There
> are relatively few well-established small/embeddable interpreters. Lua,
> TCL, ... none others come to mind (which are small).

Would using something similar to Vim's mechanism (allowing Fossil to be
compiled with Lua/Ruby/MZScheme/Python/etc. support) be more acceptable?

Of course, Vim does have its own scripting language (Vimscript) but this
"language support" is available to users via the Makefiles.

My own humble requests for Fossil v2 include:

- use of Markdown formatting in tickets (unless, this is already possible?)

- +1 for full text search in tickets...and maybe embedded wiki files if
possible

- support for kernel-style commit messages:
For example, you have a commit message like this:

Capitalized, short summary line

<Detailed explanatory text which can be long, preferably wrapped to
around 72 characters or less>

Fossil would then show the summary line in the Timeline view, both in
the web GUI and commandline. Clicking the specific commit would then
display both the summary line and the explanatory text.

(I tried altering the Timeline view CSS for this from a few threads
back, but it made the Timeline page graph ugly.)

Thanks, Stephan!

--Arnel
Laurens Van Houtven
2013-07-22 10:24:08 UTC
Permalink
+1 for a more common markup language (e.g. markdown) :)
Stephan Beal
2013-07-22 10:28:58 UTC
Permalink
On Mon, Jul 22, 2013 at 12:21 PM, Arnel Legaspi <***@gmail.com>wrote:

> Would using something similar to Vim's mechanism (allowing Fossil to be
> compiled with Lua/Ruby/MZScheme/Python/etc. support) be more acceptable?
>

If the core library has a sane interface, there's no reason we can't have
ALL of those bindings - they just need to be implemented by someone. For
example, as soon as fossil has a library API, i will certainly write a
binding for it for my own personal scripting language (not because the
world needs it, but because that's the type of thing i do in my free time),
and being able to implement such bindings will be a point i focus on when
designing any fossil library interface.


> - use of Markdown formatting in tickets (unless, this is already possible?)
>

It might make sense long-term to combine the wiki/ticket editing features,
or replace the wiki with the embedded docs system. There is no reason why
we need separate editing systems (wiki vs tickets). Being able to attach a
Content Type to a wiki page is on the todo list, so that people can flag
their pages as being for a specific parser.


> - +1 for full text search in tickets...and maybe embedded wiki files if
> possible
>

It's more difficult than it sounds, unfortunately. My current thinking is
that this would be best done via an add-on tool which normalizes the db
structure into something we can search.


> - support for kernel-style commit messages:
> For example, you have a commit message like this:
>
> Capitalized, short summary line
>
> <Detailed explanatory text which can be long, preferably wrapped to around
> 72 characters or less>
>

IMO fossil must not force arbitrary conventions like this onto the users.
(To be clear: my opinion is not the one which ultimately counts!)

(I tried altering the Timeline view CSS for this from a few threads back,
> but it made the Timeline page graph ugly.)
>

Yes, it's difficult to modify the timeline view - it has grown into a truly
complex beast over the years. When working on the JSON bits, the timeline
part was certainly the most painful one to port. One of the goals of a new
API would be to replace such monoliths with smaller parts.


--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Isaac Jurado
2013-07-22 11:04:51 UTC
Permalink
On Mon, Jul 22, 2013 at 12:28 PM, Stephan Beal <***@googlemail.com> wrote:
> On Mon, Jul 22, 2013 at 12:21 PM, Arnel Legaspi <***@gmail.com>
> wrote:
>>
>> Would using something similar to Vim's mechanism (allowing Fossil to
>> be compiled with Lua/Ruby/MZScheme/Python/etc. support) be more
>> acceptable?
>
>
> If the core library has a sane interface, there's no reason we can't
> have ALL of those bindings - they just need to be implemented by
> someone. For example, as soon as fossil has a library API, i will
> certainly write a binding for it for my own personal scripting
> language (not because the world needs it, but because that's the type
> of thing i do in my free time), and being able to implement such
> bindings will be a point i focus on when designing any fossil library
> interface.

Converting Fossil into a library has a lot of downsides:

- Building a library in a platform-independent manner is non-trivial
(unless making use of things such as libtool or CMake is alright).

- A lot of additional code has to be added to avoid being tricked by
users passing incorrect parameters.

- Testing is more tedious; test code needs to be developed in C to
ensure the calls work. Unless the test code can be generated
automatically through a scripting language, which is not simple
either.

- Ensuring API/ABI compatibility is harder. And this actually slows
down development because new features have to be implemented

- Sooner or later, a useful library has to match the following
requirements:

* Re-entrant
* Thread-safe
* Support both synchronous and asynchronous remote operations
* Friendly to bindings

To match the previous requirements, designs can be come very
baroque, and therefore, harder to read and maintain.

- A thorough documentation would be necessary.

But, the worst of all, is that even if the library is greande-proof, you
would be indirectly hit by bugs in external software using it.

Even Git, with its humongous manpower, has struggled a couple of years
to make an incomplete library.

I'd suggest the Mercurial approach: focus on the JSON API and offer it
over multiple transports (HTTP, SSH, UNIX sockets, pipes). Then let
anybody interested on integrating develop a client in their language of
choice. Because JSON has first-class support in almost any computer
language, bindings would not be necessary and the current process model
could be kept.

Best regards.

--
Isaac Jurado

"The noblest pleasure is the joy of understanding"
Leonardo da Vinci
Isaac Jurado
2013-07-22 11:06:28 UTC
Permalink
On Mon, Jul 22, 2013 at 1:04 PM, Isaac Jurado <***@gmail.com> wrote:
>
> - Ensuring API/ABI compatibility is harder. And this actually slows
> down development because new features have to be implemented

Sorry, incomplete sentence:

New features would have to be implemented in the library first and
then exposed. That's 150% development overhead.

--
Isaac Jurado

"The noblest pleasure is the joy of understanding"
Leonardo da Vinci
Stephan Beal
2013-07-22 11:32:30 UTC
Permalink
On Mon, Jul 22, 2013 at 1:06 PM, Isaac Jurado <***@gmail.com> wrote:

> On Mon, Jul 22, 2013 at 1:04 PM, Isaac Jurado <***@gmail.com> wrote:
> >
> > - Ensuring API/ABI compatibility is harder. And this actually slows
> > down development because new features have to be implemented
>
> Sorry, incomplete sentence:
>
> New features would have to be implemented in the library first and
> then exposed. That's 150% development overhead.


That may be the case, but isn't necessarily always so. But once we invest
the 150% then the next 3 guys who want to re-use that functionality spend
nearly 0% of the time they would have had to without a library.


--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Stephan Beal
2013-07-22 11:30:55 UTC
Permalink
On Mon, Jul 22, 2013 at 1:04 PM, Isaac Jurado <***@gmail.com> wrote:

> Converting Fossil into a library has a lot of downsides:
>
> - Building a library in a platform-independent manner is non-trivial
> (unless making use of things such as libtool or CMake is alright).
>

It can't be any more problematic than making a portable binary. The
majority of fossil's portability-related problems have been in app-level
code on Windows (e.g. converting paths from UTF8 to whatever Windows uses).


> - A lot of additional code has to be added to avoid being tricked by
> users passing incorrect parameters.
>

That's just part of - a library has to be more careful with its parameters
AND must also report problems instead of just dying immediately.



>
> - Testing is more tedious; test code needs to be developed in C to
> ensure the calls work.


Not if we script it. sqlite3 runs all its tests in TCL, and it is arguably
the single best-tested open source library in the world.



> Unless the test code can be generated
> automatically through a scripting language, which is not simple
> either.
>

Once a library interface is in place, adding a script binding is, as a
general rule, very simple. i've bound literally dozens of 3rd-party APIs to
3 different JS interpreters, and the process is more or less always the
same.


> - Ensuring API/ABI compatibility is harder. And this actually slows
> down development because new features have to be implemented
>

i don't envision us having to work about this. Fossil is not a library with
the same uses as, say, sqlite3. If we have to concern ourselves with this
level of compatibility then fossil has probably gotten more popular than i
expected. In any case, i do not envision most people using the library -
they will have a single binary (built on top of the library). The library
would be there so that fossil can finally be integrated into places like
IDEs.



> - Sooner or later, a useful library has to match the following
> requirements:
>
> * Re-entrant
> * Thread-safe
> * Support both synchronous and asynchronous remote operations
> * Friendly to bindings
>

Thread-safety is not a factor here, i think. Any given instance of a Fossil
context only need be single-threaded (all of the multi-threaded use cases
go out of the project's scope, i think, e.g. parallel threads for
searching). The is, IMO, no reason why fossil would need to support
multiple threads. Reentrace is, in my experience, not a problem so long as
one uses a Context parameter. Asynchronous operations are another place
which i consider to be out of scope for fossil. Being friendly to bindings
will be a focus of mine - i've got a considerable amount of experience in
this area.


> To match the previous requirements, designs can be come very
> baroque, and therefore, harder to read and maintain.
>

True, but fossil's current code is monolithic and relatively inflexible in
the face of change.



> - A thorough documentation would be necessary.
>

Not a problem - both Richard and i are documentation maniacs. My one-man
open source projects are regularly rated in the top 10% most-documented
projects by ohloh.net :). Writing technical docs is a hobby/paste-time of
mine:

https://docs.google.com/presentation/d/1plJbtQZXKBymEiw9sYy9xqnzMleWuYO461R3VrocOWQ/view



>
> But, the worst of all


You say that as if writing docs is a bad thing ;)



> , is that even if the library is greande-proof, you
> would be indirectly hit by bugs in external software using it.
>

That's just part of it. Right now we're in the situation that fossil cannot
fit in places it should be able to fit (e.g. SCM support in IDEs) because
it is monolithic.


> Even Git, with its humongous manpower, has struggled a couple of years
> to make an incomplete library.
>

Git has a level of popularity/penetration which (i hope!) never plagues
fossil. As long as fossil stays niche-market, the dev team will have a lot
more freedom in the form of a _lack_ of pressure to conform to requirements
of popular plugins/add-ons.



>
> I'd suggest the Mercurial approach: focus on the JSON API and offer it
> over multiple transports (HTTP, SSH, UNIX sockets, pipes).


Actually, that's kind of how the JSON API was started. Richard asked, "what
does Fossil need?" to which i answered, "a library." After agreeing that
the effort would be huge, i settled for JSON (which is, in effect, a
network-linked DLL). However, some of fossil's primary features cannot fit
into JSON without significant shoe-horning. e.g. commits may require binary
data, for which there is no JSON equivalent.



> Then let
> anybody interested on integrating develop a client in their language of
> choice. Because JSON has first-class support in almost any computer
> language, bindings would not be necessary and the current process model
> could be kept.
>

the JSON API can, in principal, proxy most of fossil's operations. There
are a handful of things (namely binary data) which is cannot do. Another
potential pain point is JSON (despite common misconception) does NOT
specify an integer precision, which means that any large integers it
generates might or might not be portable to other systems. This can be
worked around by using doubles instead of integers, and we have not _yet_
seen such incompatibilities, but i expect to at some point.


--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Isaac Jurado
2013-07-22 12:38:24 UTC
Permalink
On Mon, Jul 22, 2013 at 1:30 PM, Stephan Beal <***@googlemail.com> wrote:
>
>> - Ensuring API/ABI compatibility is harder. And this actually
>> slows down development because new features have to be
>> implemented
>
> i don't envision us having to work about this. Fossil is not a
> library with the same uses as, say, sqlite3. If we have to concern
> ourselves with this level of compatibility then fossil has probably
> gotten more popular than i expected. In any case, i do not envision
> most people using the library - they will have a single binary (built
> on top of the library). The library would be there so that fossil can
> finally be integrated into places like IDEs.

I'm sorry but I find this a bit confusing. If you want to offer a
library where programs can link into, what difference does it make by
assuming it will have a small and concrete set of users? That code will
need maintenance too, and a compatibility compromise if often necessary
for practical reasons.

Regards.

--
Isaac Jurado

"The noblest pleasure is the joy of understanding"
Leonardo da Vinci
Stephan Beal
2013-07-22 12:47:38 UTC
Permalink
On Mon, Jul 22, 2013 at 2:38 PM, Isaac Jurado <***@gmail.com> wrote:

> I'm sorry but I find this a bit confusing. If you want to offer a
> library where programs can link into, what difference does it make by
> assuming it will have a small and concrete set of users?


That's a fair question. My _assumption_ is that fossil's user base would
not be one who is affected by binary incompatibility problems, for a couple
reasons:

a) the vast majority won't be directly using the library, but will be using
the binary
b) the number of users is small compared to, e.g., git.
c) the preferred embedding approach (IMO) should be an "amalgamation build"
(like sqlite does), which basically gets rid of binary compatibility
problems.
d) after this restructuring/rewrite, i expect more services to be more
readily available over network interfaces (which don't have binary
compatibility problems, just data format compatibility).



@All: i really appreciate the continued feedback. The Google Doc has had
1-5 users viewing it almost full-time since yesterday. (i cannot see who
they are with the exception of those who have explicit write access - all
others show up as anonymous.)

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Trevor Davel (Twylite)
2013-07-22 13:16:17 UTC
Permalink
Hi,

On 2013/07/21 12:54 PM, Stephan Beal wrote:
> To help bootstrap the process of figuring out what Fossil v2 might
> look like i have started writing down ideas in a public Google Doc:
>
> https://docs.google.com/document/d/12g0s5A2TPX7-y47Nsw235rvsjcuh49TnHfMDB4ASvlo/view

One feature I would love to see in Fossil v2 (or v1) is support for
tracking code reviews. This would cover finding commits that require
review, flagging the commit once the review has been completed, and
reporting on what has been reviewed (and by whom).

One approach would be to have non-propagating review tags on commits,
specifically including the ability to search for unreviewed commits by
branch.

An alternative realisation would be to automatically create "Review"
tickets for any commit on a branch that has a specific propagating tag
indicating that review is required.

Example interaction sequence:
1) Find commits on branch "release" that have not been reviewed.
2) Perform review. Flag commit as "reviewed-ok" or "reviewed-rework" or
"noreview (part of [abcdef])".
3) Repeat until all commits have been reviewed.
4) Show timeline for "release", highlights unreviewed commits; includes
detail of which user marked each commit as reviewed.

Regards,
Twylite
Stephan Beal
2013-07-22 13:21:53 UTC
Permalink
On Mon, Jul 22, 2013 at 3:16 PM, Trevor Davel (Twylite) <***@crypt.co.za
> wrote:

> Hi,
>
>
> On 2013/07/21 12:54 PM, Stephan Beal wrote:
>
>> To help bootstrap the process of figuring out what Fossil v2 might look
>> like i have started writing down ideas in a public Google Doc:
>>
>> https://docs.google.com/**document/d/12g0s5A2TPX7-**
>> y47Nsw235rvsjcuh49TnHfMDB4ASvl**o/view<https://docs.google.com/document/d/12g0s5A2TPX7-y47Nsw235rvsjcuh49TnHfMDB4ASvlo/view>
>>
>
> One feature I would love to see in Fossil v2 (or v1) is support for
> tracking code reviews. This would cover finding commits that require
> review, flagging the commit once the review has been completed, and
> reporting on what has been reviewed (and by whom).
>
> One approach would be to have non-propagating review tags on commits,
> specifically including the ability to search for unreviewed commits by
> branch.
>

Speaking of: being able to tag non-branches (e.g. tagging individual wiki
pages or files) is something i'd like to add/see added to v2.


> An alternative realisation would be to automatically create "Review"
> tickets for any commit on a branch that has a specific propagating tag
> indicating that review is required.
>

That sounds like a fairly straightforward implementation which could fit in
the current architecture.

All added to the list...

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Remigiusz Modrzejewski
2013-07-22 13:28:39 UTC
Permalink
On Jul 21, 2013, at 12:54 , Stephan Beal wrote:

> To help bootstrap the process of figuring out what Fossil v2 might look
> like i have started writing down ideas in a public Google Doc:
>
> https://docs.google.com/document/d/12g0s5A2TPX7-y47Nsw235rvsjcuh49TnHfMDB4ASvlo/view

And why not a public Fossil repo?

The answer to this question may give some insight into what's not perfect with Fossil...


Kind regards,
Remigiusz Modrzejewski
Stephan Beal
2013-07-22 13:41:44 UTC
Permalink
On Mon, Jul 22, 2013 at 3:28 PM, Remigiusz Modrzejewski
<***@maxnet.org.pl>wrote:

>
> On Jul 21, 2013, at 12:54 , Stephan Beal wrote:
>
> > To help bootstrap the process of figuring out what Fossil v2 might look
> > like i have started writing down ideas in a public Google Doc:
> >
> >
> https://docs.google.com/document/d/12g0s5A2TPX7-y47Nsw235rvsjcuh49TnHfMDB4ASvlo/view
>
> And why not a public Fossil repo?
>

Because GDocs allows multiple people to edit at the same time (or watch
while someone else edits). Fossil won't ever do that (until/unless there is
a simple 3rd-party JS API for doing so). i've been typing on that doc the
past 24 hours or so and the people who have it opened can see what i type
as i type it. (Side-note: once you work with gdocs long enough, then every
time you type an email you will start to have the feeling that whatever you
type is visible in real time to those who you will _eventually_ send it to!
Strange but true.)


> The answer to this question may give some insight into what's not perfect
> with Fossil...
>

We could also argue that fossil is not "perfect" because it does not
support real-time collaborative editing, but i think that would be
overstating the problem. There are lots of things fossil neither does nor
should do.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Remigiusz Modrzejewski
2013-07-22 13:46:03 UTC
Permalink
On Jul 22, 2013, at 15:41 , Stephan Beal wrote:

>> And why not a public Fossil repo?
>
> Because GDocs allows multiple people to edit at the same time (or watch
> while someone else edits). Fossil won't ever do that (until/unless there is
> a simple 3rd-party JS API for doing so). i've been typing on that doc the
> past 24 hours or so and the people who have it opened can see what i type
> as i type it.

Ah, this didn't occur to me.
In our usage GDocs is a poor man's scm.

>> The answer to this question may give some insight into what's not perfect
>> with Fossil...
>>
>
> We could also argue that fossil is not "perfect" because it does not
> support real-time collaborative editing, but i think that would be
> overstating the problem. There are lots of things fossil neither does nor
> should do.

I agree. I just didn't think of "watch me as I type" aspect of GDocs.
If it got implemented in Fossil, that would be a huge red blinking warning sign that it went astray.


Kind regards,
Remigiusz Modrzejewski
Stephan Beal
2013-07-22 13:54:15 UTC
Permalink
On Mon, Jul 22, 2013 at 3:46 PM, Remigiusz Modrzejewski
<***@maxnet.org.pl>wrote:

> ...
>


> Ah, this didn't occur to me.
> In our usage GDocs is a poor man's scm.
>

And google takes over the user administration ;)


>
> I agree. I just didn't think of "watch me as I type" aspect of GDocs.
>

Once you get used to it, it ALWAYS feels like someone is watching what you
type. :/ Kinda creepy.

If it got implemented in Fossil, that would be a huge red blinking warning
> sign that it went astray.
>

Amen!

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Baruch Burstein
2013-07-22 20:23:37 UTC
Permalink
On Mon, Jul 22, 2013 at 4:41 PM, Stephan Beal <***@googlemail.com> wrote:

> On Mon, Jul 22, 2013 at 3:28 PM, Remigiusz Modrzejewski <
> ***@maxnet.org.pl> wrote:
>
>>
>> On Jul 21, 2013, at 12:54 , Stephan Beal wrote:
>>
>> > To help bootstrap the process of figuring out what Fossil v2 might look
>> > like i have started writing down ideas in a public Google Doc:
>> >
>> >
>> https://docs.google.com/document/d/12g0s5A2TPX7-y47Nsw235rvsjcuh49TnHfMDB4ASvlo/view
>>
>> And why not a public Fossil repo?
>>
>
> Because GDocs allows multiple people to edit at the same time (or watch
> while someone else edits). Fossil won't ever do that (until/unless there is
> a simple 3rd-party JS API for doing so). i've been typing on that doc the
> past 24 hours or so and the people who have it opened can see what i type
> as i type it. (Side-note: once you work with gdocs long enough, then every
> time you type an email you will start to have the feeling that whatever you
> type is visible in real time to those who you will _eventually_ send it to!
> Strange but true.)
>

I am sure I am not the only one who, when typing an email, find myself
typing the first few letters of a word I don't know how to spell or is just
long, and then pressing CTRL+SPACE (for the die-hard vim users, that is
auto-complete in many IDE's)
--
˙uʍop-ǝpısdn sı ɹoʇıuoɯ ɹnoʎ 'sıɥʇ pɐǝɹ uɐɔ noʎ ɟı
Joseph R. Justice
2013-07-23 02:30:43 UTC
Permalink
On Sun, Jul 21, 2013 at 6:54 AM, Stephan Beal <***@googlemail.com> wrote:

Hi, all,
>
> This topic has been tossed around before, but the amount of effort
> involved in its undertaking has always kept us from actually doing it...
>
> To help bootstrap the process of figuring out what Fossil v2 might look
> like i have started writing down ideas in a public Google Doc:
>


> If you don't have access to that doc, either send me your gmail address
> and i'll gladly add you, or post your ideas here and i'll integrate them
> into the doc.
>

Some random thoughts on Fossil v2 as a library (call it "libfossil2") and
as a default client / server binary which makes use of the library (call it
"fossil2client" or "fossil2scm", I seem to have used both names), which I
realize you might already know about or have thought of but just in case...
(if only because I have a terminal case of Male Answer Syndrome) (and, wow,
this got longer than I expected, and I apologize in advance for telling you
in detail all about everything you Probably Already Knew and moreover Knew
Far More About Than I Do...):



* Fossil2client should *not* expect or require libfossil2 to be compiled
into it statically, or have an embedded copy of the source for libfossil2
within the source of fossil2client.

Reason why: The various Linux distributions, certainly the major FOSS ones
such as Fedora, Debian, etc, will require this; indeed, if fossil2client is
shipped with an embedded copy of the source for libfossil2, that embedded
copy will be stripped out in favor of the independent library version of
libfossil2 and fossil2client will be compiled to use that independent
library copy. They don't want to worry about security and other bugs they
have to patch in embedded code copies, etc; they want to fix the bug in one
place, the independent library itself, and let all the users of the library
just naturally benefit from that fix.

Debian, which is the Linux distro I am most familiar with, already does
this (or appears to anyway) with fossil v1, see
http://packages.debian.org/jessie/fossil where the fossil binary depends on
libsqlite3-0 (>= 3.7.11 at this instant). I don't know if fossil v1 ships
with an embedded code copy of sqlite (I haven't looked), but if it does
they're stripping it in favor of sqlite as an independent library object
and making their binary for fossil v1 dependent on that independent sqlite
library object.

I fully expect they would / will do the same with fossil2scm as a
dependency of libfossil2.



* The library should be implemented in such a way that multiple versions /
generations of it can be installed simultaneously. Perhaps alternatively,
there should be a mechanism where the library can be queried as to which
API version it is supporting, and/or instructed as to which API version a
client using it understands how to use.

Reason why: Let's say libfossil2 is installed and is supporting API version
Apple. Let's say two clients making use of libfossil2, namely fossil2scm
and IDEusingfossil2, are installed, and both know how to use libfossil2
with API version Apple.

Now, let's say a new release of libfossil2 is made with API version Berry
(Berry > Apple). Let's say there is some reason that libfossil2 with API
version Berry is not fully compatible with clients that only know how to
use API version Apple. Now, fossil2scm, because it comes from the fine
folks who brought you libfossil2, has a new version released in conjunction
with the new version of libfossil2 and the new version of fossil2scm is
compatible with API version Berry. However, IDEusingfossil2 does not yet
have a new release that understands API version Berry.

A Linux distribution faced with this, which wishes to offer working
versions of libfossil2, fossil2scm, and IDEusingfossil2, needs to be able
to allow for multiple versions of libfossil2 to be installed simultaneously
if it wishes to offer the very latest available versions of libfossil2 and
fossil2scm (because until IDEusingfossil2 can catch up in terms of
development it cannot use the very latest available version of
libfossil2). Alternatively, the Linux distribution is forced to not offer
the very latest available version of libfossil2 and fossil2scm until such
time as other things which depend on libfossil2 (e.g. IDEusingfossil2) can
be updated to use API version Barry, or else the distribution needs to make
things using API version Berry (e.g. fossil2scm) not simultaneously
installable with things using API version Apple (e.g. IDEusingfossil2).

However, note that if IDEusingfossil2 is able to tell whatever version of
libfossil2 it is linked with "I understand API version Apple, and I need
you to behave precisely as if you are API version Apple", and if libfossil2
is willing and able to obey this requirement, then it would be possible to
update libfossil2 (and other dependencies) to API version Berry even tho
IDEusingfossil2 is not yet updated to understand that API.



* The library should be implemented in such a way that multiple instances
of the library, and/or multiple different clients using either the same
instance of the library and/or multiple different instances of the library,
will not accidentally corrupt fossil repositories. Note that under
Unix/Linux many types of file locking (especially network file locking) are
less than fully secure or reliable.



* It might be worthwhile to reach out to the various major Linux and BSD
Unix distros to see what will make their lives easier in terms of packaging
and distributing libfossil2 and fossil2scm. This probably isn't as much of
an issue for people using Microsoft Windows and I assume MacOS/X, because
those people will mostly or solely use the binaries the fossil project
itself provides for libfossil2 and fossil2scm; only the most dedicated
developer will want to compile it themselves.



While I am reading the "Random Thoughts on Fossil v2" Google Doc circa the
time I am writing this:

Localization / internationalization: For at least FOSS projects, I know
there are organizations out there who will do much of the work of
localizing and internationalizing the software for you (in particular
various language strings) As Long As you use software libraries which make
it easy for them to do this. Whether or not these software libraries would
affect libfossil2 and fossil2scm in terms of the software licenses
applicable to them (I'm thinking in particular of GPL / strong copyleft
licensing as opposed to LGPL / weak copyleft or MIT/BSD/Apache/etc
non-copyleft licensing), I do not know off the top of my head. But, the
possibility is out there at least to leverage the work these groups do.



Abstraction around the repo and config DBs: I see two main areas here --
(1) abstractions which maintain the use of a SQL database, but don't
necessarily mandate sqlite, e.g. MySQL (and its siblings) or PostgreSQL, or
on Windows Microsoft SQL Server, or on everything Oracle, could possibly be
used as the database engine; (2) abstractions which do not mandate the use
of SQL, and in particular ones which permit use of some flavor of a
NoSQL-type engine. (I note that there are several distinct and not
necessarily cooperative things which all fall under the banner of NoSQL.)
Whether or not any of this would be desirable or even worthwhile I
certainly have no idea.

On a semi-related note... I know (or am under the impression) that many
database engines like to try to get down to the "raw disk" / bare metal,
and not have the platform OS be an intermediary, in terms of their access
to their files. I don't know if or how this is applicable to libfossil2
but I thought I would mention it.

On another note... We have SSDs now which are very fast but have limited
write cycles. We also have RAM disks available on many platforms. Does
this have any significance to libfossil2 and/or fossil2scm?



OS-level peculiarities: There are other multi-platform software projects
which have to deal with these, both with peculiarities between similar
types of platform, such as the various flavors of Unix and Linux, and
between drastically different platforms, such as between the former versus
the different versions of Windows, MacOS/X, ummm, VMS, the IBM mainframe
OSen, etc etc etc. All the major FOSS SQL databases have to deal with
this, the Apache web server (Apache HTTPD) has to deal with this (they have
Apache Portable Runtime IIRC), Gnu Emacs has to do this (tho that would
certainly involve GPL licensed software), I'm sure there's plenty more...
Maybe there's stuff that can be stolen ^H^H borrowed from one or more of
those to avoid having to roll your own.



Strict C89 or commonly-supported C99 extensions: What platforms, and
compilers on those platforms, do you want to be portable to? If everything
you care about is (non-buggy) C99, well...



HTTP/CGI servers: I could see people wanting to use libfossil2 with HTTP
servers other than what is provided by fossil2scm, e.g. Apache, Nginx,
Microsoft Windows IIS, whatever MacOS/X uses, etc.



Consistent CLI args/flags handling: For Linux, the Gnu C Library (libc) has
getopt and argp_parse (see
http://www.gnu.org/software/libc/manual/html_node/Parsing-Program-Arguments.htmlfor
details). However, this is certainly not available as a standard
library for platforms like Microsoft Windows and VMS, and if it is provided
as a library which is not a standard system library there are likely GPL
licensing issues. It might be worth seeing what the projects I mentioned
in OS-level peculiarities do about this, and what the various BSD Unix
flavors do about this.

On a semi-related note, and because I'd written most of it before realizing
this wasn't applicable to CLI args/flags... If fossil2scm is to be used in
an interactive mode (e.g. where it does not fire up, do something, and
terminate, but instead provides some sort of internal command line or
windowing interface as applicable)... Certainly there's Very Well Known
libraries, at least for Unix/Linux, which Just Handle This, e.g. Gnu
Readline. However, Gnu Readline is GPL licensed (and will *NOT* change),
so unless this license is acceptable for libfossil2 / fossil2scm a
different library with a more acceptable license would be required. (I do
not know what the BSD Unix equivalent to Gnu Readline is, but I think one
is being worked on with a license more acceptable to the BSDs.)



Not restricting to backwards compatibility: If you do this, libfossil2 and
things dependent on it must never write something to a fossil v1 repository
which would be seen by a v1 client as corruption, unless, *perhaps*, there
is an Explicit and Unambiguous direction to do so with acknowledgement that
this will render the v1 repository corrupt in the eyes of a v1 client.
This requires that libfossil2 always be able to correctly determine that a
repository is or is not in v1 format (or perhaps more safely assume that a
repository is always in v1 format unless they can specifically determine
that it is v2 format).



Social networking logins: See the OpenID Foundation (at http://openid.net/).

On a related note... For SSL, PGP keys, etc... I know there's issues with
clients being able to tell that the server they are being directed to by
means of DNS is the real server they want to go to, e.g. that they are not
being fed poisoned or spoofed DNS data. IIRC there are protocols being
developed for DNS to deal with this sort of issue, tho I don't know how
mature they are yet or how widely they're implemented. For (public) PGP
keys, see the stuff at www.pgp.net (it redirects). Also related
http://en.wikipedia.org/wiki/CAcert and http://www.cacert.org/ .
http://www.biglumber.com/ .

On a different note... Insofar as networking is an issue, make certain
everything works with IPv6 as well as IPv4!



Goodness. I didn't realize I would write that much.

In any event. Thanks for giving me a (large) chunk of your time by reading
this. Hope at least some of it has been of some use, interest. Be well.



Joseph

>
Stephan Beal
2013-07-23 08:54:40 UTC
Permalink
On Tue, Jul 23, 2013 at 4:30 AM, Joseph R. Justice <***@gmail.com>wrote:

> ...Some random thoughts on Fossil v2 as a library (call it "libfossil2")
> and as a default client / server binary which makes use of the library
> (call it "fossil2client" or "fossil2scm", I seem to have used both names),
> which I realize you might already know about or have thought of but just in
> case... (if only because I have a terminal case of Male Answer Syndrome)
> (and, wow, this got longer than I expected, and I apologize in advance for
> telling you in detail all about everything you Probably Already Knew and
> moreover Knew Far More About Than I Do...):
>

Rabid rants are in no way excluded from participation, provided they remain
polite ;).


> * Fossil2client should *not* expect or require libfossil2 to be compiled
> into it statically, or have an embedded copy of the source for libfossil2
> within the source of fossil2client.
>

Right. My goal here is a normal library/client relationship. Static linking
is an important option for fossil (because it's popular with users) but is
not a requirement.


> Debian, which is the Linux distro I am most familiar with, already does
> this (or appears to anyway) with fossil v1, see
> http://packages.debian.org/jessie/fossil where the fossil binary depends
> on libsqlite3-0 (>= 3.7.11 at this instant). I don't know if fossil v1
> ships with an embedded code copy of sqlite (I haven't looked),
>

Strange. Yes, fossil embeds its own. That dependency was probably
automatically determined by a script which looks for common symbol names in
the resulting binary.



> but if it does they're stripping it in favor of sqlite as an independent
> library object and making their binary for fossil v1 dependent on that
> independent sqlite library object.
>

i suspect it's only a bookkeeping error, and that (ld fossil) on that
system will reveal that it does not link against /usr/lib/libsqlite3.so


> I fully expect they would / will do the same with fossil2scm as a
> dependency of libfossil2.
>

They'll probably do what's easiest - if it's distributed as a package which
builds everything together (as i suspect it will be) then they'll probably
just 'make; make install'.


> * The library should be implemented in such a way that multiple versions /
> generations of it can be installed simultaneously. Perhaps alternatively,
> there should be a mechanism where the library can be queried as to which
> API version it is supporting, and/or instructed as to which API version a
> client using it understands how to use.
>

Long term that is a good point, and if you have concrete ideas on how best
to achieve this, please elaborate.


> Now, let's say a new release of libfossil2 is made with API version Berry
> (Berry > Apple). Let's say there is some reason that libfossil2 with API
> version Berry is not fully compatible with clients that only know how to
> use API version Apple. Now, fossil2scm, because it comes from the fine
> folks who brought you libfossil2, has a new version released in conjunction
> with the new version of libfossil2 and the new version of fossil2scm is
> compatible with API version Berry. However, IDEusingfossil2 does not yet
> have a new release that understands API version Berry.
>

i'm hoping that (A) 95% of the people just keep using the standalone client
binary, (B) that the 5% who don't will use an amalgamation build so that
they do not have such problems :/. That is the official solution sqlite3
recommends, and i see no reason not to do the same for fossil. That said,
sqlite3 has much stricter compatibility requirements than fossil is likely
to have. (Maybe we'll be cursed with success and indeed have to cast the
API in stone at some point, but then there's always v3 to hack on...)


> However, note that if IDEusingfossil2 is able to tell whatever version of
> libfossil2 it is linked with "I understand API version Apple, and I need
> you to behave precisely as if you are API version Apple", and if libfossil2
> is willing and able to obey this requirement, then it would be possible to
> update libfossil2 (and other dependencies) to API version Berry even tho
> IDEusingfossil2 is not yet updated to understand that API.
>

sqlite has functions which allow you to query the linked in library
version, and fossil would of course have the same.


> * The library should be implemented in such a way that multiple instances
> of the library, and/or multiple different clients using either the same
> instance of the library and/or multiple different instances of the library,
> will not accidentally corrupt fossil repositories. Note that under
> Unix/Linux many types of file locking (especially network file locking) are
> less than fully secure or reliable.
>

Those (extremely difficult) details are 100% delegated to sqlite. There's
no reason fossil has no directly deal with them.


> * It might be worthwhile to reach out to the various major Linux and BSD
> Unix distros to see what will make their lives easier in terms of packaging
> and distributing libfossil2 and fossil2scm.
>

i can't personally commit to it, but wouldn't object to it. That said,
we're still a long way from even determining if this whole experiment is
even feasible. It might turn out that it becomes to difficult to do, that
nobody is willing to do it. And that's fine - the whole thing is an
experiment at this point and it won't break my heart if it turns out it's
just more effort than it's worth.



> Localization / internationalization: For at least FOSS projects, I know
> there are organizations out there who will do much of the work of
> localizing and internationalizing the software for you (in particular
> various language strings) As Long As you use software libraries which make
> it easy for them to do this.
>

ideally the core library won't emit any human-language strings except to
propagate OS/sqlite3-level errors.



> Whether or not these software libraries would affect libfossil2 and
> fossil2scm in terms of the software licenses applicable to them (I'm
> thinking in particular of GPL / strong copyleft licensing as opposed to
> LGPL / weak copyleft or MIT/BSD/Apache/etc non-copyleft licensing), I do
> not know off the top of my head. But, the possibility is out there at
> least to leverage the work these groups do.
>

That's good to know. i have no idea what the conventional approaches are
for this, but i also have no intention of outputting arbitrary strings from
a library (error messages excepted). The l10n/i18n will become the problem
of the concrete application(s).


> Abstraction around the repo and config DBs: I see two main areas here --
> (1) abstractions which maintain the use of a SQL database, but don't
> necessarily mandate sqlite, e.g. MySQL (and its siblings) or PostgreSQL, or
> on Windows Microsoft SQL Server, or on everything Oracle, could possibly be
> used as the database engine; (2) abstractions which do not mandate the use
> of SQL, and in particular ones which permit use of some flavor of a
> NoSQL-type engine. (I note that there are several distinct and not
> necessarily cooperative things which all fall under the banner of NoSQL.)
> Whether or not any of this would be desirable or even worthwhile I
> certainly have no idea.
>

My current thinking is, "abstracting away sqlite3 will be painful,"
particularly because so much is implemented in terms of sqlite-specific SQL.

On another note... We have SSDs now which are very fast but have limited
> write cycles. We also have RAM disks available on many platforms. Does
> this have any significance to libfossil2 and/or fossil2scm?
>

Not directly - the vast majority of the i/o happens through sqlite (or "the
storage layer") and fossil will not specify/require any particular
performance characteristics. HW-level details aren't fossil's domain. If
it's running on a 720kb floppy disk it will behave the same (just a lot
more slowly).


> OS-level peculiarities: There are other multi-platform software projects
> which have to deal with these, both with peculiarities between similar
> types of platform, such as the various flavors of Unix and Linux, and
> between drastically different platforms, such as between the former versus
> the different versions of Windows, MacOS/X, ummm, VMS, the IBM mainframe
> OSen, etc etc etc. All the major FOSS SQL databases have to deal with
> this, the Apache web server (Apache HTTPD) has to deal with this (they have
> Apache Portable Runtime IIRC), Gnu Emacs has to do this (tho that would
> certainly involve GPL licensed software), I'm sure there's plenty more...
> Maybe there's stuff that can be stolen ^H^H borrowed from one or more of
> those to avoid having to roll your own.
>

i think we currently have most of the painful parts already implemented in
fossil. e.g. the weird Windows console text conversions and conversion of
filename encodings. The existing code works fine for windows/*nix (of
various sorts), and i suspect we'll just be recycling that.

Strict C89 or commonly-supported C99 extensions: What platforms, and
> compilers on those platforms, do you want to be portable to? If everything
> you care about is (non-buggy) C99, well...
>

The only c99 feature i use regularly is <stdint.h> and <inttypes.h>, which
eliminate the more glaring of the C89 portability problems. sqlite3 uses
(long long) in one place, so it's not strictly c89 portable, but it does
indeed work on the majority of C89 platforms.



> HTTP/CGI servers: I could see people wanting to use libfossil2 with HTTP
> servers other than what is provided by fossil2scm, e.g. Apache, Nginx,
> Microsoft Windows IIS, whatever MacOS/X uses, etc.
>

That's all possible now, and don't imagine that that will go away.

Okay, it seems there are still two pages to answer here, but i need to get
back to work for a while. i'll respond to the rest later.


--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Matt Welland
2013-07-23 20:35:57 UTC
Permalink
Here are a couple features that would make fossil a reasonable replacement
for zim wiki and might be worth considering for fossil2.0.

1. Ability to Edit/save/commit files from the UI.
2. In wiki files square brackets at beginning of line parse into check box
list (as is done in zim wiki).




On Sun, Jul 21, 2013 at 3:54 AM, Stephan Beal <***@googlemail.com> wrote:

> Hi, all,
>
> This topic has been tossed around before, but the amount of effort
> involved in its undertaking has always kept us from actually doing it...
>
> To help bootstrap the process of figuring out what Fossil v2 might look
> like i have started writing down ideas in a public Google Doc:
>
>
> https://docs.google.com/document/d/12g0s5A2TPX7-y47Nsw235rvsjcuh49TnHfMDB4ASvlo/view
>
> Any of you who have write access to the JSON API docs also have write
> access to that one, so feel free to expand/comment/etc. It's just a big
> scratchpad, not a formal doc, nor does it provide any indication of what
> Fossil's future has in store - it's just ideas regarding what v2 "might"
> look like if i were to start working on it today (which, in a way, i am ;).
>
> If you don't have access to that doc, either send me your gmail address
> and i'll gladly add you, or post your ideas here and i'll integrate them
> into the doc.
>
> --
> ----- stephan beal
> http://wanderinghorse.net/home/stephan/
> http://gplus.to/sgbeal
>
> _______________________________________________
> fossil-users mailing list
> fossil-***@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
>


--
Matt
-=-
90% of the nations wealth is held by 2% of the people. Bummer to be in the
majority...
Mark Janssen
2013-07-23 20:42:45 UTC
Permalink
Bit late to the party, but my 2 cents

1) Fossil as a library or API with the fossil executable as a single file
built on top of it. One could even consider a SCGI/FCGI type of interface
where the fossil binary serves JSON+BSON requests.
2) Ticket notifications by email (notification when merged into my main
repo would be fine). Using the ticketing system for any app with a decent
userbase is a pain right now.

Mark


On Tue, Jul 23, 2013 at 10:35 PM, Matt Welland <***@gmail.com> wrote:

> Here are a couple features that would make fossil a reasonable replacement
> for zim wiki and might be worth considering for fossil2.0.
>
> 1. Ability to Edit/save/commit files from the UI.
> 2. In wiki files square brackets at beginning of line parse into check box
> list (as is done in zim wiki).
>
>
>
>
> On Sun, Jul 21, 2013 at 3:54 AM, Stephan Beal <***@googlemail.com>wrote:
>
>> Hi, all,
>>
>> This topic has been tossed around before, but the amount of effort
>> involved in its undertaking has always kept us from actually doing it...
>>
>> To help bootstrap the process of figuring out what Fossil v2 might look
>> like i have started writing down ideas in a public Google Doc:
>>
>>
>> https://docs.google.com/document/d/12g0s5A2TPX7-y47Nsw235rvsjcuh49TnHfMDB4ASvlo/view
>>
>> Any of you who have write access to the JSON API docs also have write
>> access to that one, so feel free to expand/comment/etc. It's just a big
>> scratchpad, not a formal doc, nor does it provide any indication of what
>> Fossil's future has in store - it's just ideas regarding what v2 "might"
>> look like if i were to start working on it today (which, in a way, i am ;).
>>
>> If you don't have access to that doc, either send me your gmail address
>> and i'll gladly add you, or post your ideas here and i'll integrate them
>> into the doc.
>>
>> --
>> ----- stephan beal
>> http://wanderinghorse.net/home/stephan/
>> http://gplus.to/sgbeal
>>
>> _______________________________________________
>> fossil-users mailing list
>> fossil-***@lists.fossil-scm.org
>> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>>
>>
>
>
> --
> Matt
> -=-
> 90% of the nations wealth is held by 2% of the people. Bummer to be in the
> majority...
>
> _______________________________________________
> fossil-users mailing list
> fossil-***@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
>
Stephan Beal
2013-07-24 01:06:37 UTC
Permalink
On Tue, Jul 23, 2013 at 10:42 PM, Mark Janssen <***@gmail.com>wrote:

> Bit late to the party, but my 2 cents
>

Not late at all - the ball's just getting rolling.

1) Fossil as a library or API with the fossil executable as a single file
> built on top of it. One could even consider a SCGI/FCGI type of interface
> where the fossil binary serves JSON+BSON requests.
>

i think the idea of the lib/app separation is starting to sink in with
people :).


> 2) Ticket notifications by email (notification when merged into my main
> repo would be fine). Using the ticketing system for any app with a decent
> userbase is a pain right now.
>

That one would come "almost automatically" as a side effect of triggers
support, i suspect, which is high on many peoples' wish lists.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Stephan Beal
2013-07-24 01:02:08 UTC
Permalink
On Tue, Jul 23, 2013 at 10:35 PM, Matt Welland <***@gmail.com> wrote:

> Here are a couple features that would make fossil a reasonable replacement
> for zim wiki and might be worth considering for fossil2.0.
>
> 1. Ability to Edit/save/commit files from the UI.
>

That one's been on my TODO for a long time because the JSON API really
wants this for embedded docs (which are just files, so it would then work
for any text file).


> 2. In wiki files square brackets at beginning of line parse into check box
> list (as is done in zim wiki).
>

That's at a much further downstream and the whole Wiki Wars are a topic i'm
gonna side-step for the time being ;). In case you've missed them in the
past... i think wiki-related threads are the only ones who approached this
one's size.



--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Loading...