Big problem to solve: good WYSIWYG on WMF wikis

[David Gerard]

[crossposted to foundation-l and wikitech-l]


"There has to be a vision though, of something better. Maybe something
that is an actual wiki, quick and easy, rather than the template
coding hell Wikipedia's turned into." - something Fred Bauder just
said on wikien-l.


Our current markup is one of our biggest barriers to participation.

AIUI, edit rates are about half what they were in 2005, even as our
fame has gone from "popular" through "famous" to "part of the
structure of the world." I submit that this is not a good or healthy
thing in any way and needs fixing.

People who can handle wikitext really just do not understand how
offputting the computer guacamole is to people who can cope with text
they can see.

We know this is a problem; WYSIWYG that works is something that's been
wanted here forever. There are various hideous technical nightmares in
its way, that make this a big and hairy problem, of the sort where the
hair has hair.

However, I submit that it's important enough we need to attack it with
actual resources anyway.


This is just one data point, where a Canadian government office got
*EIGHT TIMES* the participation in their intranet wiki by putting in a
(heavily locally patched) copy of FCKeditor:


http://lists.wikimedia.org/pipermail/mediawiki-l/2010-May/034062.html

"I have to disagree with you given my experience. In one government
department where MediaWiki was installed we saw the active user base
spike from about 1000 users to about 8000 users within a month of having
enabled FCKeditor. FCKeditor definitely has it's warts, but it very
closely matches the experience non-technical people have gotten used to
while using Word or WordPerfect. Leveraging skills people already have
cuts down on training costs and allows them to be productive almost
immediately."

http://lists.wikimedia.org/pipermail/mediawiki-l/2010-May/034071.html

"Since a plethora of intelligent people with no desire to learn WikiCode
can now add content, the quality of posts has been in line with the
adoption of wiki use by these people. Thus one would say it has gone up.

"In the beginning there were some hard core users that learned WikiCode,
for the most part they have indicated that when the WYSIWYG fails, they
are able to switch to WikiCode mode to address the problem. This usually
occurs with complex table nesting which is something that few of the
users do anyways. Most document layouts are kept simple. Additionally,
we have a multilingual english/french wiki. As a result the browser
spell-check is insufficient for the most part (not to mention it has
issues with WikiCode). To address this a second spellcheck button was
added to the interface so that both english and french spellcheck could
be available within the same interface (via aspell backend)."


So, the payoffs could be ridiculously huge: eight times the number of
smart and knowledgeable people even being able to *fix typos* on
material they care about.

Here are some problems. (Off the top of my head; please do add more,
all you can think of.)


- The problem:

* Fidelity with the existing body of wikitext. No conversion flag day.
The current body exploits every possible edge case in the regular
expression guacamole we call a "parser". Tim said a few years ago that
any solution has to account for the existing body of text.

* Two-way fidelity. Those who know wikitext will demand to keep it and
will bitterly resist any attempt to take it away from them.

* FCKeditor (now CKeditor) in MediaWiki is all but unmaintained.

* There is no specification for wikitext. Well, there almost is -
compiled as C, it runs a bit slower than the existing PHP compiler.
But it's a start!
http://lists.wikimedia.org/pipermail/wikitext-l/2010-August/000318.html


- Attempting to solve it:

* The best brains around Wikipedia, MediaWiki and WMF have dashed
their foreheads against this problem for at least the past five years
and have got *nowhere*. Tim has a whole section in the SVN repository
for "new parser attempts". Sheer brilliance isn't going to solve this
one.

* Tim doesn't scale. Most of our other technical people don't scale.
*We have no resources and still run on almost nothing*.

($14m might sound like enough money to run a popular website, but for
comparison, I work as a sysadmin at a tiny, tiny publishing company
with more money and staff just in our department than that to do
*almost nothing* compared to what WMF achieves. WMF is an INCREDIBLY
efficient organisation.)


- Other attempts:

* Starting from a clear field makes it ridiculously easy. The
government example quoted above is one. Wikia wrote a good WYSIWYG
that works really nicely on new wikis (I'm speaking here as an
experienced wikitext user who happily fixes random typos on Wikia). Of
course, I noted that we can't start from a clear field - we have an
existing body of wikitext.


So, specification of the problem:

* We need good WYSIWYG. The government example suggests that a simple
word-processor-like interface would be enough to give tremendous
results.
* It needs two-way fidelity with almost all existing wikitext.
* We can't throw away existing wikitext, much as we'd love to.
* It's going to cost money in programming the WYSIWYG.
* It's going to cost money in rationalising existing wikitext so that
the most unfeasible formations can be shunted off to legacy for
chewing on.
* It's going to cost money in usability testing and so on.
* It's going to cost money for all sorts of things I haven't even
thought of yet.


This is a problem that would pay off hugely to solve, and that will
take actual money thrown at it.

How would you attack this problem, given actual resources for grunt work?


- d.

[David Gerard]

On 28 December 2010 16:06, Victor Vasiliev <vasilvv-***@public.gmane.org> wrote:

> I have thought about WYSIWYG editor for Wikipedia and found it
> technically impossible. The main and key problem of WYSIWIG are
> templates. You have to understand that templates are not single
> element of Wikipedia syntax, they are integral part of page markup.
> You do not insert "infobox template", you insert infobox *itself*, and
> from what I heard the templates were the main concern of many editors
> who were scared of wikitext.
> Now think of how many templates are there in Wikipedia, how frequently
> they are changed and how much time it would take to implement their
> editing.


Yes. So how do we sensibly - usably - deal with templates in a
word-processor-like layout? Is there a way that passes usability
muster for non-geeks? How do others do it? Do their methods actually
work?

e.g. Wikia has WYSIWYG editing and templates. They have a sort of
solution to template editing in WYSIWYG. It's not great, but people
sort of cope. How did they get there? What can be done to make it
better, *conceptually*?

What I'm saying there is that we don't start from the assumption that
we know nothing and have to start from scratch, forming our answers
only from pure application of personal brilliance; we should start
from the assumption that we know actually quite a bit, if we only know
who to ask and where. Does it require throwing out all previous work?
etc., etc. And this is the sort of question that requires actual
expense on resources to answer.

Given that considerable work has gone on already, what would we do
with resources to apply to the problem?


- d.

[Brion Vibber]

On Tue, Dec 28, 2010 at 8:43 AM, David Gerard <***@gmail.com> wrote:

> e.g. Wikia has WYSIWYG editing and templates. They have a sort of
> solution to template editing in WYSIWYG. It's not great, but people
> sort of cope. How did they get there? What can be done to make it
> better, *conceptually*?
>
> What I'm saying there is that we don't start from the assumption that
> we know nothing and have to start from scratch, forming our answers
> only from pure application of personal brilliance; we should start
> from the assumption that we know actually quite a bit, if we only know
> who to ask and where. Does it require throwing out all previous work?
> etc., etc. And this is the sort of question that requires actual
> expense on resources to answer.
>
> Given that considerable work has gone on already, what would we do
> with resources to apply to the problem?
>

My primary interest at the moment in this area is to reframe the question a
bit; rather than "how do we make good WYSIWYG that works on the way
Wikipedia pages' markup and templates are structured now" -- which we know
has been extremely hard to get going -- to instead consider "how do we make
good WYSIWYG that does the sorts of things we currently use markup and
templates for, plus the things we wish we could do that we can't?"

We have indeed learned a *huge* amount from the last decade of Wikipedia and
friends, among them:

* authors and readers crave advanced systems for data & format-sharing (eg
putting structured info into infoboxes) and interactive features (even just
sticking a marker on a map!)
* most authors prefer simplicity of editing (keep the complicated stuff out
of the way until you need it)
* some authors will happily dive into hardcore coding to create the tools
they need (templates, user/site JS, gadgets)
* many other authors will very happily use those tools once they're created
* the less the guts of those tools are exposed, the easier it is for other
people to reuse them


The incredible creativity of Wikimedians in extending the frontend
capabilities of MediaWiki through custom JavaScript, and the markup system
through templates, has been blowing my mind for years. I want to find a way
to point that creativity straight forward, as it were, and use it to kick
some ass. :)


Within the Wikimedia ecosystem, we can roughly divide the world into
"Wikipedia" and "all the other projects". MediaWiki was created for
Wikipedia, based on previous software that had been adapted to the needs of
Wikipedia; and while the editing and template systems are sometimes awkward,
they work.

Our other projects like Commons, Wiktionary, Wikibooks, Wikiversity, and
Wikinews have *never* been as well served. The freeform markup model --
which works very well for body text on Wikipedia even if it's icky for
creating tables, diagrams and information sets -- has been a poorer fit, and
little effort has been spent on actually creating ways to support them well.

Commons needs better tools for annotating and grouping media resources.

Wiktionary needs structured data with editing and search tools geared
towards it.

Wikibooks needs a structure model that's based on groups of pages and media
resources, instead of just standalone freetext articles which may happen to
link to each other.

Wikiversity needs all those, and more interactive features and the ability
for users to group themselves socially and work together.


Getting anything done that would work on the huge, well-developed,
wildly-popular Wikipedia has always been a non-starter because it has to
deal with 10 years of backwards-compatibility from the get-go. I think it's
going to be a *lot* easier to get things going on those smaller projects
which are now so poorly served that most people don't even know they exist.
:)

This isn't a problem specific to Wikimedia; established organizations of all
sorts have a very difficult time getting new ideas over that hump from "not
good enough for our core needs" to "*bam* slap it everywhere". By
concentrating on the areas that aren't served at all well by the current
system, we can make much greater headway in the early stages of development;
Clayton Christensen's "The Innovator's Dilemma" calls this "competing
against non-consumption".


For the Wikipedia case, we need to incubate the next generation of
templating up to the point that they can actually undercut and replace
today's wikitext templates, or I worry we're just going to be sitting around
going "gosh I wish we could replace these templates and have markup that
works cleanly in wysiwyg" forever.


My current thoughts are to concentrate on a few areas:
1) create a widget/gadget/template/extension/plugin model built around
embedding blocks of information within a larger context...
2) ...where the data and rendering can be reasonably separate... (eg, not
having to pull tricks where you manually mix different levels of table
templates to make the infobox work right)
3) ...and the rendering can be as simple, or as fancy as complex, as your
imagination and HTML5 allow.

-- brion vibber (brion @ pobox.com)

[Rob Lanphier]

Hi Brion,

Thanks for laying out the problem so clearly! I agree wholeheartedly
that we need to avoid thinking about this problem too narrowly as a
user interface issue on top of existing markup+templates. More
inline:

On Tue, Dec 28, 2010 at 9:27 AM, Brion Vibber <***@pobox.com> wrote:
> This isn't a problem specific to Wikimedia; established organizations of all
> sorts have a very difficult time getting new ideas over that hump from "not
> good enough for our core needs" to "*bam* slap it everywhere". By
> concentrating on the areas that aren't served at all well by the current
> system, we can make much greater headway in the early stages of development;
> Clayton Christensen's "The Innovator's Dilemma" calls this "competing
> against non-consumption".

Thankfully, we at least we're not trying to defend a business model
and cost structure that's fundamentally incompatible with making a
change here. However, I know that's not the part that you're
highlighting, and I agree that Christensen's "competing against
non-consumption" concept is well worth learning about in this
context[1], as well as the concepts of "disruptive innovation" vs
"continuous innovation"[2]. As you've said, we've learned a lot in
the past decade of Wikipedia about how people use our the technology.
A new editing model that incorporates that learning will almost
certainly take a while to reach full parity in flexibility, power, and
performance. The current editor base of English Wikipedia probably
won't be patient with any changes that result in a loss of
flexibility, power and performance. Furthermore, many (perhaps even
most) things we'd be inclined to try would *not* have a measurable and
traceable impact on new editor acquisition and retention, which will
further diminish patience. A mature project like Wikipedia is a hard
place to hunt for willing guinea pigs.

> For the Wikipedia case, we need to incubate the next generation of
> templating up to the point that they can actually undercut and replace
> today's wikitext templates, or I worry we're just going to be sitting around
> going "gosh I wish we could replace these templates and have markup that
> works cleanly in wysiwyg" forever.
>
> My current thoughts are to concentrate on a few areas:
> 1) create a widget/gadget/template/extension/plugin model built around
> embedding blocks of information within a larger context...
> 2) ...where the data and rendering can be reasonably separate... (eg, not
> having to pull tricks where you manually mix different levels of table
> templates to make the infobox work right)
> 3) ...and the rendering can be as simple, or as fancy as complex, as your
> imagination and HTML5 allow.

Let me riff on what you're saying here (partly just to confirm that I
understand fully what you're saying). It'd be very cool to have the
ability to declare a single article, or probably more helpfully, a
single revision of an article to use a completely different syntax.
There's already technically a kludgy model for that now: wrap the
whole thing in a tag, and put the parser for the new syntax in a tag
extension. That said, it would probably exacerbate our problems if we
allowed intermixing of old syntax and new syntax in a single revision.
The goal should be to move articles irreversibly toward a new model,
and I don't think it'd be possible to do this without the tools to
prevent us from backsliding (for example, tools that allow editors to
convert an article from old syntax to new syntax, and also tools that
allow administrators to lock down the syntax choice for an article
without locking down the article).

Still, it's pretty alluring to think about the upgrade of syntax as an
incremental problem within an article. We could figure out how to
solve one little corner of the data/rendering separation problem and
then move on to the next. For example, we could start with citations
and make sure it's possible to insert citations easily and cleanly,
and to extract citations from an article without relying on scraping
the HTML to get them. Or maybe we do that certain types of infoboxes
instead, and then gradually get more general. We can take advantage
of the fact that we've got millions of articles to help us choose
which particular types of data will benefit from a targeted approach,
and tailor extensions to very specific data problems, and then
generalize after we sort out what works/doesn't work with a few
specific cases.

So, which problem first?

Rob

[1] Those with an aversion to business-speak will require steely
fortitude to even click on the url, let alone actually read the
article, but it's still worth extracting the non-business points from
this article:
http://businessinnovationfactory.com/weblog/christensen_worldinnovationforum
[2] While there is a Wikipedia article describing this[3], a better
description of the important bits is here:
http://www.mail-archive.com/***@haskell.org/msg18498.html
[3] Whee, footnote to a footnote!
http://en.wikipedia.org/wiki/Disruptive_technology

[Brion Vibber]

On Tue, Dec 28, 2010 at 5:28 PM, Rob Lanphier <***@wikimedia.org> wrote:

> Let me riff on what you're saying here (partly just to confirm that I
> understand fully what you're saying). It'd be very cool to have the
> ability to declare a single article, or probably more helpfully, a
> single revision of an article to use a completely different syntax.
>

Yes, though I'd recommend jettisoning the word "syntax" entirely from the
discussion at this stage, as I worry it distracts towards bikeshedding about
unimportant details.

Rather, it could be more useful to primarily think of data resources having
"features" or "structure". With images for instance, we don't make people
pay too much attention about whether something's in JPEG, PNG, GIF, or SVG
format.

At the level of actual people working with the system, the file's *format*
is completely unimportant -- only its features and metadata are relevant.
Set a size, give a caption, specify a page if it's a paged format, or a time
if it's a video format. Is it TIFF or PDF? Ogg Theora or WebM? Don't know,
don't care, and any time a user has to worry about it we've let them down.

We need to think about similarly concentrating on document structure rather
than markup syntax for text pages.


I definitely agree that the idea of progressively moving bits and pieces in
that direction is a wise one. If we can devise a *document structure* that
lets us embed magic templatey _things_ into a paragraph-oriented-text
document and maintain their structural identity all the way to browser-ready
HTML and back, then we can have a useful migration path:

* identify possibly unsafe uses of templates, extensions, and
parserfunctions (machines are great at this!)
* clean them up bit by bit (bots are often good at many common cases)
* once a page can be confirmed as not using Weird Template Magic, but only
using templates/images/plugins that fit within the structure, it's golden.
* depending on which flavor of overlords we have, we might have various ways
of enforcing that a page will always *remain* well-structured from then on.

That might not even involve changing syntax per se -- we shouldn't care too
much about whether italic is <i> or ''. But knowing where a table or a div
block starts and ends reliably is extremely important to being able to tell
which part of your document is which.


And heck, even if not everything gets fixed along that kind of path, just
being able to *have* pages and other resource types that *are*
well-structured mixed into the system is going to be hugely useful for the
non-Wikipedia projects.

-- brion vibber (brion @ pobox.com)

[Billinghurst]

(Lying on the ground in the foetal position sobbing gently ... poor poor Wiksource,
forgotten again.)

Wikisource - we have tried to get the source and structure by regulating the spaces that
we can, however, formalising template fields to forms would be great ...

* extension for DynamicPageList (previously rejected)
* search engines that work with transcluded text
* extension for music notation (Lilypond?)
* pdf text extraction tool to be implemented
* good metadata & tools
* bibliographic tools, especially tools that allow sister cross-references
* book-making tools that work with transcluded text
* tools that allow "What links here" across all of WMF
...

Hell, I could even see that text from WS references could be framed and transcluded to WP,
and provide a ready link back to the works at the sites. Same for WQ to transclude quotes
from a WS reference text, ready links from Wiktionary to usage in WS books. That should
be the value of a wiki and sister sites.

Regards, Andrew


On 28 Dec 2010 at 9:27, Brion Vibber wrote:

> On Tue, Dec 28, 2010 at 8:43 AM, David Gerard <***@gmail.com> wrote:
>
> > e.g. Wikia has WYSIWYG editing and templates. They have a sort of
> > solution to template editing in WYSIWYG. It's not great, but people
> > sort of cope. How did they get there? What can be done to make it
> > better, *conceptually*?
> >
> > What I'm saying there is that we don't start from the assumption that
> > we know nothing and have to start from scratch, forming our answers
> > only from pure application of personal brilliance; we should start
> > from the assumption that we know actually quite a bit, if we only know
> > who to ask and where. Does it require throwing out all previous work?
> > etc., etc. And this is the sort of question that requires actual
> > expense on resources to answer.
> >
> > Given that considerable work has gone on already, what would we do
> > with resources to apply to the problem?
> >
>
> My primary interest at the moment in this area is to reframe the question a
> bit; rather than "how do we make good WYSIWYG that works on the way
> Wikipedia pages' markup and templates are structured now" -- which we know
> has been extremely hard to get going -- to instead consider "how do we make
> good WYSIWYG that does the sorts of things we currently use markup and
> templates for, plus the things we wish we could do that we can't?"
>
> We have indeed learned a *huge* amount from the last decade of Wikipedia and
> friends, among them:
>
> * authors and readers crave advanced systems for data & format-sharing (eg
> putting structured info into infoboxes) and interactive features (even just
> sticking a marker on a map!)
> * most authors prefer simplicity of editing (keep the complicated stuff out
> of the way until you need it)
> * some authors will happily dive into hardcore coding to create the tools
> they need (templates, user/site JS, gadgets)
> * many other authors will very happily use those tools once they're created
> * the less the guts of those tools are exposed, the easier it is for other
> people to reuse them
>
>
> The incredible creativity of Wikimedians in extending the frontend
> capabilities of MediaWiki through custom JavaScript, and the markup system
> through templates, has been blowing my mind for years. I want to find a way
> to point that creativity straight forward, as it were, and use it to kick
> some ass. :)
>
>
> Within the Wikimedia ecosystem, we can roughly divide the world into
> "Wikipedia" and "all the other projects". MediaWiki was created for
> Wikipedia, based on previous software that had been adapted to the needs of
> Wikipedia; and while the editing and template systems are sometimes awkward,
> they work.
>
> Our other projects like Commons, Wiktionary, Wikibooks, Wikiversity, and
> Wikinews have *never* been as well served. The freeform markup model --
> which works very well for body text on Wikipedia even if it's icky for
> creating tables, diagrams and information sets -- has been a poorer fit, and
> little effort has been spent on actually creating ways to support them well.
>
> Commons needs better tools for annotating and grouping media resources.
>
> Wiktionary needs structured data with editing and search tools geared
> towards it.
>
> Wikibooks needs a structure model that's based on groups of pages and media
> resources, instead of just standalone freetext articles which may happen to
> link to each other.
>
> Wikiversity needs all those, and more interactive features and the ability
> for users to group themselves socially and work together.
>
>
> Getting anything done that would work on the huge, well-developed,
> wildly-popular Wikipedia has always been a non-starter because it has to
> deal with 10 years of backwards-compatibility from the get-go. I think it's
> going to be a *lot* easier to get things going on those smaller projects
> which are now so poorly served that most people don't even know they exist.
> :)
>
> This isn't a problem specific to Wikimedia; established organizations of all
> sorts have a very difficult time getting new ideas over that hump from "not
> good enough for our core needs" to "*bam* slap it everywhere". By
> concentrating on the areas that aren't served at all well by the current
> system, we can make much greater headway in the early stages of development;
> Clayton Christensen's "The Innovator's Dilemma" calls this "competing
> against non-consumption".
>
>
> For the Wikipedia case, we need to incubate the next generation of
> templating up to the point that they can actually undercut and replace
> today's wikitext templates, or I worry we're just going to be sitting around
> going "gosh I wish we could replace these templates and have markup that
> works cleanly in wysiwyg" forever.
>
>
> My current thoughts are to concentrate on a few areas:
> 1) create a widget/gadget/template/extension/plugin model built around
> embedding blocks of information within a larger context...
> 2) ...where the data and rendering can be reasonably separate... (eg, not
> having to pull tricks where you manually mix different levels of table
> templates to make the infobox work right)
> 3) ...and the rendering can be as simple, or as fancy as complex, as your
> imagination and HTML5 allow.
>
> -- brion vibber (brion @ pobox.com)
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-***@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[David Gerard]

On 28 December 2010 16:54, Stephanie Daugherty <sdaugherty-***@public.gmane.org> wrote:

> Not only is the current markup a barrier to participation, it's a barrier to
> development. As I argued on Wikien-l, starting over with a markup that can
> be syntacticly validated, preferably one that is XML based would reap huge
> rewards in the safety and effectiveness of automated tools - authors of
> tools like AWB have just as much trouble making software handle the corner
> cases in wikitext markup as new editors have understanding it.


In every discussion so far, throwing out wikitext and replacing it
with something that isn't a crawling horror has been considered a
non-starter, given ten years and terabytes of legacy wikitext.

If you think you can swing throwing out wikitext and barring the
actual code from human editing - XML is not safely human editable in
any circumstances - then good luck to you, but I don't like your
chances.


- d.

[George Herbert]

On Tue, Dec 28, 2010 at 3:43 PM, David Gerard <dgerard-***@public.gmane.org> wrote:
> On 28 December 2010 16:54, Stephanie Daugherty <sdaugherty-***@public.gmane.org> wrote:
>
>> Not only is the current markup a barrier to participation, it's a barrier to
>> development. As I argued on Wikien-l, starting over with a markup that can
>> be syntacticly validated, preferably one that is XML based would reap huge
>> rewards in the safety and effectiveness of automated tools - authors of
>> tools like AWB have just as much trouble making software handle the corner
>> cases in wikitext markup as new editors have understanding it.
>
>
> In every discussion so far, throwing out wikitext and replacing it
> with something that isn't a crawling horror has been considered a
> non-starter, given ten years and terabytes of legacy wikitext.
>
> If you think you can swing throwing out wikitext and barring the
> actual code from human editing - XML is not safely human editable in
> any circumstances - then good luck to you, but I don't like your
> chances.

That is true - "We can't do away with Wikitext" always been the
intermediate conclusion (in between "My god, we need to do something
about this problem" and "This is hopeless, we give up again").

Perhaps it's time to start some exercises in noneuclidian Wiki
development, and just assume the opposite and see what happens.


--
-george william herbert
george.herbert-***@public.gmane.org

[masti]

> That is true - "We can't do away with Wikitext" always been the
> intermediate conclusion (in between "My god, we need to do something
> about this problem" and "This is hopeless, we give up again").
>

between wikitext and WYSISWYG is a simple solution of colourizing text
like for hundreds of programing (and otehr) languages formats in a
simple text editor. This is not a rocket science but we still do not
have it :(

masti

[Platonides]

masti wrote:
>> That is true - "We can't do away with Wikitext" always been the
>> intermediate conclusion (in between "My god, we need to do something
>> about this problem" and "This is hopeless, we give up again").
>>
>
> between wikitext and WYSISWYG is a simple solution of colourizing text
> like for hundreds of programing (and otehr) languages formats in a
> simple text editor. This is not a rocket science but we still do not
> have it :(
>
> masti

Have you tried wikiEd ?

[masti]

On 12/31/2010 01:19 AM, Platonides wrote:
> masti wrote:
>>> That is true - "We can't do away with Wikitext" always been the
>>> intermediate conclusion (in between "My god, we need to do something
>>> about this problem" and "This is hopeless, we give up again").
>>>
>>
>> between wikitext and WYSISWYG is a simple solution of colourizing text
>> like for hundreds of programing (and otehr) languages formats in a
>> simple text editor. This is not a rocket science but we still do not
>> have it :(
>>
>> masti
>
> Have you tried wikiEd ?

yes :(

why not have this functionality in wikiedit window?

[Maciej Jaros]

masti (2010-12-31 01:33):
> On 12/31/2010 01:19 AM, Platonides wrote:
>> masti wrote:
>>>> That is true - "We can't do away with Wikitext" always been the
>>>> intermediate conclusion (in between "My god, we need to do something
>>>> about this problem" and "This is hopeless, we give up again").
>>>>
>>> between wikitext and WYSISWYG is a simple solution of colourizing text
>>> like for hundreds of programing (and otehr) languages formats in a
>>> simple text editor. This is not a rocket science but we still do not
>>> have it :(
>>>
>>> masti
>> Have you tried wikiEd ?
> yes :(
>
> why not have this functionality in wikiedit window?

Because wikEd is wicked ;-) and spoils simple editor window so that many
other scripts have problems using it. Also probably because it's not
really helpful for all users to work with and certainly is making your
computer work slower.

Regards,
Nux.

[Platonides]

masti wrote:
>> That is true - "We can't do away with Wikitext" always been the
>> intermediate conclusion (in between "My god, we need to do something
>> about this problem" and "This is hopeless, we give up again").
>>
>
> between wikitext and WYSISWYG is a simple solution of colourizing text
> like for hundreds of programing (and otehr) languages formats in a
> simple text editor. This is not a rocket science but we still do not
> have it :(
>
> masti

Have you tried wikiEd ?

[Dmitriy Sintsov]

* David Gerard <***@gmail.com> [Tue, 28 Dec 2010 23:43:14 +0000]:
> On 28 December 2010 16:54, Stephanie Daugherty <***@gmail.com>
> wrote:
>
> > Not only is the current markup a barrier to participation, it's a
> barrier to
> > development. As I argued on Wikien-l, starting over with a markup
that
> can
> > be syntacticly validated, preferably one that is XML based would
reap
> huge
> > rewards in the safety and effectiveness of automated tools - authors
> of
> > tools like AWB have just as much trouble making software handle the
> corner
> > cases in wikitext markup as new editors have understanding it.
>
>
> In every discussion so far, throwing out wikitext and replacing it
> with something that isn't a crawling horror has been considered a
> non-starter, given ten years and terabytes of legacy wikitext.
>
> If you think you can swing throwing out wikitext and barring the
> actual code from human editing - XML is not safely human editable in
> any circumstances - then good luck to you, but I don't like your
> chances.
>
New templating could be implemented in parallel - not having to abandon
these terabytes. Inserting something like XSL to a wikipage should be
orthogonal to wiki templating syntax (only some of tag hook names will
be unavailable due to XSL using these). However, I do agree to you that
XML (or XSL) tag editing is a hard job, sometimes even harder than
wikitext. Wikitext is really fast way to build articles to people who
type fast enough. Wikitext for markup and links and XSL for templates,
perhaps. There is also a HEREDOC style for almost arbitrary content.
Many possible ways to have two languages in parallel.
Dmitriy

[Philip Tzou]

Can we use a Javascript based parser to convert the wiki markup language to
some intermediate object, which can be easily converted to both languages
(wiki markup language and HTML)? I think JQuery object is a good idea. We
can extend JQuery to include such language conversion methods.

[David Gerard]

On 29 December 2010 16:20, Philip Tzou <***@gmail.com> wrote:

> Can we use a Javascript based parser to convert the wiki markup language to
> some intermediate object, which can be easily converted to both languages
> (wiki markup language and HTML)? I think JQuery object is a good idea. We
> can extend JQuery to include such language conversion methods.


Magnus has already written a quick toy to this effect, "WYSIWTF":

http://lists.wikimedia.org/pipermail/wikien-l/2010-December/108090.html

In later messages he lists various possible paths for expansion.


- d.

[Happy-melon]

There are some things that we know:

1) as Brion says, MediaWiki currently only presents content in one way: as
wikitext run through the parser. He may well be right that there is a
bigger fish which could be caught than WYSIWYG editing by saying that MW
should present data in other new and exciting ways, but that's actually a
separate question. *If* you wish to solve WYSIWYG editing, your baseline is
wikitext and the parser.

2) "guacamole" is one of the more unusual descriptors I've heard for the
parser, but it's far from the worst. We all agree that it's horribly messy
and most developers treat it like either a sleeping dragon or a *very*
grumpy neighbour. I'd say that the two biggest problems with it are that a)
it's buried so deep in the codebase that literally the only way to get your
wikitext parsed is to fire up the whole of the rest of MediaWiki around it
to give it somewhere comfy to live in, and b) there is as David says no way
of explaining what it's supposed to be doing except saying "follow the code;
whatever it does is what it's supposed to do". It seems to be generally
accepted that it is *impossible* to represent everything the parser does in
any standard grammar.

Those are all standard gripes, and nothing new or exciting. There are also,
to quote a much-abused former world leader, some known unknowns:

1) we don't know how to explain What You See when you parse wikitext except
by prodding an exceedingly grumpy hundred thousand lines of PHP and *asking
What it thinks* You Get.

2) We don't know how to create a WYSIWYG editor for wikitext.

Now, I'd say we have some unknown unknowns.

1) *is* it because of wikitext's idiosyncracies that WYSIWYG is so
difficult? Is wikitext *by its nature* not amenable to WYSIWYG editing?

2) would a wikitext which *was* representable in a standard grammar be
amenable to WYSIWYG editing?

3) would a wikitext which had an alternative parser, one that was not buried
in the depths of MW (perhaps a full JS library that could be called in
real-time on the client), be amenable to WYSIWYG editing?

4) are questions 2 and 3 synonymous?

--HM


"David Gerard" <***@gmail.com> wrote in
message news:AANLkTimthUx-UndO1CTnexcRqbPP89t2M-***@mail.gmail.com...
> [crossposted to foundation-l and wikitech-l]
>
>
> "There has to be a vision though, of something better. Maybe something
> that is an actual wiki, quick and easy, rather than the template
> coding hell Wikipedia's turned into." - something Fred Bauder just
> said on wikien-l.
>
>
> Our current markup is one of our biggest barriers to participation.
>
> AIUI, edit rates are about half what they were in 2005, even as our
> fame has gone from "popular" through "famous" to "part of the
> structure of the world." I submit that this is not a good or healthy
> thing in any way and needs fixing.
>
> People who can handle wikitext really just do not understand how
> offputting the computer guacamole is to people who can cope with text
> they can see.
>
> We know this is a problem; WYSIWYG that works is something that's been
> wanted here forever. There are various hideous technical nightmares in
> its way, that make this a big and hairy problem, of the sort where the
> hair has hair.
>
> However, I submit that it's important enough we need to attack it with
> actual resources anyway.
>
>
> This is just one data point, where a Canadian government office got
> *EIGHT TIMES* the participation in their intranet wiki by putting in a
> (heavily locally patched) copy of FCKeditor:
>
>
> http://lists.wikimedia.org/pipermail/mediawiki-l/2010-May/034062.html
>
> "I have to disagree with you given my experience. In one government
> department where MediaWiki was installed we saw the active user base
> spike from about 1000 users to about 8000 users within a month of having
> enabled FCKeditor. FCKeditor definitely has it's warts, but it very
> closely matches the experience non-technical people have gotten used to
> while using Word or WordPerfect. Leveraging skills people already have
> cuts down on training costs and allows them to be productive almost
> immediately."
>
> http://lists.wikimedia.org/pipermail/mediawiki-l/2010-May/034071.html
>
> "Since a plethora of intelligent people with no desire to learn WikiCode
> can now add content, the quality of posts has been in line with the
> adoption of wiki use by these people. Thus one would say it has gone up.
>
> "In the beginning there were some hard core users that learned WikiCode,
> for the most part they have indicated that when the WYSIWYG fails, they
> are able to switch to WikiCode mode to address the problem. This usually
> occurs with complex table nesting which is something that few of the
> users do anyways. Most document layouts are kept simple. Additionally,
> we have a multilingual english/french wiki. As a result the browser
> spell-check is insufficient for the most part (not to mention it has
> issues with WikiCode). To address this a second spellcheck button was
> added to the interface so that both english and french spellcheck could
> be available within the same interface (via aspell backend)."
>
>
> So, the payoffs could be ridiculously huge: eight times the number of
> smart and knowledgeable people even being able to *fix typos* on
> material they care about.
>
> Here are some problems. (Off the top of my head; please do add more,
> all you can think of.)
>
>
> - The problem:
>
> * Fidelity with the existing body of wikitext. No conversion flag day.
> The current body exploits every possible edge case in the regular
> expression guacamole we call a "parser". Tim said a few years ago that
> any solution has to account for the existing body of text.
>
> * Two-way fidelity. Those who know wikitext will demand to keep it and
> will bitterly resist any attempt to take it away from them.
>
> * FCKeditor (now CKeditor) in MediaWiki is all but unmaintained.
>
> * There is no specification for wikitext. Well, there almost is -
> compiled as C, it runs a bit slower than the existing PHP compiler.
> But it's a start!
> http://lists.wikimedia.org/pipermail/wikitext-l/2010-August/000318.html
>
>
> - Attempting to solve it:
>
> * The best brains around Wikipedia, MediaWiki and WMF have dashed
> their foreheads against this problem for at least the past five years
> and have got *nowhere*. Tim has a whole section in the SVN repository
> for "new parser attempts". Sheer brilliance isn't going to solve this
> one.
>
> * Tim doesn't scale. Most of our other technical people don't scale.
> *We have no resources and still run on almost nothing*.
>
> ($14m might sound like enough money to run a popular website, but for
> comparison, I work as a sysadmin at a tiny, tiny publishing company
> with more money and staff just in our department than that to do
> *almost nothing* compared to what WMF achieves. WMF is an INCREDIBLY
> efficient organisation.)
>
>
> - Other attempts:
>
> * Starting from a clear field makes it ridiculously easy. The
> government example quoted above is one. Wikia wrote a good WYSIWYG
> that works really nicely on new wikis (I'm speaking here as an
> experienced wikitext user who happily fixes random typos on Wikia). Of
> course, I noted that we can't start from a clear field - we have an
> existing body of wikitext.
>
>
> So, specification of the problem:
>
> * We need good WYSIWYG. The government example suggests that a simple
> word-processor-like interface would be enough to give tremendous
> results.
> * It needs two-way fidelity with almost all existing wikitext.
> * We can't throw away existing wikitext, much as we'd love to.
> * It's going to cost money in programming the WYSIWYG.
> * It's going to cost money in rationalising existing wikitext so that
> the most unfeasible formations can be shunted off to legacy for
> chewing on.
> * It's going to cost money in usability testing and so on.
> * It's going to cost money for all sorts of things I haven't even
> thought of yet.
>
>
> This is a problem that would pay off hugely to solve, and that will
> take actual money thrown at it.
>
> How would you attack this problem, given actual resources for grunt work?
>
>
> - d.
>
> _______________________________________________
> foundation-l mailing list
> foundation-***@lists.wikimedia.org
> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>

[Krinkle]

Hi,

When this topic was raised before a few years ago (I dont remember
which time,
it's been continuingly discussed throughout the years) I found an idea
especially
interesting but it got buried in the mass.

From memory and imagination:

The idea is to write a new parser that is not deep in MediaWiki and
can therefor be used apart from
MediaWiki and is fairly easy to be translated to, for example,
javascript.

This parser accepts similar input as we do now (ie. '''bold''',
{{template}}, [[link|text]] etc.)
however totally rewritten and with more logical behavour. Call it a
2.0 parser without any
worries about compatibilty or old wikitext edge cases which (ab)use
the edge cases of the current parser.

This would become the default in MediaWiki for new pages created, and
indicated by an int
in the revision table (ie. rev_pv (parserversion) ). A WYSIWYG editor
can be written for this
in javascript and it's great.

So what about articles with the old paser (ie. rev_pv=NULL /
rev_pv=1) ? No problem,
the old parser stick around for a while and such articles simply dont
have a WYSIWYG editor.

Editing articles with the old parser will show a small notice on top
(like the one for pages larger than
x bytes due to old browser limits) showing an option 'switch' it. That
would result in previewing the page's wikitext
with the new parser. The user can then make adjuistment as needed to
make it look good again (if neccecary at all)
and save page (which saves the new revision with rev_pv=2, like it
would do for new articles).

Since there are lots of articles which likely will have the same
output in HTML and require no modification whatshowever
there could be a script written (either as a userbot for the end user
or as a maintenance script) that would automatically check
all pages that have the old rev_pv and compare them to the output of
the new parser and automatically update the rev_rv
field if it matches. All others would be visible on a SpecialPage for
"pages of which the last revision has an older version of the parser",
with a link to an MW.org page with an overview of a few things that
regulars may want to know (ie. the most common differences).

Just an idea :)
--
Krinkle

[Andrew Dunbar]

On 29 December 2010 02:07, Happy-melon <happy-***@live.com> wrote:
> There are some things that we know:
>
> 1) as Brion says, MediaWiki currently only presents content in one way: as
> wikitext run through the parser.  He may well be right that there is a
> bigger fish which could be caught than WYSIWYG editing by saying that MW
> should present data in other new and exciting ways, but that's actually a
> separate question.  *If* you wish to solve WYSIWYG editing, your baseline is
> wikitext and the parser.

Specifically, it only presents content as HTML. It's not really a
parser because it doesn't create an AST (Abstract Syntax Tree). It's a
wikitext to HTML converter. The flavour of the HTML can be somewhat
modulated by the skin but it could never output directly to something
totally different like RTF or PDF.

> 2) "guacamole" is one of the more unusual descriptors I've heard for the
> parser, but it's far from the worst.  We all agree that it's horribly messy
> and most developers treat it like either a sleeping dragon or a *very*
> grumpy neighbour.  I'd say that the two biggest problems with it are that a)
> it's buried so deep in the codebase that literally the only way to get your
> wikitext parsed is to fire up the whole of the rest of MediaWiki around it
> to give it somewhere comfy to live in,

I have started to advocate the isolation of the parser from the rest
of the innards or MediaWiki for just this reason:
https://bugzilla.wikimedia.org/show_bug.cgi?id=25984

Free it up so that anybody can embed it in their code and get exactly
the same rendering that Wikipedia et al get, guaranteed.

We have to find all the edges where the parser calls other parts of
MediaWiki and all the edges where other parts of MediaWiki call the
parser. We then define these edges as interfaces so that we can drop
an alternative parser into MediaWiki and drop the current parser into
say an offline viewer or whatever.

With a freed up parser more people will hack on it, more people will
come to grok it and come up with strategies to address some of its
problems. It should also be a boon for unit testing.

(I have a very rough prototype working by the way with lots of stub classes)

> and b) there is as David says no way
> of explaining what it's supposed to be doing except saying "follow the code;
> whatever it does is what it's supposed to do".  It seems to be generally
> accepted that it is *impossible* to represent everything the parser does in
> any standard grammar.

I've thought a lot about this too. It certainly is not any type of
standard grammar. But on the other hand it is a pretty common kind of
nonstandard grammar. I call it a "recursive text replacement grammar".

Perhaps this type of grammar has some useful characteristics we can
discover and document. It may be possible to follow the code flow and
document each text replacement in sequence as a kind of parser spec
rather than trying and failing again to shoehorn it into a standard
LALR grammar.

If it is possible to extract such a spec it would then be possible to
implement it in other languages.

Some research may even find that is possible to transform such a
grammar deterministically into an LALR grammar...

But even if not I'm certain it would demysitfy what happens in the
parser so that problems and edge cases would be easier to locate.

Andrew Dunbar (hippietrail)

> Those are all standard gripes, and nothing new or exciting.  There are also,
> to quote a much-abused former world leader, some known unknowns:
>
> 1) we don't know how to explain What You See when you parse wikitext except
> by prodding an exceedingly grumpy hundred thousand lines of PHP and *asking
> What it thinks* You Get.
>
> 2) We don't know how to create a WYSIWYG editor for wikitext.
>
> Now, I'd say we have some unknown unknowns.
>
> 1) *is* it because of wikitext's idiosyncracies that WYSIWYG is so
> difficult?  Is wikitext *by its nature* not amenable to WYSIWYG editing?
>
> 2) would a wikitext which *was* representable in a standard grammar be
> amenable to WYSIWYG editing?
>
> 3) would a wikitext which had an alternative parser, one that was not buried
> in the depths of MW (perhaps a full JS library that could be called in
> real-time on the client), be amenable to WYSIWYG editing?
>
> 4) are questions 2 and 3 synonymous?
>
> --HM
>
>
> "David Gerard" <***@gmail.com> wrote in
> message news:AANLkTimthUx-UndO1CTnexcRqbPP89t2M-***@mail.gmail.com...
>> [crossposted to foundation-l and wikitech-l]
>>
>>
>> "There has to be a vision though, of something better. Maybe something
>> that is an actual wiki, quick and easy, rather than the template
>> coding hell Wikipedia's turned into." - something Fred Bauder just
>> said on wikien-l.
>>
>>
>> Our current markup is one of our biggest barriers to participation.
>>
>> AIUI, edit rates are about half what they were in 2005, even as our
>> fame has gone from "popular" through "famous" to "part of the
>> structure of the world." I submit that this is not a good or healthy
>> thing in any way and needs fixing.
>>
>> People who can handle wikitext really just do not understand how
>> offputting the computer guacamole is to people who can cope with text
>> they can see.
>>
>> We know this is a problem; WYSIWYG that works is something that's been
>> wanted here forever. There are various hideous technical nightmares in
>> its way, that make this a big and hairy problem, of the sort where the
>> hair has hair.
>>
>> However, I submit that it's important enough we need to attack it with
>> actual resources anyway.
>>
>>
>> This is just one data point, where a Canadian government office got
>> *EIGHT TIMES* the participation in their intranet wiki by putting in a
>> (heavily locally patched) copy of FCKeditor:
>>
>>
>>   http://lists.wikimedia.org/pipermail/mediawiki-l/2010-May/034062.html
>>
>> "I have to disagree with you given my experience. In one government
>> department where MediaWiki was installed we saw the active user base
>> spike from about 1000 users to about 8000 users within a month of having
>> enabled FCKeditor. FCKeditor definitely has it's warts, but it very
>> closely matches the experience non-technical people have gotten used to
>> while using Word or WordPerfect. Leveraging skills people already have
>> cuts down on training costs and allows them to be productive almost
>> immediately."
>>
>>   http://lists.wikimedia.org/pipermail/mediawiki-l/2010-May/034071.html
>>
>> "Since a plethora of intelligent people with no desire to learn WikiCode
>> can now add content, the quality of posts has been in line with the
>> adoption of wiki use by these people. Thus one would say it has gone up.
>>
>> "In the beginning there were some hard core users that learned WikiCode,
>> for the most part they have indicated that when the WYSIWYG fails, they
>> are able to switch to WikiCode mode to address the problem. This usually
>> occurs with complex table nesting which is something that few of the
>> users do anyways. Most document layouts are kept simple. Additionally,
>> we have a multilingual english/french wiki. As a result the browser
>> spell-check is insufficient for the most part (not to mention it has
>> issues with WikiCode). To address this a second spellcheck button was
>> added to the interface so that both english and french spellcheck could
>> be available within the same interface (via aspell backend)."
>>
>>
>> So, the payoffs could be ridiculously huge: eight times the number of
>> smart and knowledgeable people even being able to *fix typos* on
>> material they care about.
>>
>> Here are some problems. (Off the top of my head; please do add more,
>> all you can think of.)
>>
>>
>> - The problem:
>>
>> * Fidelity with the existing body of wikitext. No conversion flag day.
>> The current body exploits every possible edge case in the regular
>> expression guacamole we call a "parser". Tim said a few years ago that
>> any solution has to account for the existing body of text.
>>
>> * Two-way fidelity. Those who know wikitext will demand to keep it and
>> will bitterly resist any attempt to take it away from them.
>>
>> * FCKeditor (now CKeditor) in MediaWiki is all but unmaintained.
>>
>> * There is no specification for wikitext. Well, there almost is -
>> compiled as C, it runs a bit slower than the existing PHP compiler.
>> But it's a start!
>> http://lists.wikimedia.org/pipermail/wikitext-l/2010-August/000318.html
>>
>>
>> - Attempting to solve it:
>>
>> * The best brains around Wikipedia, MediaWiki and WMF have dashed
>> their foreheads against this problem for at least the past five years
>> and have got *nowhere*. Tim has a whole section in the SVN repository
>> for "new parser attempts". Sheer brilliance isn't going to solve this
>> one.
>>
>> * Tim doesn't scale. Most of our other technical people don't scale.
>> *We have no resources and still run on almost nothing*.
>>
>> ($14m might sound like enough money to run a popular website, but for
>> comparison, I work as a sysadmin at a tiny, tiny publishing company
>> with more money and staff just in our department than that to do
>> *almost nothing* compared to what WMF achieves. WMF is an INCREDIBLY
>> efficient organisation.)
>>
>>
>> - Other attempts:
>>
>> * Starting from a clear field makes it ridiculously easy. The
>> government example quoted above is one. Wikia wrote a good WYSIWYG
>> that works really nicely on new wikis (I'm speaking here as an
>> experienced wikitext user who happily fixes random typos on Wikia). Of
>> course, I noted that we can't start from a clear field - we have an
>> existing body of wikitext.
>>
>>
>> So, specification of the problem:
>>
>> * We need good WYSIWYG. The government example suggests that a simple
>> word-processor-like interface would be enough to give tremendous
>> results.
>> * It needs two-way fidelity with almost all existing wikitext.
>> * We can't throw away existing wikitext, much as we'd love to.
>> * It's going to cost money in programming the WYSIWYG.
>> * It's going to cost money in rationalising existing wikitext so that
>> the most unfeasible formations can be shunted off to legacy for
>> chewing on.
>> * It's going to cost money in usability testing and so on.
>> * It's going to cost money for all sorts of things I haven't even
>> thought of yet.
>>
>>
>> This is a problem that would pay off hugely to solve, and that will
>> take actual money thrown at it.
>>
>> How would you attack this problem, given actual resources for grunt work?
>>
>>
>> - d.
>>
>> _______________________________________________
>> foundation-l mailing list
>> foundation-***@lists.wikimedia.org
>> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/foundation-l
>>
>
>
>
>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-***@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>

[Andreas Jonsson]

2010-12-29 08:33, Andrew Dunbar skrev:
> I've thought a lot about this too. It certainly is not any type of
> standard grammar. But on the other hand it is a pretty common kind of
> nonstandard grammar. I call it a "recursive text replacement grammar".
>
> Perhaps this type of grammar has some useful characteristics we can
> discover and document. It may be possible to follow the code flow and
> document each text replacement in sequence as a kind of parser spec
> rather than trying and failing again to shoehorn it into a standard
> LALR grammar.
>
> If it is possible to extract such a spec it would then be possible to
> implement it in other languages.
>
> Some research may even find that is possible to transform such a
> grammar deterministically into an LALR grammar...
>
> But even if not I'm certain it would demysitfy what happens in the
> parser so that problems and edge cases would be easier to locate.
>
From my experience of implementing a wikitext parser, I would say that
it might be possible to transform wikitext to a token stream that is
possible to parse with a LALR parser. My implementation
(http://svn.wikimedia.org/svnroot/mediawiki/trunk/parsers/libmwparser)
uses Antlr (which is an LL parser generator) and only rely on context
sensitive parsing (Antlr's semantic predicates) for parsing
apostrophes (bold and italics), and this might be possible to solve in
a different way. The rest of the complex cases are handled by the
lexical analyser that produce a well behaving token stream that can be
relatively straightforwardly parsed.

My implementation is not 100% compatible, but I think that a 100%
compatible parser is not desirable since the most exotic border cases
would probably be characterized as "bugs" anyway (e.g. [[Link|<table
class="]]">). But I think that the basic idea can be used to produce
a sufficiently compatible parser.

Best Regards,

/Andreas

[Andrew Dunbar]

On 3 January 2011 21:54, Andreas Jonsson <***@kreablo.se> wrote:
> 2010-12-29 08:33, Andrew Dunbar skrev:
>> I've thought a lot about this too. It certainly is not any type of
>> standard grammar. But on the other hand it is a pretty common kind of
>> nonstandard grammar. I call it a "recursive text replacement grammar".
>>
>> Perhaps this type of grammar has some useful characteristics we can
>> discover and document. It may be possible to follow the code flow and
>> document each text replacement in sequence as a kind of parser spec
>> rather than trying and failing again to shoehorn it into a standard
>> LALR grammar.
>>
>> If it is possible to extract such a spec it would then be possible to
>> implement it in other languages.
>>
>> Some research may even find that is possible to transform such a
>> grammar deterministically into an LALR grammar...
>>
>> But even if not I'm certain it would demysitfy what happens in the
>> parser so that problems and edge cases would be easier to locate.
>>
> From my experience of implementing a wikitext parser, I would say that
> it might be possible to transform wikitext to a token stream that is
> possible to parse with a LALR parser.  My implementation
> (http://svn.wikimedia.org/svnroot/mediawiki/trunk/parsers/libmwparser)
> uses Antlr (which is an LL parser generator) and only rely on context
> sensitive parsing (Antlr's semantic predicates) for parsing
> apostrophes (bold and italics), and this might be possible to solve in
> a different way.  The rest of the complex cases are handled by the
> lexical analyser that produce a well behaving token stream that can be
> relatively straightforwardly parsed.
>
> My implementation is not 100% compatible, but I think that a 100%
> compatible parser is not desirable since the most exotic border cases
> would probably be characterized as "bugs" anyway (e.g. [[Link|<table
> class="]]">).  But I think that the basic idea can be used to produce
> a sufficiently compatible parser.

In that case what is needed is to hook your parser into our current code
and get it create output if you have not done that already. Then you will
want to run the existing parser tests on it. Then you will want to run both
parsers over a large sample of existing Wikipedia articles (make sure
you use the same revisions on both parsers!) and run them through diff.
Then we'll have a decent idea of whether there are any edge cases you
didn't spot or whether any of them are exploited in template magic.

Let us know the results!

Andrew Dunbar (hippietrail)


> Best Regards,
>
> /Andreas
>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-***@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>

[Neil Kandalgaonkar]

I've been inspired by the discussion David Gerard and Brion Vibber
kicked off, and I think they are headed in the right direction.

But I just want to ask a separate, but related question.

Let's imagine you wanted to start a rival to Wikipedia. Assume that you
are motivated by money, and that venture capitalists promise you can be
paid gazillions of dollars if you can do one, or many, of the following:

1 - Become a more attractive home to the WP editors. Get them to work on
your content.

2 - Take the free content from WP, and use it in this new system. But
make it much better, in a way Wikipedia can't match.

3 - Attract even more readers, or perhaps a niche group of
super-passionate readers that you can use to build a new community.

In other words, if you had no legacy, and just wanted to build something
from zero, how would you go about creating an innovation that was
disruptive to Wikipedia, in fact something that made Wikipedia look like
Friendster or Myspace compared to Facebook?

And there's a followup question to this -- but you're all smart people
and can guess what it is.

--
Neil Kandalgaonkar ( <***@wikimedia.org>

[MZMcBride]

Neil Kandalgaonkar wrote:
> Let's imagine you wanted to start a rival to Wikipedia. Assume that you
> are motivated by money, and that venture capitalists promise you can be
> paid gazillions of dollars if you can do one, or many, of the following:
>
> 1 - Become a more attractive home to the WP editors. Get them to work on
> your content.
>
> 2 - Take the free content from WP, and use it in this new system. But
> make it much better, in a way Wikipedia can't match.
>
> 3 - Attract even more readers, or perhaps a niche group of
> super-passionate readers that you can use to build a new community.
>
> In other words, if you had no legacy, and just wanted to build something
> from zero, how would you go about creating an innovation that was
> disruptive to Wikipedia, in fact something that made Wikipedia look like
> Friendster or Myspace compared to Facebook?
>
> And there's a followup question to this -- but you're all smart people
> and can guess what it is.

[quote]
The "Viable alternative to Wikipedia" isn't going to be another Mediawiki
site, in any event - it's going to be something that someone puts some real
effort into developing the software for, not to mention the user
experience...
[/quote]

http://wikipediareview.com/index.php?showtopic=31808&view=findpost&p=262671

I largely agree with that.

You can't make more than broad generalizations about what a "Wikipedia
killer" would be. If there were a concrete answer or set of answers,
Wikipedia would be dead already. A number of organizations and companies
have tried to replicate Wikipedia's success (e.g. Wikia) with varying
degrees of success. The most common factor to past Wikipedia competitors has
been MediaWiki (though if someone can refute this, please do). To me (and
others), that leaves the question of what would happen if you wrote some
software that was actually built for making an encyclopedia, rather than the
jack of all trades product that MediaWiki is.

As for follow-up questions, be explicit.

MZMcBride

[Alex Brollo]

2010/12/29 MZMcBride <***@mzmcbride.com>

> Neil Kandalgaonkar wrote:
> > Let's imagine you wanted to start a rival to Wikipedia. Assume that you
> > are motivated by money, and that venture capitalists promise you can be
> > paid gazillions of dollars if you can do one, or many, of the following:
> >
> > 1 - Become a more attractive home to the WP editors. Get them to work on
> > your content.
> >
> > 2 - Take the free content from WP, and use it in this new system. But
> > make it much better, in a way Wikipedia can't match.
> >
> > 3 - Attract even more readers, or perhaps a niche group of
> > super-passionate readers that you can use to build a new community.
> >
> > In other words, if you had no legacy, and just wanted to build something
> > from zero, how would you go about creating an innovation that was
> > disruptive to Wikipedia, in fact something that made Wikipedia look like
> > Friendster or Myspace compared to Facebook?
> >
> > And there's a followup question to this -- but you're all smart people
> > and can guess what it is.
>

It's simply evolution rule! The day this would happen - that something will
appear, collecting all the best from wiki, adding too something better and
successful - wiki will slowly disappear. But all the better of wiki will
survive into the emerging "species".... where's the problem, you you don't
consider wiki in terms of competition, but in terms of utiliy? I'm actively
working for wiki principles, not at all for wiki project! I hope, that this
will be not considered offensive for wiki community.

Alex

[David Gerard]

On 29 December 2010 08:24, MZMcBride <***@mzmcbride.com> wrote:

> To me (and
> others), that leaves the question of what would happen if you wrote some
> software that was actually built for making an encyclopedia, rather than the
> jack of all trades product that MediaWiki is.


MediaWiki is precisely that software. And there's any number of
specialist wikis using it that are basically Wikipedia in a specialist
area.

This sounds like "software that looks to me on the surface like it was
actually built for making an encyclopedia". This is, of course, not at
all the same as success.

Note the number of competitors, forks and comolements that have
already beached having assumed "I can do a better Wikipedia if it fits
my idea of how an encyclopedia *should* look" and been dead wrong.
You're reasoning from the assumption of no knowledge, rather than one
of considerable knowledge.


- d.

[MZMcBride]

David Gerard wrote:
> On 29 December 2010 08:24, MZMcBride <***@mzmcbride.com> wrote:
>
>> To me (and
>> others), that leaves the question of what would happen if you wrote some
>> software that was actually built for making an encyclopedia, rather than the
>> jack of all trades product that MediaWiki is.
>
> MediaWiki is precisely that software. And there's any number of
> specialist wikis using it that are basically Wikipedia in a specialist
> area.

No, I don't think MediaWiki is precisely that software. MediaWiki is a wiki
engine that can be used for a variety of purposes. It may have started out
as a tool to make an encyclopedia, but very shortly after its mission
drifted.

Since Wikipedia's creation, there have been countless debates about what an
"encyclopedia" is. However, at a most basic level, we can say that an
encyclopedia is its content. As an exercise, try retrieving the first
sentence of every article on the English Wikipedia. You'll quickly discover
it's a real pain in the ass. Or try extracting the birth year from every
living person's article on the English Wikipedia that uses an infobox. Even
more of a difficult task, if not an impossible one.

MediaWiki was designed to fit a number of ideas: free dictionary, free
encyclopedia, free news site, free media repo, etc. And thus its design has
been held back in many areas in order to ensure that any change doesn't
break its various use-cases.

How do you build a better Wikipedia? By building software designed to make
an encyclopedia. That leaves two options: abandon MediaWiki or re-focus
MediaWiki. The current MediaWiki will never lead to a "Wikipedia killer." I
firmly believe that.

Assuming you focused on only building a better encyclopedia, a MediaWiki 2.0
would put meta-content in a separate area, so that clicking edit doesn't
stab the user in the eye with nasty infobox content. MW2.0 would use actual
input forms for data, instead of the completely hackish hellhole that is
"[[Category:]]" and "{{Infobox |param}}". MW2.0 would standardize and
normalize template parameters to something more sane and would allow
categories to be added, removed, and moved without divine intervention (and
a working knowledge of Python). MW2.0 would have the ability to edit pages
without knowing an esoteric, confusing, and non-standardized markup.

All of this (and much more) is possible, but it requires killing the
one-size-fits-all model that allows MediaWiki to work (ehhh, function) as a
dictionary, media repository, news site, etc. For an encyclopedia, you want
to use categories and make a category interface as nice as possible, for
example. For a media repository, the categories are in[s]ane and would be
replaced by tags. And we won't begin to discuss the changes needed to make
Wiktionary not the scrambled, hacked-up mess that it currently is.

You make a "Wikipedia killer" by building software that's actually designed
to kill Wikipedia, not kill Wikipedia, Wiktionary, Wikimedia Commons,
Wikinews, and whatever else.

> This sounds like "software that looks to me on the surface like it was
> actually built for making an encyclopedia". This is, of course, not at
> all the same as success.

I'm not sure what "this" is. Can you clarify?

MZMcBride

[David Gerard]

On 29 December 2010 11:21, MZMcBride <***@mzmcbride.com> wrote:
> David Gerard wrote:

>> MediaWiki is precisely that software. And there's any number of
>> specialist wikis using it that are basically Wikipedia in a specialist
>> area.

> No, I don't think MediaWiki is precisely that software. MediaWiki is a wiki
> engine that can be used for a variety of purposes. It may have started out
> as a tool to make an encyclopedia, but very shortly after its mission
> drifted.
> MediaWiki was designed to fit a number of ideas: free dictionary, free
> encyclopedia, free news site, free media repo, etc. And thus its design has
> been held back in many areas in order to ensure that any change doesn't
> break its various use-cases.


No, it's pretty much a simple free-encyclopedia engine. Ask people on
the other projects about how hard it is to get anyone interested in
what they need.


>> This sounds like "software that looks to me on the surface like it was
>> actually built for making an encyclopedia". This is, of course, not at
>> all the same as success.

> I'm not sure what "this" is. Can you clarify?


Your original statement of what you thought was needed. I'm not
convinced that putting more policy into the engine will make a
Wikipedia killer. There's already rather a lot of
encyclopedia-directed policy in the WMF deploy.


- d.

[Neil Kandalgaonkar]

On 12/29/10 3:21 AM, MZMcBride wrote:
> David Gerard wrote:
>> On 29 December 2010 08:24, MZMcBride<***@mzmcbride.com> wrote:
>>
>>> To me (and
>>> others), that leaves the question of what would happen if you wrote some
>>> software that was actually built for making an encyclopedia, rather than the
>>> jack of all trades product that MediaWiki is.
>>
>> MediaWiki is precisely that software. And there's any number of
>> specialist wikis using it that are basically Wikipedia in a specialist
>> area.
>
> No, I don't think MediaWiki is precisely that software. MediaWiki is a wiki
> engine that can be used for a variety of purposes. It may have started out
> as a tool to make an encyclopedia, but very shortly after its mission
> drifted.

I agree. If MediaWiki is software for creating an encyclopedia then why
are the tools for creating references and footnotes optional extras?
It's rather difficult to set up MediaWiki so that it works in a manner
similar to Wikipedia or Wikimedia Commons (and I know this from experience).

Scale matters too. Right now, any new feature for MediaWiki has to be
considered in the light of some tiny organization running it on an aging
Windows PC, as well as running a top ten website. It's amazing that
MediaWiki has managed to bridge this gap at all, but it's come at a
noticeable cost.

Question: assuming that our primary interest is creating software for
Wikipedia and similar WMF projects, do we actually get anything from the
Windows PC intranet users that offsets the cost of keeping MediaWiki
friendly to both environments? In other words, do we get contributions
from them that help us do Wikipedia et al,?


> MW2.0 would use actual
> input forms for data, instead of the completely hackish hellhole that is
> "[[Category:]]" and "{{Infobox |param}}". MW2.0 would standardize and
> normalize template parameters to something more sane and would allow
> categories to be added, removed, and moved without divine intervention (and
> a working knowledge of Python). MW2.0 would have the ability to edit pages
> without knowing an esoteric, confusing, and non-standardized markup.

+1

I think it is vital to keep templates -- it's a whole new layer of
creativity and MediaWiki's shown that many powerful features can come
about that way. That said, we also want a template to have some
guaranteee of sane inputs and outputs, and maybe a template could also
suggest how its data should be indexed in search engines or for internal
search, or how to create a friendly GUI interface. Perhaps that would be
impossible for all cases, but if XML made it easy in 95% of cases, I'd
take that tradeoff.

As a programmer I am somewhat dismayed at the atrocities that have been
perpetrated with MediaWiki. (Wikimedia's hacking language variants to
show licensing options is my favorite). As someone who believes that
given freedom, users will create amazing things, I'm blown away by the
creativity. I think those show the need for a *more* powerful template
system, that is hopefully easier to use.

Maybe this is anathema here, but XML seems like a logical choice to me.
While inefficient to type in raw code, it is widely understood, and we
can use existing tools to make WYSIWYG editors easily. So perhaps an
infobox could be edited with a sort of form that was autogenerated from
some metadata that described the possible contents of an infobox.

Also, XML can encapsulate data and even other templates to an infinite
degree. A few months ago somebody asked how they could implement a third
layer of "quoting" in the geocoding template syntax and it just seemed
to me like this problem shouldn't have to exist.

--
Neil Kandalgaonkar ( <***@wikimedia.org>

[Ryan Lane]

> Question: assuming that our primary interest is creating software for
> Wikipedia and similar WMF projects, do we actually get anything from the
> Windows PC intranet users that offsets the cost of keeping MediaWiki
> friendly to both environments? In other words, do we get contributions
> from them that help us do Wikipedia et al,?
>

As someone who originally started contributing from maintaining a
small MediaWiki instance, I kind of dislike this question. I also
don't think we should be mixing "we" when discussing WMF and
MediaWiki.

But to answer your question: yes. We get contributions, we get
employees, and we get a larger, more vibrant community. A number of
contributors come from enterprises and small shops, but they often
don't contribute directly to Wikimedia projects. However, their
contributions often allow other people to use the software in
environments they couldn't be used in otherwise (LDAP authentication
is a perfect example of this). The people who then get to use the
software may turn into contributors that do benefit WMF.

MediaWiki is created primarily for WMF use, but a lot of other people
depend on it. I advocate the use of the software by everyone, and
emphasize in talks that we want contributions from everyone, even if
they don't benefit WMF. I don't think we should discourage this. We
should really try harder to embrace enterprise users to get *more*
non-WMF specific extensions and features.

It doesn't take that much effort to keep core small, and maintain
extensions for WMF use. I honestly don't think this is a limiting
factor to the usability of WMF projects, either.

Respectfully,

Ryan Lane

[Chad]

On Wed, Dec 29, 2010 at 4:55 PM, Ryan Lane <***@gmail.com> wrote:
> As someone who originally started contributing from maintaining a
> small MediaWiki instance, I kind of dislike this question. I also
> don't think we should be mixing "we" when discussing WMF and
> MediaWiki.
>
> But to answer your question: yes. We get contributions, we get
> employees, and we get a larger, more vibrant community. A number of
> contributors come from enterprises and small shops, but they often
> don't contribute directly to Wikimedia projects. However, their
> contributions often allow other people to use the software in
> environments they couldn't be used in otherwise (LDAP authentication
> is a perfect example of this). The people who then get to use the
> software may turn into contributors that do benefit WMF.
>

QFT.

-Chad

[Neil Kandalgaonkar]

Thanks... I know this is a provocative question but I meant it just as
it was stated, nothing more, nothing less. For better or worse my
history with the foundation is too short to know the answers to these
questions.

All the assumptions in my question are up for grabs, including the
assumption that we're even primarily developing MediaWiki for WMF
projects. Maybe we think it's just a good thing for the world and that's
that.

Anyway, I would question that it doesn't take a lot of effort to keep
the core small -- it seems to me that more and more of the things we use
to power the big WMF projects are being pushed into extensions and
templates and difficult-to-reproduce configuration and even data entered
directly into the wiki, commingled indistinguishably with documents. (As
you are aware, it takes a lot of knowledge to recreate Wikipedia for a
testing environment. ;)

Meanwhile, MediaWiki is perhaps too powerful and too complex to
administer for the small organization. I work with a small group of
artists that run a MediaWiki instance and whenever online collaboration
has to happen, nobody in this group says "Let's make a wiki page!" That
used to happen, but nowadays they go straight to Google Docs. And that
has a lot of downsides; no version history, complex to auth credentials,
lack of formatting power, can't easily transition to a doc published on
a website, etc.

I'm not saying MediaWiki has to be the weapon of choice for lightweight
collaboration. Maybe that suggests maybe we should narrow the focus of
what we're doing. Or, get more serious about going after those use cases.



On 12/29/10 1:55 PM, Ryan Lane wrote:
>> Question: assuming that our primary interest is creating software for
>> Wikipedia and similar WMF projects, do we actually get anything from the
>> Windows PC intranet users that offsets the cost of keeping MediaWiki
>> friendly to both environments? In other words, do we get contributions
>> from them that help us do Wikipedia et al,?
>>
>
> As someone who originally started contributing from maintaining a
> small MediaWiki instance, I kind of dislike this question. I also
> don't think we should be mixing "we" when discussing WMF and
> MediaWiki.
>
> But to answer your question: yes. We get contributions, we get
> employees, and we get a larger, more vibrant community. A number of
> contributors come from enterprises and small shops, but they often
> don't contribute directly to Wikimedia projects. However, their
> contributions often allow other people to use the software in
> environments they couldn't be used in otherwise (LDAP authentication
> is a perfect example of this). The people who then get to use the
> software may turn into contributors that do benefit WMF.
>
> MediaWiki is created primarily for WMF use, but a lot of other people
> depend on it. I advocate the use of the software by everyone, and
> emphasize in talks that we want contributions from everyone, even if
> they don't benefit WMF. I don't think we should discourage this. We
> should really try harder to embrace enterprise users to get *more*
> non-WMF specific extensions and features.
>
> It doesn't take that much effort to keep core small, and maintain
> extensions for WMF use. I honestly don't think this is a limiting
> factor to the usability of WMF projects, either.
>
> Respectfully,
>
> Ryan Lane
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-***@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

--
Neil Kandalgaonkar ( <***@wikimedia.org>

[Dmitriy Sintsov]

* Neil Kandalgaonkar <***@wikimedia.org> [Wed, 29 Dec 2010 14:40:13
-0800]:
> Thanks... I know this is a provocative question but I meant it just as
> it was stated, nothing more, nothing less. For better or worse my
> history with the foundation is too short to know the answers to these
> questions.
>
> All the assumptions in my question are up for grabs, including the
> assumption that we're even primarily developing MediaWiki for WMF
> projects. Maybe we think it's just a good thing for the world and
that's
> that.
>
> Anyway, I would question that it doesn't take a lot of effort to keep
> the core small -- it seems to me that more and more of the things we
use
> to power the big WMF projects are being pushed into extensions and
> templates and difficult-to-reproduce configuration and even data
entered
> directly into the wiki, commingled indistinguishably with documents.
(As
> you are aware, it takes a lot of knowledge to recreate Wikipedia for a
> testing environment. ;)
>
> Meanwhile, MediaWiki is perhaps too powerful and too complex to
> administer for the small organization. I work with a small group of
> artists that run a MediaWiki instance and whenever online
collaboration
> has to happen, nobody in this group says "Let's make a wiki page!"
That
> used to happen, but nowadays they go straight to Google Docs. And that
> has a lot of downsides; no version history, complex to auth
credentials,
> lack of formatting power, can't easily transition to a doc published
on
> a website, etc.
>
MediaWIki wasn't always so complex. The first version, I've used in 2007
(1.9.3) was reasonably simpler than current 1.17 / 1.18 revisions. And
one might learn it gradually, step by step in many months or even years.
Besides of writing extensions for various clients, I do use it for my
own small memo / blog, where I do put code samples, useful links
(bookmarking) and a lot of various texts (quotations and articles to
read later).

To me, a standalone MediaWiki on a flash drive sounds like a good idea.
However, there are many limitations, although SQLite support have become
much better and there is a Nanoweb http server; some computers might
already listen to 127.0.0.1:80. I wish it was possible to run a kind of
web server with system sockets, or even no sockets at all, however
browsers probably do not support this :-( Otherwise, one should pre-run
a port scanner (not a very good thing).
Dmitriy

[Jay Ashworth]

----- Original Message -----
> From: "Neil Kandalgaonkar" <***@wikimedia.org>

> Meanwhile, MediaWiki is perhaps too powerful and too complex to
> administer for the small organization. I work with a small group of
> artists that run a MediaWiki instance and whenever online collaboration
> has to happen, nobody in this group says "Let's make a wiki page!"

Why not?

> That used to happen, but nowadays they go straight to Google Docs.

Oh.

Well, that's bad. But people will choose the wrong tools; I don't think
that's evidence that MediaWiki's Broken As Designed.

"Too powerful and complex to administer"?

It needs administration? In a small organization?

I set one up at my previous employers, and used it to take all my notes,
which required exactly zero administration: I just slapped it on a box,
and I was done.

And my successor is *very* happy about it. :-)

Cheers,
-- jra

[Ryan Kaldari]

On this note, MTV Networks (my previous job) switched from using
Mediawiki to Confluence a couple years ago. They mainly cited ease of
use and Microsoft Office integration as the reasons. Personally I hated
it, except for the dashboard interface, which was pretty slick. Some
Wikipedia power-users have similar dashboard style interfaces that they
have custom built on their User Pages, but I think it would be cool if
we let people add these sort of interfaces without having to be a
template-hacker.

The sort of interface I'm talking about would include stuff like
community and WikiProject notices and various real-time stats. If you
were a vandal fighter, you would get a vandalism thermometer, streaming
incident notices, a recent changes feed, etc. If you were a content
reviewer, you would get lists of the latest Featured Article and Good
Article candidates, as well as the latest images nominated for Featured
Picture Status, and announcements from the Guild of Copyeditors. The
possibilities are endless.

Ryan Kaldari


On 12/31/10 4:35 PM, Jay Ashworth wrote:
> ----- Original Message -----
>
>> From: "Neil Kandalgaonkar"<***@wikimedia.org>
>>
>
>> Meanwhile, MediaWiki is perhaps too powerful and too complex to
>> administer for the small organization. I work with a small group of
>> artists that run a MediaWiki instance and whenever online collaboration
>> has to happen, nobody in this group says "Let's make a wiki page!"
>>
> Why not?
>
>
>> That used to happen, but nowadays they go straight to Google Docs.
>>
> Oh.
>
> Well, that's bad. But people will choose the wrong tools; I don't think
> that's evidence that MediaWiki's Broken As Designed.
>
> "Too powerful and complex to administer"?
>
> It needs administration? In a small organization?
>
> I set one up at my previous employers, and used it to take all my notes,
> which required exactly zero administration: I just slapped it on a box,
> and I was done.
>
> And my successor is *very* happy about it. :-)
>
> Cheers,
> -- jra
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-***@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>

[Tei]

On 1 January 2011 03:03, Ryan Kaldari <***@wikimedia.org> wrote:
> On this note, MTV Networks (my previous job) switched from using
> Mediawiki to Confluence a couple years ago. They mainly cited ease of
> use and Microsoft Office integration as the reasons. Personally I hated
> it, except for the dashboard interface, which was pretty slick. Some
> Wikipedia power-users have similar dashboard style interfaces that they
> have custom built on their User Pages, but I think it would be cool if
> we let people add these sort of interfaces without having to be a
> template-hacker.
>
> The sort of interface I'm talking about would include stuff like
> community and WikiProject notices and various real-time stats. If you
> were a vandal fighter, you would get a vandalism thermometer, streaming
> incident notices, a recent changes feed, etc. If you were a content
> reviewer, you would get lists of the latest Featured Article and Good
> Article candidates, as well as the latest images nominated for Featured
> Picture Status, and announcements from the Guild of Copyeditors. The
> possibilities are endless.
>
> Ryan Kaldari
>

So, what stop people from writing a "dashboard wizard" that let people
select a predefined one?



--
--
ℱin del ℳensaje.

[David Gerard]

On 1 January 2011 02:03, Ryan Kaldari <***@wikimedia.org> wrote:

> On this note, MTV Networks (my previous job) switched from using
> Mediawiki to Confluence a couple years ago.


There's a certain large media organisation in the UK that uses
Confluence for WYSIWYG and access control lists. And not MediaWiki. I
could have talked them past the ACLs, but not the lack of WYSIWYG.
That's one of the reasons I'm so very gung-ho on the stuff.


> They mainly cited ease of
> use and Microsoft Office integration as the reasons.


It doesn't have ease of use at all. What it has is a features list and
a sales team.

In terms of ease of use, my current workplace has an official
Plone-based intranet and a few less-official MediaWiki installations.
Our office wiki is ridiculously easier to actually use than the Plone
site, despite the lack of WYSIWYG (FCK was pretty good, but not quite
good enough). The Plone site is a write-only


Personally I hated
> it, except for the dashboard interface, which was pretty slick. Some
> Wikipedia power-users have similar dashboard style interfaces that they
> have custom built on their User Pages, but I think it would be cool if
> we let people add these sort of interfaces without having to be a
> template-hacker.
>
> The sort of interface I'm talking about would include stuff like
> community and WikiProject notices and various real-time stats. If you
> were a vandal fighter, you would get a vandalism thermometer, streaming
> incident notices, a recent changes feed, etc. If you were a content
> reviewer, you would get lists of the latest Featured Article and Good
> Article candidates, as well as the latest images nominated for Featured
> Picture Status, and announcements from the Guild of Copyeditors. The
> possibilities are endless.
>
> Ryan Kaldari
>
>
> On 12/31/10 4:35 PM, Jay Ashworth wrote:
>> ----- Original Message -----
>>
>>> From: "Neil Kandalgaonkar"<***@wikimedia.org>
>>>
>>
>>> Meanwhile, MediaWiki is perhaps too powerful and too complex to
>>> administer for the small organization. I work with a small group of
>>> artists that run a MediaWiki instance and whenever online collaboration
>>> has to happen, nobody in this group says "Let's make a wiki page!"
>>>
>> Why not?
>>
>>
>>> That used to happen, but nowadays they go straight to Google Docs.
>>>
>> Oh.
>>
>> Well, that's bad.  But people will choose the wrong tools; I don't think
>> that's evidence that MediaWiki's Broken As Designed.
>>
>> "Too powerful and complex to administer"?
>>
>> It needs administration?  In a small organization?
>>
>> I set one up at my previous employers, and used it to take all my notes,
>> which required exactly zero administration: I just slapped it on a box,
>> and I was done.
>>
>> And my successor is *very* happy about it.  :-)
>>
>> Cheers,
>> -- jra
>>
>> _______________________________________________
>> Wikitech-l mailing list
>> Wikitech-***@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-***@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>

[David Gerard]

On 1 January 2011 15:03, David Gerard <***@gmail.com> wrote:

> It doesn't have ease of use at all. What it has is a features list and
> a sales team.
>
> In terms of ease of use, my current workplace has an official
> Plone-based intranet and a few less-official MediaWiki installations.
> Our office wiki is ridiculously easier to actually use than the Plone
> site, despite the lack of WYSIWYG (FCK was pretty good, but not quite
> good enough). The Plone site is a write-only

... document graveyard. It's where documentation goes to die, unloved
and unnoticed. The wiki is what people actually read and update.

But I do think WYSIWYG could give it about eight times the participation.

So, yeah. I'm picturing a happy world of bunnies and flowers where the
MediaWiki tarball includes WYSIWYG right there and people use an
office wiki as the massively multiplayer office whiteboard it should
be, and the sysadmin gets treated like a hero with very little work.
Because MediaWiki is very little work. And we like to be treated like
heroes every now and then.


- d.

[MZMcBride]

Neil Kandalgaonkar wrote:
> Question: assuming that our primary interest is creating software for
> Wikipedia and similar WMF projects, do we actually get anything from the
> Windows PC intranet users that offsets the cost of keeping MediaWiki
> friendly to both environments? In other words, do we get contributions
> from them that help us do Wikipedia et al,?

Not generally, no.

MediaWiki is just one of Wikimedia's projects, something that I think is
sometimes overlooked or forgotten. Probably as it's the current base upon
which all the other projects are built. To me, that appears to be the
fundamental problem here. I've said this in a roundabout way a few times
now, but the horse is still whimpering, so let's try once more.

I don't think the software that a dictionary or quote database needs is ever
going to be the same as the software that an encyclopedia or news site
needs. And I don't think the software options that fit those four use-cases
will ever work (well!) for a media repository. I don't think it's a lack of
creativity. Given the hacks put in place on sites like the English
Wiktionary, it's clearly not. But at some point there has to be a
recognition that using a screwdriver to put nails in the wall is a bad idea.
You need a hammer.

Tim wrote a blog on techblog.wikimedia.org in July 2010 about MediaWiki
version statistics. Someone commented that it was ironic that Wikimedia was
using WordPress instead of MediaWiki as a blogging platform. Tim's response:
they do different things.[1]

This isn't a matter of not knowing what the problem is. The problem is
recognized by the leading MediaWiki developers and it's an old software
principle (cf. Unix's philosophy[2] of doing one thing and doing it well).
The full phrase quoted earlier is "jack of all trades, master of none." I
think MediaWiki fits this perfectly.

MediaWiki needs to re-focus or fork. Ultimately, however, there are not
enough resources to maintain every current Wikimedia project and nobody is
willing to make the necessary cuts. So we end up with a lot of mediocre
projects/products rather than one or two great ones.

MZMcBride

[1] http://techblog.wikimedia.org/?p=970#comment-819
[2] http://en.wikipedia.org/wiki/Unix_philosophy

[Alex]

On 12/29/2010 5:14 PM, MZMcBride wrote:
> Neil Kandalgaonkar wrote:
>> Question: assuming that our primary interest is creating software for
>> Wikipedia and similar WMF projects, do we actually get anything from the
>> Windows PC intranet users that offsets the cost of keeping MediaWiki
>> friendly to both environments? In other words, do we get contributions
>> from them that help us do Wikipedia et al,?
>
> Not generally, no.
>
> MediaWiki is just one of Wikimedia's projects, something that I think is
> sometimes overlooked or forgotten. Probably as it's the current base upon
> which all the other projects are built. To me, that appears to be the
> fundamental problem here. I've said this in a roundabout way a few times
> now, but the horse is still whimpering, so let's try once more.
>
> I don't think the software that a dictionary or quote database needs is ever
> going to be the same as the software that an encyclopedia or news site
> needs. And I don't think the software options that fit those four use-cases
> will ever work (well!) for a media repository. I don't think it's a lack of
> creativity. Given the hacks put in place on sites like the English
> Wiktionary, it's clearly not. But at some point there has to be a
> recognition that using a screwdriver to put nails in the wall is a bad idea.
> You need a hammer.
>
> Tim wrote a blog on techblog.wikimedia.org in July 2010 about MediaWiki
> version statistics. Someone commented that it was ironic that Wikimedia was
> using WordPress instead of MediaWiki as a blogging platform. Tim's response:
> they do different things.[1]
>
> This isn't a matter of not knowing what the problem is. The problem is
> recognized by the leading MediaWiki developers and it's an old software
> principle (cf. Unix's philosophy[2] of doing one thing and doing it well).
> The full phrase quoted earlier is "jack of all trades, master of none." I
> think MediaWiki fits this perfectly.
>

I don't think using a general purpose wiki engine for every project is
inherently a poor idea. MediaWiki is highly extensible. We just, for
some reason, haven't really taken advantage of that where it could
really matter. Most of the extensions we use just kind of work in the
background. I don't know if its due to lack of resources, or whether the
WMF wants all the projects to look and work the same.

Wiktionary is probably the easiest example. All of the entries follow a
fairly rigid layout that lends itself rather easily to a form, yet we're
still inputting them using a single big textarea.

Though that's not to say we couldn't still do better than we are with a
general purpose wiki engine. I still stand by my earlier suggestion that
we drop the requirement that everything WMF uses has to be able to work
for others right out of the box using only PHP. We should use PHP when
possible, but it shouldn't be a limitation.

--
Alex (wikipedia:en:User:Mr.Z-man)

[Bryan Tong Minh]

On Wed, Dec 29, 2010 at 9:58 PM, Neil Kandalgaonkar <***@wikimedia.org> wrote:
> Question: assuming that our primary interest is creating software for
> Wikipedia and similar WMF projects, do we actually get anything from the
> Windows PC intranet users that offsets the cost of keeping MediaWiki
> friendly to both environments? In other words, do we get contributions
> from them that help us do Wikipedia et al,?
>
Why would I contribute to software that I can't even run?

[George Herbert]

On Wed, Dec 29, 2010 at 2:26 AM, David Gerard <***@gmail.com> wrote:
> On 29 December 2010 08:24, MZMcBride <***@mzmcbride.com> wrote:
>
>> To me (and
>> others), that leaves the question of what would happen if you wrote some
>> software that was actually built for making an encyclopedia, rather than the
>> jack of all trades product that MediaWiki is.
>
>
> MediaWiki is precisely that software.

However - see also the other threads on other lists recently, about MW
core failings.

MW was designed to build an encyclopedia with Web 1.5 technology. It
was a major step forwards compared to its contemporaries, but sites
like Gmail, Facebook, Twitter are massive user experience advances
over where we are and can credibly go with MediaWiki.

So, to modify David's comment -

MediaWiki *was* precisely that software.


--
-george william herbert
***@gmail.com

[Jay Ashworth]

----- Original Message -----
> From: "George Herbert" <***@gmail.com>

> MW was designed to build an encyclopedia with Web 1.5 technology. It
> was a major step forwards compared to its contemporaries, but sites
> like Gmail, Facebook, Twitter are massive user experience advances
> over where we are and can credibly go with MediaWiki.

MediaWiki is nearly perfectly usable from my Blackberry with CSS, images,
and JavaScript disabled; please don't break that.

Cheers,
-- jra

[Maciej Jaros]

2010-12-29 08:31, Neil Kandalgaonkar:
> I've been inspired by the discussion David Gerard and Brion Vibber
> kicked off, and I think they are headed in the right direction.
>
> But I just want to ask a separate, but related question.
>
> Let's imagine you wanted to start a rival to Wikipedia. Assume that you
> are motivated by money, and that venture capitalists promise you can be
> paid gazillions of dollars if you can do one, or many, of the following:
>
> 1 - Become a more attractive home to the WP editors. Get them to work on
> your content.
>
> 2 - Take the free content from WP, and use it in this new system. But
> make it much better, in a way Wikipedia can't match.
>
> 3 - Attract even more readers, or perhaps a niche group of
> super-passionate readers that you can use to build a new community.
>
> In other words, if you had no legacy, and just wanted to build something
> from zero, how would you go about creating an innovation that was
> disruptive to Wikipedia, in fact something that made Wikipedia look like
> Friendster or Myspace compared to Facebook?
>
> And there's a followup question to this -- but you're all smart people
> and can guess what it is.

If one would have a budget of gazillions of dollars then it would be
quite easy ;-). The problem is - what would be the point of investing
such money if you wouldn't get it back from this investment?

If you wouldn't have such money (mostly to pay users for creating
content), then the most problematic part would be to convince community
you are OK. IMHO this has nothing to do with usability or any such thing
it's rather a matter of gaining trust. A part from that you would have
to make all (or almost all) the things that work now work. If you would
make a brand new software then you would have to rewrite at least most
popular user scripts which would alone be a lot work. You would probably
also have to make a nice WYSIWYG to make your site worth moving to. To
make it worth to change at least some of user habits. Not to mention
your site would need to build on a quikcly scalable infrastructure to
guarantee high availabilty (at least as high as Wikipedia is).

In general, you have to remember that even if something is technically
better it's not guaranteed to be successful. For example I think that DP
(pgdp.net) and Rastko are technically better equiped for proofreading
then Wikisource, but I guess for thoose already familiar with MediaWiki
it's easier to create texts for Wikisource.

Regards,
Nux.

[Bryan Tong Minh]

On Wed, Dec 29, 2010 at 12:36 PM, Maciej Jaros <***@wp.pl> wrote:
> If one would have a budget of gazillions of dollars then it would be
> quite easy ;-). The problem is - what would be the point of investing
> such money if you wouldn't get it back from this investment?
>
While money can fix a lot of things, I don't think the current
bottleneck is money. To break stuff you need to find community
consensus, developer consensus, somebody willing to implement it and
somebody to review it. Of course for a gazillion dollars you could
perhaps the eliminate a few of these steps, but in general they are
not really easy to solve with money I think.

[Maciej Jaros]

Bryan Tong Minh (2010-12-29 13:05):
> On Wed, Dec 29, 2010 at 12:36 PM, Maciej Jaros<***@wp.pl> wrote:
>> If one would have a budget of gazillions of dollars then it would be
>> quite easy ;-). The problem is - what would be the point of investing
>> such money if you wouldn't get it back from this investment?
>>
> While money can fix a lot of things, I don't think the current
> bottleneck is money. To break stuff you need to find community
> consensus, developer consensus, somebody willing to implement it and
> somebody to review it. Of course for a gazillion dollars you could
> perhaps the eliminate a few of these steps, but in general they are
> not really easy to solve with money I think.

Well if you would pay users for editing you could attract more users (at
least those that are not willing to work for free). But I guess that
would only work if you would have practically unlimited resources...
Having said that I just remembered that Youtube works like that - you
can get money if get a lot of viewers on your movies.

Regards,
Nux.

[Alex]

On 12/29/2010 7:05 AM, Bryan Tong Minh wrote:
> On Wed, Dec 29, 2010 at 12:36 PM, Maciej Jaros <***@wp.pl> wrote:
>> If one would have a budget of gazillions of dollars then it would be
>> quite easy ;-). The problem is - what would be the point of investing
>> such money if you wouldn't get it back from this investment?
>>
> While money can fix a lot of things, I don't think the current
> bottleneck is money. To break stuff you need to find community
> consensus, developer consensus, somebody willing to implement it and
> somebody to review it. Of course for a gazillion dollars you could
> perhaps the eliminate a few of these steps, but in general they are
> not really easy to solve with money I think.
>

I think one of the biggest obstacles to improving the Wikipedia user
experience is the requirement that the content has to not only be
reusable, but reusable with a minimum amount of effort - i.e. on a free
shared hosting environment with neither shell access nor the ability to
install or compile programs. With only a couple exceptions[1] any
software that's required to display Wikipedia content has to be PHP,
have a PHP implementation available, or be done client-side (and of
course, we can't use Flash). We're hamstrung by the limitations of what
can be reasonably done in pure PHP even in cases when we would be using
a C extension or shelling out to an executable.

The recently revived discussion on StringFunctions is a good example of
this. Tim and others don't want to install StringFunctions because it
will just increase the complexity of wikitext and, like ParserFunctions,
will only be a temporary fix until template coders write new templates
that reach new limits created. A real solution to the issue is to use a
real programming language in place of wikitext for complex templates.
But until the aforementioned limitation is relaxed, that's likely never
going to happen. We have to either implement an existing language like
Lua in PHP or write our own language and maintain 2 implementations of
it (the compiled version for WMF and the pure PHP version).

[1] LaTeX and EasyTimeline

--
Alex (wikipedia:en:User:Mr.Z-man)

[Neil Kandalgaonkar]

On 12/29/10 4:05 AM, Bryan Tong Minh wrote:
> On Wed, Dec 29, 2010 at 12:36 PM, Maciej Jaros<***@wp.pl> wrote:
>> If one would have a budget of gazillions of dollars then it would be
>> quite easy ;-). The problem is - what would be the point of investing
>> such money if you wouldn't get it back from this investment?
>>
> While money can fix a lot of things, I don't think the current
> bottleneck is money.

I apologize for sending this discussion in a direction I hadn't
intended. The money was purely to imply that you had to be motivated,
not that you had a vast budget.

Let me be more explicit. The "innovator's dilemma" problem, already
referred to in this discussion, occurs because the successful innovator
can't see past the goal of defending their earlier successes, and
working with their existing assets.

The thought experiment of working for a competitor was meant to suggest
this: what would you do if you wanted to make Wikipedia's earlier
successes *obsolete*? The point is to then try to look at some of our
greatest assets and see if, in the current environment, they could be
potential liabilities.

And the followup question was "if a competitor can do this, why don't WE
do this?"

Brion already suggested something like this, where we would end up with
a transition regime between old and new.

P.S. All due respect to RobLa, but "Microsoft tried this and failed"
doesn't exactly convince me it's impossible. ;)

--
Neil Kandalgaonkar ( <***@wikimedia.org>

[Fred Bauder]

> And the followup question was "if a competitor can do this, why don't WE
> do this?"

Now you're talking. Being static because a competitor might be able to
adopt your changes better than you can is ultimately self-defeating.

Fred

[Maciej Jaros]

Neil Kandalgaonkar (2010-12-29 21:40):
> On 12/29/10 4:05 AM, Bryan Tong Minh wrote:
>> On Wed, Dec 29, 2010 at 12:36 PM, Maciej Jaros<***@wp.pl> wrote:
>>> If one would have a budget of gazillions of dollars then it would be
>>> quite easy ;-). The problem is - what would be the point of investing
>>> such money if you wouldn't get it back from this investment?
>>>
>> While money can fix a lot of things, I don't think the current
>> bottleneck is money.
> I apologize for sending this discussion in a direction I hadn't
> intended. The money was purely to imply that you had to be motivated,
> not that you had a vast budget.
>
> Let me be more explicit. The "innovator's dilemma" problem, already
> referred to in this discussion, occurs because the successful innovator
> can't see past the goal of defending their earlier successes, and
> working with their existing assets.
>
> The thought experiment of working for a competitor was meant to suggest
> this: what would you do if you wanted to make Wikipedia's earlier
> successes *obsolete*? The point is to then try to look at some of our
> greatest assets and see if, in the current environment, they could be
> potential liabilities.

My original point was that the community is the power of WMF sites and
that this alone is IMHO hard to beat. To be more exact this is a
community that I believe is loyal and needs to trust the
corporation/founder/foundation behind the site (I've seen a community
driven project fall after loosing this trust).

> And the followup question was "if a competitor can do this, why don't WE
> do this?"

We don't because it would probably be more reasonable for our competitor
to do something completely different to gather different community or he
would have to make a gigantic effort to steal current community (both in
technical and PR terms). I think the effort would simply be inefficient.

In any case - the next killer functionality (if that's what you're
asking) is well known and already mentioned - WYSIWYG. WYSIWYG that
makes edits easy for new users and make them not break existing markup.
And yes I believe present markup needs to be preserved. Not because it's
good, it's because it is well know to many current users. It's because
community is accustomed with it. Loosing users after changing markup
drastically would certainly not be a good idea. You have to remember how
many disappointment brought a simple change of default skin. Something
that can be changed back in 3 clicks. And so new markup (if such would
be used) would have to at least be parseable back to wikitext.

Regards,
Nux.

[Lars Aronsson]

On 12/29/2010 08:31 AM, Neil Kandalgaonkar wrote:
> Let's imagine you wanted to start a rival to Wikipedia. Assume that you
> are motivated by money, and that venture capitalists promise you can be
> paid gazillions of dollars if you can do one, or many, of the following:

This is a foolish discussion for two reasons. First, wikitech-l is
a technical list, and not suited for talk on organizational change.
Second, innovation doesn't come from within. Encyclopaedia
Britannica didn't invent Wikipedia, AT&T didn't invent the
Internet, Gorbachev didn't succeed in implementing
competition within the communist party (although he tried),
and dinosaurs weren't invited to the design committee for
the surviving mammals. If there is a new alternative to Wikipedia,
we, the subscribers to wikitech-l, are not invited to design it.

What we can easily achieve is to make bureaucracy so
slow and rigid (and our discussions derailed, like now)
that more people will leave WMF projects. But this is
not enough to create a working alternative.


--
Lars Aronsson (***@aronsson.se)
Aronsson Datateknik - http://aronsson.se

[Rob Lanphier]

On Tue, Dec 28, 2010 at 11:31 PM, Neil Kandalgaonkar
<***@wikimedia.org> wrote:
> Let's imagine you wanted to start a rival to Wikipedia. Assume that you
> are motivated by money, and that venture capitalists promise you can be
> paid gazillions of dollars if you can do one, or many, of the following:
>
> 1 - Become a more attractive home to the WP editors. Get them to work on
> your content.
>
> 2 - Take the free content from WP, and use it in this new system. But
> make it much better, in a way Wikipedia can't match.
>
> 3 - Attract even more readers, or perhaps a niche group of
> super-passionate readers that you can use to build a new community.

I'll start off by saying that I have no idea how anyone would do it,
realistically. I'm pretty sure it's possible, but I think a big
reason that it hasn't happened yet is because the economics of
creating a competitor are really difficult. There are very few
markets that Microsoft completely gave up in (especially markets in
which they've had success), but yet that's exactly what they did with
Encarta. Good luck getting VC money to take on a market that
Microsoft abandons. ;-)

I suspect if I had to choose, though, I'd go with #2. I'd probably
bootstrap by creating tools *for* Wikipedia editors rather than trying
right off the bat to create a wholly separate site. For example, it'd
probably be possible to scrape our data to create a really fantastic
citation database, which then could be used to build tools that make
creating citations much easier. The goal would be to make it easier
for editors to keep *my* database up-to-date, and push a copy to
Wikipedia, rather than having to constantly suck things out of
Wikipedia.

That's such a small part of the overall editing problem that I'm not
sure how I'd bootstrap that into something much larger (and in the
case of a citation database, it wouldn't be necessary for Wikipedia to
lose in order to have a modest ad-supported business).

As to the implied question, I think we need to figure out ways of
making things like this easier for third parties to tackle. If we can
make it easier for third parties to create tools for editing Wikipedia
(regardless of their motivations), we'll probably accidentally make it
easier for us to make it easier to edit Wikipedia.

Rob

[Ryan Kaldari]

I would steal some of the better ideas from Wikia like the "hot article"
lists, user polls, user avatars, and throw in some real-time
collaboration software a la Etherpad.

Ryan Kaldari

On 12/28/10 11:31 PM, Neil Kandalgaonkar wrote:
> I've been inspired by the discussion David Gerard and Brion Vibber
> kicked off, and I think they are headed in the right direction.
>
> But I just want to ask a separate, but related question.
>
> Let's imagine you wanted to start a rival to Wikipedia. Assume that you
> are motivated by money, and that venture capitalists promise you can be
> paid gazillions of dollars if you can do one, or many, of the following:
>
> 1 - Become a more attractive home to the WP editors. Get them to work on
> your content.
>
> 2 - Take the free content from WP, and use it in this new system. But
> make it much better, in a way Wikipedia can't match.
>
> 3 - Attract even more readers, or perhaps a niche group of
> super-passionate readers that you can use to build a new community.
>
> In other words, if you had no legacy, and just wanted to build something
> from zero, how would you go about creating an innovation that was
> disruptive to Wikipedia, in fact something that made Wikipedia look like
> Friendster or Myspace compared to Facebook?
>
> And there's a followup question to this -- but you're all smart people
> and can guess what it is.
>
>

[Soxred93]

Of course, you have to remember that Wikipedia is a top 10 website. Wikia is a top 200 website. "hot article"s just don't scale that well to a wiki like Wikipedia. It's fundamentally flawed.

On the flip side, an Etherpad-like feature would be nice.

-X!

On Dec 29, 2010, at 6:41 PM, Ryan Kaldari wrote:

> I would steal some of the better ideas from Wikia like the "hot article"
> lists, user polls, user avatars, and throw in some real-time
> collaboration software a la Etherpad.
>
> Ryan Kaldari
>
> On 12/28/10 11:31 PM, Neil Kandalgaonkar wrote:
>> I've been inspired by the discussion David Gerard and Brion Vibber
>> kicked off, and I think they are headed in the right direction.
>>
>> But I just want to ask a separate, but related question.
>>
>> Let's imagine you wanted to start a rival to Wikipedia. Assume that you
>> are motivated by money, and that venture capitalists promise you can be
>> paid gazillions of dollars if you can do one, or many, of the following:
>>
>> 1 - Become a more attractive home to the WP editors. Get them to work on
>> your content.
>>
>> 2 - Take the free content from WP, and use it in this new system. But
>> make it much better, in a way Wikipedia can't match.
>>
>> 3 - Attract even more readers, or perhaps a niche group of
>> super-passionate readers that you can use to build a new community.
>>
>> In other words, if you had no legacy, and just wanted to build something
>> from zero, how would you go about creating an innovation that was
>> disruptive to Wikipedia, in fact something that made Wikipedia look like
>> Friendster or Myspace compared to Facebook?
>>
>> And there's a followup question to this -- but you're all smart people
>> and can guess what it is.
>>
>>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-***@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Ryan Kaldari]

Actually, I would implement "hot articles" per WikiProject. So, for
example, you could see the 5 articles under WikiProject Arthropods that
had been edited the most in the past week. That should scale well. In
fact, I would probably redesign Wikipedia to be WikiProject-based from
the ground up, rather than as an afterthought. Like when you first sign
up for an account it asks you which WikiProjects you want to join, etc.
and there are cool extensions for earning points and awards within
WikiProjects (that don't require learning how to use templates).

Ryan Kaldari

On 12/29/10 3:49 PM, Soxred93 wrote:
> Of course, you have to remember that Wikipedia is a top 10 website. Wikia is a top 200 website. "hot article"s just don't scale that well to a wiki like Wikipedia. It's fundamentally flawed.
>
> On the flip side, an Etherpad-like feature would be nice.
>
> -X!
>
> On Dec 29, 2010, at 6:41 PM, Ryan Kaldari wrote:
>
>
>> I would steal some of the better ideas from Wikia like the "hot article"
>> lists, user polls, user avatars, and throw in some real-time
>> collaboration software a la Etherpad.
>>
>> Ryan Kaldari
>>
>> On 12/28/10 11:31 PM, Neil Kandalgaonkar wrote:
>>
>>> I've been inspired by the discussion David Gerard and Brion Vibber
>>> kicked off, and I think they are headed in the right direction.
>>>
>>> But I just want to ask a separate, but related question.
>>>
>>> Let's imagine you wanted to start a rival to Wikipedia. Assume that you
>>> are motivated by money, and that venture capitalists promise you can be
>>> paid gazillions of dollars if you can do one, or many, of the following:
>>>
>>> 1 - Become a more attractive home to the WP editors. Get them to work on
>>> your content.
>>>
>>> 2 - Take the free content from WP, and use it in this new system. But
>>> make it much better, in a way Wikipedia can't match.
>>>
>>> 3 - Attract even more readers, or perhaps a niche group of
>>> super-passionate readers that you can use to build a new community.
>>>
>>> In other words, if you had no legacy, and just wanted to build something
>>> from zero, how would you go about creating an innovation that was
>>> disruptive to Wikipedia, in fact something that made Wikipedia look like
>>> Friendster or Myspace compared to Facebook?
>>>
>>> And there's a followup question to this -- but you're all smart people
>>> and can guess what it is.
>>>
>>>
>>>
>> _______________________________________________
>> Wikitech-l mailing list
>> Wikitech-***@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-***@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>

[Platonides]

Ryan Kaldari wrote:
> Actually, I would implement "hot articles" per WikiProject. So, for
> example, you could see the 5 articles under WikiProject Arthropods that
> had been edited the most in the past week. That should scale well. In
> fact, I would probably redesign Wikipedia to be WikiProject-based from
> the ground up, rather than as an afterthought. Like when you first sign
> up for an account it asks you which WikiProjects you want to join, etc.
> and there are cool extensions for earning points and awards within
> WikiProjects (that don't require learning how to use templates).
>
> Ryan Kaldari

Well, that's an interesting point. People ask for things like a "chat
per article" without realising what that would mean.
Grouping communication in bigger "wikiproject" channels could work.
Although some tree-like structure would be needed to manually split /
magically join depending on the amount of people there.

[Aryeh Gregor]

On Wed, Dec 29, 2010 at 2:31 AM, Neil Kandalgaonkar <***@wikimedia.org> wrote:
> In other words, if you had no legacy, and just wanted to build something
> from zero, how would you go about creating an innovation that was
> disruptive to Wikipedia, in fact something that made Wikipedia look like
> Friendster or Myspace compared to Facebook?

By having content that's consistently better. It doesn't matter how
easy your site is to edit. Even if your site is so easy to edit that
you get 10% of viewers editing, 10% of your few million (at best)
viewers is still going to get you vastly worse content than a small
fraction of a percent of Wikipedia's billions. Wikipedia survives off
network effects; it's not even remotely a level playing field. People
who are focusing on things like WYSIWYG or better-quality editing
software are missing the point. You need to have better *content* to
attract viewers, before you even stand a chance of edits through your
site being meaningful.

If you somehow manage to have content that's consistently better than
Wikipedia's, though, people will figure out over time, as long as you
can maintain the quality advantage. One obvious strategy would be to
mirror Wikipedia in real time and send viewers to Wikipedia proper to
edit it, but to have more useful features or a better experience.
Maybe a better mobile site, maybe faster page load times, maybe easier
navigation or search. Maybe more content, letting people put up
vanity bios or articles about obscure webcomics that integrate more or
less seamlessly with the Wikipedia corpus. You could even compete by
putting up a better editing interface, conceivably, although auth
would be tricky to work out. If you ever got a majority of viewers
coming to your site, you could fork transparently.

On Wed, Dec 29, 2010 at 6:59 PM, Brion Vibber <***@pobox.com> wrote:
> I think this isn't as useful a question as it might be; defining a project
> in terms of competing with something else leads to stagnation, not
> innovation.

I agree. The correct strategy to take down Wikipedia would involve
overcoming the network effect that locks it into its current position
of dominance, and that's not something that would be useful for
Wikipedia itself to do. To fend off attacks of this sort, what you'd
want is to make your content harder to reuse, which we explicitly
*don't* want to do. Better to ask: how can we enable more people to
contribute who want to but can't be bothered?

[David Gerard]

On 30 December 2010 00:27, Aryeh Gregor <Simetrical+***@gmail.com> wrote:

>  You could even compete by
> putting up a better editing interface, conceivably, although auth
> would be tricky to work out.


You know, this is something that would be extremely easy to experiment
with right now,


> On Wed, Dec 29, 2010 at 6:59 PM, Brion Vibber <***@pobox.com> wrote:

>> I think this isn't as useful a question as it might be; defining a project
>> in terms of competing with something else leads to stagnation, not
>> innovation.

> I agree.  The correct strategy to take down Wikipedia would involve
> overcoming the network effect that locks it into its current position
> of dominance, and that's not something that would be useful for
> Wikipedia itself to do.  To fend off attacks of this sort, what you'd
> want is to make your content harder to reuse, which we explicitly
> *don't* want to do.  Better to ask: how can we enable more people to
> contribute who want to but can't be bothered?


Making Wikipedia easy to mirror and fork is the best protection I can
think of for the content itself. It also keeps the support structures
(Foundation) and community good and honest. Comparison: People keep
giving Red Hat money; Debian continues despite a prominent and
successful fork (Ubuntu), and quite a bit goes back from the fork
(both pull and push).


- d.

[Tim Starling]

On 29/12/10 18:31, Neil Kandalgaonkar wrote:
> I've been inspired by the discussion David Gerard and Brion Vibber
> kicked off, and I think they are headed in the right direction.
>
> But I just want to ask a separate, but related question.
>
> Let's imagine you wanted to start a rival to Wikipedia. Assume that you
> are motivated by money, and that venture capitalists promise you can be
> paid gazillions of dollars if you can do one, or many, of the following:
>
> 1 - Become a more attractive home to the WP editors. Get them to work on
> your content.
>
> 2 - Take the free content from WP, and use it in this new system. But
> make it much better, in a way Wikipedia can't match.

This has been done before: Wikinfo, Citizendium, etc.

> 3 - Attract even more readers, or perhaps a niche group of
> super-passionate readers that you can use to build a new community.

This is basically Wikia's business model. I think you need to think
outside the box.

I would make it more like World of Warcraft. We should incentivise
people to set up wiki sweatshops in Indonesia, paying local people to
"grind" all day, cleaning up articles, in order to build up a level 10
admin character that can then be sold for thousands of dollars on the
open market. Also it should have cool graphics.

OK, if you want a real answer: I think if you could convince admins to
be nicer to people, then that would make a bigger impact to
Wikipedia's long-term viability than any ease-of-editing feature.
Making editing easier will give you a one-off jump in editing
statistics, it won't address the trend.

We know from interviews and departure messages that the editing
interface creates an initial barrier for entry, but for people who get
past that barrier, various social factors, such as incivility and
bureaucracy, limit the time they spend contributing.

Once you burn someone out, they don't come back for a long time, maybe
not ever. So you introduce a downwards trend which extends over
decades, until the rate at which we burn people out meets the rate at
which new editors are born.

Active, established editors have a battlefront mentality. They feel as
if they are fighting for the survival of Wikipedia against a constant
stream of newbies who don't understand or don't care about our
policies. As the stream of newbies increases, they become more
desperate, and resort to more desperate (and less civil) measures for
controlling the flood.

Making editing easier could actually be counterproductive. If we let
more people past the editing interface barrier before we fix our
social problems, then we could burn out the majority of the Internet
population before we figure out what's going on. Increasing the number
of new editors by a large factor will increase the anxiety level of
admins, and thus accelerate this process.

I think there are things we can do in software to help de-escalate
this conflict between established editors and new editors.

One thing we can do is to reduce the sense of urgency. Further
deployment of FlaggedRevs (pending changes) is the obvious way to do
this. By hiding recent edits, admins can deal with bad edits in their
own time, rather reacting in the heat of the moment.

Another thing we could do is to improve the means of communication.
Better communication often helps to de-escalate a conflict.

We could replace the terrible user talk page interface with an
easy-to-use real-time messaging framework. We could integrate polite
template responses with the UI. And we could provide a centralised
forum-like view of such messages, to encourage mediators to review and
de-escalate emotion-charged conversations.

-- Tim Starling

[Neil Kandalgaonkar]

On 12/29/10 7:26 PM, Tim Starling wrote:

> OK, if you want a real answer: I think if you could convince admins to
> be nicer to people, then that would make a bigger impact to
> Wikipedia's long-term viability than any ease-of-editing feature.
> Making editing easier will give you a one-off jump in editing
> statistics, it won't address the trend.
>
> We know from interviews and departure messages that the editing
> interface creates an initial barrier for entry, but for people who get
> past that barrier, various social factors, such as incivility and
> bureaucracy, limit the time they spend contributing.

For me the usability projects always had the unstated intent of
broadening the pool of good editors. More hands to ease the burdens of
the beleagured admins, and also fresher blood that wasn't quite as
ensconced in wikipolitics.

But overall I agree.


> Making editing easier could actually be counterproductive. If we let
> more people past the editing interface barrier before we fix our
> social problems, [...]

This is an interesting insight!

I have been thinking along these lines too, although in a more haphazard
way.

At some point, if we believe our community is our greatest asset, we
have to think of Wikipedia as infrastructure not only for creating high
quality articles, but also for generating and sustaining a high quality
editing community. My sense is that the Wiki* communities are down with
goal #1, but goal #2 is not on their radar at all.

So we probably need an employee dedicated to this. (I think? Arguments?)

When the Usability Project closed down, the team was also unhappy with
the narrow focus paid to editing. Research showed the most serious
problems were elsewhere. We then said we were going to address UX issues
in a very broad way, which included social issues. Unfortunately the
person in charge of that left the Foundation soon after and in the
kerfuffle I'm not sure if we now have anybody whose primary job it is to
think about the experience of the user in such broad terms.

--
Neil Kandalgaonkar ( <***@wikimedia.org>

[Alex Brollo]

2010/12/30 Neil Kandalgaonkar <***@wikimedia.org>

> On 12/29/10 7:26 PM, Tim Starling wrote:
>
> > Making editing easier could actually be counterproductive. If we let
> > more people past the editing interface barrier before we fix our
> > social problems, [...]
>
> This is an interesting insight!
>

Yes it's really interesting and highlighting!

I'm following another talk about StringFunctions; and I recently got an
account into toolserver (I only hope that my skill is merely sufficient!).
In both cases, there's an issue of "security by obscurity". I hate it at
beginning, but perhaps such an approach is necessary, it's the simplest way
to get a very difficult result.

So, what's important is, the balance between simplicity and complexity,
since this turns out into a "contributor filter". At the beginning, wiki
markup has been designed to be very simple. A very important feature of
markup has been sacrificed: the code is not "well formed". There are lots of
simple, but ambiguous tags (for bold and italic characters, for lists); tags
don't need to be closed; text content and tags/attributes are mixed freely
into the template code. This makes simpler their use but causes terrible
quizzes for advanced users facing with unusual cases or trying to parse
wikitext by scripts or converting wikitext into a formally well formed
markup. My question is: can we imagine to move a little bit that balance
accepting a little more complexity and to think to a well formed wiki
markup?

Alex

[Platonides]

Neil Kandalgaonkar wrote:
>
> I have been thinking along these lines too, although in a more haphazard
> way.
>
> At some point, if we believe our community is our greatest asset, we
> have to think of Wikipedia as infrastructure not only for creating high
> quality articles, but also for generating and sustaining a high quality
> editing community. My sense is that the Wiki* communities are down with
> goal #1, but goal #2 is not on their radar at all.
>
> So we probably need an employee dedicated to this. (I think? Arguments?)

He would be quite busy (and polyglot!) to keep an eye over the community
of +800 projects.

[Neil Kandalgaonkar]

On 12/30/10 10:24 AM, Platonides wrote:
> Neil Kandalgaonkar wrote:
>> At some point, if we believe our community is our greatest asset, we
>> have to think of Wikipedia as infrastructure not only for creating high
>> quality articles, but also for generating and sustaining a high quality
>> editing community.
>>
>> So we probably need an employee dedicated to this. (I think? Arguments?)
>
> He would be quite busy (and polyglot!) to keep an eye over the community
> of +800 projects.

Why is this a requirement?

If you think about the sum total of user-hours spent on Wikipedia, the
vast majority of them are spent in just three or four interface flows.

But you're right; they can't be everywhere, so maybe there should be a
guidelines page on design principles. We have WP:CIVILITY, do we have
similar guidelines for software developers, on how to make it easy for
the community to be civil?

Frankly I don't think I'm qualified to do this. I know of a few people
are brilliant at this, and who do this sort of thing for a living, but
they are consultants. Fostering community on the web is generally
considered a sort of black art... does anybody know of any less
mystified way of dealing with the problem?

--
Neil Kandalgaonkar ( <***@wikimedia.org>

[Platonides]

Neil Kandalgaonkar wrote:
> On 12/30/10 10:24 AM, Platonides wrote:
>> Neil Kandalgaonkar wrote:
>>> At some point, if we believe our community is our greatest asset, we
>>> have to think of Wikipedia as infrastructure not only for creating high
>>> quality articles, but also for generating and sustaining a high quality
>>> editing community.
>>>
>>> So we probably need an employee dedicated to this. (I think? Arguments?)
>>
>> He would be quite busy (and polyglot!) to keep an eye over the community
>> of +800 projects.
>
> Why is this a requirement?

The point is, there's no "one community" to "watch". Most people think
in enwiki, for being the biggest project, and most probably the base
project of those people.

But one must not forget that there are many WMF projects out there. It
doesn't end in enwp. They have similar problems, but cannot be
generalised either.
There's a risk of contracting someone as an injerence on the project
(seems the role for a facilitator, but I'd only place people that were
already in the community -the otrs folks seem a good fishing pool-, if
doing such thing). Plus, there's the view on how it may be perceived
(WMF trying to impose its views over the community, WMF really having
power on the project and thus being liable...).



> If you think about the sum total of user-hours spent on Wikipedia, the
> vast majority of them are spent in just three or four interface flows.

What are you thinking about? Things such as talk page messages. There
are shortcuts for those interfaces. Several gadgets/scripts provide a
tab for adding a template to a page + leave a predefined message to the
author talk page. That's good in a sense as the users *get* messages
(eg. when listing images for deletion), they are also quite full and
translated (relevant just for commons). But it also means that it's a
generic message, so not as appropiate for everyone.

We can make the flow faster, but we lose precision.


> But you're right; they can't be everywhere, so maybe there should be a
> guidelines page on design principles. We have WP:CIVILITY, do we have
> similar guidelines for software developers, on how to make it easy for
> the community to be civil?

I'm lost here. Are you calling uncivil the developer community for this
thread? You mean that WP:CIVILITY should be enforced by mediawiki?
Developers should be more helopful when dealing bug reports? What do you
mean?


> Frankly I don't think I'm qualified to do this. I know of a few people
> are brilliant at this, and who do this sort of thing for a living, but
> they are consultants. Fostering community on the web is generally
> considered a sort of black art... does anybody know of any less
> mystified way of dealing with the problem?

[Neil Kandalgaonkar]

On 12/30/10 3:33 PM, Platonides wrote:

>> But you're right; they can't be everywhere, so maybe there should be a
>> guidelines page on design principles. We have WP:CIVILITY, do we have
>> similar guidelines for software developers, on how to make it easy for
>> the community to be civil?
>
> I'm lost here. Are you calling uncivil the developer community for this
> thread? You mean that WP:CIVILITY should be enforced by mediawiki?
> Developers should be more helopful when dealing bug reports? What do you
> mean?

I guess I have not been clear... I was picking up on what Tim said, that
we have to work on making WP and other projects into places where people
feel more welcome.

Telling people to be nicer may help, but I actually think that people
are more shaped by their environment. If you go from a party at a
friend's warm apartment to an anonymous street your mood and
receptiveness to others changes instantly.

The point is to make MediaWiki more like the friend's apartment, and
less like the anonymous street. If we have interfaces that make it easy
for admins to be rude to new editors, they will be more rude. If we make
it easy to be nice, then maybe they'll also be nicer. This isn't a
radical new idea.

Tim already noted that he hopes Pending Changes (nee FlaggedRevs) would
help people be less brusque with one another. Polite template responses,
things like that.

Users are influenced by very subtle cues. Understanding how they work is
a very rare ability. So I was suggesting we collect rules of thumb for
people who are making interfaces. Not policies to bash each other with.

--
Neil Kandalgaonkar ( <***@wikimedia.org>

[MZMcBride]

Tim Starling wrote:
> OK, if you want a real answer: I think if you could convince admins to
> be nicer to people, then that would make a bigger impact to
> Wikipedia's long-term viability than any ease-of-editing feature.
> Making editing easier will give you a one-off jump in editing
> statistics, it won't address the trend.
>
> We know from interviews and departure messages that the editing
> interface creates an initial barrier for entry, but for people who get
> past that barrier, various social factors, such as incivility and
> bureaucracy, limit the time they spend contributing.

Is there any evidence to support these claims? From what I understand, a lot
of Wikipedia's best new content is added by anonymous users.[1] Thousands
more editors are capable of registering and editing without much interaction
with the broader Wikimedia community at all. If there's evidence that mean
admins are a credible threat to long-term viability, I'd be interested to
see it.

Given that there are about 770 active administrators[2] on the English
Wikipedia and I think you could reasonably say that a good portion are not
mean, is it really quite a few people who are having this far-reaching
impact that you're suggesting exists? That seems unlikely.

> Making editing easier could actually be counterproductive. If we let
> more people past the editing interface barrier before we fix our
> social problems, then we could burn out the majority of the Internet
> population before we figure out what's going on. Increasing the number
> of new editors by a large factor will increase the anxiety level of
> admins, and thus accelerate this process.

I think the growth should be organic. With a better interface in place, a
project has a much higher likelihood of successful, healthy growth.

> One thing we can do is to reduce the sense of urgency. Further
> deployment of FlaggedRevs (pending changes) is the obvious way to do
> this. By hiding recent edits, admins can deal with bad edits in their
> own time, rather reacting in the heat of the moment.

Endless backlogs are going to draw people in? Delayed gratification is going
to keep people contributing? This proposal seems anti-wiki in a literal and
philosophical sense.

MZMcBride

[1] http://www.cs.dartmouth.edu/reports/abstracts/TR2007-606/
[2] http://en.wikipedia.org/wiki/Wikipedia:List_of_administrators

[David Gerard]

On 30 December 2010 11:06, MZMcBride <***@mzmcbride.com> wrote:
> Tim Starling wrote:

>> OK, if you want a real answer: I think if you could convince admins to
>> be nicer to people, then that would make a bigger impact to
>> Wikipedia's long-term viability than any ease-of-editing feature.
>> Making editing easier will give you a one-off jump in editing
>> statistics, it won't address the trend.

> Given that there are about 770 active administrators[2] on the English
> Wikipedia and I think you could reasonably say that a good portion are not
> mean, is it really quite a few people who are having this far-reaching
> impact that you're suggesting exists? That seems unlikely.


There is some discussion of how the community and ArbCom enable
grossly antisocial behaviour on internal-l at present. Admin behaviour
is enforced by the ArbCom, and the AC member on internal-l has mostly
been evasive. It's not clear what approach would work at this stage;
it would probably have to get worse before the Foundation could
reasonably step in.


- d.

[Risker]

On 30 December 2010 09:07, David Gerard <***@gmail.com> wrote:

> On 30 December 2010 11:06, MZMcBride <***@mzmcbride.com> wrote:
> > Tim Starling wrote:
>
> >> OK, if you want a real answer: I think if you could convince admins to
> >> be nicer to people, then that would make a bigger impact to
> >> Wikipedia's long-term viability than any ease-of-editing feature.
> >> Making editing easier will give you a one-off jump in editing
> >> statistics, it won't address the trend.
>
> > Given that there are about 770 active administrators[2] on the English
> > Wikipedia and I think you could reasonably say that a good portion are
> not
> > mean, is it really quite a few people who are having this far-reaching
> > impact that you're suggesting exists? That seems unlikely.
>
>
> There is some discussion of how the community and ArbCom enable
> grossly antisocial behaviour on internal-l at present. Admin behaviour
> is enforced by the ArbCom, and the AC member on internal-l has mostly
> been evasive. It's not clear what approach would work at this stage;
> it would probably have to get worse before the Foundation could
> reasonably step in.
>
>

Perhaps if communication actually took place with Arbcom itself, rather than
on a list in which there is no Arbcom representative, there might be a
better understanding of the concerns you have mentioned. There's no "Arbcom
representative" on internal-L, and in fact this is something of a bone of
contention.

Nonetheless, I think the most useful post in this entire thread has been Tim
Starling's, and I thank him for it.


Risker
(who is coincidentally an enwp Arbitration Committee member but is in no way
an Arbcom representative on this list)

[John Vandenberg]

On Fri, Dec 31, 2010 at 1:07 AM, David Gerard <***@gmail.com> wrote:
> There is some discussion of how the community and ArbCom enable
> grossly antisocial behaviour on internal-l at present. Admin behaviour
> is enforced by the ArbCom, and the AC member on internal-l has mostly
> been evasive.

Wtf?
ArbCom members are expected to be responsive to discussions about
English Wikipedia occurring on internal-l?
Could you please clarify who are you're obliquely attacking here?

--
John Vandenberg

[David Gerard]

Blog post on this topic:

http://davidgerard.co.uk/notes/2010/12/30/how-does-a-project-bite-only-the-proper-number-of-newbies/


- d.

[Tei]

>With open source software, there are people who think “that’s dumb,” there are people who think “I want to see it >fixed” and there are people who think “I can do something about it.” The people at the intersection of all three >power open source.


A lot of people in the open source project Y will not see a problem
with X, being X a huge usability problem that stop a lot of people
from using Y.

So what you have is a lot of people "I don't see the problem with
that" ( realistically, a lot of people that will talk about a lot of
things, and not about X ), and maybe some of the people that have
problems with X that don't know how to communicate his problem, or
don't care enough.

Any open source project work like a club. The club work for the
people that is part of the club, and does the things that the people
of the club enjoy. If you like chess, you will not join the basket
club, and probably the basket club will never run a chess competition.
Or the chess club a basket competition.

If anything, the "Problem" with open source, is that any change is
incremental, and there's a lot of endogamy.

Also user suggestions are not much better. Users often ask for things
that are too hard, or incremental enhancements that will result on
bloat on the long term.

So really, what you may need is one person that can see the problems
of the newbies, of the devs, of the people with a huge investment on
the project, and make long term decisions, and have a lot of influence
on the people, while working on the shadows towards that goal.


--
--
ℱin del ℳensaje.

[Aryeh Gregor]

On Wed, Dec 29, 2010 at 10:26 PM, Tim Starling <***@wikimedia.org> wrote:
> I think there are things we can do in software to help de-escalate
> this conflict between established editors and new editors.
>
> One thing we can do is to reduce the sense of urgency. Further
> deployment of FlaggedRevs (pending changes) is the obvious way to do
> this. By hiding recent edits, admins can deal with bad edits in their
> own time, rather reacting in the heat of the moment.
>
> Another thing we could do is to improve the means of communication.
> Better communication often helps to de-escalate a conflict.
>
> We could replace the terrible user talk page interface with an
> easy-to-use real-time messaging framework. We could integrate polite
> template responses with the UI. And we could provide a centralised
> forum-like view of such messages, to encourage mediators to review and
> de-escalate emotion-charged conversations.

We could also try to work out ways to make adminship less important.
If protection, blocking, and deletion could be made less necessary and
important in day-to-day editing, that would reduce the importance of
admins and reduce the difference between established and new
contributors. You could often make do with much "softer" versions of
these three things, which could be given out much more liberally.

For instance, to replace blocking, you could have a system whereby any
reasonably established editor (> X edits/Y days) can place another
editor or IP address in moderation, so that their edits have to be
approved before going live, in Flagged Revs style. As with blocking,
any established editor could also reverse such a block. Abuse would
thus be easily reversed and fairly harmless (since the edits could go
through automatically when it's lifted, barring conflicts). Sysops
would only be necessary if people with established accounts abuse
their rights.

Likewise, most deletion doesn't really need to make anything private.
Reasonably established editors could be given the right to soft-delete
a page such that any other such editor could read or undelete it.
This would be fine for the vast majority of deletions, like vanity
pages and spam. Sysops would only have to get involved for copyright
infringement, privacy issues, and so on.

As for protection, we already have Flagged Revs. Lower levels of
flagging should be imposable by people other than sysops, and since
those largely supersede semiprotection, sysops would again only be
needed to adjudicate disputes between established editors (like
full-protecting an edit-warred page). Obviously, all these rights
would be revocable by sysops in the event of abuse.


Unfortunately, I don't think that technical solutions are going to fix
the problem on enwiki. I think the only thing that will do it is if
Wikimedia adopts more explicit policies about creating a friendly
editing environment, and enforces them in the same vein as it does
copyright policies. But that's easier said than done for a number of
reasons.

[Platonides]

Aryeh Gregor wrote:
> We could also try to work out ways to make adminship less important.
> If protection, blocking, and deletion could be made less necessary and
> important in day-to-day editing, that would reduce the importance of
> admins and reduce the difference between established and new
> contributors. You could often make do with much "softer" versions of
> these three things, which could be given out much more liberally.
>
> For instance, to replace blocking, you could have a system whereby any
> reasonably established editor (> X edits/Y days) can place another
> editor or IP address in moderation, so that their edits have to be
> approved before going live, in Flagged Revs style. As with blocking,
> any established editor could also reverse such a block. Abuse would
> thus be easily reversed and fairly harmless (since the edits could go
> through automatically when it's lifted, barring conflicts). Sysops
> would only be necessary if people with established accounts abuse
> their rights.
>
> Likewise, most deletion doesn't really need to make anything private.
> Reasonably established editors could be given the right to soft-delete
> a page such that any other such editor could read or undelete it.
> This would be fine for the vast majority of deletions, like vanity
> pages and spam. Sysops would only have to get involved for copyright
> infringement, privacy issues, and so on.
>
> As for protection, we already have Flagged Revs. Lower levels of
> flagging should be imposable by people other than sysops, and since
> those largely supersede semiprotection, sysops would again only be
> needed to adjudicate disputes between established editors (like
> full-protecting an edit-warred page). Obviously, all these rights
> would be revocable by sysops in the event of abuse.


There's an extension to 'delete' pages by blanking. I find that approach
much more wiki.
We should also work on allowing more protection levels. Fixing problems
with the "if you can protect, you can edit anything" behavior and such.

[masti]

On 12/31/2010 01:02 AM, Platonides wrote:
> There's an extension to 'delete' pages by blanking. I find that approach
> much more wiki.

if you like to be blocked for blanking ...

masti

[Platonides]

masti wrote:
> On 12/31/2010 01:02 AM, Platonides wrote:
>> There's an extension to 'delete' pages by blanking. I find that approach
>> much more wiki.
>
> if you like to be blocked for blanking ...
>
> masti

If it was the right way of deleting, it would actually be the way
specified by the policy... if that page really deserves to be deleted.

[David Gerard]

On 31 December 2010 00:02, Platonides <***@gmail.com> wrote:

> There's an extension to 'delete' pages by blanking. I find that approach
> much more wiki.


"Pure wiki deletion" is a perennial proposal. One problem is that
there doesn't appear to be a wiki anywhere that actually uses it, or
ever have been one. (I've asked for examples before - does anyone have
any?) This suggests that the biggest wiki in the world might not be
the greatest place to be the very first.


- d.

[Conrad Irwin]

On 31 December 2010 00:08, David Gerard <***@gmail.com> wrote:
> On 31 December 2010 00:02, Platonides <***@gmail.com> wrote:
>
>> There's an extension to 'delete' pages by blanking. I find that approach
>> much more wiki.
>
>
> "Pure wiki deletion" is a perennial proposal. One problem is that
> there doesn't appear to be a wiki anywhere that actually uses it, or
> ever have been one. (I've asked for examples before - does anyone have
> any?) This suggests that the biggest wiki in the world might not be
> the greatest place to be the very first.
>

If you want to being the biggest wiki in the world to mean anything,
you need to innovate. Wikipedia will continue to stagnate if everyone
is too scared to try out new stuff. This is in my mind the biggest
problem facing Wikimedia — it's suffering from complete feature-freeze
because everyone is so scared of making a mistake. On all fronts,
encyclopedic, social, technical, nothing has really moved forward at
all for the last year or two. Sure, we've optimized a few workflows,
tightened a few procedures, and added some content — but there's no
innovation, nothing exciting and new.

Evolution is the best model we have for how to build something, the
way to keep progress going is to continually try new things; if they
fail, "meh", if they succeed — "yay"! There are no planning meetings,
no months of deliberation about exactly what shape a finger should be.
Sure, nothing built by evolution is "perfect", but that's fine, it
will continue to get better in ways not even imaginable from this
point in time (everyone knows you can't see into the future, so stop
wasting time trying). One reason that wikis are such a good way of
creating content is that they use the same process — anyone can make a
random change. If it is good, it is kept; if not it isn't. The same
model is appearing in other places too. Github allows random people to
change software, and only the good stuff gets merged. Google does the
same: Wave was a fun idea, it turns out it was also useless — oh well,
lesson learnt, move on.

There is no Wikipedia-killer in a concrete sense. The world will
continue to evolve. Wikipedia has a simple choice: evolve or get left
behind.

Conrad

[Alex Brollo]

2010/12/31 Conrad Irwin <***@gmail.com>

>
>
> Evolution is the best model we have for how to build something, the
> way to keep progress going is to continually try new things; if they
> fail, "meh", if they succeed — "yay"!
>

Just to add a little bit of "pure theory" into the talk, wiki project is
simply one of the most interesting, and successful, models of "adaptive
complex systems theory". I encourage anyone to take a deeper look into it.
It's both interesting for wiki users/sysops/high level managers and for
complex systems researchers.

I guess, complex system theory wuld suggest too politics. Just an example:
as in evolution, best environment where something new appears is not the
wider environment, but the small ones, the "islands", just like Galapagos in
evolution! This would suggest a great attention about what happens into
smaller wiki projects. I guess, the most interesting things could be found
there, while not so much evolution can be expected into the "mammoth"
project. ;-)

Alex (from it.source)

[Alex]

On 12/29/2010 10:26 PM, Tim Starling wrote:
> On 29/12/10 18:31, Neil Kandalgaonkar wrote:
>> I've been inspired by the discussion David Gerard and Brion Vibber
>> kicked off, and I think they are headed in the right direction.
>>
>> But I just want to ask a separate, but related question.
>>
>> Let's imagine you wanted to start a rival to Wikipedia. Assume that you
>> are motivated by money, and that venture capitalists promise you can be
>> paid gazillions of dollars if you can do one, or many, of the following:
>>
>> 1 - Become a more attractive home to the WP editors. Get them to work on
>> your content.
>>
>> 2 - Take the free content from WP, and use it in this new system. But
>> make it much better, in a way Wikipedia can't match.
>
> This has been done before: Wikinfo, Citizendium, etc.
>
>> 3 - Attract even more readers, or perhaps a niche group of
>> super-passionate readers that you can use to build a new community.
>
> This is basically Wikia's business model. I think you need to think
> outside the box.
>
> I would make it more like World of Warcraft. We should incentivise
> people to set up wiki sweatshops in Indonesia, paying local people to
> "grind" all day, cleaning up articles, in order to build up a level 10
> admin character that can then be sold for thousands of dollars on the
> open market. Also it should have cool graphics.
>
> OK, if you want a real answer: I think if you could convince admins to
> be nicer to people, then that would make a bigger impact to
> Wikipedia's long-term viability than any ease-of-editing feature.
> Making editing easier will give you a one-off jump in editing
> statistics, it won't address the trend.
>
> We know from interviews and departure messages that the editing
> interface creates an initial barrier for entry, but for people who get
> past that barrier, various social factors, such as incivility and
> bureaucracy, limit the time they spend contributing.
>
> Once you burn someone out, they don't come back for a long time, maybe
> not ever. So you introduce a downwards trend which extends over
> decades, until the rate at which we burn people out meets the rate at
> which new editors are born.
>
> Active, established editors have a battlefront mentality. They feel as
> if they are fighting for the survival of Wikipedia against a constant
> stream of newbies who don't understand or don't care about our
> policies. As the stream of newbies increases, they become more
> desperate, and resort to more desperate (and less civil) measures for
> controlling the flood.
>
> Making editing easier could actually be counterproductive. If we let
> more people past the editing interface barrier before we fix our
> social problems, then we could burn out the majority of the Internet
> population before we figure out what's going on. Increasing the number
> of new editors by a large factor will increase the anxiety level of
> admins, and thus accelerate this process.

One thing that I think could help, at least on the English Wikipedia,
would be to further restrict new article creation. Right now, any
registered user can create a new article, and according to some
statistics I gathered a few months ago[1], almost 25% of new users make
their first edit creating an article. 81% of those users had their
article deleted and <0.1% of them were still editing a few (6-7) months
later, compared to 4% for the 19% whose articles were kept, giving a
total retention rate of 1.3%.

However, for the 75% of users who started by editing an existing
article, the overall retention rate was 2.5%. Still a small number, but
almost double the rate for the article creation route.

The English Wikipedia, with 3.5 million articles, has been scraping the
bottom of the notability barrel for a while. Creating a proper new
article is not an especially easy task in terms of editing, yet the
project practically encourages new users to do it. We're dropping new
users into the deep end of the pool, then getting angry at them when
they start to drown. What we should be doing instead is suggesting that
users add their information to an existing article somewhere (with
various tools to help them find it). And if they can't find anything
remotely related in 3.5 million articles, ask themselves whether they
still think its an appropriate topic.

This is an area where the foundation potentially could step in to change
things. Its never going to happen through the community, since there's
too many people (or at least too many loud people) with a "more is
better" mentality. (Part of the reason I gathered the stats was to prove
that most new users don't start by creating an article). They'll scream
and moan for a while about how we're being anti-wiki, but in the end,
most probably won't really care that much.

--
Alex (wikipedia:en:User:Mr.Z-man)

[Platonides]

Alex wrote:
> One thing that I think could help, at least on the English Wikipedia,
> would be to further restrict new article creation. Right now, any
> registered user can create a new article, and according to some
> statistics I gathered a few months ago[1], almost 25% of new users make
> their first edit creating an article. 81% of those users had their
> article deleted and <0.1% of them were still editing a few (6-7) months
> later, compared to 4% for the 19% whose articles were kept, giving a
> total retention rate of 1.3%.
>
> However, for the 75% of users who started by editing an existing
> article, the overall retention rate was 2.5%. Still a small number, but
> almost double the rate for the article creation route.


This is significant, but I'm not convinced about the reason.

There is surely an attacking factor. You make them go through hoops,
having to register an account, then destroy its work. It's normal that
some potentially good contributors leave. But many of those are single
purpose accounts which would only be interested in adding its myspace
band, ever.
We should support the first type users, but we don't want even its
register for the second type.


> The English Wikipedia, with 3.5 million articles, has been scraping the
> bottom of the notability barrel for a while. Creating a proper new
> article is not an especially easy task in terms of editing, yet the
> project practically encourages new users to do it. We're dropping new
> users into the deep end of the pool, then getting angry at them when
> they start to drown.

Completely. This mentality should be changed.


> What we should be doing instead is suggesting that
> users add their information to an existing article somewhere (with
> various tools to help them find it). And if they can't find anything
> remotely related in 3.5 million articles, ask themselves whether they
> still think its an appropriate topic.

That's a good point, but not suitable for all topics.
If I want to create an article that would have been considered relevant
you shouldn't make me wander in circles. Some people shouldn't be
treated as babies, while others should.

[Erik Moeller]

2010/12/29 Tim Starling <***@wikimedia.org>:
> One thing we can do is to reduce the sense of urgency. Further
> deployment of FlaggedRevs (pending changes) is the obvious way to do
> this. By hiding recent edits, admins can deal with bad edits in their
> own time, rather reacting in the heat of the moment.

The actual effect of FlaggedRevs on revert behavior appears to be, if
anything, to accelerate reverts. See Felipe Ortega's presentation at
Wikimania 2010, page 18 and following:
http://en.wikipedia.org/wiki/File:Felipe_Ortega,_Flagged_revisions_study_results.pdf

Performing review actions as quickly as possible is generally seen by
FlaggedRevs-using communities as one of the key performance indicators
connected with the feature. The moment of performing the review action
also tends to be the moment of reverting. I see no evidence, on the
other hand, that FlaggedRevs has contributed to a decreased sense of
urgency anywhere it's been employed.

It's important to note that FlaggedRevs edits aren't like patches
awaiting review. They must be processed in order for anyone's
subsequent edits to be reader-visible. Logged-in users, on the other
hand, always see the latest version by default. These factors and
others may contribute to a sense that edits must be processed as
quickly as possible.

I do fully agree with the rest of your note. We have sufficient data
to show not only that the resistance against new edits as indicated by
the revert ratio towards new users has increased significantly in the
last few years, but also that only very few of the thousands of new
users who complete their first 10 edits in any given month stick
around. Our former contributors survey showed that among people with
more than 10 edits/month who had stopped editing, 40% did so because
of unpleasant experiences with other editors.

http://strategy.wikimedia.org/wiki/File:Former_contributors_survey_presentation_-_wiki.pdf

While fixing the editing UI is absolutely essential, I strongly agree
with your hypothesis that doing so without regard for the problematic
social dynamics is likely to only accelerate people's negative
experiences. Useful technology changes in the area of new user
interaction are a lot harder to anticipate, however, and the only way
we're going to learn is through lots of small experiments. We can
follow in the footsteps of the GroupLens researchers and others who
have experimented with interface changes such as changes to the revert
process, and how these affect new user retention:

http://en.wikipedia.org/wiki/User:EpochFail/NICE
(See their publications to-date at http://www.grouplens.org/biblio )

Once we've identified paths that are clearly fruitful (e.g. if we find
that an experiment with real-time chat yields useful results), we can
throw more resources at them to implement proper functionality.

Over the holidays, my mother shared her own "newbie biting" story.
She's 64 years old and a professional adult educator. Her clearly
constructive good faith edit in the FlaggedRevs-using German Wikipedia
[1] was reverted within the minute it was made, without a comment of
any kind. She explained that she doesn't have enough frustration
tolerance to deal with this kind of behavior.

It's quite likely that we won't be able to make Wikipedia
frustration-free enough to retain someone like my mother as an editor,
but we should be able to make it a significantly more pleasant
experience than it is today.

[1] http://de.wikipedia.org/w/index.php?title=Transaktionsanalyse&diff=76794057&oldid=75722161


--
Erik Möller
Deputy Director, Wikimedia Foundation

Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate

[Alex Brollo]

Can I suggest a really simple trick to inject something new into
"stagnating" wikipedia?

Simply install Labeled Section Trasclusion into a large pedia project; don't
ask, simply install it. If you'd ask, typical pedian boldness would raise a
comment "Thanks, we don't need such a thing" for sure. They need it... but
they don't know, nor they can admit that a small sister project like source
uses currently something very useful.

Let they discover the #lst surprising power.

Alex

[Roan Kattouw]

2011/1/4 Alex Brollo <***@gmail.com>:
> Simply install Labeled Section Trasclusion into a large pedia project;
Just from looking at the LST code, I can tell that it has at least one
performance problem: it initializes the parser on every request. This
is easy to fix, so I'll fix it today. I can also imagine that there
would be other performance concerns with LST preventing its deployment
to large wikis, but I'm not sure of that.

Roan Kattouw (Catrope)

[Alex Brollo]

2011/1/4 Roan Kattouw <***@gmail.com>

> Just from looking at the LST code, I can tell that it has at least one
> performance problem: it initializes the parser on every request. This
> is easy to fix, so I'll fix it today. I can also imagine that there
> would be other performance concerns with LST preventing its deployment
> to large wikis, but I'm not sure of that.
>

Excellent, I'm a passionate user of #lst extension, and I like that its code
can be optimized (so I feel combortable to use it more and more). I can't
read php, and I take this opportunity to ask you:

1. is #lsth option compatible with default #lst use?
2. I can imagine that #lst simply runs as a "substring finder", and I
imagine that substring search is really an efficient, fast and
resource-sparing server routine. Am I true?
3. when I ask for a section into a page, the same page is saved into a
cache, so that next calls for other sections of the same page are fast and
resource-sparing?

What a "creative" use of #lst allows, if it is really an efficient, light
routine, is to build named variables and arrays of named variables into one
page; I can't imagine what a good programmer could do with such a powerful
tool. I'm, as you can imagine, far from a good programmer, nevertheless I
built easily routines for unbeliavable results. Perhaps, coming back to the
topic..... a good programmer would disrupt wikipedia using #lst? :-)

Alex

[Roan Kattouw]

2011/1/4 Alex Brollo <***@gmail.com>:
> Excellent, I'm a passionate user of #lst extension, and I like that its code
> can be optimized (so I feel combortable to use it more and more). I can't
> read php, and I take this opportunity to ask you:
>
I haven't read the code in detail, and I can't really answer these
question until I have. I'll look at these later today, I have some
other things to do first.

> 1. is #lsth option compatible with default #lst use?
No idea what #lsth even is or does, nor what you mean by 'compatible'
in this case.

> 2. I can imagine that #lst simply runs as a "substring finder", and I
> imagine that substring search is really an efficient, fast and
> resource-sparing server routine. Am I true?
It does seem to load the entire page text (wikitext I think, not sure)
and look for the section somehow, but I haven't looked at how it does
this in detail.

> 3. when I ask for a section into a page, the same page is saved into a
> cache, so that next calls for other sections of the same page are fast and
> resource-sparing?
>
I'm not sure whether LST is caching as much as it should. I can tell
you though that the "fetch the wikitext of revision Y of page Z"
operation is already cached in MW core. Whether the "fetch the
wikitext of section X of revision Y of page Z" operation is cached
(and whether it makes sense to do so), I don't know.

> What a "creative" use of #lst allows, if it is really an efficient, light
> routine, is to build named variables and arrays of named variables into one
> page; I can't imagine what a good programmer could do with such a powerful
> tool. I'm, as you can imagine, far from a good programmer, nevertheless I
> built easily routines for unbeliavable results. Perhaps, coming back to the
> topic.....  a good programmer would disrupt wikipedia using #lst? :-)
>
Using #lst to implement variables in wikitext sounds like a terrible
hack, similar to how using {{padleft:}} to implement string functions
in wikitext is a terrible hack.

Roan Kattouw (Catrope)

[Paul Houle]

On 12/29/2010 2:31 AM, Neil Kandalgaonkar wrote:
> Let's imagine you wanted to start a rival to Wikipedia. Assume that you
> are motivated by money, and that venture capitalists promise you can be
> paid gazillions of dollars if you can do one, or many, of the following:
>
Ok, first of all you need a pot of gold at the end of the
rainbow. Let's assume it's a real business model and not that you know
a few folks who have $1B burning a hole in their pocket. Let's also
assume that it's a business model basic on getting a lot of traffic...

Secondly, if you want to go up against 'Wikipedia as a whole',
that's a very difficult problem. Wikipedia is one of the strongest
sites on the internet in terms of S.E.O., not because of any nasty
stuff, but because so many people link to Wikipedia articles from all
over the web. Wikipedia ranks highly for many terms and that's a
situation that Google & Bing don't mind, since Wikipedia has something
halfway decent to say about most topics... It makes search engines seem
smart.

To overturn Wikipedia on the conventional web, you'd really need
to beat it at S.E.O. Sneaky-peet tricks won't help you that much when
you're working at this scale, because if you're able to make enough
phony links to challenge one of the most-linked sites on Earth, you're
probably going to set off alarm bells up and down the West coast.
Thus, the challenge of a two-sided market faces anybody who wants to
'beat' Wikipedia, and I think it's just too hard a nut to crack, even
if you've got software that's way better and if you've got a monster
marketing budget.

I think there are three ways you can 'beat' Wikipedia in a smaller
sense. (i) in another medium, (ii) by targeting very specific
verticals, or (iii) by creating derivative products that add a very
specific kind of value (that is, targeting a horizontal)

In (i) I think of companies like Foursquare and Fotopedia that
follow a mobile-first strategy. If mobile apps got really big and
eclipsed the 'web as we know it', I can see a space for a Wikipedia
successor. This could entirely bypass the S.E.O. problem, but couldn't
Wikipedia fight back with a mobile app of it's own? On the other hand,
this might not be so plausible: the better mobile devices do an O.K.
job with 'HTML 5' and with improvements in hardware, networking and in
HTML-related specifications, so there might be no real advantage in
having 'an app for that'. Already people are complaining that a
collection of apps on your device creates a number of 'walled gardens'
that can't be searched in aggregate, and these kinds of pressures may
erode the progress of apps.

For (ii) I think of Wikia, which hosts things like

http://mario.wikia.com/wiki/MarioWiki

Stuff like this drives deletionists nuts on Wikipedia, but having
a place for them to live in Wikia makes everybody happy. Here's a place
where the Notability policy means that Wikipedia isn't competitive.
Now, in general, Wikia is trying to do this for thousands of subjects
(which might compete with Wikipedia overall) and they've had some
success, but not an overwhelming amount.

Speaking of notability, another direction is to make something
that's more comprehensive than Wikipedia. Consider Freebase, which
accepts Person records for any non-fictional person and has detailed
records of millions of TV episodes, music tracks, books, etc. If
Wikipedia refuses to go someplace, they create opportunities.

As for (iii) you're more likely to have a complementary
relationship with Wikipedia. You can take advantage of Wikipedia's
success and get some income to pay for people and machines. There
wouldn't be any possibility of 'replacing' Wikipedia except in a crazy
long-term scenario where, say, we can convert Wikipedia into a
knowledge base that can grow and update itself with limited human
intervention. (Personally I think this is 10-25 years off)

Anyhow, I could talk your ear off about (iii) but I'd make you
sign an N.D.A. first. ;-)

[Michael Dale]

Looking over the thread, there are lots of good ideas. Its really
important to have some plan towards cleaning up abstractions between
"structured data", "procedures in representation", "visual
representation" and "tools for participation".

But, I think its correct to identify the social aspects of the projects
as more critical than purity of abstractions within wikitext. Tools,
bots and scripts and clever ui components can abstract away some of the
pain of the underlining platform as long as people are willing to accept
a bit of abstraction leakage / lack of coverage in some areas as part of
moving to something better.

One area that I did not see much mention of in this thread is automated
systems for reputation. Reputation systems would be useful both for user
interactions and for gauging expertise within particular knowledge domains.

Social capital within wikikmedia projects is presently stored in
incredibly unstructured ways and has little bearing on user privileges
or how the actions of others are represented to you, and how your
actions are represented to others. Its presently based on traditional
small scale capacities of individuals to gauge social standing within
their social networks and or to read user pages.

We can see automatic reputation system emerging anytime you want to
share anything online be it making a small loan to trading used DVDs.
Sharing information should adopt some similar principals.

There has been some good work done in this area with wikitrust system (
and other user moderation / karma systems ). Tying that data into smart
interface flows that reward positive social behaviour and productive
contributions, should make it "more fun" to participate in the projects
and result in more fluid higher quality information sharing.

peace,
--michael

On 12/29/2010 01:31 AM, Neil Kandalgaonkar wrote:
> I've been inspired by the discussion David Gerard and Brion Vibber
> kicked off, and I think they are headed in the right direction.
>
> But I just want to ask a separate, but related question.
>
> Let's imagine you wanted to start a rival to Wikipedia. Assume that you
> are motivated by money, and that venture capitalists promise you can be
> paid gazillions of dollars if you can do one, or many, of the following:
>
> 1 - Become a more attractive home to the WP editors. Get them to work on
> your content.
>
> 2 - Take the free content from WP, and use it in this new system. But
> make it much better, in a way Wikipedia can't match.
>
> 3 - Attract even more readers, or perhaps a niche group of
> super-passionate readers that you can use to build a new community.
>
> In other words, if you had no legacy, and just wanted to build something
> from zero, how would you go about creating an innovation that was
> disruptive to Wikipedia, in fact something that made Wikipedia look like
> Friendster or Myspace compared to Facebook?
>
> And there's a followup question to this -- but you're all smart people
> and can guess what it is.
>

[Happy-melon]

"Michael Dale" <***@wikimedia.org> wrote in message
news:***@wikimedia.org...
> One area that I did not see much mention of in this thread is automated
> systems for reputation. Reputation systems would be useful both for user
> interactions and for gauging expertise within particular knowledge
> domains.
>
> Social capital within wikikmedia projects is presently stored in
> incredibly unstructured ways and has little bearing on user privileges
> or how the actions of others are represented to you, and how your
> actions are represented to others. Its presently based on traditional
> small scale capacities of individuals to gauge social standing within
> their social networks and or to read user pages.
>
> We can see automatic reputation system emerging anytime you want to
> share anything online be it making a small loan to trading used DVDs.
> Sharing information should adopt some similar principals.
>
> There has been some good work done in this area with wikitrust system (
> and other user moderation / karma systems ). Tying that data into smart
> interface flows that reward positive social behaviour and productive
> contributions, should make it "more fun" to participate in the projects
> and result in more fluid higher quality information sharing.
>
> peace,
> --michael

I think this is a fascinating idea, and one that I think meets a very
valuable criterion: being more useful to newcomers, who are used to seeing
such things on other sites, than to established editors (who will inevitably
hate it). I can see a deployment path along the lines of the Foundation
saying "we are going to enable this extension, whether or not you ask for
it. You do not have to use it, but you may not disable it.", and watching
what happens. It could well be months or years before people get over
complaining about it and it start to bed down. Of course in that time (and
generally) it needs to be immune to various forms of credit farming, which
could lead to some interesting metrics to try and ensure that sockmasters
cannot earn huge reputations by passing credit amongst their socks, while
ordinary users can be rewarded.

With ideas like this, and more generally, I think the Foundation has an
increasing role to play in fighting the growing inertia in the projects.
It's easy to say that any intervention will damage the community and so
should be avoided; but let's not forget that a (mildly) torn muscle will
heal stronger, and that the alternative is, as mentioned, complete
stagnation. It's important that the community keep evolving and innovating
as well; and if that means that some of its more inflexible members cannot
keep up and leave, so be it, as long as they are replaced by new and excited
members such that the community as a whole remains vibrant. Of course
that's a horribly difficult balance to strike, and it would be easy to kill
the golden goose. But the goose is now getting rather grey and arthritic,
and we could really do with some golden goslings right now.

--HM

[Brion Vibber]

On Dec 28, 2010 11:31 PM, "Neil Kandalgaonkar" <***@wikimedia.org> wrote:
>
> I've been inspired by the discussion David Gerard and Brion Vibber
> kicked off, and I think they are headed in the right direction.
>
> But I just want to ask a separate, but related question.
>
> Let's imagine you wanted to start a rival to Wikipedia. Assume that you

I think this isn't as useful a question as it might be; defining a project
in terms of competing with something else leads to stagnation, not
innovation.

A better question might be: what would be a project that would help people
involving freely distributable and modifiable educational and reference
materials, that you could create with today and tomorrow's tech and
sufficient resources, that doesn't exist or work well today?

-- brion vibber (brion @ pobox.com)

[Joe Corneli]

"In dotCommunist Ew Ork, Aris, and Ome, Wikipedia disrupts you!"

Suggestion: Set the sights a little higher, and I'd say start by
ditching the "disruption" metaphor, which is fine and good for firms,
but less sensible in a landscape that's already massively and
"organically" distributed (I'm thinking of the free culture movement
as a whole). If the rhetorical question is "how to build a better
encyclopedia?" or "how to further the WMF's mission?" -- here's
something: how about some specific and well-thought out proposals and
a way to discuss them that doesn't devolve to some sort of punditry
pissing contest? Like, a UserVoice-style feedback system (instead of
this: http://www.mediawiki.org/wiki/Bugzilla#Requesting_a_feature) and
clear way to keep track of project and subproject progress (Redmine?),
including a way to make sense of the priorities and other trends that
govern progress on the current set of open bugs
(https://bugzilla.wikimedia.org/buglist.cgi?quicksearch=status%3Aopen).

In real simple terms, know thyself!

[lampak]

I've been following the discussion and as I can see it's already become
rather unproductive*. So I hope my cutting in will not be very much out
of place (even if I don't really know what I'm talking about).

Many people here has stated the main reason why a WYSIWYG editor is not
feasible is the current wikitext syntax.

What's actually wrong with it?

The main thing I can thing of is the fact one template may include an
opening of a table etc. and another one a closing (e.g. {{col-begin}},
{{col-end}}). It makes it impossible to isolate the template from the
rest of the article - draw a frame around it, say "this box here is a
template".

It could be fixed by forbidding leaving unclosed tags in templates. As a
replacement, a kind of foreach loop could be introduced to iterate
through an unspecified number of arguments.

Lack of standardisation has also been mentioned. Something else?

I've tried to think how a perfect parser should work. Most of this has
been already mentioned. I think it should work in two steps: first
tokenise the code and transform it into an intermediate tree structure like
*paragraph
title:
* plain text: "Section 1"
content:
* plain text: "foo"
* bold text:
* plain text: "bar"
* template
name: "Infobox"
* argument
name: "last name":
value:
* plain text: "Shakespear"
and so on. Then this structure could be transformed into a) HTML for
display, b) JSON for the WYSIWYG editor. Thanks for this you wouldn't
need to write a whole new JS parser. The editor would get a half-ready
product. The JS code would need to be able to: a) transform this
structure into HTML, b) modify the structure, c) transform this
structure back into wikitext.

But I guess it's more realistic to write a new JS parser than to write a
new PHP parser. The former can start as a stub, the latter would need to
be fully operational from the beginning.

Stephanie's suggestions are also interesting.

lampak

* (except the WYSIWTF, of course)

[Roan Kattouw]

2011/1/1 lampak <***@gmail.com>:
> It could be fixed by forbidding leaving unclosed tags in templates.
[...]
> I've tried to think how a perfect parser should work. Most of this has
> been already mentioned. I think it should work in two steps: first
> tokenise the code and transform it into an intermediate tree structure like
[...]
> and so on. Then this structure could be transformed into a) HTML for
> display, b) JSON for the WYSIWYG editor. Thanks for this you wouldn't
> need to write a whole new JS parser. The editor would get a half-ready
> product. The JS code would need to be able to: a) transform this
> structure into HTML, b) modify the structure, c) transform this
> structure back into wikitext.
>
Trevor Parscal already has a proof-of-concept parser that follows this
philosophy pretty much to the letter. I don't think it's in our SVN
repository yet (he said he would commit it some time ago) and I
haven't succeeded in convincing him to reply on this list (holidays, I
guess), but he's been playing around for it for about nine months now,
on and off, and from what I've heard and seen it's promising and
entirely in the spirit of your post.

Roan Kattouw (Catrope)

[Jay Ashworth]

----- Original Message -----
> From: "lampak" <***@gmail.com>

> I've been following the discussion and as I can see it's already
> become rather unproductive*. So I hope my cutting in will not be very much
> out of place (even if I don't really know what I'm talking about).
>
> Many people here has stated the main reason why a WYSIWYG editor is
> not feasible is the current wikitext syntax.
>
> What's actually wrong with it?

Oh god! *Run*!!!!!

:-)

This has been done a dozen times in the last 5 years, lampak. The short
version, as much as *I* am displeased with the fact that we'll never have
*bold*, /italic/ and _underscore_, is that the installed base, both of
articles and editors, means that Mediawikitext will never change.

It *might* be possible to *extend* it, but that requires that at least
one of the 94 projects to write a formally defined parser for it, in
something resembling yacc, would have to complete -- and to my knowledge,
none has done so.

Cheers,
-- jra

[George Herbert]

On Sun, Jan 2, 2011 at 6:28 AM, Jay Ashworth <***@baylink.com> wrote:
> [...]
> This has been done a dozen times in the last 5 years, lampak.  The short
> version, as much as *I* am displeased with the fact that we'll never have
> *bold*, /italic/ and _underscore_, is that the installed base, both of
> articles and editors, means that Mediawikitext will never change.


That we've multiply concluded that it will never change doesn't mean
it won't; as a thought exercise, as I suggested in OtherThread, we
should consider negating that conclusion and seeing what happens.


--
-george william herbert
***@gmail.com

[Rob Lanphier]

On Mon, Jan 3, 2011 at 4:59 PM, George Herbert <***@gmail.com> wrote:
> That we've multiply concluded that it will never change doesn't mean
> it won't; as a thought exercise, as I suggested in OtherThread, we
> should consider negating that conclusion and seeing what happens.

Agreed. I think part of the problem in the past is that the
conversation generally focused on the actual syntax, and not enough on
the incremental changes that we can make to MediaWiki to make this
happen.

If, for example, we can build some sort of per-revision indicator of
markup language (sort of similar to mime type) which would let us
support multiple parsers on the same wiki, then it would be possible
to build alternate parsers that people could try out on a per-article
basis (and more importantly, revert if it doesn't pan out). The
thousands of MediaWiki installs could try out different syntax
options, and maybe a clear winner would emerge.

Rob

[Chad]

On Mon, Jan 3, 2011 at 8:41 PM, Rob Lanphier <***@wikimedia.org> wrote:
> If, for example, we can build some sort of per-revision indicator of
> markup language (sort of similar to mime type) which would let us
> support multiple parsers on the same wiki, then it would be possible
> to build alternate parsers that people could try out on a per-article
> basis (and more importantly, revert if it doesn't pan out).  The
> thousands of MediaWiki installs could try out different syntax
> options, and maybe a clear winner would emerge.
>

Or you end up supporting 5 different parsers that people like
for slightly different reasons :)

-Chad

[Rob Lanphier]

On Mon, Jan 3, 2011 at 5:54 PM, Chad <***@gmail.com> wrote:
> On Mon, Jan 3, 2011 at 8:41 PM, Rob Lanphier <***@wikimedia.org> wrote:
>> If, for example, we can build some sort of per-revision indicator of
>> markup language (sort of similar to mime type) which would let us
>> support multiple parsers on the same wiki, then it would be possible
>> to build alternate parsers that people could try out on a per-article
>> basis (and more importantly, revert if it doesn't pan out).  The
>> thousands of MediaWiki installs could try out different syntax
>> options, and maybe a clear winner would emerge.
>
> Or you end up supporting 5 different parsers that people like
> for slightly different reasons :)

Yup, that would definitely be a strong possibility without a
disciplined approach. However, done correctly, killing off fringe
parsers on a particular wiki would be fairly easy to do. Just because
the underlying wiki engine allows for 5 different parsers, doesn't
mean a particular wiki would need to allow the creation of new pages
or new revisions using any of the 5. If we build the tools that allow
admins some ability to constrain the choices, it doesn't have to get
too out of hand on a particular wiki.

If we were to go down this development path, we'd need to commit ahead
of time to be pretty stingy about what we bless as a "supported"
parser, and brutal about killing off support for outdated parsers.

Rob

[Alex Brollo]

2011/1/4 Rob Lanphier <***@robla.net>

> On Mon, Jan 3, 2011 at 5:54 PM, Chad <***@gmail.com> wrote:
> > On Mon, Jan 3, 2011 at 8:41 PM, Rob Lanphier <***@wikimedia.org>
> wrote:
> >> If, for example, we can build some sort of per-revision indicator of
> >> markup language (sort of similar to mime type) which would let us
> >> support multiple parsers on the same wiki, then it would be possible
> >> to build alternate parsers that people could try out on a per-article
> >> basis (and more importantly, revert if it doesn't pan out). The
> >> thousands of MediaWiki installs could try out different syntax
> >> options, and maybe a clear winner would emerge.
> >
> > Or you end up supporting 5 different parsers that people like
> > for slightly different reasons :)
>
> Yup, that would definitely be a strong possibility without a
> disciplined approach. However, done correctly, killing off fringe
> parsers on a particular wiki would be fairly easy to do. Just because
> the underlying wiki engine allows for 5 different parsers, doesn't
> mean a particular wiki would need to allow the creation of new pages
> or new revisions using any of the 5. If we build the tools that allow
> admins some ability to constrain the choices, it doesn't have to get
> too out of hand on a particular wiki.
>
> If we were to go down this development path, we'd need to commit ahead
> of time to be pretty stingy about what we bless as a "supported"
> parser, and brutal about killing off support for outdated parsers.
>
> Rob
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-***@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>

[Alex Brollo]

I apologyze, I sent an empty reply. :-(

Just a brief comment: there's no need of seaching for "a perfect wiki
syntax", since it exists: it's the present model of well formed markup, t.i.
xml.

While digging into subtler troubles from wiki syntax, t.i. difficulties in
parsing it by scripts or understanding fuzzy behavior of the code, I always
find a trouble coming from tha simple fact, that wiki is a markup that isn't
intrinsecally well formed - it doen't respect the simple, basic rules of a
well formed syntax: strict and evident rules about beginning-ending of a
modifier; no mixing of attributes and content inside its "tags", t.i.
templates.

In part, wiki markup can be hacked to take a step forward; I'm using more
and more "well formed templates", splitted into two parts, a "starting
template" and an "ending template". Just a banal example: it.source users
are encouraged to use {{Centrato!l=20em}}.... text ...</div> syntax, where
text - as you see - is outside the template, while the usual
syntax {{Centrato|.... text ... |l=20em}} mixes tags and contents (Centrato
is Italian name of "center" and l attribute states the width of centered
div). I find such a trick extremely useful when parsind text, since - as
follows by the use of a well-formed marckup - I can retrieve the whole text
simply removing any template code and any html tag; an impossible task using
the common "not well formed" syntax, where nothing tells about the nature
of parameters: they only can be classified by "human understanding" of the
template code.... or by the whole body of wiki parser.

Alex

[Ryan Kaldari]

The perfect wiki syntax would be XML (at least behind the scenes). Then
people could use whatever syntax they want and have it easily translated
via XSLT.

Ryan Kaldari

On 1/1/11 9:51 AM, lampak wrote:
> I've been following the discussion and as I can see it's already become
> rather unproductive*. So I hope my cutting in will not be very much out
> of place (even if I don't really know what I'm talking about).
>
> Many people here has stated the main reason why a WYSIWYG editor is not
> feasible is the current wikitext syntax.
>
> What's actually wrong with it?
>
> The main thing I can thing of is the fact one template may include an
> opening of a table etc. and another one a closing (e.g. {{col-begin}},
> {{col-end}}). It makes it impossible to isolate the template from the
> rest of the article - draw a frame around it, say "this box here is a
> template".
>
> It could be fixed by forbidding leaving unclosed tags in templates. As a
> replacement, a kind of foreach loop could be introduced to iterate
> through an unspecified number of arguments.
>
> Lack of standardisation has also been mentioned. Something else?
>
> I've tried to think how a perfect parser should work. Most of this has
> been already mentioned. I think it should work in two steps: first
> tokenise the code and transform it into an intermediate tree structure like
> *paragraph
> title:
> * plain text: "Section 1"
> content:
> * plain text: "foo"
> * bold text:
> * plain text: "bar"
> * template
> name: "Infobox"
> * argument
> name: "last name":
> value:
> * plain text: "Shakespear"
> and so on. Then this structure could be transformed into a) HTML for
> display, b) JSON for the WYSIWYG editor. Thanks for this you wouldn't
> need to write a whole new JS parser. The editor would get a half-ready
> product. The JS code would need to be able to: a) transform this
> structure into HTML, b) modify the structure, c) transform this
> structure back into wikitext.
>
> But I guess it's more realistic to write a new JS parser than to write a
> new PHP parser. The former can start as a stub, the latter would need to
> be fully operational from the beginning.
>
> Stephanie's suggestions are also interesting.
>
> lampak
>
> * (except the WYSIWTF, of course)
>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-***@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>

[Andreas Jonsson]

Have a look at the xwiki/2.0 syntax
(http://platform.xwiki.org/xwiki/bin/view/Main/XWikiSyntax) for an
example of a wikitext syntax that works with WYSIWYG editing.

Best regards,

Andreas Jonsson

2011-01-01 18:51, lampak skrev:
> I've been following the discussion and as I can see it's already become
> rather unproductive*. So I hope my cutting in will not be very much out
> of place (even if I don't really know what I'm talking about).
>
> Many people here has stated the main reason why a WYSIWYG editor is not
> feasible is the current wikitext syntax.
>
> What's actually wrong with it?
>
> The main thing I can thing of is the fact one template may include an
> opening of a table etc. and another one a closing (e.g. {{col-begin}},
> {{col-end}}). It makes it impossible to isolate the template from the
> rest of the article - draw a frame around it, say "this box here is a
> template".
>
> It could be fixed by forbidding leaving unclosed tags in templates. As a
> replacement, a kind of foreach loop could be introduced to iterate
> through an unspecified number of arguments.
>
> Lack of standardisation has also been mentioned. Something else?
>
> I've tried to think how a perfect parser should work. Most of this has
> been already mentioned. I think it should work in two steps: first
> tokenise the code and transform it into an intermediate tree structure like
> *paragraph
> title:
> * plain text: "Section 1"
> content:
> * plain text: "foo"
> * bold text:
> * plain text: "bar"
> * template
> name: "Infobox"
> * argument
> name: "last name":
> value:
> * plain text: "Shakespear"
> and so on. Then this structure could be transformed into a) HTML for
> display, b) JSON for the WYSIWYG editor. Thanks for this you wouldn't
> need to write a whole new JS parser. The editor would get a half-ready
> product. The JS code would need to be able to: a) transform this
> structure into HTML, b) modify the structure, c) transform this
> structure back into wikitext.
>
> But I guess it's more realistic to write a new JS parser than to write a
> new PHP parser. The former can start as a stub, the latter would need to
> be fully operational from the beginning.
>
> Stephanie's suggestions are also interesting.
>
> lampak
>
> * (except the WYSIWTF, of course)
>
>
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-***@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
>