[tw] TiddlyWiki in Education

[Steven Schneider]

There seems to be some interest in this group in using TiddlyWiki as a
platform for teaching, and for delivering educational resources such as
textbooks and presentations. I thought we might use this thread to identify
those interested in this kind of work, and see if there is critical mass to
spin off a few collaborative projects, or at least to bring together some
existing resources.

So, here goes:

I'm a professor in Information Design at SUNY Polytechnic Institute in NY,
USA (hence my user name, stevesuny :). I've used TW in classes for the past
6-7 years, both wikifying texts and teaching students to design and write
interactive texts. At the moment, I am interested in:

# Rendering open texts published as ePub (or mobi or wxr or odt: this is
one example <http://open.lib.umn.edu/americangovernment/>) in TW so that
students can submit assignments as comments on the text.

# Developing tools that use resources from the Internet Archive
<https://archive.org/> as teaching materials.

# Expanding and enhancing DesignWriteStudio
<https://designwritestudio.updog.co/> as a platform for teaching reading &
writing of interactive texts, using TiddlyWiki as both the object of study
as well as object of activity.

If there is interest, perhaps we might spend the next few weeks writing
brief case studies of what we've done, and what we'd like to do? Or,
especially for those in higher education, perhaps there is a conference we
could collaboratively prepare a session proposal for? Anyone with
particular interests in education or textbooks planning on attending the
Euro TW Meet-Up in Oxford next month?

//steve.

[@TiddlyTweeter]

Ciao Steve

Very interested in this. I have long career as an ethnographer (backround:
LSE, MPhil, anthropologist) and archivist (some on-line archives).

I have a large live project right now I am working on with other people. I
hope to use TW for it.

It is all about Charles Dickens' *Great Expectations* and works towards a
fully annotated version of the text. The project also examines numerous
films that have been made of the book for possible new film versions of it.

I'm interested in TW as a solution because it seems to be able to combine
rigor of fidelity to original texts, whilst allowing a lot of customized
freedom on annotation, commentary AND new innovation.

I am not a programmer, which is my handicap, but good on specifying need. I
can willingly, given a real group doing real things, take the take to
document in detail the requirements of the project, so long as there is a
chance it helps me actually instantiate it in TW.

Best wishes
Josiah

Post by Steven Schneider
There seems to be some interest in this group in using TiddlyWiki as a
platform for teaching, and for delivering educational resources such as
textbooks and presentations. I thought we might use this thread to identify
those interested in this kind of work, and see if there is critical mass to
spin off a few collaborative projects, or at least to bring together some
existing resources.
I'm a professor in Information Design at SUNY Polytechnic Institute in NY,
USA (hence my user name, stevesuny :). I've used TW in classes for the past
6-7 years, both wikifying texts and teaching students to design and write
# Rendering open texts published as ePub (or mobi or wxr or odt: this is
one example <http://open.lib.umn.edu/americangovernment/>) in TW so that
students can submit assignments as comments on the text.
# Developing tools that use resources from the Internet Archive
<https://archive.org/> as teaching materials.
# Expanding and enhancing DesignWriteStudio
<https://designwritestudio.updog.co/> as a platform for teaching reading
& writing of interactive texts, using TiddlyWiki as both the object of
study as well as object of activity.
If there is interest, perhaps we might spend the next few weeks writing
brief case studies of what we've done, and what we'd like to do? Or,
especially for those in higher education, perhaps there is a conference we
could collaboratively prepare a session proposal for? Anyone with
particular interests in education or textbooks planning on attending the
Euro TW Meet-Up in Oxford next month?
//steve.

[Lost Admin]

Post by Steven Schneider
...
# Rendering open texts published as ePub (or mobi or wxr or odt: this is
one example <http://open.lib.umn.edu/americangovernment/>) in TW so that
students can submit assignments as comments on the text.
...

A couple of times now, I've seen this talk of TW and ePub. I've done a bit
of digging into the epub format and I don't understand how you can
integrate TW and epub. ePub is a container format for text documents that
uses HTML to encode the document but it also places some pretty tight
limits on the HTML and per-html-file size. On top of all that, the document
needs to fall-back gracefully to the limits imposed by ebook readers with
limited RAM. So, how do you propose to merge the two?

I suppose, if you kept the TW file small enough you could wrap it in the
structure of an epub and it might render properly in some ebook readers but
I don't think you could save changes back into the epub file without a
custom epub reader.

['Mark S.' via TiddlyWiki]

I think he's saying that he wants to convert existing texts into TW.

I've gone through this exercise with some classical texts (e.g. a couple
plays at shakespeare.tiddlyspot.com, bibles). Classical text doesn't have
special formatting to worry about. Well structured data can be imported
fairly quickly. The questions for the mentioned epubs would be, do you want
to bring in all the text formatting, and if so, do you want to do it as TW
markup, Markdown markup, or original HTML ?

The broader questions I would have is, how do you want to "chunk" the text
into TW, and then how do you want overlay or link commentaries? The smaller
the level of chunking, the easier it is to comment on precise bits of text,
but the larger the file size and the slower the TW will be.

I can imagine that you could show all the verses in a text and then have a
highlighted asterisk after the text. Click on the asterisk and there would
appear a note box that a student could record notes about that passage. A
report tiddler would aggregate all the comments made. I'm guessing that
that would be too inflexible.

Good luck,
Mark

Post by Lost Admin

Post by Steven Schneider
...
# Rendering open texts published as ePub (or mobi or wxr or odt: this is
one example <http://open.lib.umn.edu/americangovernment/>) in TW so that
students can submit assignments as comments on the text.
...

A couple of times now, I've seen this talk of TW and ePub. I've done a bit
of digging into the epub format and I don't understand how you can
integrate TW and epub. ePub is a container format for text documents that
uses HTML to encode the document but it also places some pretty tight
limits on the HTML and per-html-file size. On top of all that, the document
needs to fall-back gracefully to the limits imposed by ebook readers with
limited RAM. So, how do you propose to merge the two?
I suppose, if you kept the TW file small enough you could wrap it in the
structure of an epub and it might render properly in some ebook readers but
I don't think you could save changes back into the epub file without a
custom epub reader.

[@TiddlyTweeter]

Ciao Mark S.

I agree that "chunking" is the central issue on performance. It is my
experience in testing that the more fine grained the chunking the slower
the performance. And it gets worse the larger the file.

An interesting issue is, once the text is completed, is whether it would be
possible to output the "ePub version" with optimized JavaScript dedicated
to specific limited functions & not all of what TW normally does. It might
be worth thinking on since ePubs are increasingly ubiquitous.

Josiah

Post by 'Mark S.' via TiddlyWiki
The broader questions I would have is, how do you want to "chunk" the text
into TW, and then how do you want overlay or link commentaries? The smaller
the level of chunking, the easier it is to comment on precise bits of text,
but the larger the file size and the slower the TW will be.

[Steven Schneider]

Hi folks - I made a major faux pas in the group by cc'ing this message to
others -- I meant to forward it to them. If you could, please take care not
to "reply all" so we don't flood their mailbox. My apologies to all.
//steve.

Post by @TiddlyTweeter
Ciao Mark S.
I agree that "chunking" is the central issue on performance. It is my
experience in testing that the more fine grained the chunking the slower
the performance. And it gets worse the larger the file.
An interesting issue is, once the text is completed, is whether it would
be possible to output the "ePub version" with optimized JavaScript
dedicated to specific limited functions & not all of what TW normally does.
It might be worth thinking on since ePubs are increasingly ubiquitous.
Josiah

Post by 'Mark S.' via TiddlyWiki
The broader questions I would have is, how do you want to "chunk" the
text into TW, and then how do you want overlay or link commentaries? The
smaller the level of chunking, the easier it is to comment on precise bits
of text, but the larger the file size and the slower the TW will be.

[@TiddlyTweeter]

Ciao Steve

Its also an issue using Google Groups on web. As a web user of it I'm not
really thinking its also an email list. Looking at it I see there is a CC
field I should uncheck.

BUT the problem is, you as the author maybe want copies to go to those
people, so how would I know when to UNcheck it?

C'est la vie. I will now :-)

Best wishes
Josiah

Post by Steven Schneider
Hi folks - I made a major faux pas in the group by cc'ing this message to
others -- I meant to forward it

['Mark S.' via TiddlyWiki]

I've never seen an epub with javascript ability. I know PDFs can have some
ability, but I don't think I've ever seen an example of one. The problem is
that javascript could be used in theory to hack a system, so adoption has
been slow.

Each tiddler adds 100 to 200 bytes of overhead, depending on additional
fields and tags.

That book you mentioned must have been huge. Unless my math is off, or your
paragraphs are really small, it could be 10 to 20 megs. Even Moby Dick is
only 1.2 megs

Text compresses really well. Thinking out loud, I wonder if there could be
a Tiddler Template to decompress text. Then huge books could be packed into
a TW. You would view the content through a viewer page that would display
the content normally.

-- Mark

Post by @TiddlyTweeter
Ciao Mark S.
I agree that "chunking" is the central issue on performance. It is my
experience in testing that the more fine grained the chunking the slower
the performance. And it gets worse the larger the file.
An interesting issue is, once the text is completed, is whether it would
be possible to output the "ePub version" with optimized JavaScript
dedicated to specific limited functions & not all of what TW normally does.
It might be worth thinking on since ePubs are increasingly ubiquitous.
Josiah

Post by 'Mark S.' via TiddlyWiki
The broader questions I would have is, how do you want to "chunk" the
text into TW, and then how do you want overlay or link commentaries? The
smaller the level of chunking, the easier it is to comment on precise bits
of text, but the larger the file size and the slower the TW will be.

[@TiddlyTweeter]

Ciao Mark

... your paragraphs are really small,

Absolutely right. Novels that have lots of dialogue like ...

"Did you eat the sandwich?", she said

"Are you questioning my motives?", he replied

She responded, "No, just whether you liked cheese with pickle."

... etc

End up as vast numbers of Tiddlers if para chunked.

Josiah

[Lost Admin]

Post by 'Mark S.' via TiddlyWiki
I've never seen an epub with javascript ability. I know PDFs can have some
ability, but I don't think I've ever seen an example of one. The problem is
that javascript could be used in theory to hack a system, so adoption has
been slow.

As I understand it, EPUB version 3 is supposed to support some sort of
interactivity through javascript. However, I don't know how many ebook
readers that support epubs actually fully support version 3.

[@TiddlyTweeter]

There is a fairly good overview of Interactivity in EPUB 3 here:
http://epubzone.org/news/epub-3-and-interactivity

I think the point is that its not gonna universally embrace freed JS. That
does not stop US developing a different approach that approximates them AND
adds other functions.

A sort of honed TW-Pub with better performance at scale than TW native. Or
a neat way to produce ePubs that then get wrapped conventionally. Or both.

Just thoughts
Josiah

Post by Lost Admin
As I understand it, EPUB version 3 is supposed to support some sort of
interactivity through javascript. However, I don't know how many ebook
readers that support epubs actually fully support version 3.

[Steven Schneider]

Got a chance to do a bit more on the epub > html > tiddlywiki workflow.

See http://american-government-imported-text.tiddlyspot.com/ which imports
all tiddlers matching filter from
http://american-government-in-the-information-age.tiddlyspot.com/, which
used text-slicer to parse the html of an epub. I then transclude all
tiddlers using a series of <$list> commands. Not bad for a quick import of
epub. Work to be done, but proof of concept, at least.

//steve.

[Stephen Wilson]

To bring the thread back a little to where it started, I would be
interested in developing tiddlywiki for education.
I would also be interested in looking at ways to export tiddlers easily
into a read only format.
And finally, I would definitely be interested in looking at systems to
either generate questions/ mark questions and give feedback automatically.

My use case is for engineering at level 3 (A level/ Pre University stuff).
My tiddlywiki heavily nested eg :
http://stephenteacher.tiddlyspot.com/#Statics I would need a simple way to
parse that tree, collect the tiddlers and assemble them in order.
On the exercise side I suspect my needs are a little different as ideal I
need maths focused solutions. Essentials would be random question
generation eg the question is a+b=? but a and b can be any number from 1 to
10 so each student gets a slightly different question. Feedback from the
student answer would then be good. The recent if plugin springs to mind as
something which could be leveraged to this. There also needs to be some
way for students to own their work.
I have been curiously looking at http://webwork.maa.org/ and
http://www.u-psud.fr/fr/universite/organisation-generale/services/direction-de-l-innovation-pedagogique/utiliser-wims.html
(Google translate on Chrome to the rescue) for this functionality but have
not had the time to install and play with them. I have also looked at
doing similar in tiddlyspot using the matthcell plugin
http://stephenteacher.tiddlyspot.com/#Interactive%20Steps%20to%20Solving%20Problems
So that's where I'm at and what I'd like to do educationally.

Stephen Wilson

stephenteacher.tiddlyspot.com/

Post by Steven Schneider
Got a chance to do a bit more on the epub > html > tiddlywiki workflow.
See http://american-government-imported-text.tiddlyspot.com/ which
imports all tiddlers matching filter from
http://american-government-in-the-information-age.tiddlyspot.com/, which
used text-slicer to parse the html of an epub. I then transclude all
tiddlers using a series of <$list> commands. Not bad for a quick import of
epub. Work to be done, but proof of concept, at least.
//steve.

[Lost Admin]

I'm curious about something... your student base is smart and technically
savvy (based on your statement of them being pre-university engineering
students), how can you possibly prevent them from gaming the system if you
use tiddlywiki? They will have full access to all the code (all the logic
is in JavaScript that runs on the browser),

Post by Stephen Wilson
To bring the thread back a little to where it started, I would be
interested in developing tiddlywiki for education.
I would also be interested in looking at ways to export tiddlers easily
into a read only format.
And finally, I would definitely be interested in looking at systems to
either generate questions/ mark questions and give feedback automatically.
My use case is for engineering at level 3 (A level/ Pre University
http://stephenteacher.tiddlyspot.com/#Statics I would need a simple way
to parse that tree, collect the tiddlers and assemble them in order.
On the exercise side I suspect my needs are a little different as ideal I
need maths focused solutions. Essentials would be random question
generation eg the question is a+b=? but a and b can be any number from 1 to
10 so each student gets a slightly different question. Feedback from the
student answer would then be good. The recent if plugin springs to mind as
something which could be leveraged to this. There also needs to be some
way for students to own their work.
I have been curiously looking at http://webwork.maa.org/ and
http://www.u-psud.fr/fr/universite/organisation-generale/services/direction-de-l-innovation-pedagogique/utiliser-wims.html
(Google translate on Chrome to the rescue) for this functionality but have
not had the time to install and play with them. I have also looked at
doing similar in tiddlyspot using the matthcell plugin
http://stephenteacher.tiddlyspot.com/#Interactive%20Steps%20to%20Solving%20Problems
So that's where I'm at and what I'd like to do educationally.
Stephen Wilson
stephenteacher.tiddlyspot.com/

Post by Steven Schneider
Got a chance to do a bit more on the epub > html > tiddlywiki workflow.
See http://american-government-imported-text.tiddlyspot.com/ which
imports all tiddlers matching filter from
http://american-government-in-the-information-age.tiddlyspot.com/, which
used text-slicer to parse the html of an epub. I then transclude all
tiddlers using a series of <$list> commands. Not bad for a quick import of
epub. Work to be done, but proof of concept, at least.
//steve.

[Stephen Wilson]

That is a very good question.......I guess I hadn't really thought that far
ahead :)
At least it would take a little more effort to try and game the system than
simply submitting a friends assignment and hoping I don't notice.
They are less tech savvy than you think...

Stephen

Post by Lost Admin
I'm curious about something... your student base is smart and technically
savvy (based on your statement of them being pre-university engineering
students), how can you possibly prevent them from gaming the system if you
use tiddlywiki? They will have full access to all the code (all the logic
is in JavaScript that runs on the browser),

Post by Stephen Wilson
To bring the thread back a little to where it started, I would be
interested in developing tiddlywiki for education.
I would also be interested in looking at ways to export tiddlers easily
into a read only format.
And finally, I would definitely be interested in looking at systems to
either generate questions/ mark questions and give feedback automatically.
My use case is for engineering at level 3 (A level/ Pre University
http://stephenteacher.tiddlyspot.com/#Statics I would need a simple way
to parse that tree, collect the tiddlers and assemble them in order.
On the exercise side I suspect my needs are a little different as ideal I
need maths focused solutions. Essentials would be random question
generation eg the question is a+b=? but a and b can be any number from 1 to
10 so each student gets a slightly different question. Feedback from the
student answer would then be good. The recent if plugin springs to mind as
something which could be leveraged to this. There also needs to be some
way for students to own their work.
I have been curiously looking at http://webwork.maa.org/ and
http://www.u-psud.fr/fr/universite/organisation-generale/services/direction-de-l-innovation-pedagogique/utiliser-wims.html
(Google translate on Chrome to the rescue) for this functionality but have
not had the time to install and play with them. I have also looked at
doing similar in tiddlyspot using the matthcell plugin
http://stephenteacher.tiddlyspot.com/#Interactive%20Steps%20to%20Solving%20Problems
So that's where I'm at and what I'd like to do educationally.
Stephen Wilson
stephenteacher.tiddlyspot.com/

Post by Steven Schneider
Got a chance to do a bit more on the epub > html > tiddlywiki workflow.
See http://american-government-imported-text.tiddlyspot.com/ which
imports all tiddlers matching filter from
http://american-government-in-the-information-age.tiddlyspot.com/,
which used text-slicer to parse the html of an epub. I then transclude all
tiddlers using a series of <$list> commands. Not bad for a quick import of
epub. Work to be done, but proof of concept, at least.
//steve.

[Lost Admin]

Post by Stephen Wilson
That is a very good question.......I guess I hadn't really thought that
far ahead :)
At least it would take a little more effort to try and game the system
than simply submitting a friends assignment and hoping I don't notice.
*They are less tech savvy than you think...*
Stephen

That is disappointing.

[PMario]

Post by Stephen Wilson
That is a very good question.......I guess I hadn't really thought that
far ahead :)
At least it would take a little more effort to try and game the system
than simply submitting a friends assignment and hoping I don't notice.
They are less tech savvy than you think...

If they trick the system, they have to dig relatively deep. .. So you can
use this "assignment work" as the base for a grade :)) ... once

-m

[@TiddlyTweeter]

The context of teaching & "cheating" is interesting. That TW could be
cheated by someone who knew what they were doing is a plus in my view :-).
A+ for smarts. Though B- for not playing fair:-)

On a more serious note, how you could solve this without a server solution?
There is a point at which TW, as currently implemented, isn't really
anywhere an elegant solution.

I am very interested in the potential of Danielo's TW version, *NOTESELF
<https://noteself.github.io/>*, that (1) saves to browser storage, not
files; (2) can co-ordinate its running on multiple devices via a Cloudant
server. I think its a very interesting approach that might be leveraged
towards multi-user approaches and passworded levels of authentication.
Danielo created it for himself, not for explicit multi-user situations, but
both the Cloudant (server database) and PouchDB (local storage database)
have built in potential in the required direction. Its very interesting
because its keeps the full local ownership of TW, yet combines it with a
remote database. It runs with or without it the remote database.

My 2 cents
Josiah

Stephen Wilson wrote...

... it would take a little more effort to try and game the system than

Post by Stephen Wilson
simply submitting a friends assignment and hoping I don't notice.

PMario replied ...
If they trick the system, they have to dig relatively deep. .. So you can
use this "assignment work" as the base for a grade :)) ... once

['Mark S.' via TiddlyWiki]

Owning the work. You could distribute to each student their own password.
Before uploading the assignment, they would encrypt using the password you
provided. This would be effectively like "signing" the document. The
documents could even share the same open file folder safely. Not as
sophisticated as GPG, but should work unless students share their passwords
or have trouble copying/pasting them.

Mark

Post by Stephen Wilson
To bring the thread back a little to where it started, I would be
interested in developing tiddlywiki for education.
I would also be interested in looking at ways to export tiddlers easily
into a read only format.
And finally, I would definitely be interested in looking at systems to
either generate questions/ mark questions and give feedback automatically.
My use case is for engineering at level 3 (A level/ Pre University
http://stephenteacher.tiddlyspot.com/#Statics I would need a simple way
to parse that tree, collect the tiddlers and assemble them in order.
On the exercise side I suspect my needs are a little different as ideal I
need maths focused solutions. Essentials would be random question
generation eg the question is a+b=? but a and b can be any number from 1 to
10 so each student gets a slightly different question. Feedback from the
student answer would then be good. The recent if plugin springs to mind as
something which could be leveraged to this. There also needs to be some
way for students to own their work.
I have been curiously looking at http://webwork.maa.org/ and
http://www.u-psud.fr/fr/universite/organisation-generale/services/direction-de-l-innovation-pedagogique/utiliser-wims.html
(Google translate on Chrome to the rescue) for this functionality but have
not had the time to install and play with them. I have also looked at
doing similar in tiddlyspot using the matthcell plugin
http://stephenteacher.tiddlyspot.com/#Interactive%20Steps%20to%20Solving%20Problems
So that's where I'm at and what I'd like to do educationally.
Stephen Wilson
stephenteacher.tiddlyspot.com/

Post by Steven Schneider
Got a chance to do a bit more on the epub > html > tiddlywiki workflow.
See http://american-government-imported-text.tiddlyspot.com/ which
imports all tiddlers matching filter from
http://american-government-in-the-information-age.tiddlyspot.com/, which
used text-slicer to parse the html of an epub. I then transclude all
tiddlers using a series of <$list> commands. Not bad for a quick import of
epub. Work to be done, but proof of concept, at least.
//steve.

[Steven Schneider]

Stephen,

First, apologies for cc on initial message.

Second, thanks for the response. I suppose if we get 4-5-6 folks interested
in teaching / education / making resources, we can spin off into another
group to talk? But for now, stick with this thread....

Post by Stephen Wilson
To bring the thread back a little to where it started, I would be
interested in developing tiddlywiki for education.
I would also be interested in looking at ways to export tiddlers easily
into a read only format.

I think this is best done using node.js, the way that tiddlywiki.com static
<http://tiddlywiki.com/static/Generating%2520Static%2520Sites%2520with%2520TiddlyWiki.html>
is presented. I'm just on the intro side of learning node.js, but it seems
necessary for anything along these lines.

Post by Stephen Wilson
And finally, I would definitely be interested in looking at systems to
either generate questions/ mark questions and give feedback automatically.

YES!! But feedback/grading gets tricky. It used to be more possible in the
tiddlyspace days, but TW Federation is not super easy (again, probably
requires node.js). One way I've done this is to use google forms for
student submissions. For example,
https://sunypoly-com216.updog.co/syllabus.html#Reflections%3A%20Chapter%202 asks
students to submit their thoughts in a google form, which I then import
(using xlsx import tool; see demo
<https://groups.google.com/forum/#!topic/tiddlywiki/PAqApq8wvGE>) to create
this:
https://sunypoly-com216.updog.co/syllabus.html#Chapter%202%20Reflections.
Grading could be done in the results spreadsheet...

I've also created a google form in which students "voted" using pre-formed
google forms to record their submissions

Post by Stephen Wilson
My use case is for engineering at level 3 (A level/ Pre University
http://stephenteacher.tiddlyspot.com/#Statics I would need a simple way
to parse that tree, collect the tiddlers and assemble them in order.

I think that we can do that already using toc filter
http://tobibeer.github.io/tw5-plugins/#toc


On the exercise side I suspect my needs are a little different as ideal I

Post by Stephen Wilson
need maths focused solutions. Essentials would be random question
generation eg the question is a+b=? but a and b can be any number from 1 to
10 so each student gets a slightly different question.Feedback from the
student answer would then be good. The recent if plugin springs to mind as
something which could be leveraged to this.

I think you are referring to Thomas Elmiger IfAisB
<https://groups.google.com/forum/#!searchin/tiddlywiki/if$20macro%7Csort:relevance/tiddlywiki/s5ntFB3DE_g/V1FzwyGyAgAJ> work
- if so, this might be the approach what we need.

Post by Stephen Wilson
There also needs to be some way for students to own their work.

Well, yes, but let's not miss the opportunity to do half of what we need.
And, I believe, that as Xememex moves forward, that opportunity will
present itself.

Post by Stephen Wilson
So that's where I'm at and what I'd like to do educationally.

Very exciting! Thanks for the response.

//steve.

Post by Stephen Wilson
Stephen Wilson
stephenteacher.tiddlyspot.com/

[Steven Schneider]

Post by 'Mark S.' via TiddlyWiki
I think he's saying that he wants to convert existing texts into TW.

Yes - that is exactly what I am saying. Here is an
example: http://open.lib.umn.edu/americangovernment/

How would I begin to import this into TW?

//steve.

[Steven Schneider]

OK, then, here is a workflow to go from epub -> html -> tiddlywiki.

(1) I download the epub version of
http://open.lib.umn.edu/americangovernment/
(2) Use pandoc to convert to html (http://pandoc.org/)

pandoc -f epub -t html American-Government-and-Politics-in-the-Information-
Age-1479837299._3.epub -o Amgov.html


(3) Import Amgov.html to a text-slicer edition of TiddlyWiki
http://tiddlywiki.com/editions/text-slicer/


(4) The results are here:


http://american-government-in-the-information-age.tiddlyspot.com/


Pretty good for 5 minutes of work - not counting the time to download
pandoc & teach myself to use it :)


(5) Next steps: work on displaying TW tiddlers in an organized fashion
using the list fields. That should be relatively straightforward. I'll
report back to this thread soon...


//steve.

Post by Steven Schneider

Post by 'Mark S.' via TiddlyWiki
I think he's saying that he wants to convert existing texts into TW.

http://open.lib.umn.edu/americangovernment/
How would I begin to import this into TW?
//steve.

['Mark S.' via TiddlyWiki]

Wow - amazing. While I was typing up some approaches, you already did it!

Post by Steven Schneider
OK, then, here is a workflow to go from epub -> html -> tiddlywiki.
(1) I download the epub version of
http://open.lib.umn.edu/americangovernment/
(2) Use pandoc to convert to html (http://pandoc.org/)
pandoc -f epub -t html American-Government-and-Politics-in-the-Information
-Age-1479837299._3.epub -o Amgov.html
(3) Import Amgov.html to a text-slicer edition of TiddlyWiki
http://tiddlywiki.com/editions/text-slicer/
http://american-government-in-the-information-age.tiddlyspot.com/
Pretty good for 5 minutes of work - not counting the time to download
pandoc & teach myself to use it :)
(5) Next steps: work on displaying TW tiddlers in an organized fashion
using the list fields. That should be relatively straightforward. I'll
report back to this thread soon...
//steve.

Post by Steven Schneider

Post by 'Mark S.' via TiddlyWiki
I think he's saying that he wants to convert existing texts into TW.

http://open.lib.umn.edu/americangovernment/
How would I begin to import this into TW?
//steve.

['Mark S.' via TiddlyWiki]

I didn't realize that Jeremy's slicer could handle HTML directly. That's a
real time saver.

I see that some formatting is lost. In particular, the <em> tag used in the
preamble of chapter 1. This might mean that other formatting (like bold)
might also be lost in some places.

It appears to me that you might want to start over with a different
starting tiddler name, since "Sliced up Amgov.htm..." appears in every
tiddler and in every link. I can imagine how students would respond to
that.

I think if it was me I'd want some way to get rid of the pink text.

Have fun!
Mark

Post by Steven Schneider
OK, then, here is a workflow to go from epub -> html -> tiddlywiki.
(1) I download the epub version of
http://open.lib.umn.edu/americangovernment/
(2) Use pandoc to convert to html (http://pandoc.org/)
pandoc -f epub -t html American-Government-and-Politics-in-the-Information
-Age-1479837299._3.epub -o Amgov.html
(3) Import Amgov.html to a text-slicer edition of TiddlyWiki
http://tiddlywiki.com/editions/text-slicer/
http://american-government-in-the-information-age.tiddlyspot.com/
Pretty good for 5 minutes of work - not counting the time to download
pandoc & teach myself to use it :)
(5) Next steps: work on displaying TW tiddlers in an organized fashion
using the list fields. That should be relatively straightforward. I'll
report back to this thread soon...
//steve.

Post by Steven Schneider

Post by 'Mark S.' via TiddlyWiki
I think he's saying that he wants to convert existing texts into TW.

http://open.lib.umn.edu/americangovernment/
How would I begin to import this into TW?
//steve.

[@TiddlyTweeter]

Jeremy's Slicer is excellent for simple cases. It requires certain criteria
to be met to function well.

One thing about PRE-PROCESSORS is how much of the heavy lifting they do or
not.

Its all very well to chunk some vast document into Tiddler units. But what
also matters is what they are capable of for generating organizational
metadata.

I'm personally very interested in how smart pre-processing could be. My aim
being to absolutely reduce to as low a level as possible later manual
editing.

Josiah

['Mark S.' via TiddlyWiki]

There are multiple approaches, depending on how you want things chunked.
Chunking matters because your students will probably be commenting at
whatever "chunk" level you provide. So if you want paragraph-by-paragraph
chunking, then you're going to need to chunk things into single-paragraph
level tiddler chunks.

The more chunks, the more overhead, which is why it becomes tempting to
chunk at the chapter or section level.

One way to proceed would be to install tiddlyclip. Then go through the
original text and clip the text you wanted piece by piece. You could do
this as plain text or as converted TW markup text. I don't know if
tiddlyclip can clip the original HTML or not -- I'd have to study the docs.
The problems with this approach is that you would need to tag or add fields
to your resulting tiddlers so that you could link a series of paragraphs up
together as chapters and sections. So, it's labor intensive.

Another approach is to drag and drop chapters (or selected paragraphs) into
your new TiddlyWiki and then import the text. This brings in the original
HTML formatting so it looks like the original text but the actual text is
harder to edit and it's going to be a little bit bigger (because the text
has HTML tags in it). You will want to tweak the HTML to remove the
next/back links (either eliminate them or point them at the appropriate
tiddler). Of course, you probably won't need to edit the text after this
tweaking so HTMN might be all right. You will need to add your own titles
to each tiddler chunk you import. You will probably want to either come up
with titles that will allow the entire text to be sorted by title and/or
add fields and tags that will allow the text to be reassembled as one
document.

Another approach would be to find the original HTML source text. I think
the epub could be decompiled to do this. Put all the source HTML into one
big HTML file and then convert it by a series of replacements into TW
markup. Then paste the whole thing at once into a giant tiddler. Jeremy has
a slicer tool/plugin that can then break out the text into smaller tiddlers
of connected text. This last approach may actually be the easiest in the
long run.

As you can see, there's a bunch of things to think about. What size
chunking? How much of the original formatting? How to connect the pieces
together?

HTH
Mark

Post by Steven Schneider

Post by 'Mark S.' via TiddlyWiki
I think he's saying that he wants to convert existing texts into TW.

http://open.lib.umn.edu/americangovernment/
How would I begin to import this into TW?
//steve.

[@TiddlyTweeter]

Ciao Mark & Steve

I have two comments ...

1 - CHUNKING is also about the semantics of the original text. In novels
the working optimal seems to be the paragraph. Though in some cases (Like *Finnegan's
Wake*) it might be the sentence. For poetry its the line. I think its a
mistake to allow practical workability to override the meaning units
needed. Of course, there are cases where chapter units might work. It very
much depends on the nature of the work. And on the purpose of doing it.

2 - HOW to CHUNK is I think very important and interesting issue.
Personally I use a Regular Expression engine that lets me input a vanilla
text and directly output fully formed Tiddlers. Of course I have to put
hours into getting it to work. But the output requires me to do nothing.
Its already fully chunked and ordered with embedded fields. I don't even
JSON. I just inject it directly into a TW.

Best wishes
Josiah

Post by Steven Schneider
OK, then, here is a workflow to go from epub -> html -> tiddlywiki.
There are multiple approaches, depending on how you want things chunked.
Chunking matters because your students will probably be commenting at
whatever "chunk" level you provide.

[@TiddlyTweeter]

Ciao Lost Admin

I agree with you that beyond a certain scale native TW may not work for
ePubs. In tests I did full scale novels (where the unit was 1 tid = 1
paragraph of novel of over 4,000 paras) became unwieldy. HOWEVER, it may be
possible to optimize further. And TW has potential for authoring texts
ready for ePub-ing inside a wrapper. I think there is definite potential.

Josiah

Post by Lost Admin
A couple of times now, I've seen this talk of TW and ePub. I've done a bit
of digging into the epub format and I don't understand how you can
integrate TW and epub. ePub is a container format

[Handoko Suwono]

nice discussion here. I am posting to follow the updates. I am not an
educator, however been a mentor to google code-in which provides TiddlyWiki
some intro tasks for teens pre-university students. And perhaps doing it in
some workshops in conferences. So it's good to know more.

handoko -

Post by Steven Schneider
There seems to be some interest in this group in using TiddlyWiki as a
platform for teaching, and for delivering educational resources such as
textbooks and presentations. I thought we might use this thread to identify
those interested in this kind of work, and see if there is critical mass to
spin off a few collaborative projects, or at least to bring together some
existing resources.