Discussion:
[Axiom-developer] build-improvements and latex
Waldek Hebisch
2006-10-21 14:11:26 UTC
Permalink
Gaby, you probably know this, but just in case: running
Rosetta.pamphlet trough noweb messes it. This is using
Debian noweb. Package identification:

ii nowebm 2.10c-3.3 A WEB-like literate-programming tool
--
Waldek Hebisch
***@math.uni.wroc.pl
Bill Page
2006-10-21 14:32:50 UTC
Permalink
On October 21, 2006 10:11 AM Waldek Hebisch
Post by Waldek Hebisch
Gaby, you probably know this, but just in case: running
Rosetta.pamphlet through noweb messes it. This is using
ii nowebm 2.10c-3.3 A WEB-like literate-programming tool
There is a better version of Rosetta.pamphlet online here:

http://wiki.axiom-developer.org/axiom--test--1/src/doc/Rosetta

I have previously discussed this version with Clifford Yap on this
list.

Shall I check this into Silver /trunk?

Regards,
Bill Page.
Waldek Hebisch
2006-10-21 14:47:37 UTC
Permalink
Post by Bill Page
http://wiki.axiom-developer.org/axiom--test--1/src/doc/Rosetta
I have previously discussed this version with Clifford Yap on this
list.
Shall I check this into Silver /trunk?
I fetched the source. Diff tells me:

--- build-improvements/src/doc/Rosetta.pamphlet 2006-10-21 02:54:55.000000000 +0200
+++ Rosetta.pamphlet 2006-10-21 16:39:28.000000000 +0200
@@ -1,4 +1,4 @@
-\documentclass{book}
+\documentclass{article}
\normalsize\baselineskip=12pt
\parskip=0pt
\parindent=10pt

(and another hunk due to missing newline at the end of file). So
it seems that build-improvements essentially uses your version
(except for \documentclass line).
--
Waldek Hebisch
***@math.uni.wroc.pl
Ralf Hemmecke
2006-10-21 15:54:50 UTC
Permalink
The pamphlet ending is a bit misleading. It is a perfect latex file.

latex Rosetta.pamplet

In a sense you are right. Running that file through noweave (with any
option you like) doesn't give a proper (standalone) latex file.

Ralf
Post by Waldek Hebisch
Post by Bill Page
http://wiki.axiom-developer.org/axiom--test--1/src/doc/Rosetta
I have previously discussed this version with Clifford Yap on this
list.
Shall I check this into Silver /trunk?
--- build-improvements/src/doc/Rosetta.pamphlet 2006-10-21 02:54:55.000000000 +0200
+++ Rosetta.pamphlet 2006-10-21 16:39:28.000000000 +0200
@@ -1,4 +1,4 @@
-\documentclass{book}
+\documentclass{article}
\normalsize\baselineskip=12pt
\parskip=0pt
\parindent=10pt
(and another hunk due to missing newline at the end of file). So
it seems that build-improvements essentially uses your version
(except for \documentclass line).
Bill Page
2006-10-21 15:19:33 UTC
Permalink
Post by Waldek Hebisch
Post by Bill Page
http://wiki.axiom-developer.org/axiom--test--1/src/doc/Rosetta
I have previously discussed this version with Clifford Yap on this
list.
Shall I check this into Silver /trunk?
--- build-improvements/src/doc/Rosetta.pamphlet 2006-10-21
02:54:55.000000000 +0200
+++ Rosetta.pamphlet 2006-10-21 16:39:28.000000000 +0200
@@ -1,4 +1,4 @@
-\documentclass{book}
+\documentclass{article}
\normalsize\baselineskip=12pt
\parskip=0pt
\parindent=10pt
(and another hunk due to missing newline at the end of file). So
it seems that build-improvements essentially uses your version
(except for \documentclass line).
Hmmm... that doesn't seem significant does it?

So you say that the version of noweb that you are using (Debian
version 2.10c-3.3) is messes it? It seems to work fine on the
axiom-developer.org server which is running noweb-2.11b. The
dvi output that you can download from the web is generated by
running noweave and latex on this source file.

BTW, I have been thinking about changing "axiom--test--1" (which
currently is not linked to any repository) to "axiom/trunk" and
re-basing it on Axiom Silver at SourceForge. This would allow me
implement, for example, a button to generate the output of 'svn diff'
of the source for each pamphlet file. This would simplify applying
patches from the online source back to the Silver repository but
without linking it too strongly to the underlying repository.

To update the repository from the online source one could then

What do you think?

Regards,
Bill Page.
root
2006-10-21 22:31:39 UTC
Permalink
Post by Bill Page
BTW, I have been thinking about changing "axiom--test--1" (which
If you bring axiom--test--1 up to silver I'll check in the
--patch-51 changes so far.

arch is much easier to maintain for me.

t
Bill Page
2006-10-21 15:49:58 UTC
Permalink
Post by Bill Page
...
...
So you say that the version of noweb that you are using (Debian
version 2.10c-3.3) is messes it? It seems to work fine on the
axiom-developer.org server which is running noweb-2.11b. The
dvi output that you can download from the web is generated by
running noweave and latex on this source file.
Ooops I have to take that back, I just checked again and I see that
I am actually running the following command on the axiom-developer.org
server:

axiom/mnt/linux/bin/lib/noweave -delay ...

which means that I am probably running Tim Daly's patched version
of noweb from Axiom Gold, not the version of noweb that is actually
installed on the server.

This is significant because the Rosetta.pamphlet file contains some
unescaped << ... >> and [[ ... ]] sequences that are not chunk
references.

I think we should fix these so that Rosetta.pamphlet conforms to the
standard noweb syntax and behaviour.

And I will update the way axiom-developer.org runs noweb so that it
is actually running the system installed version.

Regards,
Bill Page.
Bill Page
2006-10-21 16:21:22 UTC
Permalink
Post by Ralf Hemmecke
The pamphlet ending is a bit misleading. It is a perfect latex file.
latex Rosetta.pamplet
In a sense you are right. Running that file through noweave (with any
option you like) doesn't give a proper (standalone) latex file.
Yes, you are right. One of the advantages of the patch that Tim made
to noweb was that (usually) one does not need to make the distinction
between a noweb format file and a pure latex format file because the
default handling of undefined chunk names is less disruptive.

Note: We can achieve that same affect with an unmodified version
of noweb if we use Norman Ramsey's suggested filter. See extensive
discussion of this in the email archive.

Regards,
Bill Page.
Ralf Hemmecke
2006-10-21 18:46:02 UTC
Permalink
Post by Bill Page
Post by Ralf Hemmecke
The pamphlet ending is a bit misleading. It is a perfect latex file.
latex Rosetta.pamplet
In a sense you are right. Running that file through noweave (with any
option you like) doesn't give a proper (standalone) latex file.
Yes, you are right. One of the advantages of the patch that Tim made
to noweb was that (usually) one does not need to make the distinction
between a noweb format file and a pure latex format file because the
default handling of undefined chunk names is less disruptive.
That has nothing to do with the issue that src/doc/Rosetta.pamphlet does
not contain any code chunk nor any unescaped << (in Gold, Silver or
build-improvements).

Rosetta.pamphlet is a full latex file with \documentclass,
\begin{document} ... \end{document}. If you do

noweave Rosetta.pamphlet > Rosetta.tex

then noweave adds some commands that are defined in noweb.sty (no matter
whether you add switches like -n or -delay). Since Rosetta.pamphlet does
not have any \usepackage in it, latex cannot succeed.

In fact src/doc/Makefile.pamplet has a target where I don't see 'noweave'.

Ralf

${DVI}/Rosetta.dvi: ${IN}/Rosetta.pamphlet
@echo 5 building ${DVI}/Rosetta.dvi from ${IN}/Rosetta.pamphlet
@(cd ${MID} ; \
cp ${IN}/Rosetta.pamphlet ${MID} ; \
if [ -z ${NOISE} ] ; then \
(latex Rosetta.pamphlet --interaction nonstopmode >${TMP}/trace ; \
latex Rosetta.pamphlet --interaction nonstopmode >${TMP}/trace ) ; \
else \
(latex Rosetta.pamphlet --interaction nonstopmode ; \
latex Rosetta.pamphlet --interaction nonstopmode ) ; \
fi ; \
cp Rosetta.dvi ${DVI} )

@
root
2006-10-21 22:36:21 UTC
Permalink
Post by Bill Page
Note: We can achieve that same affect with an unmodified version
of noweb if we use Norman Ramsey's suggested filter. See extensive
discussion of this in the email archive.
why not submit a patch to build-improvements that works just like
the gold version but uses the suggested filter? just make sure that
all of the noweb patches are properly covered.

t
Gabriel Dos Reis
2006-10-22 00:27:34 UTC
Permalink
"Bill Page" <***@synthesis.anikast.ca> writes:

[...]

| Note: We can achieve that same affect with an unmodified version
| of noweb if we use Norman Ramsey's suggested filter.

That would be my preference.

-- Gaby
Bill Page
2006-10-21 19:47:43 UTC
Permalink
Post by Ralf Hemmecke
... One of the advantages of the patch that Tim made
to noweb was that (usually) one does not need to make the
distinction between a noweb format file and a pure latex
format file because the default handling of undefined chunk
names is less disruptive.
That has nothing to do with the issue that src/doc/Rosetta.pamphlet
does not contain any code chunk nor any unescaped << (in Gold,
Silver or build-improvements).
Look at line 273

\Axiom & set [1, 2] & [1, 2] & matrix([[1, 2],[3, 4]])

and many lines after that.

When these lines are processed by the unpatched noweb program the
[[...]] sections are expanded as noweb "quotes", but when processed
by Tim's patched version of noweb this does not happen.
Post by Ralf Hemmecke
Rosetta.pamphlet is a full latex file with \documentclass,
\begin{document} ... \end{document}. If you do
noweave Rosetta.pamphlet > Rosetta.tex
then noweave adds some commands that are defined in noweb.sty
(no matter whether you add switches like -n or -delay). Since
Rosetta.pamphlet does not have any \usepackage in it, latex
cannot succeed.
Actually noweave is usually called with the option -delay like
this

noweave -delay Rosetta.pamphlet > Rosetta.tex

In that case the only noweb-ism that is added occurs after the
\end{document} and has no affect on the latex. You can see this
on the axiom-test--1/src/doc/Rosetta wiki page because the only
error in the latex log is due to the \chapter command which is
not defined for \documentclass{article}.
Post by Ralf Hemmecke
In fact src/doc/Makefile.pamplet has a target where I don't
see 'noweave'.
${DVI}/Rosetta.dvi: ${IN}/Rosetta.pamphlet
@echo 5 building ${DVI}/Rosetta.dvi from ${IN}/Rosetta.pamphlet
@(cd ${MID} ; \
cp ${IN}/Rosetta.pamphlet ${MID} ; \
if [ -z ${NOISE} ] ; then \
(latex Rosetta.pamphlet --interaction nonstopmode
${TMP}/trace ; \
latex Rosetta.pamphlet --interaction nonstopmode
${TMP}/trace ) ; \
else \
(latex Rosetta.pamphlet --interaction nonstopmode ; \
latex Rosetta.pamphlet --interaction nonstopmode ) ; \
fi ; \
cp Rosetta.dvi ${DVI} )
@
You are right and the Makefile is wrong. Everything with extension
.pamphlet should be processed by noweave before calling latex.

Regards,
Bill Page.
Ralf Hemmecke
2006-10-21 20:24:06 UTC
Permalink
Post by Bill Page
Post by Ralf Hemmecke
... One of the advantages of the patch that Tim made to noweb was
that (usually) one does not need to make the distinction between
a noweb format file and a pure latex format file because the
default handling of undefined chunk names is less disruptive.
That has nothing to do with the issue that src/doc/Rosetta.pamphlet
does not contain any code chunk nor any unescaped << (in Gold,
Silver or build-improvements).
Look at line 273
\Axiom & set [1, 2] & [1, 2] & matrix([[1, 2],[3, 4]])
and many lines after that.
When these lines are processed by the unpatched noweb program the
[[...]] sections are expanded as noweb "quotes", but when processed
by Tim's patched version of noweb this does not happen.
I really could not believe that Tim's patch handles that case correctly.
Note that inside [[...]] noweb does never require a chunk name.

If I say

noweave -delay Rosetta.pamphlet > Rosetta.tex

then I get for the above input line (actually it is line 271) the line

\Axiom & set [1, 2] & [1, 2] & matrix({\Tt{}1,\ 2],[3,\
4\nwendquote}) \\

as output and of course

latex Rosetta.tex

ends with the error message

l.271 ... & set [1, 2] & [1, 2] & matrix({\Tt
{}1,\ 2],[3,\
4\nwendquote...

since \Tt is defined in noweb.sty (which is never included).

OK let's do the same thing with the patch.

$AXIOM/bin/lib/noweave -delay Rosetta.pamphlet > Rosetta1.tex

That leads to the line

\Axiom & set [1, 2] & [1, 2] & matrix({\tt{}1,\ 2],[3,\ 4})
\\

and you certainly agree that the original definition of \tt does not
produce [[ in the output.

latex Rosetta.tex

succeeds but the output is simply wrong. The patch is a misfeature!!!
Post by Bill Page
Everything with extension .pamphlet should be processed by noweave
before calling latex.
I completely agree. Doing otherwise just causes confusion.

Ralf
root
2006-10-21 22:42:10 UTC
Permalink
Post by Bill Page
You are right and the Makefile is wrong. Everything with extension
.pamphlet should be processed by noweave before calling latex.
nope. that's not the case.

noweb is just a preprocessing tool, not a requirement for
literate programming.

there is no requirement (or intended requirement) that a .pamphlet
file implies running noweb. some of the files in any documented
axiom system will not contain code and will not need noweb. they
are, however, perfectly vaild .pamphlet files.

t
Ralf Hemmecke
2006-10-21 23:08:33 UTC
Permalink
Post by root
Post by Bill Page
You are right and the Makefile is wrong. Everything with extension
.pamphlet should be processed by noweave before calling latex.
nope. that's not the case.
noweb is just a preprocessing tool, not a requirement for
literate programming.
there is no requirement (or intended requirement) that a .pamphlet
file implies running noweb. some of the files in any documented
axiom system will not contain code and will not need noweb. they
are, however, perfectly vaild .pamphlet files.
t
Please define the format of a .pamphlet file so that people can write
tools to support that format. (If the format is moving, that is not a
problem, but we should always have an up-to-date description of that
format.)

Up to now I thought the format is like the one in noweb, but already
with your patch to noweb, you modified that. Without a documented format
we end up like SPAD is now. A language without formal specification.
Good luck.

Ralf
root
2006-10-22 00:18:59 UTC
Permalink
Post by Ralf Hemmecke
Up to now I thought the format is like the one in noweb, but already
with your patch to noweb, you modified that. Without a documented format
we end up like SPAD is now. A language without formal specification.
Good luck.
It seems that you've tripped across a noweb bug and called it a feature.
The noweb tool mangles files. We need to fix it.

The [[ form is valid axiom syntax and should not be wholly owned by noweb.
(e.g [[1,2],[3,4]]). We can step around the noweb bug by simply placing a
space between the [[ form (e.g [ [) in the Rosetta document. I did that
in the Axiom book. Or we could outlaw the [[ noweb form, since it is
trivial to write the correct latex.

In all of axiom there are straight latex files and many that include
extensions for including code in a verbatim manner. These are the
only features in use that I'm aware of:

0) straight latex, the base case

1) Definition and use, hashing strings to text blocks

<<chunk>>=
@

<<chunk>>

2) Definition extension, adding to existing hash entries

<<chunk>>=
@
<<chunk>>=
@

3) special symbol quoting, latex synomym for \verb usage

[[thing]]


If we were to require that ALL files MUST be passed thru noweb then
there are more changes required to noweb. In particular, noweb needs
to ensure that it produces a valid latex file from a latex file. It
can do this by inserting a \usepackage if it finds <<>>= or [[]] usage
and one is not provided. Or it should "inline" the noweb latex definitions.
Or if there is no \usepackage for noweb, then do nothing harmful.

Although noweb implements three of four features I don't see it as a
requirement. Indeed there has been discussion of other formats such as

* ALLPROSE,
* "booklets", which are pamphlets where the chunk names optionally
contain URIs, see src/doc/booklet.c.pamphlet
* latex macro forms which replace the <<>> syntax with
\begin{chunk}{chunkname}, not yet public.

I agree that it is reasonable to try to define what a pamphlet file
format is and that we should create a DOCUMENT command and style files
that handles any valid pamphlet file correctly. Indeed, that was the
original motivation for the document command; to hide these details.

But we haven't even figured out how we plan to handle multiple
embedded \usepackage lines, hyperlinks, indexing, bibtex,
animated gifs, or video clips.

I think that we need to lift our eyes away from "what is" and focus
a bit on the future. It is too early to state that noweb is THE format
that will carry us onward. Drag-and-drop allows dragged files to be
exported in more than one format which may require additional support
within the pamphlet file structure. Pamphlet files need to consider
cases of standalone, embedded (e.g. as book chapters), and
conflated (collections of partial sections into "topical booklets"
that slice "across pamphlets" such as an ALL-MATRIX booklet")
files which the document command might produce.

The 30-year-horizon won't be reached by limiting our imagination
to our current tools. Research projects demand experimentation,
not standardization. The question of what is a valid pamphlet
file requires definition, not standardization. We need to be able
to expand, extend, and warp the definition as we learn, not fix
on a trivial preprocessing tool as "the standard".

Think 30 year horizon.

t
Gabriel Dos Reis
2006-10-22 00:50:27 UTC
Permalink
root <***@axiom-developer.org> writes:

[...]

| The question of what is a valid pamphlet
| file requires definition, not standardization.

In this specific case, from my perspective, that is the same thing.
People need to communicate and they need to agree on a protocol.
The protocol can and should evolve, but there needs to be agreement on
what is what. Otherwise, we'll be lost in a 30-light-year deep black hole.

-- Gaby
root
2006-10-22 01:11:13 UTC
Permalink
Post by Gabriel Dos Reis
| The question of what is a valid pamphlet
| file requires definition, not standardization.
In this specific case, from my perspective, that is the same thing.
People need to communicate and they need to agree on a protocol.
The protocol can and should evolve, but there needs to be agreement on
what is what. Otherwise, we'll be lost in a 30-light-year deep black hole.
So if straight latex is the base case of a valid literate pamphlet file
(since it contains 0 lines of code) then we have a working definition.

There are many ways to implement that definition, maybe use the latex
command directly or fix noweb to stop mangling latex. noweb applied
to a pamphlet file which does not use noweb should be the identity
function.

noweb has a bug, not a feature.

or make the document command "just work" and do the right thing,
calling noweb only when required.




look to the future. what happens when someone drag-and-drops a
latex file onto axiom that describes a piece of research about
axiom but contains no code? do we mangle it? or do we "just work"?
this is a very likely case for the near future as most research
papers of interest won't contain code yet.




noweb was chosen by me as a simple, language neutral tool that would
achieve the FIRST step of literate programming. there was no intention
of claiming that it was in any way vital or final.

the essense of literate programming can be done without noweb.

restricting our definition of literate programming to require that we
work on the apt-get version of a non-essential tool which fails to
work properly in the base case seems to me to limit ourselves to a
trivial, broken case of literate programming.

tools limit what you can think.
think, then shape the tools.

t
Ralf Hemmecke
2006-10-22 13:15:08 UTC
Permalink
Post by root
So if straight latex is the base case of a valid literate pamphlet file
(since it contains 0 lines of code) then we have a working definition.
Cool, I have never read this somewhere before. ;-)
Post by root
There are many ways to implement that definition, maybe use the latex
command directly or fix noweb to stop mangling latex. noweb applied
to a pamphlet file which does not use noweb should be the identity
function.
noweb has a bug, not a feature.
Wrong. For your definition, noweb is simply the wrong tool. Norman
defined his format and where does he say that running noweb over a latex
file should be the identity?

Anyway, I agree with Bill. We should take the noweb format and
concentrate on other things until somebody comes and invests lots of
time to

1) define a good pamphlet format and
2) provide convenient tools to handle this format.

Maybe noweb is not perfect. But it answers 1) and 2) pretty well. And
seemingly most Axiom developers are happy with it.

If somebody comes forth and defines a new format (clearly that would be
done in a .pamphlet file), then most developers would first have to
agree that this is a better format. Then the format translation should
be done *automatically*.
Post by root
or make the document command "just work" and do the right thing,
calling noweb only when required.
Tim, I think, you are contradicting yourself. You want us to explain
everything and here the "just works" seems to be stronger than the
literate document that explains why sometimes one must call noweb and
sometimes not. BTW, the documentation of "/src/scripts/document"
explains nothing and thus it does "not just work". It *always* applies
noweave. ;-)

Ralf
Gabriel Dos Reis
2006-10-23 01:08:35 UTC
Permalink
root <***@axiom-developer.org> writes:

[...]

| the essense of literate programming can be done without noweb.

I'm glad to hear that; because it was not obvious to me people agreed
when I suggested similar ideas in different terms some months ago.

-- Gaby
Jay Belanger
2006-10-22 17:07:45 UTC
Permalink
root <***@axiom-developer.org> writes:
...
Post by root
Although noweb implements three of four features I don't see it as a
requirement. Indeed there has been discussion of other formats such as
...
Post by root
* latex macro forms which replace the <<>> syntax with
\begin{chunk}{chunkname}, not yet public.
Can one infer that there is (the beginnings of) one which is private?

Jay
root
2006-10-22 17:31:36 UTC
Permalink
Post by Jay Belanger
Post by root
Although noweb implements three of four features I don't see it as a
requirement. Indeed there has been discussion of other formats such as
...
Post by root
* latex macro forms which replace the <<>> syntax with
\begin{chunk}{chunkname}, not yet public.
Can one infer that there is (the beginnings of) one which is private?
yes. i'm learning a lot about the tex language so it's taking time.
latex does similiar processing to handle bibligraphic and index files.

if latex can do code chunks directly, format them properly in
a latex document and dump the code to a file there is no need
for any other tool.

t
Ralf Hemmecke
2006-10-22 21:46:30 UTC
Permalink
Post by root
Post by Jay Belanger
Post by root
Although noweb implements three of four features I don't see it as a
requirement. Indeed there has been discussion of other formats such as
...
Post by root
* latex macro forms which replace the <<>> syntax with
\begin{chunk}{chunkname}, not yet public.
Can one infer that there is (the beginnings of) one which is private?
yes. i'm learning a lot about the tex language so it's taking time.
latex does similiar processing to handle bibligraphic and index files.
if latex can do code chunks directly, format them properly in
a latex document and dump the code to a file there is no need
for any other tool.
I am not sure whether I should support that, but surely formatting of
code chunks is not a problem. Already the verbatim or fancyvrb packages
can do that. Maybe the listings package is well suited for this task.

When it comes to code generation, that is probably getting a bit tricky.
Suppose you want to have

\begin{chunk}{chunkname}
b: Boolean := true;
for i in mylist repeat {
\usechunk{anotherchunk}
b := b/\odd? i;
}
\end{chunk}

If you produce TeX code that can master that correctly, I'll call you a
TeX-guru. (Note that I mean /\ and not \odd.) That the braces after
\usechunk are treated differently than the braces of 'repeat'. But that
is probably an easier exercise.

TeX can write files, but note that before it has to bring the code
chunks into the right order. That basically means that TeX must keep the
whole file in memory if the <<*>> definition comes at the end. One
problem here is that not every TeX-installation has set the pool sizes
high enough. So I expect that the TeX approach very much depends on the
TeX installation and cannot "just work".

And last, I don't think that the TeX approach will be faster than noweb.

Just my two cents on that approach.

Ralf
Jay Belanger
2006-10-23 20:44:56 UTC
Permalink
root <***@axiom-developer.org> writes:
...
Post by root
yes. i'm learning a lot about the tex language so it's taking time.
latex does similiar processing to handle bibligraphic and index files.
For LaTeX, isn't an external program usually called to format the
index files?
Post by root
if latex can do code chunks directly, format them properly in
a latex document and dump the code to a file there is no need
for any other tool.
That sounds like a pretty neat idea. But wouldn't using TeX to do the
tangling be inefficient for large programs? (This isn't a rhetorical
question; I have no idea.)

Jay
root
2006-10-23 21:18:25 UTC
Permalink
Post by Jay Belanger
That sounds like a pretty neat idea. But wouldn't using TeX to do the
tangling be inefficient for large programs? (This isn't a rhetorical
question; I have no idea.)
i have no idea but i do have practical experience.
in work i have a project that is a literate program.

the pamphlet file has 127862 lines.
the generated lisp file has 60184 lines.
the dvi file has 3240 pages.

remaking the whole pile takes (on a 3.2GHz machine)
about 25 seconds for noweb
about 3 seconds for latex -> dvi
about 3 seconds for clisp executable image
about 3 seconds for regression tests

noweb is a bottleneck for my work project
and all it has to do is mmap the file,
create a hash table of the chunks,
and recursively walk the <<*>> tag.

so i'm not worried about the time latex might take.

t

Bill Page
2006-10-21 20:43:39 UTC
Permalink
Ralf,

Thanks for taking this issue to a proper conclusion. :-)
Post by Ralf Hemmecke
...
OK let's do the same thing with the patch.
$AXIOM/bin/lib/noweave -delay Rosetta.pamphlet > Rosetta1.tex
That leads to the line
\Axiom & set [1, 2] & [1, 2] & matrix({\tt{}1,\ 2],[3,\ 4})
\\
and you certainly agree that the original definition of \tt does not
produce [[ in the output.
latex Rosetta.tex
succeeds but the output is simply wrong. The patch is a misfeature!!!
Yes, now I do agree. I think the patched noweb should be eliminated.
Post by Ralf Hemmecke
Post by Bill Page
Everything with extension .pamphlet should be processed by noweave
before calling latex.
I completely agree. Doing otherwise just causes confusion.
So how do we escape these [[...]] things so that noweb doesn't touch
them? Is it the same as escaping << ... >> ? I don't think Norman
Ramsey's filter will handle this case since it only affects how
undefined chunk names are treated.

Regards,
Bill Page.
Ralf Hemmecke
2006-10-21 21:47:08 UTC
Permalink
Post by Bill Page
So how do we escape these [[...]] things so that noweb doesn't touch
them? Is it the same as escaping << ... >> ?
sed -e 's/\[\[/@[[/g' Rosetta.pamphlet > R.pamphlet

Then R.pamphlet *must* be run through 'noweave -delay' in order to get
the right latex file. Actually it also is a latex file, but 'latex
Post by Bill Page
I don't think Norman
Ramsey's filter will handle this case since it only affects how
undefined chunk names are treated.
As I said. It's totally unrelated.

BTW, I do not think that [[...]] should be used. As far as I understand
it is intended in noweb to be just like \verb only that one could have
some fancy stuff (see below).

But, actually, I believe that better tags should be given. For example,
I use \adtype{Integer} and \adname{foo} for types and functions in
ALLPROSE and have some scripts dealing with the proper hyper-linking.
Note that [[...]] says nothing about the semantics of the stuff inside.

Ralf




Example....

Try the following commands on the file below.

noweave -delay -index aaa.nw > aaa.tex
latex aaa.tex
latex aaa.tex
xdvi aaa.dvi

I have no idea why I get

! Bad space factor (0).
<recently read> \@savsf

l.6 ...\nwbegincode{1}\sublabel{NW21NIyD-1p0Y9w-1}

\nwmargintag{{\nwtagstyle{...

Just ignore it by pressing 'q'.


%%% aaa.nw %%%
\documentclass{article}
\usepackage{noweb}
\usepackage{hyperref}
\begin{document}
blah blah
<<*>>=
define IntegerCategory: Category == with {
foo: % -> %;
bar: % -> %;
}
@ %def IntegerCategory foo bar

I now can use [[foo]] and [[bar]] and even [[IntegerCategory]] and
[[Integer]] which is quite convenient. The
\verb'%def' stuff even gets linked (see next page).

\newpage

But that is not proper tagging.

<<*>>=
Integer: IntegerCategory == add {
foo(x: %) == x;
...
}
@ %def Integer

The problem is that we consider [[Integer]] and [[IntegerCategory]] as
types while [[foo]] and [[bar]] are just functions.

\end{document}
Gabriel Dos Reis
2006-10-22 00:31:23 UTC
Permalink
"Bill Page" <***@synthesis.anikast.ca> writes:

| Ralf,
|
| Thanks for taking this issue to a proper conclusion. :-)
|
| On October 21, 2006 4:24 PM you wrote:
| > ...
| > OK let's do the same thing with the patch.
| >
| > $AXIOM/bin/lib/noweave -delay Rosetta.pamphlet > Rosetta1.tex
| >
| > That leads to the line
| >
| > \Axiom & set [1, 2] & [1, 2] & matrix({\tt{}1,\ 2],[3,\ 4})
| > \\
| >
| > and you certainly agree that the original definition of \tt does not
| > produce [[ in the output.
| >
| > latex Rosetta.tex
| >
| > succeeds but the output is simply wrong. The patch is a misfeature!!!
| >
|
| Yes, now I do agree. I think the patched noweb should be eliminated.

The build-improvements branch does not apply Tim's patch.

-- Gaby
Bill Page
2006-10-21 23:27:46 UTC
Permalink
Post by Ralf Hemmecke
Post by root
Post by Bill Page
You are right and the Makefile is wrong. Everything with extension
.pamphlet should be processed by noweave before calling latex.
nope. that's not the case.
????
Post by Ralf Hemmecke
Post by root
noweb is just a preprocessing tool, not a requirement for
literate programming.
Well, yah. That's true.
Post by Ralf Hemmecke
Post by root
there is no requirement (or intended requirement) that a .pamphlet
file implies running noweb. some of the files in any documented
axiom system will not contain code and will not need noweb. they
are, however, perfectly vaild .pamphlet files.
Tim, that's just stupid. A .pamphlet file has to have a well defined
format or it's useless. Why not just call then .apple or .macintosh
files, then? ;)

I didn't mean *necessarily* noweb, but right now that is the best
definition we have of what a pamphlet file is.
Post by Ralf Hemmecke
Please define the format of a .pamphlet file so that people can
write tools to support that format. (If the format is moving,
that is not a problem, but we should always have an up-to-date
description of that format.)
I agree. Or rather I would very much prefer to *define* .pamphlet
format as *exactly* the same as noweb format. That way we can
forget about it and do something more important then vaguely
trying to re-invent the wheel ... or something sort of round
that rolls and makes transportation easy ... whatever that
might be. ;)
Post by Ralf Hemmecke
Up to now I thought the format is like the one in noweb, but
already with your patch to noweb, you modified that. Without a
documented format we end up like SPAD is now. A language without
formal specification.
Good luck.
Good grief!

Regards,
Bill Page.
Gabriel Dos Reis
2006-10-22 00:22:07 UTC
Permalink
Waldek Hebisch <***@math.uni.wroc.pl> writes:

| Gaby, you probably know this, but just in case: running
| Rosetta.pamphlet trough noweb messes it. This is using
| Debian noweb. Package identification:
|
| ii nowebm 2.10c-3.3 A WEB-like literate-programming tool

Thanks; I was completely unaware of this issue.

-- Gaby
C Y
2006-10-22 02:08:28 UTC
Permalink
Post by root
But we haven't even figured out how we plan to handle multiple
embedded \usepackage lines, hyperlinks, indexing, bibtex,
animated gifs, or video clips.
Is this a topic of current interest? I confess it fell off of my radar
screen somewhat, but I do have some ideas I would like to explore.

hyperlinks I think are fairly well addressed by hyperref - maybe a bit
of extension work is needed, but it solves the basic problem at least.

indexing is a bit more of a challenge. To the best of my knowledge
most indicies are created by humans, since no automatic tool can do a
"context" sensitive indexing. Since that might prove a bit impractical
for us, we probably need to look into other alternatives. Some useful
tools might include:
http://www.tug.org/tex-archive/help/Catalogue/entries/addindex.html
http://www.tug.org/tex-archive/help/Catalogue/entries/splitindex.html

I would suggest the following, at least: a "human generated" list of
key words related to specific topics of interest as the general index,
a "Categories and Domains" index of all such definitions, and perhaps
an Authors index as well.

BibTeX I already did some work with using the packages prepared by the
ToC guys http://theoryofcomputing.org/ (the eprint package), which is
available here:
http://portal.axiom-developer.org/Members/starseeker/axiombibliographysystem.tar.gz/file_view
It's not very pretty and would need to be strengthened/cleaned up
considerably before being ready for prime time, but I think it did
address some of the original goals for the bibliography. I don't know
if anyone else agrees with me on the desirability of using MSC2000 and
PACS to make a large scale summary of all works referenced in Axiom by
topic, but I did produce a proof-of-concept document with a limited set
of examples:
http://portal.axiom-developer.org/Members/starseeker/axiombibliography.pdf/file_view

No clue how to deal with animated gifs.

Movies and 3D however have an interesting alternative. There is a
package called movie15 for LaTeX which can embed some interesting
content in pdf files. I don't know how many pdf viewers support these
abilities yet, but the U3D file format mentioned here is an interesting
idea for a graphics output target in Axiom and I have been able to view
such a file successfully in Acrobat Reader on Linux in the past. The
standard document for this format is here:
http://www.ecma-international.org/publications/standards/Ecma-363.htm
and the movie15 page is here:
http://www.ctan.org/tex-archive/help/Catalogue/entries/movie15.html

An obvious problem is how to get this and pstricks to play nice, but it
may be worth looking into. Perhaps dvipdf could be extended to handle
it the same way it can handle hyperref.
Post by root
I think that we need to lift our eyes away from "what is" and focus
a bit on the future. It is too early to state that noweb is THE
format that will carry us onward. Drag-and-drop allows dragged files
to be exported in more than one format which may require additional
support within the pamphlet file structure. Pamphlet files need to
consider cases of standalone, embedded (e.g. as book chapters), and
conflated (collections of partial sections into "topical booklets"
that slice "across pamphlets" such as an ALL-MATRIX booklet")
files which the document command might produce.
For this I think the first thing to explore is this LaTeX package:
http://www.tug.org/tex-archive/macros/latex/contrib/combine/

I don't know how well it does in practice but conference proceedings
would seem to face many problems similar to the ones we do.
Post by root
The 30-year-horizon won't be reached by limiting our imagination
to our current tools. Research projects demand experimentation,
not standardization. The question of what is a valid pamphlet
file requires definition, not standardization. We need to be able
to expand, extend, and warp the definition as we learn, not fix
on a trivial preprocessing tool as "the standard".
However, by the same token we hope to have a HUGE body of work written
in this file format - for those files to remain maintainable and
relevant over time, we need to have most of the fundamentals down
before large scale (re)writing takes place. At the very least, changes
should be such that they can be applied automatically in most cases to
update files without human intervention. That means many of these
issues should be resolved sooner rather than later.

Cheers,
CY

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Waldek Hebisch
2006-10-22 13:38:51 UTC
Permalink
Post by Gabriel Dos Reis
| Gaby, you probably know this, but just in case: running
| Rosetta.pamphlet trough noweb messes it. This is using
|
| ii nowebm 2.10c-3.3 A WEB-like literate-programming tool
Thanks; I was completely unaware of this issue.
There are more problems: apparently noweb can not correctly handle
files which have underscores in their names. For example latex run
on src/hyper/parse_aux.tex gives the following errors (I removed
other messages which are caused by the two errors):

! Missing $ inserted.
<inserted text>
$
l.13 ...aux.pamphlet}\nwbegincode{1}\moddef{parse_
aux.h}\endmoddef\nwstartde...

.....
! Missing $ inserted.
<inserted text>
$
l.742 \LA{}parse_
aux.c\RA{}

similar for other files containing underscores in their names. It
looks that silver somewhat handles this problem (Tim's patches ???).
--
Waldek Hebisch
***@math.uni.wroc.pl
Gabriel Dos Reis
2006-10-23 01:13:15 UTC
Permalink
Waldek Hebisch <***@math.uni.wroc.pl> writes:

| There are more problems: apparently noweb can not correctly handle
| files which have underscores in their names. For example latex run
| on src/hyper/parse_aux.tex gives the following errors (I removed
| other messages which are caused by the two errors):

I know of that one. And, actually, my recent tweak to dvi-ax is
supposed to be followed up by a renaming of parse_xxx to parse-xxx.
I did not have time to commit it.

[...]

| similar for other files containing underscores in their names. It
| looks that silver somewhat handles this problem (Tim's patches ???).

silver escapses the underscore. build-improvements explicitly removed
the underscore a while ago.

-- Gaby
Bill Page
2006-10-22 14:03:59 UTC
Permalink
Post by Waldek Hebisch
Post by Gabriel Dos Reis
| ii nowebm 2.10c-3.3 A WEB-like
| literate-programming tool
...
There are more problems: apparently noweb can not correctly
handle files which have underscores in their names.
?

I do not get any errors in files containing _ in either Tim's
version distributed with Gold nor with a more recent unmodified
version of noweb (noweb-2.11b).
Post by Waldek Hebisch
For example latex run on src/hyper/parse_aux.tex gives the
following errors (I removedother messages which are caused by
! Missing $ inserted.
<inserted text>
$
l.13 ...aux.pamphlet}\nwbegincode{1}\moddef{parse_
aux.h}\endmoddef\nwstartde...
.....
! Missing $ inserted.
<inserted text>
$
l.742 \LA{}parse_
aux.c\RA{}
similar for other files containing underscores in their names.
It looks that silver somewhat handles this problem (Tim's
patches ???).
Here is my console log:

[***@axiom-developer hyper]$ noweave -delay parse_aux.pamphlet >
parse_aux.tex
[***@axiom-developer hyper]$ latex parse_aux.tex
This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4)
entering extended mode
(./parse_aux.tex
LaTeX2e <2003/12/01>
Babel <v3.8d> and hyphenation patterns for american, french, german,
ngerman, b
ahasa, basque, bulgarian, catalan, croatian, czech, danish, dutch,
esperanto, e
stonian, finnish, greek, icelandic, irish, italian, latin, magyar, norsk,
polis
h, portuges, romanian, russian, serbian, slovak, slovene, spanish, swedish,
tur
kish, ukrainian, nohyphenation, loaded.
(/usr/local/teTeX/share/texmf-dist/tex/latex/base/article.cls
Document Class: article 2004/02/16 v1.4f Standard LaTeX document class
(/usr/local/teTeX/share/texmf-dist/tex/latex/base/size10.clo))
(/usr/local/teTeX/share/texmf-dist/tex/latex/base/axiom.sty)
(./parse_aux.aux)
[1] (./parse_aux.toc) [2] [3] [4] [5] [6]
Overfull \hbox (89.24922pt too wide) in paragraph at lines 161--161
[]\OT1/cmtt/m/n/10 read_ht_file(HashTable *page_hash, HashTable
*macro_hash, H
ashTable *patch_hash, FILE *db_fp, char *db_file)[]
[7]
Overfull \hbox (20.99982pt too wide) in paragraph at lines 236--236
[] \OT1/cmtt/m/n/10 fprintf(stderr, "(HyperDoc)
read_ht_file: H
yperDoc file %s has been updated\n",[]

Overfull \hbox (5.24995pt too wide) in paragraph at lines 239--239
[] \OT1/cmtt/m/n/10 fprintf(stderr, "(HyperDoc) Issue htadd
%s
to update database\n", fullname);[]
[8]
Overfull \hbox (31.49973pt too wide) in paragraph at lines 261--261
[] \OT1/cmtt/m/n/10 fprintf(stderr, "(HyperDoc) Page
na
me %s occurred twice\n", page->name);[]

Overfull \hbox (20.99982pt too wide) in paragraph at lines 286--286
[] \OT1/cmtt/m/n/10 fprintf(stderr, "(HyperDoc)
The
Version in %s is being ignored \n",[]
[9]
Overfull \hbox (41.99963pt too wide) in paragraph at lines 309--309
[] \OT1/cmtt/m/n/10 fprintf(stderr, "(HyperDoc)
Patch n
ame %s occurred twice\n", patch->name);[]

Overfull \hbox (41.99963pt too wide) in paragraph at lines 318--318
[] \OT1/cmtt/m/n/10 fprintf(stderr, "(HyperDoc)
read_ht_db:
Unknown type %s in ht.db\n", token.id);[]
[10]
Overfull \hbox (15.74986pt too wide) in paragraph at lines 371--371
[] \OT1/cmtt/m/n/10 link->win = XCreateWindow(gXDisplay,
gWindow->f
DisplayedWindow, 0, 0, 100, 100, 0,[]
[11] [12] [13] [14] [15]
Overfull \hbox (57.7495pt too wide) in paragraph at lines 586--586
[] \OT1/cmtt/m/n/10 fprintf(stderr, "(HyperDoc) Error trying to read
%s
, unexpected end-of-file.\n",db_file_name);[]
[16] [17] [18] [19] [20] (./parse_aux.aux) )
(see the transcript file for additional information)
Output written on parse_aux.dvi (20 pages, 23836 bytes).
Transcript written on parse_aux.log.

-----------

Same result with

[***@axiom-developer hyper]$ /usr/local/axiom/mnt/linux/bin/lib/noweave \
-delay parse_aux.pamphlet > parse_aux2.tex

Regards,
Bill Page.
Gabriel Dos Reis
2006-10-23 01:14:15 UTC
Permalink
"Bill Page" <***@synthesis.anikast.ca> writes:

| On October 22, 2006 9:39 AM Waldek Hebisch wrote:
| > > Waldek Hebisch <***@math.uni.wroc.pl> writes:
| > >
| > > | ii nowebm 2.10c-3.3 A WEB-like
| > > | literate-programming tool
| > >
| > ...
| > There are more problems: apparently noweb can not correctly
| > handle files which have underscores in their names.
|
| ?
|
| I do not get any errors in files containing _ in either Tim's
| version distributed with Gold nor with a more recent unmodified
| version of noweb (noweb-2.11b).

That is because both say

<<parse\_aux.h>>=

and not

<<parse_aux.h>>=

as build-improvements does.

-- gaby
Bill Page
2006-10-22 14:39:00 UTC
Permalink
Post by Waldek Hebisch
For example latex run on src/hyper/parse_aux.tex gives the
following errors (I removedother messages which are caused by
! Missing $ inserted.
<inserted text>
$
l.13 ...aux.pamphlet}\nwbegincode{1}\moddef{parse_
aux.h}\endmoddef\nwstartde...
...
I do not get any errors in files containing _ in either Tim's
version distributed with Gold nor with a more recent unmodified
version of noweb (noweb-2.11b).
In contrast to the output which Waldek shows, in both versions
of noweb run on axiom-developer.org the _ is removed from the
generated noweb commands.\:

\section{parse\_aux.h}
\nwfilename{parse_aux.pamphlet}\nwbegincode{1}\moddef{parseaux.h}\endmoddef
\LA{}license\RA{}

-----

Could this be a LOCALE/LANG issue? Or is this behaviour specific
to the version of noweb distributed by Debian?

Regards,
Bill Page.
Gabriel Dos Reis
2006-10-23 01:16:31 UTC
Permalink
"Bill Page" <***@synthesis.anikast.ca> writes:

| Could this be a LOCALE/LANG issue?

No. It is an issue on build-improvements while back when I made a
uniform rule to extracting the C files from the pamphlets.

| Or is this behaviour specific
| to the version of noweb distributed by Debian?

No, it is a uniform behaviour of all recent noweb versions.
Noweb is unable to unsecaped "_" in a chunk-name.

-- Gaby
Waldek Hebisch
2006-10-22 15:01:02 UTC
Permalink
Post by Bill Page
Post by Waldek Hebisch
For example latex run on src/hyper/parse_aux.tex gives the
following errors (I removedother messages which are caused by
! Missing $ inserted.
<inserted text>
$
l.13 ...aux.pamphlet}\nwbegincode{1}\moddef{parse_
aux.h}\endmoddef\nwstartde...
...
I do not get any errors in files containing _ in either Tim's
version distributed with Gold nor with a more recent unmodified
version of noweb (noweb-2.11b).
In contrast to the output which Waldek shows, in both versions
of noweb run on axiom-developer.org the _ is removed from the
\section{parse\_aux.h}
\nwfilename{parse_aux.pamphlet}\nwbegincode{1}\moddef{parseaux.h}\endmoddef
\LA{}license\RA{}
-----
Could this be a LOCALE/LANG issue? Or is this behaviour specific
to the version of noweb distributed by Debian?
On 32-bit Debian Sarge with "nowebm 2.10c-3.2" I get the output you
show. But on 64-bit Debian testing/unstable using "nowebm 2.10c-3.3"
I get:

\section{parse\_aux.h}
\nwfilename{parse_aux.pamphlet}\nwbegincode{1}\moddef{parse_aux.h}\endmoddef\nwstartdeflinemarkup\nwenddeflinemarkup
\LA{}license\RA{}


Note that strictly speaking both forms are problematic: the first mangles
names, the second passes naked underscore to latex (maybe the intention
was to update '\moddef' so that it can handle underscores).
--
Waldek Hebisch
***@math.uni.wroc.pl
Bill Page
2006-10-22 15:18:59 UTC
Permalink
Post by Ralf Hemmecke
Post by root
So if straight latex is the base case of a valid literate
pamphlet file (since it contains 0 lines of code) then we
have a working definition.
Cool, I have never read this somewhere before. ;-)
Yes, one might have thought that Knuth would have started with
a definition like this, wouldn't you? But he didn't.
Post by Ralf Hemmecke
Post by root
There are many ways to implement that definition, maybe use
the latex command directly or fix noweb to stop mangling latex.
noweb applied to a pamphlet file which does not use noweb
should be the identity function.
noweb has a bug, not a feature.
Wrong. For your definition, noweb is simply the wrong tool.
Norman defined his format and where does he say that running
noweb over a latex file should be the identity?
Ralf is correct. Tim is wrong. We can not say that noweb has a
bug. However I do agree that it is a pity that noweb wasn't
designed the way Tim suggests. This would have been nice but
perhaps it conflicts with other design goals that Norman had in
mind when he wrote noweb.
Post by Ralf Hemmecke
Anyway, I agree with Bill. We should take the noweb format and
concentrate on other things until somebody comes and invests
lots of time to
1) define a good pamphlet format and
2) provide convenient tools to handle this format.
Maybe noweb is not perfect. But it answers 1) and 2) pretty well.
And seemingly most Axiom developers are happy with it.
And I agree with Ralf... :-) Axiom is an open source project. As
such it should "play nicely" with the rest of the open source
development community and collaborate with the authors of the
tools on which we depend. We can not afford the time and resouces
to solve all of these problems for ourselves or else we will never
get around to actually implementing new computer algebra! We *must*
take advantage of the open source development environment and
depend on other open source developers to provide the tools.
Post by Ralf Hemmecke
If somebody comes forth and defines a new format (clearly that
would be done in a .pamphlet file), then most developers would
first have to agree that this is a better format. Then the format
translation should be done *automatically*.
As I have said before in this frequently recurring discussion,
*if* we were to choose a new literate programming tool, then I
would much prefer a more radical solution than simply changing
the underlying file format. There are much better tools available
for literate programming, such as Leo, which provide a complete
and integrated literate programming environment.
Post by Ralf Hemmecke
...
Regards,
Bill Page.
Bill Page
2006-10-22 15:23:30 UTC
Permalink
I think we should ask Norman Ramsey about this "name mangling".

Does it have a nice solution that would allow us to use
the version(s) of noweb distributed by Debian?
-----Original Message-----
Sent: October 22, 2006 11:01 AM
To: Bill Page
Subject: Re: [Axiom-developer] build-improvements and latex
Post by Bill Page
Post by Waldek Hebisch
For example latex run on src/hyper/parse_aux.tex gives the
following errors (I removedother messages which are caused by
! Missing $ inserted.
<inserted text>
$
l.13 ...aux.pamphlet}\nwbegincode{1}\moddef{parse_
aux.h}\endmoddef\nwstartde...
...
I do not get any errors in files containing _ in either Tim's
version distributed with Gold nor with a more recent unmodified
version of noweb (noweb-2.11b).
In contrast to the output which Waldek shows, in both versions
of noweb run on axiom-developer.org the _ is removed from the
\section{parse\_aux.h}
\nwfilename{parse_aux.pamphlet}\nwbegincode{1}\moddef{parseaux
.h}\endmoddef
Post by Bill Page
\LA{}license\RA{}
-----
Could this be a LOCALE/LANG issue? Or is this behaviour specific
to the version of noweb distributed by Debian?
On 32-bit Debian Sarge with "nowebm 2.10c-3.2" I get the output you
show. But on 64-bit Debian testing/unstable using "nowebm 2.10c-3.3"
\section{parse\_aux.h}
\nwfilename{parse_aux.pamphlet}\nwbegincode{1}\moddef{parse_au
x.h}\endmoddef\nwstartdeflinemarkup\nwenddeflinemarkup
\LA{}license\RA{}
Note that strictly speaking both forms are problematic: the
first mangles
names, the second passes naked underscore to latex (maybe the
intention
was to update '\moddef' so that it can handle underscores).
--
Waldek Hebisch
Norman Ramsey
2006-10-22 22:38:15 UTC
Permalink
Post by Bill Page
I think we should ask Norman Ramsey about this "name mangling".
Does it have a nice solution that would allow us to use
the version(s) of noweb distributed by Debian?
Sounds like you guys have a local hack.

The underscore issue is a FAQ.


Norman
Post by Bill Page
-----Original Message-----
Sent: October 22, 2006 11:01 AM
To: Bill Page
Subject: Re: [Axiom-developer] build-improvements and latex
Post by Bill Page
Post by Waldek Hebisch
For example latex run on src/hyper/parse_aux.tex gives the
following errors (I removedother messages which are caused by
! Missing $ inserted.
<inserted text>
$
l.13 ...aux.pamphlet}\nwbegincode{1}\moddef{parse_
aux.h}\endmoddef\nwstartde...
...
I do not get any errors in files containing _ in either Tim's
version distributed with Gold nor with a more recent unmodified
version of noweb (noweb-2.11b).
In contrast to the output which Waldek shows, in both versions
of noweb run on axiom-developer.org the _ is removed from the
\section{parse\_aux.h}
\nwfilename{parse_aux.pamphlet}\nwbegincode{1}\moddef{parseaux
.h}\endmoddef
Post by Bill Page
\LA{}license\RA{}
-----
Could this be a LOCALE/LANG issue? Or is this behaviour specific
to the version of noweb distributed by Debian?
On 32-bit Debian Sarge with "nowebm 2.10c-3.2" I get the output you
show. But on 64-bit Debian testing/unstable using "nowebm 2.10c-3.3"
\section{parse\_aux.h}
\nwfilename{parse_aux.pamphlet}\nwbegincode{1}\moddef{parse_au
x.h}\endmoddef\nwstartdeflinemarkup\nwenddeflinemarkup
\LA{}license\RA{}
Note that strictly speaking both forms are problematic: the
first mangles
names, the second passes naked underscore to latex (maybe the
intention
was to update '\moddef' so that it can handle underscores).
--
Waldek Hebisch
Waldek Hebisch
2006-10-22 18:09:49 UTC
Permalink
Post by Bill Page
I think we should ask Norman Ramsey about this "name mangling".
Does it have a nice solution that would allow us to use
the version(s) of noweb distributed by Debian?
-----Original Message-----
Subject: Re: [Axiom-developer] build-improvements and latex
Post by Waldek Hebisch
For example latex run on src/hyper/parse_aux.tex gives the
following errors (I removedother messages which are caused by
! Missing $ inserted.
<inserted text>
$
l.13 ...aux.pamphlet}\nwbegincode{1}\moddef{parse_
aux.h}\endmoddef\nwstartde...
...
On 32-bit Debian Sarge with "nowebm 2.10c-3.2" I get the output you
show. But on 64-bit Debian testing/unstable using "nowebm 2.10c-3.3"
\section{parse\_aux.h}
\nwfilename{parse_aux.pamphlet}\nwbegincode{1}\moddef{parse_au
x.h}\endmoddef\nwstartdeflinemarkup\nwenddeflinemarkup
\LA{}license\RA{}
Correction: both verion of noweb give the same result. The real reason
is that parse_aux.pamphlet in build-improvements is modified:

--- ../../silver/axiom/src/hyper/parse_aux.pamphlet 2006-10-20 23:54:15.000000000 +0200
+++ parse_aux.pamphlet 2006-10-22 19:42:42.000000000 +0200
@@ -10,7 +10,7 @@
\tableofcontents
\eject
\section{parse\_aux.h}
-<<parseaux.h>>=
+<<parse_aux.h>>=
<<license>>
#ifndef _PARSE_AUX_H_
#define _PARSE_AUX_H_ 1

and three similar chunks.
--
Waldek Hebisch
***@math.uni.wroc.pl
Bill Page
2006-10-22 18:58:44 UTC
Permalink
Post by Waldek Hebisch
...
Correction: both verion of noweb give the same result. The real
--- ../../silver/axiom/src/hyper/parse_aux.pamphlet
2006-10-20 23:54:15.000000000 +0200
+++ parse_aux.pamphlet 2006-10-22 19:42:42.000000000 +0200
@@ -10,7 +10,7 @@
\tableofcontents
\eject
\section{parse\_aux.h}
-<<parseaux.h>>=
+<<parse_aux.h>>=
<<license>>
#ifndef _PARSE_AUX_H_
#define _PARSE_AUX_H_ 1
and three similar chunks.
The following changeset annotation

http://page.axiom-developer.org/darcsweb.cgi?r=axiom.build-improvements;a=an
notate_shade;f=/src/hyper/parse_aux.pamphlet

Shows that Gaby made the change to include the underscore _
on 2006-09-18.

Regards,
Bill Page.
Gabriel Dos Reis
2006-10-23 01:18:20 UTC
Permalink
"Bill Page" <***@synthesis.anikast.ca> writes:

[...]

| Shows that Gaby made the change to include the underscore _
| on 2006-09-18.

Yes, that is correct.

-- Gaby
Bill Page
2006-10-22 23:30:54 UTC
Permalink
Thanks, Norman. I think we figured it out. There is really no
mangling - just an "innocent" change with unexpected behaviour.
Your FAQ is very useful:

http://www.eecs.harvard.edu/nr/noweb/FAQ.html

Q. How do I get an output file (root chunk) with an underscore
in its name? LaTeX complains about the underscore.

A. All chunk names are set in ordinary TeX mode. This means
that underscore is a subscript character. Rather than use special
hacks, I just avoid underscores in file names, since I prefer to
use hyphens anyway. However, you can write a simple sed script to
filter the underscores for use by noweave:

noweave -filter "sed '/^@use /s/_/\\_/g;/^@defn /s/_/\\_/g'"

and the filter will change _ to \_ before TeX sees it.

-----------

I've got to remember to check the FAQ more often.

Regards,
Bill Page.
-----Original Message-----
Sent: October 22, 2006 6:38 PM
To: Bill Page
Subject: Re: [Axiom-developer] build-improvements and latex
Post by Bill Page
I think we should ask Norman Ramsey about this "name mangling".
Does it have a nice solution that would allow us to use
the version(s) of noweb distributed by Debian?
Sounds like you guys have a local hack.
The underscore issue is a FAQ.
Norman
Waldek Hebisch
2006-10-23 01:19:56 UTC
Permalink
Post by Bill Page
Thanks, Norman. I think we figured it out. There is really no
mangling - just an "innocent" change with unexpected behaviour.
http://www.eecs.harvard.edu/nr/noweb/FAQ.html
Q. How do I get an output file (root chunk) with an underscore
in its name? LaTeX complains about the underscore.
A. All chunk names are set in ordinary TeX mode. This means
that underscore is a subscript character. Rather than use special
hacks, I just avoid underscores in file names, since I prefer to
use hyphens anyway. However, you can write a simple sed script to
and the filter will change _ to \_ before TeX sees it.
The filter from the FAQ does not work. I had to use:

noweave -filter 'sed "/^@use /s/_/\\\\_/g;/^@defn /s/_/\\\\_/g"'

(apstrophes inside filter interacted with quoting inside noweave).
--
Waldek Hebisch
***@math.uni.wroc.pl
Gabriel Dos Reis
2006-10-23 01:33:03 UTC
Permalink
Waldek Hebisch <***@math.uni.wroc.pl> writes:

| The filter from the FAQ does not work. I had to use:
|
| noweave -filter 'sed "/^@use /s/_/\\\\_/g;/^@defn /s/_/\\\\_/g"'
|
| (apstrophes inside filter interacted with quoting inside noweave).

I'm in the process of "trimming out" other changes I have so that the
file renaming patch can go in.

-- Gaby
Norman Ramsey
2006-10-23 03:15:31 UTC
Permalink
Post by Waldek Hebisch
(apstrophes inside filter interacted with quoting inside noweave).
Urk! At some point I must have changed noweave without changing the FAQ.
Thanks for the bug report; I have now changed the FAQ.


Norman
Waldek Hebisch
2006-10-23 01:29:59 UTC
Permalink
Post by Gabriel Dos Reis
| There are more problems: apparently noweb can not correctly handle
| files which have underscores in their names. For example latex run
| on src/hyper/parse_aux.tex gives the following errors (I removed
I know of that one. And, actually, my recent tweak to dvi-ax is
supposed to be followed up by a renaming of parse_xxx to parse-xxx.
I did not have time to commit it.
Inspired by noweb FAQ in am testing the following patch -- tohether
with previous patch it allowed me to run 'make dvi' with only two
problems (Rosetta.pamphlet and more-rules.mk). 'more-rules.mk' is
build only during main build, so for standalone 'make dvi' (without
previous build) I had to add it by hand. Now I am tesing full build.

diff -ru build-improvements.nn2/src/hyper/parse_aux.pamphlet build-improvements.nn/src/hyper/parse_aux.pamphlet
--- build-improvements.nn2/src/hyper/parse_aux.pamphlet 2006-10-23 01:38:47.000000000 +0200
+++ build-improvements.nn/src/hyper/parse_aux.pamphlet 2006-10-23 03:08:28.000000000 +0200
@@ -20,7 +20,7 @@

#endif
@
-\section{parse_aux.c}
+\section{parse\_aux.c}
<<parse_aux.c>>=
#define _PARSE_AUX_C
#include "useproto.h"
diff -ru build-improvements.nn2/src/scripts/document.in build-improvements.nn/src/scripts/document.in
--- build-improvements.nn2/src/scripts/document.in 2006-10-23 01:38:47.000000000 +0200
+++ build-improvements.nn/src/scripts/document.in 2006-10-23 02:50:15.000000000 +0200
@@ -100,7 +100,8 @@

if test x$do_weave = xyes; then
file=`basename $1 .pamphlet`
- $weave -delay $1 > $file.tex
+ $weave -delay -filter 'sed "/^@use /s/_/\\\\_/g;/^@defn /s/_/\\\\_/g"' \
+ $1 > $file.tex
if test x$do_latex != xyes; then
exit 0;
fi
--
Waldek Hebisch
***@math.uni.wroc.pl
Bill Page
2006-10-23 02:20:06 UTC
Permalink
Post by Waldek Hebisch
...
Inspired by noweb FAQ in am testing the following patch --
tohether with previous patch it allowed me to run 'make dvi'
with only two problems (Rosetta.pamphlet and more-rules.mk).
'more-rules.mk' is build only during main build, so for
standalone 'make dvi' (without previous build) I had to add
it by hand. Now I am tesing full build.
Here is a patch to Rosetta.pamphlet to make it a real pamphlet
(noweb) file:

[***@axiom-developer doc]$ diff -au Rosetta.pamphlet_orig Rosetta.pamphlet
--- Rosetta.pamphlet_orig 2006-09-11 22:16:29.000000000 -0500
+++ Rosetta.pamphlet 2006-10-22 21:05:28.000000000 -0500
@@ -1,4 +1,4 @@
-\documentclass{book}
+\documentclass{article}
\normalsize\baselineskip=12pt
\parskip=0pt
\parindent=10pt
@@ -15,8 +15,6 @@
\renewcommand{\textfraction}{.1}
\renewcommand{\floatpagefraction}{.75}
%
-\def\chaptername{}
-%
\catcode`@=11
\def\***@plain{\let\@mkboth\@gobbletwo%
\let\@oddhead\@empty\def\@oddfoot{\sysdetails}
@@ -59,8 +57,6 @@
\newcommand{\Sumit}{{\sf Sumit}}
\newcommand{\Yacas}{{\sf Yacas}}

-\chapter{Rosetta Translations}
-
\section{Introduction}

The following is a collection of synonyms for various operations in
@@ -268,19 +264,19 @@
\begin{tabular}{l|lll}
& \h{Set} & \h{List} & \h{Matrix} \\
\hline
-\Axiom & set [1, 2] & [1, 2] & matrix([[1, 2],[3, 4]])
\\
-\Derive & \{1, 2\} & [1, 2] & [[1,2], [3,4]]
\\
+\Axiom & set [1, 2] & [1, 2] & matrix(@[[1, 2],[3, 4]])
\\
+\Derive & \{1, 2\} & [1, 2] & @[[1,2], [3,4]]
\\
\DoCon & & & \\
-\GAP & Set([1,2]) & [1, 2] & [[1,2], [3,4]]\fnm
\\
+\GAP & Set([1,2]) & [1, 2] & @[[1,2], [3,4]]\fnm
\\
\Gmp & & & \\
\Macsyma & [1, 2] & [1, 2] & matrix([1, 2], [3, 4])
\\
\Magnus & & & \\
\Maxima & [1, 2] & [1, 2] & matrix([1, 2], [3, 4])
\\
-\Maple & \{1, 2\} & [1, 2] & matrix([[1, 2], [3, 4]])
\\
+\Maple & \{1, 2\} & [1, 2] & matrix(@[[1, 2], [3, 4]])
\\
\Mathematica & \{1, 2\} & \{1, 2\} & \{\{1, 2\}, \{3, 4\}\}
\\
\MuPAD & \{1, 2\} & [1, 2] & export(Dom): \q export(linalg):
\\
& & & matrix:= ExpressionField(normal)):
\\
- & & & matrix([[1, 2], [3, 4]])
\\
+ & & & matrix(@[[1, 2], [3, 4]])
\\
\Octave & & & \\
\Pari & & & \\
\Reduce & \{1, 2\} & \{1, 2\} & mat((1, 2), (3, 4))
\\
@@ -301,7 +297,7 @@
\Magnus & & & & \\
\Maxima & x = 0 & l[2] & m[2, 3] & length(l) \\
\Maple & x = 0 & l[2] & m[2, 3] & nops(l) \\
-\Mathematica & x == 0 & l[[2]] & m[[2, 3]] & Length[l] \\
+\Mathematica & x == 0 & l@[[2]] & m@[[2, 3]] & Length[l] \\
\MuPAD & x = 0 & l[2] & m[2, 3] & nops(l) \\
\Octave & & & & \\
\Pari & & & & \\
@@ -345,7 +341,7 @@
\Magnus & & \\
\Maxima & mat\_\,ncols(m) & transpose(matrix(l))
\\
\Maple & linalg[coldim](m) & linalg[transpose](matrix([l]))
\\
-\Mathematica & Dimensions[m][[2]] & Transpose[\{l\}]
\\
+\Mathematica & Dimensions[m]@[[2]] & Transpose[\{l\}]
\\
\MuPAD & linalg::ncols(m) & transpose(matrix([l]))\,\fnm
\\
\Octave & & \\
\Pari & & \\
@@ -463,8 +459,8 @@
\Magnus & \\
\Maxima & for x in [2, 3, 5] while x\^{}2 < 10 do print(x)\$
\\
\Maple & for x in [2, 3, 5] while x\^{}2 < 10 do print(x) od:
\\
-\Mathematica & For[l = \{2, 3, 5\}, l != \{\} \&\& l[[1]]\^{}2 < 10,
\\
- & \q l = Rest[l], Print[l[[1]]] ]
\\
+\Mathematica & For[l = \{2, 3, 5\}, l != \{\} \&\& l@[[1]]\^{}2 < 10,
\\
+ & \q l = Rest[l], Print[l@[[1]]] ]
\\
\MuPAD & for x in [2, 3, 5] do if x\^{}2 < 10 then print(x) end\_if
\\
& \q end\_for:
\\
\Octave & \\
@@ -683,7 +679,7 @@
\Magnus & & & \\
\Maxima & part(e, 0) & part(e, 1) & args(e) \\
\Maple & op(0, e) & op(1, e) & [op(e)] \\
-\Mathematica & Head[e] & e[[1]] & ReplacePart[e, List, 0] \\
+\Mathematica & Head[e] & e@[[1]] & ReplacePart[e, List, 0] \\
\MuPAD & op(e, 0) & op(e, 1) & [op(e)] \\
\Octave & & & \\
\Pari & & & \\
@@ -789,7 +785,7 @@
\Maple & writedata("file", xy);
\\
\Mathematica & outfile = OpenWrite["file"];
\\
& Do[WriteString[outfile,
\\
- & \q xy[[i, 1]], " ", xy[[i, 2]], "$\backslash$n"], \{i, 1,
n\}] \\
+ & \q xy@[[i, 1]], " ", xy@[[i, 2]], "$\backslash$n"], \{i, 1,
n\}] \\
& Close[outfile];
\\
\MuPAD & fprint(Unquoted, Text, "file",
\\
& \q ("$\backslash$n", xy[i, 1], xy[i, 2]) \$ i = 1..n):
\\
@@ -1513,4 +1509,3 @@
\end{tt}
\endgroup
\end{document}
-

------------

Regards,
Bill Page.
Gabriel Dos Reis
2006-10-23 05:48:21 UTC
Permalink
"Bill Page" <***@synthesis.anikast.ca> writes:

| Here is a patch to Rosetta.pamphlet to make it a real pamphlet
| (noweb) file:

This looks OK to me. Tim?

-- Gaby
Loading...