DMD 1.005 release

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

Good stuff.

janderson

2007-02-06 07:11:21 UTC

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

The new mixin stuff with the quotes seems a bit left of field. However
I'd imagine you could write some very reusable code with the string
con-concatenation stuff, although I'll bet it'll make things very hard
to debug. Maybe even use the strings for some kinda meta/reflection
coding. I can't wait to see some real-world examples.

-Joel

Kirk McDonald

2007-02-06 07:25:01 UTC

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

Mwahaha! This program, when run, prints out a copy of its own source. It
also has some completely gratuitous new-style mixins.

// file test.d
mixin(`import std.stdio : writefln;`);

mixin(`void main() {
mixin("writefln(import(\"test.d\"));");
}`);

--
Kirk McDonald
Pyd: Wrapping Python with D
http://pyd.dsource.org

BCS

2007-02-06 16:46:05 UTC

Post by Kirk McDonald

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

Mwahaha! This program, when run, prints out a copy of its own source. It
also has some completely gratuitous new-style mixins.
// file test.d
mixin(`import std.stdio : writefln;`);
mixin(`void main() {
mixin("writefln(import(\"test.d\"));");
}`);

Without the gratuitous stuff that has to be the cleanest quine outside
of bash (in bash an empty file prints nothing)

import std.stdio;
void main(){writef(import(__FILE__));}

Jari-Matti Mäkelä

2007-02-06 16:58:58 UTC

Post by BCS
Without the gratuitous stuff that has to be the cleanest quine outside
of bash (in bash an empty file prints nothing)
import std.stdio;
void main(){writef(import(__FILE__));}

And if the strings don't mess up with printf, it can be made even shorter:

void main(){printf(import(__FILE__));}

Jarrett Billingsley

2007-02-07 05:45:14 UTC

Post by Jari-Matti MÃ¤kelÃ¤

"Jari-Matti Mäkelä" <jmjmak at utu.fi.invalid> wrote in message

Post by BCS
Without the gratuitous stuff that has to be the cleanest quine outside
of bash (in bash an empty file prints nothing)
import std.stdio;
void main(){writef(import(__FILE__));}

void main(){printf(import(__FILE__));}

"writef".length == "printf".length

Chris Nicholson-Sauls

2007-02-07 05:56:36 UTC

Post by Jari-Matti MÃ¤kelÃ¤

Post by Jarrett Billingsley
"Jari-Matti Mäkelä" <jmjmak at utu.fi.invalid> wrote in message

Post by BCS
Without the gratuitous stuff that has to be the cleanest quine outside
of bash (in bash an empty file prints nothing)
import std.stdio;
void main(){writef(import(__FILE__));}

void main(){printf(import(__FILE__));}

"writef".length == "printf".length

But the "printf" version is -= "import std.stdio;".length + 1;

That said, for examplar|demonstrative D code I'd just assume avoid printf regardless. It
just isn't "the D way." Personal preferance, though, would be:

import tango.io.Stdout;
void main(){Stdout(import(__FILE__));}

-- Chris Nicholson-Sauls

Anders F Björklund

2007-02-07 10:54:24 UTC

Post by Chris Nicholson-Sauls

Post by Jarrett Billingsley
"writef".length == "printf".length

But the "printf" version is -= "import std.stdio;".length + 1;

If you use printf, you should use "import std.c.stdio;"

That it works without it is a long-standing bug, IMHO.

Post by Chris Nicholson-Sauls
That said, for examplar|demonstrative D code I'd just assume
avoid printf regardless. It just isn't "the D way."

I don't think there is anything inherently wrong with
using printf or the rest of the C standard library,
as long as it is explicitly imported by the D code ?
Having "printf" defined in Object is evil, though...

--anders

Stewart Gordon

2007-02-07 18:46:04 UTC

BCS Wrote:
<snip>

Post by BCS
Without the gratuitous stuff that has to be the cleanest quine
outside of bash (in bash an empty file prints nothing)
import std.stdio;
void main(){writef(import(__FILE__));}

What is your definition of "clean"?

Moreover, there are many languages in which an empty source file is a null program - BASIC, Perl and probably most shell scripting languages (indeed, probably most scripting languages) have this characteristic. In the course of history there have even been one or two C compliers that did this.

Stewart.

Kevin Bealer

2007-02-06 07:28:55 UTC

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

You fixed all the bugs I've added in recent memory. Plus, if I
understand correctly, the implications of some of these features is
staggering...

It looks like one could write a few hundred line module that can pull in
and do compile-time interpreting of a language of the complexity of say,
Scheme. And the code in the module could be both readable and
straightforward... And the results would be absorbed into the calling
code as normal optimizable statements...

"I warn you, Doctor -- man was not meant to have this kind of power!"

Kevin

Don Clugston

2007-02-06 07:54:36 UTC

Post by Kevin Bealer

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

You fixed all the bugs I've added in recent memory. Plus, if I
understand correctly, the implications of some of these features is
staggering...
It looks like one could write a few hundred line module that can pull in
and do compile-time interpreting of a language of the complexity of say,
Scheme. And the code in the module could be both readable and
straightforward... And the results would be absorbed into the calling
code as normal optimizable statements...
"I warn you, Doctor -- man was not meant to have this kind of power!"
Kevin

My thoughts exactly.
And it only warrants a version number increase of 0.001 ??? <g>

Walter Bright

2007-02-06 08:33:48 UTC

Post by Kevin Bealer
You fixed all the bugs I've added in recent memory. Plus, if I
understand correctly, the implications of some of these features is
staggering...
It looks like one could write a few hundred line module that can pull in
and do compile-time interpreting of a language of the complexity of say,
Scheme. And the code in the module could be both readable and
straightforward... And the results would be absorbed into the calling
code as normal optimizable statements...

The irony is that it only took 3 hours to implement, which shows the
power of having the lexing, parsing, and semantic passes be logically
distinct.

The idea is to enable the creation of DSLs (Domain Specific Languages)
that don't have the crippling problem C++ expression templates have -
that of being stuck with C++ operators and precedence.

To make this work, however, one must be able to manipulate strings at
compile time. I've made a start on a library to do this,
std.metastrings, based on earlier work by Don Clugston and Eric Anderton.

This is just the start of what's going to happen with D 2.0.

Yauheni Akhotnikau

2007-02-06 10:16:00 UTC

Post by Walter Bright
The idea is to enable the creation of DSLs (Domain Specific Languages)
that don't have the crippling problem C++ expression templates have -
that of being stuck with C++ operators and precedence.

Not only that. This opens clean and simple way to use pre-compile-time
code generation. For example:

// greeting.d:
import std.stdio;

class Greeting
{
mixin( import( "greeting.impl.d" ) );
}

void
main()
{
auto g = new Greeting;
g.hello();
g.bye();
}

// greeting.impl.d:
void
hello()
{
writefln( "Hello!" );
}

void
bye()
{
writefln( "Bye!" );
}

Where the content of greeting.impl.d can be generated by some
domain-specific tool (such as ASN.1 serializator/deserializator
generators).

It's a very good news! Thanks!

--
Regards,
Yauheni Akhotnikau

BLS

2007-02-06 14:35:11 UTC

Post by Walter Bright
The idea is to enable the creation of DSLs (Domain Specific Languages)

to figure it out / This means /

procedure gcd( num1, num2 )
// calc. the greatest C D
return aValue

or

PROCEDURE gcd
IN PARAMETERS num1, num2
OUT PARAMETERS aValue

Is this something I can establish in D since 1.005 ?

Bjoern

Bill Baxter

2007-02-06 14:59:31 UTC

Post by Kevin Bealer
You fixed all the bugs I've added in recent memory. Plus, if I
understand correctly, the implications of some of these features is
staggering...
It looks like one could write a few hundred line module that can pull
in and do compile-time interpreting of a language of the complexity of
say, Scheme. And the code in the module could be both readable and
straightforward... And the results would be absorbed into the calling
code as normal optimizable statements...

The irony is that it only took 3 hours to implement, which shows the
power of having the lexing, parsing, and semantic passes be logically
distinct.
The idea is to enable the creation of DSLs (Domain Specific Languages)
that don't have the crippling problem C++ expression templates have -
that of being stuck with C++ operators and precedence.
To make this work, however, one must be able to manipulate strings at
compile time. I've made a start on a library to do this,
std.metastrings, based on earlier work by Don Clugston and Eric Anderton.
This is just the start of what's going to happen with D 2.0.

For those who haven't seen it, Walter updated
http://www.digitalmars.com/d/mixin.html with a simple example of what
you can do with the mixin string mojo.

Phobos also has new documentation for metastrings:
http://www.digitalmars.com/d/phobos/std_metastrings.html

It looks like this feature is calling out for some sort of new 'here
document' syntax, so that the code in strings can look and read more
like code.

It certainly seems to solve the long standing 'can't generate
identifiers at compile time' feature request. And then some. :-)

It'll be very exciting to see what people come up with!

--bb

Pragma

2007-02-06 15:52:27 UTC

Post by Kevin Bealer
You fixed all the bugs I've added in recent memory. Plus, if I
understand correctly, the implications of some of these features is
staggering...
It looks like one could write a few hundred line module that can pull
in and do compile-time interpreting of a language of the complexity of
say, Scheme. And the code in the module could be both readable and
straightforward... And the results would be absorbed into the calling
code as normal optimizable statements...

It's funny you should say that. I was kidding with Kris in IRC last week about how you could just slap a copy of
DMDScript in the compiler and let us talk to it directly from within templates. While this isn't letting us muck about
with the AST, to create specalized grammars, this is certainly a more elegant solution.

... and it doesn't even require a separate syntax.

Post by Walter Bright
To make this work, however, one must be able to manipulate strings at
compile time. I've made a start on a library to do this,
std.metastrings, based on earlier work by Don Clugston and Eric Anderton.
This is just the start of what's going to happen with D 2.0.

--
- EricAnderton at yahoo

Walter Bright

2007-02-06 16:24:42 UTC

It's funny you should say that. I was kidding with Kris in IRC last
week about how you could just slap a copy of DMDScript in the compiler
and let us talk to it directly from within templates. While this isn't
letting us muck about with the AST, to create specalized grammars, this
is certainly a more elegant solution.
... and it doesn't even require a separate syntax.

Andrei and I toyed with that exact idea for a while. It got shot down
after it became clear that since DMDScript has a "same-only-different"
syntax from D, it would be terribly confusing.

Lionello Lunesu

2007-02-06 16:38:52 UTC

Post by Kevin Bealer
You fixed all the bugs I've added in recent memory. Plus, if I
understand correctly, the implications of some of these features is
staggering...
It looks like one could write a few hundred line module that can pull
in and do compile-time interpreting of a language of the complexity of
say, Scheme. And the code in the module could be both readable and
straightforward... And the results would be absorbed into the calling
code as normal optimizable statements...

The irony is that it only took 3 hours to implement, which shows the
power of having the lexing, parsing, and semantic passes be logically
distinct.
The idea is to enable the creation of DSLs (Domain Specific Languages)
that don't have the crippling problem C++ expression templates have -
that of being stuck with C++ operators and precedence.
To make this work, however, one must be able to manipulate strings at
compile time. I've made a start on a library to do this,
std.metastrings, based on earlier work by Don Clugston and Eric Anderton.

You know the next step, right? A template version of htod!

include!("gl.h");

:D

L.

Walter Bright

2007-02-06 16:56:48 UTC

Post by Lionello Lunesu
You know the next step, right? A template version of htod!
include!("gl.h");

I did shy away from the "execute this shell command and insert its
output into a string literal" because that would turn a D compiler into
a huge security risk.

BCS

2007-02-06 16:57:41 UTC

it needs some string manipulation stuff. I'd be more than happy to let
you put the string templates from dparse in. It has template to:

Discard leading white space
return a slice up to the first white space char
return a slice starting with the first white space char
Return a slice up-to but not including the first instance of t.
Return a slice starting after the first instance of t and containing the
rest of the string.
discard [ ]* then return [a-zA-Z_][a-zA-Z0-9_]*

non-string type template

return tuple with string broken up by d
return tuple with string broken up by white space
tuple cdr (think lisp)
check if anything in tuple begins with a given prefix

source at:
http://www.dsource.org/projects/scrapple/browser/trunk/dparser/dparse.d

BLS

2007-02-06 14:45:26 UTC

Post by Walter Bright
The idea is to enable the creation of DSLs (Domain Specific Languages)
It looks like one could write a few hundred line module that can pull
in and do compile-time interpreting of a language of the complexity

of say Scheme.

to figure it out / This means /

procedure foo( num1, num2 )
return aValue

or

PROCEDURE fo
IN PARAMETERS num1, num2
OUT PARAMETERS aValue

Is this something I can establish in D since 1.005 ?
?
Bjoern

Kevin Bealer schrieb:

BLS

2007-02-06 15:17:47 UTC

I guess here is a need for further explaination.

Either I am an complete idiot (not completely unrealistic) and
missunderstood something, or a new, quit radical, programming paradigmn
change is on it s way. I mean it is difficult to realize the implications.
Bjoern

Post by BLS

of say Scheme.

to figure it out / This means /
procedure foo( num1, num2 )
return aValue
or
PROCEDURE fo
IN PARAMETERS num1, num2
OUT PARAMETERS aValue
Is this something I can establish in D since 1.005 ?
?
Bjoern

Pragma

2007-02-06 15:57:46 UTC

Post by BLS
I guess here is a need for further explaination.
Either I am an complete idiot (not completely unrealistic) and
missunderstood something, or a new, quit radical, programming paradigmn
change is on it s way. I mean it is difficult to realize the implications.
Bjoern

Just try to wrap your head around this: http://www.digitalmars.com/d/mixin.html

template GenStruct(char[] Name, char[] M1)
{
const char[] GenStruct = "struct " ~ Name ~ "{ int " ~ M1 ~ "; }";
}

mixin(GenStruct!("Foo", "bar"));

//which generates:

struct Foo { int bar; }

In short this means that we can have *100%* arbitrary code generation at compile time, w/o need of a new grammar to
support the capability.

--
- EricAnderton at yahoo

BLS

2007-02-06 17:02:09 UTC

Post by BLS
I guess here is a need for further explaination.
Either I am an complete idiot (not completely unrealistic) and
missunderstood something, or a new, quit radical, programming
paradigmn change is on it s way. I mean it is difficult to realize
the implications.
Bjoern

http://www.digitalmars.com/d/mixin.html
template GenStruct(char[] Name, char[] M1)
{
const char[] GenStruct = "struct " ~ Name ~ "{ int " ~ M1 ~ "; }";
}
mixin(GenStruct!("Foo", "bar"));
struct Foo { int bar; }
In short this means that we can have *100%* arbitrary code generation at
compile time, w/o need of a new grammar to support the capability.

Hi Eric,
I am able to read and understand the code. (not nessesarily the far
reaching implications)

Post by Bill Baxter
The idea is to enable the creation of DSLs (Domain Specific Languages)

How ?
Bjoern
Post scriptum
I can imagine the following scenario : D Compiler is calling a
Translator, a modified Enki f.i. to translate a Domain Specific Language
into D ... strange

Pragma

2007-02-06 17:42:24 UTC

Post by BLS

Post by BLS
I guess here is a need for further explaination.
Either I am an complete idiot (not completely unrealistic) and
missunderstood something, or a new, quit radical, programming
paradigmn change is on it s way. I mean it is difficult to realize
the implications.
Bjoern

http://www.digitalmars.com/d/mixin.html
template GenStruct(char[] Name, char[] M1)
{
const char[] GenStruct = "struct " ~ Name ~ "{ int " ~ M1 ~ "; }";
}
mixin(GenStruct!("Foo", "bar"));
struct Foo { int bar; }
In short this means that we can have *100%* arbitrary code generation
at compile time, w/o need of a new grammar to support the capability.

Hi Eric,
I am able to read and understand the code. (not nessesarily the far
reaching implications)

Post by Bill Baxter
The idea is to enable the creation of DSLs (Domain Specific Languages)

How ?
Bjoern

I think you answered your own question. :)

Take the compile-time regexp lib that Don and I wrote a while back. Technically, Regular-expressions are a DSL of
sorts. This feature just makes the implementation of stuff like that easier. The end result will still be D code.

auto widget = CreateNewWidget!("Some DSL Code");

I was confused too, since the wording could be interpreted as allowing you to just code in some other language, wherever
you want. This is not the case. Ultimately, any DSL implemented in this fashion is going to have to operate on static
strings.

Post by BLS
I can imagine the following scenario : D Compiler is calling a
Translator, a modified Enki f.i. to translate a Domain Specific Language
into D ... strange

I've thought about that too- much like BCS's work. The only thing keeping me from doing this *was* that the code
generated would be largely inferior to that created by an external program. Thanks to the new syntax of mixin(), this
is no longer the case.

--
- EricAnderton at yahoo

Chris Nicholson-Sauls

2007-02-06 21:13:27 UTC

Post by BLS

Post by BLS
I guess here is a need for further explaination.
Either I am an complete idiot (not completely unrealistic) and
missunderstood something, or a new, quit radical, programming
paradigmn change is on it s way. I mean it is difficult to realize
the implications.
Bjoern

http://www.digitalmars.com/d/mixin.html
template GenStruct(char[] Name, char[] M1)
{
const char[] GenStruct = "struct " ~ Name ~ "{ int " ~ M1 ~ "; }";
}
mixin(GenStruct!("Foo", "bar"));
struct Foo { int bar; }
In short this means that we can have *100%* arbitrary code generation
at compile time, w/o need of a new grammar to support the capability.

Hi Eric,
I am able to read and understand the code. (not nessesarily the far
reaching implications)

Post by Bill Baxter
The idea is to enable the creation of DSLs (Domain Specific Languages)

How ?
Bjoern

You just showed something that I've been pondering attempting with this. Namely a GUI
library that builds all forms/controls/etc from some sort of markup (probably modified XML
or JSON), either at runtime /or/ compile-time.

-- Chris Nicholson-Sauls

renoX

2007-02-07 19:12:54 UTC

Post by BLS
I can imagine the following scenario : D Compiler is calling a
Translator, a modified Enki f.i. to translate a Domain Specific Language
into D ... strange

I've thought about that too- much like BCS's work. The only thing

Enki?
BCS?

Could you avoid mysterious references?

Regards,
renoX

Frits van Bommel

2007-02-07 19:20:09 UTC

Post by renoX

Post by BLS
I can imagine the following scenario : D Compiler is calling a
Translator, a modified Enki f.i. to translate a Domain Specific Language
into D ... strange

I've thought about that too- much like BCS's work. The only thing

Enki?
BCS?
Could you avoid mysterious references?

Enki: http://www.dsource.org/projects/ddl/wiki/Enki

BCS is a poster in these newsgroups. He's been mentioning a project of
his called dparser lately:
http://www.dsource.org/projects/scrapple/browser/trunk/dparser/dparse.d

BCS

2007-02-07 19:27:33 UTC

Post by renoX

Post by BLS
I can imagine the following scenario : D Compiler is calling a
Translator, a modified Enki f.i. to translate a Domain Specific Language
into D ... strange

I've thought about that too- much like BCS's work. The only thing

Enki?

http://www.dsource.org/projects/ddl/browser/trunk/enki
written by Pragma

Post by renoX
BCS?

http://www.dsource.org/projects/scrapple/browser/trunk/dparser/dparse.d
written by me (BCS)

Post by renoX
Could you avoid mysterious references?
Regards,
renoX

Chris Nicholson-Sauls

2007-02-06 21:03:01 UTC

Post by BLS
I guess here is a need for further explaination.
Either I am an complete idiot (not completely unrealistic) and
missunderstood something, or a new, quit radical, programming
paradigmn change is on it s way. I mean it is difficult to realize
the implications.
Bjoern

Combine that with the ever increasing power of Tuples, some of the slowly improving type
info, and compile-time string manipulation.... and you've got programs that write programs
for writing programs that generate the original program automatically. o_O

I like it. Might even scrap some old code in favor of it.

-- Chris Nicholson-Sauls

Walter Bright

2007-02-06 16:28:22 UTC

I think you're right. The only thing that makes me uneasy is the
"preprocessor abuse" that comes up in C++. We should be careful in how
we use this, lest the cuticle side of the thumb take over.

Sean Kelly

2007-02-06 17:01:02 UTC

Post by BLS
I guess here is a need for further explaination.
Either I am an complete idiot (not completely unrealistic) and
missunderstood something, or a new, quit radical, programming
paradigmn change is on it s way. I mean it is difficult to realize
the implications.

I think you're right. The only thing that makes me uneasy is the
"preprocessor abuse" that comes up in C++. We should be careful in how
we use this, lest the cuticle side of the thumb take over.

The most obvious danger is simply being able to eyeball what the source
code for a module actually is, but that's been an issue for any
sufficiently complex template code anyway. What I like about this
feature is that it improves upon the power of macros but does so without
providing a method for changing the meaning of existing symbols (the
"#define if while" problem). It also requires almost no new language
features, so it shouldn't have a tremendous learning curve. Finally,
since all this works via strings, it should be easy to determine what's
actually going on simply by tossing in a few pragma(msg) statements. If
there were a way to emit the "expanded" source we could even use this as
a "standalone" code generation tool of sorts. Nice work!

Sean

BCS

2007-02-06 17:24:34 UTC

Post by Sean Kelly
The most obvious danger is simply being able to eyeball what the source
code for a module actually is, but that's been an issue for any
sufficiently complex template code anyway.

How are #line directives handled? is their any way to tell the debugger
to look at another file:

mixin(MixInThisFile("foo"));

// results in this

// stuff
#line foo, 127
// stuff from foo:127

#line ... // revert back to original file:line

Then, in the debugger, it would start stepping you through foo in the
correct place.

Post by Sean Kelly
If
there were a way to emit the "expanded" source we could even use this as
a "standalone" code generation tool of sorts. Nice work!

Put in a pragma msg in place of the mixin and you get the code.

maybe a mixin(string, filename) form would be nice. It would dump to
given file as well as generate code.

Post by Sean Kelly
Sean

Sean Kelly

2007-02-06 20:01:58 UTC

Post by Sean Kelly
The most obvious danger is simply being able to eyeball what the
source code for a module actually is, but that's been an issue for any
sufficiently complex template code anyway.

How are #line directives handled? is their any way to tell the debugger
mixin(MixInThisFile("foo"));
// results in this
// stuff
#line foo, 127
// stuff from foo:127
#line ... // revert back to original file:line
Then, in the debugger, it would start stepping you through foo in the
correct place.

I suspect that generating debug info will require the mixed-in code to
be expanded in place with the proper #line directives, etc, in the
object file.

Post by Sean Kelly
If there were a way to emit the "expanded" source we could even use
this as a "standalone" code generation tool of sorts. Nice work!

Put in a pragma msg in place of the mixin and you get the code.

Yup. I think for a standalone code generator, it would probably be
better to generate the output file completely through pragma(msg) so as
to omit the template code used for processing the mixins. For example,
I figure it shouldn't be terribly difficult to do D codegen from a UML
file in template code, etc.

Sean

Pragma

2007-02-06 17:30:42 UTC

If there were a way to emit the "expanded" source we could even use this as
a "standalone" code generation tool of sorts.

Now there's something that's missing. At least with C compilers, they can usually be asked to spit out what things look
like after the pre-processor is done with it. Getting the same kind of results from DMD, after all templates are
evaluated would be great for diagnostics and debugging.

--
- EricAnderton at yahoo

BCS

2007-02-06 17:05:33 UTC

This could do thing like this:

BuildBarserFromFileSpec!("foo.bnf")

that would import "foo.bnf", parser a BNF grammar, and build a parser
from it. It even avoids the need to use functions for callbacks.

p.s.
I'm going to have to try and re-implement dparse using this.

Darn you Walter!!! I don't have time for this (cool stuff)!!! <G>

Pragma

2007-02-06 17:32:06 UTC

Post by BLS
I guess here is a need for further explaination.
Either I am an complete idiot (not completely unrealistic) and
missunderstood something, or a new, quit radical, programming
paradigmn change is on it s way. I mean it is difficult to realize
the implications.
Bjoern

BuildBarserFromFileSpec!("foo.bnf")
that would import "foo.bnf", parser a BNF grammar, and build a parser
from it. It even avoids the need to use functions for callbacks.
p.s.
I'm going to have to try and re-implement dparse using this.
Darn you Walter!!! I don't have time for this (cool stuff)!!! <G>

BCS: I may be tempted to Enki-ize your work once you're done with that. I think a compile-time rendition is due. ;)

--
- EricAnderton at yahoo

BCS

2007-02-06 17:39:47 UTC

Post by BCS
p.s.
I'm going to have to try and re-implement dparse using this.

BCS: I may be tempted to Enki-ize your work once you're done with that.
I think a compile-time rendition is due. ;)

Done? What is that? I haven't heard the term before. <g>

Pragma

2007-02-06 17:51:16 UTC

Post by BCS
p.s.
I'm going to have to try and re-implement dparse using this.

BCS: I may be tempted to Enki-ize your work once you're done with
that. I think a compile-time rendition is due. ;)

Done? What is that? I haven't heard the term before. <g>

BCS, you're not working at 3D-realms by any chance, are you? <eg>

--
- EricAnderton at yahoo

BCS

2007-02-06 18:36:43 UTC

Post by BCS
p.s.
I'm going to have to try and re-implement dparse using this.

BCS: I may be tempted to Enki-ize your work once you're done with
that. I think a compile-time rendition is due. ;)

Done? What is that? I haven't heard the term before. <g>

BCS, you're not working at 3D-realms by any chance, are you? <eg>

no, I'm an Mechanical engineering undergraduate student working in
declarative code/system generation (professionally) and secure software
systems (academically)

Andreas Kochenburger

2007-02-06 20:41:28 UTC

I am not a D programmer (yet) only observing what is happening.

I compare the new "code generation at compile-time" stuff in D with
Forth. Forth also has a built-in interpreter & compiler which extends
the language and can also execute macros at compile-time through
EVALUATE. Of course Forth is much more low-level than D. But IMO the new
mixins are not a "radical programming paradigm change".

Perhaps I just did misunderstand something?

Andreas

janderson

2007-02-07 07:45:28 UTC

Post by Andreas Kochenburger

Post by BLS
I guess here is a need for further explaination.
Either I am an complete idiot (not completely unrealistic) and
missunderstood something, or a new, quit radical, programming
paradigmn change is on it s way. I mean it is difficult to realize
the implications.
Bjoern

I am not a D programmer (yet) only observing what is happening.
I compare the new "code generation at compile-time" stuff in D with
Forth. Forth also has a built-in interpreter & compiler which extends
the language and can also execute macros at compile-time through
EVALUATE. Of course Forth is much more low-level than D. But IMO the new
mixins are not a "radical programming paradigm change".
Perhaps I just did misunderstand something?
Andreas

I'm not familiar with forth. Can you provide some examples? Does it
allow partial macro definitions. Can you apply string operations on
them at compile time?

-Joel

Andreas Kochenburger

2007-02-07 19:39:34 UTC

Post by janderson
I'm not familiar with forth. Can you provide some examples? Does it
allow partial macro definitions. Can you apply string operations on
them at compile time?

Forth is not a traditional compiler that generates executables from
source files. A Forth system includes a built-in interpreter and
compiler (and most systems have an assembler too).

Source definitions are compiled and linked to the internal dictionary
i.e. you extend the system itself. You can not only create application
programms, but you can also easily add new features to the compiler e.g.
new compiler commands.

Macro functions at compile-time are only a small exercise for a Forth
programmer. You mark the last defined function IMMEDIATE, and the next
time the function is used it is executed _at_compile-time_ !

Please note, Forth and D are playing in different leagues. But you don't
always have large object-oriented applications. For the more "bare
metal" stuff Forth is flexibler than D. If you never had contact with
Forth you will probably find it rather strange: it is a stack-based
language and uses post-fix annotation like HP calculators.

Andreas

Lionello Lunesu

2007-02-06 08:53:27 UTC

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

!!!! This is just what I needed for a compile-time .rc compiler!

L.

Lars Ivar Igesund

2007-02-06 09:42:53 UTC

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

Sounds like some nice new features, but even though the compiler seems to
know that these new features are D 2.0, the spec don't show it. I'd suggest
to branch the specification now, after all 1.0 shouldn't see any new
feature changes. Without this, there is no point in the 1.0 marker
whatsoever.

--
Lars Ivar Igesund
blog at http://larsivi.net
DSource & #D: larsivi
Dancing the Tango

Frank Benoit (keinfarbton)

2007-02-06 13:30:02 UTC

Post by Lars Ivar Igesund
Sounds like some nice new features, but even though the compiler seems to
know that these new features are D 2.0, the spec don't show it. I'd suggest
to branch the specification now, after all 1.0 shouldn't see any new
feature changes. Without this, there is no point in the 1.0 marker
whatsoever.

I second that.

BCS

2007-02-06 16:48:45 UTC

Post by Lars Ivar Igesund

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

I also second that, branch the spec or annotate it *Vary* well. Either
way, the change log should say v2.0 as well

mike

2007-02-06 15:19:38 UTC

Nice!

That new mixin stuff will be great for that stacktrace I'm planning to do
for months now ... gotta try that out asap :)

-Mike

Am 06.02.2007, 05:54 Uhr, schrieb Walter Bright

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

--
Erstellt mit Operas revolutionärem E-Mail-Modul: http://www.opera.com/mail/

Pragma

2007-02-06 15:44:38 UTC

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

"The AssignExpression must evaluate at compile time to a constant string. The text contents of the string must be
compilable as a valid StatementList, and is compiled as such.:"

Arbitrary code generation?! This ought to make for some really slick compile-time code generators - say goodbye to
delegate calling overhead and static variable bloat.

The import expression thing has me scratching my head though: what path does DMD use to determine where to find the
imported file? (it's not clear in the documentation)

Awesome update Walter - thanks again. :)

--
- EricAnderton at yahoo

Walter Bright

2007-02-06 16:33:39 UTC

Post by Pragma
The import expression thing has me scratching my head though: what path
does DMD use to determine where to find the imported file? (it's not
clear in the documentation)

It just looks in the default directory. I know this is inadequate for a
long term solution, but I wanted to see what people thought of it before
spending a lot of effort on the details.

janderson

2007-02-06 17:06:05 UTC

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

The import stuff has been part of C for a long time (in the form of
#include), however I've never seen it used. Maybe with string
operations it will be useful, but otherwise I don't see the point.

=Joel

BCS

2007-02-06 17:27:14 UTC

Post by janderson

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

not quite, I don't think this works in c

char string[] = "#import<bigstring.txt>"

sting gets the value of "#import<bigstring.txt>", not the contents of
"bigstring.txt"

janderson

2007-02-07 04:32:35 UTC

Post by janderson

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

not quite, I don't think this works in c
char string[] = "#import<bigstring.txt>"
sting gets the value of "#import<bigstring.txt>", not the contents of
"bigstring.txt"

No but you could do:

char* string =
#import "bigstring.txt"

and have quotes in the bigstring.txt

D's look nicer but its been around for ages. I'm sure combining it with
the mixins will make all the difference.

-Joel

Walter Bright

2007-02-06 18:07:23 UTC

Post by janderson
The import stuff has been part of C for a long time (in the form of
#include), however I've never seen it used. Maybe with string
operations it will be useful, but otherwise I don't see the point.

The fundamental difference is that #include inserts *program text*,
while import inserts the contents as a *string literal*.

Some things that you can do with import that you cannot do with #include:

1) You can have tech writers write "help text" files, which can then be
imported by the programmers as string literals. This means the tech
writers do not have to be concerned in the slightest with string syntax.

2) It's an easy way to bind binary data into a program. For example,
let's say one wants to embed an icon (.ico) file into your program
binary. In C, one would have to write:

static unsigned char icon[] = { 0x00, 0x10, 0x53, 0x29, ... };

meaning one must translate the binary data in foo.ico to the hex notation.

In D, one can write:

static ubyte[] icon = cast(ubyte[])import("foo.ico");

janderson

2007-02-07 04:36:18 UTC

The fundamental difference is that #include inserts *program text*,
while import inserts the contents as a *string literal*.
1) You can have tech writers write "help text" files, which can then be
imported by the programmers as string literals. This means the tech
writers do not have to be concerned in the slightest with string syntax.

They could just add quotes around them in C. Slightly less convenient,
never seen anyone use that feature.

Post by Walter Bright
2) It's an easy way to bind binary data into a program. For example,
let's say one wants to embed an icon (.ico) file into your program
static unsigned char icon[] = { 0x00, 0x10, 0x53, 0x29, ... };
meaning one must translate the binary data in foo.ico to the hex notation.
static ubyte[] icon = cast(ubyte[])import("foo.ico");

This is a good point. Maybe it should be on the webpage.

-Joel

Walter Bright

2007-02-07 05:13:02 UTC

Post by janderson

Post by Walter Bright
1) You can have tech writers write "help text" files, which can then
be imported by the programmers as string literals. This means the tech
writers do not have to be concerned in the slightest with string syntax.

They could just add quotes around them in C.

Not exactly. You cannot have multiline string literals in C. You'd have
to use \n\ line splicing, and escape any " and \ embedded in the text:

"Your text\n\
would have to\n\
look like this\n\
and be careful to\n\
escape any \"s in\n\
the text, as well as\n\
any \\s."

Furthermore, string literals are limited to 4095 characters (if one
cares about portability). C99 5.2.4.1

These restrictions are annoying enough that I've preferred using runtime
reading and loading of the message file instead.

mike

2007-02-06 18:23:33 UTC

Updated DMD to 1.005 from 1.0 today and now I get this error:

' C:\dmd\src\ext\derelict\sdl\ttf.d(79): struct derelict.sdl.ttf._TTF_Font
unknown
' size
' C:\dmd\src\ext\derelict\sdl\ttf.d(79): struct derelict.sdl.ttf._TTF_Font
no size
' yet for forward reference

Does anybody know how to fix that? I've already searched the NG, the
derelict forum, upgraded to the latest trunk ... nothing helped so far.

-Mike

Am 06.02.2007, 05:54 Uhr, schrieb Walter Bright

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

--
Erstellt mit Operas revolutionärem E-Mail-Modul: http://www.opera.com/mail/

Thomas Brix Larsen

2007-02-06 18:28:06 UTC

Post by mike
' C:\dmd\src\ext\derelict\sdl\ttf.d(79): struct derelict.sdl.ttf._TTF_Font
unknown
' size
' C:\dmd\src\ext\derelict\sdl\ttf.d(79): struct derelict.sdl.ttf._TTF_Font
no size
' yet for forward reference
Does anybody know how to fix that? I've already searched the NG, the
derelict forum, upgraded to the latest trunk ... nothing helped so far.
-Mike

It should read: struct _TTF_Font {}
http://dsource.org/projects/derelict/browser/trunk/DerelictSDLttf/derelict/sdl/ttf.d

- Brix

mike

2007-02-06 22:09:51 UTC

Thanks! That worked :)

Post by Thomas Brix Larsen

Post by mike
' C:\dmd\src\ext\derelict\sdl\ttf.d(79): struct
derelict.sdl.ttf._TTF_Font
unknown
' size
' C:\dmd\src\ext\derelict\sdl\ttf.d(79): struct
derelict.sdl.ttf._TTF_Font
no size
' yet for forward reference
Does anybody know how to fix that? I've already searched the NG, the
derelict forum, upgraded to the latest trunk ... nothing helped so far.
-Mike

It should read: struct _TTF_Font {}
http://dsource.org/projects/derelict/browser/trunk/DerelictSDLttf/derelict/sdl/ttf.d
- Brix

--
Erstellt mit Operas revolutionärem E-Mail-Modul: http://www.opera.com/mail/

Hasan Aljudy

2007-02-06 19:03:05 UTC

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

Wow, this is no small change .. this should've ben dmd 1.2 or something.

Now, there's already been alot of talk about what new doors this might
open, so I'm not gonna talk about that.

What concerns me is that this will make semantic analysis more difficult
to implement.

Just think about "build/bud" for example, now the author will have to
worry about things like:
mixin("import x.y.z");

or even worse:
mixin(templ1!(something, templ2!(somethingelse), "x.y"));

I don't see how it's possible to interpret that without implementing a
full compiler.

P.S. I know that for "build" all we need is a list of import files, and
dmd already has a switch to do that.

Walter Bright

2007-02-06 19:55:18 UTC

Post by Hasan Aljudy
I don't see how it's possible to interpret that without implementing a
full compiler.

You're right, it isn't possible.

Post by Hasan Aljudy
P.S. I know that for "build" all we need is a list of import files, and
dmd already has a switch to do that.

DMD will also output a list of files that are textually imported, so bud
can pick them up at least the second time around.

Derek Parnell

2007-02-06 23:27:53 UTC

Post by Hasan Aljudy
I don't see how it's possible to interpret that without implementing a
full compiler.

You're right, it isn't possible.

Post by Hasan Aljudy
P.S. I know that for "build" all we need is a list of import files, and
dmd already has a switch to do that.

DMD will also output a list of files that are textually imported, so bud
can pick them up at least the second time around.

Thanks Walter! :-(

Bud is no longer a useful tool because it can no longer do what it was
trying to do - namely find out which files needed recompiling and get only
that done. Because in order to do that now, it first has to recursively
compile each command line file and imported file using the -c -v switches
to get a list of the potential files needing to be checked for
recompilation. But seeing I've just compiled them to get this list, there
is not much point now in /recompiling/ them. Also, mixin-imported files are
not necessarily modules but must be treated as code fragments, so they
can't be compiled to see if they in-turn effectively import other files!

My work here is (un)done.

It seems that DMD now needs to be enhanced to do what Rebuild and Bud were
trying to do.

--
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"Down with mediocrity!"
7/02/2007 10:18:52 AM

BCS

2007-02-06 23:59:03 UTC

Reply to Derek,

Post by Derek Parnell
Thanks Walter! :-(
Bud is no longer a useful tool because it can no longer do what it was
trying to do - namely find out which files needed recompiling and get
only that done. Because in order to do that now, it first has to
recursively compile each command line file and imported file using the
-c -v switches to get a list of the potential files needing to be
checked for recompilation. But seeing I've just compiled them to get
this list, there is not much point now in /recompiling/ them. Also,
mixin-imported files are not necessarily modules but must be treated
as code fragments, so they can't be compiled to see if they in-turn
effectively import other files!
My work here is (un)done.
It seems that DMD now needs to be enhanced to do what Rebuild and Bud
were trying to do.

If bud keep around meta data about what happened last time (building N required
A, B, C) then if none of those changed, the set of files that can be used
can't change. Having that kind of tree would let you do a minimal rebuild
even with the new import stuff.

Once you go that direction (DMD would have to report the files used) why
not have DMD report the public interface of each module? that would let bud
notice when a change in a module doesn't demands a rebuild of the modules
that import it. Some of this might be possible by watching for actual changes
in .di file, not just checking modification dates.

Kirk McDonald

2007-02-07 00:39:55 UTC

Post by Derek Parnell

Post by Hasan Aljudy
I don't see how it's possible to interpret that without implementing a
full compiler.

You're right, it isn't possible.

Post by Hasan Aljudy
P.S. I know that for "build" all we need is a list of import files, and
dmd already has a switch to do that.

DMD will also output a list of files that are textually imported, so bud
can pick them up at least the second time around.

Thanks Walter! :-(
Bud is no longer a useful tool because it can no longer do what it was
trying to do - namely find out which files needed recompiling and get only
that done. Because in order to do that now, it first has to recursively
compile each command line file and imported file using the -c -v switches
to get a list of the potential files needing to be checked for
recompilation. But seeing I've just compiled them to get this list, there
is not much point now in /recompiling/ them. Also, mixin-imported files are
not necessarily modules but must be treated as code fragments, so they
can't be compiled to see if they in-turn effectively import other files!
My work here is (un)done.
It seems that DMD now needs to be enhanced to do what Rebuild and Bud were
trying to do.

Perhaps it would be feasible to turn bud (and perhaps rebuild) into
Makefile generators.

--
Kirk McDonald
Pyd: Wrapping Python with D
http://pyd.dsource.org

Walter Bright

2007-02-07 03:04:11 UTC

Post by Derek Parnell
Bud is no longer a useful tool because it can no longer do what it was
trying to do - namely find out which files needed recompiling and get only
that done. Because in order to do that now, it first has to recursively
compile each command line file and imported file using the -c -v switches
to get a list of the potential files needing to be checked for
recompilation. But seeing I've just compiled them to get this list, there
is not much point now in /recompiling/ them. Also, mixin-imported files are
not necessarily modules but must be treated as code fragments, so they
can't be compiled to see if they in-turn effectively import other files!
My work here is (un)done.
It seems that DMD now needs to be enhanced to do what Rebuild and Bud were
trying to do.

The compiler cannot tell what file it'll need to textually import
without compiling either, so cannot do a 'make' on textual imports.

No tool is perfect; I recommend just ignoring the problem with textual
imports. Such shouldn't be used outside of specialized modules.

Hasan Aljudy

2007-02-07 06:15:07 UTC

Post by Derek Parnell

Post by Hasan Aljudy
I don't see how it's possible to interpret that without implementing a
full compiler.

You're right, it isn't possible.

Post by Hasan Aljudy
P.S. I know that for "build" all we need is a list of import files, and
dmd already has a switch to do that.

DMD will also output a list of files that are textually imported, so bud
can pick them up at least the second time around.

Thanks Walter! :-(
Bud is no longer a useful tool because it can no longer do what it was
trying to do - namely find out which files needed recompiling and get only
that done.

Really? I always use -full -clean switches. I use build/bud because dmd
is not smart enough to compile all the files needed to make the program.
dmd is fast enough to not require one to bother do "selective"
compliation. (if one can say so ..)

Ary Manzana

2007-02-06 19:56:29 UTC

Post by Hasan Aljudy

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

Wow, this is no small change .. this should've ben dmd 1.2 or something.
Now, there's already been alot of talk about what new doors this might
open, so I'm not gonna talk about that.
What concerns me is that this will make semantic analysis more difficult
to implement.
Just think about "build/bud" for example, now the author will have to
mixin("import x.y.z");
mixin(templ1!(something, templ2!(somethingelse), "x.y"));
I don't see how it's possible to interpret that without implementing a
full compiler.
P.S. I know that for "build" all we need is a list of import files, and
dmd already has a switch to do that.

I also like the new features and think the same as you.

First, I don't know what is the real beneffit of this features. I mean,
I want to see a real world example using mixins before thinking they are
great (BTW, the first time I saw them PHP inmediately came into my
head... even more after the post about """ @msg """).

Second, this makes even harder to get good IDE support for D. You can
have syntax coloring, and that's it. Autocompletion is going to be a
very though part: the IDE must act as a compiler, as you say it, to
figure out what the program will look like so that it can know what are
the declarations available to the programmer.

Anyway, I'm still working on Descent, and when I get to that part (which
I plan to implement because it's all there, in DMD... I guess?), I'll
tell you. :-)

Walter Bright

2007-02-06 20:08:05 UTC

Post by Ary Manzana
Second, this makes even harder to get good IDE support for D. You can
have syntax coloring, and that's it. Autocompletion is going to be a
very though part: the IDE must act as a compiler, as you say it, to
figure out what the program will look like so that it can know what are
the declarations available to the programmer.

True, but on the other hand, specifically not supporting it in the IDE
may act as a needed brake on evil uses of it.

Ary Manzana

2007-02-06 20:16:14 UTC

True, but on the other hand, specifically not supporting it in the IDE
may act as a needed brake on evil uses of it.

I didn't say an IDE won't support it, I said it'll be very hard to get
there :-)

But... I'm wondering which are the evil uses of it. For me it's now
almost impossible not to program with an IDE (big projects, I mean). At
least in Java. Maybe compile time stuff will make it such that an IDE
won't be needed anymore. But it's very hard for me to see that happening.

Sean Kelly

2007-02-06 20:45:49 UTC

True, but on the other hand, specifically not supporting it in the IDE
may act as a needed brake on evil uses of it.

I didn't say an IDE won't support it, I said it'll be very hard to get
there :-)
But... I'm wondering which are the evil uses of it. For me it's now
almost impossible not to program with an IDE (big projects, I mean). At
least in Java. Maybe compile time stuff will make it such that an IDE
won't be needed anymore. But it's very hard for me to see that happening.

Oddly, I've found myself moving away from IDEs over the years, perhaps
partially because the editors I like to use aren't IDEs. About the only
time I use an IDE any more is for debugging... coding happens elsewhere.

Sean

Charlie

2007-02-06 22:27:18 UTC

Post by Sean Kelly

Post by Ary Manzana
Second, this makes even harder to get good IDE support for D. You
can have syntax coloring, and that's it. Autocompletion is going to
be a very though part: the IDE must act as a compiler, as you say
it, to figure out what the program will look like so that it can
know what are the declarations available to the programmer.

True, but on the other hand, specifically not supporting it in the
IDE may act as a needed brake on evil uses of it.

I didn't say an IDE won't support it, I said it'll be very hard to get
there :-)
But... I'm wondering which are the evil uses of it. For me it's now
almost impossible not to program with an IDE (big projects, I mean).
At least in Java. Maybe compile time stuff will make it such that an
IDE won't be needed anymore. But it's very hard for me to see that
happening.

When you inherit code or start to code on an existing project, the
ability to 'ctrl+click' to jump to that variables definition is a huge
time saver, otherwise you have to grep through tons of files , and in
large libraries the classes may be nested very deep, you might have to
ctrl+click 5 times before you get to what you're looking for.

I've actually gone the opposite way, drifting towards IDE's over time,
even though for 'one-moduler's it remains <insert favorite editor here>.

I am really looking forward to descent, I hope you can devote enough
time to Ary.

Charlie

Walter Bright

2007-02-07 03:20:51 UTC

Post by Walter Bright
True, but on the other hand, specifically not supporting it in the IDE
may act as a needed brake on evil uses of it.

But... I'm wondering which are the evil uses of it.

From my point of view, evil uses of it are things like version control:

mixin(import("versions.txt"));

where versions.txt contains:

version = FOO;
version = BAR;

etc., or other uses that subvert the right way to do things. One should
think long and hard about using textual import to import D code.

What it's for is to:

1) import data for a string constant
2) import code that's in DSL (Domain Specific Language), not D, form.

Hasan Aljudy

2007-02-07 06:11:48 UTC

Post by Walter Bright
True, but on the other hand, specifically not supporting it in the
IDE may act as a needed brake on evil uses of it.

But... I'm wondering which are the evil uses of it.

mixin(import("versions.txt"));
version = FOO;
version = BAR;
etc., or other uses that subvert the right way to do things. One should
think long and hard about using textual import to import D code.

Why is that evil? I think it's actually a great idea. "versions" are a
sort of configuration that determines which code should be compiled and
which code shouldn't. Storing this configuration in a separate file
makes sense to me.

Post by Walter Bright
1) import data for a string constant
2) import code that's in DSL (Domain Specific Language), not D, form.

Walter Bright

2007-02-07 07:16:17 UTC

Post by Hasan Aljudy

Post by Walter Bright
mixin(import("versions.txt"));
version = FOO;
version = BAR;
etc., or other uses that subvert the right way to do things. One
should think long and hard about using textual import to import D code.

The right way to do versions that cut across multiple files is to
abstract the versioning into an API, and implement the different
versions in different modules.

This is a lot easier to manage when you're dealing with larger, more
complex code, although it is more work up front.

Anders F Björklund

2007-02-07 11:01:17 UTC

Post by Hasan Aljudy
Why is that evil? I think it's actually a great idea. "versions" are a
sort of configuration that determines which code should be compiled
and which code shouldn't. Storing this configuration in a separate
file makes sense to me.

The right way to do versions that cut across multiple files is to
abstract the versioning into an API, and implement the different
versions in different modules.
This is a lot easier to manage when you're dealing with larger, more
complex code, although it is more work up front.

How would you use something like autoconf with this approach ?

Would it need to generate different Makefiles / D file lists for
different options/versions, instead of just -version statements ?
Example of things I am thinking about are "__WXGTK__", "UNICODE",
or "HAVE_OPENGL". With C/C++, they're usually in a config.h file.

--anders

Walter Bright

2007-02-07 19:00:43 UTC

Post by Anders F BjÃ¶rklund

Post by Hasan Aljudy
Why is that evil? I think it's actually a great idea. "versions" are
a sort of configuration that determines which code should be compiled
and which code shouldn't. Storing this configuration in a separate
file makes sense to me.

The right way to do versions that cut across multiple files is to
abstract the versioning into an API, and implement the different
versions in different modules.
This is a lot easier to manage when you're dealing with larger, more
complex code, although it is more work up front.

How would you use something like autoconf with this approach ?
Would it need to generate different Makefiles / D file lists for
different options/versions, instead of just -version statements ?
Example of things I am thinking about are "__WXGTK__", "UNICODE",
or "HAVE_OPENGL". With C/C++, they're usually in a config.h file.

I've worked with the C approach for many years, and have gotten
increasingly dissatisfied with it. Over time, it leads to conflicting,
misused, overlapping version macros.

I've also tried the "make an API for the version" method, and have been
much more satisfied with it. You can see it at work in the gc
implementation (see gclinux.d and win32.d).

BCS

2007-02-07 18:52:59 UTC

Post by Walter Bright
The right way to do versions that cut across multiple files is to
abstract the versioning into an API, and implement the different
versions in different modules.

What about cases where 90% of the code is identical but small bits and
peaces are different? If I understand correctly, to do what you suggest
would requirer that those bits be put in functions and have several
versions of the function somewhere else. This could be a problem in
several ways

===Tiny bits of code would requirer tiny functions that would hide what
is going on.

version(RowMajor)
x = table[i][j];
else // RowMinor
x = table[j][i];

====Empty else cases would result in do nothing functions:

version(StrongChecks)
{
if(foo) ...
if(bar) ...
...
}
//empty else

====You can't break across function calls

switch(i)
{
case 1:
version(Baz)
if(baz) break;
else
break;
case 2:

...// lots of un versioned code
}

or

switch(i)
{
version(Baz)
{
case 1:
if(baz) break;
}
case 2:

...// lots of un versioned code

version(!Baz)
{
case 1:
}
}

====lots of version combinations

version(Foo) i = foo(i);
version(Boo) i = boo(i);
version(Fig) i = fig(i);
version(Baz) i = baz(i);
version(Bar) i = bar(i); //32 options???

Are these valid concerns? Am I misunderstanding what you said?

Walter Bright

2007-02-07 19:16:06 UTC

Post by Walter Bright
The right way to do versions that cut across multiple files is to
abstract the versioning into an API, and implement the different
versions in different modules.

Yes, it would require tiny functions, though I don't agree they hide
what is going on. Presumably a descriptive name would be used for it.
One of the nice things about it is that porting the code requires
generating a new module with the right implementations in it, which is a
lot easier than going through checking all the #ifdef's (yes, I know one
shouldn't have to do that, but in practice you ALWAYS have to because
the macros always get misapplied, and often get forgotten to even
apply). With an API, it is difficult to forget to use it, and difficult
to use the wrong macro.

For example, when writing portable C code, one is often faced with the
macros:
__GNUC__ for the Gnu compiler
linux for the host operating system
TARGET_LINUX for the target operating system we're cross compiling for

I can't even count the number of times __GNUC__ was being used to decide
whether we're compiling for windows or linux, or the host operating
system was confused with the target:

#if __GNUC__
#include <pthreads.h>
#else
#include <windows.h>
#endif

AAAARRRRRGGGGGHHHHH!!! That, my friends, is evil.

Now, if one abstracted away what one was *doing* with threads, then one
just does:

import mythreadapi;

and provide a different implementation of mythreadapi for windows or
linux. It's a LOT harder to screw that up.

Post by BCS
version(RowMajor)
x = table[i][j];
else // RowMinor
x = table[j][i];

int GetRow(i,j) { return table[i][j]; }

The function gets inlined.

Post by BCS
version(StrongChecks)
{
if(foo) ...
if(bar) ...
...
}
//empty else
====You can't break across function calls
switch(i)
{
version(Baz)
if(baz) break;
else
break;
...// lots of un versioned code
}

if (globalversion.baz && baz)
break;

Post by BCS
====lots of version combinations
version(Foo) i = foo(i);
version(Boo) i = boo(i);
version(Fig) i = fig(i);
version(Baz) i = baz(i);
version(Bar) i = bar(i); //32 options???

i = abc(i); // a different abc is implemented for each version.

Post by BCS
Are these valid concerns? Am I misunderstanding what you said?

They are valid concerns, you're just used to thinking in terms of the C
preprocessor.

BCS

2007-02-07 19:48:43 UTC

Post by Walter Bright
#if __GNUC__
#include <pthreads.h>
#else
#include <windows.h>
#endif
AAAARRRRRGGGGGHHHHH!!! That, my friends, is evil.

agreed.

Post by BCS
version(RowMajor)
x = table[i][j];
else // RowMinor
x = table[j][i];

int GetRow(i,j) { return table[i][j]; }
The function gets inlined.

The equivalent of this doesn't

version(RowMajor)
{
x = 0
for(i=0;i<max;i++)
x+=table[i][j];
}
else // RowMinor
{
x = 0
for(i=0;i<max;i++)
x+=table[j][i];
}

and IIRC this doesn't ether

version(X86) // avoid overflow in midpoint calculations
{
asm
{
MOV low EAX;
ADD EAX hi;
RCR EAX 1;
MOV EAX mid;
}
}
else
{
mid = (hi-low)/2 + low;
}

Post by BCS
====You can't break across function calls
switch(i)
{
version(Baz)
if(baz) break;
else
break;
...// lots of un versioned code
}

if (globalversion.baz && baz)
break;

The point is to have all of the versioning done by the time you link,
that leaves a runtime check for version info.

Post by BCS
====lots of version combinations
version(Foo) i = foo(i);
version(Boo) i = boo(i);
version(Fig) i = fig(i);
version(Baz) i = baz(i);
version(Bar) i = bar(i); //32 options???

i = abc(i); // a different abc is implemented for each version.

All 32 possibilities??? What if there are 16 independent versions?
that's 64K functions! And no that is not an unlikely case, say "i" is a
parse tree and we want to add different types of annotation depending on
what features are enabled.

Post by BCS
Are these valid concerns? Am I misunderstanding what you said?

They are valid concerns, you're just used to thinking in terms of the C
preprocessor.

I have barely ever used CPP for that type of thing so I wasn't ever used
to thinking that way in the first place.
<g>

BCS

2007-02-07 18:32:52 UTC

Post by Hasan Aljudy
Why is that evil? I think it's actually a great idea. "versions" are a
sort of configuration that determines which code should be compiled and
which code shouldn't. Storing this configuration in a separate file
makes sense to me.

rename versions.txt to versions.d

and use

import versions;

same effect, less confusion

Walter Bright

2007-02-07 19:16:49 UTC

rename versions.txt to versions.d
and use
import versions;
same effect, less confusion

No, versions defined in an import do NOT affect the importer.

Frits van Bommel

2007-02-07 19:30:22 UTC

rename versions.txt to versions.d
and use
import versions;
same effect, less confusion

It doesn't work like that:
-----
urxae at urxae:~/tmp$ cat test.d
import std.stdio;

import test2;

void main() {
version(Foo)
writefln("version=Foo");
else
writefln("Not version=Foo");
}
urxae at urxae:~/tmp$ cat test2.d
version=Foo;
urxae at urxae:~/tmp$ dmd test.d test2.d -oftest && ./test
gcc test.o test2.o -o test -m32 -lphobos -lpthread -lm -Xlinker
-L/home/urxae/opt/dmd/lib
Not version=Foo
-----

Version specifications ("version=X" lines) don't get imported.

Yauheni Akhotnikau

2007-02-07 09:39:59 UTC

On Wed, 07 Feb 2007 06:20:51 +0300, Walter Bright

Post by Walter Bright
1) import data for a string constant
2) import code that's in DSL (Domain Specific Language), not D, form.

Can you provide some examples of DSL the new D feature is intended for?

For example, what if I want to implement something like to RubyOnRails
ActiveRecord DSL
(http://api.rubyonrails.org/files/vendor/rails/activerecord/README.html)?
This is an ordinal Ruby code:

class Account < ActiveRecord::Base
validates_presence_of :subdomain, :name, :email_address, :password
validates_uniqueness_of :subdomain
validates_acceptance_of :terms_of_service, :on => :create
validates_confirmation_of :password, :email_address, :on => :create
end

I may wish to translate it in the following D fragment (with an exception
that in D I must explicitly describe all table fields):

mixin( DActiveRecord!(
`class Account
field subdomain varchar(20)
field name varchar(100)
field email_address varchar(255)
field password varchar(32)
field term_of_service int
validates_presence_of subdomain, name, email_address, password
validates_uniqueness_of subdomain
validates_acceptance_of terms_of_service, on => create
validates_confirmation_of password, email_address, on => create
end` ) );

The template DActiveRecord must parse DSL string at compile time and
produce another string with Account class implementation in D. With all
necessary symantic analysis and constraints (for example, it is impossible
to use name of field in 'validate_presence_of' if it isn't described as
'field').

Do you think this task can be done with D templates at complile time?

--
Regards,
Yauheni Akhotnikau

Kyle Furlong

2007-02-07 10:17:48 UTC

Post by Yauheni Akhotnikau
On Wed, 07 Feb 2007 06:20:51 +0300, Walter Bright

Post by Walter Bright
1) import data for a string constant
2) import code that's in DSL (Domain Specific Language), not D, form.

Can you provide some examples of DSL the new D feature is intended for?
For example, what if I want to implement something like to RubyOnRails
ActiveRecord DSL
(http://api.rubyonrails.org/files/vendor/rails/activerecord/README.html)?
class Account < ActiveRecord::Base
validates_presence_of :subdomain, :name, :email_address, :password
validates_uniqueness_of :subdomain
validates_acceptance_of :terms_of_service, :on => :create
validates_confirmation_of :password, :email_address, :on => :create
end
I may wish to translate it in the following D fragment (with an
mixin( DActiveRecord!(
`class Account
field subdomain varchar(20)
field name varchar(100)
field email_address varchar(255)
field password varchar(32)
field term_of_service int
validates_presence_of subdomain, name, email_address, password
validates_uniqueness_of subdomain
validates_acceptance_of terms_of_service, on => create
validates_confirmation_of password, email_address, on => create
end` ) );
The template DActiveRecord must parse DSL string at compile time and
produce another string with Account class implementation in D. With all
necessary symantic analysis and constraints (for example, it is
impossible to use name of field in 'validate_presence_of' if it isn't
described as 'field').
Do you think this task can be done with D templates at complile time?
--Regards,
Yauheni Akhotnikau

Almost certainly.

Yauheni Akhotnikau

2007-02-07 12:05:09 UTC

On Wed, 07 Feb 2007 13:17:48 +0300, Kyle Furlong <kylefurlong at gmail.com>

Post by Kyle Furlong

Post by Yauheni Akhotnikau
The template DActiveRecord must parse DSL string at compile time and
produce another string with Account class implementation in D. With all
necessary symantic analysis and constraints (for example, it is
impossible to use name of field in 'validate_presence_of' if it isn't
described as 'field').
Do you think this task can be done with D templates at complile time?

Almost certainly.

:)
I know, but at which price? ;)

--
Regards,
Yauheni Akhotnikau

Walter Bright

2007-02-07 19:18:28 UTC

Post by Yauheni Akhotnikau
Do you think this task can be done with D templates at complile time?

Yes, that's exactly the intent. If this can't be made to work, we'll fix
D so it can.

Serg Kovrov

2007-02-07 06:11:14 UTC

Post by Ary Manzana
But... I'm wondering which are the evil uses of it. For me it's now
almost impossible not to program with an IDE (big projects, I mean). At
least in Java. Maybe compile time stuff will make it such that an IDE
won't be needed anymore. But it's very hard for me to see that happening.

I believe by 'evil use', Walter meant evil use of mixins, not IDE's.
Isn't he?

--
serg.

Walter Bright

2007-02-07 07:17:40 UTC

Post by Serg Kovrov

Post by Ary Manzana
But... I'm wondering which are the evil uses of it. For me it's now
almost impossible not to program with an IDE (big projects, I mean).
At least in Java. Maybe compile time stuff will make it such that an
IDE won't be needed anymore. But it's very hard for me to see that
happening.

I believe by 'evil use', Walter meant evil use of mixins, not IDE's.
Isn't he?

Yes. IDEs aren't evil.

BCS

2007-02-06 20:50:56 UTC

Reply to Ary,

Cool thought:

build a compiler / IDE where the would front end short of code-gen is ued
to read in code and the editor actually edits the parsed code tree. Say goodbye
to syntax errors, maybe even semantic ones as well because you can't edit
code to something that isn't correct... Er, maybe that wouldn't be such a
good idea. Anyway, what is on the screen is actually a rendering of the parsed
stuff. Talk about fast compile times:

codetree.CodeGen(); // no parsing etc.

Also the same functions that do name scopeing can do auto-compleat (or the
other way around).

Vladimir Panteleev

2007-02-07 06:30:31 UTC

Post by Walter Bright
http://www.digitalmars.com/d/changelog.html

Hmm. What would prevent someone from writing programs like:
writef(import("/etc/passwd"));
and trick someone to compile this program for them (under the pretext that they don't have a D compiler, for example) to steal the user list (or the contents of any other file with a known absolute or relative path on the victim's system)?

IMO, the compiler should at least issue a warning when importing a file not located in/under the source file's directory. Although, if the source emits a lot of pragma(msg) messages, the warning might get cluttered by those - or this might be concealed in a large program with a lot of files. A better security-wise solution is to disallow importing files outside the source file's directory, unless specified by the user on the command-line.

--
Best regards,
Vladimir mailto:thecybershadow at gmail.com

Hasan Aljudy

2007-02-07 07:11:23 UTC

Post by Vladimir Panteleev

Post by Walter Bright
http://www.digitalmars.com/d/changelog.html

writef(import("/etc/passwd"));
and trick someone to compile this program for them (under the pretext that they don't have a D compiler, for example) to steal the user list (or the contents of any other file with a known absolute or relative path on the victim's system)?
IMO, the compiler should at least issue a warning when importing a file not located in/under the source file's directory. Although, if the source emits a lot of pragma(msg) messages, the warning might get cluttered by those - or this might be concealed in a large program with a lot of files. A better security-wise solution is to disallow importing files outside the source file's directory, unless specified by the user on the command-line.

Well, theoretically nothing prevents someone from writing a virus in C++
and trick someone to compile and run it.

Andy Knowles

2007-02-07 08:37:19 UTC

"Hasan Aljudy" <hasan.aljudy at gmail.com> wrote in message

Post by Hasan Aljudy

On Tue, 06 Feb 2007 06:54:18 +0200, Walter Bright

Post by Walter Bright
http://www.digitalmars.com/d/changelog.html

writef(import("/etc/passwd"));
and trick someone to compile this program for them (under the pretext
that they don't have a D compiler, for example) to steal the user list
(or the contents of any other file with a known absolute or relative path
on the victim's system)?
IMO, the compiler should at least issue a warning when importing a file
not located in/under the source file's directory. Although, if the source
emits a lot of pragma(msg) messages, the warning might get cluttered by
those - or this might be concealed in a large program with a lot of
files. A better security-wise solution is to disallow importing files
outside the source file's directory, unless specified by the user on the
command-line.

Well, theoretically nothing prevents someone from writing a virus in C++
and trick someone to compile and run it.

But you don't need them to run Vladimir's example, just compile it. They
send you back the compiled program ("Thanks for compiling it for me!") and
you run it. The passwords are imbedded in the binary. C++ can't do this
quite as easily.

Andrei Alexandrescu (See Website For Email)

2007-02-07 07:51:17 UTC

Post by Vladimir Panteleev

Post by Walter Bright
http://www.digitalmars.com/d/changelog.html

How would the bad person see the output of the compilation?

Andrei

Vladimir Panteleev

2007-02-07 09:07:19 UTC

Post by Andrei Alexandrescu (See Website For Email)

Post by Vladimir Panteleev

Post by Walter Bright
http://www.digitalmars.com/d/changelog.html

How would the bad person see the output of the compilation?

In this particular example, the idea is to trick someone to compile a program for you and send you back the binary, under a pretext similar to "I don't have a D compiler" or "I can't/don't want to install the D compiler on my system". The fact that a compiler can embed random files in the resulting binary from his filesystem isn't obvious to a person familiar with compilers in general and not expecting similar behavior from a tool which is supposed to work with just a given set of source files.

--
Best regards,
Vladimir mailto:thecybershadow at gmail.com

Henning Hasemann

2007-02-07 09:46:20 UTC

On Wed, 07 Feb 2007 08:30:31 +0200

Post by Vladimir Panteleev
writef(import("/etc/passwd"));
and trick someone to compile this program for them (under the pretext that they don't have a D compiler, for example) to steal the user list (or the contents of any other file with a known absolute or relative path on the victim's system)?
IMO, the compiler should at least issue a warning when importing a file not located in/under the source file's directory. Although, if the source emits a lot of pragma(msg) messages, the warning might get cluttered by those - or this might be concealed in a large program with a lot of files. A better security-wise solution is to disallow importing files outside the source file's directory, unless specified by the user on the command-line.

I whould even go one step further and limit import() to just import files in the -I path.
That whould have a few implications:

- Except you have weird import paths, import() can not lead to include any
given "evil" file, it should then be as secure or insecure as C's #include
- One whould not be able to include any given file on the fs, but I think
that shouldnt be a problem, since most of the time the dsl files should lie
somewhere around in the source-tree.
- You whould use import() much like import, which at least sounds more consistent.
For example:
import("foo.d"); whould include the same file as
import foo; whould import.
Note that the semantic is still different and that import needs the file extension (as
it might often be used to include non-d files) where import does not.
Alternative: Allow import() just to include files with a predifined extension, then
one could use the package.subpackage.module syntax as well.

Questions that arise to me while writing this:
* since import() does not the same thing as import ...; shouldnt it be renamed to something else?
(say include)

* if one whould restrict import() to import-paths whould it be senseful to allow subdirectories
to be specified?
(say import("files_written_in_my_dsl/foo.dsl"))
Note here that this whould still not allow directories "above" the import paths.

Henning

Jarrett Billingsley

2007-02-07 14:12:19 UTC

"Walter Bright" <newshound at digitalmars.com> wrote in message

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

I am amazed at the mixin/import features. The ability we have to
arbitrarily generate code at runtime is.. something I never would have
imagined would come until D2.0.

I've been thinking about it, though, and I'm not sure if it's 100% the best
way to do it. The major advantage, of course, is that it doesn't introduce
tons of new syntax for building up code. However, I'm not sure I really
like the idea of building up strings either. It seems.. preprocessor-y.

Don't get me wrong, I'm still giddy with the prospects of this new feature.
But something about it just seems off.

Jarrett Billingsley

2007-02-07 14:24:04 UTC

"Jarrett Billingsley" <kb3ctd2 at yahoo.com> wrote in message
news:eqcmo3$2u17$1 at digitaldaemon.com...

Just wanted to add that something I find severely lacking is that there's no
way to get a pretty (usable, non-mangled) string representation of a type.
Instead we have to write large template libraries to accomplish something
that the compiler does _all the time_. This means it's currently not
possible to do something like

template MakeVariable(Type, char[] name)
{
const char[] MakeVariable = Type.nameof ~ " " ~ name ~ ";";
}

mixin(MakeVariable!(int, "x"));

Instead we have to use a template library to demangle the name:

import ddl.meta.nameof;

template MakeVariable(Type, char[] name)
{
const char[] MakeVariable = prettytypeof!(Type) ~ " " ~ name ~ ";";
}

mixin(MakeVariable!(int, "x"));

:\

Andrei Alexandrescu (See Website For Email)

2007-02-07 18:03:34 UTC

Post by Jarrett Billingsley
"Walter Bright" <newshound at digitalmars.com> wrote in message

Post by Walter Bright
Fixes many bugs, some serious.
Some new goodies.
http://www.digitalmars.com/d/changelog.html
http://ftp.digitalmars.com/dmd.1.005.zip

I am amazed at the mixin/import features. The ability we have to
arbitrarily generate code at runtime is.. something I never would have
imagined would come until D2.0.

At compile time you mean.

Post by Jarrett Billingsley
I've been thinking about it, though, and I'm not sure if it's 100% the best
way to do it. The major advantage, of course, is that it doesn't introduce
tons of new syntax for building up code. However, I'm not sure I really
like the idea of building up strings either. It seems.. preprocessor-y.
Don't get me wrong, I'm still giddy with the prospects of this new feature.
But something about it just seems off.

The ability to transform true code trees will come with D's macro
abilities. But that's a few months ahead at least.

Andrei

Stewart Gordon

2007-02-07 15:49:26 UTC