GlobalAlloc, HeapAlloc, VirtualAlloc, or malloc

[Jake Montgomery]

What function (GlobalAlloc, HeapAlloc,VirtualAlloc or malloc) should be
used to allocate large blocks of memory (1-100MB). There are no issues
of passing the pointes across processes or between dlls.

What are the speed implications of these different routines?
For malloc, does it matter which version of the CRT I link with?
I assume that once I have the memory, the speed of access to it will not
depend on how it is allocated?

Thanx in advace.

[Carl Daniel [VC++ MVP]]

Post by Jake Montgomery
What function (GlobalAlloc, HeapAlloc,VirtualAlloc or malloc) should
be used to allocate large blocks of memory (1-100MB). There are no
issues of passing the pointes across processes or between dlls.
What are the speed implications of these different routines?
For malloc, does it matter which version of the CRT I link with?
I assume that once I have the memory, the speed of access to it will
not depend on how it is allocated?

It makes virtually no difference (no pun intended). Under the covers, every
one of those will eventually call (the moral equivalent of) VirtualAlloc.
The difference in time to allocate a large block is probably immeasurable
(compared to the noise in the measurement) and yes, once allocated, there's
no difference in accessing the memory regardless of how it was allocated.

-cd

[Simon Cooke]

Post by Carl Daniel [VC++ MVP]

Post by Jake Montgomery
What function (GlobalAlloc, HeapAlloc,VirtualAlloc or malloc) should
be used to allocate large blocks of memory (1-100MB). There are no
issues of passing the pointes across processes or between dlls.
What are the speed implications of these different routines?
For malloc, does it matter which version of the CRT I link with?
I assume that once I have the memory, the speed of access to it will
not depend on how it is allocated?

It makes virtually no difference (no pun intended). Under the covers, every
one of those will eventually call (the moral equivalent of) VirtualAlloc.
The difference in time to allocate a large block is probably immeasurable
(compared to the noise in the measurement) and yes, once allocated, there's
no difference in accessing the memory regardless of how it was allocated.
-cd

Please... it's the "equivalent" of. Not "moral equivalent". There is no
morality judgement involved in the choice of which API function you use.

[Igor Tandetnik]

Post by Simon Cooke
Please... it's the "equivalent" of. Not "moral equivalent". There is
no morality judgement involved in the choice of which API function
you use.

In the expression "moral equivalent", "moral" does not mean "ethical" or
"virtuous", but rather "supported by reason or probability; practically
sufficient" <http://dictionary.reference.com/search?q=moral>

[Simon Cooke]

Post by Igor Tandetnik

Post by Simon Cooke
Please... it's the "equivalent" of. Not "moral equivalent". There is
no morality judgement involved in the choice of which API function
you use.

In the expression "moral equivalent", "moral" does not mean "ethical" or
"virtuous", but rather "supported by reason or probability; practically
sufficient" <http://dictionary.reference.com/search?q=moral>

How long did you have to search before you found that particular definition
of moral?

Please don't try to twist words around. The word moral implies a judgement
call of right vs. wrong in an ethics / behavioral sense.

Please also look around on usenet and the web, and see how that phrase is
commonly used and what it means. It is NOT used to mean "supported by
reason or probability". It is used to mean good vs. evil.

[William DePalo [MVP VC++]]

Post by Simon Cooke
How long did you have to search before you found that particular definition
of moral?

Less than 30 seconds for me.

Post by Simon Cooke
Please don't try to twist words around. The word moral implies a judgement
call of right vs. wrong in an ethics / behavioral sense.

<LOL> You must be kidding, right? Carl did not coin that usage of "moral
equivalent".

It would be easy for a reasonable person to conclude that it is you who is
trying to "twist" words for some pedantic purpose or other.

Post by Simon Cooke
Please also look around on usenet and the web, and see how that phrase is
commonly used and what it means. It is NOT used to mean "supported by
reason or probability". It is used to mean good vs. evil.

The reason why dictionaries go for pages and pages is precisely because of
the complex ways in which words are used. Carl's usage of the phrase is but
one example.

Regards,
Will

P.S. You don't post here under another alias, do you?

[Simon Cooke]

Post by Simon Cooke

Post by Simon Cooke
How long did you have to search before you found that particular

definition

Post by Simon Cooke
of moral?

Less than 30 seconds for me.

Post by Simon Cooke
Please don't try to twist words around. The word moral implies a judgement
call of right vs. wrong in an ethics / behavioral sense.

<LOL> You must be kidding, right? Carl did not coin that usage of "moral
equivalent".
It would be easy for a reasonable person to conclude that it is you who is
trying to "twist" words for some pedantic purpose or other.

Let me guess... you also see nothing wrong with the use of words such as
"irregardless". You probably also make "mute points". But I digress.

Answer me this:

If you believe that the phrase "moral equivalent" should be used when
describing software, please explain why the word "equivalent" is not
sufficient by itself.

Equivalent means: "corresponding or virtually identical especially in
effect or function"

What does the word "Moral" added to that description convey?

Post by Simon Cooke

Post by Simon Cooke
Please also look around on usenet and the web, and see how that phrase is
commonly used and what it means. It is NOT used to mean "supported by
reason or probability". It is used to mean good vs. evil.

The reason why dictionaries go for pages and pages is precisely because of
the complex ways in which words are used. Carl's usage of the phrase is but
one example.

And it's an invalid one.

Post by Simon Cooke
Regards,
Will
P.S. You don't post here under another alias, do you?

No - why, do you?

[William DePalo [MVP VC++]]

Post by Simon Cooke
Let me guess... you also see nothing wrong with the use of words such as
"irregardless". You probably also make "mute points". But I digress.

Still laughing.

Post by Simon Cooke
If you believe that the phrase "moral equivalent" should be used when
describing software, please explain why the word "equivalent" is not
sufficient by itself.

Rolling on the floor now: from reading what part of my reply did you infer
that I meant "should".

Post by Simon Cooke
What does the word "Moral" added to that description convey?

Here's a clue for you: Get a dictionary and look up the words "idiomatic"
and "expression".

Post by Simon Cooke
No - why, do you?

No never. I always use my name. But reading this I thought it might have
been written by Bonj.

Regards,
Will

[Simon Cooke]

Post by William DePalo [MVP VC++]

Post by Simon Cooke
Let me guess... you also see nothing wrong with the use of words such as
"irregardless". You probably also make "mute points". But I digress.

Still laughing.

Post by Simon Cooke
If you believe that the phrase "moral equivalent" should be used when
describing software, please explain why the word "equivalent" is not
sufficient by itself.

Rolling on the floor now: from reading what part of my reply did you infer
that I meant "should".

You certainly believe it to be valid. I do note that you still have not
explained why describing something as "equivalent" to something else is not
sufficient in and of itself.

So I'll ask again:

What is the difference between the phrase "moral equivalent" and
"equivalent"?

Post by William DePalo [MVP VC++]

Post by Simon Cooke
What does the word "Moral" added to that description convey?

Here's a clue for you: Get a dictionary and look up the words "idiomatic"
and "expression".

Here's another clue for you: look up the words "idiotic" and "phrase".

Plenty of people describe things as being "mute" when they mean "moot". Do
you believe them to be correct also?

Post by William DePalo [MVP VC++]

Post by Simon Cooke
No - why, do you?

No never. I always use my name. But reading this I thought it might have
been written by Bonj.

No idea who Bonj is, but no.

[William DePalo [MVP VC++]]

Post by Simon Cooke
You certainly believe it to be valid.

Exactly right. The dictionary lists it in a definition as a valid usage of
the phrase.

Simon Cooke says it is not valid.

I'll take the dictionary, thanks. :-)

Regards,
Will

[Simon Cooke]

Post by William DePalo [MVP VC++]

Post by Simon Cooke
You certainly believe it to be valid.

Exactly right. The dictionary lists it in a definition as a valid usage of
the phrase.
Simon Cooke says it is not valid.
I'll take the dictionary, thanks. :-)

The dictionary does not list the phrase, nor does it specify its usage.

So I'll ask again:

What is the difference between the phrase "moral equivalent" and
"equivalent"?

[Igor Tandetnik]

Post by Simon Cooke
What is the difference between the phrase "moral equivalent" and
"equivalent"?

Roughly, "equivalent" means "identical in every way", or at least "very
similar to the point of being practically indistinguishable". "Moral
equivalent" means "somewhat similar", "having the same general idea".

In other words, "A is equivalent to B" is a much stronger statement than
"A is a moral equivalent of B". Neither has anything to do with morals
or ethics.

[Igor Tandetnik]

Post by Simon Cooke
Please also look around on usenet and the web, and see how that
phrase is commonly used and what it means. It is NOT used to mean
"supported by reason or probability". It is used to mean good vs.
evil.

http://research.compaq.com/SRC/m3sources/html/ui/src/winvbt/WinContext.i3.html
Modify hdc to be suitable for tint painting. This procedure is the moral
equivalent of XGC.ResolveTintGC in xvbt.

http://dev.zope.org/Wikis/DevSite/Proposals/CleanSignalHandling
Add signal handling to ZServer?, to permit more uniform control of the
server from the command line. In particular:
- Trap the signal.SIGTERM signal, and call the moral equivalent of
manage_shutdown.
- Trap the signal.SIGHUP signal, and call the moral equivalent of
manage_restart.

http://lists.xml.org/archives/xml-dev/200305/msg00917.html
|> Yup. It can be as simple as that. The XSLT case is actually easier
than
|> the general problem, because there are no overlaps. All XSLT needs
is the
|> moral equivalent of Lisp's quote/backquote operator.

http://mail.python.org/pipermail/python-dev/2002-August/028003.html

Post by Simon Cooke
the question was whether
adding a method to a gf
is always the moral equivalent of

[code fragment snipped]

http://info.astrian.net/jargon/terms/r/reinvent_the_wheel.html
reinvent the wheel v. To design or implement a tool equivalent
to an existing one or part of one, with the implication that doing so is
silly or a waste of time. This is often a valid criticism. On the other
hand, automobiles don't use wooden rollers, and some kinds of wheel have
to be reinvented many times before you get them right. On the third
hand, people reinventing the wheel do tend to come up with the moral
equivalent of a trapezoid with an offset axle.

Looks like you don't know your native language as well as I know my
second one (assuming English is your native language - I apologize if it
is not so).

[Simon Cooke]

Post by Igor Tandetnik
Looks like you don't know your native language as well as I know my
second one (assuming English is your native language - I apologize if it
is not so).

http://www.google.com/search?q=%22moral+equivalent%22&hl=en&lr=&ie=UTF-8&oe=UTF-8&start=10&sa=N

Funny... your links don't appear in this search. At least nowhere near the
top.

[Igor Tandetnik]

Post by Simon Cooke

Post by Igor Tandetnik
Looks like you don't know your native language as well as I know my
second one (assuming English is your native language - I apologize
if it is not so).

http://www.google.com/search?q=%22moral+equivalent%22&hl=en&lr=&ie=UTF-8&oe=UTF-8&start=10&sa=N

Post by Simon Cooke
Funny... your links don't appear in this search. At least nowhere
near the top.

Try

http://groups.google.com/groups?q=moral+equivalent&group=comp.*

"Moral equivalent" does have moral and ethical connotations when
discussing politics and such, but none when used in technical
discussions - like the one that was occuring in this thread before you
joined.

[Simon Cooke]

Post by Simon Cooke

Post by Simon Cooke

Post by Igor Tandetnik
Looks like you don't know your native language as well as I know my
second one (assuming English is your native language - I apologize
if it is not so).

http://www.google.com/search?q=%22moral+equivalent%22&hl=en&lr=&ie=UTF-8&oe=UTF-8&start=10&sa=N

Post by Simon Cooke
Funny... your links don't appear in this search. At least nowhere
near the top.

Try
http://groups.google.com/groups?q=moral+equivalent&group=comp.*
"Moral equivalent" does have moral and ethical connotations when
discussing politics and such, but none when used in technical
discussions - like the one that was occuring in this thread before you
joined.

No, it has no place at all in a tehcnical discussion. Those that use it as
such are ignorant.

BTW: Your logic is faulty. You just created a list of all the times people
misused it in a technical setting - not its common usage.

[Igor Tandetnik]

Post by Simon Cooke

Post by Igor Tandetnik

Post by Simon Cooke
Funny... your links don't appear in this search. At least nowhere
near the top.

Try
http://groups.google.com/groups?q=moral+equivalent&group=comp.*
"Moral equivalent" does have moral and ethical connotations when
discussing politics and such, but none when used in technical
discussions - like the one that was occuring in this thread before
you joined.

No, it has no place at all in a tehcnical discussion. Those that use
it as such are ignorant.
BTW: Your logic is faulty. You just created a list of all the times
people misused it in a technical setting - not its common usage.

Well, that search on Google Groups returns 5,210 hits. At which point
would you classify an expression as "generally accepted" rather than
"misused"? And, in case you have not noticed, we are in a newsgroup
dedicated to a programming language - that would be "technical setting"
in my book. You asked me to

Post by Simon Cooke
Please also look around on usenet and the web, and see how that phrase is
commonly used and what it means

That's what I did. Too bad the outcome of this experiment does not
support your theory.

As another example, the terms "master" and "slave" have strong ethical
connotations when discussing politics, but are widely accepted technical
terms when discussing hard drives (do you live in L.A.. by any chance?
See [1]). "R*pe" (intentionally mangled to avoid possible spam filters)
has one meaning when discussing crime, but another when talking about
agriculture (in case you don't know, it's a plant from which rapeseed
aka Canola oil is produced).

[1] http://slashdot.org/article.pl?sid=03/11/25/0014257

[Simon Cooke]

Post by Igor Tandetnik

Post by Simon Cooke

Post by Igor Tandetnik

Post by Simon Cooke
Funny... your links don't appear in this search. At least nowhere
near the top.

Try
http://groups.google.com/groups?q=moral+equivalent&group=comp.*
"Moral equivalent" does have moral and ethical connotations when
discussing politics and such, but none when used in technical
discussions - like the one that was occuring in this thread before
you joined.

No, it has no place at all in a tehcnical discussion. Those that use
it as such are ignorant.
BTW: Your logic is faulty. You just created a list of all the times
people misused it in a technical setting - not its common usage.

Well, that search on Google Groups returns 5,210 hits. At which point
would you classify an expression as "generally accepted" rather than
"misused"?

The phrase "mute point" returns 14000 hits. It's still wrong. The correct
phrase is "moot point".

But hey, if you want to sound like a fool, go ahead and have your morally
equivalent function implementations. I'm not interested in debating this
any more.

[William DePalo [MVP VC++]]

Post by Simon Cooke
But hey, if you want to sound like a fool, go ahead and have your morally
equivalent function implementations. I'm not interested in debating this
any more.

<ROFL>

In reading this thread from start to finish, I can't help but think of a
famous quip of Abraham Lincoln (16th US president):

"It is better to remain silent an be thought a fool than to speak and remove
all doubt".

Regards,
Will

[Simon Cooke]

Post by William DePalo [MVP VC++]

Post by Simon Cooke
But hey, if you want to sound like a fool, go ahead and have your morally
equivalent function implementations. I'm not interested in debating this
any more.

<ROFL>
In reading this thread from start to finish, I can't help but think of a
"It is better to remain silent an be thought a fool than to speak and remove
all doubt".

You're welcome to your opinions about me. But on the subject of whether or
not "moral equivalent" is the right phrase to use when comparing software
implementations, you're dead wrong.

As I said, I'm not interested in debating this any more. Good day to you.

[Doug Harrison [MVP]]

Post by Jake Montgomery
What function (GlobalAlloc, HeapAlloc,VirtualAlloc or malloc) should be
used to allocate large blocks of memory (1-100MB). There are no issues
of passing the pointes across processes or between dlls.

GlobalAlloc - Use only when required, such as when dealing with certain
clipboard formats.

HeapAlloc - IIRC, it's documented to be a poor choice on Win9X for blocks >
4 MB. I've not found much reason to use this API.

VirtualAlloc - This one allows you to decommit memory you've freed, which
shrinks your process's address space and pagefile usage, which can be
desirable when talking about very large blocks. It also provides a
guaranteed and efficient way to start a buffer small and grow it
incrementally in place; as long as you don't need to exceed the maximum size
you designated when you reserved the memory, you can commit pages within the
reserved area without causing any reallocation, thus avoiding expensive
copying. It's also the basis for dynamically committing memory upon first
use when using SEH. On the downside, the granularity of allocations is
rather high, being the page size of the machine.

malloc - This is what a C program should generally use. It's portable and is
sufficient for most needs.

Post by Jake Montgomery
What are the speed implications of these different routines?

For allocating a single large block of memory, they're probably all about
the same, modulo things like the Win9X caveat concerning HeapAlloc.

Post by Jake Montgomery
For malloc, does it matter which version of the CRT I link with?

The multithreaded CRT guards the heap with a single critical section, which
can be a bottleneck for some applications whose threads make frequent
allocations and end up contending for the heap a lot of the time.

Post by Jake Montgomery
I assume that once I have the memory, the speed of access to it will not
depend on how it is allocated?

Correct.

See this MSDN topic for a brief comparison of allocation methods and links
to further documentation:

Comparing Memory Allocation Methods
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/comparing_memory_allocation_methods.asp?frame=true

[Alexander Grigoriev]

Even worse, granularity is 64 kbytes. This is how VirtualAlloc'ated blocks
are aligned in x86 systems.

Post by Doug Harrison [MVP]
VirtualAlloc - This one allows you to decommit memory you've freed, which
use when using SEH. On the downside, the granularity of allocations is
rather high, being the page size of the machine.

[Doug Harrison [MVP]]

Post by Alexander Grigoriev
Even worse, granularity is 64 kbytes. This is how VirtualAlloc'ated blocks
are aligned in x86 systems.

I think it would be correct to say the granularity for memory reservation is
64 KB, for commitment within the reserved area, 4 KB (the page size on x86).

[Simon Cooke]

Post by Jake Montgomery
What function (GlobalAlloc, HeapAlloc,VirtualAlloc or malloc) should be
used to allocate large blocks of memory (1-100MB). There are no issues
of passing the pointes across processes or between dlls.

VirtualAlloc should be used for large allocations of memory (eg. > 4Mb).
HeapAlloc/GlobalAlloc are pretty much identical, but today you should use
HeapAlloc and not use GlobalAlloc unless you need to use DDE, the clipboard
or OLE Data.

In this case, use VirtualAlloc.

BTW: If you did need to pass pointers across processes, you would use a
memory mapped file.

[William DePalo [MVP VC++]]

Post by Simon Cooke
BTW: If you did need to pass pointers across processes, you would use a
memory mapped file.

Just to be clear:

1) The pointers would have to be in the shared segment
2) The things that they point to would have to be in
the shared segment
3) The shared segment would have to be mapped at
the same virtual address in both processes.

Point #3 is by no means guaranteed in the general case.

Regards,
Will

[Carl Daniel [VC++ MVP]]

Post by William DePalo [MVP VC++]
Point #3 is by no means guaranteed in the general case.

Unless you're running on Win9x, when it's always guaranteed. Except on
Thursdays...

-cd

[William DePalo [MVP VC++]]

Post by Carl Daniel [VC++ MVP]

Post by William DePalo [MVP VC++]
Point #3 is by no means guaranteed in the general case.

Unless you're running on Win9x, when it's always guaranteed.

Zactly. :-)

Regards,
Will

[Repstat]

What's wrong with the good trusty old 'new'?

[Jake Montgomery]

Post by Repstat
What's wrong with the good trusty old 'new'?

That was my question (OP) ... although, since I wanted a block of
memory, not of any specific type, i choose malloc. (new uses malloc in
the MS CRT.) My question was, is there anything wrong with it, or is
one of the MS specific allocators better?

Thanx,
Jake.

[r***@gmail.com]

Please refer this link for better understanding....

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/comparing_memory_allocation_methods.asp?frame=true

Post by Jake Montgomery
What function (GlobalAlloc, HeapAlloc,VirtualAlloc or malloc) should be
used to allocate large blocks of memory (1-100MB). There are no issues
of passing the pointes across processes or between dlls.
What are the speed implications of these different routines?
For malloc, does it matter which version of the CRT I link with?
I assume that once I have the memory, the speed of access to it will not
depend on how it is allocated?
Thanx in advace.

[Simon Trew]

Post by Jake Montgomery
What function (GlobalAlloc, HeapAlloc,VirtualAlloc or malloc) should be
used to allocate large blocks of memory (1-100MB). There are no issues
of passing the pointes across processes or between dlls.
What are the speed implications of these different routines?
For malloc, does it matter which version of the CRT I link with?
I assume that once I have the memory, the speed of access to it will not
depend on how it is allocated?
Thanx in advace.

I have read the answers above. I know I am not an MVP and my opinion counts for nothing. But I would add one thing: measure measure measure. Yes, HeapAlloc and VirtualAlloc and so on will boil down to the same thing. But they all have a memory fence on them (I have tried to find out if there is a fence at the kernel level, but I am not sure and I don't know who knows, Raymond Chen describes it as "looking at the world through kernerl coloured glasses". There is and always has been a second template paramter to the container classes, an allocator. Use it. Put a test allocator in, not difficult to scrub up a trivial allocator that you malloc explicitly, then you can measure that and see how many calls there are to the allocator. (A lot more than you suspect with that code excerpt, it comes down to when C++ is allowed to elide a copy, and although move semantics make it a lot easier to elide it now, there are still implicit copies going on that under the rules the compiler cannot elide).

I think Stephen Leacock in his essay "How to borrow money" put it quite well: If you want to borrow money, borrow a whole lot. And I know that was written about a hundred years ago but it now pretty much applies to memory. Memory is cheap and programmers are expensive--- but moreso now, memory fences are very expensive so we don't want to go jumping over them more often than we have to, and any kind of allocation will be a memory fence, a processor stop, and all your threads stop, to ensure referential integrity. That is true even with the C++11 memory model. In fact it is almost always true with that model, std::atomic is rather a back door to make an exception, since it is assumed (essentially for legacy code to work) that you have to have the traditional guarantee of sequence, which of course is not true any more, but we have to pretend it is unless we say is is atomic. But since there is only one bunch of memory (whether you split it into pools or however you fancy dividing it up) and only one hard disk and only one network connection (yes there can be more than one, I am just making a point) at the end you come down to contention for a limited resource. And the way to alleviate that is to try to avoid the contention. On MSVC98, it is much quicker to allocate two 2Mb arrays of floats as a 4Mb chunk and a pointer to half way through, than give them each their own allocation of 2Mb. I don't have to argue that cos I have the test figures.

Don't take too many trips to the well, take one with a big bucket. It's only virtual memory anyway, so it won't be physically allocated until you need it.