Discussion:
[Catalyst] Catalyst vs Rails vs Django Cook off
Octavian Rasnita
2007-01-14 21:26:50 UTC
Permalink
Hi,

Here is ablog I just found. Is it true that Catalyst is so slow comparing
with other frameworks?
http://letsgetdugg.com/category/rails

Octavian
Brian Kirkbride
2007-01-14 21:56:10 UTC
Permalink
Post by Octavian Rasnita
Hi,
Here is ablog I just found. Is it true that Catalyst is so slow
comparing with other frameworks?
http://letsgetdugg.com/category/rails
Octavian
See the exhaustive thread in which this was discussed earlier:

http://www.mail-archive.com/***@lists.rawmode.org/msg02329.html

Perhaps a quick archive search can answer your next question faster than I can.

Cheers,
Brian
Zbigniew Lukasiak
2007-01-15 14:12:33 UTC
Permalink
I remember the discussion here - but it seems that nobody tried to
independently verify the results. I did compare Rails and Catalyst
and on my pretty standard Debian box Catalyst was about 50% faster
than Rails.

--
Zbyszek
Post by Octavian Rasnita
Hi,
Here is ablog I just found. Is it true that Catalyst is so slow comparing
with other frameworks?
http://letsgetdugg.com/category/rails
Octavian
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Dev site: http://dev.catalyst.perl.org/
--
Zbigniew Lukasiak
http://brudnopis.blogspot.com/
Victor Igumnov
2007-01-15 14:29:17 UTC
Permalink
The main devs confirmed my results. Concerning your benchmark, I am
pretty sure you screwed up some where, ie: running web brick in
development mode.

Regardless, the benchmark was fairly simplistic to begin with which
only stressed the dispatcher.

-Victor
Post by Zbigniew Lukasiak
I remember the discussion here - but it seems that nobody tried to
independently verify the results. I did compare Rails and Catalyst
and on my pretty standard Debian box Catalyst was about 50% faster
than Rails.
--
Zbyszek
Post by Octavian Rasnita
Hi,
Here is ablog I just found. Is it true that Catalyst is so slow comparing
with other frameworks?
http://letsgetdugg.com/category/rails
Octavian
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/
Dev site: http://dev.catalyst.perl.org/
--
Zbigniew Lukasiak
http://brudnopis.blogspot.com/
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/
Dev site: http://dev.catalyst.perl.org/
Zbigniew Lukasiak
2007-01-15 14:52:24 UTC
Permalink
I don't want to start another heated debate - but if you say that
someone confirmed your results then please link to the relevant emails
(from the archive). I did follow the whole discussion but I don't
remember anyone reporting repeating the experiment, I also talked
about that on IRC and got the same information.

Best Regards,
Zbigniew Lukasiak
Post by Victor Igumnov
The main devs confirmed my results. Concerning your benchmark, I am
pretty sure you screwed up some where, ie: running web brick in
development mode.
Regardless, the benchmark was fairly simplistic to begin with which
only stressed the dispatcher.
-Victor
Post by Zbigniew Lukasiak
I remember the discussion here - but it seems that nobody tried to
independently verify the results. I did compare Rails and Catalyst
and on my pretty standard Debian box Catalyst was about 50% faster
than Rails.
--
Zbyszek
Post by Octavian Rasnita
Hi,
Here is ablog I just found. Is it true that Catalyst is so slow comparing
with other frameworks?
http://letsgetdugg.com/category/rails
Octavian
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/
Dev site: http://dev.catalyst.perl.org/
--
Zbigniew Lukasiak
http://brudnopis.blogspot.com/
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/
Dev site: http://dev.catalyst.perl.org/
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Dev site: http://dev.catalyst.perl.org/
--
Zbigniew Lukasiak
http://brudnopis.blogspot.com/
Robert 'phaylon' Sedlacek
2007-01-15 16:08:24 UTC
Permalink
Regardless, the benchmark was fairly simplistic to begin with which only
stressed the dispatcher.
Didn't you say at one point you changed it to not use the templating
systems? Because it says

Document Path: /

and

sub default : Private {
my ( $self, $c ) = @_;
$c->stash->{template}="index.tt2";
}
--
# Robert 'phaylon' Sedlacek
# Perl 5/Catalyst Developer in Hamburg, Germany
{ EMail => ' ***@474.at ', Web => ' http://474.at ' }
Zbigniew Lukasiak
2007-01-15 16:43:11 UTC
Permalink
I did try my tests once again and they do indeed use WEBrick. I'll
try to fix that. By the way - is it possible to deploy Catalyst over
lighttpd?

--
Zbyszek
Post by Victor Igumnov
The main devs confirmed my results. Concerning your benchmark, I am
pretty sure you screwed up some where, ie: running web brick in
development mode.
Regardless, the benchmark was fairly simplistic to begin with which
only stressed the dispatcher.
-Victor
Post by Zbigniew Lukasiak
I remember the discussion here - but it seems that nobody tried to
independently verify the results. I did compare Rails and Catalyst
and on my pretty standard Debian box Catalyst was about 50% faster
than Rails.
--
Zbyszek
Post by Octavian Rasnita
Hi,
Here is ablog I just found. Is it true that Catalyst is so slow comparing
with other frameworks?
http://letsgetdugg.com/category/rails
Octavian
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/
Dev site: http://dev.catalyst.perl.org/
--
Zbigniew Lukasiak
http://brudnopis.blogspot.com/
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/
Dev site: http://dev.catalyst.perl.org/
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Dev site: http://dev.catalyst.perl.org/
--
Zbigniew Lukasiak
http://brudnopis.blogspot.com/
Kieren Diment
2007-01-15 16:52:34 UTC
Permalink
Post by Zbigniew Lukasiak
I did try my tests once again and they do indeed use WEBrick. I'll
try to fix that. By the way - is it possible to deploy Catalyst over
lighttpd?
Yup:


http://search.cpan.org/~mramberg/Catalyst-Runtime-5.7006/lib/Catalyst/Engin=
e/FastCGI.pm#Lighttpd

and for fun with testing: http://catalystframework.org/calendar/7




--
Post by Zbigniew Lukasiak
Zbyszek
Post by Victor Igumnov
The main devs confirmed my results. Concerning your benchmark, I am
pretty sure you screwed up some where, ie: running web brick in
development mode.
Regardless, the benchmark was fairly simplistic to begin with which
only stressed the dispatcher.
-Victor
Post by Zbigniew Lukasiak
I remember the discussion here - but it seems that nobody tried to
independently verify the results. I did compare Rails and Catalyst
and on my pretty standard Debian box Catalyst was about 50% faster
than Rails.
--
Zbyszek
Post by Octavian Rasnita
Hi,
Here is ablog I just found. Is it true that Catalyst is so slow comparing
with other frameworks?
http://letsgetdugg.com/category/rails
Octavian
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/
Dev site: http://dev.catalyst.perl.org/
--
Zbigniew Lukasiak
http://brudnopis.blogspot.com/
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/
Dev site: http://dev.catalyst.perl.org/
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Dev site: http://dev.catalyst.perl.org/
--
Zbigniew Lukasiak
http://brudnopis.blogspot.com/
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Dev site: http://dev.catalyst.perl.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.scsys.co.uk/pipermail/catalyst/attachments/20070115/7d51f=
311/attachment-0001.htm
Robert 'phaylon' Sedlacek
2007-01-15 16:56:39 UTC
Permalink
Post by Zbigniew Lukasiak
I did try my tests once again and they do indeed use WEBrick. I'll
try to fix that. By the way - is it possible to deploy Catalyst over
lighttpd?
FastCGI? :)
--
# Robert 'phaylon' Sedlacek
# Perl 5/Catalyst Developer in Hamburg, Germany
{ EMail => ' ***@474.at ', Web => ' http://474.at ' }
Carl Johnstone
2007-01-15 15:18:35 UTC
Permalink
Post by Octavian Rasnita
Is it true that Catalyst is so slow comparing
with other frameworks?
Does it matter?

If speed is so important, you should write your own custom httpd that does
exactly what you need in assembly language.

Carl
Hermida, Leandro
2007-01-15 15:35:28 UTC
Permalink
Sent: Monday, January 15, 2007 10:21
Subject: Re: [Catalyst] Catalyst vs Rails vs Django Cook off
Is it true that Catalyst is so slow comparing with other frameworks?
Does it matter?
If speed is so important, you should write your own custom
httpd that does exactly what you need in assembly language.
Carl
Speed does matter and I believe the original thread question is a valid
one. Not everyone has the time or the know-how to do wheel reinvention
and write custom daemons (I know I don't). That's why people write
kernels and libraries and abstraction of lower level things so that
others can build things on top. Otherwise we would never get anything
done.
Carl Johnstone
2007-01-15 18:07:41 UTC
Permalink
Leandro Hermida >>>>>
Speed does matter and I believe the original thread question is a valid
one. Not everyone has the time or the know-how to do wheel reinvention
and write custom daemons (I know I don't). That's why people write
kernels and libraries and abstraction of lower level things so that
others can build things on top. Otherwise we would never get anything
done.
<<<<< <<<<<

So here you're saying that getting things done is more important than pure
speed?

So surely you pick the framework that most helps you get things done rather
than the one that works fastest?

Carl
A. Pagaltzis
2007-01-15 18:19:34 UTC
Permalink
Post by Carl Johnstone
So surely you pick the framework that most helps you get things
done rather than the one that works fastest?
Yes and no. Depends on what you?re doing. But in the case of
Catalyst, you?ll probably get much more speed out of switching
to another templating engine and/or ORM than out of switching
the framework, whose total performance impact is negligible. So
I don?t know what the point of benchmarking frameworks against
each other is, particularly for such an unrepresentative case.

Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>
Jonathan Rockway
2007-01-16 00:10:44 UTC
Permalink
Post by A. Pagaltzis
I don?t know what the point of benchmarking frameworks against
each other is, particularly for such an unrepresentative case.
Ad impressions.
--
package JAPH;use Catalyst qw/-Debug/;($;=JAPH)->config(name => do {
$,.=reverse qw[Jonathan tsu rehton lre rekca Rockway][$_].[split //,
";$;"]->[$_].q; ;for 1..4;$,=~s;^.;;;$,});$;->setup;
Daniel McBrearty
2007-01-16 01:27:14 UTC
Permalink
I don't see any reason why it shouldn't be meaningful if it was done
well. Not that anyone should choose their framework on the basis of
such a benchmark, but it's a factor to throw into the mix.
Post by Jonathan Rockway
I don't know what the point of benchmarking frameworks against
each other is, particularly for such an unrepresentative case.
Ad impressions.
--
package JAPH;use Catalyst qw/-Debug/;($;=JAPH)->config(name => do {
$,.=reverse qw[Jonathan tsu rehton lre rekca Rockway][$_].[split //,
";$;"]->[$_].q; ;for 1..4;$,=~s;^.;;;$,});$;->setup;
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Dev site: http://dev.catalyst.perl.org/
--
Daniel McBrearty
email : danielmcbrearty at gmail.com
www.engoi.com : the multi - language vocab trainer
BTW : 0873928131
Christopher Hicks
2007-01-16 02:51:36 UTC
Permalink
Post by Daniel McBrearty
I don't see any reason why it shouldn't be meaningful if it was done
well. Not that anyone should choose their framework on the basis of
such a benchmark, but it's a factor to throw into the mix.
What about the benchmark for how many lines/words of code it takes to build the test monkey in each framework?
--
</chris>

The whole problem with the world is that fools and fanatics are always so
certain of themselves, and wiser people so full of doubts.
- Bertrand Russell, philosopher, mathematician, author,
Nobel laureate (1872-1970)
David Morel
2007-01-16 03:43:42 UTC
Permalink
Post by Daniel McBrearty
I don't see any reason why it shouldn't be meaningful if it was done
well. Not that anyone should choose their framework on the basis of
such a benchmark, but it's a factor to throw into the mix
Does that include dynamic content caching wizardry ? It is
meaningless if you don't take into account real-life scenarios like
reverse proxy cache invalidation policies (and tricks). This is just
to say that all this perf talk is meaningless : sometimes the power
you get from a well thought out framework allows you to do things
that are close to magick, speed-wise among others. Comparing simple
setups is ridiculous IMHO.

David Morel
Marcello Romani
2007-01-17 21:46:59 UTC
Permalink
Post by Daniel McBrearty
I don't see any reason why it shouldn't be meaningful if it was done
well. Not that anyone should choose their framework on the basis of
such a benchmark, but it's a factor to throw into the mix
Does that include dynamic content caching wizardry ? It is meaningless
if you don't take into account real-life scenarios like reverse proxy
cache invalidation policies (and tricks). This is just to say that all
this perf talk is meaningless : sometimes the power you get from a well
thought out framework allows you to do things that are close to magick,
speed-wise among others. Comparing simple setups is ridiculous IMHO.
David Morel
If a framework makes development easier because it's more elegant, easy
to use, or whatever, then you may have more time to think about setting
up a more efficient deployment architecture (i.e. the thinks mentioned
above).
Therefore it seems to me that ease of developement might be more
important to the overall app performance than the raw speed in simple
test cases.

Just my 2 (euro)cents.
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Dev site: http://dev.catalyst.perl.org/
--
Marcello Romani
Responsabile IT
Ottotecnica s.r.l.
http://www.ottotecnica.com
Jay K
2007-01-17 23:05:02 UTC
Permalink
I agree 100% on this... if we are judging Catalyst, et al, as simple
dispatchers, then we should consider apache+cgi in the discussion as
well - as apache is obviously one of the most venerable and widely
deployed dispatchers out there.

A framework is much more than that. In my experience, the costs in
time and money involved in building and maintaining the code for an
app outweigh the cost of deploying it by huge margins. Furthermore,
by the time you are experiencing enough traffic to be analyzing the
performance at this granular a level, there are much better ways to
improve your responses per second that cost much less in terms of
time and money.

These days, for less than the cost of a week of work, you can double
your serving capacity easily - and because of the 'fringe benefits'
of using Catalyst this is usually simply a matter of swapping out
session and caching plugins (if you even use them) to the more cross-
server compatible modules.

If you are using a frontend cache like squid, you can do even more
cost-for-performance-wise.

Personally, I use Catalyst because it takes care of a lot of details
I would rather not worry about. When I am free of worrying about
all those details, I can focus on building JUST my application
logic. This means that my application logic tends to be more solid,
because I am not tracking a ton of specifics outside of my app.

Because I'm not constantly crossing the line between my app and base
functionality (responding to HTTP, getting the correct bit of code
executed based on the request, etc.), I am not chasing bugs related
to that line. I can rely on the fact that it will always happen in a
particular way and if I have a bug, it's more than likely in my
application, so I can focus there.

That is the power of a Framework any 'benchmark' that doesn't take
those things into account is so much fluff and of no use to me.

JayK
Post by Marcello Romani
Post by David Morel
Post by Daniel McBrearty
I don't see any reason why it shouldn't be meaningful if it was done
well. Not that anyone should choose their framework on the basis of
such a benchmark, but it's a factor to throw into the mix
Does that include dynamic content caching wizardry ? It is
meaningless if you don't take into account real-life scenarios
like reverse proxy cache invalidation policies (and tricks). This
is just to say that all this perf talk is meaningless : sometimes
the power you get from a well thought out framework allows you to
do things that are close to magick, speed-wise among others.
Comparing simple setups is ridiculous IMHO.
David Morel
If a framework makes development easier because it's more elegant,
easy to use, or whatever, then you may have more time to think
about setting up a more efficient deployment architecture (i.e. the
thinks mentioned above).
Therefore it seems to me that ease of developement might be more
important to the overall app performance than the raw speed in
simple test cases.
Just my 2 (euro)cents.
Post by David Morel
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/
Dev site: http://dev.catalyst.perl.org/
--
Marcello Romani
Responsabile IT
Ottotecnica s.r.l.
http://www.ottotecnica.com
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/
Dev site: http://dev.catalyst.perl.org/
---
America will never be destroyed from the outside. If we falter and
lose our freedoms, it will be because we destroyed ourselves. --
Abraham Lincoln
Marcello Romani
2007-01-18 14:04:18 UTC
Permalink
Post by Jay K
I agree 100% on this... if we are judging Catalyst, et al, as simple
dispatchers, then we should consider apache+cgi in the discussion as
well - as apache is obviously one of the most venerable and widely
deployed dispatchers out there.
...and is probably faster than Catalyst at doing that.
Post by Jay K
A framework is much more than that. In my experience, the costs in time
and money involved in building and maintaining the code for an app
outweigh the cost of deploying it by huge margins. Furthermore, by the
time you are experiencing enough traffic to be analyzing the performance
at this granular a level, there are much better ways to improve your
responses per second that cost much less in terms of time and money.
These days, for less than the cost of a week of work, you can double
your serving capacity easily - and because of the 'fringe benefits' of
using Catalyst this is usually simply a matter of swapping out session
and caching plugins (if you even use them) to the more cross-server
compatible modules.
This brings another point into the mix: how easy is to swap those
modules? How much code do you have to change in you app to make it work?
If a framework has a good plugin system, you can just tell it to use a
different cache module (e.g. swap out Cache::FastMMap in favor of
memcached) and restart the application.
Post by Jay K
If you are using a frontend cache like squid, you can do even more
cost-for-performance-wise.
Personally, I use Catalyst because it takes care of a lot of details I
would rather not worry about. When I am free of worrying about all
those details, I can focus on building JUST my application logic. This
means that my application logic tends to be more solid, because I am not
tracking a ton of specifics outside of my app.
This could be a good starting point for a comparison between web frameworks.
Catalyst is very good in this area (IMHO) because for example I can
develop on WinXP using the test server and then deploy under
Linux/mod_perl *without changing a single line of code in my app* (there
could be issues if I use perl modules that are less portable than
Catalyst, but that's not Cat's fault).
That's a pretty impressive cross-platform-ness IMHO ! :-)
Post by Jay K
Because I'm not constantly crossing the line between my app and base
functionality (responding to HTTP, getting the correct bit of code
executed based on the request, etc.), I am not chasing bugs related to
that line. I can rely on the fact that it will always happen in a
particular way and if I have a bug, it's more than likely in my
application, so I can focus there.
That is the power of a Framework any 'benchmark' that doesn't take those
things into account is so much fluff and of no use to me.
Agreed.
Post by Jay K
JayK
--
Marcello Romani
Responsabile IT
Ottotecnica s.r.l.
http://www.ottotecnica.com
A. Pagaltzis
2007-01-18 19:47:48 UTC
Permalink
Post by Jay K
I agree 100% on this... if we are judging Catalyst, et al, as
simple dispatchers, then we should consider apache+cgi in the
discussion as well - as apache is obviously one of the most
venerable and widely deployed dispatchers out there.
Eh? Apache doesn?t dispatch anything unless maybe you?re talking
about mod_perl, and CGI.pm certainly doesn?t dispatch anything.
Not in the web framework sense of dispatch anwyay, which was born
precisely out of the desire to avoid having to write an ad-hoc
dispatcher in every CGI script.

Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>
Jonathan Rockway
2007-01-18 22:48:28 UTC
Permalink
Post by A. Pagaltzis
Eh? Apache doesn?t dispatch anything unless maybe you?re talking
about mod_perl
If that's true, then GETting http://mysite.com/foo/bar/baz.html would
get a file named $DOCROOT/foo\/bar\/baz.html. Considering that's never
the case (you can't have / in UNIX filenames, only paths), I would say
Apache is a dispatcher. Add mod_alias and mod_rewrite into the mix, and
Apache is definitely a URL dispatcher.
--
package JAPH;use Catalyst qw/-Debug/;($;=JAPH)->config(name => do {
$,.=reverse qw[Jonathan tsu rehton lre rekca Rockway][$_].[split //,
";$;"]->[$_].q; ;for 1..4;$,=~s;^.;;;$,});$;->setup;
A. Pagaltzis
2007-01-19 00:24:59 UTC
Permalink
Post by Jonathan Rockway
Post by A. Pagaltzis
Eh? Apache doesn?t dispatch anything unless maybe you?re
talking about mod_perl
If that's true, then GETting http://mysite.com/foo/bar/baz.html
would get a file named $DOCROOT/foo\/bar\/baz.html.
Considering that's never the case (you can't have / in UNIX
filenames, only paths), I would say Apache is a dispatcher.
Add mod_alias and mod_rewrite into the mix, and Apache is
definitely a URL dispatcher.
Hmm, you?re right. I guess I?ll start writing my apps as lots of
little CGI scripts so that I can use the Apache dispatching.

No wait, this isn?t 1992 anymore.

(Except for many PHP people who?re still stuck there.)

Last I checked, Apache?s URI translation won?t route requests to
particular methods within an app that?s launched from a single
script, so no, Apache is not a dispatcher in the sense that
Catalyst or RoR is.

You can of course load your app *into* the webserver with
mod_perl, in which case you can configure Apache to dispatch the
URI to a particular method in your app; but I mentioned that as
an exception in the previous mail anyway. I assumed that would be
enough of a hint, but I guess I expected too much.

Are we done splitting this hair now or do you need more
clarification?

Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>
Jonathan Rockway
2007-01-19 00:30:32 UTC
Permalink
Post by A. Pagaltzis
Are we done splitting this hair now or do you need more
clarification?
Depends on what you mean by "methods":

foo.pl:
use MyApp;
print MyApp->foo($ENV{QUERY_STRING});

bar.pl:
use MyApp;
print MyApp->bar($ENV{QUERY_STRING});

I don't think this is a good idea, but you can do it if you want to.

"can" and "good idea" is the hair that I'm splitting.
--
package JAPH;use Catalyst qw/-Debug/;($;=JAPH)->config(name => do {
$,.=reverse qw[Jonathan tsu rehton lre rekca Rockway][$_].[split //,
";$;"]->[$_].q; ;for 1..4;$,=~s;^.;;;$,});$;->setup;
A. Pagaltzis
2007-01-19 00:58:30 UTC
Permalink
Post by Jonathan Rockway
Post by A. Pagaltzis
Are we done splitting this hair now or do you need more
clarification?
use MyApp;
print MyApp->foo($ENV{QUERY_STRING});
use MyApp;
print MyApp->bar($ENV{QUERY_STRING});
I don't think this is a good idea, but you can do it if you want to.
"can" and "good idea" is the hair that I'm splitting.
Yes, sure, as I already said in my previous mail [1], you can
abuse Apache?s URI-to-filesystem mapper as a dispatcher with
the right contortions as long as your URI structure is static.
(No wait, your next mail will point out that you can create
directories and generate scripts from a template; so the URI
structure needn?t be static.) Noone in their right mind is going
to want to in practice, of course. So when the subject was what
dispatchers should be benchmarked, why point it out? Is anyone
going to care?

I guess it?s me who shouldn?t have. Sorry. I should know better.


[1] Does anyone notice a pattern here?

Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>
Jay K
2007-01-19 01:04:42 UTC
Permalink
I find it interesting how people will grab one particular part of a
post and completely disregard the rest. I don't really care too much
about this rather silly discussion, but ability to bludgeon doesn't
make you correct and I feel the need to point that out.

Apache takes a request (string + other request information) and
translates it to a specific action. That may be showing an directory
index, serving up the contents of a file, or executing a particular
bundle of code (via CGI or any number of other methods). Depending
on how it is configured, it can perform access control /
authorization and numerous other bits of functionality.

That is what a dispatcher does. There is no requirement that the
thing a dispatcher decides to do is in one particular arbitrarily
defined bundle of code.

Using apache as your only dispatcher may be out of fashion, or
outmoded, or whatever you choose to call it, but that doesn't mean it
isn't a dispatcher.

Point being that frameworks provide more than choosing what to do,
and if you stop the comparison at that point, then you might as well
include every web server in your comparison.

It was intended to point out one of the problems with the benchmark,
using an extreme example.

I need to remember that subtlety has no place on mailing lists.

Jay
Post by A. Pagaltzis
Hmm, you?re right. I guess I?ll start writing my apps as lots of
little CGI scripts so that I can use the Apache dispatching.
No wait, this isn?t 1992 anymore.
(Except for many PHP people who?re still stuck there.)
Last I checked, Apache?s URI translation won?t route requests to
particular methods within an app that?s launched from a single
script, so no, Apache is not a dispatcher in the sense that
Catalyst or RoR is.
You can of course load your app *into* the webserver with
mod_perl, in which case you can configure Apache to dispatch the
URI to a particular method in your app; but I mentioned that as
an exception in the previous mail anyway. I assumed that would be
enough of a hint, but I guess I expected too much.
Are we done splitting this hair now or do you need more
clarification?
Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/
Dev site: http://dev.catalyst.perl.org/
---
America will never be destroyed from the outside. If we falter and
lose our freedoms, it will be because we destroyed ourselves. --
Abraham Lincoln
A. Pagaltzis
2007-01-19 03:33:20 UTC
Permalink
Post by Jay K
Point being that frameworks provide more than choosing what to
do,
Care to actually mention a few examples?
Post by Jay K
if you stop the comparison at that point, then you might as
well include every web server in your comparison.
As long as the web server provides a way to dispatch to a piece
of my own code, then yes, absolutely. Apache + mod_perl is a
reasonable benchmark candidate, f.ex.

Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>
A. Pagaltzis
2007-01-16 04:06:56 UTC
Permalink
Post by Daniel McBrearty
I don't see any reason why it shouldn't be meaningful if it was
done well. Not that anyone should choose their framework on the
basis of such a benchmark, but it's a factor to throw into the
mix.
Because as long as the framework is not improbably slow,
its contribution to an app?s performance characteristics
will just be noise in any realworld scenario.

Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>
Octavian Rasnita
2007-01-15 16:14:13 UTC
Permalink
Post by Carl Johnstone
Post by Octavian Rasnita
Is it true that Catalyst is so slow comparing
with other frameworks?
Does it matter?
Of course it does.
Post by Carl Johnstone
If speed is so important, you should write your own custom httpd that does
exactly what you need in assembly language.
It is too hard to write it in assembly, but it could be more easy to use
Ruby on Rails instead if RoR is really faster.
That's why I have asked.

Now to say the truth, I won't use RoR because I don't know Ruby, but I want
to know which are the advantages and disadvantages of Catalyst comparing
with other frameworks.

Octavian
Daniel McBrearty
2007-01-15 16:25:05 UTC
Permalink
you honestly can't come to much of a conclusion about a test that
just hits the docroot over and over again, and sees how many
connections per second happen as a result. The test may say that one
aprticular aspect of RoR is quicker ... but it's not an aspect that
has a huge impact, given that the speed of most apps is affected by
what happens in the db layer.

After all, if you wanted you app to do this, you would just apache
serving a static file :-)\


Also, I noted that perrin's comment about using the -r option for
dprof hasn't been answered - maybe there was something else altogether
going on.
Post by Octavian Rasnita
Post by Carl Johnstone
Post by Octavian Rasnita
Is it true that Catalyst is so slow comparing
with other frameworks?
Does it matter?
Of course it does.
Post by Carl Johnstone
If speed is so important, you should write your own custom httpd that does
exactly what you need in assembly language.
It is too hard to write it in assembly, but it could be more easy to use
Ruby on Rails instead if RoR is really faster.
That's why I have asked.
Now to say the truth, I won't use RoR because I don't know Ruby, but I want
to know which are the advantages and disadvantages of Catalyst comparing
with other frameworks.
Octavian
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Dev site: http://dev.catalyst.perl.org/
--
Daniel McBrearty
email : danielmcbrearty at gmail.com
www.engoi.com : the multi - language vocab trainer
BTW : 0873928131
Robert 'phaylon' Sedlacek
2007-01-15 16:35:35 UTC
Permalink
Post by Octavian Rasnita
Now to say the truth, I won't use RoR because I don't know Ruby, but I
want to know which are the advantages and disadvantages of Catalyst
comparing with other frameworks.
To summarize (again): The benchmark doesn't benchmark Catalyst, only
it's dispatcher (and as I see it from the site and tarball, TT) and the
way it does it hasn't even anything to do with dispatching in real
applications.

I never used RoR, but as I understood it, it's dispatching facilities
are no way near those of Catalyst.
--
# Robert 'phaylon' Sedlacek
# Perl 5/Catalyst Developer in Hamburg, Germany
{ EMail => ' ***@474.at ', Web => ' http://474.at ' }
Perrin Harkins
2007-01-15 21:52:29 UTC
Permalink
Post by Robert 'phaylon' Sedlacek
To summarize (again): The benchmark doesn't benchmark Catalyst, only
it's dispatcher
I think it's a lame benchmark too, but isn't a dispatcher mostly what
Catalyst is? DBIx::Class and TT are not Catalyst, as people often
mention on the list.

- Perrin
Daniel McBrearty
2007-01-16 01:25:59 UTC
Permalink
they are not, but when you choose a framework you don't just choose a
dispatcher. You choose all the other design options that go with it.
Post by Perrin Harkins
Post by Robert 'phaylon' Sedlacek
To summarize (again): The benchmark doesn't benchmark Catalyst, only
it's dispatcher
I think it's a lame benchmark too, but isn't a dispatcher mostly what
Catalyst is? DBIx::Class and TT are not Catalyst, as people often
mention on the list.
- Perrin
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Dev site: http://dev.catalyst.perl.org/
--
Daniel McBrearty
email : danielmcbrearty at gmail.com
www.engoi.com : the multi - language vocab trainer
BTW : 0873928131
A. Pagaltzis
2007-01-16 04:13:50 UTC
Permalink
Post by Daniel McBrearty
Post by Perrin Harkins
Post by Robert 'phaylon' Sedlacek
To summarize (again): The benchmark doesn't benchmark
Catalyst, only it's dispatcher
I think it's a lame benchmark too, but isn't a dispatcher
mostly what Catalyst is? DBIx::Class and TT are not Catalyst,
as people often mention on the list.
they are not, but when you choose a framework you don't just
choose a dispatcher. You choose all the other design options
that go with it.
Like what? And what about those other design options is
benchmarkable?

Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>
Daniel McBrearty
2007-01-16 16:28:03 UTC
Permalink
Post by A. Pagaltzis
Like what? And what about those other design options is
benchmarkable?
1. the language. For instance, a key factor against RoR for me was the
fact that Ruby doesn't know where its going w.r.t. unicode. Perl has
mature support for that. There are multiple other reasons why people
like/dislike various langauges, and a lot of it, if we are honest, is
taste, or factors specific to that project.

2. the library support in the language. Ditto.

3. as a subset of 2, templating systems, ORM's ...

It may be that the differences between these things, platform to
platform, are insignificant compared to other factors. OK. But if I
was setting out to do this exercise (which I'm not, right now ...) I
would make some basic measurements anyhow, at least as a start point.

<snip>
Because as long as the framework is not improbably slow,
its contribution to an app's performance characteristics
will just be noise in any realworld scenario.
</snip>

So what are the key factors that influence performance? Why not design
a benchmark such that it can show up those differences?

I'm not pretending to know in advance what makes the difference. I
don't. I just don't think that saying "there's no point measuring it
... " and expecting the world to just believe is a very realistic
combination. Hence the question I was trying to ask in the other
thread - what DOES make a realistic benchmark?


<snip>
Does that include dynamic content caching wizardry ? It is
meaningless if you don't take into account real-life scenarios like
reverse proxy cache invalidation policies (and tricks). This is just
to say that all this perf talk is meaningless : sometimes the power
you get from a well thought out framework allows you to do things
that are close to magick, speed-wise among others. Comparing simple
setups is ridiculous IMHO.
</snip>

Fair enough. So why not try to design a benchmark in such a way that
those techniques can be exploited? What is the simplest set of tests
that has some meaning for you?
--
Daniel McBrearty
email : danielmcbrearty at gmail.com
www.engoi.com : the multi - language vocab trainer
BTW : 0873928131
David Morel
2007-01-16 16:54:37 UTC
Permalink
Post by Daniel McBrearty
Fair enough. So why not try to design a benchmark in such a way that
those techniques can be exploited? What is the simplest set of tests
that has some meaning for you?
I don't know :) I'm thinking benchmarking simple things don't work.
Pushing this a bit further, I realized I didn't really care about
speed as long as the apps are resonably fast. I don't really care if
a framework gives me 150 hits/s and another one 130. Maybe what
matters in the end is the resources you have to allocate (financially
speaking, which implies how many servers, elcetricity, developer
time, etc) to a given project. This implies observing the projects on
the long term, and encompasses such concerns as application
lifecycle, security (how much did the downtime to plug that hole cost
us ?), extensibility, developer turnover ("Bye boss, had enough of
your php toys"), etc. Don't you think ?

I don't know of anyone wishing to write the same app twice, or a
client willing to pay two teams of developers to produce the same app
in two different languages, so I guess the comparison cannot be done :)

Am I a would-be suit -or moron, pick one ? Of course it would be nice
to say : Catalyst is the best AND fastest framework there is. But
IMHO it's not that important (considering establishing such a fact is
nearly impossible). I'm pretty satisfied with the fact that I can use
the whole of CPAN at great speed, and I think that matters more.

David Morel <***@amakuru.net>
-----------------------------------
OpenPGP pubkey :
http://www.amakuru.net/dmorel.asc
A. Pagaltzis
2007-01-17 00:36:40 UTC
Permalink
[ snip mile-long excursion totally missing the point ]
What Perrin said is that Catalyst (and RoR and all the others)
are really just dispatchers, so if you wanted to benchmark them,
what else than dispatching would you benchmark?

You replied by saying a web framework is not just a dispatcher,
you choose it for lots of other related reasons. Well d?uh, but
that doesn?t have to anything to do with ?if you were to
benchmark Catalyst, what functionality is it that Catalyst itself
provides that you would be excercising??

Answer: the dispatcher.

Catalyst, Jifty, RoR, Camping, Django, Turbogears and all the
others are dispatchers with bits of glue for other components
thrown in. That?s all they are.

And that?s what Perrin said.

Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>
Robert 'phaylon' Sedlacek
2007-01-16 16:13:44 UTC
Permalink
Post by Perrin Harkins
Post by Robert 'phaylon' Sedlacek
To summarize (again): The benchmark doesn't benchmark Catalyst, only
it's dispatcher
I think it's a lame benchmark too, but isn't a dispatcher mostly what
Catalyst is? DBIx::Class and TT are not Catalyst, as people often
mention on the list.
I wouldn't say so. You can build applications that only use the
dispatcher of Catalyst, but the model interface, the view (not TT, just
the view classes), the plugin system, inherited actions, the config
system, action classes, etc. are pretty well used in most of my apps too.
--
# Robert 'phaylon' Sedlacek
# Perl 5/Catalyst Developer in Hamburg, Germany
{ EMail => ' ***@474.at ', Web => ' http://474.at ' }
Perrin Harkins
2007-01-16 20:01:38 UTC
Permalink
Post by Robert 'phaylon' Sedlacek
Post by Perrin Harkins
I think it's a lame benchmark too, but isn't a dispatcher mostly what
Catalyst is? DBIx::Class and TT are not Catalyst, as people often
mention on the list.
I wouldn't say so. You can build applications that only use the
dispatcher of Catalyst, but the model interface, the view (not TT, just
the view classes), the plugin system, inherited actions, the config
system, action classes, etc. are pretty well used in most of my apps too.
In the context of benchmarks, those things all seem pretty minimal to
me. The model interface is a razor-thin wrapper over calls to your ORM
of choice. There's not much reason to use it that I can see. Same with
the view: very little difference from calling templating tools directly.
(Somewhat more reason to use it though, since people often mess up the
caching in TT when they write their own calls to it.)

Most plugins don't do anything except pass your config to some module
and put a method in $c for something you could call directly. A few
specific plugins that contain some code do add real value -- session,
auth, pagecache -- and are frequently used, so I would consider them a
core part of Catalyst's functionality. Those would be fair game for a
benchmark.

Inherited actions and action classes are part of the dispatcher in a
broad sense (i.e. part of how URLs get mapped to methods).

The config system does provide convenient centralization but is mostly a
call to YAML or similar and won't contribute to runtime performance.

The biggest other thing that Catalyst itself provides is an abstraction
layer over runtime environments. That's a standard part of all modern
web framework, and shows up in any benchmark naturally.

The point being, saying that benchmarking the mapping of URLs to methods
isn't a good test of Catalyst is like saying that running SQL queries
isn't a good test of DBI. The bulk of the actual Catalyst code (not ORM
or templating) that will run on any request to a Catalyst app is the
dispatcher and the runtime environment abstraction. It's a *terrible*
test of the real-world performance of real web applications, which is
mostly about the database access and templates, but a good test of the
Catalyst code.

- Perrin
Robert 'phaylon' Sedlacek
2007-01-16 20:44:26 UTC
Permalink
Post by Perrin Harkins
In the context of benchmarks, those things all seem pretty minimal to
me. The model interface is a razor-thin wrapper over calls to your ORM
of choice. There's not much reason to use it that I can see. Same with
the view: very little difference from calling templating tools directly.
(Somewhat more reason to use it though, since people often mess up the
caching in TT when they write their own calls to it.)
It might be small pieces individually, but that can sum up. There seem
to be some talks about optimising the way uri_for works to make it
faster, so this doesn't seem to be that small of a value.
Post by Perrin Harkins
Inherited actions and action classes are part of the dispatcher in a
broad sense (i.e. part of how URLs get mapped to methods).
Well, in a _very_ broad sense IMHO ;) but no harm in differing here.
Post by Perrin Harkins
The config system does provide convenient centralization but is mostly a
call to YAML or similar and won't contribute to runtime performance.
Correct. This wasn't an argument that they do, rather one against
"Catalyst is only a fancy dispatcher," with which I still disagree.
Post by Perrin Harkins
The point being, saying that benchmarking the mapping of URLs to methods
isn't a good test of Catalyst is like saying that running SQL queries
isn't a good test of DBI.
That's a rather odd comparison. I'd say "benchmarking mapping of URLs to
methods isn't a good test of Catalyst like benchmarking DBI isn't a good
test of DBIx-Class."
Post by Perrin Harkins
The bulk of the actual Catalyst code (not ORM
or templating) that will run on any request to a Catalyst app is the
dispatcher and the runtime environment abstraction. It's a *terrible*
test of the real-world performance of real web applications, which is
mostly about the database access and templates, but a good test of the
Catalyst code.
I'm not sure what definition you'd use for the word "test" here. With
some I can agree, with "benchmark against $X" I still have to disagree.
Sure, a benchmark once in a while can show you how things in Catalyst
got faster/slower. I'm just saying that you can't get a "X is faster
than Y" but only a "X is faster as Y in the way Z uses it" out of it.
--
# Robert 'phaylon' Sedlacek
# Perl 5/Catalyst Developer in Hamburg, Germany
{ EMail => ' ***@474.at ', Web => ' http://474.at ' }
A. Pagaltzis
2007-01-17 00:40:37 UTC
Permalink
Post by Robert 'phaylon' Sedlacek
Post by Perrin Harkins
The point being, saying that benchmarking the mapping of URLs
to methods isn't a good test of Catalyst is like saying that
running SQL queries isn't a good test of DBI.
That's a rather odd comparison. I'd say "benchmarking mapping
of URLs to methods isn't a good test of Catalyst like
benchmarking DBI isn't a good test of DBIx-Class."
I agree that Perrin?s analogy was a bit odd, but yours is even
odder.

Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>
Robert 'phaylon' Sedlacek
2007-01-17 20:12:58 UTC
Permalink
Post by Robert 'phaylon' Sedlacek
That's a rather odd comparison. I'd say "benchmarking mapping
of URLs to methods isn't a good test of Catalyst like
benchmarking DBI isn't a good test of DBIx-Class."
I agree that Perrin???s analogy was a bit odd, but yours is even
odder.
You are of course free to think so, but without some Why's I'm hardly able
to argue about it. But I always wanted to be psychic, so I'll try it
anyway :)

When I request a resource from a Catalyst application, two things are
executed: The framework logic, and my application logic. But this is not a
"first the one, then the other" execution. During the request, the
framework calls my application logic to get some response together, and my
application logic uses the framework logic in a convenient way. The
dispatching (read: The decisio on what part of my application logic to
execute) is a rather small part.
--
# Robert 'phaylon' Sedlacek
# Perl 5/Catalyst Developer in Hamburg, Germany
{ EMail => ' ***@474.at ', Web => ' http://474.at ' }
Perrin Harkins
2007-01-17 23:30:40 UTC
Permalink
Post by Robert 'phaylon' Sedlacek
When I request a resource from a Catalyst application, two things are
executed: The framework logic, and my application logic. But this is not a
"first the one, then the other" execution. During the request, the
framework calls my application logic to get some response together, and my
application logic uses the framework logic in a convenient way. The
dispatching (read: The decisio on what part of my application logic to
execute) is a rather small part.
That was my point actually: the dispatching (plus the abstraction layer
for accessing things like the query string from the web environment) is
nearly all of it, in terms of performance. A couple of calls to
$c->forward() and $c->stash() are negligible, and all the rest is either
your code or some non-Catalyst CPAN module.

Regarding the DBI analogy, the thinking is pretty simple:

- DBI is big, much bigger than Catalyst.
- DBI has tons of functionality in it, like a profiler, database schema
introspection, and a proxy system.
- If someone made a benchmark comparing DBI to JDBC based on query
speed, no one would complain that they didn't test the speed of the
other features, even though you might use them. Query speed is the most
relevant thing to benchmark for DBI, just like mapping URLs to methods
is the most relevant thing to benchmark for Catalyst.

The benchmark still sucks though, even as a dispatcher test. He gave it
the easiest possible URL.

- Perrin
Sébastien Wagener
2007-01-18 00:24:38 UTC
Permalink
What about profiling real world applications where the Catalyst seems to
be the bottleneck?

On my production server, database requests are usually quite fast, so
most of the time is spent in perl code, and here are the first lines of
a "dprofpp -r" on my local 2.8 Ghz Laptop (production database,
Algorithm::C3 0.06, mod_perl deployment, as I do not want to use FastCGI
and dprofpp -F), for a repeated lost-password/login/change
password/logout cycle (totally about 1000 requests):

Total Elapsed Time = 142.8513 Seconds
Real Time = 142.8513 Seconds
Exclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c Name
13.7 19.56 19.569 53920 0.0004 0.0004 NEXT::ELSEWHERE::ancestors
11.0 15.83 77.300 57155 0.0003 0.0014 NEXT::AUTOLOAD
2.61 3.731 3.731 485987 0.0000 0.0000
Class::Accessor::Fast::__ANON__

It appears that 39.121 (exclusive) seconds out of 142.8513 seconds, i.e.
about 27.4% of the running time of my application are spent in
subroutines which would probably not even exist in a compiled language
like C++, or which could at least be ignored in such a language...
(correct me if I am wrong)

I do not want to criticise my personal profiling here, I am going to do
that if I have more time. My point is: Do not talk about benchmarks but
try to make real world applications faster. There are real world
instances where Catalyst, DBIx::Class and related modules contribute a
significant part to the running time of the application. If it is
possible to reduce this part by a reasonable amount of work, try to
reduce it. I do not mind a framework that speeds up the development
time, but results in slightly slower requests if I know that the
Catalyst team does not ignore the speed of its framework. I am not
claiming that the team ignores performance, there have actually been
made impressive improvements in Algorithm::C3 and so on.
Just go on, and do not waste your time to build benchmarks to impress
marketing people ;-)

S?bastien
Post by Perrin Harkins
Post by Robert 'phaylon' Sedlacek
When I request a resource from a Catalyst application, two things are
executed: The framework logic, and my application logic. But this is not a
"first the one, then the other" execution. During the request, the
framework calls my application logic to get some response together, and my
application logic uses the framework logic in a convenient way. The
dispatching (read: The decisio on what part of my application logic to
execute) is a rather small part.
That was my point actually: the dispatching (plus the abstraction layer
for accessing things like the query string from the web environment) is
nearly all of it, in terms of performance. A couple of calls to
$c->forward() and $c->stash() are negligible, and all the rest is either
your code or some non-Catalyst CPAN module.
- DBI is big, much bigger than Catalyst.
- DBI has tons of functionality in it, like a profiler, database schema
introspection, and a proxy system.
- If someone made a benchmark comparing DBI to JDBC based on query
speed, no one would complain that they didn't test the speed of the
other features, even though you might use them. Query speed is the most
relevant thing to benchmark for DBI, just like mapping URLs to methods
is the most relevant thing to benchmark for Catalyst.
The benchmark still sucks though, even as a dispatcher test. He gave it
the easiest possible URL.
- Perrin
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Dev site: http://dev.catalyst.perl.org/
Perrin Harkins
2007-01-18 00:44:10 UTC
Permalink
Post by Sébastien Wagener
On my production server, database requests are usually quite fast, so
most of the time is spent in perl code, and here are the first lines of
a "dprofpp -r" on my local 2.8 Ghz Laptop (production database,
Algorithm::C3 0.06, mod_perl deployment, as I do not want to use FastCGI
and dprofpp -F), for a repeated lost-password/login/change
password/logout cycle (totally about 1000 requests)
One tip for profiling with mod_perl:
If you preload your modules in httpd.conf (or a startup.pl called from
it), make sure you initialize the debugger before you load them or else
they will not show up in DProf output.

For example, put this before any PerlModule or PerlRequire statements:
<Perl>
require Apache::DB;
Apache::DB->init;
</Perl>

- Perrin
Matt S Trout
2007-01-22 19:39:39 UTC
Permalink
Post by Sébastien Wagener
What about profiling real world applications where the Catalyst seems to
be the bottleneck?
On my production server, database requests are usually quite fast, so
most of the time is spent in perl code, and here are the first
lines of
a "dprofpp -r" on my local 2.8 Ghz Laptop (production database,
Algorithm::C3 0.06, mod_perl deployment, as I do not want to use FastCGI
and dprofpp -F), for a repeated lost-password/login/change
That happens with lots of plugins; suggest trying
Catalyst::Plugin::C3 to remove that overhead (note that it's not 100%
bug free but the performance increase is worth a go)
--
Matt S Trout, Technical Director, Shadowcat Systems Ltd.
Offering custom development, consultancy and support contracts for
Catalyst,
DBIx::Class and BAST. Contact mst (at) shadowcatsystems.co.uk for
details.
+ Help us build a better perl ORM: http://dbix-
class.shadowcatsystems.co.uk/ +
Robert 'phaylon' Sedlacek
2007-01-18 20:54:44 UTC
Permalink
Post by Perrin Harkins
That was my point actually: the dispatching (plus the abstraction layer
for accessing things like the query string from the web environment) is
nearly all of it, in terms of performance. A couple of calls to
$c->forward() and $c->stash() are negligible, and all the rest is either
your code or some non-Catalyst CPAN module.
Well, let's just agree that we have differing opinions on this. Otherwise
we'd only start repeating ourselves.
Post by Perrin Harkins
The benchmark still sucks though, even as a dispatcher test. He gave it
the easiest possible URL.
And I think we never disagreed on this point :)
--
# Robert 'phaylon' Sedlacek
# Perl 5/Catalyst Developer in Hamburg, Germany
{ EMail => ' ***@474.at ', Web => ' http://474.at ' }
Carl Johnstone
2007-01-15 18:26:19 UTC
Permalink
Post by Octavian Rasnita
Now to say the truth, I won't use RoR because I don't know Ruby, but I
want to know which are the advantages and disadvantages of Catalyst
comparing with other frameworks.
The most important advantage/disadvantage *to you* must be that Catalyst is
Perl and you know that, and RoR is Ruby and you don't know that. So it
becomes more about Ruby vs Perl than Catalyst vs Rails.

It's also entirely personal to you, as you've got to factor in your desire
to learn Ruby.

Carl
Octavian Rasnita
2007-01-15 18:55:53 UTC
Permalink
Post by Carl Johnstone
Post by Octavian Rasnita
Now to say the truth, I won't use RoR because I don't know Ruby, but I
want to know which are the advantages and disadvantages of Catalyst
comparing with other frameworks.
The most important advantage/disadvantage *to you* must be that Catalyst
is Perl and you know that, and RoR is Ruby and you don't know that. So it
becomes more about Ruby vs Perl than Catalyst vs Rails.
It's also entirely personal to you, as you've got to factor in your desire
to learn Ruby.
Carl
I have seen fewer and fewer people start learning perl, and more become
interested about Python and Ruby (not mentioning those that like C#, Java,
C...).
They can say that their preferate language is better, that it is newer and
that it took what's the best from perl and from other languages, that its
object orientation is better, that even though it doesn't have so many
libraries as perl does, they are better than those from perl, that the few
frameworks available for their language are faster than the frameworks that
work under perl, and so on.

I would like to say that it is not true, but I cannot see any benchmarks,
and I don't know those languages for beeing able to create them, and that's
why I would like to know.

Octavian
Robert 'phaylon' Sedlacek
2007-01-15 19:18:46 UTC
Permalink
Post by Octavian Rasnita
I have seen fewer and fewer people start learning perl, and more become
interested about Python and Ruby (not mentioning those that like C#,
Java, C...).
They can say that their preferate language is better, that it is newer
and that it took what's the best from perl and from other languages,
that its object orientation is better, that even though it doesn't have
so many libraries as perl does, they are better than those from perl,
that the few frameworks available for their language are faster than the
frameworks that work under perl, and so on.
I would like to say that it is not true, but I cannot see any
benchmarks, and I don't know those languages for beeing able to create
them, and that's why I would like to know.
In my experience, you can't prove anything to people who argue like "my
language is better." The real reason for them to say that Perl stinks
is, that most people say that Perl stinks.

I have given up on it. Tell it to the people who are able to listen and
don't waste your energy on the rest.
--
# Robert 'phaylon' Sedlacek
# Perl 5/Catalyst Developer in Hamburg, Germany
{ EMail => ' ***@474.at ', Web => ' http://474.at ' }
Perrin Harkins
2007-01-15 22:29:11 UTC
Permalink
Post by Octavian Rasnita
I would like to say that it is not true, but I cannot see any benchmarks
I don't think anyone disputes that Perl (and Python and Java) are much
faster than Ruby. You can find benchmarks showing that all over the
web. The RoR boosters are usually the ones on the defensive over
performance, saying that language performance doesn't matter because the
database is always the bottleneck (which I agree with, most of the
time). It would be very hard to justify choosing RoR for a project
based on performance.

- Perrin
Octavian Rasnita
2007-01-15 23:22:10 UTC
Permalink
Post by Perrin Harkins
I don't think anyone disputes that Perl (and Python and Java) are much
faster than Ruby. You can find benchmarks showing that all over the
web. The RoR boosters are usually the ones on the defensive over
performance, saying that language performance doesn't matter because the
database is always the bottleneck (which I agree with, most of the
time). It would be very hard to justify choosing RoR for a project
based on performance.
- Perrin
That's why I was curious and I have sent to the list that blog with the
comparison between RoR and Catalyst.

Octavian
Nilson Santos Figueiredo Junior
2007-01-15 23:57:29 UTC
Permalink
Post by Octavian Rasnita
That's why I was curious and I have sent to the list that blog with the
comparison between RoR and Catalyst.
You need to keep in mind that sometimes it's easier to optimize things
for benchmarks than for real world applications. That happens in
almost every field of computer technology: from video cards to web
development frameworks.

The only real way of comparing frameworks is to build a small (but
complex) application using each one of them. I honestly don't know if
Catalyst would perform better than the others but I think that if it
performs worse, it won't be by a large margin under these conditions.

Of course, no one wants to do that. People want to do quick and dirty
benchmarks (and, thus, get quick and dirty results). I wish I had
enough knowledge regarding RoR and Ruby so that I could perform a
proper benchmark but I don't even know if it supports things such as
chained actions which we take for granted when using Catalyst.

Leaving performance aside, I think Catalyst is probably the best web
framework out there feature-wise. And, for most people who have a
clue, that's usually what really matters.

-Nilson Santos F. Jr.
Peter Edwards
2007-01-15 18:26:25 UTC
Permalink
For businesses the cost-to-develop and cost-to-maintain are usually more
important than handler performance.
The reason is that in most medium-large transactional web systems the
bottleneck is the database and not the framework.
A 10-100x slowdown in using an ORM or your framework of choice doesn't make
much difference in terms of end user perceived speed (assuming you serve
static content from a lightweight proxy web server). It does save a lot of
money in cost-to-maintain.

Sure there is a place for optimised libraries and so on, but it's only worth
optimising bottlenecks and the handler usually isn't one.
I recently had to hand-write a TCP/IP server in Perl using POE to handle one
particularly heavy case but most situations are easily dealt with by Apache
mod_perl or FastCGI, in fact running Perl in CGI mode is often adequate.

More important questions to me about frameworks are how:
- easy to learn
- quick to develop
- flexible
- comprehensive in terms of supporting infrastructure (templates,
components)
- easy/expensive to recruit staff
- easy to integrate with other systems

Realistic approaches include
.NET
J2EE
Catalyst, Jifty
RoR

They all have strengths/weaknesses.
For example, you'll get a Jifty or RoR app up quickly if you do it the
standard way. If you want to build a bigger system that doesn't quite fit
their model you're probably better off with Catalyst or another framework.
I did one large system with Catalyst last year and it was great for that.
I did a smaller system just before Christmas and wrote my own simple MVC
framework for convenience.
It's a question of choosing the right tool for the job.

Regards,
Peter Edwards
www.dragonstaff.com Business IT Consultancy
-----Original Message-----
From: Hermida, Leandro [mailto:***@fmi.ch]
Sent: 15 January 2007 09:30
To: The elegant MVC web framework
Subject: RE: [Catalyst] Catalyst vs Rails vs Django Cook off
Sent: Monday, January 15, 2007 10:21
Subject: Re: [Catalyst] Catalyst vs Rails vs Django Cook off
Is it true that Catalyst is so slow comparing with other frameworks?
Does it matter?
If speed is so important, you should write your own custom
httpd that does exactly what you need in assembly language.
Carl
Speed does matter and I believe the original thread question is a valid
one. Not everyone has the time or the know-how to do wheel reinvention
and write custom daemons (I know I don't). That's why people write
kernels and libraries and abstraction of lower level things so that
others can build things on top. Otherwise we would never get anything
done.
Jonathan Rockway
2007-01-16 00:09:32 UTC
Permalink
Post by Octavian Rasnita
Hi,
Here is ablog I just found. Is it true that Catalyst is so slow comparing
with other frameworks?
http://letsgetdugg.com/category/rails
Rails can do nothing* faster than Catalyst. Good for it.

* Dispatch to an action that returns "Hello world."

Try adding a templating system and a database (it's an MVC framework after
all) and then see what the results look like.
--
package JAPH;use Catalyst qw/-Debug/;($;=JAPH)->config(name => do {
$,.=reverse qw[Jonathan tsu rehton lre rekca Rockway][$_].[split //,
";$;"]->[$_].q; ;for 1..4;$,=~s;^.;;;$,});$;->setup;
Matt S Trout
2007-01-16 18:27:08 UTC
Permalink
Post by Octavian Rasnita
Hi,
Here is ablog I just found. Is it true that Catalyst is so slow
comparing with other frameworks?
http://letsgetdugg.com/category/rails
Is it true that for incredibly trivial applications Catalyst might
serve less rps? Yes.

Have I ever seen a real-world scenario where the things this
benchmarks account for more than 0.1% of runtime? No.

We discussed this. Re-read the old discussion. Thread over :)
--
Matt S Trout, Technical Director, Shadowcat Systems Ltd.
Offering custom development, consultancy and support contracts for
Catalyst,
DBIx::Class and BAST. Contact mst (at) shadowcatsystems.co.uk for
details.
+ Help us build a better perl ORM: http://dbix-
class.shadowcatsystems.co.uk/ +
Daniel McBrearty
2007-01-16 20:26:04 UTC
Permalink
I looked for that discussion - didn't find it. Do you have a pointer?

In your experience, what is accounting for the other 99.9% fo runtime?
Db access? templates? other?
Post by Matt S Trout
Post by Octavian Rasnita
Hi,
Here is ablog I just found. Is it true that Catalyst is so slow
comparing with other frameworks?
http://letsgetdugg.com/category/rails
Is it true that for incredibly trivial applications Catalyst might
serve less rps? Yes.
Have I ever seen a real-world scenario where the things this
benchmarks account for more than 0.1% of runtime? No.
We discussed this. Re-read the old discussion. Thread over :)
--
Matt S Trout, Technical Director, Shadowcat Systems Ltd.
Offering custom development, consultancy and support contracts for
Catalyst,
DBIx::Class and BAST. Contact mst (at) shadowcatsystems.co.uk for
details.
+ Help us build a better perl ORM: http://dbix-
class.shadowcatsystems.co.uk/ +
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Dev site: http://dev.catalyst.perl.org/
--
Daniel McBrearty
email : danielmcbrearty at gmail.com
www.engoi.com : the multi - language vocab trainer
BTW : 0873928131
Daniel McBrearty
2007-01-16 20:36:49 UTC
Permalink
forget that ... found it

http://www.gossamer-threads.com/lists/catalyst/users/10756?search_string=data%20retrieval;#10756
Post by Daniel McBrearty
I looked for that discussion - didn't find it. Do you have a pointer?
In your experience, what is accounting for the other 99.9% fo runtime?
Db access? templates? other?
Post by Matt S Trout
Post by Octavian Rasnita
Hi,
Here is ablog I just found. Is it true that Catalyst is so slow
comparing with other frameworks?
http://letsgetdugg.com/category/rails
Is it true that for incredibly trivial applications Catalyst might
serve less rps? Yes.
Have I ever seen a real-world scenario where the things this
benchmarks account for more than 0.1% of runtime? No.
We discussed this. Re-read the old discussion. Thread over :)
--
Matt S Trout, Technical Director, Shadowcat Systems Ltd.
Offering custom development, consultancy and support contracts for
Catalyst,
DBIx::Class and BAST. Contact mst (at) shadowcatsystems.co.uk for
details.
+ Help us build a better perl ORM: http://dbix-
class.shadowcatsystems.co.uk/ +
_______________________________________________
Listinfo: http://lists.rawmode.org/mailman/listinfo/catalyst
Dev site: http://dev.catalyst.perl.org/
--
Daniel McBrearty
email : danielmcbrearty at gmail.com
www.engoi.com : the multi - language vocab trainer
BTW : 0873928131
--
Daniel McBrearty
email : danielmcbrearty at gmail.com
www.engoi.com : the multi - language vocab trainer
BTW : 0873928131
Daniel McBrearty
2007-01-18 21:28:49 UTC
Permalink
Post by Matt S Trout
We discussed this. Re-read the old discussion. Thread over :)
ooooh nooo it's not ....
--
Daniel McBrearty
email : danielmcbrearty at gmail.com
www.engoi.com : the multi - language vocab trainer
BTW : 0873928131
Loading...