|
»
»
»
some flawed benchmarks
|
View:
New views
10 Messages
—
Rating Filter:
Alert me
|
 |
some flawed benchmarks
by adam.prime
::
Rate this Message:
A couple of months ago i was going through slides from gozers "From
CGI to mod_perl 2.0, Fast!" talk, which has some benchmarks comparing CGI, perlrun and registry to each other. At which point i realized that i've never really known how much faster using straight handlers is than using one of the CGI emulation layers. I also didn't have any idea how much faster SetHandler modperl was vs SetHandler perl-script. So i decided to see what i could figure out. I took gozers CGI from the slides (slightly modified) and ran it through the paces on my laptop, then converted the script to run as a straight handler. here's the CGI version: #!/usr/bin/perl print qq[Content-Type: text/html\r\n\r\n]; print(qq[ <html><body> <h1>Hello Worlds</h1> <pre> GATEWAY_INTERFACE: $ENV{GATEWAY_INTERFACE} MOD_PERL: $ENV{MOD_PERL} </pre> </body></html> ]); Here's the Handler version package Kabob::HelloWorld; use strict; use warnings; use Apache2::RequestRec (); use Apache2::Const -compile =>qw(:common); sub handler { my $r = shift; $r->content_type('text/html'); $r->print(qq[ <html><body> <h1>Hello Worlds</h1> <pre> GATEWAY_INTERFACE: $ENV{GATEWAY_INTERFACE} MOD_PERL: $ENV{MOD_PERL} </pre> </body></html> ]); return Apache2::Const::OK; } 1; and here's the conf (these tests were all running through a light mod_proxy front end too) <Location /cgi/> <IfDefine FrontEnd> ProxyPass http://localhost:8080/cgi/ ProxyPassReverse http://localhost:8080/cgi/ </IfDefine> </Location> <IfDefine BackEnd> ScriptAlias /cgi/ /www/p/ </IfDefine> Alias /perlrun/ /www/p/ <Location /perlrun/> <IfDefine FrontEnd> ProxyPass http://localhost:8080/perlrun/ ProxyPassReverse http://localhost:8080/perlrun/ </IfDefine> <IfDefine BackEnd> SetHandler perl-script PerlHandler ModPerl::PerlRun Options +ExecCGI PerlSendHeader On </IfDefine> </Location> Alias /registry/ /www/p/ <Location /registry/> <IfDefine FrontEnd> ProxyPass http://localhost:8080/registry/ ProxyPassReverse http://localhost:8080/registry/ </IfDefine> <IfDefine BackEnd> SetHandler perl-script PerlHandler ModPerl::Registry Options +ExecCGI PerlSendHeader On </IfDefine> </Location> <Location /perlscript/> <IfDefine FrontEnd> ProxyPass http://localhost:8080/perlscript/ ProxyPassReverse http://localhost:8080/perlscript/ </IfDefine> <IfDefine BackEnd> SetHandler perl-script PerlResponseHandler Kabob::HelloWorld </IfDefine> </Location> <Location /modperl/> <IfDefine FrontEnd> ProxyPass http://localhost:8080/modperl/ ProxyPassReverse http://localhost:8080/modperl/ </IfDefine> <IfDefine BackEnd> SetHandler modperl PerlResponseHandler Kabob::HelloWorld </IfDefine> </Location> and here's the results (which are no doubt flawed for a number of reasons) running: ab -n 10000 [url] CGI Requests per second: 217.80 [#/sec] (mean) Time per request: 4.591 [ms] (mean) Transfer rate: 53.17 [Kbytes/sec] received PerlRun Requests per second: 482.49 [#/sec] (mean) Time per request: 2.073 [ms] (mean) Transfer rate: 114.49 [Kbytes/sec] received Registry Requests per second: 693.33 [#/sec] (mean) Time per request: 1.442 [ms] (mean) Transfer rate: 164.53 [Kbytes/sec] received SetHandler perl-script Requests per second: 772.12 [#/sec] (mean) Time per request: 1.295 [ms] (mean) Transfer rate: 189.94 [Kbytes/sec] received SetHandler modperl Requests per second: 1048.66 [#/sec] (mean) Time per request: 0.954 [ms] (mean) Transfer rate: 250.84 [Kbytes/sec] received I'm not sure how well you can really compare the CGI emulation numbers to the PerlHandler numbers, but personally i think the 30%ish improvement from perl-script to modperl is pretty amazing. I wouldn't have imagined it would have been that high. Adam |
|
|
Re: some flawed benchmarks
by Dodger-3
::
Rate this Message:
I appreciate this, as I'd been wondering.
But it also prompts me to.. I gotta ask... No offence but... Don't you know what a here_doc is? -- Dodger 2008/7/9 <adam.prime@...>: > A couple of months ago i was going through slides from gozers "From CGI to > mod_perl 2.0, Fast!" talk, which has some benchmarks comparing CGI, perlrun > and registry to each other. At which point i realized that i've never > really known how much faster using straight handlers is than using one of > the CGI emulation layers. I also didn't have any idea how much faster > SetHandler modperl was vs SetHandler perl-script. > > So i decided to see what i could figure out. I took gozers CGI from the > slides (slightly modified) and ran it through the paces on my laptop, then > converted the script to run as a straight handler. > > here's the CGI version: > > #!/usr/bin/perl > > > print qq[Content-Type: text/html\r\n\r\n]; > > print(qq[ > <html><body> > <h1>Hello Worlds</h1> > <pre> > GATEWAY_INTERFACE: $ENV{GATEWAY_INTERFACE} > MOD_PERL: $ENV{MOD_PERL} > </pre> > </body></html> > > ]); > > > Here's the Handler version > > package Kabob::HelloWorld; > > use strict; > use warnings; > > use Apache2::RequestRec (); > > use Apache2::Const -compile =>qw(:common); > > sub handler { > my $r = shift; > > $r->content_type('text/html'); > $r->print(qq[ > <html><body> > <h1>Hello Worlds</h1> > <pre> > GATEWAY_INTERFACE: $ENV{GATEWAY_INTERFACE} > MOD_PERL: $ENV{MOD_PERL} > </pre> > </body></html> > > > ]); > > return Apache2::Const::OK; > } > > 1; > > and here's the conf (these tests were all running through a light mod_proxy > front end too) > > > <Location /cgi/> > <IfDefine FrontEnd> > ProxyPass http://localhost:8080/cgi/ > ProxyPassReverse http://localhost:8080/cgi/ > </IfDefine> > </Location> > <IfDefine BackEnd> > ScriptAlias /cgi/ /www/p/ > </IfDefine> > > > Alias /perlrun/ /www/p/ > <Location /perlrun/> > <IfDefine FrontEnd> > ProxyPass http://localhost:8080/perlrun/ > ProxyPassReverse http://localhost:8080/perlrun/ > </IfDefine> > <IfDefine BackEnd> > SetHandler perl-script > PerlHandler ModPerl::PerlRun > Options +ExecCGI > PerlSendHeader On > </IfDefine> > </Location> > > Alias /registry/ /www/p/ > <Location /registry/> > <IfDefine FrontEnd> > ProxyPass http://localhost:8080/registry/ > ProxyPassReverse http://localhost:8080/registry/ > </IfDefine> > <IfDefine BackEnd> > SetHandler perl-script > PerlHandler ModPerl::Registry > Options +ExecCGI > PerlSendHeader On > </IfDefine> > </Location> > > > > <Location /perlscript/> > <IfDefine FrontEnd> > ProxyPass http://localhost:8080/perlscript/ > ProxyPassReverse http://localhost:8080/perlscript/ > </IfDefine> > <IfDefine BackEnd> > SetHandler perl-script > PerlResponseHandler Kabob::HelloWorld > </IfDefine> > </Location> > > <Location /modperl/> > <IfDefine FrontEnd> > ProxyPass http://localhost:8080/modperl/ > ProxyPassReverse http://localhost:8080/modperl/ > </IfDefine> > <IfDefine BackEnd> > SetHandler modperl > PerlResponseHandler Kabob::HelloWorld > </IfDefine> > </Location> > > > and here's the results (which are no doubt flawed for a number of reasons) > > running: ab -n 10000 [url] > > CGI > Requests per second: 217.80 [#/sec] (mean) > Time per request: 4.591 [ms] (mean) > Transfer rate: 53.17 [Kbytes/sec] received > > PerlRun > Requests per second: 482.49 [#/sec] (mean) > Time per request: 2.073 [ms] (mean) > Transfer rate: 114.49 [Kbytes/sec] received > > Registry > Requests per second: 693.33 [#/sec] (mean) > Time per request: 1.442 [ms] (mean) > Transfer rate: 164.53 [Kbytes/sec] received > > SetHandler perl-script > Requests per second: 772.12 [#/sec] (mean) > Time per request: 1.295 [ms] (mean) > Transfer rate: 189.94 [Kbytes/sec] received > > SetHandler modperl > Requests per second: 1048.66 [#/sec] (mean) > Time per request: 0.954 [ms] (mean) > Transfer rate: 250.84 [Kbytes/sec] received > > I'm not sure how well you can really compare the CGI emulation numbers to > the PerlHandler numbers, but personally i think the 30%ish improvement from > perl-script to modperl is pretty amazing. I wouldn't have imagined it would > have been that high. > > Adam > > > > > > > > > -- Dodger |
|
|
Re: some flawed benchmarks
by Dodger-3
::
Rate this Message:
Oh. I would also recommend three variants, based on what people often
do, what people sometimes do, and what people probably should do when using CGI.pm, which can make a difference (just for thoroughness): Usually done: #!/usr/bin/perl use CGI; print header; print <<"EOF"; <html> <body> <h1>Environment dump:</h1> <dl> @{[map "<dt>$_</dt>\n<dd>$ENV{$_}</dd>\n", sort keys %ENV]} </dl> </body> </html> EOF Sometimes do: #!/usr/bin/perl use strict; use CGI; my $cgi = new CGI; print $cgi->header; print <<"EOF"; <html> <body> <h1>Environment dump:</h1> <dl> @{[map "<dt>$_</dt>\n<dd>$ENV{$_}</dd>\n", sort keys %ENV]} </dl> </body> </html> EOF Might do occassionally, and probably should do all the time if using CGI: #!/usr/bin/perl use strict; use CGI(); # note the difference -- using CGI in OO mode, don't import *anything* my $cgi = new CGI; print $cgi->header; print <<"EOF"; <html> <body> <h1>Environment dump:</h1> <dl> @{[map "<dt>$_</dt>\n<dd>$ENV{$_}</dd>\n", sort keys %ENV]} </dl> </body> </html> EOF 2008/7/9 <adam.prime@...>: > A couple of months ago i was going through slides from gozers "From CGI to > mod_perl 2.0, Fast!" talk, which has some benchmarks comparing CGI, perlrun > and registry to each other. At which point i realized that i've never > really known how much faster using straight handlers is than using one of > the CGI emulation layers. I also didn't have any idea how much faster > SetHandler modperl was vs SetHandler perl-script. > > So i decided to see what i could figure out. I took gozers CGI from the > slides (slightly modified) and ran it through the paces on my laptop, then > converted the script to run as a straight handler. > > here's the CGI version: > > #!/usr/bin/perl > > > print qq[Content-Type: text/html\r\n\r\n]; > > print(qq[ > <html><body> > <h1>Hello Worlds</h1> > <pre> > GATEWAY_INTERFACE: $ENV{GATEWAY_INTERFACE} > MOD_PERL: $ENV{MOD_PERL} > </pre> > </body></html> > > ]); > > > Here's the Handler version > > package Kabob::HelloWorld; > > use strict; > use warnings; > > use Apache2::RequestRec (); > > use Apache2::Const -compile =>qw(:common); > > sub handler { > my $r = shift; > > $r->content_type('text/html'); > $r->print(qq[ > <html><body> > <h1>Hello Worlds</h1> > <pre> > GATEWAY_INTERFACE: $ENV{GATEWAY_INTERFACE} > MOD_PERL: $ENV{MOD_PERL} > </pre> > </body></html> > > > ]); > > return Apache2::Const::OK; > } > > 1; > > and here's the conf (these tests were all running through a light mod_proxy > front end too) > > > <Location /cgi/> > <IfDefine FrontEnd> > ProxyPass http://localhost:8080/cgi/ > ProxyPassReverse http://localhost:8080/cgi/ > </IfDefine> > </Location> > <IfDefine BackEnd> > ScriptAlias /cgi/ /www/p/ > </IfDefine> > > > Alias /perlrun/ /www/p/ > <Location /perlrun/> > <IfDefine FrontEnd> > ProxyPass http://localhost:8080/perlrun/ > ProxyPassReverse http://localhost:8080/perlrun/ > </IfDefine> > <IfDefine BackEnd> > SetHandler perl-script > PerlHandler ModPerl::PerlRun > Options +ExecCGI > PerlSendHeader On > </IfDefine> > </Location> > > Alias /registry/ /www/p/ > <Location /registry/> > <IfDefine FrontEnd> > ProxyPass http://localhost:8080/registry/ > ProxyPassReverse http://localhost:8080/registry/ > </IfDefine> > <IfDefine BackEnd> > SetHandler perl-script > PerlHandler ModPerl::Registry > Options +ExecCGI > PerlSendHeader On > </IfDefine> > </Location> > > > > <Location /perlscript/> > <IfDefine FrontEnd> > ProxyPass http://localhost:8080/perlscript/ > ProxyPassReverse http://localhost:8080/perlscript/ > </IfDefine> > <IfDefine BackEnd> > SetHandler perl-script > PerlResponseHandler Kabob::HelloWorld > </IfDefine> > </Location> > > <Location /modperl/> > <IfDefine FrontEnd> > ProxyPass http://localhost:8080/modperl/ > ProxyPassReverse http://localhost:8080/modperl/ > </IfDefine> > <IfDefine BackEnd> > SetHandler modperl > PerlResponseHandler Kabob::HelloWorld > </IfDefine> > </Location> > > > and here's the results (which are no doubt flawed for a number of reasons) > > running: ab -n 10000 [url] > > CGI > Requests per second: 217.80 [#/sec] (mean) > Time per request: 4.591 [ms] (mean) > Transfer rate: 53.17 [Kbytes/sec] received > > PerlRun > Requests per second: 482.49 [#/sec] (mean) > Time per request: 2.073 [ms] (mean) > Transfer rate: 114.49 [Kbytes/sec] received > > Registry > Requests per second: 693.33 [#/sec] (mean) > Time per request: 1.442 [ms] (mean) > Transfer rate: 164.53 [Kbytes/sec] received > > SetHandler perl-script > Requests per second: 772.12 [#/sec] (mean) > Time per request: 1.295 [ms] (mean) > Transfer rate: 189.94 [Kbytes/sec] received > > SetHandler modperl > Requests per second: 1048.66 [#/sec] (mean) > Time per request: 0.954 [ms] (mean) > Transfer rate: 250.84 [Kbytes/sec] received > > I'm not sure how well you can really compare the CGI emulation numbers to > the PerlHandler numbers, but personally i think the 30%ish improvement from > perl-script to modperl is pretty amazing. I wouldn't have imagined it would > have been that high. > > Adam > > > > > > > > > -- Dodger |
|
|
Re: some flawed benchmarks
by Andy Armstrong
::
Rate this Message:
On 10 Jul 2008, at 06:28, adam.prime@... wrote:
> I'm not sure how well you can really compare the CGI emulation > numbers to the PerlHandler numbers, but personally i think the > 30%ish improvement from perl-script to modperl is pretty amazing. I > wouldn't have imagined it would have been that high. It would be interesting to see how FCGI compares to those numbers. -- Andy Armstrong, Hexten |
|
|
Re: some flawed benchmarks
by adam.prime
::
Rate this Message:
I deliberately removed CGI from the script because i personally would
never use CGI in something written to be run as straight handlers, and it obviously wouldn't make any sense to use CGI in the CGI emulations, and then not use it in the Handler version. Not using heredoc's shouldn't really have any effect on the (already flawed) results, because they weren't used in any of the examples. But yeah, it was late and i should have been sleeping but i was dorking around with this instead. I'm personally not really interested in how using CGI affects the numbers, but if other people would like to see them i can do this later tonight. Adam Dodger wrote: > Oh. I would also recommend three variants, based on what people often > do, what people sometimes do, and what people probably should do when > using CGI.pm, which can make a difference (just for thoroughness): > > Usually done: > #!/usr/bin/perl > use CGI; > print header; > > print <<"EOF"; > <html> > <body> > <h1>Environment dump:</h1> > <dl> > @{[map "<dt>$_</dt>\n<dd>$ENV{$_}</dd>\n", sort keys %ENV]} > </dl> > </body> > </html> > EOF > > Sometimes do: > #!/usr/bin/perl > use strict; > use CGI; > my $cgi = new CGI; > print $cgi->header; > > print <<"EOF"; > <html> > <body> > <h1>Environment dump:</h1> > <dl> > @{[map "<dt>$_</dt>\n<dd>$ENV{$_}</dd>\n", sort keys %ENV]} > </dl> > </body> > </html> > EOF > > Might do occassionally, and probably should do all the time if using CGI: > > #!/usr/bin/perl > use strict; > use CGI(); # note the difference -- using CGI in OO mode, don't import > *anything* > my $cgi = new CGI; > print $cgi->header; > > print <<"EOF"; > <html> > <body> > <h1>Environment dump:</h1> > <dl> > @{[map "<dt>$_</dt>\n<dd>$ENV{$_}</dd>\n", sort keys %ENV]} > </dl> > </body> > </html> > EOF > |
|
|
Re: some flawed benchmarks
by adam.prime
::
Rate this Message:
Andy Armstrong wrote:
> It would be interesting to see how FCGI compares to those numbers. I don't know anything about fastcgi, but i suppose i could look at that also this evening. Adam |
|
|
Re: some flawed benchmarks
by Perrin Harkins
::
Rate this Message:
On Thu, Jul 10, 2008 at 1:28 AM, <adam.prime@...> wrote:
> and here's the conf (these tests were all running through a light mod_proxy > front end too) Note that CGI and FastCGI don't need the proxy frontend. > I'm not sure how well you can really compare the CGI emulation numbers to > the PerlHandler numbers, but personally i think the 30%ish improvement from > perl-script to modperl is pretty amazing. I wouldn't have imagined it would > have been that high. It is pretty cool that it works so well. I feel like I should point out though, for the benefit of those using Registry, that the difference is only this big because the code isn't doing anything. In a real-world scenario you'd see an improvement, but nothing close to 30%. - Perrin |
|
|
Re: some flawed benchmarks
by adam.prime
::
Rate this Message:
Quoting Perrin Harkins <perrin@...>:
> Note that CGI and FastCGI don't need the proxy frontend. The only reason I did it that way was because that's how apache was already set up on my laptop, and i didn't feel like dorking around with it too much. I could certainly change it around so that CGI and FGCGI (if added) were handled by the light apache. I wasn't sure how apples to oranges that would be, but i guess in reality, people that are running plain CGI or fast CGI probably aren't doing it on a mod_perl enabled server. > It is pretty cool that it works so well. I feel like I should point > out though, for the benefit of those using Registry, that the > difference is only this big because the code isn't doing anything. In > a real-world scenario you'd see an improvement, but nothing close to > 30%. It's probably also worth noting that under sethandler modperl the GATEWAY_INTERFACE env variable has no value, which results in the page being 3 bytes shorter (iirc), which is actually a pretty big deal since the whole page is pretty short. The examples should likely be modified to actually return pages that are the same length. anything else stupid i'm missing? adam |
|
|
|
|
|
Re: some flawed benchmarks
by Perrin Harkins
::
Rate this Message:
On Thu, Jul 10, 2008 at 11:43 PM, <adam.prime@...> wrote:
> If anyone's got any ideas about why the proxy is having as significant an > impact as it is, i'd love to tweak it. There are some proxy settings in recent mod_proxy versions (apache 2.2) which you could experiment with. I haven't tried them yet. I think you can use persistent connections to the backend with this setup, since it can pool them with the worker MPM. You could also experiment with the config on the mod_perl server. For the most part, if you run with concurrency below MaxClients and you do a warmup run before each benchmark, it should mitigate common config issues. > Also worth noting is that the front > end is running worker, the backend is running prefork. That's a good setup. Some people use lighttpd, perlbal, nginx, etc. for proxying. If you feel ambitious you could try one of those. - Perrin |
| Free Forum Powered by Nabble | Forum Help |