And as if by magic, Geoserver doubled its WFS serving speed :-)

[aaime]

Hi all,
I bring very good news.
Todays I was thinking about the issue reported by GeoUser, the WFS
slowness, and also about one Chris blog, both reporting to the
information that we do serve GML at 4MB/s.

And I was think, that can't be right, we should be faster than that,
4MB/s is slow, but I could not pinpoint the issue. Well, today I did.
I tried the SPEED strategy, and boom, Geoserver was going 8+MB/s.
The default strategy was twice as slow... so I profiled, and located
the issue: our partially buffering stream wasn't implementing the
write(byte[], int, int) method, which is key to good performance,
and only provided the single byte method.

Now, this difference is big, but we're late in the release cycle.
So, I implemented a new partial buffer strategy, partial-buffer2,
that can be configured in web.xml, but it's not the default. This will
allow users to try the new strategy out, but fall back on the safe and
tested partial-buffer one in case of trouble.

On, in case you needed more reasons to dump jdk 1.4 and try
out jdk 1.6, this is the serving rate with the new partial buffer
strategy with three different jdks:
* jdk 1.4.2_12:  4.1 MB/s
* jdk 1.5.0_09:  8.8 MB/s (that's the one I used in prelim tests).
* jkd 1.6.0   : 10.7 MB/s

Oh, btw, don't expect the same improvement with WMS serving, though
the improved buffer strategy will do some good to WMS too (sorry
I don't have raw numbers).

Bye bye
Andrea


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@...
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

[Randy George]

Hi Andrea,

* jkd 1.6.0   : 10.7 MB/s !!

        This is good news. I'm especially glad to see it set for the
Geoserver 1.5.0 release.

I'm also happy to see your work over at GDAL_Ogr patching the Java SWIG
wrappers for ogr. A bridge of this nature leverages a lot of work on
existing formats.

        Your expertise is greatly appreciated.

Thanks
Randy


-----Original Message-----
From: geoserver-devel-bounces@...
[mailto:geoserver-devel-bounces@...] On Behalf Of Andrea
Aime
Sent: Saturday, April 07, 2007 10:27 AM
To: Geoserver-devel
Subject: [Geoserver-devel] And as if by magic,Geoserver doubled its WFS
serving speed :-)

Hi all,
I bring very good news.
Todays I was thinking about the issue reported by GeoUser, the WFS
slowness, and also about one Chris blog, both reporting to the
information that we do serve GML at 4MB/s.

And I was think, that can't be right, we should be faster than that,
4MB/s is slow, but I could not pinpoint the issue. Well, today I did.
I tried the SPEED strategy, and boom, Geoserver was going 8+MB/s.
The default strategy was twice as slow... so I profiled, and located
the issue: our partially buffering stream wasn't implementing the
write(byte[], int, int) method, which is key to good performance,
and only provided the single byte method.

Now, this difference is big, but we're late in the release cycle.
So, I implemented a new partial buffer strategy, partial-buffer2,
that can be configured in web.xml, but it's not the default. This will
allow users to try the new strategy out, but fall back on the safe and
tested partial-buffer one in case of trouble.

On, in case you needed more reasons to dump jdk 1.4 and try
out jdk 1.6, this is the serving rate with the new partial buffer
strategy with three different jdks:
* jdk 1.4.2_12:  4.1 MB/s
* jdk 1.5.0_09:  8.8 MB/s (that's the one I used in prelim tests).
* jkd 1.6.0   : 10.7 MB/s

Oh, btw, don't expect the same improvement with WMS serving, though
the improved buffer strategy will do some good to WMS too (sorry
I don't have raw numbers).

Bye bye
Andrea


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@...
https://lists.sourceforge.net/lists/listinfo/geoserver-devel


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@...
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

[aaime]

Randy George ha scritto:
> Hi Andrea,
>
> * jkd 1.6.0   : 10.7 MB/s !!

Heh, hold the horses a second :-)
That result should not be taken as a general statement of
Geoserver performance, it's a very specific use case, to
reproduce it you got to use a ton (100.000)
of point geometries in a shapefile.

Today I tried with a big road network sitting in Postgis
with a lot more of coordinates and small attributes,
that does generate more or less the same amount of GML,
but the speed in JDK 1.6 is around 4MB/s.
I have other patches sitting on my disk to bring even
this one to around 8MB/s, but they are more profound and
extensive (JDK class replacements) and they will need to be tested
a lot more before hitting Geoserver for good.

That first change, in any cause, will bring goodness to all
services, but in general, a lot of gain if the operation is
basically moving a big amount of data and not doing any computation,
quite a few if on the other side the operation is CPU bound and
moves very few data (such as WMS is). The improvement can go
from 2 times in the luckiest case (the one I was reporting)
to maybe 10%-20% in WMS serving.

Cheers
Andrea

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@...
https://lists.sourceforge.net/lists/listinfo/geoserver-devel