|
»
»
»
Need suggestions
|
View:
New views
4 Messages
—
Rating Filter:
Alert me
|
 |
Need suggestions
by anil_jacob
::
Rate this Message:
I would like to get some feed back on collecting grinder results and graphing and what everyone else does. I am currently running grinder via ANT and increment Threads in blocks of 25, for each 25 users I stop grinder, run a script to total up number of tests and divide by the duration of the run this is the Transactions/sec(TPS) and also get average response time and write them to a Summary file. I do this for each step size of 25 upto the maximum number of Threads my system can handle. Then I run another script which reads the TPS numbers for each set of users from Summary file and creates a graph. This has worked quite well. Is there a different way to do it? One idea was to ramp up users within grinder by using sleep and then use GrinderAnalyser to chart. However I don't see the graph that I am expecting i.e TPS going up with the user ramp up and then stabilizing. Any suggestions? AJ ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ grinder-use mailing list grinder-use@... https://lists.sourceforge.net/lists/listinfo/grinder-use |
|
|
Re: Need suggestions
by RUSSELL Scott-2
::
Rate this Message:
AJ,
It is interesting to know what you are doing. I am doing mostly performance/scalability testing, so I probably have a different emphasis. Firstly, I do not look at all at TPS. It is an industry obsession, to me that misses the point totally. Here is what I do, and what I look at. 1. Generally, I run a test for 2hours. I use a fixed number of users and simply loop them. Normally, I would be running 3 or 4 scripts simultaneously, with an agreed percentage of each in advance. 2. Now I would 4, 2hr tests with, 50 users, 100 users, 200 users and maybe 500 users(or more depending on customer requirements. 3. Once I have the results, I would separate static(ie apache) and dynamic(ie weblogic, jboss) test results, and make graphs accordingly. Currently I use a spreadsheet, from the summary results, but I can also use Calum's postgres db thingy with graphing in pdf format, available here(http://sourceforge.net/projects/ground/). But to be honest, I only use it for step 5 below. 4. I would tend to make comparative graphs, usually, of the 50, 100, 200, 500 users response times. Normally, I would concentrate on the dynamic content, as you can usually make the apache response so fast that it is not worth analysing(your mileage may vary). Typically, I would graph response times, for various tests/url's versus number of users(50, 100, 200, 500). I may also graph, cpu, memory or network bandwidth. 5. I also concentrate on the outlyers, or worst responding urls. This would be from the raw client data. This is to avoid the random user reporting your site as "slow". I try to narrow the distribution as far as possible to make sure the worst case outlyers are not much worse than the mean. It may mean slowing every url by a second or so, but generally results in a much more even user experience. (ie everyone thinks the response times are OK). No random joe's with performance issues. 6. Throughput. I like to think of this in terms of questions you would ask the curator of a museum. You would never ask:- "The mona lisa is a wonderful painting, for your visitors, how many transactions per second do you have on that painting?". I regard web applications in exactly the same way. It is not about how many people look at one url. It is about how many visitors(of a given type) per hour you can handle. Refine this a little further, and you can quote this figure for each script. This gives a nice number that managers can understand, and people with simple arithmetic, and a basic business understanding of what your scripts do, can make calculations that are sensible and easy. 7. Errors. Often the most ignored parameter in any test. I like to see less than 0.5% errors across a 2hour test period. I will constantly run tests, and tweak scripts and have developers fix code until, I get down to this figure. It may seem impossible on some projects, but generally I have found it not to be the case. Low error rates, mean good repeatability, which means good data for testing. Anyway, hope that helps a little. Comments appreciated. Sc0tt.... -----Original Message----- From: anil_jacob@... [mailto:anil_jacob@...] Sent: Tuesday, September 23, 2008 10:41 PM To: grinder-use@... Subject: [Grinder-use] Need suggestions I would like to get some feed back on collecting grinder results and graphing and what everyone else does. I am currently running grinder via ANT and increment Threads in blocks of 25, for each 25 users I stop grinder, run a script to total up number of tests and divide by the duration of the run this is the Transactions/sec(TPS) and also get average response time and write them to a Summary file. I do this for each step size of 25 upto the maximum number of Threads my system can handle. Then I run another script which reads the TPS numbers for each set of users from Summary file and creates a graph. This has worked quite well. Is there a different way to do it? One idea was to ramp up users within grinder by using sleep and then use GrinderAnalyser to chart. However I don't see the graph that I am expecting i.e TPS going up with the user ramp up and then stabilizing. Any suggestions? AJ ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ grinder-use mailing list grinder-use@... https://lists.sourceforge.net/lists/listinfo/grinder-use ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ grinder-use mailing list grinder-use@... https://lists.sourceforge.net/lists/listinfo/grinder-use |
|
|
Re: Need suggestions
by RUSSELL Scott-2
::
Rate this Message:
Aj,
I am interested in doing something similar to your methodology, any possibility you are willing to share your code? Thanks Sc0tt... -----Original Message----- From: anil_jacob@... [mailto:anil_jacob@...] Sent: Tuesday, September 23, 2008 10:41 PM To: grinder-use@... Subject: [Grinder-use] Need suggestions I would like to get some feed back on collecting grinder results and graphing and what everyone else does. I am currently running grinder via ANT and increment Threads in blocks of 25, for each 25 users I stop grinder, run a script to total up number of tests and divide by the duration of the run this is the Transactions/sec(TPS) and also get average response time and write them to a Summary file. I do this for each step size of 25 upto the maximum number of Threads my system can handle. Then I run another script which reads the TPS numbers for each set of users from Summary file and creates a graph. This has worked quite well. Is there a different way to do it? One idea was to ramp up users within grinder by using sleep and then use GrinderAnalyser to chart. However I don't see the graph that I am expecting i.e TPS going up with the user ramp up and then stabilizing. Any suggestions? AJ ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ grinder-use mailing list grinder-use@... https://lists.sourceforge.net/lists/listinfo/grinder-use ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ grinder-use mailing list grinder-use@... https://lists.sourceforge.net/lists/listinfo/grinder-use |
|
|
|
| Free Forum Powered by Nabble | Forum Help |
