Writing robust selenium tests for qooxdoo

View: New views
5 Messages — Rating Filter:   Alert me  

Writing robust selenium tests for qooxdoo

by stefan.hansel :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


Hi there,

as there seems to be some qooxdoo-related selenium knowledge available here, I hope someone might give some useful hints.

Currently I'm trying to write some Selenium tests with Selenium IDE for a RAP-application and I have my first testcases working (thanks to the selenium-qooxdoo-extensions).
The problem is - by 'working' I mean - it's only working when running the script very slow or inserting 'pause' - steps.

I want to replace this arbitrary 'pause'-steps with robust waiting.

Let's just explain my problems with a very simple use-case:
'starting a RAP-Application, and then clicking on just one button'

Behind the scenes the following is happening:
1) URL is entered into the browser, RAP-HTML-Page ist loaded with default qx-library-Script, Main-RAP-Class is started.
2) MainClass creates a first request to the serverside (using qx.io.remote.Request), the serverside sends back a qooxdoo script with the initial GUI
3a) this qooxdoo-script is evaluated and
3b) the initial widgets are created.
4) I want to click a button.

The first problem of course is waiting for the remote request to finish (step2). Actually this is not too hard, because in RAP there is a qx-singleton (org.eclipse.swt.Request) which keeps a counter of all running request. So I just have to wait for this counter to be 0 again.
I also have to wait until the qooxdoo-script is evaluated (3a), but I have luck - the request counter is only set back after the script from the server was evaluated !

My Selenium-Script thus looks like this:
open rapApp/start
waitForCondition var tmp=selenium.browserbot.getCurrentWindow();tmp.wrappedJSObject.org.eclipse.swt.WidgetManager.$$instance 10000
waitForCondition var tmp=selenium.browserbot.getCurrentWindow();tmp.wrappedJSObject.org.eclipse.swt.Request.$$instance._runningRequestCount==0 10000
qxClick //img[@src="images//16x16/calendar.png"]


(The first 'waitForCondition' checks for something - creating another RAP-singleton - that happens in step 3a, just to make sure that we are really within step 3a)
Nevertheless ... even with those two waits, 'qxClick' is still too fast and not finding the image without a prior 'pause'.

Could it be that qooxdoos widget queues are interfering ?
I'm quite sure, that with the script above I correctly wait for the qx-script to be evaluated (3a) - but ... isn't qooxdoo deferring/delaying all things that manipulate the browser DOM  (i.e. qx.ui.core.Widget._globalWidgetQueue in qooxdoo 0.7.3) ?

I think that even in non-RAP applications these queue's could make testing complicated, if for instance a click on a button opens a dialog ?
Would it make sense to provide a 'waitForQxIdle()' function, that somehow checks, that all queues are empty ?

How are other people making their selenium-tests wait robustly ?

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
qooxdoo-devel@...
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel

Re: Writing robust selenium tests for qooxdoo

by Fabian Jakobs-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

stefan.hansel@... schrieb:

>
> Hi there,
>
> as there seems to be some qooxdoo-related selenium knowledge available
> here, I hope someone might give some useful hints.
>
> Currently I'm trying to write some Selenium tests with Selenium IDE
> for a RAP-application and I have my first testcases working (thanks to
> the selenium-qooxdoo-extensions).
> The problem is - by 'working' I mean - it's only working when running
> the script very slow or inserting 'pause' - steps.
>
> I want to replace this arbitrary 'pause'-steps with robust waiting.
>
> Let's just explain my problems with a very simple use-case:
> 'starting a RAP-Application, and then clicking on just one button'
>
> Behind the scenes the following is happening:
> 1) URL is entered into the browser, RAP-HTML-Page ist loaded with
> default qx-library-Script, Main-RAP-Class is started.
> 2) MainClass creates a first request to the serverside (using
> qx.io.remote.Request), the serverside sends back a qooxdoo script with
> the initial GUI
> 3a) this qooxdoo-script is evaluated and
> 3b) the initial widgets are created.
> 4) I want to click a button.
>
> The first problem of course is waiting for the remote request to
> finish (step2). Actually this is not too hard, because in RAP there is
> a qx-singleton (org.eclipse.swt.Request) which keeps a counter of all
> running request. So I just have to wait for this counter to be 0 again.
> I also have to wait until the qooxdoo-script is evaluated (3a), but I
> have luck - the request counter is only set back after the script from
> the server was evaluated !
>
> My Selenium-Script thus looks like this:
> open rapApp/start
> waitForCondition var
> tmp=selenium.browserbot.getCurrentWindow();tmp.wrappedJSObject.org.eclipse.swt.WidgetManager.$$instance
> 10000
> waitForCondition var
> tmp=selenium.browserbot.getCurrentWindow();tmp.wrappedJSObject.org.eclipse.swt.Request.$$instance._runningRequestCount==0
> 10000
> qxClick //img[@src="images//16x16/calendar.png"]
>
>
>
> (The first 'waitForCondition' checks for something - creating another
> RAP-singleton - that happens in step 3a, just to make sure that we are
> really within step 3a)
> Nevertheless ... even with those two waits, 'qxClick' is still too
> fast and not finding the image without a prior 'pause'.
>
> Could it be that qooxdoos widget queues are interfering ?
> I'm quite sure, that with the script above I correctly wait for the
> qx-script to be evaluated (3a) - but ... isn't qooxdoo
> deferring/delaying all things that manipulate the browser DOM  (i.e.
> qx.ui.core.Widget._globalWidgetQueue in qooxdoo 0.7.3) ?
Yes qooxdoo queues pretty much all widget and DOM related changes. You
could try to flush the queues manually by calling
"qx.ui.core.Widget.flushGlobalQueues()". This way you don't have to wait
for the asynchronous automatic flush.

> I think that even in non-RAP applications these queue's could make
> testing complicated, if for instance a click on a button opens a dialog ?
> Would it make sense to provide a 'waitForQxIdle()' function, that
> somehow checks, that all queues are empty ?
This is definitely a good idea. I don't know enough about Selenium to
suggest a way this could be implemented. Thomas any idea?

> How are other people making their selenium-tests wait robustly ?


Best Fabian


--
Fabian Jakobs
JavaScript Framework Developer

1&1 Internet AG
Brauerstraße 48
76135 Karlsruhe

Amtsgericht Montabaur HRB 6484

Vorstand: Henning Ahlert, Ralph Dommermuth, Matthias Ehrlich, Thomas Gottschlich, Matthias Greve, Robert Hoffmann, Markus Huhn, Oliver Mauss, Achim Weiss
Aufsichtsratsvorsitzender: Michael Scheeren


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
qooxdoo-devel@...
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel

Re: Writing robust selenium tests for qooxdoo

by thron7 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message



Fabian Jakobs wrote:

> stefan.hansel@... schrieb:
>  
>> Hi there,
>>
>> as there seems to be some qooxdoo-related selenium knowledge available
>> here, I hope someone might give some useful hints.
>>
>> Currently I'm trying to write some Selenium tests with Selenium IDE
>> for a RAP-application and I have my first testcases working (thanks to
>> the selenium-qooxdoo-extensions).
>> The problem is - by 'working' I mean - it's only working when running
>> the script very slow or inserting 'pause' - steps.
>>
>> I want to replace this arbitrary 'pause'-steps with robust waiting.
>>
>> Let's just explain my problems with a very simple use-case:
>> 'starting a RAP-Application, and then clicking on just one button'
>>
>> Behind the scenes the following is happening:
>> 1) URL is entered into the browser, RAP-HTML-Page ist loaded with
>> default qx-library-Script, Main-RAP-Class is started.
>> 2) MainClass creates a first request to the serverside (using
>> qx.io.remote.Request), the serverside sends back a qooxdoo script with
>> the initial GUI
>> 3a) this qooxdoo-script is evaluated and
>> 3b) the initial widgets are created.
>> 4) I want to click a button.
>>
>> The first problem of course is waiting for the remote request to
>> finish (step2). Actually this is not too hard, because in RAP there is
>> a qx-singleton (org.eclipse.swt.Request) which keeps a counter of all
>> running request. So I just have to wait for this counter to be 0 again.
>> I also have to wait until the qooxdoo-script is evaluated (3a), but I
>> have luck - the request counter is only set back after the script from
>> the server was evaluated !
>>
>> My Selenium-Script thus looks like this:
>> open rapApp/start
>> waitForCondition var
>> tmp=selenium.browserbot.getCurrentWindow();tmp.wrappedJSObject.org.eclipse.swt.WidgetManager.$$instance
>> 10000
>> waitForCondition var
>> tmp=selenium.browserbot.getCurrentWindow();tmp.wrappedJSObject.org.eclipse.swt.Request.$$instance._runningRequestCount==0
>> 10000
>> qxClick //img[@src="images//16x16/calendar.png"]
>>
>>
>>
>> (The first 'waitForCondition' checks for something - creating another
>> RAP-singleton - that happens in step 3a, just to make sure that we are
>> really within step 3a)
>> Nevertheless ... even with those two waits, 'qxClick' is still too
>> fast and not finding the image without a prior 'pause'.
>>    

I'm not sure since I don't have much experience with Selenium
assertions. But your approach seems quite cumbersome. Have you
considered to e.g. use the 'waitForVisible()' assertion? You could check
for the //img[@src="images//16x16/calendar.png" itself, thereby directly
checking for the button to appear before clicking.

>> Could it be that qooxdoos widget queues are interfering ?
>> I'm quite sure, that with the script above I correctly wait for the
>> qx-script to be evaluated (3a) - but ... isn't qooxdoo
>> deferring/delaying all things that manipulate the browser DOM  (i.e.
>> qx.ui.core.Widget._globalWidgetQueue in qooxdoo 0.7.3) ?
>>    
> Yes qooxdoo queues pretty much all widget and DOM related changes. You
> could try to flush the queues manually by calling
> "qx.ui.core.Widget.flushGlobalQueues()". This way you don't have to wait
> for the asynchronous automatic flush.
>
>  
>> I think that even in non-RAP applications these queue's could make
>> testing complicated, if for instance a click on a button opens a dialog ?
>> Would it make sense to provide a 'waitForQxIdle()' function, that
>> somehow checks, that all queues are empty ?
>>    
> This is definitely a good idea. I don't know enough about Selenium to
> suggest a way this could be implemented. Thomas any idea?
>  

If there is a qooxdoo method to check the queues, this shouldn't be a
problem. But you are actually the first to report such an issue.

>  
>> How are other people making their selenium-tests wait robustly ?
>>    

Maybe Hugh and his guys can comment on this?!

Thomas

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
qooxdoo-devel@...
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel

Re: Writing robust selenium tests for qooxdoo

by Hugh Gibson :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

> >> How are other people making their selenium-tests wait robustly ?
> >>    
>
> Maybe Hugh and his guys can comment on this?!

Calling Chi...

Hugh

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
qooxdoo-devel@...
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel

Re: Writing robust selenium tests for qooxdoo

by stefan.hansel :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

>> I'm not sure since I don't have much experience with Selenium
>> assertions. But your approach seems quite cumbersome. Have you
>> considered to e.g. use the 'waitForVisible()' assertion?

Yes - this might be working ... but I had to do it all the time with
different assertions. 'waitForIdle' at least tells from the name what I'm
trying to do and could be implemented in a generic way.

>> If there is a qooxdoo method to check the queues, this shouldn't be a
problem.
If there was a method I could have called it ;-) ...

Currently I use the following to wait for the qooxdoo-queues (as a proof
of concept) and it seems to work.
Of course it could be that these two statements change timing anyway so
that it's a selffulfilling prophecy.

waitForCondition
var
tmp=selenium.browserbot.getCurrentWindow();tmp.wrappedJSObject.qx.ui.core.Widget._fastGlobalDisplayQueue.length==0
10000
waitForCondition
var tmp=selenium.browserbot.getCurrentWindow();var lazyQueueEmpty=true;for
(vKey in tmp.wrappedJSObject.qx.ui.core.Widget._lazyGlobalDisplayQueues)
{lazyQueueEmpty=false;break;};lazyQueueEmpty
10000
(Don't be confused - this is 0.7.3 qooxdoo If you still remember it ;-) )

As all this stuff looks very awful, that's why I thought about putting
such stuff in the js-extension.

>> But you are actually the first to report such an issue.
Let's wait some more, maybe we get other useful strategies.




-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
qooxdoo-devel mailing list
qooxdoo-devel@...
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
LightInTheBox - Buy quality products at wholesale price!