[scala-tools] Debugging unit tests

[Rob Hasselbaum-4]

How do you approach running unit tests in the Eclipse debugger? I'd like to use ScalaTest or one of the other Scala test frameworks, but as far as I can tell, Eclipse integration isn't there yet. I assume I can use ScalaTest's integration with either TestNG or JUnit, but what do I lose compared to a "pure" ScalaTest solution? Thanks!

[Tony Sloane]

Hi Rob,

On 20/09/2008, at 3:15 PM, Rob Hasselbaum wrote:

How do you approach running unit tests in the Eclipse debugger? I'd like to use ScalaTest or one of the other Scala test frameworks, but as far as I can tell, Eclipse integration isn't there yet. I assume I can use ScalaTest's integration with either TestNG or JUnit, but what do I lose compared to a "pure" ScalaTest solution? Thanks!

I'm not using ScalaTest by itself, but I've had some success with a mixture of JUnit, ScalaCheck and ScalaTest. There is some material in the Programming in Scala book about this that was helpful for me.

My basic outline is a class that looks like this:

import junit.framework.Assert._

import junit.framework.TestCase

import org.scalacheck._

import org.scalacheck.Prop._ 

import org.scalatest.junit.JUnit3Suite 

import org.scalatest.prop.Checkers 

class XTests extends TestCase with JUnit3Suite with Checkers { ... }

Of course you should mixin or import anything else you need for the tests.  Inside XTests you write JUint tests, but you can use ScalaCheck generators etc.  E.g., here are a couple of mine for a lambda calculus project I've done recently, one just using JUnit and another using a checker from ScalaCheck (assertEval is included at the end of this message):

    def testAssociativity () {

        assertEval ("""x y 42""", App (App (Var ("x"), Var ("y")), Num (42)))

    } 

    def testLeaves () {

        check ((i : Int) => (i >= 0) ==> evalTo (i.toString, Num (i)))

        check ((v : Var) => evalTo (v.toString, v))

    }

With this setup you can run the XTests within Eclipse using a JUnit launch configuration.  I have managed to get the single test version going, but not the "run all JUnit tests in this project" version (let me know if you work out how to do this).

Also, you can use the ScalaTest GUI outside of Eclipse.  I use the following ant target:

  <target name="scalatest" depends="build">

    <java classname="org.scalatest.tools.Runner" classpathref="kiama.classpath" fork="true">

      <arg value="-g"/>

      <arg value="-p"/>

      <arg value="bin"/>

    </java>

  </target>

I hope this helps.  Let me know if you need more detail. 

cheers,

Tony

    def assertEval (term : String, result : Exp) {

        val in = new CharArrayReader (term.toArray)

        parse (in) match {

            case Success (e, in) if in.atEnd =>

                normal (e) match {

                    case Some (r) => assertEquals (result, r)

                    case None     => fail ("reduction failed: " + term)

                }

            case Success (_, in) =>

                fail ("extraneous input at " + in.pos + ": " + term)

            case f @ Failure (_, _) =>

                fail ("parse failure: " + f)

        }

    }

[Ismael Juma]

Rob Hasselbaum <rob@...> writes:
> How do you approach running unit tests in the Eclipse debugger?

Personally I use Specs with the JUnit4 runner.

> but what do I lose compared to a "pure" ScalaTest solution? Thanks!

I don't use ScalaTest, so can't comment on that.

Regards,
Ismael

[Rob Hasselbaum-4]

On 09/20/2008 09:53 AM, Tony Sloane wrote:

With this setup you can run the XTests within Eclipse using a JUnit launch configuration.  I have managed to get the single test version going, but not the "run all JUnit tests in this project" version (let me know if you work out how to do this).

Thanks Tony. I decided to give TestNG a try because I'm already familiar with JUnit and I want to broaden my horizons a bit. :-) The TestNG plugin seems to work fine with suites that extend ScalaTest's TestNGSuite. The only minor annoyance is that I have to manually set the class name in the launch configuration because the browse dialog doesn't see any of my Scala classes. But still sufficient for my needs.

Regards,
-Rob

[Arco Oost-3]

Rob,

For TestNG, I add a "testng.xml" file to my test directory. When you
right-click on the file you can just run it.
The file is defined like:


<suite name="OMeta" verbose="1">
  <test name="OMeta">
    <packages>
      <package name="test.*"/>
    </packages>
  </test>
</suite>

Regards,
  Arco Oost

Rob Hasselbaum wrote:

> On 09/20/2008 09:53 AM, Tony Sloane wrote:
>> With this setup you can run the XTests within Eclipse using a JUnit
>> launch configuration.  I have managed to get the single test version
>> going, but not the "run all JUnit tests in this project" version (let
>> me know if you work out how to do this).
>
> Thanks Tony. I decided to give TestNG a try because I'm already
> familiar with JUnit and I want to broaden my horizons a bit. :-) The
> TestNG plugin seems to work fine with suites that extend ScalaTest's
> TestNGSuite. The only minor annoyance is that I have to manually set
> the class name in the launch configuration because the browse dialog
> doesn't see any of my Scala classes. But still sufficient for my needs.
>
> Regards,
> -Rob

[Rob Hasselbaum-4]

Perfect! Thank you.


On 09/20/2008 04:56 PM, Arco Oost wrote:

Rob,

For TestNG, I add a "testng.xml" file to my test directory. When you right-click on the file you can just run it.
The file is defined like:


<suite name="OMeta" verbose="1">
 <test name="OMeta">
   <packages>
     <package name="test.*"/>
   </packages>
 </test>
</suite>

Regards,
 Arco Oost

Rob Hasselbaum wrote:

On 09/20/2008 09:53 AM, Tony Sloane wrote:

With this setup you can run the XTests within Eclipse using a JUnit launch configuration.  I have managed to get the single test version going, but not the "run all JUnit tests in this project" version (let me know if you work out how to do this).

Thanks Tony. I decided to give TestNG a try because I'm already familiar with JUnit and I want to broaden my horizons a bit. :-) The TestNG plugin seems to work fine with suites that extend ScalaTest's TestNGSuite. The only minor annoyance is that I have to manually set the class name in the launch configuration because the browse dialog doesn't see any of my Scala classes. But still sufficient for my needs.

Regards,
-Rob

[Bill Venners-3]

Hi Rob,

Sounds like you found a solution already. If there are minor tweaks
that you think could help in the short run, please let me know. Longer
run I'd like to somehow make it easier to use any of ScalaTest, specs,
ScalaCheck, JUnit 3/4, TestNG with the big 3 IDEs. Normally one would
write an IDE plug-in, but I may have to somehow help with the Scala
IDE plug-ins. But all that is on hold until I get the book out the
door.

Bill

On Sat, Sep 20, 2008 at 3:15 PM, Rob Hasselbaum <rob@...> wrote:
> How do you approach running unit tests in the Eclipse debugger? I'd like to
> use ScalaTest or one of the other Scala test frameworks, but as far as I can
> tell, Eclipse integration isn't there yet. I assume I can use ScalaTest's
> integration with either TestNG or JUnit, but what do I lose compared to a
> "pure" ScalaTest solution? Thanks!
>

[Rob Hasselbaum-4]

Thanks Bill. I'll keep an eye out. The combination of TestNG with TestNGSuite and the "testng.xml" file makes running the tests in Eclipse pretty convenient. It might be helpful to others in the short-term to describe this kind of setup on the ScalaTest home page, as I'm sure there must be broad interest in running tests within the debugger.


On 09/21/2008 12:48 AM, Bill Venners wrote:

Hi Rob,

Sounds like you found a solution already. If there are minor tweaks
that you think could help in the short run, please let me know. Longer
run I'd like to somehow make it easier to use any of ScalaTest, specs,
ScalaCheck, JUnit 3/4, TestNG with the big 3 IDEs. Normally one would
write an IDE plug-in, but I may have to somehow help with the Scala
IDE plug-ins. But all that is on hold until I get the book out the
door.