Are you really using JUnit during your development?

[J. B. Rainsberger]

Andrew McDonagh wrote:

> I've found once I give in to Rail's 'convention over configuration' and
> accept I need a db (even though I shouldn't) it helped a lot.

I think I've done that, even though there's a recipe for running without
a database.

> The second biggest hurdle was deciding to not unit test sometimes, as
> the difference between a Rails unit test and a Rails Functional test can
> be very little, so sometimes I've just written acceptance tests

I see the Functional test as an object test for the controller, and Unit
test as an object test for the model. I tried RailsViews, or whatever
it's called, for an object test for the view, but I couldn't get it to work.

Don't you find, when writing functional tests, that they are too big?
Don't you find annoying irrelevant details in your tests? Don't you find
yourself duplicating setup code just to test a small variation in the
happy path?
--
J. B. (Joe) Rainsberger :: http://www.jbrains.info
Your guide to software craftsmanship
JUnit Recipes: Practical Methods for Programmer Testing
2005 Gordon Pask Award for contribution Agile Software Practice


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[J. B. Rainsberger]

Shane Mingins wrote:

> (JB) Is the TDD list a good place to discuss the TDD aspects/challenges with
> Rails??? I would be interested in following any discussions.

Yes, except that I don't read that list, so I wanted the discussion to
stay here for a while. I'm 9,362 messages behind on the TDD list.
--
J. B. (Joe) Rainsberger :: http://www.jbrains.info
Your guide to software craftsmanship
JUnit Recipes: Practical Methods for Programmer Testing
2005 Gordon Pask Award for contribution Agile Software Practice


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Simon Chappell]

On 8/23/06, J. B. Rainsberger <jbrains@...> wrote:
> Simon Chappell wrote:
>
> >  > I usually only run tests before checking in, not every single time I
> > save a
> >  > file.
>
> Really? Do you check in that frequently, or do you run your tests that
> rarely?

Errr, I don't think I said that. Better check that attribution.

I run the test for the object I'm working on very often, sometimes
multiple times per minute. A single test object typically takes less
than a second, maybe two to run. Then, when the object is at an
advanced % of being done, I start periodically running the full test
suite.

The last big project I was on at my previous employer, the entire
build took a minute and a half (that includes compiling everything and
building all of the JavaDoc). This was fine by me, given that it was
enough time to get another cup of tea. :-)

Simon

--
www.simonpeter.org


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Jeff Nielsen]

Coincidentally, I am presenting a tutorial at SD Best Practices on
this topic.  It is entitled: "The Psychology of Build Times: Keeping
the Tests Fast Enough."  We discuss a model of different kinds of
builds for different situations, based on how long people are "willing
to wait."

For example, here are some observations I have made in my work with
coaching teams.

- How long will an average developer spend running tests locally
  (usually in the IDE) after changing a few lines of code?

    5-10 secs max. before they feel interrupted or bothered


- How much time is an average developer willing to spend running tests
  locally (by hand) when they are ready to check in?

    20 - 30 secs before they start doing something else


- How long is an average developer waiting to hang around the team
  room/office after checking in?

    10 minutes before they have to leave or decide to defer checking
    in until later


The more tests you can fit into each window, the more feedback you'll
get more often.


Jeff Nielsen
Digital Focus
www.digitalfocus.com



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Elliotte Harold]

Michael Feathers wrote:

> Unfortunately, programmers who want to unit test their code suffer.  API
> designers rarely think about testability of client code.  I see the
> effect nearly every week at client sites.

What sort of problems do you see? I admit I don;t think much about the
testability of client code either, but I'm not sure how that practically
impacts anything. Can you elaborate on these problems you've seen?


--
Elliotte Rusty Harold  elharo@...
Java I/O 2nd Edition Just Published!
http://www.cafeaulait.org/books/javaio2/
http://www.amazon.com/exec/obidos/ISBN=0596527500/ref=nosim/cafeaulaitA/


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Elliotte Harold]

J. B. Rainsberger wrote:

> I find it difficult not to interpret this as a (very mild) personal
> attack. You seem to imply that we are not also practitioners. I suspect
> you're not an asshole, so I imagine you mean something else. I would
> appreciate you clarifying this position. Is it necessarily the case,
> then, that "the sort of programmers who... hang out on lists like this
> one" don't know what we're talking about, because I kinda think I do and
> I kinda think I've proven that.

You do know what you're talking about, and you are practitioners, as am
I. However you're making the classic mistake of assuming that everyone
else is just like you, and that what works for you works for them. This
is a *HUGE* and unrecognized fallacy that underlies a lot more than
programming methodologies. For instance, it's largely responsible for
the failure of modern elementary and secondary education, which works
for maybe 5% of children. However that's those 5% that go onto become
teachers and principles and propagate the same techniques to future
generations, while ignoring the failure of 95% of their students.

The simple fact is that people learn and think differently. No one
technique works for everyone, but some techniques do work for more
people. We tend to self-select into groups that think similarly and
learn similarly to ourselves. The people who like interfaces are a
self-selected group of programmers who think at a very high degree of
abstraction, even for programmers (already a self-selected and abstract
bunch). This doesn't mean they don't produce quality software. It does
mean that you can't expect what works for them to work for all or even
most developers.

--
Elliotte Rusty Harold  elharo@...
Java I/O 2nd Edition Just Published!
http://www.cafeaulait.org/books/javaio2/
http://www.amazon.com/exec/obidos/ISBN=0596527500/ref=nosim/cafeaulaitA/


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Michael Feathers]

Elliotte Harold wrote:

>Michael Feathers wrote:
>
>  
>
>>Unfortunately, programmers who want to unit test their code suffer.  API
>>designers rarely think about testability of client code.  I see the
>>effect nearly every week at client sites.
>>    
>>
>
>What sort of problems do you see? I admit I don;t think much about the
>testability of client code either, but I'm not sure how that practically
>impacts anything. Can you elaborate on these problems you've seen?
>  
>

Sure, I once worked on a system for instrument automation.  The library
supplied by the vendor presented an API that contained only non-virtual
functions.  The code that used the library could not be tested because
there was no way to make sure that it was making the right calls.

We ended up wrapping the entire API so that we could mock it.  If the
vendor had made the functions virtual, or at least supplied mocking
points, we would have been saved a lot of trouble.  But, the thing is,  
API vendors just don't think about these things.  I used a DOM parsing
library a few months ago which, by all appearances, didn't let you
create a document in memory.  You had to either point it to a URL or the
disk.  So, to write tests that could otherwise easily be written in
memory, you needed either a server or a scattering of files on disk.

API designers just don't seem to think about testing.

Michael Feathers
www.objectmentor.com




 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Michael Feathers]

Elliotte Harold wrote:

>J. B. Rainsberger wrote:
>
>  
>
>>I find it difficult not to interpret this as a (very mild) personal
>>attack. You seem to imply that we are not also practitioners. I suspect
>>you're not an asshole, so I imagine you mean something else. I would
>>appreciate you clarifying this position. Is it necessarily the case,
>>then, that "the sort of programmers who... hang out on lists like this
>>one" don't know what we're talking about, because I kinda think I do and
>>I kinda think I've proven that.
>>    
>>
>
>You do know what you're talking about, and you are practitioners, as am
>I. However you're making the classic mistake of assuming that everyone
>else is just like you, and that what works for you works for them. This
>is a *HUGE* and unrecognized fallacy that underlies a lot more than
>programming methodologies. For instance, it's largely responsible for
>the failure of modern elementary and secondary education, which works
>for maybe 5% of children. However that's those 5% that go onto become
>teachers and principles and propagate the same techniques to future
>generations, while ignoring the failure of 95% of their students.
>
>The simple fact is that people learn and think differently. No one
>technique works for everyone, but some techniques do work for more
>people. We tend to self-select into groups that think similarly and
>learn similarly to ourselves. The people who like interfaces are a
>self-selected group of programmers who think at a very high degree of
>abstraction, even for programmers (already a self-selected and abstract
>bunch). This doesn't mean they don't produce quality software. It does
>mean that you can't expect what works for them to work for all or even
>most developers.
>  
>

I think you're mixing pedagogical techniques with working techniques.  
Average programmers can think in terms of interfaces.. it's a matter of
acclimation.  Note, I'm talking about average programmers.  There are,
unfortunately, some people who really shouldn't be programming at all.

I know that there are cases where you know that the people who come
after you in a project are going to be of uncertain caliber.  I was
talking to a friend a month ago who was part of a small team that
deliberately didn't use patterns.   They programmed for the least common
denominator because that was all that they could count on for
maintenance.  They made a calculated bet.  The thing is, though, that
I've seen enough teams raise their level of practice to bet on that when
I know that they have the opportunity.

Michael Feathers
www.objectmentor.com





 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[ddoss_2006]

Not sure if this is what you are talking about, but it reminds me of something that I was thinking about this morning,...I'm using an API, and I want to test my use of that API - I was thinking that it would have been nice if they had also provided a test harness for the API,...assuming (possibly erroneously) that they have tested their code, then they probably had to write some testing structures of their objects (like structures that set up HttpRequests to include everything that one of their objects needs/expects to work),...wouldn't it be so much easier for me if they just provided that structure for me, instead of me having to build it up also.

Not sure if that gives you an idea,...or is just completely off-track.  :)  It's early after a late night of coding.  :)

-------------- Original message --------------
From: Elliotte Harold <elharo@...>
Michael Feathers wrote:

> Unfortunately, programmers who want to unit test their code suffer. API
> designers rarely think about testability of client code. I see the
> effect nearly every week at client sites.

What sort of problems do you see? I admit I don;t think much about the
testability of client code either, but I'm not sure how that practically
impacts anything. Can you elaborate on these problems you've seen?

--
Elliotte Rusty Harold elharo@...
Java I/O 2nd Edition Just Published!
http://www.cafeaulait.org/books/javaio2/
http://www.amazon.com/exec/obidos/ISBN=0596527500/ref=nosim/cafeaulaitA/

 

[Non-text portions of this message have been removed]



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Michael Feathers]

No, that helps a lot.  In an ideal world, all libraries would
distributed test cases also.  And, yes, it would go far in dealing with
the issue I'm talking about, but it doesn't go far enough.  A while
back, I blogged something I call The Golden Rule of API Design:

http://blogs.objectmentor.com/ArticleS.MichaelFeathers.TheGoldenRuleOfApiDesign

Michael Feathers

didoss@... wrote:

>Not sure if this is what you are talking about, but it reminds me of something that I was thinking about this morning,...I'm using an API, and I want to test my use of that API - I was thinking that it would have been nice if they had also provided a test harness for the API,...assuming (possibly erroneously) that they have tested their code, then they probably had to write some testing structures of their objects (like structures that set up HttpRequests to include everything that one of their objects needs/expects to work),...wouldn't it be so much easier for me if they just provided that structure for me, instead of me having to build it up also.
>
>Not sure if that gives you an idea,...or is just completely off-track.  :)  It's early after a late night of coding.  :)
>
>-------------- Original message --------------
>From: Elliotte Harold <elharo@...>
>Michael Feathers wrote:
>
>  
>
>>Unfortunately, programmers who want to unit test their code suffer. API
>>designers rarely think about testability of client code. I see the
>>effect nearly every week at client sites.
>>    
>>
>
>What sort of problems do you see? I admit I don;t think much about the
>testability of client code either, but I'm not sure how that practically
>impacts anything. Can you elaborate on these problems you've seen?
>
>  
>

 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Elliotte Harold]

Michael Feathers wrote:

> Sure, I once worked on a system for instrument automation.  The library
> supplied by the vendor presented an API that contained only non-virtual
> functions.  The code that used the library could not be tested because
> there was no way to make sure that it was making the right calls.

You're going to have to elaborate on that case. I don't doubt you had a
problem, but I don't understand what it was. How do non-virtual
functions imply that you couldn't tell  if it was making the right
calls? You say "non-virtual". Was this a C++ library?

>  I used a DOM parsing
> library a few months ago which, by all appearances, didn't let you
> create a document in memory.  You had to either point it to a URL or the
> disk.  So, to write tests that could otherwise easily be written in
> memory, you needed either a server or a scattering of files on disk.
>

That seems OK to me. If the library doesn't need a means of creating
documents in memory for its own purposes, then it shouldn't add those
just for testing. Had it done so, you would have used those functions.
However, because it didn't you tested the code in a more realistic
fashion that is more likely to expose bugs. That's a good thing.

Now if the library had a legitimate need to create documents in memory
for reasons other than testing, then I would add such functions. However
I wouldn't do it just for testing.

--
Elliotte Rusty Harold  elharo@...
Java I/O 2nd Edition Just Published!
http://www.cafeaulait.org/books/javaio2/
http://www.amazon.com/exec/obidos/ISBN=0596527500/ref=nosim/cafeaulaitA/


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Michael Feathers]

Elliotte Harold wrote:

>Michael Feathers wrote:
>
>  
>
>>Sure, I once worked on a system for instrument automation.  The library
>>supplied by the vendor presented an API that contained only non-virtual
>>functions.  The code that used the library could not be tested because
>>there was no way to make sure that it was making the right calls.
>>    
>>
>
>You're going to have to elaborate on that case. I don't doubt you had a
>problem, but I don't understand what it was. How do non-virtual
>functions imply that you couldn't tell  if it was making the right
>calls? You say "non-virtual". Was this a C++ library?
>  
>

Yes, C++.  We couldn't introduce mocks for testing.

>  
>
>> I used a DOM parsing
>>library a few months ago which, by all appearances, didn't let you
>>create a document in memory.  You had to either point it to a URL or the
>>disk.  So, to write tests that could otherwise easily be written in
>>memory, you needed either a server or a scattering of files on disk.
>>
>>    
>>
>
>That seems OK to me. If the library doesn't need a means of creating
>documents in memory for its own purposes, then it shouldn't add those
>just for testing. Had it done so, you would have used those functions.
>However, because it didn't you tested the code in a more realistic
>fashion that is more likely to expose bugs. That's a good thing.
>  
>

Nope.  We wrapped it.  Yes, we had system tests, but in order to test
our logic independently, we pretty much had to wrap it.

>Now if the library had a legitimate need to create documents in memory
>for reasons other than testing, then I would add such functions. However
>I wouldn't do it just for testing.
>  
>
The fact is, testing is a legitmate need.  Designing for testability is
simply good engineering.

Michael Feathers
www.objectmentor.com




 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Elliotte Harold]

Michael Feathers wrote:

> Yes, C++.  We couldn't introduce mocks for testing.


So you tested without mocks. I can accept that. I vastly prefer
mock-free testing to mocked testing.


--
Elliotte Rusty Harold  elharo@...
Java I/O 2nd Edition Just Published!
http://www.cafeaulait.org/books/javaio2/
http://www.amazon.com/exec/obidos/ISBN=0596527500/ref=nosim/cafeaulaitA/


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Elliotte Harold]

Michael Feathers wrote:

>>
> Nope.  We wrapped it.  Yes, we had system tests, but in order to test
> our logic independently, we pretty much had to wrap it.

I don't want the tests to be independent of the libraries they depend
on. I want them to depend on exactly what the code depends on: no more;
no less. That way you find bugs in the boundaries between model classes
and libraries as well as in the model classes and libraries themselves.

>> Now if the library had a legitimate need to create documents in memory
>> for reasons other than testing, then I would add such functions. However
>> I wouldn't do it just for testing.
>>  
>>
> The fact is, testing is a legitmate need.  Designing for testability is
> simply good engineering.
>

The tail is wagging the dog. Testing is done to support the project. It
is not the project, and it should not control the project. I am willing
to consider that a library might need to adjust itself to enable client
applications to test *themselves* (Testing the library is the library
vendor's responsibility) but I still haven't seen a detailed, worked out
example demonstrating how a library might prevent client applications
from testing themselves. Pithy sayings are well and good; but they need
to be backed up by case studies with enough detail to allow them to be
independently verified.

I'll keep an eye out for such a case in the future, but I don;t recall
ever noticing one. I do recall numerous examples where I had trouble
testing code that wasn't intended to be used as a library in the first
place. However in all cases I can think of the problems that prevented
testing also prevented library usage. The problems were in no way unique
to testability, nor did you have to resort to testability to justify
fixing the problems,

--
Elliotte Rusty Harold  elharo@...
Java I/O 2nd Edition Just Published!
http://www.cafeaulait.org/books/javaio2/
http://www.amazon.com/exec/obidos/ISBN=0596527500/ref=nosim/cafeaulaitA/


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Michael Feathers]

Elliotte Harold wrote:

>Michael Feathers wrote:
>
>  
>
>>Yes, C++.  We couldn't introduce mocks for testing.
>>    
>>
>
>
>So you tested without mocks. I can accept that. I vastly prefer
>mock-free testing to mocked testing.
>
>
>  
>

No actually, we couldn't write automated tests.  There was no path back
to check results, just instrument actions.

Michael Feathers
www.objectmentor.com



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Elliotte Harold]

Michael Feathers wrote:

> No actually, we couldn't write automated tests.  There was no path back
> to check results, just instrument actions.

I suspect in that case the library is dysfunctional irrespective of
testing. There should be some form of return value, error code, or
callback that indicates whether the instrument took the action. That's
useful for a lot more than just testing.

Perhaps the instrument only had a one-way communication channel? I've
seen a few like that. If so, you could add a pass-through device that
logs all the requests, or use some other device to check what the
instrument does. That's certainly non-trivial, but not impossible
either. Furthermore the test would be of the instrument itself rather
than of what the instrument is supposed to do.

--
Elliotte Rusty Harold  elharo@...
Java I/O 2nd Edition Just Published!
http://www.cafeaulait.org/books/javaio2/
http://www.amazon.com/exec/obidos/ISBN=0596527500/ref=nosim/cafeaulaitA/


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/