Time to learn TDD and impact on productivity?

[Mark Levison-2]

Two recent conversations I've had, leave me with questions about my
assumptions in how long it takes to really learn TDD and the short term
effects on productivity.

Ideally I would like pointers to peer review studies (I seem to remember a
few kicking around at Agile 2007, but can't find them now). Failing that
anecdotal would a be good start.

1) How long does it take to get a developer back to where they were before
in terms of productivity?
2) To become really productive? I.E. more so than they were before.

In both cases it can be assumed the developers are working as part of
supportive agile, have had training, access to books and ongoing coding
dojos (http://www.notesfromatooluser.com/2008/10/tdd-randori-workshop.html).
Effectively with everything put in place so they can succeed.

3) What is the effect on their productivity during the learning period?
Guesstimates will be fine here. In this case the point is to communicate
with management and make clear what effects the adoption of TDD will have in
the short term. Yes of course will emphasize reduction in complexity and
therefore the improved quality.

Cheers
Mark Levison

Blog: http://www.notesfromatooluser.com/
Recent Entries: Agile/Scrum Smells:
http://www.notesfromatooluser.com/2008/06/agilescrum-smells.html
Agile Games for Making Retrospectives Interesting:
http://www.notesfromatooluser.com/2008/10/agile-games-for-making-retrospectives-interesting.html


[Non-text portions of this message have been removed]

[Casey Charlton]

1) 3 months
2) 6+ months
3) 50% as productive but higher quality code

Casey Charlton
http://devlicio.us/blogs/casey


On 27 Nov 2008, at 20:51, "Mark Levison" <mark@...> wrote:

> Two recent conversations I've had, leave me with questions about my
> assumptions in how long it takes to really learn TDD and the short  
> term
> effects on productivity.
>
> Ideally I would like pointers to peer review studies (I seem to  
> remember a
> few kicking around at Agile 2007, but can't find them now). Failing  
> that
> anecdotal would a be good start.
>
> 1) How long does it take to get a developer back to where they were  
> before
> in terms of productivity?
> 2) To become really productive? I.E. more so than they were before.
>
> In both cases it can be assumed the developers are working as part of
> supportive agile, have had training, access to books and ongoing  
> coding
> dojos (http://www.notesfromatooluser.com/2008/10/tdd-randori-workshop.html 
> ).
> Effectively with everything put in place so they can succeed.
>
> 3) What is the effect on their productivity during the learning  
> period?
> Guesstimates will be fine here. In this case the point is to  
> communicate
> with management and make clear what effects the adoption of TDD will  
> have in
> the short term. Yes of course will emphasize reduction in complexity  
> and
> therefore the improved quality.
>
> Cheers
> Mark Levison
>
> Blog: http://www.notesfromatooluser.com/
> Recent Entries: Agile/Scrum Smells:
> http://www.notesfromatooluser.com/2008/06/agilescrum-smells.html
> Agile Games for Making Retrospectives Interesting:
> http://www.notesfromatooluser.com/2008/10/agile-games-for-making-retrospectives-interesting.html
>
> [Non-text portions of this message have been removed]
>
>

[Non-text portions of this message have been removed]

[nat_pryce]

2008/11/27 Casey Charlton <casey@...>:
> 1) 3 months
> 2) 6+ months
> 3) 50% as productive but higher quality code

How do you measure productivity?

And what do you mean by quality? Easier to maintain?  Or less defects?
 In either case, are you factoring that into the 50%?

--Nat

[Mark Levison-2]

On Thu, Nov 27, 2008 at 5:27 PM, Nat Pryce <nat.pryce@...> wrote:
>
> 2008/11/27 Casey Charlton <casey@...>:
>
> > 1) 3 months
> > 2) 6+ months
> > 3) 50% as productive but higher quality code
>
> How do you measure productivity?

Nat as I'm sure you know this has been hotly debated on
AgileProjectManagement recently. In this case I'm sure all answers will be
by gut feel - which while not the best, will better than throwing darts at a
dart board.

I'm rather hoping that this thread stays focused so I can get enough
information to help my colleague. If people want to debate productivity can
we do it in another thread.

> And what do you mean by quality? Easier to maintain? Or less defects?
> In either case, are you factoring that into the 50%?

Now that's an important question - but again I'm hoping it doesn't become
the core of the thread.

Thanks
Mark Levison

Blog: http://www.notesfromatooluser.com/
Recent Entries: Agile/Scrum Smells:
http://www.notesfromatooluser.com/2008/06/agilescrum-smells.html
Agile Games for Making Retrospectives Interesting:
http://www.notesfromatooluser.com/2008/10/agile-games-for-making-retrospectives-interesting.html


[Non-text portions of this message have been removed]

[nat_pryce]

2008/11/27 Mark Levison <mark@...>:
> I'm rather hoping that this thread stays focused so I can get enough
> information to help my colleague. If people want to debate productivity can
> we do it in another thread.
>
>> And what do you mean by quality? Easier to maintain? Or less defects?
>> In either case, are you factoring that into the 50%?
>
> Now that's an important question - but again I'm hoping it doesn't become
> the core of the thread.

Aren't both issues at the core of the thread?

If you measure productivity in "lines-of-system-code-written-per-day"
the benefit of TDD will look very different than if you measure it as
"features-deployed-and-running-with-a-specific-MTBF" or
"profit-made-by-the-system-per-month".

To decide whether TDD makes a team more or less productive you must
take into account the time that a team would otherwise spend coping
with code that is more difficult to maintain than if they had had to
write unit tests, or diagnosing and fixing bugs that were not
eliminated early.

--Nat

[Mark Levison-2]

On Thu, Nov 27, 2008 at 5:51 PM, Nat Pryce <nat.pryce@...> wrote:
> Aren't both issues at the core of the thread?
>
> If you measure productivity in "lines-of-system-code-written-per-day"
> the benefit of TDD will look very different than if you measure it as
> "features-deployed-and-running-with-a-specific-MTBF" or
> "profit-made-by-the-system-per-month".

I assume that most in this group are intuitively measuring using one
of the latter two - I can't imagine anyone (including our management)
thinking that lines of code are a good measure.

> To decide whether TDD makes a team more or less productive you must
> take into account the time that a team would otherwise spend coping
> with code that is more difficult to maintain than if they had had to
> write unit tests, or diagnosing and fixing bugs that were not
> eliminated early.

Agreed and I will convey that, but again I assume that most in this
group are already factoring this in.

I think the key point here is that I'm not trying to write an academic
paper - just give management a first order idea of the effect on their
projects.

Cheers
Mark

Blog: http://www.notesfromatooluser.com/
Recent Entries: Agile/Scrum Smells:
http://www.notesfromatooluser.com/2008/06/agilescrum-smells.html
Agile Games for Making Retrospectives Interesting:
http://www.notesfromatooluser.com/2008/10/agile-games-for-making-retrospectives-interesting.html

[nat_pryce]

2008/11/27 Mark Levison <mark@...>:
> On Thu, Nov 27, 2008 at 5:51 PM, Nat Pryce <nat.pryce@...> wrote:
>> To decide whether TDD makes a team more or less productive you must
>> take into account the time that a team would otherwise spend coping
>> with code that is more difficult to maintain than if they had had to
>> write unit tests, or diagnosing and fixing bugs that were not
>> eliminated early.
>
> Agreed and I will convey that, but again I assume that most in this
> group are already factoring this in.

That's why I was querying the statement that TDD initially makes a
team "50% as productive but [produce] higher quality code".  It sounds
as if the quality was not factored in, in which case the team was not
really 50% as productive.

--Nat

[George Dinwiddie]

Mark Levison wrote:

> Two recent conversations I've had, leave me with questions about my
> assumptions in how long it takes to really learn TDD and the short term
> effects on productivity.
>
> Ideally I would like pointers to peer review studies (I seem to remember a
> few kicking around at Agile 2007, but can't find them now). Failing that
> anecdotal would a be good start.
>
> 1) How long does it take to get a developer back to where they were before
> in terms of productivity?

I can only speak for myself, but I'd say it took two or three weeks.

> 2) To become really productive? I.E. more so than they were before.

In my case, a couple months.  It was when I discovered I'd made a
tremendous blunder of an assumption, and it only took me half a day to
correct it, get all the tests running again, and then realize I was
done.  I had estimated it would take me a week to fix the problem
because it touched so much code.

  - George

--
  ----------------------------------------------------------------------
   * George Dinwiddie *                      http://blog.gdinwiddie.com
   Software Development                    http://www.idiacomputing.com
   Consultant and Coach                    http://www.agilemaryland.org
  ----------------------------------------------------------------------

[Mark-429]

I'd say my experience was similar. I had read a fair bit before starting my
first real TDD project but getting a handle on it all took a while (few
weeks?). Payback for me was within a couple of months - I'm positive the
framework I wrote was of a much better standard than if I had dived in
without TDD.  I backed out of 2 wrong paths with confidence I wasn't
breaking anything - not sure I would have risked it without tests. And they
were the majors. plenty of minors along the way too.

 

HTH

 

Mark

 

From: testdrivendevelopment@...
[mailto:testdrivendevelopment@...] On Behalf Of George Dinwiddie
Sent: Friday, 28 November 2008 4:21 p.m.
To: testdrivendevelopment@...
Subject: Re: [TDD] Time to learn TDD and impact on productivity?

 

Mark Levison wrote:

> Two recent conversations I've had, leave me with questions about my
> assumptions in how long it takes to really learn TDD and the short term
> effects on productivity.
>
> Ideally I would like pointers to peer review studies (I seem to remember a
> few kicking around at Agile 2007, but can't find them now). Failing that
> anecdotal would a be good start.
>
> 1) How long does it take to get a developer back to where they were before
> in terms of productivity?

I can only speak for myself, but I'd say it took two or three weeks.

> 2) To become really productive? I.E. more so than they were before.

In my case, a couple months. It was when I discovered I'd made a
tremendous blunder of an assumption, and it only took me half a day to
correct it, get all the tests running again, and then realize I was
done. I had estimated it would take me a week to fix the problem
because it touched so much code.

- George



.

 
<http://geo.yahoo.com/serv?s=97359714/grpId=5031255/grpspId=1705007181/msgId
=29468/stime=1227842449/nc1=3848583/nc2=5202322/nc3=4763762>
 



[Non-text portions of this message have been removed]

[Adam Sroka-3]

On Thu, Nov 27, 2008 at 12:51 PM, Mark Levison <mark@...> wrote:
> Two recent conversations I've had, leave me with questions about my
> assumptions in how long it takes to really learn TDD and the short term
> effects on productivity.
>

I have seen some of these studies (No links. If I feel inclined later
I may look them up.) For the most part they tend to look favorable for
TDD and show only a brief initial decline in productivity followed by
an eventual return to approximately the same level. OTOH, they tend to
show a significant increase in quality as measured by the number of
bugs found post development.

However, I have never placed a lot of stock in what they find. The
problem is that you aren't really comparing apples to apples. The real
benefits of TDD, in my opinion, have to do with the way that the tests
drive the design and facilitate frequent, merciless (regression proof)
refactoring. The increased malleability of test-driven code makes it
easy to change directions safely and frequently.

I find that with TDD I am more inclined to get it right and have less
of a tendency to muck up my brain with "what-abouts" and
"don't-forget-were-gonna-needs." For the most part, I just don't have
to think about anything other than the task at hand. This change in
perspective is so valuable to me that I would still use TDD as my
primary development technique even if I knew that my productivity was
lower.

[Casey Charlton]

50% lines of code, or 50% deliverables completed

And yes easier to maintain and with less bugs

Casey Charlton
http://devlicio.us/blogs/casey


On 27 Nov 2008, at 22:27, "Nat Pryce" <nat.pryce@...> wrote:

> 2008/11/27 Casey Charlton <casey@...>:
> > 1) 3 months
> > 2) 6+ months
> > 3) 50% as productive but higher quality code
>
> How do you measure productivity?
>
> And what do you mean by quality? Easier to maintain?  Or less defects?
> In either case, are you factoring that into the 50%?
>
> --Nat
>

[Non-text portions of this message have been removed]

[Olof Bjarnason]

Mark; I can only speak for myself.

I did some TDD-reading during december 2006, maybe a week or two.
Primarily tutorials on the net / the main ideas. No book. But I think
I got the "seed" of the idea from the Pragmatic Programmer book
(Hunt-Thomas).

I then developed my first real-world program during christmas same
year. Didn't have any unit testing tool, used only assert (it was
C++). This was the "wetting my feet" period.

It worked so smoothly and the solution and feeling I got was so good,
that I started using the technique at work in a small scale the
following months. Mostly with small concepts near the model of the
system to begin with (no GUI/DB/file related stuff yet). Let us call
this the "trial-and-error" period. That period was maybe 6-8 months. I
bought and read parts of "Test driven development in .NET" during this
period.

During this trial-and-error-period I was positively encouraging other
developers to try it out at work, even had half-a-day
tutorial/demonstration (the stack example from the book) in C# for the
other developers. I know now that it was a bit premature because I
hadn't yet come to the next stage: the "scaling up" period.

With that I mean I actively tried to TDD other parts of systems, even
legacy systems, for example DB/file related things. Learning about
mocks and dependency injection, and the MVP pattern. At the same time
figuring out test smells and finding readability to be my number one
guidance when writing test code.

Now we are close to christmas 2008, two years later, and I'm still in
the "scaling up" period. I'm not doing TDD in 100% of the system I'm
currently developing - maybe close 50% is more like it. Both because
it is so hard and that I have the other developers against me in that
they don't write test-friendly code (not against me developing
software my own way though). I've wet my feet in the BDD ideas, and
found some of it great. One lesson learned from BDD is that TDD is
possible top-down, not just bottom-up as I did in my
"trial-and-error"-period.

And I've stopped trying to encourage my collegues of even doing unit
tests. It seems my initial opportunism was a bit naive.

2008/11/28 Adam Sroka <adam.sroka@...>:

> On Thu, Nov 27, 2008 at 12:51 PM, Mark Levison <mark@...> wrote:
>> Two recent conversations I've had, leave me with questions about my
>> assumptions in how long it takes to really learn TDD and the short term
>> effects on productivity.
>>
>
> I have seen some of these studies (No links. If I feel inclined later
> I may look them up.) For the most part they tend to look favorable for
> TDD and show only a brief initial decline in productivity followed by
> an eventual return to approximately the same level. OTOH, they tend to
> show a significant increase in quality as measured by the number of
> bugs found post development.
>
> However, I have never placed a lot of stock in what they find. The
> problem is that you aren't really comparing apples to apples. The real
> benefits of TDD, in my opinion, have to do with the way that the tests
> drive the design and facilitate frequent, merciless (regression proof)
> refactoring. The increased malleability of test-driven code makes it
> easy to change directions safely and frequently.
>
> I find that with TDD I am more inclined to get it right and have less
> of a tendency to muck up my brain with "what-abouts" and
> "don't-forget-were-gonna-needs." For the most part, I just don't have
> to think about anything other than the task at hand. This change in
> perspective is so valuable to me that I would still use TDD as my
> primary development technique even if I knew that my productivity was
> lower.
>

[Mark Levison-2]

I forgot to say thanks for all the feedback. Its been useful to a colleague
who is talking to her uber manager about the short term effects that
adopting TDD will have on their team. In addition it will also be useful in
my TDD adoption Strategies article.

Cheers
Mark

Blog: http://www.notesfromatooluser.com/
Recent Entries: Agile/Scrum Smells:
http://www.notesfromatooluser.com/2008/06/agilescrum-smells.html
Agile Games for Making Retrospectives Interesting:
http://www.notesfromatooluser.com/2008/10/agile-games-for-making-retrospectives-interesting.html


[Non-text portions of this message have been removed]

[Steve Freeman-4]

There's a recent article from MS research which claims that "the pre-
release defect density of the four products decreased between 40% and  
90% relative to similar projects that did not use the TDD practice.  
Subjectively, the teams experienced a 15-35% increase in initial  
development time after adopting TDD." Which seems like a good deal to  
me, although it was TDD with BDUF. Link via

http://www.m3p.co.uk/blog/2008/12/08/tdd-fewer-bugs-to-production-longer-to-write/

S.


On 4 Dec 2008, at 21:07, Mark Levison wrote:

> I forgot to say thanks for all the feedback. Its been useful to a  
> colleague
> who is talking to her uber manager about the short term effects that
> adopting TDD will have on their team. In addition it will also be  
> useful in
> my TDD adoption Strategies article.
>
> Cheers
> Mark
>
> Blog: http://www.notesfromatooluser.com/
> Recent Entries: Agile/Scrum Smells:
> http://www.notesfromatooluser.com/2008/06/agilescrum-smells.html
> Agile Games for Making Retrospectives Interesting:
> http://www.notesfromatooluser.com/2008/10/agile-games-for-making-retrospectives-interesting.html
>
>
> [Non-text portions of this message have been removed]
>

Steve Freeman
http://www.mockobjects.com

Winner of the Agile Alliance Gordon Pask award 2006