Ever see a burndown chart like the one to the left?  I’ve been on a big burndown chart kick lately and when I saw this one I just couldn’t resist using it.  This one is a bit different from my previous blog entries here and here.  This burndown chart is a release burndown rather than an iteration burndown.  It sure looks like the team was incredibly successful and finished early, right?  Wrong!  What this burndown actually shows is all of the stories being “done” and the release not actually occurring for several more iterations.  How is that possible?

Unfortunately this is a symptom all too common for agile teams, so learn from this blog entry and don’t repeat the mistakes of others!  What this shows is a lack of commitment to a definition of done.  I touched on this briefly when I wrote about using a rules of engagement document.  If you don’t settle on a definition of done the pattern in this burndown chart can easily occur.  This pattern can also occur when a team is trying to hard to meet their deadlines.

If you went back and examined what occurred in this example you would find a team pressured to try to hit a release date by working at an unrealistic pace rather than a sustainable pace.  In order for the team to keep looking like they were on track for an ontime release they built technical debt by not sticking to a strong definition of done.  The result is a tremendous amount of testing which was delayed and triggered massive test-fix cycles.  In fact, the original release date was projected for after iteration 6, but instead it took 4 more iterations to actually release the product.  Technical debt cannot simply be allowed to build.  Software projects don’t have the luxury of deficit spending like the US Government.  At some point the bill will come due and it will almost always come due prior to release.  In fact, if it comes due after release it is even worse – you now are in emergency mode because customers are suffering!

Rally Software recently posted their definition of done on their blog.  It makes for some interesting reading and gives good perspective for other agile SaaS companies.

Richard Lawrence has a recent blog entry on this topic as well.  Richard encourages teams to not build technical debt and in fact he encourages growing your definition of done in order to release high value, high quality software as quickly as possible.  It is fascinating reading based on his work at a large organization.

Until next time I’ll be Making Agile a Reality^® by helping teams agree and abide by their definition of done.

Related posts:

1. Agile antipatterns: Agile burn-down chart roundup post Do you want to see several different ways agile and scrum burn-down charts...
2. Agile antipattern: Another burndown chart that lies! That burndown chart looks sweet doesn’t it?  The team finished the iteration...
3. Holistic vs. Dogmatic Agile Definition – Results Matter! I originally wrote about this topic in our September email newsletter.  In...

Does your team have an iteration burndown chart (giving credit only for completed stories) look like the one to the left? If so there are a couple of possible explanations.  Last week I blogged about how this could be a symptom of working on user stories that are too large.  However, there is another possible explanation, and it is one that is FAR harder to solve.  The culprit is pushing testing to the end of the iteration.  The difference between the two is the steepness and abruptness of the wall.  When testing at the end is truly being done there are no stories finished until the last couple of days of the iteration while large stories pushes most until the last half of the iteration.  See last week’s entry on large stories to see the subtle difference in the burndown charts.

This is basically a version of the code freeze during an iteration antipattern.  It is all too common for an organization which switches from waterfall to agile to continue to try to do testing the same way it had been done previously.  This generally is done by having a handoff from development to testing during the iteration.  Since waterfall tested large chunks of code, most organizations have developers write as much code as they can prior to sending a large chunk to testing.  In order for this to occur the development goes for as long as possible during the iteration.  Stories can’t be completed until they are tested, so all of the completion of stories happens in the last couple of days of the iteration.  This forms the burndown “wall” shown in the sample burndown chart.

So what is wrong with having a burndown wall if everything gets completed?  I’m going to surprise you and say there is still a LOT wrong even if everything gets completed in this structure.  Here is a list of issues you probably want to look at if this is your current situation:

1. What are the developers doing while testing is going on?  They SHOULD NOT be working on the next iteration.
2. If developers are spending their time fixing the defects found during testing how do you guarantee all the defects will get fixed and tested in time for the end of the iteration?
3. What happens if testing runs up against a big problem and can’t finish testing the stories prior to the end of the iteration?
4. If there is pressure to finish testing in time does testing subconsciously cut down on the breadth or depth of testing?
5. What are the testers doing during the early part of the iteration when the developers are writing a pile of code?

These are real concerns and in my experience one or more of them will cause significant issues in almost every iteration.  If a team actually manages to overcome all of these issues in the majority of their iterations then there is still one more question to answer: Is it the most efficient way?

Team after team after team has proven it is not the most efficient way to do things.  When teams start doing things differently using Acceptance Test-Driven Development (ATDD) using automated tests they are much more efficient.  It also gives them the ability to accept stories every day.  Being able to accept stories every day gets rid of the wall and makes a more classic looking burndown chart.  Knowing things are completed and working every day will help the entire team feel better about how they are performing.

Until next time work at Making Agile a Reality^® by integrating testing and development so stories can be accepted every single day and see how much your team can improve!

Related posts:

1. Agile antipattern: Another burndown chart that lies! That burndown chart looks sweet doesn’t it?  The team finished the iteration...
2. Agile antipattern: Burndown charts that hide the truth See that burndown chart over there to the left?  It looks beautiful...
3. Agile antipattern: Changing the definition of done Ever see a burndown chart like the one to the left?  I’ve...

As this blog entry is published I am teaching an agile/scrum course to a client in Flanders, New Jersey.  You might want to ask “Bob, how can you do that?  Isn’t the client upset when you blogwhile they are paying for your time?”  Certainly they would be upset and they would be well within their rights to be upset about it!  However, I’m not BLOGGING during their time, the blog is simply taking a pre-scheduled action.  I actually wrote the blog entry while watching the 3rd quarter of the Steelers vs. Ravens football (American style) game on Sunday night.  Fortunately, WordPress has a way to “schedule” a blog entry for later publication.  In other words I am using a simple scheduler to automate a task to occur at a specific future time.  I’m mentioning all of this because automation is one of the areas where I see many agile teams fail.

Before talking about agile teams let me give another image of automation which sticks with me.  My wife uses the phrase “keeping the slaves working” when she talks about using our dishwasher, washer and dryer.  What she means is external devices are doing the work of humans and saving human time – HER time in many cases!

Agile teams need to start thinking of their computers, servers and other devices as slaves to the team.  The slaves can work 24/7 while the team cannot.  If a computer can do the work much faster or the computer can do it while humans aren’t available doesn’t it make sense to use that capability?  There are some testing big-wigs who say you can’t automate everything.  I agree with that, but where I differ with them is on how much can or should be automated.  I say automate as much as possible, but do it in a maintainable and sustainable way.  Others may disagree.  Their disagreement doesn’t mean I’m wrong

In my experience I have seen three main reasons for teams not using automation (in testing, continuous integration or build/deploy):

1. They simply do not know it is possible.
2. They assume the learning curve is too long for a reasonable ROI.
3. Management has said it costs too much money for a reasonable ROI.

Let me debunk all of these fairly quickly.  The first one is no longer an issue because every reader is going to forward this blog entry to their friends who will all forward to their friends, etc. and within a few days the entire world will know automation is possible!  The second may have been a factor in the past, but with contemporary tools like Cucumber, Selenium and FitNesse the learning curve is MUCH lower than ever before.  Teams can become reasonably proficient with many tools with just a few days of training and coaching from a qualified individual.  Because it only takes a few days the last issue is debunked as well because the cost for a few days of time is negligible in the big picture when compared to the savings in productivity.

Look into automation.  Email me and ask about how current clients are being successful with very little time and dollars invested.  Keep the slaves working while you aren’t at the office and get a better balance between work and home life!

Until next time I’ll be Making Agile a Reality^® for my clients by helping them learn how to get maximum leverage from their use of automation.

Related posts:

1. New to agile? Remember one thing: Just enough, just in time If you lived through the past few decades you have undoubtedly heard...
2. New to agile? Remember how to say “No” No.  Only two letters.  Very simple word.  Yet for some reason, with...
3. New to agile? Remember to respect people One of the Lean Principles is “Respect People.”  I think it may...

This is another common theme among teams just starting with agile.  It usually goes something like this:

1. The team has an unsuccessful iteration.
2. They determine all the unfinished work is testing.
3. During the retrospective they resolve to give more help to the testers so they can finish in time.
4. The next iteration is also unsuccessful.
5. The team determines once again all unfinished work is testing.
6. During the retrospective they decide the only way to have enough time for testing is to move to a longer iteration length.

Uh oh, Houston we have a problem!

This is not going to lead to success.  Think about it for just a minute.  The reasoning behind doing this is to give testers more time to test.  That sounds great in theory, but what are the developers doing during the extra time?  No, sorry, but they aren’t doing anything like sitting on their hands.  They are creating more software.  So if they are creating more software, when does it get tested?  Well, during the extra time of course!  But, but, the extra time was for testing the stuff they ALREADY created.  That’s right, we give extra time to test the stuff they created during the original iteration length, while then taking that time to create proportionally more software.  The result is pretty ugly – the testers now fall short by an amount of work which is directly proportional to the change in iteration length.  If the original iteration was 2 weeks, and the new iteration is 3 weeks, testers will fail to complete 50% more work in the new iteration length.  Bottom line, we didn’t make the problem better.

This is one of the worst antipatterns because for some reason people actually believe it should work.  Somehow they will wave a magic wand and developers will just slow down enough to let testers catch up with the new iteration length.  Sorry folks, it doesn’t work that way.  So what can you do instead?  You might try making a new rule:

There is no code except tested code

How does this help?  Well, developers can no longer say “I coded that already” for just one example.  It isn’t “coded” until it has passed testing.  This means developers get zero credit for anything which isn’t tested.  Now it is not just testers which fell short during the iteration, it is the team.  If testing is constantly falling behind and coders can’t just blame the testers, what are their options?  Well, for starters they can help the testers out.  Maybe figure out how to automate more tests.  Maybe use unit test-driven development so the software that gets tested is more solid and likely to pass the acceptance tests.  Maybe developers turn into testers in order to complete the iteration.  Believe me, the more times developers have to run tests, the more automation tools and frameworks will start to show up or become easier to use (and this is a good thing!).

This is based on the lean principle to improve the system by doing two things:  measuring UP and optimizing the whole.  I’ll almost certainly have blog posts on both of those topics in the near future, but feel free to purchase Mary and Tom Poppendieck’s books on Lean Software Development (see our books page) if you want to read about them.

Thanks for reading this entry.  If you were considering a longer iteration length I hope this post stopped you from making a big mistake!

Until next time I’ll be Making Agile a Reality™ by having my clients expose the real elephants in the room and getting rid of them.

Related posts:

1. Agile antipattern: Burndown “wall” Does your team have an iteration burndown chart (giving credit only for...
2. Agile antipattern: Code freezes during each iteration Over the past 18 months I’ve encountered a number of teams where...
3. Agile Architecture and Agile Testing – New Courses on the horizon Exciting news!  Some associates of mine are currently completing work on some...

Over the past 18 months I’ve encountered a number of teams where it is standard practice to have a code freeze late in the iteration.  The reason given for this was “to allow QA to test what we created during the iteration.” I’m sorry, but I have to be blunt here – this isn’t agile! It leads to 3 main questions:

1. What are the developers doing after the code freeze?
2. What happens if defects are uncovered after the code freeze?
3. Why could testing not be done earlier in the iteration?

The answers to these questions are often enlightening.  Let’s take them one at a time:

1.  What are developers doing after the code freeze?

I’ve heard two primary answers to this.  The first is that developers are fixing defects during that time.  When pressed, I usually find out these are defects from earlier iterations (although this isn’t always the case – more on that later).  If we are fixing defects from previous iterations while QA is working on testing the current iteration, how do we ever get caught up?  Maybe I’m being a bit dense, but I just don’t see the math working on this one.

The second most common answer is developers are working ahead on things for the next iteration.  Said a different way, developers are creating yet more untested code before the commitment for the current iteration has been met.  Let’s think about this model for just a minute.  Let’s assume we are in iteration 1 and the team does a code freeze on day 7.  Coding occurred for 7 days and testing will go for 3 days.  But the developers start stuff from the next iteration on day 7, so for the next iteration they get 3 days from this iteration and 7 from the following iteration for a total of 10 coding days, followed by 3 testing days.  This cycle continues until the project completes.  Anyone feeling sick yet?  Again, being blunt, this is not agile, it is mini-waterfall or what I prefer to call “wagile.”

2.  What happens if defects are discovered after the code freeze?

Again, in Family Feud style, the most common answer is the defects are fixed in the following iteration.  This is a continuation of the first half of the previous answer.  How can the math ever work?

However, this isn’t always the answer.  Sometimes the answer is developers fix them as they are found.  This answer is MUCH less common, but does exist.  The problem with this answer is very simple: if it is possible for developers to fix defects in real time after the code freeze, why could this not be done earlier in the iteration?  Which leads to question 3…

3.  Why could testing not be done earlier in the iteration?

Two answers here are equally common.  The first I’ll cover is the “it takes too much time/effort to move the code to our QA environment so we only do it once per iteration.”  I understand the need for a QA environment for testing, but is it necessary for ALL testing?  In most of the cases I’ve seen it is possible to do a tremendous amount of testing in a development environment to make sure everything is basically working before promotion to a QA environment.  The QA environment should be renamed the Verification environment.  Ideally we want to use it to verify everything works in the stable environment and passes all tests just like it did in the development environment.  So step one is to do more testing in the development environment.  Step two is to use some automation tools to build a stable environment quickly.  When pressed, most teams can create an automated process to do a bare-metal environment build within a short period of time.  It is worth the effort.  If this exists, a new build could be deployed at any time, but at least nightly.  Combine it with automated regression testing and really get some value from it!

The second answer I usually hear is “we don’t know what to test until the code is done anyway.”  Uh oh, can anyone even count how many ways this statement isn’t agile in nature???  Let’s remember the agile process:

1. Product Owner (or similar role) creates stories and some basic acceptance criteria.
2. During iteration planning more acceptance criteria are created based on conversations between tester, developer and Product Owner.
3. During iteration the developer and tester collaborate on their understanding so the code can be written and acceptance criteria can be turned into tests.
4. During development the developer can (and should) access any acceptance tests already written to see how their code is doing.
5. The developer isn’t “done” until the code passes all unit tests and acceptance tests.  Remember the definition of done!

When things are done in this manner the tester ALWAYS knows what to test because they are in close communication with the developer.  Collaboration is vital to agile success.  This is just one example driving it home.

The bottom line for me is there is no legitimate reason for having a code freeze during an iteration.  Teams need to either invest the time/effort to put together a system which doesn’t require a code freeze, or stop calling themselves agile.  This is a nasty antipattern which will cause confusion or worse.

Thank you for reading.  Until next time I’ll be Making Agile a Reality™ for my clients by helping them avoid code freezes during iterations.

Related posts:

1. Agile antipattern: Burndown “wall” Does your team have an iteration burndown chart (giving credit only for...
2. Agile antipattern: Moving work from one iteration to the next All agile teams start at something less than the completely proficient level. ...
3. Agile antipattern: Extending an iteration I had a previous blog post about stopping an iteration and how...

In an agile environment manual testing is fine – except for when it isn’t!  In particular, everyone recognizes manual regression testing takes time.  When using a traditional development process companies use it as an argument for longer release cycles saying something like “It takes 8 weeks of regression testing anyway, so a large release makes sense otherwise the overhead of the testing would be too high.”  It is easy to see how people could buy into that argument.  Because of similar arguments it is hard to get organizations to understand regression testing outside the context of RELEASE regression testing.  The primary cadence of an agile process is the iteration cadence which puts a much different spin on regression testing.

Let me back up for a moment and sum up what most people believe about testing within iterations during an agile project.  Put very simply, teams are led to believe they need to get each user story to the “done” state which matches the team’s definition of done (DoD).  In most cases the DoD says the story has been unit tested, acceptance tested and has been accepted by the Product Champion, Product Owner, Customer or other equivalent role.  Using this definition will lead to having stories which all work in isolation.  The way I put it in my classes is the team has proven there is a finger, a thumb, a knee, etc. but have they proven they are part of the same body and the body works?  Concentrating on a story-centric DoD can lead to significant issues if things don’t work at a system level.  Using a story-centric DoD means it is possible to create “a finger” in one iteration and not know it completely destroyed “a knee” created in an earlier iteration.

Some of you may be thinking “But Bob, you can’t be saying we have to do our regression testing every single iteration!  It takes 8 weeks to do regression testing and if iterations are two weeks long the math won’t work.”  You are right, the math doesn’t work, but not in the way you are thinking.  What I am suggesting is during an agile process you have to have a DoD which is iteration-centric as well as one that is story-centric.  An iteration-centric DoD should include something saying all of the tests from previous iterations still pass.  In other words, tests from prior iterations become regression tests.  There are many great reasons for this, but let me describe just one situation which should make the point.

This chart shows the amount of testing done on each feature if regression testing is done every iteration.  Notice that in 5 iterations feature #1 gets tested 5 times.  Because teams develop software in priority order this means the highest value feature is also the most tested.  The lowest value feature is the least tested.  Common sense tells us this is a good thing!  Wouldn’t it be great if the most important features were the least likely to have bugs?

When using manual regression testing at some point there will be a limit to how much regression testing can be done.  This will directly impact how often the highest priority features get tested prior to release.  If a team can do no regression testing then every feature gets tested once.  If they can regression test just one iteration then almost all features will be tested twice, and so on.  For most teams the lack of regression testing leads to “hardening iterations” prior to release.  This point is very important – if you do not do full regression testing each iteration, you will pay a penalty of both time and money later!

When I present this information to executives they quickly realize an investment in some sort of automated test strategy could save them quite a bit of money.  Automated testing can be done using a variety of free tools so the only investment is in research and learning curve.  I usually recommend tools like FitNesse, Fit, Selenium, Cucumber, Watir and WatiN as starting points.  If you are using Java or a .NET language there are many available frameworks with small learning curves which can deliver value very quickly.  I tend to stay away from tools like HP Quality Test Pro only because it is a heavyweight tool requiring specialized knowledge in order to do automated testing effectively.  Note I am not talking about record/playback automated testing!  I want teams to strive for automation which doesn’t require too much incremental knowledge from what they already know – if it is too big a learning curve they just won’t do it.

If you want to take this to the next level I strongly recommend reading this article.  It takes a holistic view of the defintion of done which really pushes teams to think of everything!

Thank you for reading!  Until next time I’ll continue recommending automated testing frameworks so organizations can continue Making Agile a Reality™.

Related posts:

1. Agile antipattern: Burndown “wall” Does your team have an iteration burndown chart (giving credit only for...
2. Agile antipattern: Code freezes during each iteration Over the past 18 months I’ve encountered a number of teams where...
3. Are you agile – the Nokia test I’ve been helping lots of clients start new agile initiatives recently.  This...

Tired of not knowing exactly what to create or test? Get in the habit of asking the magic question “How will I know I’ve done that?” In other words, ask the Product Owner (or whatever person you have filling that role on your team) to express some acceptance criteria!  Asking this question will make miserable days seem bright and sunny!  Well, ok, maybe not that, but asking this question will brighten your mood.  There are at least three good reasons why asking this question is important.

First, the answer clears up uncertainty.  As I mentioned in my blog post Bob-ism #1 – the good developer it is very easy for developers to build the wrong product if they aren’t careful.  Being careful means uncertainty on the desired functionality needs to be cleared up as early as possible.  There is no such thing as the perfect Product Owner who creates 100% unambiguous stories that never generate questions.  This is impossible because it absolutely should not happen.  The concept behind stories is they are an invitation to a conversation. The conversation should start with “How will I know I’ve done that?”  Members of agile teams need to recognize the necessity of clearing up uncertainty, and Product Owner’s need to understand the necessity of making time available to help.

Second, the answer gives acceptance criteria. Acceptance criteria are useful to every person who deals with a user story.  Developers know how the code will be tested. Testers have a basis for knowing what tests to create.  Wouldn’t it be nice if acceptance criteria could directly be translated into automated acceptance tests?  Well, we have a class on Real World Agile Testing with Fit and FitNesse coming up.  The class is basically to teach how to do that!  Even if you don’t take the class, look at Fit or FitNesse as potential ways to translate between spoken language and automated tests.  But all of this is based on knowing what the acceptance criteria should be, and the answer to the question “How will I know I’ve done that?” gives the criteria.

Finally, asking the question fosters collaboration.  Remember, a user story is an invitation to a conversation.  By asking “How will I know I’ve done that?” you have accepted the invitation.  Imagine if all teams had developers, testers and Product Owners collaborating up front on stories and all being in agreement on what “done” would look like - all before a line of code was written.  For those of us who have seen this in action it is truly amazing.  Teams become hyper-productive.  Features match customer wants and needs much more accurately.  Quality improves dramatically.  Morale on the team improves.  All of these are downstream effects of effective and meaningful collaboration.

Let me close this post with a couple of very important caveats:

• Don’t collaborate in a vaccuum.  If one person needs to ask this question, there are likely others also needing to know the answer.  Make sure you have the proper people as part of the conversation.  This usually means developer, tester and Product Owner are all involved.
• Capture the answer!  This seems so silly to have to say, but it is quite common for people to have a conversation and not record the results anywhere.  This conversation affects the user story, so capture the answer in a place people will know to look.  The answer is acceptance criteria.  It may even clarify some wording in the story itself.  Whatever it is affects the story, so save it appropriately.  Remember, knowledge in your head is lost to the rest of the team – so what happens when you aren’t available???

My user story today was “My readers want a new blog entry so that they can be more agile.”  I asked myself the question “How will I know I’ve done that?”  The answer was “I’ll know I’ve done that when I have a published blog entry my users can access which gives them a useful agile tip.”  So the acceptance criteria are:

• blog entry published
• users can access
• contains a useful tip

I tested the first two and they work.  Did I accomplish the mission for the 3rd bullet point?  I certainly hope so.  If not, let me know!

Related posts:

1. Agile antipattern: Code freezes during each iteration Over the past 18 months I’ve encountered a number of teams where...
2. Agile Architecture and Agile Testing – New Courses on the horizon Exciting news!  Some associates of mine are currently completing work on some...
3. New to agile? INVEST in good user stories As a <user> I want <function> so that<value>. Above is a very...

Another short blog entry.  This time it is to announce that we’ll be hosting Rob Myers teaching a great agile testing course using Fit and FitNesse on March 23 and 24.  The course will be held at the Denver PPA Event Center (right across from Invesco Field at Mile Hi).  This course was just given for a local client and the reviews were fantastic.  Get more details at http://realworldfit-bl.eventbrite.com.  There you will also see some of the testimonials from the recent course.

And, just because I like all of you, use discount code EARLYEARLY before February 25 to get an additional 10% discount.

If your organization struggles with getting testing completed within an iteration and you use either C# or Java, this course is a must.  It can mean the difference between agile success and failure.  Is that wor$895?  And, if your organization is interested in sending more than one person (the course works best with tester/developer pairs), contact me and I’ll give an additional discount for the 2nd person (or any beyond that).

Don’t delay, sign up now.  You absolutely don’t want to miss this course.  It gives you the keys to true automated acceptance testing, not the type most companies are doing when they delay automation for an interation or two!

Related posts:

1. Agile Architecture and Agile Testing – New Courses on the horizon Exciting news!  Some associates of mine are currently completing work on some...
2. Testing to find defects is waste Have you ever heard someone say that testing to find defects is...
3. Free Event! Agile Adoption: The Real Story On October 20, the Agile Cooperative will be hosting a free one-day...

Exciting news!  Some associates of mine are currently completing work on some new courses Agile For All will be able to offer.  The two courses are Agile Architecture and Agile Testing.  In fact, it is really three courses because Agile Testing has a version that is all about using FIT (Framework for Integrated Testing) and the other is how to do acceptance testing using VSTS (Visual Studio Team System) with FIT.

To me all of these courses are vitally important.  One constant question in the Agile community is “How do we create an architecture that doesn’t break when we change things?”  Another is “How do we do coding and testing in the same iteration?”  These courses are designed to answer those specific questions.  I am very excited by these courses because I know the backgrounds and abilities of the course creators.  I respect their abilities and after many conversations with them I am certain they will hit the mark with the courses.

I already have outlines for the courses, we just haven’t had time to post them on our website yet.  Be looking for more information on each of these (and more!) soon.  Agile For All is devoted to making sure each role within an organization has agile training available.  With these two new courses we can address specific needs of architects, designers and testers.  While there are some courses available in the agile testing arena (mostly for unit-testing rather than acceptance-testing), I have not yet found any that will have the same in-depth approach taken by our new course.  I have only found one agile architecture course and it is a single day course which will simply not be enough time to give students the ability to understand and TRY various techniques.

If you are interested in any of these courses, or anything else that we may not have listed yet, please email info@agileforall.com and I’ll be sure to send you more information.  For me this is Christmas coming a month early because I’ve felt for 2+ years that these courses are needed in the industry!

Related posts:

1. Testing to find defects is waste Have you ever heard someone say that testing to find defects is...
2. Real World Agile Testing with Fit and FitNesse Another short blog entry.  This time it is to announce that we’ll...
3. Agile Architecture – It is NOT an Oxymoron! Many companies adopting agile have a hard time with the architecture and...

Have you ever heard someone say that testing to find defects is waste?  I’ve heard it and I’ve said it when teaching courses. But people aren’t hearing the whole message!The other half of the message is that testing to PREVENT defects is ESSENTIAL!  I recently read a comment to a question posed on LinkedIn which stated “Sufficiently professional and motivated development teams should (must) be able to produce defect-free code without the need for testing. All testing to find defects is waste!”  I’m sorry, but this comment is absolutely crazy.  This person is lumping ALL testing into the testing to find defects bucket.

Remember, there are two agile practices around testing that are both important.  They are unit testing and acceptance testing.  Both can be done in test first ways which give us UTDD and ATDD (unit and acceptance test-driven development respectively).  In UTDD the developer is testing that the code they wrote does what they expected it to do.  In ATDD tests are created to ensure that what the developer expected the code to do actually matches the requirement or user story.  You simply can’t have developers do both, or a misunderstood requirement will never generate an error!  The code will work, it will do what the developer expects, and amazingly what they expect it to do will also match the requirement.  Duh!

UTDD and ATDD are both examples of testing to PREVENT defects.  We can’t prevent a developer from writing code that has a defect in it, but we can prevent that defect from getting into the system.  That is the purpose of any sort of TDD.  Once the code passes all the tests the developer is done and as far as anyone knows the code works.  Now if anyone ever refactors the code, or changes something in another area that affects the code the automated tests will catch it before it gets to a verification stage or production stage.  Testing to prevent defects is a bit of a wrong phrase – it should be testing to prevent defects from being checked into the code base.  That’s really what we are trying to do.  When we use that definition it becomes clear that the testing MUST be done early and there is no “throw it over the wall to QA” mentality.

Figure out how to use both UTDD and ATDD in your organization.  Figure out how to automate both.  Figure out how to have continuous integration, nightly builds, etc.  If you can’t figure it out, invest in bringing in some consulting help to get over the hump.  The investment in time, training, tooling and dollars will be well worth it in a very short period of time.  Just ask yourself how many dollars are spent on support staff and defect fixing for production code, then determine how the organization would change if there were 50% fewer defects.  UTDD/ATDD will reduce defects in most cases by even more than that, but for most organizations a 50% reduction makes the investment in using the practices have a very short payback period.  Remember too that it is usually the best people in the organization being used to fix the most egregious defects – isn’t it worth a lot of money to have those same people be writing the next generation product rather than fixing the previous generation?

If you want to know more about one possible ATDD tool, go to our resources page and check out the book titled “FIT For Developing Software.”  It is a book about the open source tool FIT (Framework for Integrated Testing) available here.  It works for .NET languages and Java.  Some teams are incredibly effective with this tool, but it takes making the effort.

Again, if you need help with any of this, please, please call or email us.  We can provide assistance or point you to someone else that can help.

Related posts:

1. Real World Agile Testing with Fit and FitNesse Another short blog entry.  This time it is to announce that we’ll...
2. New to agile? Remember to eliminate waste When I teach any agile course I start out with the principles...
3. Agile antipattern: Burndown “wall” Does your team have an iteration burndown chart (giving credit only for...