Webinar: Preventing Embedded Software Bugs with TDD

Webinar: Preventing Embedded Software Bugs with TDD


Thank you for joining us today to hear about
Test-Driven Development. There is a lot to Test-Driven Development
and in this webinar we will only scratch the surface. We are going to start with one of the more
obvious benefits to Test-Driven Development, and that benefit is Defect Prevention. To give yourself a fighting chance to prevent
defects, first you have to make an admission. Slide 6: The First Step So repeat after me: “I am a programmer and
I write bugs!” Slide 7: The Waterfall Does your company follow a software development
life cycle modeled after the waterfall? The waterfall is a nice model, but it’s far
from reality. Let me ask you, when are requirements done? Given the timeline on the slide, you might
say, May 1st. Now that’s when the requirements were signed
off, but were they really done? Rarely, requirements are done when you discontinue
support for the product. Consequently, the other phases, Design, Code
and Test are also never done. There are feedback loops needed, like it or
not. Winston Royce, the first to define waterfall, warned that it does not work except for the simpler systems. How many of you are building the simpler systems? Slide 8: Iterative and Incremental Instead of waterfall, maybe you follow an
Iterative and Incremental approach like Agile or Scrum. The Iterative model is more honest as it attempts
to manage the evolving requirements design and code. Too many adoptions of Agile that I have seen
in the industry ignore the hard part adopting the engineering practices of extreme programming
to support iteration. Slide 9: Saving Test for the End No matter what approach you take, and how
careful you are, saving testing till the end risks it all. When testing is not continual and automated,
problems are sprinkled around the code, only to be discovered later. Slide 10: Doesn’t End Well When we get close to release, the problems
become all too evident and our efforts often degrade into a chaotic and reactive firefight. Slide 11: Wisdom From John Gall Why does this happen? John Gall, in The Systems Bible offered some
wisdom to us all. Your program will have bugs, and they will
surprise you when you find them. Slide 12: Celebration Spoiled CommitStrip.com caught the all-to-frequent
release party spoiler in this cartoon. The developers are celebrating the release
and the bug comes and interrupts a party. Quickly, one of the programmers Samurais the
bug and everyone feels relief. Slide 13: The Last Bug, Really? Now they can get back to their celebration
after killing the last bug. But look what’s hiding nearby in virtually
every corner. Some of the undiscovered bugs looked pretty
worried, why does this happen? Is there some law of Software Physics that
says, we must write bugs and then kill them later? Do the bug just magically appear from nowhere? Slide 14: Our Workflow Produces Defects Of course not, we put the bugs there. It’s because of how we work. When development is followed by test what
will we find? Defects, bugs, but unfortunately this approach
leaves many bugs still in hiding. Slide 15: The Physics of Debug Later Programming The most popular way to program on planet
Earth is what I call Debug Later Programming. Let’s look at the physics of this popular
technique and its risk. Slide 16: The Physics of Debug Later Programming People make mistakes; we are quite good at
it. Unfortunately, many of our programming mistakes
go unnoticed and are left in the code. Slide 17: The Physics of Debug Later Programming Sometime later, the mistake is discovered,
maybe you discovered the mistake yourself a few hours after writing the code. Maybe your colleague in system test discovers
a bug a few months after you wrote the code. Maybe the customer discovers the defect months
or years after the mistake was originally made and the writer of the bug is long forgotten. Slide 18: The Physics of Debug Later Programming Now that we know that bug exists, and it’s
been deemed important enough to fix. How long does it take to find the cause of
the bug? That question is not answerable. This is a huge risk to our product, customer,
schedule and reputation. Slide 19: The Physics of Debug Later Programming But once you find the cause, it may be very
easy and fast to fix. But what if the broken code has been part
of the system for a while? What if you had a Boolean function that is
exactly wrong? Other working features would depend on it
being wrong. Your system is working by accident due to
offsetting errors. Fix this bug and you create a batch of new
problems to be discovered and debugged later. Slide 20: Edgar Dijkstra’s Vision Let’s hear what Dr. Edgar Dijkstra pioneering
Computer Scientist, has to say about debugging. “If you want more effective programmers, you
will discover that they should not waste their time debugging, they should not introduce
the bugs to start with.” Unfortunately, Dr. Dijkstra did not tell us
how to not write bugs! He was researching former proofs of code correctness;
he discovered no silver bullet. What if there was a way to find an approximation
of Dijkstra’s dream of not writing bugs in the first place. Would you be interested in giving up Debug
Later Programming? Let’s look at the case study. Slide 21: The Famous Zune Bug Some of you may recognize this device. It’s a Microsoft Zune, a music player that
competed with the iPod. On December 31st, 2008, the Zune had a problem,
it did not play music, it became a pocket warmer as its battery quietly drained. For a music player, not working on New Year’s
Eve is certainly a category one defect. What else is special about December 31st 2008,
besides being the worst day for a music player in the western world to break? It was a last day of leap year. Slide 22: The API Responsible for the Zune
Bug Many bloggers dug into this bug. The broken function was in the clock driver,
converts the number of days, since January 1st, 1980 to a data structure containing the
year, the month, the day of the month and the day of the week, except for the last day
of the leap year where it never returns. Slide 23: The Function Responsible for the
Zune Bug I downloaded the driver and inspected it. I concluded along with many bloggers who said
that the cause the defect was this line of code. Slide 24: It Only Takes One Line of Code Was the defect cause by a missed requirement? No, the programmer knew leap years have 366
days. How do I know the programmer’s mind? The evidence, Your Honor, is in the code before
you. About 9 out of 10 articles offered that if
the conditional that specified greater than or equal rather than just greater than, there
would have been no Zune bug. I came to the same conclusion, but I have
come to not completely trust myself in these matters, so I wrote a test. Slide 25: The Test That Could Have Prevented
the Zune Bug First, I had to write a function to pretend
to be the clock chip and produce the right number of days for the 366th day of 2008,
the last day of leap year. I ran the test. My test run never returned, just as expected. I applied the fix, but much to my surprise
the test failed. The code under test reported that it was Tuesday,
2009, December 0. Now, good news is, we just have a category
two defect for a music player that also doubles as an alarm clock. Slide 26: Defect Not Prevented One test could have prevented the Zune bug! Slide 27: Bugs Can Hide Anywhere, What Has
To Be Tested? The programmer knew about the leap year requirement,
they just got the code wrong. They did not choose to make a mistake there. Bugs can hide anywhere, so what do we need
to test? Slide 28: We Need to Test We need to test everything. That sure sounds like a lot to test. Slide 29: How Many Tests Are Needed To Test
These Modules Together? Don’t worry too much … yet. Let’s look at a couple simple models. If you had three modules that each have 10
paths through them and you chose only to test these modules together, how many test cases
would you need? The math is not difficult but the answer is
discouraging, you need a thousand test for the simple system, forget it. We are not writing a thousand tests for this
unless there are lives on the line. Slide 30: How Many Tests Are Needed To Fully
Unit Test These Three Modules Is there something else we can do? Yes, unit testing. If you unit test each of these modules, you
will need 10 tests for each module and a handful of integration tests. For the skilled engineer, this is probably
less than a day’s work. A modular system can be thoroughly unit tested. Slide 31: Integration Tests Are Needed Great news, the code compiled and linked,
the APIs are compatible, the unit test pass, ship it. Wait, not so fast. Just because you cannot practically do thorough
testing, of the system as a whole, does not mean you do not need integration of load test. You need a subset of these tests to make sure
that the parts of the system are interacting properly. So how do we get started? Slide 32: The First Step To Recovery Repeat after me, I am a programmer, and I
write bugs. We need to accept that we are responsible
for the bugs we produce and look for way to improve. Maybe we have to stop doing Debug Later Programming,
what should we do instead? Test-Driven Development, also known as TDD. Slide 33: Test-Driven Development Micro-Cycle In Test-Driven Development, programming is
done in a tight feedback loop. Start by writing a test for a small behavior
you want to add to the code. Just one test not all the tests, just one. Slide 34: Test-Driven Development Micro-Cycle In the early stages, the test calls a function
that does not exists, the test should fail the build. If it does build, it means that the function
name is already declared and implemented somewhere. But in the early cycles, you would first expect
an undeclared function error. Look at the compiler error, make sure it’s
the one you expected. Slide 35:Test-Driven Development Micro-Cycle Add just enough code to satisfy the compiler
and then the linker, rigging the function to fail the test. By writing code that will fail the test first,
we assure that the test can tell when the production code misbehaves. Slide 36: Test-Driven Development Micro-Cycle Once you have seen the test fail, make it
pass, but don’t write the expected full implementation of the function, write just enough code to
assure that the current implementation will pass the new test and pass all the prior tests. You are moving the code, one small step closer
to being done. One defined and test behavior at a time. It’s normal at this point to have hard-coded
return values and partial implementations. Writing just enough code to pass the test,
means the implementation will be incomplete until all the tests have been written. By the way, people new to this are going to
find this disturbing. Slide 37: Test-Driven Development Micro-Cycle After several TDD cycles are complete, you
may think of a better way to solve the problem, you might want to change your name of some
variables or functions or remove some duplication. You may discover a flaw in your approach,
the tests are there to help. You can safely refactor your code because
you have tests to finding each behavior. Refactoring is how Test-Driven Developers
keep code clean for long and useful life. Slide 38: Test-Driven Development Micro-Cycle You continue test driving, one behavior at
a time. Cleaning the code as you go and once you can’t
think of any more tests, you are done. Not only does TDD help you get the code to
work in the first place, it produces a regression test suite to support you or someone else
when you find that code needed to be change in the future. It provides a document defining each behavior. Making sure that code is correct is so important;
we test every line, every branch, every conditional. Making sure, code is designed well and understood
and changed is so important. We make design part of everyday. We make it safe to refactor and we do refactor. Slide 39: Development and Test Are Done In
Parallel You can see that the Test-Driven Developer
does not do all the tests and then the code, as the name might imply. We do test and code in parallel. Slide 40: Development and Test Are Done In
Parallel Each TDD cycle we add one small behavior. When all the tests are written, the code is
done, at least for now. Slide 41: The Physics of Test-Driven Development Let’s look at the physics of Test-Driven Development
and how it is radically different from Debug Later Programming. And my apologies to any physicists out there
for taking liberty with your science. Slide 42: The Physics of Test-Driven Development As humans we make mistakes, we are really
good at this. Slide 43: The Physics of Test-Driven Development But with the small steps of TDD and the immediate
feedback of the Unit Test, the mistake is right in your face, not hidden like the bugs
in the comic we saw earlier. Slide 44: The Physics of Test-Driven Development Usually the mistake is obvious, we can find
the mistake quickly. Why? It’s in the code you just changed in the last
few minutes. It’s fresh in your mind. Slide 45: The Physics of Test-Driven Development So fix it, mistake corrected, defect prevented. But sometimes your new test passes and several
previous tests fail, warning you about a previously defined behavior that’s now broken. You likely overlooked some detail that’s not
currently foremost in your mind, but it really matters. Back the change out preserving existing behaviors
and rethink the problem. When this happens to you, that’s when you
really get hooked on test-driven. Before I started doing TDD, I thought I was
good at programming. Now I realize that mistakes are the norm and
not the exception. I am a programmer and I write bugs, though
not so many anymore. Slide 46: Test-Driving a Circular Buffer Let’s look at an example. You are all likely familiar with the Circular
Buffer or a FIFO: an array of integers with a couple of indexes to manage where to put
new integers and where to get all values from, so we can get values in our First-In First-Out
order. Slide 47: Envision A Solution You could envision an implementation with
the data structure holding couple of indexes, maybe a length, maybe some other things and
a pointer to an array, to hold the values in. Slide 48: Make a Test List Before writing the first test case, we could
make a list of what to test. We can anticipate the need for tests around
various special cases, like tests around the empty case, tests around the full case, making
sure that we get items in First-In and First-Out order. What do we do about some of the error cases
like putting to a full buffer and getting from an empty? Did wrap-around really work? We don’t have to worry about completely being
exhaustive when we are looking at this list of tests because when we get into the details,
we will find the tests we didn’t think of while we were brainstorming here. Slide 49: TDD State Machine There is a very regular pattern that we are
going to follow while we are test driving. And this pattern helps us to free our brain
to problem solve, and to better manage some of the details we are working with. So when you start, you choose a test, you
write the test. The test probably won’t compile, make it compile. Then the test probably won’t link, make it
link. But make it link such that the implementation
is wrong and will fail. Watch it fail, then make it pass. And once you make it pass, after you have
done a few iterations through here maybe you are going to want to clean up. But right now it’s our first iteration. It won’t matter. Pick another test. Keep going around this loop. When you can’t think of any more tests, you
are done. Slide 50: CPPUTEST Test_Group To test drive, you will need a test harness. You can build your own or you can choose one
of many open source test harnesses available. You don’t need to buy expensive test tools
to do TDD. Here is CPPUTest. I am one of the authors of CPPUTest. Now, we built it to do TDD. It uses C++ but the C++ is hidden in macros
and works well for unit testing both C and C++. Tests are organized into groups in this page,
you can see there is a test group and in parenthesis its called (CircularBuffer), each test is
part of a group, we will get a fresh copy of the CircularBuffer because setup is called
before every test case and teardown is called after every test case. In addition, the variable buffer will be available
to every test case. Slide 51: The First Test CPPUTEST Test Case Let’s look at a test case. See the macro test and it has two parameters:
CircularBuffer which is the name of the test group and empty_after_creation, that’s the
name of the test case. In the body of the test we are checking a
condition; we are checking that the return result for CircularBuffer_ IsEmpty that comes
back true, this is what we would expect after creating a CircularBuffer, that it would be
empty. From this point we are going to have to go
through a few steps where we will get a compiler error so we go add the signature up to the
header file, then we will get a linker error in which case we will go ahead in implementation
but we will rig the implementation to return false, so that this test will fail. And once we have seen that failed test, then
we will go and make this change. Slide 52: The Simplest Implementation to Pass
All the Written Tests To make the test pass, we don’t go put the
final implementation that we can imagine in the code now. What we do is we hardcode it to return the
value true for the test cases we have, that’s the appropriate implementation. The fact that we can get away with hardcoding
a return result needs we don’t have enough test cases so what’s the next test case we
will write? Slide 53: Write the Next Text to Drive Out
the Hardcoded Return Result Let’s put something in the buffer and that
it won’t be empty anymore. This gives us two things to do, we are going
to have to define an API for put and we are going to have to evolve the implementation
a bit to, IsEmpty to make it work for both test cases. Hardcoded return result won’t work anymore,
so we are creating a new API, we are going to go through the compile link fail stage
for that and then we are going to advance the implementation a little bit towards the
end that we envision for this function. Slide 54: The Simplest Implementation to Pass
All the Tests We have in mind that we can use the input
index and the output index as a way to distinguish empty from not empty. So if the input index and the output index,
both point to the same location, the CircularBuffer would be empty. To do a put, we would advance the index. We will need to do some other things, we will
need to store a value etc., but we are not even getting a value at, so we don’t need
to store anything. And likewise, well, input index equals 0 that
will be true for the test cases we have right now, later we will need to make the conditional
more comprehensive, but right now, I think this will work for the test cases that we
have. Slide 55: Grow Your Code One Behavior at a
Time We know IsEmpty is incomplete, but close to
being done. If we test the transition back to empty, we
are covering another boundary case and defining a new API, CircularBuffer_Get. Let’s drive our CircularBuffer another step
closer to being complete by implementing the empty after removing last item test. Slide 56: Grow Your Code One Behavior at a
Time After going through the header file and link
stage and watching the test fail, our implementation that passes the test looks like this, CircularBuffer_Get
increments in outdex, number of the output index that we have just added to the data
structure, returns a (-1) because we have to return something to keep their compiler
happy. We didn’t have to do any change to put and
IsEmpty now we are doing the comparison to make sure that we are defining empty to be
the condition on both the input index and the output index to pointed to the same cell,
that function might be done now, even though putting together are not done, IsEmpty might
be done depending on the approach that we are taking for CircularBuffer. Slide 57: Repeat Until Done We continue in small verifiable steps, one
behavior at a time, creating an executable specification of all the behaviors of the
CircularBuffer. And as we advance our design, if we discover
something is not quite with our implementation, if we have a wrong idea of what might work,
we have the existing behaviors, well defined, executable; the cost of retesting is zero,
that I can run as I restructure the inner workings of my CircularBuffer. Slide 58: One Test at a Time is a Fundamentally
Different Way I expect that looks really strange to most
of you watching and listening to this. What you are used to, Debug Later Programming
is having some code without a new feature, envisioning a design that would handle that
feature, making all those changes all at once and then spending time, trying to figure out
what went wrong and actually ending up somewhere different than you envisioned. With Test-Driven Development, what we do is
we try to keep the code working all the time. We are looking at programming differently. We are adding one behavior at a time rather
than writing one file or writing one function or adding one data structure. We are thinking about one behavior at a time;
we are establishing cause and effect. I believe it’s a very engineering focused
way of working. What problem am I going to solve? Well, my problem is I don’t have a CircularBuffer,
they can tell if it’s empty, well I can do that. Now I can’t tell if it knows when it’s not
empty, well if I put something and it wouldn’t be empty, one behavior at a time, I grow the
behavior of the code undertest, of the code under development. Slide 59: You Can’t Do It All At Once John Gall, in his wisdom told us that “A complex
system that works is invariably found to have evolved from a simple system that worked.” I read that quote first in the early 90s in
the context of Object- Oriented Design. I consider TDD is built on that observation. In TDD we evolve code, one behavior at a time. Slide 60: A Skill To Hone People new to TDD think it is odd and dangerous,
they see all these procrastination going on. Procrastination is a skill to be honed in
practice by Test-Driven Developers. Procrastination is a survival technique; it’s
gotten a bad name from lazy people using it. I have come to see that procrastination helps
Test-Driven Developers manage complexity. Slide 61: Managing Complexity It seems that there is psychological research
to back that up. I have been reading a book called Thinking,
Fast and Slow. The author explains that humans can only manage
so much complexity. We manage complexity by holding most variables
constant in changing one at a time. We focus best on one problem at a time. How many details is a human mind good at keeping
in the foreground? Well not so many. How many details must be correct for CircularBuffer
to be correct? By my account at least a dozen. Debug Later Programming requires you to keep
a lot of balls in the air. TDD limits number of balls in the air to just
a couple. Tests become part of your long-term memory,
defining what the code is supposed to or what the code must do. These requirements are checked every time
you make any change to the code automatically. Slide 62: How I Discovered TDD I would like to tell you about how I discovered
Test-Driven Development? In 1999, I attended the first Extreme Programming
Immersion, a five day training class where I met Kent Beck, the author of Extreme Programming
Explained. Kent was performing a demo of test-first programming
as TDD used to be called. The demo was in Java using the JUnit test
harness. At first it looked kind of crazy, but as he
proceeded, it struck me, this is really important. Slide 63: Target Hardware Bottleneck I had 20 years of experience when I saw Kent’s
demo. Every product development effort met waiting
for hardware to run my code. Once we had hardware, we chased down the bugs. We tried everything to get that code in shape
before the hardware arrived. And virtually every case, extreme heroics
were needed to make the date. And in too many cases, extreme heroics were
not enough. As I watched Kent, it struck me. Code can be run in a test harness. In an embedded systems development, it’s a
norm to only run code in the embedded system. It is also not uncommon to be debugging the
hardware platform along with the code right up to the ship date. Integration never goes smoothly. Why can’t embedded programmers test code in
a test harness independent of the target platform? Well, of course they can, maybe not all the
code but a lot of it. And then we would not have to wait for the
execution environment. We could get code working in advanced of hardware
delivery. We can avoid the target hardware bottleneck. Slide 64: TDD for Embedded Uses Dual Targeting! You probably never knew what Homer’s Doh was
all about, frustration from debugging on the hardware. Let’s look at how we can separate production
code from its platform dependencies and test it effectively in the unit test harness. Slide 65: Work In The TDD Fast Feedback Environment To avoid debugging on hardware, move some
of your code off-target, and then you can do TDD in a fast feedback unit test environment. You should be able to do, an incremental unit
test build in only a few seconds. This really helps you keep your mind on the
programming problem. Slide 66: Periodically (or on check-in) Run
a Target Build Too When you are satisfied with your code working
off target, run the target build and make sure that you have written code that’s compatible
with the target compiler. Slide 67: Run Test in Eval Hardware You can’t setup a test build for Eval hardware
that uses the same processor as the target hardware. With an Eval system you can relieve memory
constraints and possibly have a more reliable execution environment that is close to your
target environment. And this lets you find portability problems
with your code. Slide 68: Run Test in the Target Depending on your target platform, it may
be possible for you to run the tests in the target hardware. And if this is the case, you can also start
to write some test that exercise your actual hardware. Slide 69: See If the System Works Finally, you have to see if your product works,
you are likely to find integration problems and timing problems, but you probably won’t
find many annoying defects making it into the target. You have prevented those defects from seeing
the light of day. Slide 70: Each Stage Detects Different Kinds
Of Problems Each stage finds different kinds of problems,
in Stage 1, when you are doing TDD, you are finding logic, interface, boundary conditions,
design, and modularity problems. Stage 2, you are finding compiler compatibility
problems. Stage 3, you are finding portability problems. Stage 4, same as Stage 3 as well as you can
start to use test cases to explore the functioning of your hardware. And finally, Stage 5, you are finding out,
did I get the features right? Do all the pieces fit together? Slide 71: Automate Builds With CI Server You may think that this is a lot of extra
work, it does take some time to setup, and it does help you go faster, it relives tedium
and boredom and makes sure you detect problems quickly. Stages 2 through 4 can be automated through
applying continuous integration servers, many of you are probably using CI servers like
Jenkins and CruiseControl. These handy tools are direct descendants of
Extreme Programming, and the practice continuous integration. Slide 72: Continuous Integration For Embedded Let’s look at what CI means to embedded programmers. Slide 73: Developer Works in the TDD Fast
Feedback Environment The developer programs in the fast feedback
environment of TDD and gets all their unit tests to pass. Slide 74: Check In Their Work When Tests Are
Passing Developers check in their working code. Slide 75: Continuous Integration Server The CI system monitors your source code repository
and what it notices it changes, it checks out the work, does its own local builds to
make sure that software is rebuildable and runs a unit tests that the developers have
created. Slide 76: Test Management System After the off target tests, have been successfully
run, the continuous integration server, builds for the target or builds for Eval board and
maybe builds for simulator and runs a code there. If any problems are detected, the developers
that contributed to that build are notified. Problems don’t get to stay in the system for
very long, they are detected right away which is the easiest time for a developer to find
what they did wrong. Slide 77: Test-Driven Developer Automate TDD developers, automate tedious and error
prone steps. You can see TDD is following in that same
principle. Retesting code is boring although writing
tests is interesting and being able to rerun all the test you wrote before is very rewarding
when they help you find errors and help you keep your code working. So, enough about that continuous integration,
let’s get back to another example of TDD in an embedded system. Slide 78: Testing Code with Dependencies The CircularBuffer is a nice example to get
started with because people understand it, and it’s small, but what about real code? Real code has dependencies; well CircularBuffer
is real code that most of the code that really matters to us has dependencies. Look at this code, what if we had a home automation
system with a Light Scheduler and every minute the Light Scheduler is going to get a callback
from your real-time operating system to wake up and looking at schedule. And if there is anything on schedule that
needs to be done, the Light Scheduler would either turn on or turn off a light. And you can see here this network of objects
will need a real-time operating system, will need some hardware to run this. Slide 79: Managing Dependencies with Interfaces The test-driven approach makes us look at
our work little bit differently. Instead of thinking the Light Scheduler, is
using a real-time operating system and using a Light Controller, let’s think of it as using
interfaces, an interface to a time service, an interface to a Light Controller. Let’s break the dependencies on the hardware
so that we can actually test this code somewhere else. Slide 80: Testing With Test-Doubles (Spies
and Fakes) If we live the knowledge of the Light Scheduler
to just the interfaces for the time service in the Light Controller, we can do something
interesting. We can swap the hardware and operating system
dependencies for a Light Controller Spy in a Fake Time Service. For a Fake Time Service, it can be whatever
time we want. And for Light Controller Spy we don’t need
any hardware for that, the spy is just going to remember what the Light Scheduler told
us. We can setup various scenarios of things scheduled
and times that occur and see what the scheduler does. And get this all ready without hardware. We will not deploy it and run it in the hardware
as well. But we would like to work in this fast feedback
environment of Test-Driven Development, so we can make sure that this important code
works. Slide 81: Test Each Behavior Independent of
HW and OS We could write a test like this, a test for
the LighScheduler that assures that the light does not get turned down at the wrong time. You have got the API, LightScheduler_AddTurnOn,
so we are going to add a TurnOn event for Light number 3 everyday at 1200. We are going to fake out the day in the minute
to not be the right time, not be 1200. And then we are going to simulate the callback
to the scheduler and we are going to make sure that the spy reports it nothing has happened. Slide 82: Test Each Behavior Independent of
HW and OS The next test, LightSchedulaer, light_turns_on_at_the_right_time
is almost the same as the previous test except you will notice that the minute setting in
the FakeTimeService says to 1200th minute. So this is the scheduled time and when we
simulate the callback to the LightScheduler, you will see that when we ask the spy, it
tells us that light 3 is on. Slide 83: I Don’t Like Testing An alternative to proactively test-driving
features is to manually test some after the code is written. This may be interesting at first but quickly
becomes boring and repetitive and error prone. Maybe we can find somebody else to do it to
make sure that our code works. I don’t like testing, that seems like something
odd for the author of Test-Driven Development for Embedded C to say, I don’t like testing. Testing is a pain, Test-Driven Development
isn’t. Slide 84: TDD is Fun! TDD is Fun!, it’s a challenge, it’s like navigating
a maze, if you like programming you will like TDD, you are writing code. It helps me see my mistakes, it helps me learn
it structured problem solving, there is a lot to like there. Slide 85: Adding tests after is not so fun Adding tests after that’s not so much fun,
especially after manually testing code first, either way it feels like extra work. It’s harder to be thorough, you get this idea,
your code is well tested before it is. Slide 86: TDD Cannot Be Dictated TDD cannot be dictated to a development team. It is voluntary, it is a skill that must be
learned, practice and mastered. But the business can and should dictate test
automation. Slide 87: Companies Are Organized As If This
Was True Let me go through a simple model. Your company is probably organized as if this
were true. The effort to test is proportional to the
effort to develop. How can I say that? Well, do you have a fixed number of developers
and a fixed number of testers? Or do you allocate fixed proportions of development
and test effort that are fairly consistent year after year. This is true in most companies I visit. Slide 88: But Systems Don’t Like Being Changed But systems don’t like to be changed, when
you add a new set of features, you must retest all prior working features. But who does that, when they rely on manual
test. Slide 89: With Each Feature All Prior Features
Need To be Retested When you add the third set of features you
need to retest the first two. Slide 90: With Each Feature All Prior Features
Need To be Retested And so on. Slide 91: With Each Feature All Prior Features
Need To be Retested Don’t worry about the proportions, I think
I am under-representing the effort growth by assuming the retest time is less than the
original test time. The model also shows growth is linear, let’s
say that’s optimistic. This is a conservative model. Slide 92: Unsustainable Growth of Manual Test
Effort Manual test is unsustainable. The available test effort is fixed while the
needed effort continues to grow. What happens next, we cut corners and test
what we think needs to be tested. Slide 93: The Untested Code Gap But more test effort was needed and this produces
what I call the untested code gap. Slide 94: Lightning Strikes In the untested code gap, risk accumulates
and at the worst possible time probably right before release, lightening strikes and it
all goes up in flames. A firefighting ensues a death march. How do we conquer the untested code gap? Test automation. Automation keeps the cost of retest low. We can’t make it zero but we can get a lot
closer than we are. TDD is an effective way to produce unit test,
in the skill to automate the other test you are going to need. Slide 95: Defect Prevention Repeat after me, “I am a programmer and I
write bugs!” If we want to prevent defects, we need to
stop practicing debug later programming and rely on manual test. I hope you can see the possibilities of TDD
and how it could help you detect your programming mistakes before they become bugs. There is more to do besides TDD. TDD won’t prevent all defects but any serious defect prevention effort must include Test Drive Development. Slide 96: There is a lot more to Test Driven
Development There is a lot more to Test Driven Development than
we can cover in one short webinar. We encourage you to learn more. You can read my book, you can attend one of our training courses.

1 thought on “Webinar: Preventing Embedded Software Bugs with TDD”

Leave a Reply

Your email address will not be published. Required fields are marked *