|
|
|
|
Brian Marick is
the author of "The
Craft of Software Testing", editor of STQE, and Chair of Agile Alliance
board among other things. We interviewed him recently and asked him
questions about his current work, book that he plans to write, Agile
manifesto, testing patterns and some more.
If
you want
to ask him more questions or you want to send any feedback on this
interview, please send it to webmaster @ whatistesting.com.
|
1. Brian, please tell us
about something about yourself, your interests, your hobbies and
what do you do in your free time.
|
I
graduated from the University of Illinois in 1981, wanting nothing more than to be a
programmer. In my first job, my first boss told me "We don't really
know anything about you, so we'll put you in testing where at least you
can't do any harm." (How wrong he was!) After that, I vacillated
between programmer and tester for years, finally settling on testing. I
became an independent consultant in 1992. In recent years, I've become
highly associated with the Agile software movement (I'm chair of the
Agile Alliance board and webmaster). Perhaps that means I'm again
vacillating between programmer and tester, but I prefer to think that
it's time for the two roles to be much more blended than they have been.
I don't have any free
time to speak of. Seriously.
|
2.
You wrote "The craft of software testing"
in 1995. This book is different from most of the testing books because
it targets
developers or
testers who do structural testing. What made you write that
book?
Why do you think
there are so few books on the topic?
|
At that time, it seemed to me
that all
testing books left big gaps for the reader to fill. They didn't work
through their examples in enough detail. I wanted to give that detail.
And at that time there were two contending camps of academic testing:
one stressed testing based on faults in code (mutation testing and
allied techniques) and the other stressed programs as formal
mathematical objects that you could relatively mechanically derive
tests from. I favored the first camp, but in a less academic way. The
core idea of the book is that testers learn by experiencing bugs and
cataloging ways to find them.
I don't particularly care for the book, by the way. The core idea is
good, but the presentation is lousy. And its assumptions about the
programming process is quaint in this day of test-driven design.
|
3.
Do
you have plans to write another book?
|
I've just started one, to
be titled _Driving Projects with
Examples: a Handbook for Agile Teams_. Ten years
ago, it would have been called something like _System Testing_. That
"testing" isn't even in the title indicates that we've come a good way.
The drafts of the chapters are available at http://www.exampler.com/book.
|
4. You are
doing PhD. Can you please share the
subject with us? How do you think this is going to impact the software
development and
testing as we understand today.
|
I
gave up. I am likely to contribute to an edited sociology volume this
year with a paper called "Rule Following as Manglish Behavior", which
casts test-driven design and refactoring as an example of what the
sociologist Andy Pickering calls "the dance of agency". There's no way
I could get that kind of stuff past an engineering PhD committee, and
what would I want a sociology degree for?
|
5.
Would you
like to share some of interesting experiences of being a technical
editor of STQE magazine?
|
I'm
afraid that there aren't really any. It's a rewarding job, especially when I take a new author and
help her get her first article down, but it doesn't make for stories.
|
|
6. How did
Agile Manifesto happen and Where do you think it stands
today? What is your assessment of the impact agile alliance
has made?
|
It happened
because there were a bunch of people who had devised similar
methodologies that were starkly different from the orthodoxy.
It was Bob Martin, I
believe, who decided that some of these people should get together and
hash out what their methodologies had in common. What was the core that
made Scrum and XP feel so compatible, whereas even a stripped-down
Rational Unified Process felt just... different? The Manifesto was our
best statement of that core.
As for the impact? I'm
writing this on the plane back from the Agile Development Conference.
Do you realize what it's like to talk to people who love their job? Who
feel like _at_ _last_ they're allowed to produce at their peak? Without
the Agile Manifesto, many people's jobs would be worse.
|
|
7. How
do
you think that large organizations can adopt agile methods and what are
the hurdles that many of these organizations see? How
do you think these organizations can be educated about it?
|
I'm not the best person to ask
this. I usually work bottom-up rather than top-down.
So what are
some of the differences between big companies and small companies? Some projects will be larger,
and they're more likely to be
distributed. There are likely to be more layers of hierarchy. They might have more of an atmosphere of
formality. They exist because of economies of scale that capitalize on
uniform processes. People can
specialize more.
It seems to me that
those things make agile processes less appealing, sometimes for good reasons, sometimes
for bad.
- Agile's sweet spot is
the small collocated team. People have had success with larger teams and
distributed teams, but it's probably not the place you want to start. Now,
there's no reason not to do small projects agilely, but the desire for
uniform processes (perhaps in an Infinitely Tailorable Form) argues
against that.
- Agile seeks to put
the people who produce the code in direct contact with the people who pay for it. The
larger the organization, the more traditional intermediaries who are ready
for a political struggle.
- It's so messy and
unsophisticated. 3x5 cards everywhere! Crude charts drawn on walls! Stuff piled everywhere,
right out in the open, instead of hidden away behind cubicle walls. A
scorn of Powerpoint status reports.
No project tracking tools that produce arbitrary reports at the click of button.
- Agile projects favor
generalists over specialists. That breeds resistance from specialists.
But there are other
hurdles that don't (maybe) depend on company size:
- Project managers are
often a hurdle. Agile projects represent a shift from directing other people to being
directed by them. The best agile project managers serve their team by
protecting them from distraction and by removing obstacles from their
path. For example, if the team wants an open workspace, it's the
project manager's job to make it happen, no matter what
the Facilities Department says. Since a lot of project managers are good at directing
people, it's not surprising they resist such a big shift.
- The message of an
agile project is "You *will* be able to ship an X on the date you choose for the amount
you choose to spend. On that day, it will be the most valuable X we could
have produced for you. But you can't know what that X is today." That
drives some people crazy. They want to know everything: scope, time,
and budget. I think they're fooling
themselves -
but they don't, and they want to keep trying for all three.
Right now, agile is for
the visionary business people, ones who believe it's actually possible for software not
to disappoint. It will move into
the mainstream as the majority talks to enough visionaries to gain confidence that it can work for them
(and it will not work for all of them).
|
8. What is
the motive behind http://www.testingcraft.com? Do you think it is
receiving enough support from the community. Especially for the
techniques and projects? If you would some help, what kind of help
would it be?
|
I don't think I
can describe the motivation better than in the motivation page: http://www.testingcraft.com/motivation.html.
In short, expertise isn't all that much about grand unifying theories;
it's about knowing a wealth of small techniques and having a tacit
understanding of when to apply them. The site was a place for testers
to share techniques. But they didn't. I don't really know why. It
probably has something to do with the way very few testers test in
their spare time.
I've moved on from that project. I'm content to let it stagnate.
|
9.
What open source testing tools you are working on? What open source
testing tools would you like to see being developed?
|
Most of my work these days is on Ward
Cunningham's Fit
(http://fit.c2.com). But I don't do
that much tool work these days.
When I'm not wearing my consulting hat, I'm mostly experimenting with
different styles of blending testing and programming. I do that by
building code test-first.
In many ways, we live in a golden age of open source test tools. That's
driven by the fact that many programmers on agile projects love testing
so much that they are building testing tools for us.
What we need most isn't tools. We need testers who are able to use
scripting languages. My favorite is Ruby. (http://www.rubycentral.com/book/)
Such testers will be able to
use tools more effectively, and do great chunks of their job more
effectively.
http://www.testing.com/writings/bypassing-the-gui.pdf
|
11.
How do you see the play between testing by developers and testing by
testers? My personal thought process is that once unit testing and its
automation becomes a norm, higher level tests would be created by
combining these unit tests the same way programs are created from units
and testers will have to acquire those skills. But 'black box testing'
as we know is not going to go away and testers who have domain
knowledge, user interface knowledge etc.
|
Except for the agile programmers, I
haven't seen any particular
evidence that
programmers are testing any more than they ever did. And it's now the
common wisdom that the "unit testing" that programmers in XP (say) do
is not primarily testing. It's instead a way of working through the
design and smoothing the act of programming. I expect that to continue
and to be extended to system tests (whole product tests, end-to-end
tests). Such "business-facing" examples will increasingly play the role
that requirements are supposed to. But I don't find it that useful to
think of them as tests (though it is awfully nice that they tell
programmers when they've changed something and caused unintended consequences
- so they are also "regression" tests).
Of course, those
examples are made by people, so they'll be incomplete, wrong, etc.
There's still a need to critique work after it's been completed. I hope
that exploratory testing really comes into its own in agile projects -
my experience is that programmers love it, and they appreciate the
testing techniques that make exploratory testing more than just poking
at keys.
But there's an
interesting twist here too, as well. In an agile project, the backlog
of stories (descriptions of desirable business value) can grow at any
time as people get ideas. I see exploratory testing as not just about
finding bugs, but also about getting ideas for new features. Elisabeth
Hendrickson and I did a tutorial on exploratory testing this past week.
But instead of having the attendees test software, we had them design a
game, exploratorily playtest the game, revise the game, and then test
it again. If I recall correctly, the attendees said that the first set
of tests were mainly about finding problems, but the second set both
found bugs and also led to cool new ideas. I want teams doing
exploratory testing to have the the attitude that they're in it to find
both.
So it seems that
testing, in the conventional sense, is getting squeezed out. And I
think it is. That's not to say that the people who are testers now have
no place on an agile project. Good testers are skilled at translating
between the technology domain and the business domain. They're better
at coming up with telling examples than programmers or business people.
They also seem to be better at noticing when the business experts
become "captured" by one of the interest groups that cares about the
project. They seem better at maintaining objectivity about the product
even when part of the team. All these are traits that testers can bring
to the team (partly by spreading them). So the *good* testers will fit
well in the agile team, though not doing the same thing they do
today.
What they'll do, I
think and hope, is collaborate very closely with the business experts
and programmers to get a first set of guiding examples running quickly
in an iteration. The programmers can then launch into making those
examples work while the testers and business experts collaborate to
generate more tests (bringing in the programmers as needed). The goal
is that the programmers should never lack for an example to guide them
and give them confidence. As soon as they finish one, the next one
should be ready.
There is, however, a
set of testers who will be doing the same thing they do today, I think.
They are those testers with a specialized technical skill, especially
ones whose tests don't lend themselves to the kind of up-front
automation that I've been talking about. I'm thinking of configuration
testers, security testers, performance testers, usability testers, and
the like. What are often called "ilities". What Cem Kaner calls the
"para-functional requirements".
I have a longish list
of ideas on this theme here: http://www.testing.com/cgi-bin/blog/2004/05/26#directions-toc
|
12. What are your
views on the testing patterns? What kind of favor have they found in
testers? What future do you see for them?
|
I
did start up an effort to get people together to create testing
patterns. It didn't work out well. We didn't get much. I think we just
didn't have enough of the right people. The agile movement grew out of
the patterns movement in a lot of ways, and the programmers-who-test
tend to be pattern-aware, so it's likely we'll get some patterns from
that direction. (There are some, for example, in Kent Beck's
_Test-driven Development by Example_, and there were mock object
patterns at last year's PLoP.) So we shall see what comes up in the
years ahead. |
|
