Automation people should be a lot more talented that your developers

Typical test automation people should be a lot more talented that your developers. That being said, all your activities related to test automation should be done in context. What do I mean? Well, Jonathan Kohl has a great article on when to do effective automation. Really, it comes down to having tools in your tool box and knowing when to use a hammer vs. a saw. You also get more out of having the machine run the work while a person evaluates it.

Given that, I do see a huge role in automation, but I'll preface it by saying it requires a critical thinking cap to be running to evaluate each test that is being automated, and in turn being evaluated. I have seen recently where manual testers handed their tests to automators who automated the test case. Sounds great, right? It took over 6 days to run ONE regression.

A critical thinker would have taken a group of tests and figured out the best path to automate the manual case. Also, they would have to adjust the data to get this completed. The manual tester also would need to log the nuisances of test they were doing, as if they were meticulous to detail.

So, developers, take note. If you are planning a code activity, talk to a critical thinking tester first. Get their input BEFORE you start coding. In fact, you could even get them to build (or better yet, review) your test fixtures and xUnit test suites so that you can have an idea of what they are trying to test. This is just a starting point, there are still many other ways this interaction can occur, but always within context!

Tony Stark, Engineer Extraordinaire ...

For many years, I recall watching movies and seeing how things didn't meet reality. There are a few things that bug me about movies, especially when computers are involved. For instance, the incessant "beep" noises computers make when the monitor is updating. Also, that computers seem to have a screen draw rate that of a typewriter. I know, leave your brain at the door, but its hard sometimes. Movies like Hackers drive me up a wall and try as I might to erase it from my memory, it appears that its still there. ... oh, I'll get back to the point.

I watched Iron Man this weekend, and it had its fair share of these annoyances, but the thing that surprised me the most is that it really stuck to some simple engineering principles to help create the story line.

I won't give away the plot or ending, but I'll elude to some of the details of him and his suit. He ends up building his suit, needless to say because they show part of the fabrication process in all the trailers. During that time he employs engineering principles quite well, started with a design phase, followed by a fabrication phase (but he built the first one by hand, after a few iterations, he automated the fabrication process). After that, he went into a recorded/session captured test cycle with a digital camera. During his flight tests, he records the session and debriefs after the session is complete. The whole test cycle was exploratory in nature, the only way to try it is by doing... in his words "you sometimes have to run before you can walk"

After he create the initial prototypes, he automated the fabrication processes after he put the suit through the paces. Five hours to build a new suit, so he can always have the latest build, usually with improvements after the test

Remind you of anything? Good lesson learned about engineering and how often we can easily take our processes and forget the basics. Great job by the movie makers too on keeping this philosophy alive and keeping true to the basics. Made it seem realistic by employing realistic techniques.

Presentation Zen and Garr Reynolds

I like non-fiction and documentaries. So, I also like learning stuff, kind of like TVO or PBS styles of shows. Have you heard of Google Talks? If you don't know about these Google Talks on YouTube, highly suggest going there and finding out. Very handy resource and a good way to feed your brain with some knowledge. I have basic cable (around 30 channels) at home and often there's nothing on. So, I sometimes feed on this stuff.

So, I was at home watching this presentation by Garr Reynolds about "Presentation Zen". Garr's whole thing is about presentations and how they can just be better. His style has a Steve Job's vibe, but with more pictures. His ideas are distilled down to simplicity for your presentations. I have often been caught in the trap of putting too much information on the power point.

Do you do presentations? I'm sure you do. I do. Either phone calls, webinars (I hate that name, but it makes sense), real presentations in front of people, etc. I tend do at least one of these types of presentations a week. Over the years, I try to do the Toast Master's stuff (speak slow, don't use "umm's", etc), but there's so much more to presenting than just the speaking style.

So, he's basically saying that there are many ways to present data. Lots of ways to represent data and power point is not the answer, its just a tool. Some of the thoughts are from other people and he distills down to a few points. Here are some of the basic tips I got from the presentation and they might help you out (I'm trying them out now):

• go to a white board or get a pen an paper and write out/draw out your presentations. Getting away from your computer will help you clear your mind and get away from the bullet point mentality. It also will get your creativity working because you're not locked down by the computer.
• don't use bullet points, but simple one line ideas. Feel free to use props or other visuals. If you need bullets to give people information, make a handout or give them a PDF so they can have it available. They are there to see the content, not read it.
• don't forget to play and have fun. That includes putting humor or your personality into the presentation.

Great video, got me started on other books to read too. Now it's time for me to go pickup the book! If you just want to see the video, the YouTube, link is above. If anything, take a look at his blog since there's a lot of information up there and its free.

Design for testability, TestNG

Another friend from work sent me an MSN link last Friday about an interview with the guy who created TestNG (a JUnit replacement) that allows for a few things beyond what JUnit can provide. The thing that piqued my interest was that it allows distribution of tests on slave machines, beanshell integration, and dependent methods for application server testing. Also, that TestNG is designed to cover all categories of tests: unit, functional, end-to-end, integration, etc. So I can get someone to create data driven or keyword driven tests and I can remotely execute them. I may not need FiT to do it now!

Anyway, I need to try this out. It would be nice to get this to be used for the simple remote execution aspect. I can then have unit tests tested across multiple environments and centrally controlled, rather than the other way around.

Oh! The creator of TestNG guy also has a great post about how TDD is not the end-all for software development; the anti-TDD movement. Interesting to see that they consider TDD just another tool in the toolbox.

More contemplation is required here for sure. But all really good information about how software development does coincide with testing on many levels.

Oh, here's an aside - I used to help teach people how to do TDD. Mostly as a 2 hour seminar. I'm going to redo the seminar to use testNG now so I can show off the evolution of test tools. I think there's a thought-lock on using xUnit for applications. How I see it, things change and sometimes its good to update the tools. This will be my way of evaluating this tool and I'll let you know how it goes.

Automate!

One my colleagues found this program called Automate! and I found it to be a nice way to do some simple application scripting.

After playing with it makes, the system seemed to break down all the functionality down into simple terms. I also like it from a perspective that it will take a non-coder less time to absorb - plain english tests. The language is so plain that you don't need to know about variables and structures to get productive. It did seem a little more than record/playback because you have to structure your script to look for object; meaning that you don't "record" in regular terms, you actually have to know how to use the application to determine locations, etc.

Why am I writing about this tool, well from my experience, I have a hard time getting people to write scripts from a point of view of automating redundant tasks, that seems to be a hard concept to get people to do absorb. Part of it is because often we don't treat our automation efforts like development projects (although that is changing).

I think context is what matters. For me, I'm relatively language agnostic meaning that I don't care what language I'm in, I'll learn it and try it out. But for some people, that's not their strength. Automate is language independent, albeit vendor lock, it can be a useful tool.

I'm only saying that this is an interesting tool for those people are out there. I wish there was an open source alternative for this type of application, however many of the developers out in the open source world can already program, so this is definitely filling a niche. Actually, if there is an open source alternative (and I'm not talking about AutoIT or autohotkey), please let me know. I know a bunch of people who would like to know this information.

It comes with 30 eval, but after that it can range from $800 to sky's the limit.

Expected quality of the experience...

On my trip out west, I got to go to a restaurant that I wanted to go to for a while. The reason I went was because I read that they had some of the best Indian food around. I bought the cookbook written by the owner of the restaurant and made some of the recipes. It all seemed very appealing and given the opportunity to experience the cooking first hand.

My expectation was to have some of the best Indian food around. When I got there, I was met by the owner at the door, explained how it works around the restaurant, so there was no guessing from my part. He told me there was a 2 hour waiting list and they had a nice lounge for drinks and they offered free h'dourves. The guy taking our names remembered my name and came and found me. He didn’t just yell out my name, but gave the personal touch. They also told us that we didn’t have to wait the whole time in the restaurant, so we went for a walk around the area. For the most part, even with the wait, it was an enjoyable experience because of the service level given and the atmosphere.

When we eventually got to our table, and no one person was responsible for order, it seemed that everyone did everything. The owner went out of his way to come talk to us too, even when it was so busy, he could easily spend his whole night at the door. We got there around 7, ate at 8, left around 9:30 and it still had a 2 hour wait at the door. Good prices and overall impression left me wanting for more... to the point where when I didn't finish my meal (it was huge), the owner came over at said "eat, you look famished" and proceeded to put more food my plate.

All of this was above and beyond the experience I expected. The principles are simple and applicable anywhere. Give you customer the easiest possible experience and give them the best service you can.

In software, we should do the same exact thing. From a testing perspective, so as above - test of the restaurant condition - will it meet your expectations of a good product? Software should be easy to use and complicated software sometimes screams to me that we put too much on the user. Sometime users are power users, but if you're planning on making something useful, do it right.

In that, simplicity is the key. Think of Google, it beats all the other search engines out there because of its simplicity. In the background, the system is very sophisticated, but the key in how it works is in the user experience and the simplicity of finding information. The results from most searches meet my expectations (which means my result is usually somewhere on the first page).

So, another lesson learned for me, and a personal test heuristic too – the restaurant condition. Or is it a test oracle? I’ll explain more about that on a later posting.

CAST 2008

The 3rd Annual Conference of the Association of Software Testing (CAST) 2008

Toronto, Ontario, Canada, July 14-16, 2008

Beyond the Boundaries: Interdisciplinary Approaches to Software Testing

Keynote Presentations by Gerald M. Weinberg,

Cem Kaner, Robert Sabourin, and Brian Fisher

Tutorials by Gerald M. Weinberg, Scott Barber, Hung Nguyen, and Julian Harty

The Association for Software Testing is pleased to announce its third annual conference (CAST 2008), to be held July 14-16. The meeting will be held in Toronto, Canada, a city which features enormous diversity in culture, businesses, educational institutions, and the arts. Toronto is the perfect location for a conference on this year’s theme: "Beyond the Boundaries: Interdisciplinary Approaches to Software Testing".

You can view the most recent brochure here, and you can see the conference program here.

(text stolen from Scott Barber's blog)

Jerry Seinfeld has a trick

I read in some magazine a number of years ago about Jerry Seinfeld and one of the reasons he was successful. When he started his career, he had a hard time finding new material. Sometimes it came to him, but often he would have a writing block. Then he tried a technique where he uses a calender to track that he writes one new joke every day.

So, here are some new things I'm practising everyday:

1. Trying to write this blog everyday. I have made sure I have enough material to cover me for a while, its just that sometimes I need to edit, which takes me the most time.
2. Going for a walk every day, even if its for 20 minutes. Sometimes I bike to work, but I try to get some blood flowing each day.
3. Not eating snacks after dinner. Basically making sure I'm hungry before I go bed. I have a huge problem with snacking, so I decided to not cut it out, but make a conscience effort to remove it from the evening.

Anyway, so far, I have been able to keep this blog floating for the time being. As for the walking, some days are better than others. Same goes for the snacking.

I guess what I'm trying to say is that practise makes perfect (or at least consistant)...Same goes with software development. Any of our efforts require you to take the time to learn and practice the art. So, resolve to do something every day to improve what you do at your job.

Zipline and single points of failure

Ever heard of a Zipline? Basically, you attach yourself via a harness to a rope and you zip along a line, except the line happens to be 500 to 1000 feet off the ground and your line can be up to 1000 feet long. So, after doing this on my recent vacation out to Western Canada, I occasionally thought to myself, what's the single points of failure on this thing? A little morbid, but I like trying to understand my surroundings, and my instincts tend to lean towards breaking things.

The rope... each "rope" made of steel was comprised of 200 threads of steel. This is the type you use when you're stringing heavy objects to buildings, etc. The harness had at least 5 different points of failure with some redundancy. The zip line harness had two points, with two carabeeners available too. The platform in the trees had redundancy too...It keeps going on and on. A seemingly unsafe ride was probably more safer than walking across a street.

A good lesson from this is that we should take our single points of failure and make sure we fortify them or make them redundant in the systems we build and support. The feeling of being secure made the overall experience quite enjoyable. The fact that it was secure was not the most exciting part, the part where I was going around 120 km/h at around 500-1000 feet off the ground was the enjoyable part. I highly suggest shelling out the money if you want a great ride.

PS: That's a picture of me on the zipline.

Scheduled posting now live in Blogger

I went on vacation for a few weeks, so you may have noticed that there were posts available before the date had arrived. Good to see that when I get back from vacation that this feature is now available on blogger!

Anyway, I'm going to try to make use of this feature since I keep about 10 posts on my computer before I post them, so this might be a good add-on for me. If anything, it will prevent me from moving to Wordpress or something like that in the future.

Books - Managing Humans

Not often do I read a book as funny about management than this one. I've read the Mythical Man-Month, PeopleWare but those books were quite insightful, but d-r-y. This book was refreshing because the point of view is from someone who's lived through the industry.

There's a lot of biting humour and the thing I got from it is that people are not broken down into simple categories. Management from a producer or consumer perspective is "an art", and this book tries to convey that message. Anyone trying to manage people or if you're being managed by someone else, try this book out before the two I mentioned above. Again, I think it has taken the best parts of the dry books and distilled it down into a semi-fictional, funny and witty book.

The chapters are broken into small manageable chunks that you could work through over a quick period of time. I read the book in a few nights. Here's a quick sampling of the chapters:

• Ch. 1 - Don't be a prick
• Ch 18 - Status reports 2.0
  
• Ch 25 - A nerd in the cave
• Ch 31 - Rules for the reorg

Let me know how it goes if you end up reading it because of this post.

The Nine Forgettings

This is a presentation I saw at the STARWest 2007 Conference last year. It was a great presentation because as software testers, I think a lot of people don't know how the industry has grown up. In fact, I am one of those people who didn't know where we came from. After this presentation, I decided to find out as much as I could about many of the people mentioned. I would suggest to you that you use your Google-fu to find more information, just so you can arm yourself against people who might steer you into 1983. :-)

Here the Google Video link. I would suggest you watch it since its a great lecture and not too long (around 35 minutes).

However, if you don't have the time, to tie you over, I attached a copy of the power point he created.

KWSQA Quality Conference 2008

I had the privilege of going last week to the KWSQA Conference

This was a great conference because it was developed by practitioners for practitioners. The speakers they had lined up were also quite interesting. I tried to pick speakers that met a few criteria:

1. They spent some time putting themselves on the Internet.
2. Entertainment value
3. I have read their work or attended a presentation before and they were worth coming back for.

Michael Bolton is one of these guys who seems to be published everywhere and has a great teaching style. I was very lucky to be able to participate in a open discussion with him and a few people. I brought my performance testing scenario and we debated and discussed in an open forum our issues. It was great to have that level of interaction with people who have far more experience in the field than I do. They gave me insight into things like "am I really done testing?". I said no because testing goes on, but in fact because of the lack of some cohesive interfaces, my answer changed to, "yes, it is done... for now". It gave me a perspective on how to deal with people. Interestingly enough, things you can do to drive performance tests are:

1. Create interfaces that are scriptable
2. logging and capturing information from that
3. training to build up tools to do performance testing
4. outside consulting
5. test labs (either external, or internal) for any large system

Also, three major types of sizing:

• customer sizing so they know what to buy/maintain, etc;
• development sizing, to see where development should spend the money or time for load, etc;
• product management sizing, they want to see the product perform at a certain level because the market drives the metric.

Given those criteria and advice, I think I have an excellent way to help manage and consult on my next task. I'm also taking a risk-based approach to this whole problem whereby I'm going to ask the question "I want to know what's important to you" and let them drive the tests. There's a lot more to this point and I'll save that for another day.

xUnit Test Patterns

I picked up a copy of a new book called xUnit Test Patterns

Some of the developers I have given it to have said it was like gold to them with the help from Design Patterns

Built with a similar structure as the Design Patterns, the reference material at the front and back covers helped to fill in the gaps. The inside front cover shows the design patterns used to create xUnit fixtures and suites. At the back is something completely different - its like a Q&A of when people talk to you about why they can't put xUnit into their applications. My favorite section is "Legacy code cannot be automatically unit tested". Page 365, BAM! Often when I use this lookup, it start a discussion with a developer to begin thinking differently.

Its a nice all encompassing view of how to unit test applications. Although its not new, its nice to have it one place. I think these are one of those books that will be good 10 years from now because of the solid use of patterns over technologies. I'm not a fan of having a huge reference guide for many languages. I use pocket references mostly when I need reference material (or just look it up online). Its a very nice indispensable tool to look up all the different types of unit test structures and patterns in one easy location.

Check it out the reviews for yourself.

For quick reference, here are comments on some of the chapters:

• Chapter 2: Test Smells - I like this chapter because it explains why code smells and how to recognize where to apply tests


• Chapter 3: Goals of Test Automation


• Chapter 7: xUnit Basics - this has saved me a few times now on having to explain to developers how to integrate unit tests into legacy applications. I'm sure I'll get some more running ground on this chapter in the next few years


• Chapter 25: Database Patterns - this is something I have not got a lot of ground on. People still tell me they cannot abstract out databases. I don't believe you need to, you just should consider the contents of this chapter in order to unit test your database.

To get a copy for yourself, you can order it online or take a look at the online version for free. Again, I like free stuff, so go and educate yourself for free!

Books - Geekonomics

Here's a book you should avoid. Geekonomics: The Real Cost of Insecure Software and how it supposedly gives you the idea that software is like any other commodity out there.

I recently obtained a free copy of this book and thought "wow, I like free stuff, I might as well give it a read". Well, after a few nights of reading it, I came to the conclusion that this book was meant for those who don't develop software and for someone who only wants to understand how most of their life is ruled by how bad software works within their own lives.

My first reaction was "apparently, we don't test 'our' software enough". This wasn't the author's take, instead he was relating it directly to negative economic and business impacts (which are still true for bad software).

The analogy's used were nicely told; the one story that sticks out in my mind was about how cement/concrete was standardized. Basically, the moral of the cement standardization was that testing was rigorous and people held to that standard for testing it. However, where the author goes awry is when he says the same style of standardization should be applied to software development. I disagree with this statement because software if infinitely more complex than concrete or automobile manufacturing.

The moral of the story is that poor quality controls leads to a economic loss, which in turn hits the bottom line.

Also, don't be fooled by the title, its not that geeky. This is geeky.

A context-driven online course...

BBST Foundations Course - For all you Black Box testers out there, there's some courses available that might of interest to you. From the website:

"The Association for Software Testing is offering a series of free online courses in software testing to its members."

I would take the course, but I'm away during that first sessions. I suggest you take the opportunity to take this free course. Considering its in the lines of what Michael Bolton says about certification (and why it is a biased system of making people pay money to organizations who actually never watch you "drive the car"), this one might actually give you some suggestions and help towards being a better tester. My guess is that it will given the people who are developing/delivering the material are the experts in this field.

Since I can't take the course, I would suggest you sign up for AST (its only $50), and get yourself educated! Best deal in town!

UPDATE: I think I may have found the course-ware online (see video's), but you'll need to view it from home. I think in order to take the course, it would be like taking a driving exam... someone will watch you drive the car.

Google Apps Engine

Google has some pretty neat things. Its more than just the products, but its the archtiecture. They often are on the forefront of putting applications out there that no one else has done. Google Apps Engine is no different.

Remember my post about SOA, well this is it in a nut shell. Other people have SOA apps, but not against the computing power that Google is unleashing.

All in all, its not much different than any web hosting service, except for the fact that you have access to the whole google computer farm! Whoa! now that's a spicy meatball.

Man-o-man! - Tried signing up for the open source, but it was gone in a few minutes. After the initial signup, I read that the load on the system caused slow downs, which is probably a product of it being in a beta.

Entrepreneurs can use this app engine to build their systems without having to buy a lot of infrastructure. This frees up money for more R in the R&D.

From a testing perspective, I would definitely need to play with it to understand what they are doing. I read some of the docs to try and figure out what it does, but playing is what will tell me the most about the application.

I like free stuff...

Information is great, but when someone takes the time to put it into nice digestible chunks, I love that even more. However, when someone gives that away for nothing, well, that puts me over the top. So, I'm sharing with you what I just found...

Five more free magazines for you today:

1. One of the better magazines, even if they don't have a lot of issues out yet, is from the Association of Software Testers called Sapient testing. I like it because the articles speak directly to the context-driven tester community.
   
2. A testing magazine out of the UK, lots of good information here and regularly updated, more Agile and certification focus
   
3. Although more infrequent, the Toronto Association of Systems and Software Quality publish a nice little newsletter that has some really good information about testing from some of the major experts.
4. Software testing in general, good ideas for new people and experts in the field. Good advice as well
5. Scott Barber and regular articles about performance

That's all.

Initialisms, Abbreviation, Acronyms, Oh MY!

k.d.s. says:
are you in the CTC
k.d.s. says:
?
Marc says:
y
k.d.s. says:
are my sunglasses are in there?
Marc says:
i'll hal, jas
k.d.s. says:
?
k.d.s. says:
oh
k.d.s. says:
haha
k.d.s. says:
I get it
Marc says:
idst
Marc says:
sylth?
k.d.s. says:
I think so
k.d.s. says:
sometime yesterday
Marc says:
atoyh?
Marc says:
dyekwtm?
k.d.s. says:
enough with the arconyms. ;-)
Marc says:
(chuckle)
k.d.s. says:
hehe
Marc says:
there's a word not many people know
Marc says:
many people don't know that an acronym must be a recognizable work or else it's just an abbreviation or an initialism
Marc says:
an fact, things like RCMP and FBI aren't even abbreviations, they're initialisms
k.d.s. says:
oh
k.d.s. says:
I didn't know that
Marc says:
things you probably forgot were acronyms: radar & laser
k.d.s. says:
laser?
Marc says:
light amplification by stimulation of electron radiation or something close to that
k.d.s. says:
neat!
Marc says:
i remember diving into all this because of all the workflow names in tm.
Marc says:
if they are abbreviated in print but you say them expanded (e.g. "etc.") it's an abbreviation.
Marc says:
if the abbreviation forms a recognizable or pronounceable word, it's an acronym (NATO, UNESCO, radar)
k.d.s. says:
ohhh
k.d.s. says:
neat
Marc says:
if you say it letter by letter, it's an initialism
Marc says:
and to think, there are people out there who spend more time on this then i just did!
Marc says:
now i don't feel like i'm so lacking for a life anymore
Marc says:
it's always so good to talk to you, Ken
Marc says:
or at you.

Smoke Jumper

A friend of mine is a smoke jumper. She spends 6 months of the year up in Northern Ontario (about 15 hours drive away from here) in order to make sure the forests don't completely burn down.

I read an article recently that compared software projects in emergency situations to firefighting. While I disagree with the analogy because I have heard some of the experiences from my friend, I do like the quick synopsis of what to do if you are dropped into a critical situation.

The things advocated were:

• Determine your role
• Build trust
• Learn the territory
• Gather information
• Do your job
• Declare victory

I won't go into the details of what the article does an adequate job conveying, I'll just ask you to go read the article and see for yourself. Do you ever find yourself in these situations? Over the years, I've had my share of firefights.

I think a lot of firefights could have been avoided by paying attention to some simple and basic testing methodologies. I'm a little bias, but I do believe that beyond deadlines and politics, what remains is how you finally test your product and how to repeat that test the rest of the product life cycle.

The one that started me off - the Pragamatic Programmer

Every software developer should read this book. The perspective is interesting because it comes from someone who decided to write down the lessons he learned after working for 10 years. Lots of simple advice that schools don't promote. In fact, most of the advice lends itself well to people who have worked for a few years and now want to learn how to get better at what they do.

In general, it gave me a language to speak to senior developers, project managers and managers. The book is broken down in to "tips" and how to handle many different situations. I have a copy at my desk so that I can refer to things (at least the table of contents). Its by no means a bible for software development, its a nice book to have when faced with problems that are common in the industry.

There are a series of books available and another group I would recommend is the Pragamatic Project Automation, version control and unit testing series.

The idea of this post wasn't to sell you on books, it was to expose you to the whole library of pragmatic books out there. Take a look at Amazon for reviews before you purchase since they tend to have more in-depth ideas behind each book.

From a testing perspective, this is where a dovetail occurs. If I understand how developers should think, then I should be able to communicate to them things they may forget to do. In turn, I can help implement and change the organization for the better based on some basic fundamental ideas. This book is a landing platform for me to start this conversation.

So, go find out more. Over the next few posts, my books will become more testing oriented, some more obvious than others (aka: I'll throw a few curve balls, just to see what you think).

SOA and testing resources

Have you heard of SOA? Service Oriented Architecture. What does that mean for you?

Well, considering you're reading something on blogger, there's an API behind the scenes doing heavy lifting. A lot of this can be considered part of an SOA world.

SOA can take familiar applications and put them easily on the web. Actually, taking existing applications and moving them to a SOA world is much harder, but after that is built, ingesting the outputs from an SOA system is quite easy. Why does that matter? Think of your cell phone, a lot of the interoperability comes from SOA, such as being able to check your Facebook information, update your location based on your GPS co-ordinates, etc. These are all applications built because Facebook built a front-end for people to write applications for.

SOA made it possible for me to post this without the use of the web client. Since I can create applications far more robust on the operating system side, then publish to a website, this is the simplest of SOA systems available.

Here's some articles to get you started - Getting familiar with SOA.

What you might not know is that much of the internet and specifically Web 2.0 is using SOA. Facebook, Amazon, Google, MySpace, etc. all have some sort of SOA

What does that mean to you? The world is becoming even more connected. With the Internet applications being available, anyone can get started and build your own web application. Its not as hard as it sounds. To see it, sometimes with the right plugins, you only need a good IDE and the ability to draw.

Draw? What's this about? I'll save that for another post. Also for another post, how to test an SOA application and how to break it. For some quick info on the testing side, here's some articles for you.

What's performance testing to a tester?

I think performance testing requires more than just seeing how many users can bang on the system. A lot of systems don't differentiate between performance testing and subsections of that type of testing. Yesterday I gave a quick intro to a tool that allows you to "performance" test a web application. In fact, that covers everything but storage testing.

When I'm thinking of heuristics and how to remember all of them, it helps me to use analogies. Here are some of the analogies I use regarding how performance testing should work. Feel free to comment if think I'm way off base.

• Volume Testing - Think of your body. Try drinking a glass of water. Now, drink a glass of water each hour for the rest of the day. Tomorrow, do the same thing execpt drink one glass every half and hour. Every day keep decrementing the time spent between each glass. This is what volume testing an application is like. Finding the breaking point, except if you combine this will all-pairs testing, you can get glimpses into cross-configuration testing as well.
• Stress Testing - Think of any stress you have had on your body; have you been in a car accident? Have you had someone shoot you out of a cannon? These are all stresses, whether plausabile or not, each still have the same type of result... or do they? Stress testing an application is the same and two ways of stress testing an application may not fully express all the stresses that a system can take.
• Load Testing - Have you ever studied for an exam? How about two at the same time? how about 10? As you go up, the harder it is to juggle all of the data. Some people can handle 10 exams while others may not be able to handle one. If that person who could handle 10 exams, the load test here would be to pump that up incementally until they cannot handle any more.
• "Performance" Testing - How fast can you run? I can't run that fast. Some people aren't built for speed or sometimes they have other factors that limit their speed. Same in software - sometimes millisecond response time might lead you in a certain direction, but more often than not it is because of something you didn't expect.
• Storage Testing - How much can you eat? How far back can you remember in your memory? This one is pretty self explanatory. One could say that I'm somewhat famine proof because of the result of my storage capacity. :-)

These are just a few heuristics I use when I do performance testing. Load generation and configuration management is a huge part of this too, but I'll get into those another time.

Performance testing of your web application – JMeter

Performance testing doesn’t require expensive tools to simulate thousands of users. Apache JMeter is a 100% pure Java desktop application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.

What can I do with it?

Apache JMeter may be used to test performance both on static and dynamic resources on:

• files
• Servlets
• Perl scripts
• Java Objects
• Databases and Queries
• FTP Servers and more…

It can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. You can use it to make a graphical analysis of performance or to test your server/script/object behavior under heavy concurrent load.

Why would I use it?

• Start doing performance testing as part of your unit testing. You can call your JMeter test from your JUnit (or NUnit, CPPUnit, or any xUnit framework) using an Ant task
• It can be used with many programming languages, not just Java. This includes Perl, C#, C++, etc.
• With the system running jmeter-server, which takes commands from the GUI (see screenshot on right) and send requests to the target system(s), you can deploy to multiple systems simultaneously.
• Its open source and freely available!

JMeter is not a browser. As far as web-services and remote services are concerned, JMeter looks like a browser (or rather, multiple browsers); however JMeter does not perform all the actions supported by browsers. In particular, JMeter does not execute the Javascript found in HTML pages. Nor does it render the HTML pages as a browser does

Fully multithreading framework allows concurrent sampling by many threads and simultaneous sampling of different functions by separate thread groups. Thousands of threads can be simulated on multiple machines. This can all be called from your xUnit framework inline with your unit tests.

VMWare and applicances

Yet another tool that I use on a daily basis is the use of virtual machines. If you don't know what this is, basically its a complete emulator for an operating system within your current operating system. Think of it this way - you can run a complete version of Windows XP within Windows XP. This is useful when you don't want to contaminate your own environment with software that might be unstable, or vise versa. Another reason is that you can create a client-server environment that is completely isolated and running on a desktop without the need for multiple physical machines.

Now, this is not an out-of-the-box tool. Companies give this technology away for free to run on your own system. Microsoft Virtual PC/Server and VMWare Server are both freely available for you to try out. (At work, I run VMWare Workstation because it has a lot of the branch/snapshot features I like).

What got me jazzed a few days ago was that MS is giving free 30 day eval of Windows 2003! Very useful given the limited amount of licenses I have access to. This is an interesting development too because MS has their own competing product and they are bowing to VMWare's dominance at the moment.

What are appliances? Basically pre-built versions of OS's that have the basic configuration completed so you can run your tests. In general, this is useful because you don't spend time having to setup a whole new environment for this effort.

I have been using appliances for quite a while. I use VMWare at home to run a Linux box (well, that VM on a Mac, so... Linux on UNIX is an interesting divide too) This would be super cool - I can run windows on my mac box for free! I run VMWare Fusion on my Mac at home.

So, why I am I talking about VMWare and how is this useful for testing? Basically if you're doing any type of testing, such as but not limited to:

• Regression tests
• Configuration Testing
• Compatibility / Conversion Testing
• Installability Testing
• Functionality Analysis Testing
• Cross-function analysis
• etc.

... this gives you the ability to do a whole lot of testing without having to have different physical environments available. All of this can be done virtually and with a lower setup time. A very effective tool for those in software development.

My bookshelf

This is my current bookshelf.



This picture is some of the books I'm reading now. Except for the Da Vinci Code which my wife sneaked on to my shelf, this is what I'm reading right now. Why do I read so many books at the same time?

Back in University, I spent a good part of my time reading. Most of the time the books were technical or contained information that was good for long term references. Occasionally I got to put some fiction in there, but mostly it was school related. There was a lot of concurrent reading going on and it kept me quite interested. One of the few books from school I still refer to is the design patterns book I used in 4th year.

I used to love going to a coffee shop and reading a novel, or a non-fiction book. My preferences were towards the non-fiction like biographies or technically oriented.

After leaving school, I had an allergic reaction to reading. I didn't enjoy it the same way I used to, for which I blame on having way too much non-useful reading to do. It took me some time to get back to a place were I'm reading stuff I enjoy.

So, in order to keep me interested in what I'm reading, I'll put one or two books out a night and read a bit from each. There are some that are always in constant rotation and I read a few pages from each night.

Over the next few posts, I'll be sharing with you my reading collection in relation to software development. I'll try not to bore you with details but give you the reason why you should be reading these books. These are books I think are a must have (unless I say otherwise) for anyone in software development. This isn't limited to testers, but anyone from project managers to developers.

Google Charts

Quick post today: Huge fan of google's little toys and API's. Everyone knows about Google maps and mashups, but they go a lot deeper than that. They occasionally release tools they use in-house for public consumption. One of them is called Google Charts.


Basically, you can provide a simple URL with enough data to form a chart, this in turn gives you an image to display. Charting from the URL! Amazing! I wish they would open source the backend code so we can all have a graphics library to do charting without needing the webservice.

I'm using this now to remove the need for an et.xls for Session Based Test Management scripts we use. I'll post the files up here once they are complete.

A replacement for Installshield?

I'm a developer/tester now with more of a slant towards test tools, technologies and technquies. However, I still care about how the tools we select for development impact our every day lives, mostly from a sustainability point of view.

Installshield is a necessary evil for getting software onto Windows machines. Over the years, I have my share of exposure to IS, including developer, professional and new versions of it. Each time, there was a new paradigm for developing in IS, from changes to how features, folders, file groups (whatever they are called now) to how things compile from a command line.

Yet another tool I came across called WIX which stands for "Windows Installers from XML". After downloading and taking a look, seems like a good alternative to IS and many of the proprietary systems for installing software. Ultimately, I want an MSI package that I can pull into a VS project and I don't want to learn a whole new language to do it. This is all in XML and the system is open source, so I can use it in every day life to make changes as I see fit.

This also means I can distribute my source for installing my software without having the burden of telling someone "Oh, you don't have that license for IS, sorry" or "I have no clue where my CD is anymore, so you'll need to hunt around for it". I'm getting frustrated with proprietary formats. It would be nice to see software developers who make software for developers create language agnostic systems for their applications. Some people already do this, like Selenium. These organizations can own the engine to compile, but don't create a new paradigm for your product.

This little utility look like it could be sustainable longterm since XML isn't going anywhere anytime soon and the source to compile the source is freely available. I'm a fan.

Perl IDE

I just found from a co-worker about this IDE for Perl called PerlIDE. It has some nice debugging features and the ability to look at scalars, hashes and watches on basic variables. Not sure how it would fair against a multi-threaded Perl app (this is a shout to Jarrett), but I have been using it for the past few days to debug some stuff I had worked on a few months ago. Its been great to use something a little more modern than Perl Debug.

That's all, quick post today. More to come.

Martin Fowler and CDT

Martin Fowler of Test-Driven Development and Agile Manifesto fame wrote on his blog this weekend about context-driven testing and how it relates to himself. He goes on to say that he doesn't know a similar thread regarding software developers in general and that there is an opportunity to help define this for the future.

I find it interesting that he sees this in himself at the same time I do. I'm of the TDD/Agile school and I'm learning how to become more context-driven. I believe there is a dovetail here and that in order to help define it, there needs to some thought into software development schools of thought.

My first thought on that school is more of an analogy: You can't take a C driver developer and move them into a BPEL Java Web Services environment overnight, you need some mentoring and retraining to make this transition effective. Same with testing, different schools need the same level of retrain/rethink in order to move in to other schools. I'm on my way now to the CDT world, but still bringing my TDD world with me.

Interesting post, and if you don't know who Martin Fowler is, you should find out! I love the tools his company makes and I'm constantly using them in my everyday testing world.

My two favorite work related PDF magazines...

My two favorite work related PDF magazines are out:

Methods & Tools - Spring 2008

• OpenUP
• How Quality is Assured by Evolutionary Method
• Creating an Agile environment
• Real Reuse for Requirements

Click here to view or download this issue PDF file, 1140 K

and

Software Test & Performance - April 2008 - Vol. 5, No. 4

• Fine-Tuning SOA in Search of Sweet Harmony
• Whip Those Test into Shape
• A Bug-List Rx Can Turn the Tables on Testing
• Trust, But Verify: Close-Up on Tester Objectivity
• Best Practices: Profiling: Leavin' on a Jet Plane
• Future Test: The Software Tester's Bill of Rights

Click here to view or download this issue PDF file, 6000 K

The best part, they are both free and the archives are fully accessible! They both have great information for project managers, developers and testers.

Conferences...

Well, its been a while since I posted. I was away at a conference earlier this week (Monday and Tuesday). The conference was IT360 and I wouldn't recommend it to any software development shop. Good place if you want to get your feet wet with speaking at conferences, but otherwise steer clear. They are not a conference "by developers, for developers", its more like it was done by professional conference people for anyone who will pay money.

Anyway, why did I go? I had won a "super pass", value >$1000 from a local IT organization. So, it was a very low cost event for me to attend and I thought I would learn a lot. When I got there, speakers (the ones I wanted to see) didn't show up and that almost killed a whole day. Anyway, this conference really was for people who don't work in software development and are looking for some information about topics they may not get exposed to (think government with an IT organization).

This isn't a complaint - I really appreciate going to the event, its just that I learned a few things:
- Developer conferences usually have enough people to fill in for other people if they are sick. There's enough people in the audience to recover from someone unable to show up.
- I'm only going to conferences by practitioners, for practitioners. Considering this is my second conference, its a good lesson to learn.
- Free is always good, but sometimes there are different levels of "free". :)

I'm going to a one day local conference at the end of the month. Should be pretty good and today was the last day for registration. This is a conferences by practitioners, for practitioners, so I'm pretty psyched about it!

“Quidquid latine dictum sit, altum sonatur”

My own skill set is changing lately. I used to call myself a periphery developer in relation to core developers. I used to do the installsheilds for complex applications, some GUI type programming, enterprise level build systems and test automation systems. Today, I mentor and consult on many aspect of these parts. One of the things I advocate is making code maintainable. That means, can the next person who needs to deal with the code use it well.

I had a discussion with a colleague recently and he recently inherited a "pristine enterprise application", >90% code coverage, thousands of JUnit tests, and properly written, from a tester point of view. In fact, when I had the chance to make it fall over, it was during a performance test and we simulated 40000 users hammering on this thing concurrently, and it held up to that torrent of traffic. Nicely done. Anyway, he was asked to make a quick fix for some new feature that may require a "hack". He told me in the parking lot that he was "violently opposed to any hacks because he came from a world of hacks". I was ecstatic to hear that, when someone care about the code so much to keep it testable. I think this an attitude that everyone should approach their code with.

Every time you make a hack, you make it difficult for the next person. On the karma scale, its pretty low for developers to do this. I also understand why developers do that since I am also guilty for making hacks to brand new code. I'm not perfect and the code I had to "hack", I made sure the next person knew that by commenting in giant letter "THIS IS A HACK, I'm really sorry". Sometime there are just time pressures and limitation that require it and sometimes its for the greater good.

Testers can just hack up tests too, this karma scale applies (even if the political situation negates the time for testing).

So, I know why things get hacked, but if you make it maintainable from the start, its best to defend that type of code base at all costs.

Anyway, the reason I thought about this was because of this website on how to create unmaintainable code. Basically, do the exact opposite of everything he says. :-)

“Quidquid latine dictum sit, altum sonatur” which translates to “Whatever is said in Latin sounds profound.”

Testable Passwords

Have you ever noticed when your password was rejected because it wasn't strong enough? I have found over the years that many password schemes (including most online banking) doesn't require strong passwords. Is your password strong enough?

I found a great online utility for such a check called Password Meter that verifies the strength of any given password you provide.

I tried looking under the hood to see what this client side application was doing. Basically its a quick AJAX application that does all the math on fly. Pretty neat and if you're looking for some really obfuscated code, take a look at this one (although it was easy to push it through a pretty-print/beautifier utility)

For your information, @Pa55w0Rd# is a 100%.

This is just another little tool that if you're testing your systems out, good to have a quick peak to see if your application can handle the password scrutiny.

You lost my luggage?

Over the years I have had my share of running through Heathrow Airport in London. If you have ever had to get a connecting flight through Heathrow from Canada on British Airways, you usually have to take a train between Terminal 1 and Terminal 4. In fact, I had to take the train from T1 to T4 to get on my plane through Europe and I only had 45 minutes. When I saw they were opening Terminal 5, I was somewhat enthusiastic because I thought it would help eliminate the delays and all the running around. (Personal aside: I was less enthusiastic about it because I read that it would handle 20 million more passengers per year, which from an environmental perspective is appalling).

Well, it opened up a few days ago and its reported that over 20,000 baggages were lost on the way between planes and destinations. It appears the luggage system, touted as one of the most advanced in the world, was unable to handle the load because "it could not be used to process delayed bags".

In simple terms, this means flights were canceled or delayed, and people who used to fly BA are now flying on other carriers who could accommodate their needs.

So, what does this have to do with testing? Everything. A couple of things about testing here:

• The political situation to get this terminal up and running must have been so intense that they thought this would be a temporary setback. In fact, it is but it will mean lost profits, and political attention that BA would not seek. Even the British Prime Minister was asked to comment on the situation. If I was a tester on this, I would feel it my duty to tell people the story of why their baggage is late.
  
• To the software developers - complex systems require attention to testing too. I realize that creating a system as complex as this one requires a lot of work, but testing should have been the most important factor when making design decisions. From the Times:

Sir, The teething troubles at Teminal 5 are a startling indication of the costly problems that can arise when launching complex systems that involve computers, software, machinery and people. In terms of complex software and systems it is well known that the testing phase is the most expensive part of the development of successful systems — consuming well in excess of 50 per cent of the total development costs. Part of the problem is that testing as a subject is rarely studied in universities — it is just not very fashionable — and so many graduates have no understanding of the difficulties and intellectual challenges that the testing of complex systems presents.
Professor Mike Holcombe Department of Computer Science,
University of Sheffield

• Use-case models and heuristics could have been used to anticipate and test for these conditions. The problem now exists that you need to test your software on a production system, so a word of advice, always use a scratch monkey.
  
• I am sure that the people testing this system told people and warned them of this problem. Again, a lessons learned activity from other deployments would have been useful here.
  

I realize that I'm over simplifying some of the aspects of this problem, however I'd like to point out that none of my testing errors made the six o'clock news. :-)

So, I'll take this as a lesson learned: if you plan to create a monster system, make sure that you can run regression tests extremely efficiently and abstract out the hardware where ever you can. Also, think of it from the users perspective because that will give you the best idea as to where problems will occur.

The shoe and tea test...

Really short post today -- Ever heard of the shoe test? Its a very applicable and reliable means of testing your software. Basically, you take your left shoe off (make sure its your left one) and place it on the keyboard. This test is often accompanied by the tea test, where you get up from your workstation and make a cup of tea. If you come back to your desk, take a look at the screen. Has anything changed? Did you find a problem that requires you to think about how you're replicate the problem again? Anyway, just some food for thought.

I also don't think we need to test our software anymore. Most computer system are intelligent enough to know how to fix and repair themselves. Most machines should be capable of thinking for themselves too now given the speed of the processors and how many processes each one can run. With all our automation systems we build to get the most out of exploratory testing, we should spend less hours testing and more hours just coding.

Have a great day. :-)

Which language for me?

I had a discussion at lunch with a colleague of mine who was saying Erlang was the new language to learn. Strange how the conversation turned into research for me. After lunch I had a quick peruse of digg.com and came across this article on the front page, "Learn as Many Languages as You Can (or just learn Scala)".

Interesting to find out about what languages you should learn; corollary to that, what languages testers should learn. We're in an effort to do more Java development, on a scale that we've never seen before. From a testers perspective, what language would you need to learn? Ruby with Watir? Python? I think its actually a lot more than that. I think its about making sure you testing between the cracks. Think of the tests first before you start learning to code in a new language. If you can think of the tests, your language of choice because a natural result of the test.

Its back to the hammer and what's it for... its like you learned how to use a hammer and really needed a saw. Same thing with testing and languages. Again, critical thinking is required to do a lot of things. Being multidisciplinary too is an asset since testing involves so much more than coding. I'll write about that later.

Keep your ear to the ground

I've been asked over the years, how do I know stuff about things (could I be any more vague). My usual answer is "I dunno". With regard to development and testing, I was asked this question last week, so I pondered it and thought to myself "maybe because I keep my ear to the ground". If you're wondering how to keep your ear to the ground, I can give some general ideas.

General:
Read, read, read. But what to read, that depends. I'm not saying for you to start surfing around, but to be more discriminating on how to do it. For example, I keep my ear to the ground by getting my daily dose of what's going on via RSS feeds. If you don't know what RSS is, I highly suggest finding out; its a quick way to aggregate your news/blogs/sites into one neat package. How do you read RSS feeds, well with Google reader of course! This is my RSS list for testing & development related things. Its huge source of inspiration and education and on a daily basis I have about 12 articles to skim through. Since I'm not surfing and only getting the latest materials related to those sites.

Talk to people:
I recently needed some advice that I couldn't get from any internal sources, so I went to a group discussion and met a few people there. Its easy - I went up to the person giving the presentation after the talk and said "Great job, I especially liked....". Being shy is not an excuse for me and what's the worse that can happen? They say "thanks" and the conversation ends. Or, you might spark an interesting discussion. Anyway, meet people who do things like you do.

Blogs:
Blogs are a great source of information because they tend to be from practitioners that do the work and share the experiences. I have found the more I give, the more I get, so my blog here is meant to give as much information as I can. Who knows where that might lead. For me, I have a source set of blogs I started with, specifically from Kent Beck (Agile), Sticky Minds (Testing) and TestingReflections.com. In fact, today's Testing Reflections comments on how people can get information. I'm in the same boat with getting daily information. Blogs in general give you information about the latest on what's going on in the field.

Create your own list. I wanted to do a blogroll here, but I think everyone needs to figure out their own niche. Again, its all about context. I'm trying to learn more about context-driven testing, so my list has a heavy helping of that along with all the TDD stuff I am used to doing. I don't have a lot of "automation" blogs because I found most are woefully unaware of the huge spectrum of testing there is out there. Maybe that's my new niche. :-)

Books:
There are lots of resources out there... but its hard to figure out what ones work. So much out there, but which one do you choose. I'm leaving this for future posts since I have a few books I have read and done some reviews for. Books are always a great investment, I have PDF books, but having something that you can flip to, have notes attached, and bent pages can't be emulated with a PDF.

How do you test a potato...

This is one of the questions I ask when I interview anyone/everyone. Seriously. Project managers, managers, senior developers, testers, co-ops... everyone.

I know its a total rip off of from the book from Microsoft on How do you move Mount Fuji which is filled with riddles that anyone interviewing for a technical job should study just for the sake of people like me.

I don't use this book as an interview guide though; don't fool yourself either. Its not a good guide, its just a fun read that contains a few lessons. To that, I have a real reason why I ask it. My interview tends to be very technical with the subjects ranging along the software engineering continuum and are not just about testing. When I ask this question, I'm forcing someone to look beyond an answer and to talk to me as a person. This question requires you to have a discussion on a topic that is totally out-to-lunch. Well, a lot of the subject matter made by software companies are totally out to lunch... how do you test an enterprise level web application that requires 100,000 hits per minute to be sustained? If you can answer me that question, give me a call. :)

This question requires someone to step out of the interview and evaluate the question for its merits and ask the right questions. Its not about how you answer, but how you question the question. This creates a fun little discussion that gets you talking to me.
If anyone is looking for the areas I usually interview, here's a general list of the things I cover:

• People
• Tools and Techniques
• Estimation
• Reuse
• Requirements
• Design
• Coding
• Error removal
• Testing
• Reviews and Inspections
• Maintenance
• Quality
• Reliability
• Efficiency
• Research

I also make it a standard part of the interview to solve simple math problems or have people code something for me (e.g.:how do you multiply two numbers together without using the multiply operator).

Anyway, the point here is that everyone should strive hire the best people around the world and don't settle. If you're not evaluating people against a similar set of criteria around the world, we get an organization that is disjointed from many angles or you get people who slipped through the cracks. I look to the engineering community to help drive this effort along.

So many ideas... so little time.

I'm up to 35 ideas for blog posts. I'm really thankful that I have as many ideas to talk about, but because of this writing torrent, I'm having writing paralysis because it. I need to finish one post before starting the next one. The pattern so far as been that I end up getting 2 or 3 more ideas for one post, so I move on before I finish.

So, this is a short little post for today. I'll have more stuff up later in the week.

Selenium and the Wii hunt

Selenium is a great tool for automating mouse clicks on web applications. It does AJAX and other nice things that some of the professional tools don't do. (aside: bye-bye Winrunner/SilkTest/QuickTest for your pricy way of testing web applications.)

Anyway, here's the story. About 2 years ago I was looking for a Wii (around November 2006). As like everyone else at the time, I wanted a Wii, but couldn't find one. I remember checking about 5 different sites that had online inventory, so I looked for a tool to do grep (search) of those shopping sites around Ontario. Futureshop, Best Buy, etc...

Selenium fit the bill. So I did a script to check all the sites every 20 seconds to check for stock. This was running on an old Windows XP box in my basement (I have gone to mac, so my old desktop is now my play-time server). I setup the script up to send me a text message to my cell when one was available. I was quite proud of the fact that my script was constantly checking the sites, but unfortunately weeks went by without a hit.

Then, while driving on the highway from Waterloo to Toronto I get a buzz on my cell phone. 100 Wii's available at Futureshop. Since I was in my car and my phone is a cheapy nokia flip phone, I couldn't exactly buy it.

Remember how I said the script would send me a text message every minute; I only got ONE text message. ONE? ONE! When I got home, I looked at the logs. Since the script was checking every 20 seconds, but was setup to send a text every minute while the number was not zero (for the geeks out there -- while wii_qty != 0), the logs showed a fast and steady decrease in the number of wii's during that minute. 100, 81, 65, 43, 27, 8, 0.

So, what did I learn from this?
1) I should have tested my logic first. The reality was I wasn't looking for a Wii. I was looking to purchase a Wii. If I wrote a charter to my activity, I would have recognized this flaw. So, I should have setup the script to automatically PURCHASE the Wii once one was available.
2) Notify me when I owned one.
3) I wasn't the only one with a script looking for a Wii. :-)

I had to wait until March 2007 until I got a Wii... and I got it the old fashion way; by driving clear across town after a phone call blitz to about 20 places that sold Wii's.

Are you an Agile shop when it comes to testing?

Are you an Agile shop when it comes to testing?

What do organizations do when they are trying to create an agile shop? I have been making this one of my main goals for the past few years, helping to transform a hardware focused, waterfall driven, stacks & queues organization into an effective agile software organization. We've had some success and challenges along the way. I'm a firm believer in the agile way, but what does that mean for the group that came from the waterfall way? People assume they are being led to the water, but in fact the leader may or may not know what they are doing. Over the last few years I have had opporunities to work with people that thought tools were the way to salvation. The reality (as I see it) goes a lot deeper than that.

Education is one of the keys.

If you give people tools, they may or may not know how to use them. If you give them a tool in context of what they are doing, I think then you can bridge the gap between how tools are useful for evolving the job. For a software company, I think we are interested in reducing long term costs, maturing processes and intellectual property value of regression testing. Some of this is related to automation, and some of it is not.

Think of a hammer - at some point in history, people used rocks to bang things (I assume they were banging open coconuts... I don't know why, that's just the image I have), then the introduction of the fulcrum allowed for more energy transfer to the hammer. Context: it made bashing things easier.

Same is true in software, but not always so obvious. Automation is a bad term because it takes the critical thinking out of process. A recent post talked about test automation and the snake oil (I'm hugely paraphrasing). To summarize, test automation as a concept was wildly adopted but often not understood. It can't be treated in the same way as other project because of its nature, to be curious and to find things outside the norms. You can't take the person out of the equation and I think that's where the term makes it sounds like people aren't important.

In a related post, Corey Goldberg had a different slant on the idea (and he says it better than I do) -

"When I say "this test can be automated", I don't mean a computer can magically replace a sapient process to create a useful test. All test design and methodology is devised using a sapient processes (obviously). However, testcase generation, data generation, test execution, results analysis, etc, can often be done without human interaction. This has been described using terms like: "Automated Testing, "Test Automation", etc. Among some of the more vocal people in the testing community, these terms are considered confusing, inaccurate, harmful, or just a cover up for what is actually a non-scripted test"

Goldberg coins the term called "programmatic tested" as opposed to "computer assisted testing"; I'm a fan of using this term instead and have been trying to get people to use it at work.
So, are you an Agile shop... I think the real question is are you an Agile in your job? Do you know what it means to be Agile? I'm still figuring it out. I think its about focusing on people and how they use the software as opposed to the whether or not development is complete. Automation needs to be a part of this vision (this is my stake in the ground), but remember that it is just a tool, not the end-game.

I have been playing with some blogger clients and I found one that will work with both roller and blogger. I'm using w.bloggar and this will be my first post using that. Before I was using bleezer, but it didn't quite meet my needs. Also, it seemed to be a little buggy... I'll make it a point to create a bug for them.

Perspective

Over the past few months I have been searching in myself about where I belong in software development. I felt as if I was in limbo about the work I was doing. I'm a developer who ended up doing a lot of test automation because I saw that we weren't doing that in many of our
projects. I excelled at it and then I had an great opportunity to go deeper into a testing role; in fact last year I had an amazing career year, but I still found I was searching for more. Recently, in a direction setting moment, I was able to get some clarity, direction and sense of purpose. Basically I realized that I have been making a career of being a tester but always from a developer perspective.

One of the things that set me on my direction was a presentation by Brett Pettichord on the "schools of testing". I found that I was clearly in the Agile school, with a bit of quality mixed in. The agile school was about making development complete, so I'd make sure that my unit tests were complete or the code coverage was hitting 90%+. The quality school was because I needed to get more processes in place to make life easier for me and the people around me. I was doing "agile" style development for years, but didn't know it.

From 20000 feet - quick reference on what "schools of testing" are:

• Analytic School - sees testing as rigorous and technical with many proponents in academia
  


• Standard School - sees testing as a way to measure progress with emphasis oncost and repeatable standards
  


• Quality School - emphasizes process, policing developers and acting as the gatekeeper
  


• Context-Driven School - emphasizes people, seeking bugs that stakeholders care about
  


• Agile School - uses testing to prove that development is complete; emphasizes automated testing
  

A couple of years ago we had a leader that came in and showed us that waterfall model for software development was passe; it had its time and is now only good for support systems. Agile was the way we were moving forward. Without a lot of training, we were told to make it happen. Like I said, that I idea only solidified in me recently.

What does this all mean? I'm looking to widen my scope of what I know. I think there are lots of tools and techniques that can be applied to help give a sense of direction and purpose. I want to remain a good generalist and a critical thinker. So, I decided to start looking more into context-driven testing. I'm an agile tester and I make it a responsibility to mentor my peers. From my perspective, I'm trying my best to understand them, but I think I need to understand myself too.So, hopefully my followup posts will give you the context and perspective of where I'm coming from. Anyway, enough of the heavy stuff; on to more fun posts! :-)

No time like the present ...

No time like the present to put yourself out there. I figured doing it internally would help to put some thoughts out there about things I learn.

I have been keeping a journal for years at work. Sometime in 2006 I tossed my written notebook and put stuff into a wiki style. But now I'd like to open up my journal and let everyone in. Over the past year I have had little awakenings about what I do or how I do it. Its all about context and synthesis of the information. I guess that's what blogs are for!