Thursday, January 11, 2018

The bare minimum you need to know about web application security testing in 2018

I recently had the opportunity to present my security talk via STP's Community Webinar. I've included some the links to resources that you might find useful:

Stories

Tools

Reading

Practice

Here's a link to the slides too (Slideshare)

Here's the presentation, in all its Youtube glory!

Friday, October 25, 2013

A couple of projects

At the beginning of the year, I wanted to get more involved with technology I really care about.
  • It started with me gaining some confidence by taking a few courses, then contributing back to StackOverflow.  
  • I wanted to learn more about Javascript and understand JSON, so I built a map using Open Data from Waterloo Region. 
  • I learned so much using TileMill that I created a training video since there was not a lot out there on this.
  • I maintain my VPS too, that houses my website; but it does so much more. 
  • Even with all of this, I took a few course to learn more about mobile development (I love my iPhone) and I also decided to put some of these accomplishments up on LinkedIn.

Wednesday, October 16, 2013

1100 stage alerts (one of my favorite bugs)

This is a post I have had ready for about 5 years now. A long time ago, I caused a partial meltdown of our software. This was the email thread that went out by QA, and the reply from Dev. FYI: Stage Alerts are just emails sent to everyone saying something bad has just happened.

Note: this is the first time I have looked at it in about 3 years and its still pretty funny... well to me. Maybe you had to be there.

------------------------
From: QA Group
To: Dev
Sent: 1:40pm
Subject: 1100+ Stage Alerts in 3 hours

I've given up trying to keep track of the Stage Alerts. Since noon today we've had over 1,100.

I haven't checked, but I'm pretty sure we (in QA) haven't reported bugs for every alert that has come through.

Just thought I'd send out a quick note to check and see if anyone in Dev has an eye on these alerts too.
------------------------
From: Developer
To: QA Group
Subject: RE: 1100+ Stage Alerts in 3 hours?

There alerts are caused by problems try to persist audit events to the database. I've discovered a couple bugs in our logging implementation, that were exposed by some unique data.

Specifically, there was a borrower user in the database with the following:

First Name: Jebediah Obadiah Zachariah Jedediah Springfield-KD
Last Name: Squeaky Voiced Teen né Jeremy Peterson/Freedman-KD

This extra long name ends up in an audit message, and eventually causes problems because it exceeds a length limit. I've updated the user to shorten the name so that no more alerts will be generated.

I'll attempt to cleanup any existing audits that are failing/retrying in the bowels of BizTalk.

An experiment with Open Data


TileMill - A quick tutorial on how to use TileMill with Open Data

I decided a few months ago to play with more open data sets. I went to the GO Open Data conference in May and that really got me excited about how we interact with data and use it. I thought "why not play with some new GIS software and see how it goes". With that in mind, I created a few maps and a video to help me remember how to use TileMill.

BTW: Yesterday, Github announced government.github.com which is a huge leap forward with regard to indexing and organizing open data sets. There is so much out there, now its a matter of trying to find good ways to use all of this information.

Wednesday, May 08, 2013

Career advice for testers

Here's a blog post that has taken me a few years to create.

There was a site called prizes.org. The idea was that people can post questions and responses were graded for their merits by the people posing the question. In exchange for a good response, you could earn money. In this case, someone asked the following question:
Searching for projects in test engineering to improve my technical skills
Test engineering has long been a passion of mine. I dabbled in it in school and has worked as a full time software test engineer since. I have no interest in management nor do I want to become a developer, I'd like to focus purely in the testing field. 
In the last 4 years, I've been so busy with my day job that I haven't had time to improve on my technical abilities outside of work. I want to take this opportunity to offer a prize for the best recommendation on part time gigs or projects that will challenge me mentally, sharpen my skills and push me to learn new things in the field. I don't mind if there's no compensation involved, I'd really like this to be a learning experience.
Google owned prizes.org. Like many of their other properties, it was taken down and its content removed. Rather than it being lost forever, I thought I should post it here.

The following answer was created by me and it also 'won' the competition; a whole $20. Apparently money does occasionally motivate me. :-) Anyway, it was worth something to someone once; I hope its worth something to you too.

Below is my response (circa 2011, I think):

Personally, I think your best options are to do the following:
  • Meet people: if you want to talk to other testers, http://weekendtesting.com/ is a great place to challenge your skills and meet people
  • Take a course: join AST (http://www.associationforsoftwaretesting.org/training/courses/foundations/) and take the foundations course. Great way to meet people and learn/add to your skillset. Its free once you join the AST and membership is only $80 US
  • Read some books: Depending on your perspective, find a book and read it. Read anything, fiction, blogs, etc. Anything from Gerald Weinberg or Andy Hunt would be excellent. Find your own context. My recommendation, stay away from purely technical books like "learn C++". I'm currently reading "How to Break a Terrorist" (http://www.amazon.com/How-Break-Terrorist-Interrogators-Brutality/dp/1416573151); amazingly useful when talking to developers!
  • Practice! : http://testing-challenges.org is a great place to actually try out some of the skills you have learned. Alternately, join uTest - from my experience, you get to use all your new found skills in a safe environment; and you can make money to boot.
  • If test automation is your thing: Join either an open source group and do some of their testing (see Github, bitbucket.org, SourceForge). If you don't know how to program, try Ruby as your first language. There's lots of web testing frameworks (Selenium and Watir). Remember, test design before coding... ask yourself what you are trying to test, then design the test. Maybe it can be automated, maybe it can't. Good test design is the key. See "A Practitioner's Guide to Software Test Design" (http://www.amazon.com/Practitioners-Guide-Software-Test-Design/dp/158053791X). Older book, but still really good.
  • Context Driven Testing Y! list. Lurking is good. To that end, if you are a AST member, try the new forums. Maybe there's something there.
That is easily enough to keep you busy for a year. For the last few years, I have been trying all of these and found them immensely useful. In fact, there are two more things that could be added to the list. Going to conferences of your peers and writing a blog. Both of these take a lot of money or a lot more time. Both require you to be self directed. My suggestion, follow through with one of the the points I mentioned and you'll be better suited to do the conference/blog thing later.

Frankly, don't do the "get a certification" route. there's lots of "experts" that suggest it is a title without meaning. See:
I would agree with them. Certifications are mostly memorize and regurgitate. Not terribily useful if you actually want to improve your technical skills.

JSON parsing in iOS, using the Tiny Tiny RSS API

I saw a recent question posted to reddit a few days ago. It was around the idea that there aren't any good tutorials on how to take JSON data and parse it in iOS. Since I'm a newbie to iOS, I decided that it might be a good problem to solve. Before we start, take a look at https://gist.github.com/kgdesouz/5542543/.

How the heck do you connect to a JSON API that is remote (in iOS)?

Using curl, its easy. Using iOS, there's a few more steps. First we need to put our data into some sort of dictionary (well, at least that's what I wanted to do). Inside the viewDidLoad method, you can just use NSMutableDictionary. This data will need to be serialized afterwards in order to prep it to be sent to the API.

    NSMutableDictionary *dictionary = [NSMutableDictionary dictionaryWithDictionary:@{
                                       @"op" : @"login", @"user" : login, @"password" : password
                                       }];
    [self fetchData:dictionary];
    [self getCategories:_sid];
Then we need to serialize the data. The magic is done in the fetchData method. Actually, all the heavy lifting is done the the fetchData method, let's break it down a little.
- (void)fetchData:(NSMutableDictionary *)dictionary
{
    NSError *error = nil;
    NSData *jsonData = [NSJSONSerialization
                        dataWithJSONObject:dictionary options:NSJSONWritingPrettyPrinted error:&error];
    
    
    if ([jsonData length] > 0 && error == nil){
        NSLog(@"Successfully serialized the dictionary into data."); NSString *jsonString =
        [[NSString alloc] initWithData:jsonData
                              encoding:NSUTF8StringEncoding];
        NSLog(@"JSON String = %@", jsonString);
Now that we have a serialized object, we can now get it ready to be sent.
        // create the connection and send it to the server
        NSString *requestString = kTinyTinyRSSURL;
        
        NSURL *url = [NSURL URLWithString:requestString];
        
        NSMutableURLRequest *req = [NSMutableURLRequest requestWithURL:url];
        [req setHTTPMethod:@"POST"];
        
        NSString *dataString = jsonString;
        
        NSData *requestData = [NSData dataWithBytes:[dataString UTF8String] length:[dataString length]];
        
        [req setHTTPBody:requestData];
        
        NSURLResponse *theResponse = NULL;
        NSError *theError = NULL;
        NSData *theResponseData = [NSURLConnection sendSynchronousRequest:req returningResponse:&theResponse error:&theError];

The money here is theResponseData, it allows you to POST the data to the server. Once you have sent the data, you will get back a response, so you need to deserialize that back into something useful. I'm putting it into a NSArray.
 
        //deserialize the response into something usable
        NSDictionary* json = [NSJSONSerialization
                              JSONObjectWithData:theResponseData //1
                              options:kNilOptions
                              error:&error];
        
        NSArray* validResponse = [json objectForKey:@"content"]; //2
        NSLog(@"content: %@", validResponse); //3

That's basically it. fetchData is reusable, so you can now use the same method to make other JSON queries. As per the gist, anyone should be able to use the code to connect to a JSON API.

Note: I mashed together a lot of this code from:
A few more thoughts about this little piece of code:
  • My full code snip can be found here: https://gist.github.com/kgdesouz/5542543/
  • The code was meant to be quick and dirty. I didn't really follow the MVC, since really I didn't need the view or the controller, just the model. I wanted everything to go to stdout (to keep things simple). I basically just looked at some tutorials and tried to integrate it. 
  • There's no GUI component required. Without too much effort, anyone should be able to use the code for any JSON calls, with a little tweaking.
  • There is a lot of clean up required. I'm still playing with unit testing in this language. Specifically, JSON unit testing. Its a little different since I need to figure out how to reset the JSON. I will probably create a JSON mock object to do this. 
  • I totally didn't use getter and setter methods here. I know I should, but read the first bullet point; quick and dirty.

Thursday, June 05, 2008

Automation people should be a lot more talented that your developers

Typical test automation people should be a lot more talented that your developers. That being said, all your activities related to test automation should be done in context. What do I mean? Well, Jonathan Kohl has a great article on when to do effective automation. Really, it comes down to having tools in your tool box and knowing when to use a hammer vs. a saw. You also get more out of having the machine run the work while a person evaluates it.

Given that, I do see a huge role in automation, but I'll preface it by saying it requires a critical thinking cap to be running to evaluate each test that is being automated, and in turn being evaluated. I have seen recently where manual testers handed their tests to automators who automated the test case. Sounds great, right? It took over 6 days to run ONE regression.

A critical thinker would have taken a group of tests and figured out the best path to automate the manual case. Also, they would have to adjust the data to get this completed. The manual tester also would need to log the nuisances of test they were doing, as if they were meticulous to detail.

So, developers, take note. If you are planning a code activity, talk to a critical thinking tester first. Get their input BEFORE you start coding. In fact, you could even get them to build (or better yet, review) your test fixtures and xUnit test suites so that you can have an idea of what they are trying to test. This is just a starting point, there are still many other ways this interaction can occur, but always within context!

Wednesday, May 28, 2008

Tony Stark, Engineer Extraordinaire ...

For many years, I recall watching movies and seeing how things didn't meet reality. There are a few things that bug me about movies, especially when computers are involved. For instance, the incessant "beep" noises computers make when the monitor is updating. Also, that computers seem to have a screen draw rate that of a typewriter. I know, leave your brain at the door, but its hard sometimes. Movies like Hackers drive me up a wall and try as I might to erase it from my memory, it appears that its still there. ... oh, I'll get back to the point.

I watched Iron Man this weekend, and it had its fair share of these annoyances, but the thing that surprised me the most is that it really stuck to some simple engineering principles to help create the story line.

I won't give away the plot or ending, but I'll elude to some of the details of him and his suit. He ends up building his suit, needless to say because they show part of the fabrication process in all the trailers. During that time he employs engineering principles quite well, started with a design phase, followed by a fabrication phase (but he built the first one by hand, after a few iterations, he automated the fabrication process). After that, he went into a recorded/session captured test cycle with a digital camera. During his flight tests, he records the session and debriefs after the session is complete. The whole test cycle was exploratory in nature, the only way to try it is by doing... in his words "you sometimes have to run before you can walk"

After he create the initial prototypes, he automated the fabrication processes after he put the suit through the paces. Five hours to build a new suit, so he can always have the latest build, usually with improvements after the test

Remind you of anything? Good lesson learned about engineering and how often we can easily take our processes and forget the basics. Great job by the movie makers too on keeping this philosophy alive and keeping true to the basics. Made it seem realistic by employing realistic techniques.

Monday, May 26, 2008

Presentation Zen and Garr Reynolds

I like non-fiction and documentaries. So, I also like learning stuff, kind of like TVO or PBS styles of shows. Have you heard of Google Talks? If you don't know about these Google Talks on YouTube, highly suggest going there and finding out. Very handy resource and a good way to feed your brain with some knowledge. I have basic cable (around 30 channels) at home and often there's nothing on. So, I sometimes feed on this stuff.

So, I was at home watching this presentation by Garr Reynolds about "Presentation Zen". Garr's whole thing is about presentations and how they can just be better. His style has a Steve Job's vibe, but with more pictures. His ideas are distilled down to simplicity for your presentations. I have often been caught in the trap of putting too much information on the power point.

Do you do presentations? I'm sure you do. I do. Either phone calls, webinars (I hate that name, but it makes sense), real presentations in front of people, etc. I tend do at least one of these types of presentations a week. Over the years, I try to do the Toast Master's stuff (speak slow, don't use "umm's", etc), but there's so much more to presenting than just the speaking style.

So, he's basically saying that there are many ways to present data. Lots of ways to represent data and power point is not the answer, its just a tool. Some of the thoughts are from other people and he distills down to a few points. Here are some of the basic tips I got from the presentation and they might help you out (I'm trying them out now):
  • go to a white board or get a pen an paper and write out/draw out your presentations. Getting away from your computer will help you clear your mind and get away from the bullet point mentality. It also will get your creativity working because you're not locked down by the computer.
  • don't use bullet points, but simple one line ideas. Feel free to use props or other visuals. If you need bullets to give people information, make a handout or give them a PDF so they can have it available. They are there to see the content, not read it.
  • don't forget to play and have fun. That includes putting humor or your personality into the presentation.
Great video, got me started on other books to read too. Now it's time for me to go pickup the book! If you just want to see the video, the YouTube, link is above. If anything, take a look at his blog since there's a lot of information up there and its free.

Friday, May 23, 2008

Design for testability, TestNG

Another friend from work sent me an MSN link last Friday about an interview with the guy who created TestNG (a JUnit replacement) that allows for a few things beyond what JUnit can provide. The thing that piqued my interest was that it allows distribution of tests on slave machines, beanshell integration, and dependent methods for application server testing. Also, that TestNG is designed to cover all categories of tests: unit, functional, end-to-end, integration, etc. So I can get someone to create data driven or keyword driven tests and I can remotely execute them. I may not need FiT to do it now!

Anyway, I need to try this out. It would be nice to get this to be used for the simple remote execution aspect. I can then have unit tests tested across multiple environments and centrally controlled, rather than the other way around.

Oh! The creator of TestNG guy also has a great post about how TDD is not the end-all for software development; the anti-TDD movement. Interesting to see that they consider TDD just another tool in the toolbox.

More contemplation is required here for sure. But all really good information about how software development does coincide with testing on many levels.

Oh, here's an aside - I used to help teach people how to do TDD. Mostly as a 2 hour seminar. I'm going to redo the seminar to use testNG now so I can show off the evolution of test tools. I think there's a thought-lock on using xUnit for applications. How I see it, things change and sometimes its good to update the tools. This will be my way of evaluating this tool and I'll let you know how it goes.

Thursday, May 22, 2008

Automate!

One my colleagues found this program called Automate! and I found it to be a nice way to do some simple application scripting.

After playing with it makes, the system seemed to break down all the functionality down into simple terms. I also like it from a perspective that it will take a non-coder less time to absorb - plain english tests. The language is so plain that you don't need to know about variables and structures to get productive. It did seem a little more than record/playback because you have to structure your script to look for object; meaning that you don't "record" in regular terms, you actually have to know how to use the application to determine locations, etc.

Why am I writing about this tool, well from my experience, I have a hard time getting people to write scripts from a point of view of automating redundant tasks, that seems to be a hard concept to get people to do absorb. Part of it is because often we don't treat our automation efforts like development projects (although that is changing).

I think context is what matters. For me, I'm relatively language agnostic meaning that I don't care what language I'm in, I'll learn it and try it out. But for some people, that's not their strength. Automate is language independent, albeit vendor lock, it can be a useful tool.

I'm only saying that this is an interesting tool for those people are out there. I wish there was an open source alternative for this type of application, however many of the developers out in the open source world can already program, so this is definitely filling a niche. Actually, if there is an open source alternative (and I'm not talking about AutoIT or autohotkey), please let me know. I know a bunch of people who would like to know this information.

It comes with 30 eval, but after that it can range from $800 to sky's the limit.

Tuesday, May 20, 2008

Expected quality of the experience...

On my trip out west, I got to go to a restaurant that I wanted to go to for a while. The reason I went was because I read that they had some of the best Indian food around. I bought the cookbook written by the owner of the restaurant and made some of the recipes. It all seemed very appealing and given the opportunity to experience the cooking first hand.

My expectation was to have some of the best Indian food around. When I got there, I was met by the owner at the door, explained how it works around the restaurant, so there was no guessing from my part. He told me there was a 2 hour waiting list and they had a nice lounge for drinks and they offered free h'dourves. The guy taking our names remembered my name and came and found me. He didn’t just yell out my name, but gave the personal touch. They also told us that we didn’t have to wait the whole time in the restaurant, so we went for a walk around the area. For the most part, even with the wait, it was an enjoyable experience because of the service level given and the atmosphere.

When we eventually got to our table, and no one person was responsible for order, it seemed that everyone did everything. The owner went out of his way to come talk to us too, even when it was so busy, he could easily spend his whole night at the door. We got there around 7, ate at 8, left around 9:30 and it still had a 2 hour wait at the door. Good prices and overall impression left me wanting for more... to the point where when I didn't finish my meal (it was huge), the owner came over at said "eat, you look famished" and proceeded to put more food my plate.

All of this was above and beyond the experience I expected. The principles are simple and applicable anywhere. Give you customer the easiest possible experience and give them the best service you can.

In software, we should do the same exact thing. From a testing perspective, so as above - test of the restaurant condition - will it meet your expectations of a good product? Software should be easy to use and complicated software sometimes screams to me that we put too much on the user. Sometime users are power users, but if you're planning on making something useful, do it right.

In that, simplicity is the key. Think of Google, it beats all the other search engines out there because of its simplicity. In the background, the system is very sophisticated, but the key in how it works is in the user experience and the simplicity of finding information. The results from most searches meet my expectations (which means my result is usually somewhere on the first page).

So, another lesson learned for me, and a personal test heuristic too – the restaurant condition. Or is it a test oracle? I’ll explain more about that on a later posting.

Friday, May 16, 2008

CAST 2008

Association for Software Testing

The 3rd Annual Conference of the Association of Software Testing (CAST) 2008

Toronto, Ontario, Canada, July 14-16, 2008

Beyond the Boundaries: Interdisciplinary Approaches to Software Testing

Keynote Presentations by Gerald M. Weinberg,

Cem Kaner, Robert Sabourin, and Brian Fisher

Tutorials by Gerald M. Weinberg, Scott Barber, Hung Nguyen, and Julian Harty

The Association for Software Testing is pleased to announce its third annual conference (CAST 2008), to be held July 14-16. The meeting will be held in Toronto, Canada, a city which features enormous diversity in culture, businesses, educational institutions, and the arts. Toronto is the perfect location for a conference on this year’s theme: "Beyond the Boundaries: Interdisciplinary Approaches to Software Testing".

You can view the most recent brochure here, and you can see the conference program here.

(text stolen from Scott Barber's blog)

Wednesday, May 14, 2008

Jerry Seinfeld has a trick

I read in some magazine a number of years ago about Jerry Seinfeld and one of the reasons he was successful. When he started his career, he had a hard time finding new material. Sometimes it came to him, but often he would have a writing block. Then he tried a technique where he uses a calender to track that he writes one new joke every day.

So, here are some new things I'm practising everyday:
  1. Trying to write this blog everyday. I have made sure I have enough material to cover me for a while, its just that sometimes I need to edit, which takes me the most time.
  2. Going for a walk every day, even if its for 20 minutes. Sometimes I bike to work, but I try to get some blood flowing each day.
  3. Not eating snacks after dinner. Basically making sure I'm hungry before I go bed. I have a huge problem with snacking, so I decided to not cut it out, but make a conscience effort to remove it from the evening.
Anyway, so far, I have been able to keep this blog floating for the time being. As for the walking, some days are better than others. Same goes for the snacking.

I guess what I'm trying to say is that practise makes perfect (or at least consistant)...Same goes with software development. Any of our efforts require you to take the time to learn and practice the art. So, resolve to do something every day to improve what you do at your job.

Tuesday, May 13, 2008

Zipline and single points of failure

Ever heard of a Zipline? Basically, you attach yourself via a harness to a rope and you zip along a line, except the line happens to be 500 to 1000 feet off the ground and your line can be up to 1000 feet long. So, after doing this on my recent vacation out to Western Canada, I occasionally thought to myself, what's the single points of failure on this thing? A little morbid, but I like trying to understand my surroundings, and my instincts tend to lean towards breaking things.

The rope... each "rope" made of steel was comprised of 200 threads of steel. This is the type you use when you're stringing heavy objects to buildings, etc. The harness had at least 5 different points of failure with some redundancy. The zip line harness had two points, with two carabeeners available too. The platform in the trees had redundancy too...It keeps going on and on. A seemingly unsafe ride was probably more safer than walking across a street.

A good lesson from this is that we should take our single points of failure and make sure we fortify them or make them redundant in the systems we build and support. The feeling of being secure made the overall experience quite enjoyable. The fact that it was secure was not the most exciting part, the part where I was going around 120 km/h at around 500-1000 feet off the ground was the enjoyable part. I highly suggest shelling out the money if you want a great ride.

PS: That's a picture of me on the zipline.

Monday, May 12, 2008

Scheduled posting now live in Blogger

I went on vacation for a few weeks, so you may have noticed that there were posts available before the date had arrived. Good to see that when I get back from vacation that this feature is now available on blogger!

Anyway, I'm going to try to make use of this feature since I keep about 10 posts on my computer before I post them, so this might be a good add-on for me. If anything, it will prevent me from moving to Wordpress or something like that in the future.

Wednesday, May 07, 2008

Books - Managing Humans

Not often do I read a book as funny about management than this one. I've read the Mythical Man-Month, PeopleWare but those books were quite insightful, but d-r-y. This book was refreshing because the point of view is from someone who's lived through the industry.

There's a lot of biting humour and the thing I got from it is that people are not broken down into simple categories. Management from a producer or consumer perspective is "an art", and this book tries to convey that message. Anyone trying to manage people or if you're being managed by someone else, try this book out before the two I mentioned above. Again, I think it has taken the best parts of the dry books and distilled it down into a semi-fictional, funny and witty book.

The chapters are broken into small manageable chunks that you could work through over a quick period of time. I read the book in a few nights. Here's a quick sampling of the chapters:
  • Ch. 1 - Don't be a prick
  • Ch 18 - Status reports 2.0
  • Ch 25 - A nerd in the cave
  • Ch 31 - Rules for the reorg
Let me know how it goes if you end up reading it because of this post.

Monday, May 05, 2008

The Nine Forgettings

This is a presentation I saw at the STARWest 2007 Conference last year. It was a great presentation because as software testers, I think a lot of people don't know how the industry has grown up. In fact, I am one of those people who didn't know where we came from. After this presentation, I decided to find out as much as I could about many of the people mentioned. I would suggest to you that you use your Google-fu to find more information, just so you can arm yourself against people who might steer you into 1983. :-)


Here the Google Video link. I would suggest you watch it since its a great lecture and not too long (around 35 minutes).


However, if you don't have the time, to tie you over, I attached a copy of the power point he created.

Thursday, May 01, 2008

KWSQA Quality Conference 2008

I had the privilege of going last week to the KWSQA Conference

This was a great conference because it was developed by practitioners for practitioners. The speakers they had lined up were also quite interesting. I tried to pick speakers that met a few criteria:
  1. They spent some time putting themselves on the Internet.
  2. Entertainment value
  3. I have read their work or attended a presentation before and they were worth coming back for.
Michael Bolton is one of these guys who seems to be published everywhere and has a great teaching style. I was very lucky to be able to participate in a open discussion with him and a few people. I brought my performance testing scenario and we debated and discussed in an open forum our issues. It was great to have that level of interaction with people who have far more experience in the field than I do. They gave me insight into things like "am I really done testing?". I said no because testing goes on, but in fact because of the lack of some cohesive interfaces, my answer changed to, "yes, it is done... for now". It gave me a perspective on how to deal with people. Interestingly enough, things you can do to drive performance tests are:
  1. Create interfaces that are scriptable
  2. logging and capturing information from that
  3. training to build up tools to do performance testing
  4. outside consulting
  5. test labs (either external, or internal) for any large system
Also, three major types of sizing:
  • customer sizing so they know what to buy/maintain, etc;
  • development sizing, to see where development should spend the money or time for load, etc;
  • product management sizing, they want to see the product perform at a certain level because the market drives the metric.
Given those criteria and advice, I think I have an excellent way to help manage and consult on my next task. I'm also taking a risk-based approach to this whole problem whereby I'm going to ask the question "I want to know what's important to you" and let them drive the tests. There's a lot more to this point and I'll save that for another day.

Tuesday, April 29, 2008

xUnit Test Patterns

I picked up a copy of a new book called xUnit Test Patterns

Some of the developers I have given it to have said it was like gold to them with the help from Design Patterns

Built with a similar structure as the Design Patterns, the reference material at the front and back covers helped to fill in the gaps. The inside front cover shows the design patterns used to create xUnit fixtures and suites. At the back is something completely different - its like a Q&A of when people talk to you about why they can't put xUnit into their applications. My favorite section is "Legacy code cannot be automatically unit tested". Page 365, BAM! Often when I use this lookup, it start a discussion with a developer to begin thinking differently.

Its a nice all encompassing view of how to unit test applications. Although its not new, its nice to have it one place. I think these are one of those books that will be good 10 years from now because of the solid use of patterns over technologies. I'm not a fan of having a huge reference guide for many languages. I use pocket references mostly when I need reference material (or just look it up online). Its a very nice indispensable tool to look up all the different types of unit test structures and patterns in one easy location.

Check it out the reviews for yourself.

For quick reference, here are comments on some of the chapters:

  • Chapter 2: Test Smells - I like this chapter because it explains why code smells and how to recognize where to apply tests

  • Chapter 3: Goals of Test Automation

  • Chapter 7: xUnit Basics - this has saved me a few times now on having to explain to developers how to integrate unit tests into legacy applications. I'm sure I'll get some more running ground on this chapter in the next few years

  • Chapter 25: Database Patterns - this is something I have not got a lot of ground on. People still tell me they cannot abstract out databases. I don't believe you need to, you just should consider the contents of this chapter in order to unit test your database.
To get a copy for yourself, you can order it online or take a look at the online version for free. Again, I like free stuff, so go and educate yourself for free!

Sunday, April 27, 2008

Books - Geekonomics

Here's a book you should avoid. Geekonomics: The Real Cost of Insecure Software and how it supposedly gives you the idea that software is like any other commodity out there.

I recently obtained a free copy of this book and thought "wow, I like free stuff, I might as well give it a read". Well, after a few nights of reading it, I came to the conclusion that this book was meant for those who don't develop software and for someone who only wants to understand how most of their life is ruled by how bad software works within their own lives.

My first reaction was "apparently, we don't test 'our' software enough". This wasn't the author's take, instead he was relating it directly to negative economic and business impacts (which are still true for bad software).

The analogy's used were nicely told; the one story that sticks out in my mind was about how cement/concrete was standardized. Basically, the moral of the cement standardization was that testing was rigorous and people held to that standard for testing it. However, where the author goes awry is when he says the same style of standardization should be applied to software development. I disagree with this statement because software if infinitely more complex than concrete or automobile manufacturing.

The moral of the story is that poor quality controls leads to a economic loss, which in turn hits the bottom line.

Also, don't be fooled by the title, its not that geeky. This is geeky.

A context-driven online course...

BBST Foundations Course - For all you Black Box testers out there, there's some courses available that might of interest to you. From the website:
"The Association for Software Testing is offering a series of free online courses in software testing to its members."
I would take the course, but I'm away during that first sessions. I suggest you take the opportunity to take this free course. Considering its in the lines of what Michael Bolton says about certification (and why it is a biased system of making people pay money to organizations who actually never watch you "drive the car"), this one might actually give you some suggestions and help towards being a better tester. My guess is that it will given the people who are developing/delivering the material are the experts in this field.

Since I can't take the course, I would suggest you sign up for AST (its only $50), and get yourself educated! Best deal in town!

UPDATE: I think I may have found the course-ware online (see video's), but you'll need to view it from home. I think in order to take the course, it would be like taking a driving exam... someone will watch you drive the car.

Friday, April 25, 2008

Google Apps Engine

Google has some pretty neat things. Its more than just the products, but its the archtiecture. They often are on the forefront of putting applications out there that no one else has done. Google Apps Engine is no different.

Remember my post about SOA, well this is it in a nut shell. Other people have SOA apps, but not against the computing power that Google is unleashing.

All in all, its not much different than any web hosting service, except for the fact that you have access to the whole google computer farm! Whoa! now that's a spicy meatball.

Man-o-man! - Tried signing up for the open source, but it was gone in a few minutes. After the initial signup, I read that the load on the system caused slow downs, which is probably a product of it being in a beta.

Entrepreneurs can use this app engine to build their systems without having to buy a lot of infrastructure. This frees up money for more R in the R&D.

From a testing perspective, I would definitely need to play with it to understand what they are doing. I read some of the docs to try and figure out what it does, but playing is what will tell me the most about the application.

I like free stuff...

Information is great, but when someone takes the time to put it into nice digestible chunks, I love that even more. However, when someone gives that away for nothing, well, that puts me over the top. So, I'm sharing with you what I just found...

Five more free magazines for you today:

  1. One of the better magazines, even if they don't have a lot of issues out yet, is from the Association of Software Testers called Sapient testing. I like it because the articles speak directly to the context-driven tester community.
  2. A testing magazine out of the UK, lots of good information here and regularly updated, more Agile and certification focus
  3. Although more infrequent, the Toronto Association of Systems and Software Quality publish a nice little newsletter that has some really good information about testing from some of the major experts.
  4. Software testing in general, good ideas for new people and experts in the field. Good advice as well
  5. Scott Barber and regular articles about performance

That's all.

Initialisms, Abbreviation, Acronyms, Oh MY!

k.d.s. says:
are you in the CTC
k.d.s. says:
?
Marc says:
y
k.d.s. says:
are my sunglasses are in there?
Marc says:
i'll hal, jas
k.d.s. says:
?
k.d.s. says:
oh
k.d.s. says:
haha
k.d.s. says:
I get it
Marc says:
idst
Marc says:
sylth?
k.d.s. says:
I think so
k.d.s. says:
sometime yesterday
Marc says:
atoyh?
Marc says:
dyekwtm?
k.d.s. says:
enough with the arconyms. ;-)
Marc says:
(chuckle)
k.d.s. says:
hehe
Marc says:
there's a word not many people know
Marc says:
many people don't know that an acronym must be a recognizable work or else it's just an abbreviation or an initialism
Marc says:
an fact, things like RCMP and FBI aren't even abbreviations, they're initialisms
k.d.s. says:
oh
k.d.s. says:
I didn't know that
Marc says:
things you probably forgot were acronyms: radar & laser
k.d.s. says:
laser?
Marc says:
light amplification by stimulation of electron radiation or something close to that
k.d.s. says:
neat!
Marc says:
i remember diving into all this because of all the workflow names in tm.
Marc says:
if they are abbreviated in print but you say them expanded (e.g. "etc.") it's an abbreviation.
Marc says:
if the abbreviation forms a recognizable or pronounceable word, it's an acronym (NATO, UNESCO, radar)
k.d.s. says:
ohhh
k.d.s. says:
neat
Marc says:
if you say it letter by letter, it's an initialism
Marc says:
and to think, there are people out there who spend more time on this then i just did!
Marc says:
now i don't feel like i'm so lacking for a life anymore
Marc says:
it's always so good to talk to you, Ken
Marc says:
or at you.

Smoke Jumper

A friend of mine is a smoke jumper. She spends 6 months of the year up in Northern Ontario (about 15 hours drive away from here) in order to make sure the forests don't completely burn down.

I read an article recently that compared software projects in emergency situations to firefighting. While I disagree with the analogy because I have heard some of the experiences from my friend, I do like the quick synopsis of what to do if you are dropped into a critical situation.

The things advocated were:
  • Determine your role
  • Build trust
  • Learn the territory
  • Gather information
  • Do your job
  • Declare victory
I won't go into the details of what the article does an adequate job conveying, I'll just ask you to go read the article and see for yourself. Do you ever find yourself in these situations? Over the years, I've had my share of firefights.

I think a lot of firefights could have been avoided by paying attention to some simple and basic testing methodologies. I'm a little bias, but I do believe that beyond deadlines and politics, what remains is how you finally test your product and how to repeat that test the rest of the product life cycle.