Session testing

Earlier today I made a serious attempt at performing session testing.

I had created a simple high level test plan to get started with, and I used Quality Center to log each 'session'.
For each of these, I included the area that I wanted to test, and I recorded the tests that I performed.

I created a separate test for each session and planned 30 minute increments to test. My goal was to spend that 30 minutes testing and not to stop even if I found a problem. I would make note of the problem and continue on if possible.

I logged three separate sessions today.
Session 1: I planned for 30 minutes. I installed the latest version and went to check out the new feature. I ran into a show stopping bug within 2 minutes. A problem with the build process. I made note of this problem in QC and then told the developers, who promptly corrected it.

Session 2: I again planned for 30 minutes and was confident that I would get further than I had previously.
This time it took me 6 minutes to find a bug. This was the hardest part of my day. I wanted to jump up and point this problem out, but I had to will myself to note the problem, and continue testing. This could wait 24 minutes to be reported.  By the end of this session, I had found one more bug and an issue that required further follow up.
I recorded each of these in the session notes and my teams discussion board. This way the team could decide how to handle these problems going forward before creating defect reports.

Session 3: I planned for a longer session of 1 hour now that one of the bugs I recorded had been dealt with.
My first task in this session was to verify that the bug had indeed been corrected. I went to visit that area first and found that while it had technically been corrected, it was still wrong.

Summary:
This was a good learning experience and its something that I want to do more of. I think that I need to refine my sessions a bit, but for a first attempt it went pretty well. Session testing seems to be a useful way to provide some more structure to what I already do.
Previously I would have tested each new version and I would have found those bugs. But I might not have found them as quickly if I had dropped everything to research and report the first problem I found. I may have  written some generic tests for acceptance criteria that weren't very helpful and would never be looked at again.
What I have now is a trail of documentation that better describes the testing that I did, how I encountered problems, and how I dealt with them. I'll continue to refine this idea for my own use and I'll post about it again.

Release notes as test cases

I've been testing bug fixes and writing release notes for them for years now. A release note is the high level documentation detailing a problem that used to exist in the software and explains how it has been corrected in the latest version. These notes are compiled and sent to the customer along with the software so that they can see what changes have been made.

I like writing release notes, but I never realized how important they are, and how they can be used to help test. I had just finished testing a bug fix and was going through the motions of completing the story.  The release note is typically the final act.  I had to explain to the imaginary customer what we had just changed, and how they could verify it.

The way that I explained the problem and its correction in text caused me to stop and ask myself a question. This question led to another test. That test led to a potential problem. So there I was, ready to close the story that I felt was sufficiently tested and move onto the next one, when writing the release note sent me back to the drawing board.

Crap.

Well, not crap. I found a problem.  But could I have found it sooner? My tests passed according to the acceptance criteria. My destructive tests didn't turn out to be that destructive. I felt that I had covered my bases when I started writing the release note and by the time I was finished found that I was wrong. I'm never wrong.* Thinking like a user is different than thinking like a tester. If someone was to explain a correction for a bug to you, what questions would you ask?

I had my car in for service recently because the check engine light was on.  The service technicians explained that I needed a new fuel leak detection pump. Or something. Whatever. I signed the papers and paid the bill and my only question when the work was finished was 'Is the check engine light off?'.

The next time I test a bug I'm going to try writing the release note for it first. It's not just acceptance criteria, its the information that can and will be questioned by the customer.





*I'm usually wrong.

The other side of the interview table

Today I got a chance to participate in an interview for a tester position. I sat with my supervisor and followed her lead to get a feel for things.  I was encouraged to ask questions of my own, so I consider this to be a sort of career milestone.

I did some research on 'how to interview', and naturally started with Joel Spolsky's "The Guerrilla Guide to Interviewing". Granted, it was not for a developer position, but it was helpful all the same.  I kept his two big points in mind, looking for 1. Smart and 2. Gets things done.

I also drew some questions from Cem Kaner's article "Recruiting Software Testers".  This was definitely a big help as well.  I was glad that I'd done the research before just walking into the room, because it was a good deal more challenging than I thought it would be.

I started explaining the first question I asked before I realized I needed to leave that up to the candidate. I also found that I was more concerned with what questions to ask than I was the answers provided. There were certainly follow up questions I could have come up with instead of moving onto the next one I had written down.

All in all, I think it was a fine first attempt and I was able to solicit some good conversion with the candidate.  Hopefully I will get more opportunities like this in the future.

Todd

Hello World

cout << "Hello, world!\n";

This is my first blog post.  I hope to improve my writing skills by using this blog, and plan to post often about totally awesome topics.  Some of these topics may include, but are not limited to: Software development, software testing, and movies.
Stay tuned for more.