Testing software, in the form of products, custom applications or BPM based process implementations is surprisingly very different. I watched skilled, professional testers provide amazing coverage of a product in my previous life as product manager, with automated tools preventing regression. I've been part of teams building custom business software, then working hard on testing that software so it gets through the final user acceptance without a hitch. Right now I'm working with a client to ensure that the BPM and Case Management solution we built, following an agile methodology, not only meets user requirements, but works without fail. It is this last one that has presented some interesting challenges. How do you transition from an iterative, modified scrum development model, to the formality required to ensure the solution that has evolved just works? This is hard for the analysts and end-users, and surprisingly for the developers as well.
The head of the systems group pointed something out to the team of end-users who have been working with us on the project since day one, and who have now been enrolled as testers. Paraphrased (fortunately, because the Spanish translation would have been bad) he pretty much said,
You've had your chance to say what goes into the solution. Now test what we've been delivered, not what you wish you'd asked for 6 weeks ago. Now is not the time to be suggesting new requirements, or complaining that some things are more difficult to do than you would have hoped. If you can work around a limitation or restriction, you will. Feel free to note potential improvements and we'll rank them for phase two. If the solution does not work, throws an error, or completely blocks work from moving from one activity to the next then your tests have been successful. The aim of this exercise is to come out the far end with a system we can be confident will work in production.
I have a lot of respect for that attitude, especially when it comes from the guy who is paying the bill for the solution that is delivered. Pragmatic, realistic and most likely to lead to a project that is delivered to time and budget, while meeting 90% of the user requirements. It also helped the end-users adjust their perceptions from creative and business minded analysts, to logical and step by step testers. They are testing with real data and documents, which will hopefully uncover any hidden data level issues, but will the test cases cover the 90% of insurance underwriting tasks they have to cover day to day? If you see the work-arounds they have to do today with their paper and 'systems', you'd think it would be hard to fail.
We have one thing on our (the consultants') side, which is the same thing that always represents a risk. The previous paper process effectively had no automated controls, but because of that it meant that you could do whatever was needed to get work done. We've added some controls and rules. In fact over the course of the project we've added, then refined, and finally removed many more. As expected, at the start of the project the appeal of being able to control and restrict a process and its flow was exciting for the users and analysts. They define many things that they realize later on just won't work in practice. The trick to this is helping them understand that this is a step forward from today and flexibility is a positive thing. And fortunately it means that the likelihood of testing failing due to poorly implemented, highly complex rules is much reduced.
So, my beloved user testing team, please beat up our solution this week, with logic, flow and some flair. 'Cos its a damned site easier to fix big problems now than after go live!
A post from the Improving It blog
No comments:
Post a Comment