Thursday, June 18, 2009

The missing link

I don't know when it happened, but the evolution of building software applications for business, especially those based on business processes and BPM, has led to a missing link. Not a sub-human creature of limited intelligence, although certainly a large gap in knowledge and capabilities inside the teams that define and develop solutions.

Leading a recent business process management project, I ran across this missing link several times. And it has led me to compare the profile of the software projects I was part of maybe ten years ago. Back in the good-ol'-days, software projects seemed to have a nice neat demarcation of roles: the business analyst worked with end-users and management to define the requirements that fed the current needs of the business and the perceived improvements; a software architect worked with the business analyst to translate these requirements into a software architecture of blocks, arrows, UML, and written specifications; finally the developers worked with the architect and (if they were senior and knew how to tie a tie) with the knowledgeable users to work out and eventually code software logic.

With BPM, the missing link has evolved through the pressure of vendors, industry analysts, and probably financial pressure in organizations wanting new software solutions cheaper. The roles have morphed as vendors and analysts tell companies that their business analysts can define and build workable processes without the help of their IT team (although see my post: Your 'Systems Group' is the reason you business teams (are) disintegrated if you want to see why I think that it wouldn't work even if your IT team was involved). The reality is that with the right tools, business analysts can build working processes, though they probably don't want to, because real (rather than just workable) business processes are hard, detailed, time consuming beasts.

So what is the profile of a team in the modern world? A senior business analyst has turned highly strategic (maybe because there is more caché in a hand-waving strategy role?), but still gets his or her hands dirty from time to time defining high level processes using a vague understanding of something BPMN-like. A more junior business analyst takes some of the concepts and tries to make a vague guess of how this maps to the details of the business. The great leveller of industry standards says that if you use BPMN you are sure to get a best practice result (I lie), and then fill in some spreadsheets for the other details you find out along the way that might just be useful. Now throw this information at a consulting firm's expert in your chosen BPM product, telling them that the analysis is complete and there is no room in the budget for further analysis, so just get building. Actually sounds very Dilbert-esqe.

What is happening? The software architect role is the missing link. He or she was carefully superseded by the enforced architecture of the BPM foundation the solution will be built on, forgeting that there was more to the role than blocks and arrows. The business analysts do not really know how to interact with software developers, since their contact has only been with senior strategy people they one day aspire to be, so the level of detail they are able (or willing) to collect is variable. But they try. The consultants are therefore forced to try and meet in the middle by wearing the hats of architect, process analyst, business rules analyst, process designer and software (at least customization and UI) developer. With top quality, highly experienced consultants, there is a good chance that they will fit the gap without a nervous breakdown. For less experienced consultants (or maybe worse, consultants highly experienced in traditional software lifecycles), the gap and constant flux in project requirements, definitions and uncertain rules can be highly jarring.

So, who wants to evolve to be stronger, better, faster, and take some of the traits of the extinct software architect? I don't know - and I'm open to offers from anyone who has seen this pattern in projects or believes that the architecture role is still alive and well. My guess is that the profile of projects I've been involved in has led to the customer not wanting to pay for the people I mentioned: process analysts and business rules analyst. Or maybe there still is a software architect role in BPM under a different name.

Either way, this trend of half-hearted, strategy rather than details-oriented business analysis, coupled with do-it-yourself process developers, seems to present a big risk to BPM projects. There are a limited number of BPM implementation experts out there, so if you want to implement a BPM solution, pick well!

I hope someone can help me with some solid experience, or tell me that consulting firms should stop giving way to clients and force them to pay for more time and resources. Or maybe someone will tell me to pick a different BPM tool that makes analysis, implementation and deployment so easy that the missing link was in fact a natural and necessary part of software evolution.

Comments welcome!

A post from the Improving It blog

Download the podcast of this blog post

No comments: