SOA anti-patterns in New Account Opening

I like many people have been reading Steve Jones' SOA anti-patterns post. Anti-patterns are structured ways of illustrating common issues that can be identified, understood and fixed. Steve's article is targeted at organizations that have been trying, and probably failing, to implement Service-Oriented Architectures.

The benefits of SOA have been widely touted, and have been considered a 'panacea' for many organizations with a range of information and content systems spread around, which have grown up over the years through need or more often organizational churn/acquisition/merger etc. For the level of technical impact that they require they carry a huge risk of IT upset if mis-managed.

For an organization to consider improving their New Account Opening processes, and work with business partners more effectively, they are going to have to consider SOA. In most cases the organization will have disjointed systems that will require attention to enable even the highest level of coordination with Business Process Management, let alone full Straight Through Processing.

So for that reason, I picked out this anti-pattern from Steve's list to highlight a significant issue that I see could happen as soon as organizations put down the 'process standards' PDF and start throwing people at the problem. Steve does a great job of describing the problem and how to solve it. I have added my own notes to put it into context as I have experienced the problem.

Antipattern: Percolating Process

Description

Organisations start with a detailed Process map and then attempt to "fit" this into a Service Oriented Architecture; this refactoring leads to process becoming the dominant feature and leads to a Process Oriented Architecture (POA) rather than SOA.

What does this mean?
Look at the way many business problems areas are scoped, described and often targeted for fixing. Its often the "As Is" and "To Be" process diagrams. Most business people are comfortable with those Visio drawings as a way to communicate what they are doing, and how they would like to do it better (with some random technology).

IT people look at the As Is and look shocked that real people would put up with this type of crazy way of working.

Then they look at the To Be. In their technology world this new system has to be built around SOA, right? So they basically split up a bunch of systems that could be scattered around the place to perform different tasks/services. And they change the shapes on the To Be diagram and add a few lines and feel comfortable that the new project has got some structure, with an SOA title to make them feel like they are using advanced technology. This is the refactoring in the anti-pattern description. It doesn't really make the To Be process SOA, but by believing it does the IT guys are going to incorrectly apply the concepts anyway.

Effect

An organisation's "Services" come in two basic types, firstly end-to-end processes that co-ordinate lots of individual steps, and secondly a large set of fine grained services that represent individual steps. Any hierarchy or structure is solely from the basis of process. The fine grained services proliferate and become difficult to manage while the large business process elements become difficult or impossible to change. The systems slowly, or quickly, stagnate and lead to solutions being built on top of the existing solution and the general treating of the process oriented system as a legacy application.

What does this mean?

The To Be process is typically an end-to-end process that may span a whole conference room wall when its pinned up for project meetings. It shows all the people that get involved, the documents they consume and produce and the systems they touch.

Within some of the big steps there is an arrow that points to a completely separate piece of paper that details the fine grained tasks that are performed at that step. That is the extent of the hierarchy - a drill down into more detailed processes.The consultant on the project loves this, because he or she is the only one that can piece together all of the layers of paper, and can always see another step that must be refined.

And so the IT/BPM guys start building processes and integrations. Eventually, hopefully an end to end, semi-automated process is rolled out. Changes to it are banned, because the complexity of the interactions are poorly understood, and the original consultant has left to pursue another more lucrative process reengineering project.

Cause

The organisation has decided to map the processes and end to end and in detail; the work has created a series of grand "end to end" process models that are categorised by their large number of steps and the lack of sub-processes that they use. Once this exercise has been done to a great level of detail it is decided to make it SOA. The problem is that the valid business services have not been identified and thus the process maps have been created without a proper service structure. This makes identifying valid services a tricky process particularly when looking for cross-functional or horizontal services.

The challenge here is that the organisation has not started with clarity on its services and hence has created a Process Oriented Architecture. Retrofitting services into such an environment creates a sub-optimal system that is liable to have all the issues of other process based systems, such as COBOL.

What does this mean?
The To Be complex business process has not been designed around the people and tasks AND the systems/services that should support them. Nobody really thought that some of the back end systems may actually need to change to support an SOA approach.

You can't just put a new process in place and plug in at various points to the back end and hope all will be fine. The back end system wasn't designed to operate in that way - it probably controlled some of the process you have hacked apart, and by working around that there will be strange side effects (like the system not working any more). And the work-arounds put in place instantly make the new process and systems look like the legacy code you were trying to get rid of.

Resolution

The first resolution is to independently of the process map create your services architecture. This will provide the structure for breaking down the processes and creating a clear hierarchy of use. Next, this service architecture should be over-laid onto the process map to understand where the cuts should be made. The current solutions can then be refactored to create a more service oriented solution by attacking the major inflexibilities in the system and then looking towards incremental change of the current systems.

What does this mean?
This doesn't need too much said about it. When you understand what systems you have in place today and how they can sensibly be split up into new services you can see how the To Be process can best utilize those services. Some changes will be needed to the back end systems. But at the same time, if one of the systems is not going to be open to change, maybe the To Be process should try and embrace it. Otherwise be prepared for major investment.

End note

In writing these anti-patterns Steve has produced a set of very valuable indicators to the health of current projects and red-flags for future projects that involve process, and inevitably SOA. New Account Opening improvement projects will encounter these anti-patterns in many organizations, so beware!

[Update]

Bruce Silver has an interesting take on this anti-pattern as the new business-IT non-alignment anti-pattern.

Technorati tags: Financial Services Technology New Account Opening SOA anti+patterns business+process