Email archiving and the Big Belly solar trash compactor

I ran across one of the shiny new Big Belly solar trash compactors in Boston this evening. As Mayor Menino is demonstrating in the photo, it swallows trash through a drawer in the front. Its big so that it can hold a lot of the stuff. And the innovative guys and girls at Seahorse Power Company have added a solar panel on the top to power a built in compactor, enabling Big Belly to hold even more, and without it needing to be plugged in to a nearby streetlight to power the compactor. This gives it an air of environmental friendliness that probably fools some of Boston's residents into believing that they are being very 'green' by feeding the thing. But its real reason for doing this is to save on the frequently overflowing trash cans scattered around the city, being picked up by an overstretched workforce.

I want to compare Big Belly with email archiving. Email archiving technology is only successful because of the need to overcome the issue that email, as written about by Keith Harrison-Broninski, is not suitable for business use. The fundamental reason for archiving emails was obscured long ago: ensure an auditable record of all communications. Now these systems sell on their capability to consume more email than the email server can sensibly store.

To me, Big Belly and email archiving both represent an underlying problem that we are trying to hide by their use. We create too much trash and too much email. And this was sadly reinforced when I was finishing up my stroll round the city. There was a guy wheeling a shopping cart full to overflowing with glass and metal containers that earlier in the day were holding beverages of different varieties. As a power user of his chariot he deftly ignored the 4 lanes of traffic he was steering across, maneuvering the empty cans and bottles towards an unknown (to me) destination that would presumably pay him a few cents for each pound of waste material he wheeled in.

In my mind the 'green' veneer of the Big Belly solar trash compactor was wearing off, probably as fast as its shiny paint will when attacked by Boston's road grit and salt this winter. Sure, it chews a lot of stuff up, requiring less maintenance, but none of that stuff is recycled. Boston, unlike other more progressive cities, has not really recognized the value of recycling on the street. And the people that do, like the shopping cart guy, will have an important source of revenue crushed up inside a large green box.

Along similar lines, email archiving enables the IT group to continue to operate the inefficient and ineffective business communication mechanism, requiring less emptying, while trapping all of the information value inside individual user mailboxes.

As Microsoft pushes its ECM strategy and other vendors are forced to respond, many more users will become familiar with collaborative tools for sharing documents and capturing discussions. These capabilities are offered today by MS Sharepoint, Vignette Collaboration, Documentum eRoom, amongst others. When used as most users do these tools at least provide a sensible collection point for what would otherwise have been trashed email attachments and messages, which is a start.

Hopefully the enterprise will also embrace the technologies that could really reduce email trash: blogs, wikis, RSS and IM. Only then can the email archive return to what it was really intended for, to capture auditable copies of valuable email communications, thus enabling far more effective legal discovery processes and significantly reduced storage costs.

As for the Big Belly trash compactor, I fear it may be here to stay. Good luck shopping cart guy!

Technorati tags: environment email+archive email trash collaboration

ECM in a post Microsoft world

A recurring theme is the threat that Microsoft is posing to the Enterprise Content Management (ECM) industry with its upcoming release of Sharepoint 2007, packaged with Office 2007. This release is likely to bring Sharepoint to the desktops of more Windows users than ever before, enabling them to experience first hand the advantages of using ECM technologies in their day to day business.

The key things that most users will be exposed to are:

• Metadata for the identification and classification of documents enabling them to be searched more easily within a business context
• A portal for seamless access to a range of data sources
• Saving documents to a records repository for lifecycle management
• Collaborative working environments enabling multiple users to share and create documents and knowledge

And all of this will be performed seamlessly in a familiar user environment.


Threat to the ECM industry

Microsoft has a huge marketing budget (some say $500 million) to bring awareness to their ECM offerings. This will obviously be a cause for concern, since the majority of ECM companies are not in a position to compete at this level. This has been discussed by several industry observers including:

• Russ Stalter : Hmmm…Maybe Jim Cramer From CNBC’s Mad Money Reads My Blog
• George Dearing: Microsoft's Second Mover Advantage in ECM

Much of their attention has been turned to Filenet as a prime example and how it will cope with this threat, or whether it will just become another acquisition target like Hummingbird.

The fear for many ECM vendors is that Microsoft will swallow up market share and will commoditize ECM offerings so that there is no value for anyone else to compete (except maybe the open-source players).


Its not all over yet

Microsoft will release Sharepoint 2007 early next year, meaning that many organizations will not be in a position to update most of their office suite to really benefit from it for another 18 months at least. As I suggested in Vista in 2007 v. improved New Account Opening now, some organizations probably have budgets in place and demonstrable ROIs from actually deploying much before then. But really that is just a stay of execution.

In Dear Mr CIO - don't hold off buying ECM I talked about how the players targeting different levels in the market could cope with the Microsoft invasion, suggesting that for the enterprise vendors there is still a light at the end of the tunnel. The four serious points I made here were that the enterprise vendors could:

• Extend and enhance in-house technology with the MS capabilities
• Build business solutions on top MS and in-house technology
• Embrace the user facing components, like Office 2007 with strong integrations and value-adds
• Identify and enhance enterprise technology requirements that MS can not easily deploy

One major thing that I have not seen discussed is this: Microsoft is not typically adopted by corporations to provide enterprise-wide capabilities.

In my experience, most MS deployments are targeted and implemented at the departmental level, a strategy that meets the architecture of the products. Very large deployments of MS technology require federation, deploying multiple servers and additional technology to tie them together into a whole. So it is far easier to just deploy departmental level systems and track them as independent instances, which is ideal when each department has different requirements for its systems.

E is for Enterprise

Let us focus on ECM and get back to basics: E is for Enterprise.

A major selling point for the true enterprise-level ECM vendors is the ability to deploy a single system, capable of managing the whole enterprise’s content. Especially when it comes to records management, defining records lifecycle policies centrally is essential for effective management of enterprise records through to destruction.

In the Microsoft model, having multiple instances of records management systems spread throughout the enterprise, to match the federated nature of the Microsoft Sharepoint architecture makes true enterprise content and records management complex to track and control. Microsoft is rarely truly enterprise level.


Everything has its place

Remember that $500 million that Microsoft is expected to spend on marketing its ECM vision?

Not only will that bring awareness to the MS products, but it will actually finally awaken the recognition that ECM has been struggling for years to achieve, despite the efforts of vendors, and organizations like AIIM. This should drive demand for ECM products at all levels.

For the ECM vendors, this is a golden opportunity to keep Microsoft in its place – confined to individual departments. To capitalize effectively though, there are some things that the vendors will have to do:

• Partner with Microsoft to give the appearance of embracing its model
• Integrate with the MS Office/Sharepoint 2007 suite fully and completely
• Embrace the strengths of Sharepoint and its pervasiveness to the vendor's advantage
• Identify gaps in the suite and fill them with value added features
• Estimate, measure and demonstrate the value of true enterprise centralized ECM capabilities, including records management, web content management and business process management

Act now

In my opinion, all of these items are essential as long as vendors acknowledge that eventually Microsoft will copy many of the best capabilities, functionality and arguments that an independent vendor has. Acting now will help to ensure a firm footprint, and a model that the market can take as the de-facto architecture. Investing too much will be a waste long-term. But being adaptable and a vendor can provide a long-term profitable solution.

My view (and its not an expert opinion) is that vendors can more than survive, rather thrive, in the post MS ECM world. But they need to play to their strengths, and show flexibility that few Filenet, Vignette, EMC and other companies of their type have.

[UPDATE: correction to URL for Russ Stalter's post]

Technorati tags: Financial Services Technology New Account Opening ecm microsoft sharepoint

BPM could drive online office applications

Business processes automated by a BPMS require human interaction at points in the process, like exception handling and knowledge working. In this post I want to share my opinions on how these human steps could benefit from the new generation of online word-processing and spreadsheet tools by embedding them directly into the user’s processing application. Tools like Google Spreadsheet, Zoho Writer and Office Suite represent the available capabilities. The IT|Redux blog provides a good listing of online office tools.

Use desktop tools for collaborative 'processes'

My post Collaborating in structured business processes talked about the extreme case where a dedicated collaborative application is required to enable users to complete a complex set of human driven tasks at specific steps in the process. I illustrated an example where extensive anti-money laundering reviews were required during the account opening process for a new high-risk financial customer.

In scenarios like this, collaboration enables a set of users to work together using the tools that are most convenient to them. Typically they will produce documents using MS Office products, research tools and business specific application. The users record their working and final decisions as text documents and spreadsheets, stored within a specific collaborative workplace. Due to the lengthy and varied work that the users are performing this is probably the best approach to improving their effectiveness.

Knowledge workers and other human interaction

More commonly occurring than full collaborative requirements are the steps in a business process that require input from a single knowledge worker, to process a case and make decisions based on information presented to them.

An example is an insurance new business process, where at certain steps an underwriter is required to assess policy terms. The underwriter is delivered the work, being presented the customer details and full application form to assist in the assessment. At this point, typical systems will leave the user to utilize external workbench tools to make their decisions. More often than not this requires that the underwriter re-key information from the original application form into a spreadsheet that enables him to determine risk and policy value. At the end of his manual processing he saves any documents that assisted in the decision to his desktop and attaches them manually to the customer case file or a shared file system.

Online productivity tools provide just enough

This is where I believe that online productivity applications like word-processors and spreadsheets could be valuable. Despite the fact that the functionality in these tools is probably insufficient for power-users, what is available right now is ideal for the specific tasks required at human steps in business processes, like the one described above.

Most online email users or bloggers will attest to the fact that the editing and formatting capabilities of these tools are sufficient for everyday document production, and that the vast majority of MS Word capabilities are left untouched. Especially where there are targeted requirements for document production, like customer correspondence, documenting basic research findings, and so on, more extensive features just get in the way.

Value to the user

By embedding online applications into an underwriter’s BPM application the key tools of a basic underwriter workbench can be presented seamlessly alongside the customer information and application form. For the user this has many possible advantages:

• Seamless access to key applications in a single browser application
• Ability to open template documents with data transferred directly from the application form and customer information data, avoiding re-keying
• Direct and enforced storage of documents back to a central repository, not requiring additional steps storing to the desktop first
• Simple, easy to use features appropriate to the limited user requirements

Value to the Business

From the point of view of the business, there are many advantages, including:

• Reduced cost per desktop, by not requiring expensive MS Office installations, and reduced hardware requirements due to the editing applications not being bloated with unnecessary features
• Improved audit and compliance through enforced recordkeeping of process related documents
• Improved accuracy of human processing due to the reduced requirement for re-keying information
• Improved user efficiency by using tailored, streamlined applications appropriate to the user requirements

Value to IT

The IT group will see benefits that enhance the savings the business may see:

• Zero desktop installation and upgrade requirement
• Reduced risk of macro viruses
  
• Ability to more effectively lock down user desktops, since no documents or files need to written locally
• User support calls reduced due to simplified application usage and no 'lost' files saved to an unknown directory

Summary

BPMS driven processes have always offered the option of tailoring the user facing applications used for delivery of work. In more mature customer systems these will integrate business systems specific to the users' work, but will rarely take into account the desktop tools that they must use to complete their tasks.

By including online word-processing and spreadsheet capabilities directly in the user interface there are many advantages to user, business and IT. It could be that this ability to embed seamlessly into the user process 'workbench' becomes a strong driver for this new breed of office tools.

Technorati tags: Financial Services Technology New Account Opening web+2.0 productivity BPM user+experience

How to mix users and forms

"It looks far too complex"

was the phrase that introduced my previous post about browser based forms and the perception of application that include them. The phrase illustrates a common issue that faces many software developers, sales engineers and anyone that demonstrates software that includes a browser-based form for capturing data and attributes. Software applications that need users to enter structured data are typically poorly received.

In that previous post I suggested some opinions as to why users hate forms and more broadly any applications that attempt to impose structure. In this post I'm going to offer some opinions that represent my non-expert status in "user experience" design, to help encourage less of the original response that "it looks far too complex", and more of the "when can we start using it!".

How to fix the problem

I am not a "user experience" consultant of great repute, so my thoughts are just observations - that I'm going to write down anyway. The following lucky seven short sections describe how we might approach making data entry less odious to the user.

1) Try the MS Office 2007 approach

MS Office 2007 introduces a new approach to capturing document attributes or properties , along with other usability enhancements. The experts at MS have also noticed that users hate entering data, so they have provided a new novel approach to the problem. These are some suggestions that I have drawn from observing the new application (Office 2007 beta 2)

• Constantly display the document or case attributes in an information panel, so it becomes less of a big surprise and threat
• Allow the user to enter the information at any time that suits them, without having to artificially navigate through a new dialog or form
• Only bully the user for entry of information when they have failed to enter it in the information panel, as late as possible

The lesson here is that users might actually add structured data if it doesn't break up their other work processes with annoying dialogs, and may actually provide them with some informational value.

2) Avoid the traditional forms display

Many forms on websites and enterprise applications alike are long and linear, since this easiest to design and implement. Even LinkedIn, a popular application designed to encourage you to register confronts new users in this way (although their form is at least segmented). This is because vVertically oriented forms may appear more lengthy than necessary. Also, when a form is positioned in a vertical panel to the left of a document it appears more urgent, as western users read left to right.

Here are some thoughts to improve this with a different orientation:

• MS Office 2003 used vertical panels on the right, maybe for this reason
• MS Office 2007 uses a horizontally oriented document information panel at the top of a document. It seems to be accepted by the eye as a more integral part of the application task ribbon (the new Office approach to toolbars) than other approaches, and does not carry the issue of appearing unnecessarily lengthy

3) Just In Time data entry

Capturing information across several screens, at relevant points in the process breaks up the appearance of a lengthy form and the monotony of entering all that data. The amount of information required at any one time is less overwhelming to the user. A key design point is that it helps the designer ensure that only relevant information entry is presented to particular a user.

4) Don't make it look like a tax return

Use a very clean, modern, friendly forms design. Investing in the best look possible will help it appeal to the user visually and will avoid clutter, making the usage more streamlined, which is essential if users are expected to return to the form in the future.

5) Know your user

Presenting the form in a presentation format most appropriate to the user profile is essential, and one item that I believe is regularly overlooked by applications. Here are some key thoughts around this:

• Know the size of the screen, and use it in a way appropriate to the application. Blog applications like Blogger are often terrible at this, presenting an editor that is too small to reasonably write and review lengthy posts.
  
• If the user doesn't need accessibility features, or keyboard shortcuts for productivity, avoid showing them them this information. It just leads to clutter and confusion.
• Where possible pick up the user profile automatically, so that accessibility features can be display only if needed.

6) Trick the user

Fool the user into believing that they are not entering information into a form, so that they are not threatened by it. This may also help the designer find more effective ways of collecting the information required. Try:

• Presenting the form as a series of friendly questions can be less threatening, and can hide unnecessary attributes that are not required
• If information entry is not enforced, experienced users will not enter it even if you show them textboxes for it
• Allow users to enter information over the course of their usage, not in a single enforced form or lengthy wizard
  

7) Share the burden

This is my favorite. The assumption in applicaitons that collect and share information, in either a collaborative or process driven tool, is that all data must be collected up front. Data entry is often performed right outside the mailroom in enterprise BPM processes. In more field based processes, the person initiating a request or document may be unfairly penalized with all of the data entry to identify the item. I would try and bear these points in mind:

• Don't make the case initiator (e.g. the salesman that creates the request for a new contract) enter all of the information. That salesman has a job to do, and its selling your stuff, not filling in forms.
• If the user can not see the advantage to them of entering the data, they are unlikely to take the care to fill it in correctly.

Let's think about the example in the previous post of Legal Contracts Management. If some of the information on the form is just there to make the legal department's job easier when it comes to searching for contracts, have them enter the information, not the salesman. For him or her the legal department's search requirements are invisible and certainly not valued.

At the end of the day

For any application utilizing data entry by typical users, some novel ideas may be required. Otherwise a typical user can often become defensive at the sight of a single data entry form, runing the impression of an otherwise incredibly usable application. Despite this issue the application requires attributes to be entered for it to work correctly or deliver its full value. A balance needs to be struck.

As with any software application, get the buy in of some positive business champions early on in the design. They can help to guide your requirements, and diffuse negative feedback out on the shop floor.

When deploying a generally accessible web application, or an internal enterprise application, training and championing is impossible on a large scale. In these scenraios you need to work harder on the design to make it really appealing to a mass audience using the application with minimal guidance, training or change management.

Take as many tips as you can from the best online web businesses and apply some of the lucky seven ideas above. With luck you will hear less of the the original response that "it looks far too complex", and more of the "when can we start using it!".

Technorati tags: Financial Services Technology New Account Opening user+experience data+entry UI
web+design

Forms and users just don't mix

"It looks far too complex"

is a common phrase heard by software consultants and professional services people when proudly presenting the new prototype UI for a document centric application. It might be a great collaborative, document management or BPM based application, but either way the business users just don't like the look of it.

Attributes for classification

Many simple automated business processes, collaborative environments and document centric applications benefit greatly from an appropriate level of structured attributes or metadata to identify documents. Otherwise it is no easier to find your documents than a large folder on a shared file system on a Windows server.

A nice example of a business process centered around documents is legal contracts authoring, negotiation and management. This is a use case that is relevant to most organizations, so I'll talk about it as its easy to understand. It uses a document-based collaborative application to facilitate the interactions between lawyers inside the company and the basic processes that they must go through to write new contracts, negotiate with external organizations, keep records of executed contracts and handle the expiration and renewal processes around their agreements.

In a reasonable sized company, to enable effective legal contracts management some attributes need to be applied to the contract package (a folder, workspace or some other object collecting all contract documents). These are there to facilitate:

• Search
• Audit
• Process enforcement (review/approval and expiration/renewal processes)

Most of the required information is entered up front, such as contract value, type, company name, expiration date, etc. On seeing the nice, structured browser-based form for capturing this information (or right at the end of the presentation) a user will come out with the comment I started with: "It looks far too complex".

A sad story

I have experienced this reaction with this type of application and more complex BPM applications for new business processes and just assumed it was my lacklustre demonstrations skills that had failed yet again to wow the users. Colleagues regularly tried to reassure me that "its the application, its just too ugly/complex [select as appropriate]". I was never entirely convinced until just last week when I stepped in to help show a new document-centric use case, based on a typically very well received collaboration product. Again, at the end of the process I got the 'too complex' line, something that I was just not expecting.

Thinking that I could not blame the application I probed a little deeper. What was the source of the problem? Surprise, surprise it was the presentation of a nice browser-based form to capture a limited set of attributes at the start of the process. Comments included:

• Users will find it too complex
• Users won't fill it in correctly
• Users will just enter random key-strokes quickly to cheat the system
• Users will just refuse to use it

What is the issue with these users?!

Why is it that users that do not typically work with structured applications become hesitant or plain defensive when they see a structured form designed to capture attributes quickly, accurately and efficiently? I have taken some guesses, and would love feedback from anyone that has more ideas or can point me at other valuable resources. My guess is that users are threatened by forms due to:

• Previous bad experiences with badly designed forms
• Overwhelming appearance of lengthy forms
• Belief that too much information is being captured
• There is no understanding of how the information is used
• The information requested is not relevant to this user or at this stage in the process
• Belief that 'data entry is not my job'

The crazy thing seems to be that many users would be happy to hammer in a freeform email that contains most of the same information and send it through to another person describing what they mean.


Not placing blame

The outstanding acceptance of document collaboration tools by users provides some insight into what a typical untrained user will accept from an application, with the common features that:

• Shy away from structured data
• Dictate little enforcement of process, defined usage patterns, document formats, styles or layouts
• Encourage freeform communications
• Do not enforce the entry of any attributes

The tools reinforce the proposition that it is good to capture unstructured documents such as the freeform email my crazy user was happy to write in my example above. The problem is that this information is often not obviously reusable in this format for the next user that needs it.

I'm not blaming collaborative tools for the problem, but they do highlight how much users hate entering data. By introducing just a small structured form and process into these tools leads to the original user feedback, as my sad story recounts.


Placing blame

There is only one place to look. I can no longer entirely beat myself up for demonstrating so many applications so badly. Rightly or wrongly I'm going to blame the forms UI. Not just the one in this application, but the whole concept of data capture forms UIs.

Many applications and business processes can benefit from a bit of data entry, and in structured business processes (like new account opening) this is even more necessary. So it is hard to escape this 'new evil', the forms UI.

In the next post I'm going to offer some opinions that represent my non-expert status in user experience design, to help encourage less of the original response that "it looks far too complex", and more of the "when can we start using it!".

Technorati tags: Financial Services Technology New Account Opening user+experience data+entry UI
web+design

Collaborating in structured business processes - Part II

In the last post Collaborating in structured business processes - Part I I talked about the need for the true collaboration capabilities offered by collaborative software at certain points within a structured business process. The challenge is that collaboration software typically does not extend itself to external processes, people or systems.

I used the new account opening use case as an example of the need for collaboration within structured business processes, especially at the point where complex exception handling is required within the process.

Bruce Silver has pointed me towards some of the vendors that provide this type of capability, in his comments on my first post. This post follows on from the first to describe how I envision that a standard BPMS could be extended to enable collaborative features if they are not already part of the architecture.

I would love to know what other use cases could benefit from this type of capability, and even better any examples of where it has been deployed in reality.

Collaboration as a service

Rather than treating collaboration as a pure user application we need to treat it as a service. This provides us an approach for BPM to orchestrate a complete process including touchpoints that interact with a collaborative environment. At specific steps in the process, BPM can call out to the collaboration service, allowing the ad-hoc work to be completed before carrying on with its structured duties.



Imagine that BPM requests the operation "collaborate around customer money-laundering risk" from the service. Under the covers the collaboration service might do the following:

• Create a new collaborative workspace from a predefined template, to contain all of the research documentation and discussions
• Assign values to a set of attributes directly from the BPM new account application, describing the basis of the exception to be handled to the collaborative users
• Create links to any documents referenced by the account application within a "background" folder in the workspace, enabling users easy access to them
• Provide links to customer information held in CRM or accounting systems
  
• Email a predefined group of users that a new exception requires processing in the collaborative environment

At this point BPM has successfully passed control of the process to the collaboration service. It can now sit back and wait. The collaboration application can now go back to being a user driven application, what it does best.

If well thought out, the collaborative users processing the exception may never need to touch the BPM system to perform (or be informed of) their tasks, centering their attention on their primary collaboration application and research tools.

Handing back control

Completion of the exception handling process happens when the collaborative users decide that they have completed their investigations, making a go/no-go decision. Several things should happen at this point: working documents should be stored to a records repository; the workspace should be locked down; BPM should be informed that it can resume the case, being handed a minimal amount of information to enable it to route the it appropriately, based on the collaborative users' decision. Of course, a reference would be available to easily get back at the collaborative workspace and all of the information it contains, if required by a user at a later stage.

Handling of timeouts, where collaborative processes exceed those configured for the business process need to be handled, through further interaction with the users through standard collaborative means like entries in group discussions, or by escalation to a manager.

Records

Storing records of the collaborative workspace is essential in many regulated or potentially litigious environments, and selecting an appropriate collaborative tool that supports integration with a records management system is key. It is also important that the collaborative tool has the flexibility to enable continuous access to these records through the collaborative UI in a completely unchangeable manner. This is essential so that audits can reveal the underlying work in context, and litigation is supported.

SOA

Although I use the word 'service' to illustrate a set of defined interactions of a BPMS with a collaborative tool, this type of integration does not require a full SOA for it to be useful. Of course, if the organization has deployed SOA, demonstrating the value of collaboration as a reusable service across the enterprise could be accelerated by deploying defined use cases for BPMS processes and collaboration.

Outcome

In this example the BPMS has successfully orchestrated the process across its own human interactions while managing data interchange with partners and business systems. By treating the collaborative application as a service that can be called from a workflow at predefined steps, BPMS can handle collaborative requirements as seamlessly as any other interaction with an external system. Now tasks like complex exception handling can be performed as well, in an environment that is most effective for the users, without them ever needing to become familiar with the BPMS controlling the process.

Technorati tags: Financial Services Technology New Account Opening BPMS collaboration SOA

Collaborating in structured business processes - Part I

Most enterprise-level collaboration products (think MS Sharepoint, Vignette Collaboration, Documentum e-Room, etc) offer some level of process automation capability or workflow engine. This is employed to enable the coordination of tasks revolving around documents, folders or workspaces, to provide some structure around what would otherwise be a completely ad-hoc (possibly chaotic) environment. Typical use cases are:

• Guided document authoring, review and approval
• Coordinated contract creation, negotiation and execution
• Tracking of business events and issues through to resolution
  

In all of these cases the workflow runs purely to manage the tasks within a single collaborative workspace populated by a defined set of users -- it never ventures outside of those four walls, and from a systems standpoint is generally very introverted, never really talking to anyone outside either.

Most structured business processes require tasks to be performed both before and after the working life of a particular document and across multiple constituencies. They run as the result of some trigger, using and abusing varying resources along the way until an eventual outcome is reached somewhere down the line. This is way out of the scope of the four walls of a collaborative workspace and BPM rises to the challenge.

The thing is that in the life of every business process there comes a time when you just need ad-hoc user interaction. BPM doesn't really do a good job of this, having neither the tools nor the inclination to let people run free and do their own thing for an extended period of time. Collaboration has both the tools and the inclination, but it doesn't have the social skills to achieve it.

New Account Opening as an example

Lets take the example of new account opening. The extended end to end process is highly summarized as the block arrows at the top of this diagram:

Collaborating in BPM

The blocks below the main process show key interactions: data interchange with business partners and systems; exception handling; audits. The last two may be very unstructured.

For instance, the requirement to perform enhanced due diligence on certain high-risk customers requires extensive ad-hoc research and human knowledge to perform. Audits, well the randomness of them goes without saying, and in this scenario they may be performed on a sample of account applications that pass through the system.

As can be seen, there is a need for true collaboration capabilities at certain points (usually well defined) within a structured business process. The challenge is to get ad-hoc, introverted collaboration to play well with highly controlling, extroverted BPM (I'm sure someone could tell me the zodiac signs for this pair!).

Collaboration as a service

Rather than treating collaboration as a pure user application we need to treat it as a service. This provides us an approach for BPM to orchestrate a complete process including points that interact with a collaborative environment. At specific steps in the process, BPM can now call out to the collaboration service, allowing the ad-hoc work to be completed before carrying on with its structured duties.

What next?

I have seen plenty of use cases that could benefit from collaboration inside structured business processes. I haven't really seen a system that does it well, so if you have, let me know.

In the next post I will talk in a little more detail how I believe collaboration as a service could work to meet this requirement.

Technorati tags: Financial Services Technology New Account Opening BPM collaboration SOA

When software has too much stuff

It’s always a great surprise to me that the Sales Engineers that accompany Account Executives on software sales calls don’t just lose it and start throwing their laptops around the room.

Sales Engineering is a special art. Its a balance of impressing the customer with unique features against showing a deliverable amount of real software, while translating everything you know about your world to the little you know about the customer’s.

Why do you care? You’re probably not in the market for software, and sales doesn’t really affect the analysis and thought-leadership that some readers purvey. Or does it?

Sales, after all, is what keeps companies afloat. Software analysts and thought-leaders are voicing an opinion on where customers should be moving towards to implement better systems, being a mouthpiece for a ‘maybe’ market, and influencing customers to look at specific groups of vendors. Corporate architects are guiding what their companies will buy and deliver. Product Managers try and listen to everybody and mix it all with a little of ‘what the market wants now’, to get their product to be most successful. There are lessons from sales processes that we should all know.

Having done my share of sales support as a sales engineer, solutions architect and general gofer, I speak with a little authority. But I still have no idea how the really experienced SEs do it day in, day out. Sure, there is a buzz when a software demo goes well, everyone in the room bonds, and the customer’s head doesn’t stop nodding the whole way through. The problem is 90% if the time you have to deliver the impossible.

As ever the problem is with personas. These are what make the ‘buyer’, ‘user’ and ‘techie’ in the room who they are. Personas are the measurable attributes like role, education, age, computer skills. And they are qualitative attributes: risk tolerance, aggressiveness and the Geoffrey Moore psychographics of technology adoption.

Every organization has different personas for each role, though similar personas often cluster in particular industries and types of organization. Obvious differences appear in corporate, non-profit and government organizations. And if the personas stay in a particular organization for a while they absorb some of the surrounding personas from around them. Think of it as persona-perspiration – a length of time in a company is like a hot rush-hour on the Tube in London or the Subway in NYC. You get some of the surrounding sweat, anxiety and attitude of the surrounding people, just in a less condensed amount of time.

When a great Sales Engineer walks into a sales call with a new customer he or she is making constant assessments of the personas in there, balancing that against their roles. SEs use this to help them judge how to respond to specific questions, who to pay a little more attention to and when to be honest about a deficiency in the demo application. Even given all their experience and attempts to show the application in the best light using the language of the customer, there is a single issue that can derail the whole thing:

Incorrectly guessing how the unspoken goals of the business, and those of the individual personas, was translated to a set of RFI requirements.

When this happens you’ll hear:

"It looks like your application has far more in it than we need"

In my experience this really means:

“You should have guessed by now that I have no idea how I will convince my staff to use this, as they have rejected the last two systems I bought”

“I’m really hiding from you that I can barely use the Internet, so the slick collaboration tool you showed scares the hell out of me”

Being open and honest with vendors is the best way to get a demo that really addresses the business goals driving a buyer's requirements. This is important, as you shouldn’t be trying to judge software on how good a mind-reader the Sales Engineer was over the last 48 hours of preparation.

Of course, the issue still remains of the always unspoken goal:

“You won't guess that my goal is to just get the IBM/EMC/etc. guy to give me the software for free when I buy his big boxes of tin”

This is completely unavoidable, and acknowledged by everyone in the airport bar after the presentation and demo.

But if the problem really is that the software appears to have too much stuff in it, however well it was shown, buyers have a responsibility to do one of two things:

• Stop the selection process and pick more appropriate vendors (not from the big names that Gartner tells you), because they all have a similar level of capabilities,
  

OR,

• Get someone who can really envision how enterprise class software can help your organization exceed its goals

No software has too much stuff if it meets total cost of ownership the goals of the business. Only with good guidance from all directions can buyers really make sure they get software that really fits their organization.

Technorati tags: sales software goals

Dear Mr CIO - please fix my business processes

Over night Neil Richardson has written a nice little post about starting the BPM journey, which approaches the issues of selecting projects suitable for BPM in organizations that don't yet have a background with the technology. Neil talks about two classes of projects that may be used to introduce BPM to an organization:

(1) under the radar, low cost-low value, administration/HR style processes
(2) pick a strategic process involved in delivering high cost services to an important client

The watercooler discussion he relates is in contrast to my post yesterday that discussed how a seasoned BPM organization could struggle to extend the value of its BPMS beyond its original high-profile efficiency goals, into smaller risk mitigation projects. I suggested that the risk-management and admin based projects are a hard entry point for the traditional BPM or imaging and workflow vendors, since the usual ROI models no longer apply. So I don't believe that these vendors would typically be drawn to an organization requiring HR as a proving ground for the technologies, trying to demonstrate success on risk mitigation and audit issues.

This of course may be dependent on the organization. Back in a post a couple of weeks ago Thomas Otter talked a little about the internal HR systems at SAP. He highlighted these stats:

17 different languages
77 company codes
40 productive payrolls (32 different country versions)
55 countries
32,000 active master data
17,500 ESS users/month using absence ESS

In that type of environment maybe there is something about the scale of the operation that would be appealing to a BPM vendor and may allow them some creativity in building out a strong ROI. For example, the consolidation of numerous disparate systems and processes into a centralized instance provides benefits both in terms of IT management and Sarbanes-Oxley documentation and audit.

I still believe that it is unlikely that BPM would make it into this environment, despite the renewed ROI. The admin/HR/finance space is owned by the ERP vendors (and yes, according to Thomas, SAP do run their own systems). But there is still a play for BPM in this ERP owned ecosystem.

A BPMS provides the ideal mechanism for tying together a range of underlying systems with a consistent business process. For typical corporations with various ERP systems, abstracting the users from the underlying ERPs through a consistent BPM process enables consolidation at the level of business process. This simplifies usability, reduces SOX issues, etc. The corporation can delay ripping out the back end systems, an expensive and risky task, while still showing significant progress towards a more unified organization.

Maybe BPM can make it in the HR and financial departments. If so, it will need a business leader with a strong head for creating an ROI on more than efficiency and risk mitigation, but someone with a real feeling for the importance of unifying IT systems. This may be one business process improvement project that needs to be led by the CIO.

Technorati tags: Financial Services Technology New Account Opening BPM HR process+improvement

Justifying BPM with a risk background

A while ago, if you had asked me to justify the need for implementing a BPMS in an organization with manual, paper-based processes or those with a lot of manual data entry, I could have reeled off the high-level justifications. With a little research targeting a key business process, the help of some templates, a spreadsheet and a bit of time I could have built out a fairly extensive ROI to back up those claims.

What were the justifications? They varied dependent on the organization and the target audience, but typically revolved around:

• Improved efficiency, reduced manual processing time, ability to 'reallocate' resources
• Tracking of work, immediate response by customer services representatives
• Enhanced supervision of workers, management information, visibility of workloads and potential issues

BPMS is often sold to a business owner, a targeted person responsible for a particularly painful process in the organization. This is a foot in the door for many BPMS vendors. Since the vendor often can cite a range of references where they have implemented exactly this type of process the business owner can be fairly sure that the project will be a success, even though it may take a little longer than they were expecting. And typically the measurable ROI will be very favorable.

If the business owner was savvy enough to see the broader benefits of BPM in his organization, it is likely that an enterprise grade BPMS will have been selected rather than a point-solution targeting just his own business problems. This gives the business owner the chance to elevate his position in the organization, heading up a broader process improvement program. Hopefully another similar process will be picked, implementing BPMS given all of the experience just gained.

Very soon the organization runs out of cookie cutter processes that the new 'head of business improvement' has been able to rework to such great advantage. What does the head of process improvement do now? Well, he's a smart guy. He goes and looks for some other parts of the organization that have business processes. He has seen that Sarbanes-Oxley (SOX) illustrated business process issues left, right and center. The question is, can his BPM baby be applied to these examples?

First off he needs a new BPMS business case, since the Efficiency, Tracking and Supervision justification from before does not have the same appeal in these other parts of the organization. One was to justify BPM in parts of the organization that have processes, but have not built their business models around them, is to focus on risk. How does BPM help here? BPM can reduce risk by:

• Enforcing high risk, high value business processes mitigating risk of fraud by preventing manipulation of the process
• Reducing risk of error and inaccuracy in data
  
• Enhancing the auditability of processes

This are valiant goals, but the head of business improvement never quite sells the vision to the executive team. Why? Business process centric departments (think insurance claims, account applications) can measurably demonstrate ROI through efficiency improvements e.g. the ability to take on more business with less people. Financial and risk centric departments can not demonstrate that so effectively. So how do you recognize ROI when there is currently no way of quantifying risk, let alone measuring that it has been reduced, and quantify the value of that reduction in dollars?

While risk management remains a black-art in many organizations, the ability to introduce BPM technology beyond the processing centers is going to be limited. Maybe BPM really isn't valuable at an enterprise level. If anyone has experience of making this leap I'd be hugely interested in learning more.

Technorati tags: Financial Services Technology New Account Opening BPM risk+management

What to do when BPM is too much and email is not enough?

I was reminded of this little problem a couple of times today. Does anyone out there have a good response to this question?

When I need a business process driven application that has a large number of users and low complexity process, where do I turn?

BPM seems overly complex, while email doesn't provide me with the control and reporting I require. Is there a toolset that fills the gap?

Should I select a point-application from a small vendor, build it myself with open source, or just build on top of a BPMS?

Looking at it from a requirements point of view:

• Complexity of process: a few steps and simple decisions
  
• Change process: rarely needs changing
  
• Number of users: 5000 (20% concurrency)
  
• Volume of cases: 200 new cases per day
  
• User interface flexibility: important
• Reliability and availability of application: important
• Security model: hierarchical
• Document storage: small number attached to cases
• Metadata: 40 attributes, many being free-text entry fields
  

Some people have said that the process engine of BPM suites is the way to go, but this has implications in terms of cost and integration complexity. I have heard others offer that they would just build the application on top of a database and some enterprise portal tools. A further offer was to use an integrated document management and workflow tool. Someone even suggested using the Microsoft workflow (I have no experience of it, but instantly scoffed when I considered the scalability aspect of many users).

What would you do with a request like this when the business needs an answer and a proof of concept system in under 2 weeks?

Technorati tags: Financial Services Technology New Account Opening BPM workflow

Feed problem

A short administrative post...

It seems that I had somehow messed up the RSS/ATOM/Feedburner feeds for this blog, getting things all mixed up with another feed. If you have tried to subscribe to this blog with your reader of choice but failed, please try again.

Feed to your favorite reader

If you still have troubles, please email me at phil_ayres[at]hotmail[dot]com

Cheers

Phil

BPM modeling as easy as a spreadsheet - Part II

My post yesterday BPM modeling as easy as a spreadsheet drew a little well deserved critism from Bruce Silver in his summary post about Excel spreadsheets and process modeling. Although the reference to 'business people who do “modeling”' was tempting to make crude comments about, I avoided it as there was some important stuff to be said.

The random wordiness of my original post confused its intended message. I was hoping to convey how unfortunately many users can not learn BPM applications by refining their skills as they do with spreadsheets. I aslo tried to extend the discussion to how spreadsheets and business processes are even more closely linked in real companies than just a comparison of learning the 'application design tool'.

In the original post I referred to the business user with spreadsheets as 'Excel Queens' - the real world highly qualified accountants of the Finance Group of every large organization who use spreadsheets for everything. We have seen over and over that in these spreadsheets are real processes being represented in tabular form, not 'merely' complex financial analysis.

Addressing representation of financial processes in this form is important, since according to Sarbanes -Oxley the output of these spreadsheet users is some of the most important information that Wall St. and the average investor could consume. The unenforceable internal controls and processes represented within their Excel spreadsheets could land the CEO and CFO in jail.

Take an example: a typical financial quarter close in a medium sized company may be well over 150 tasks. It is an ideal candidate for process automation to reduce stress on the individual accountants and the Controller and produce more accurate financial reports. But in the majority of cases it is coordinated by blobs on a grid in Excel. Is this because it is not an important process, not run regularly enough, or just easier to manage manually? I believe it is because the finance group does not have the tools, time or skills at their disposal to improve the process they own.

So as I have admitted to before, in a previous life I tried to convince finance groups that there was value in automating spreadsheet processes. How? Imagine there is something like the software CMM model for internal controls and processes (click to expand):

On explaining that every organization starts on the left and the aim is to take the most complex, highest volume or most risky processes up to the right using workflow and integration tools there were nodding heads. But the audience had little time to concentrate on the problem at any level above managing their compliance documents in a repository (the 2nd level). The best they could do was migrate a bunch of spreadsheets that represented the documentation of the whole organization's SOX internal controls and processes to a document management system with a compliance skin on it: Certus, OpenPages, Paisley, Movaris,...

Imagine how much more effective the organization could be if the business owners took some of their processes and actually automated them. And according to the intention of the original discussion, this is what Bruce et al were proposing. That business users should be able to pick up BPM modeling tools, model their processes, and have them seamlessly generate executable processes that coordinate themselves and other business users, with little or no IT interaction.

My original post seemed to imply that any business user stood around the watercooler uses Excel and can therefore use BPM tools and produce executable processes that their organizations rely on. And I suggested that this is dangerous.

Bruce (rightly so on rereading my original post) contested that I was suggesting that users should not be able to produce processes from what they model. My response was this:

[...] It is my understanding that ‘modeling’ is a skill that not every business user will have, but once acquired should be fairly transferable to any particular BPM modeling tool. A customer services supervisor (for example) who has learnt to model the processes she owns can probably pick up many modeling tools. But she does not have to have the modeling skills to do her day to day job.

On the other hand, this customer services supervisor probably did not have to learn a completely new set of skills to start (or need) to use a spreadsheet. She picked it up one day to help her find the mean call length for each of her staff.

Business modeling is a skill that not every business owner needs, or can in fact perform, despite the evident need to improve their processes. Only those with the mental aptitude to model processes and improve them will be able to pick up the tools, model and produce executable processes. The barrier to entry is higher than that of a spreadsheet, however good the vendors make the tools.

So if you pick the right business owners that can model processes I absolutely agree that they can build executable processes. Those who can’t model processes may still have the skills to run their teams very effectively, so perhaps should be supported by dedicated process analysts.

Relating this back to the finance group, there are many senior finance people that run US public companies amazingly well. But I suspect that they do not have the process analysis and modeling skills that would allow them to model their processes, and therefore implement them as executable processes using any tool.

Maybe given their other skills (and commitments) they need a little help from a professional process analyst to get some of their most valuable processes out of the spreadsheet world and into BPM.

Technorati tags: Financial Services Technology New Account Opening SOX BPM

BPM modeling as easy as a spreadsheet

It seems that spreadsheets are coming to everyone's mind at the moment. I used them as an illustration in a post about the potential use of lightweight integration(mashups) in corporate environments. Many business processes are run on Excel, which reflects a need in the organization for business users to create 'applications'.

David Ogren (and almost simultaneously it seems Keith Swenson at Fujitsu) put a new twist on the use of the spreadsheet, comparing their uptake to how BPM usage could evolve over time. Although the initial reason for Excel use is simplicity, the ability for a user to grow their skills and therefore refine a long running spreadsheet really provides the continuous improvement that many businesses seek in their processes.

In the future David, and commentators like Jesper Joergensen at BEA see that the much of the value of BPM comes from the ability for business users to model and refine their processes as their skills improve, much the same as they would refine a spreadsheet. The value that comes from the actual automation of the process by the process engine is assumed - any vendor can provide a scalable workflow engine that pushes cases around according to a business user's process design, right? The important piece and the essence of BPM is putting a design tool in front of the business user.

At the same time David correctly notes that

No one wants a compliance spreadsheet that works most of the the time or a order tracking BPM process that works most of the time.

This is a key point and implicitly hits why spreadsheets and BPM are different. In the past I have attempted to evangelize to customers why workflow tools that can rapidly deploy business processes should be used to replace many of the spreadsheets that pretend to provide internal control in the organization. Although this got nodding heads, there was little uptake in the idea.

My opposition to spreadsheets was this: they rarely have the strong IT review, versioning and deployment processes attached to them that typical applications do, nor can they really enforce very much when used as task lists for business processes.

In their defense it is likely though that the spreadsheets do not get used repeatably - by this I mean that the process they represent does not get run multiple times per day. In the scenario where a spreadsheet is being used to represent a business process and assist in decision making it is exactly this flexibility and low volume usage that sets it apart from BPM - the user needs and values the flexibility to change the spreadsheet 'mid-process', and is there to sanity check the result.

I agree with Keith and David that business users will refine their skills over time to enable them to implement more of the deeper functionality that BPM offers with less assistance from IT. And this is where I believe the similarity with spreadsheets should end. As I discussed in Right tool for the job BPM is ideally suited to business processes that do not need to change too often or those where the process needs to be documented and audited for SOX purposes (meaning that it must not change). Applying BPM to replace some of the spreadsheets representing processes in an organization, as I previously touted, may in some scenarios provide significant value and control.

Unfortunately in other scenarios BPM will appear so restrictive that the business user will not bother to refine his or her skills to solve the problem as suggested above, but will just abandon use of the tool for Excel, something familiar and flexible. It may take some time for business users to get to a point where they can select when and when not to use BPM.

Technorati tags: Financial Services Technology New Account Opening spreadsheet BPM

A real (business) problem for SOA and BPM

Core financial business processes like new account opening rely on SOA and BPM to be able to provide real improvement.

Point software 'solutions' pick off components of the problem, but the real need is for a complete end to end process. Although this can be implemented without employing the rigorous frameworks offered by BPM and SOA, the result may be a solution that is hard to change and expensive to maintain, much like the legacy systems of last century.

How does this relate to New Account Opening?

In this post I'd like to explain some of the concepts of SOA and BPM and how they relate to a real process like new account opening, to help readers picture how these technical worlds look when applied to a real business problem, and how to approach using the SOA and BPM frameworks in combination.

Through this discussion I'm going to use new account opening for annuities as an example, or at least the early presales stage of the process. Here is a high-level flowchart of the way the process typically runs (click to enlarge):

In more detail, this is what is actually happening:

1. Prospective customer chooses to by an annuity and with the help of a Producer (basically a sales agent) fills out an account application
2. The Producer uses this application information to create a Master Account for the customer, more as a holding place for additional information than anything else
3. A range of consent, profile, authentication and anti-money laundering information is collected and stored as electronic records (preferably supported by e-signature, but unlikely in the immediate future)
4. At this point the Producer continues on with the customer facing sales process, helping the customer select the appropriate product.
5. Behind the scenes the Broker / Dealer is responsible for using the information collected about the customer in (3) to ensure the suitability of the customer to the investment products being offered by the Producer in (4), based on product attributes coming from the Insurance Carrier
   
6. The Broker / Dealer is also responsible for ensuring that the Producer is licensed and authorized to offer the combined securities and insurance products to the customer (under the particular federal and state regulations)
7. Based on an appropriate product being selected by the customer, certain disclosures may be required, prior to the whole process moving to the next stage of validation, review and account setup

Complex processes make it all worthwhile

The new account opening process for annuities is complex due to the many interactions between the business partners (Producer, Broker / Dealer, Carrier) and customer, spanning potentially several weeks. Each of these constituents in the overall process has their own processes that must be followed, while accessing, creating and updating information in a range of business applications.

It is this combination of business process, information sharing and interaction with business applications that make this a decent illustration of the application of BPM and SOA.

What comes first? SOA or BPM?

Technology practitioners argue about whether SOA or BPM should be addressed first when architecting an improved business process. This is one of many examples of a top-down or bottom-up (business first or technology first) argument which seem to iterate indefinitely. The following sections provide examples that illustrate what I believe to be best practices.

SOA is not just integration

At one level SOA is about providing integrations with business applications - but doing it in a way that allows reuse across multiple parts of the business while minimizing the impact that a change to a system may have on all of the interconnected pieces. True SOA can be viewed as a way of exposing the valuable functions that business applications and legacy systems do as a set of meaningful services. These services are required by typically more than one task, user or other system, making the effort to expose them worthwhile.

You have a process, now see what systems you have

In new account opening for annuities, the Broker / Dealer can really benefit from SOA and BPM, since much of the process falls in their camp.

The original process flowchart for the Broker / Dealer doesn't explicity show all of the systems touched in this stage of the process, instead identifying the key tasks that have to be performed, either by a system or human operation. In a picture, it might look like this

The indivual systems boxes in pink don't directly relate to the required business tasks to be performed. This gives us is a decent reason to put in place a layer of abstraction, relating the business process depicted in the original flow chart to the backend systems. This enables the business analyst to avoid having to put a huge amount of system specific stuff into the business process diagram, retaining its value to the business, and making it more manageable long-term.

Abstract with Services

Since something has to go between the process tasks and the available (or maybe new) systems, we need to approach this from either top-down (business process defining what the technology should do) or bottom-up (available technology defining how the business process should run).

I would suggest that allowing the business process to drive the technology is the best approach to start with, recognizing that in the real world the legacy systems may just be too stubborn to allow this to work. Starting from top down then , we can look at integration as a set of services required to complete the tasks, rather than the technology approach that says 'here is a legacy system, here are all its functions, let's expose them as web-services'.

In this example, this is how some of the key services based on the business tasks could look:

To reiterate, for me the top-down approach is the first step in SOA: identify what I need to do, not everything I can do already. If it happens later on that I find out that its going to cost an arm and a leg to perform some of these tasks in their ideal form, then I can revert to a more pragmatic technology-based approach.

Services hide system complexity

Mapping the required services back to the available systems already in place we see the picture progress:


This helps to illustrate three points:

1. A service can touch multiple backend systems to achieve what it claims to do
2. Each capability offered by a service may perform multiple lower level functions against a single system
3. The complexity of the backend systems presented back to the business process is greatly reduced, enabling a business analyst to identify services that exist already, allowing him to reuse them

The value of SOA alongside BPM

The final point reinforces the idea of SOA. Only if the services are identifiable as valuable business functions will a business analyst be able to see what exists already and be able to reuse them. And it is reuse of services that makes SOA valuable beyond just being a technical abstraction layer between process and systems.

Using our sample process and services lets imagine the scenario where a current customer goes back to her sales agent (the Producer) and requests to buy an additional annuity. The process here is different - this is not a New Account Opening process, but the suitability components still have to be tested to ensure the products offered meet the known customer profile. In this scenario we see that this Add Product to Customer Portfolio process can make use of the same Suitability services as used by the New Account Opening process.

Now it is likely that the IT/SOA team does not have to make changes to the service or the backend systems to provide suitability to the new Add Product process. The business analyst defining the Add Product process can easily plug in the suitability service to the process.

If the services had been defined bottom-up from a more granular technology level, this would require the business analyst to be able to identify and assemble multiple technical functions that are really foreign to the business process to be able to complete the real task.

Summary

Complex business processes need BPM with all of its related tools for definition, coordination and management of change. SOA provides a framework for architecting valuable, reusable services to support the business process interaction with backend systems.

If the IT/SOA team build services based on technology functions alone, they produce a series of low-level web-services that are probably meaningless to business processes without further assembly of the right parts in the right order. Only by looking at services from a business level, rather than a technology function level can the services match the requirements of multiple business process, enabling a business analyst to identify and reuse them.

Despite believing this 'business first' approach to be best-practice, I will hedge my bets by saying that there is always room for pragmatism. Recognizing those systems and services that will be reused is the first step in setting priorities for perfect business services. The second step is the ability of the systems to be adapted to provide this. I have seen legacy systems that just would not allow simple definition of discrete services. Here the technology won out over the ideals of BPM and SOA.

Recognizing how to adapt BPM to embrace difficult systems is something I will talk about in a future post.

Some References

The thought-leaders from the BPM and SOA worlds have been visibly discussing the relationship between the two 'frameworks' in the last few days (and probably behind the scenes for much longer). Bruce Silver on BPMS Watch has provided good critical overview, and as he says Jesper Joergensen has written a good description of the relationship of BPM and SOA, effectively distilling some of the deep discussions in the SOA thread on Yahoo Groups. Steve Jones has also written a nice post illustrating why many customers probably mess up SOA, even under the expert guidance of an SI.

Updates:

Thanks to Rob Hill at Vignette for pointing out that I had incorrectly labelled the first flowchart, with Consumer instead of Carrier. This has been corrected

Also check out Steve Jones' detailed contribution to OASIS proposing a methodology for service architectures. This approaches services from the business direction, and correctly takes it a level higher than an individual business process as the starting point for enterprise SOA.

Technorati tags: Financial Services Technology New Account Opening SOA BPM reuse

Dear Mr CIO - don't hold off buying ECM

For the last few months Microsoft has been rattling ECM vendors with the hints and announcements of a range of capabilities that intrude into the ECM infrastructure space. Many of these vendors have made public their plans to embrace Microsoft building on the new capabilities, while others are keeping quiet.

How does this relate to New Account Opening?

As I wrote in my post Vista in 2007 v. improved New Account Opening now the financial services CIO has some tough decisions to make whether: to improve ailing processes now, or wait to see what comes out of the new Microsoft driven ECM world.

With some insight into how vendors and technology will look when the dust settles the CIO can hopefully continue on a progressive technology timeline and not just 'wait and see'.

So how are the ECM vendors going to respond and how will technology look.

Evolve...

For some vendors the new Microsoft technology will give them the chance to concentrate less on what they consider low-level infrastructure capabilities: records management, workflow and web content management. Instead they plan to build value on top of these features, despite that in some cases this represents serious competition to their core products.

I would suggest that for low- to mid-tier ECM vendors this may make sense. There is no competing with Microsoft at this level, especially for the SMB or departmental market, as prices will plummet. So take the opportunity to stop building valuable but expensive infrastructure and instead focus on one of two things: adding unique features and usability on top of MS; or select a business problem that you have expertise in solving and build out an out of the box offering to address it.

In the past vendors have been reasonably successful at both approaches, but unless you can imagine and market the next killer-app, the latter may be easier to sell. As a prime example, look at OpenPages - previously an attempt at getting into the document management had limited success, so molding the product to a specific business problem (governance and compliance, specifically SOX) put them into a focused and driven market segment.

Embrace...

For the mid- to upper-tier vendors the MS technology presents a challenge, as it appears that some of their core enterprise level technologies will be commoditized.

Some vendors (e.g. FileNet) have chosen to publicly embrace MS, although you wonder what they are thinking behind the scenes. They could believe that MS is not yet convincing enough in the traditional market of high-volume imaging that they can survive.

Other vendors seemed to jump right into bed with MS. For example OpenText with its announcement of almost embedding the new MS platform into their offerings, and its approach to acquire Hummingbird, a traditional MS mid-tier player. Not surprisingly though they are reserving the records management / information lifecycle management for themselves.

How are the big ECM vendors going to survive?

Not all doom and gloom (for the vendors)

As a CIO you can guarantee that large true enterprise ECM vendors will be innovative to ensure they maintain their prices and market share - both in technology and marketing.

The big ECM vendors have some options:

1. Extend and enhance in-house technology with the MS capabilities
   
2. Build business solutions on top MS and in-house technology
3. Embrace the user facing components, like Office 2007 with strong integrations and value-adds
4. Identify and enhance enterprise technology requirements that MS can not easily deploy
5. Get acquired by Sun (assuming you have a Java technology base) - hmmm maybe not, aren't Sun getting out of software? EMC already have Documentum. Who is left? HP?
   
6. Run away and hide...
   

OK maybe the last two aren't good strategies! But the first four, probably in combination offer some strong possibilities. Over the next few days I'd like to discuss these options, to help financial services organizations understand the real effects of the MS technologies.

Technorati tags: Financial Services Technology New Account Opening ECM Microsoft Records+Management

Is there a place for 'lightweight' integration?

Integration and workflow (aka SOA and BPM) are core components of new account opening. New customer applications need to be accepted, validated through multiple steps, have the data passed to and from backend systems, and eventually fulfilled as an account and welcome package for the customer.

SOA and BPM offer formal frameworks for pulling the whole end to end process together. But what if your process could do with a bit of improvement here and now, while standards (both technical and business e.g. NAVA) and your IT team get themselves to a level of completeness and competence that enables an even partial shot at providing straight-through processing?

In the world of Web 2.0, lightweight integration of online applications, services and components is termed a mashup. It often represents a small component of a larger system being incorporated to provide a distinct service, and is often performed rapidly with scripting on the front-end with zero coding on either system involved.

I believe that mashup is a great term that could be applied to an Enterprise Portal for your company's intranet applications. You know the things, where you can look up a colleague's phone number in the same screen as entering your vacation time and getting at the company's latest and greatest marketing materials. Its visual web technology in action, so it loosely couples all of the pieces successfully.

Unfortunately my time (12 months) looking at the Sarbanes-Oxley problem in US companies made me realize how much of a benefit 'agile aggregation and deployment' of applications (a mashup by another name) could be. Many companies are running their most complex compliance processes (like financial year close) on spreadsheets stored on the P: drive and distributed by email. And in these companies most of the finance department is run on Excel based data and macros, roughly put together by an Excel Queen (aka Accountant) with zero testing or management.

Given this, imagine what a benefit a semi-seamless combination of basic applications could give these finance professionals for coordination and sharing of work, tasks and data.

Its not ideal, and I wouldn't trust my paycheck to make it into my account if Oracle Financials was replaced with a mashup of the next new web accessible DB, cobbled together with the ADP outsourced check-cutter.

There are many thought-leaders in SOA, BPM and Web 2.0 that argue both for and against mashups in 'real' organizations and processes. For example, Steve Jones in his Service Architecture (SOA) blog strongly resists the idea. The programmableweb site has a different view.

For those processes and parts of the organization that have little structure anyway, or still run paper processes (for example new account opening), maybe a mashup approach could cut the mustard, enabling some level of online coordination and tracking to be put in front of over-stretched users, fast.

Its a highly flawed idea, but one that I bet many organizations do now, or are likely to follow in the near future, just by a more corporate friendly name.

[UPDATE] I forgot to mention Neil Richardson and his Little by Little blog where he talked about risk management. His post actually gave me a reason to link this topic back to compliance issues rather than just talking generally about SOA and mashups. Nice one Neil!

Technorati tags: Financial Services Technology New Account Opening SOA BPM mashup