When documents and processes come together

I'll admit that the coming-together of processes and documents (BPM+ECM) has been a common theme of this blog for the four years that I've been spouting-off here. My background was imaging and workflow, then I got trapped along the way in understanding records management and even more trapped when process improvement became BPM suites, and more about the process than the stuff in it. I can not avoid feeling that I've heard this all before, but still wasn't satisfied.

Yesterday, a forum on ebizQ brought together some of the opinions about the question "Is it inevitable that BPM and ECM will be integrated into one system in the future?". For companies running both of these technologies in house, the answer is probably a no-brainer. ECM and BPM are already integrated, because the customer did it themselves. For vendors, nobody has worked out really if they can make money from it.

I've included some quotes from the forum that I think sum up the range of opinions, though its worth reading through the whole thing if this topic interests you:

Ian Gotts of Nimus: "The two are inextricably linked already. If I want to perform some task such as sign off a PO then I probably need to access the Purchase Policy document which will be part of ECM."

Malcolm Ross of Appian: "There's a definite need to unite the features of these two worlds to create more powerful content and process systems."

Doug Mow of Virtusa: "From the customer, patient, subscriber or end user perspective I want a seamless UI that doesn't distinguish between any class of functions. "

Brian Reale of ProcessMaker: "It is one thing to route a document or a PO, add a signature, and then store it in a DMS. It is another thing to start a process in a rich contextual environment and have that environment intelligently populate with the pertinent information based on attributes like user, place, time, and company policies. "

Peter Evans-Greenwood: What's a document/record anyway? A page in a wiki? An email? SMS? A tweet? Defining the scope of BPM and ECM as processes and documents ignores the ongoing erosion of boundaries between organisations and communications channels, and existing solutions are too intrusive to push into these new channels. 

I'll quote my own response in full:

The short answer is this: in any business that uses BPM in critical applications, BPM is already integrated with ECM. I think it unlikely that we'll see anyone but IBM and maybe EMC really producing a credible E-BP-CM suite. 

Why do I believe this? Well, its important to ask why ECM is so important to businesses... Its not the collaborative-Sharepoint-less, "let's be happy and work together" stuff. Its the fact that in real business, at the end of everything we do in a business process there needs to a record of the transaction, the decisions or whatever that formed the outcome. Of course, that does not need ECM if the data we were work with at every step was fully structured (an online form producing structured database records). But for the real world, documents do exist, and ECM provides a way to handle them. 

As Doug says we must interact with documents, and as the vendors hinted, document management ain't that hard for BPM vendors. The problem is that throughout a process, documents will form part of the final business record. BPM vendors rarely manage to focus on understanding the complexities of electronic document and records management, therefore they rarely do much more than routing documents -- plain old-fashioned 'imaging and workflow'. 

What is really needed from BPM is a final step: registering the records and the decisions of the process in the corporate records management system, keeping the context of everything that was done. Working with documents and process context appears in some of the marketing around case management. Though more often than not people get more excited about how processes can change dynamically, than how they can be recorded permanently. So I expect adaptive case management (ACM) to be another missed opportunity. 

It really is a shame that records management has such a 'stodgy' appearance, since it is such an important part of business processes.

This is one of those tough areas where the integration of technology for vendor gain often eclipses the needs of the end customer. Careful consideration is required on what is needed for any particular business, and navigating the options depends largely on experience.

A post from the Improving It blog

Let us help you improve your business today. Visit www.consected.com

Cases crystallize out of chaos

Coming via the bp3 blog by Scott Francis, Frank Micheal Kraft talks about Patterns of Knowledge Work. His discussion hits the point that many activities performed that are not 'production line' processes are really based on the activities of a knowledge worker: a person who is knowledgeable about what they do, and builds up structures around their work to keep stuff organized.

This ties into my discussion a week or two back that in my own client work, I've started to see how I can examine the records of business activities, the documents and paper reports, and track back through them. In doing so I build up a view of where the major business processes in an organization live, who does them, and at a high level at least, what the activities performed are. Amazingly, the often overlooked org chart helps me in seeing order in the chaos. It is interesting to me to see that there is a natural order to processes and information in business.

For Kraft, he talks about the three ideas he has come across:

1. 
   

   Managing my own knowledge work. As I wrote my own Adaptive Case Management system for my own knowledge work, I was able to organize my own work. As the number of cases increase – 3000 now including sub-cases – I become aware of patterns.

2. 
   

   Feedback from my first pilot. This was very interesting, because the main focus for my pilot is usability. Usability is strongly interwoven with these patterns of knowledge work.

3. 
   

   The things I always wanted to model, but never was able to. I governed the modeling of thousands of models of structured processes from all areas of business processes. But because the modeling language was only able to model predictable processes, I never was able to model unpredictable processes.

In my view, he has done with these three points what many business people have had to do -- build a mechanism for structuring his own work in a way that allows him to be productive and get value out of work he has done before. For many people, this is a matter of building a filing system, putting together checklists and tasklists, maybe even putting together an Access database recording people or indexing their files. Oh that, and never going on vacation, as nobody else can quickly pick up the structure and work with it.

Kraft has written his own Adaptive Case Management system, probably because he felt he could put some structure around the work he was doing, and that structure was adaptable enough to help others in their own work. With my own consulting and software work I have done a similar thing. And as Kraft says: "As with all knowledge work the result of my effort is not completely predictable. But I am making good progress.". 

I couldn't agree more. And with every new client or process I hit, more structure and flexibility crystallizes into something that benefits the next client down the line.

A post from the Improving It blog

Let us help you improve your business today. Visit www.consected.com

'I hate working with documents on-screen', and other issues we reinforce

You've heard the phrase "Knowledge is Power". There seems to be a human character trait that says by keeping documents close at hand, preferably within steps of my office chair, I am more powerful. Filing cabinets for individuals, and "working copies" of client files are everywhere in offices. Take that easy access to information away and people fight back. You are removing some of their ability to hoard information, all of which is a duplicate of something available elsewhere, but it exposes a frailty that leaves them uncomfortable. "What if somebody else has the file the one time in a million I actually need it?", or, "What if I'm caught off-guard, a client calls and I don't have their information to hand?". This is just one of the ways employees resist the introduction of enterprise content management (ECM), case management and business process management (BPM) solutions that try to limit the amount of paper moving and filed in the organization.

I have had the 'exciting' opportunity to spend some time working with lawyers over the last couple of years. From helping smooth immigration paperwork to cleaning up company contracts, it doesn't seem to matter who I work with I experience the same thing: paper. When it comes to the workings of a law office, I am frustrated by the apparent waste of paper (and the subsequent charges for photocopying applied to my account). When I think about it though, the issue is obvious with a lawyer because the multiple copies of documents happens typically right in front of you, while you're sitting there in the office. Although not so obvious, regular offices experience the same issue. Most of us I'm sure have seen the pervasive footer on emails, "Do you really need to print this email?". There is no way to know how much paper is wasted on unnecessary printing and copying in offices, because it is not charged per page.

For the 14 years I've been working with electronic imaging technologies, I've always encountered the concern that it is hard to work with text on a screen, but early on companies worked hard to limit the concerns of employees. In the UK at least, as a user of a screen and keyboard, an employer was required to ensure your working area was set up to avoid discomfort and to improve your posture. Companies investing in document imaging put in place minimum specifications for screens, and usability requirements for applications to make it easier and more productive for people to work without paper. These lucky people of the 90's working with electronic documents had it good.

Now when I visit clients, it is not unusual to see LCD monitors in every cube. Screen technology has progressed to a level where great quality should be available on every desk, but we have instilled the concept in the heads of employees that it is hard to work with documents on screen.  Why, beyond an aversion to change, does this persist?

I think that the problem comes not from the screen, but from the documents. Many regular people think of Word documents when reading on screen. Word is an editor, not a document viewer, and it does a terrible job of making documents easy to read. People remember this. Adobe Viewer for PDFs presents nicely prepared documents beautifully. But so often we end up reading marketing brochures prepared in multiple columns for glossy printing, needing to scroll around with every paragraph read, that we learn to hate it. The same with the trend to scan documents to PDF. The viewer does a generally poor job of making scanned documents appear attractive and clean.

My feeling is that companies wanting to become more paperless need to concentrate on the underlying issues of the documents they try and have people use. If your scanned documents look ugly on screen, people won't want to use them. If you force people to use daily reports by scrolling around left, right, up and down, they complain of RSI and general reduced performance. If you archive Word documents for constant reference and record-keeping you're asking for trouble in so many ways. As an aside, I saw a Word document archived in a large government agency that was unreadable, because the pretty font used by individuals was no longer available. This applied to thousands of documents.

Maybe it is time for companies to focus on how they can make the information that gives people the power to make good decisions not only available, but easy to use. This isn't just document scanning. Good application design and recognizing that people want to organize information their way will help people work better. Companies need to focus their investments in document management, business process management, and case management on making users actually want to use the new applications, since this desire to use the application is so important to getting paper off everybody's desk.

A post from the Improving It blog

Pigeon-holes are for the successful

Unfortunately this 'fun' discussion wasn't as a result of something I blogged, but its worth a look. According to Adam Deane:

My view is that Case Management should be implemented by ECM vendors, not BPM vendors.
Case Management revolves around data, documents and data therefore should be dealt by ECM professionals.
ECM requires a different set of skills than BPM.
It would be in the customer’s best interests to have separate systems for BPM and ECM.

I personally believe that it doesn't matter where Case Management sits. Its the business value that this type of solution can bring to businesses that is important. And until my business, or somebody else's is as synonymous with the that specific category of solution as SAP is with ERP, we all need to just accept that the industry is a bunch of pigeon-holes.

Follow along with the comments on Adam Deane's blog. Its getting fun...

A post from the Improving It blog

Reporting processes - scary stuff

My excuse for my recent blogging hiatus is that I was kicking off a project (or three) with a new client - a securities firm in Vancouver. Rather than spend time expelling ideas, it was time for me to sit, listen and learn. It seems that deluge of incoming information in one direction stopped too much coming back out in the form of this blog. So now that we are all refreshed, its time for me to restart the outpourings - with a thought or two on 'reports'. Hmm, interesting stuff, all those numbers printed on pages of paper - I can't wait to hear (you might be saying to yourself with a forced smile). Hold on though...

In previous lives I have worked directly with electronic reports management tools, either selling, implementing or defining a strategy for the product. Although essential in an ECM vendor's stack, the tools rarely got much attention despite the obvious benefits they could offer customers:

1. Reduce costs printing and storing paper reports
2. Simplify the distribution of reports to consumers of the information
3. Secure notes and reviews of the reports, with a far more effective audit trail

These are just three of the many advantages that can be seen from diverting the flow of reports from real printers to a virtual printing environment that generates an on-screen representation of the original data. There are many other clever features of electronic reports management that allow the reports to be split up automatically based on text on certain pages, and indexed so that a quick database search could pull a specific piece of information (such as a customer statement) from the thousand upon thousands of pages collected each print run over the years (you've seen this with the pretty PDFs of your credit card statement online). But what happens when the report is just that - a report?

As I have been seeing on my travels, the reports generated for back office operations are often not a print-stream from a mainframe that really represents documents such as statements going to customers. They actually live up to their name and are just what they claim to be - reports: a point in time snapshot of data. Examples are: the commissions owed to your agents; the trades made in the previous 24 hours; the status of credit on customer accounts for the last week. From an audit or regulatory standpoint, these snapshots show you are paying attention to your business in a way that a dynamic query of your database can not (things change in databases, therefore generating a realistic point-in-time snapshot is often hard, ineffective or inaccurate).

Storing these reports is easy enough. Replicating what some of them are used for in the business is harder. As I have seen, it takes a real technical accountant to review a securities trading report, ensuring that there are not exceptions or compliance issues. The markups these professionals make in the review process are hieroglyphics to the untrained eye, and essential evidence to the regulators. This is something that traditional reports management software would struggle to handle - allowing someone to rapidly and naturally mark up a report in 5 hours over the course of a day.

Why do I tell you this? Because the obvious technology approaches do not fit every requirement once you really take a look at the issues. Reports management tools will not solve all the issues with real reports. I don't have an answer today. One proposal is to implement more automated solutions for spotting exceptions and helping the technical accountants work more on those exceptions than the 99.9% of things that aren't issues. But that's a challenge for somebody else. For the next week or two, those reports are going to get printed, marked up and stored with paper-cuts included.

My excuse for a blogging hiatus is over. The one way valve is broken, and information is ready to flow both ways again. Watch out!

A post from the Improving It blog

Classification of documents and processes

"Everybody focuses on document classification, but nobody ever talks about how to classify business processes. WTF?" - the considerably sanitized question I received from a client this week. Its an interesting one that maybe I will change to "How should a business align the classification of its business processes with the documents and records generated and captured within those processes?".

Back a few years, when this blog was new(er) and I had the energy to be putting some interesting thinking into writing posts with uncatchy titles, I wrote about the converging classification schemes of documents and records. This discussed how the formal taxonomies and fileplans of corporate records management might start to align in a meaningful way with the full-text indexing (Google-style search), tagging (at the time Technorati, now every blogging and bookmarking tool out there), relational databases 'indexes or titles' and a bunch of other ideas for identifying information. The conclusion was that they need to, since all these approaches have value. The cost of doing so for a business can be enormous, since adding this information can be labor-intensive.

Now add the question from my friendly client, and not only do you need to work out the meaning of corporate information, you may also need to classify the business process that produced it. The question is, if I really do go about filing documents in a structured filing system, should that be based on the content of the document, the way we need to retain and eventually dispose of the documents, the business process that produced it, the arrangement of processes documented within a COBIT structure for SOX, or some other classification that only a highly paid expert could imagine?

There are many arguments for many different approaches. We already have file plans for organizations that need to manage their formal records - the EPA for example talks about its file plan in decent depth, and how mere mortals can build on the fundamental structure within their part of the business. The focus with the EPA's guidance is to organize the records into something that allows the information to be found in the future, the security requirements and the need to archive long-term or destroy the documents according to a schedule. I don't see much business process in this.

In my opinion, a well thought out plan for storing your documents and business records automatically includes the business process that generated the information, but only if it is valuable to do so. A simplistic example based on an insurance company, could put records into this type of classification:

-> Line of business
-----> Customer
----------> Policy
--------------> Underwriting
------------------> Quote
------------------> Policy generation
------------------> Renewal
--------------> Claims
------------------> Assessment
------------------> Subrogation
--------------> Customer service

and so on...

What I see here is that at the lowest level of granularity, the business process generating or capturing the information is represented. Typically each of these items is represented by an end-to-end business process, automated or not. The details of work within the process is identifiable within these buckets (according to its own indexing scheme) and the documents must be meaningfully linked into the same work, while being available standalone. This last part is important in my opinion - the old imaging and workflow systems of years ago often held the documents within the 'work item'. If the workflow ended, the documents disappeared. Or you had to artificially tack the retention and disposition of records onto the actual business process. You could never attach more than one process to a piece of information either.

However you build out a new records structure for a business, of whatever size, the processes you run become part of it. In my opinion this can help you get to a seamless transition between what is just a written structure, and something that is actually useful for identify and finding information scattered across the business.

When it comes to technology to help simplify some of this, it makes it easier to identify if you need electronic records management, document management, report archiving, business process management, workflow or one of the many acronyms that make up a complex technology stack. Decide on the focus that helps your business work better and then ensure that fits into your information management requirements. If you try the reverse, you may not have a business worth keeping records for.

A post from the Improving It blog

To implement workflow and process automation in your business today, visit www.consected.com

Processes in the cloud

Amazon, in its new incarnation as cloud computing provider, has announced that its EC2 service will have the ability to run Microsoft Windows Server or SQL Server before the end of the year. Why does this matter?

Companies are under pressure to deliver applications for a better total cost of ownership than ever. This isn't just a matter of cheaper software and less sys admins to support it. Already we see the importance of virtualization, helping reduce the cost and increase the flexibility of corporate server rooms, at least for the products that certify themselves to run under products such as VMWare. Side this with the new 'green' push of Intel, AMD, Sun, etc - to show a reduced cost of electricity powering and cooling the masses of servers that are still required, and the complexity of organizing server rooms to do so. According to Sun, 25% of IT budgets is consumed by energy costs.

So why not just save the valuable office space that server rooms have expanded to overtake, the power costs, complex network wiring, and the cost and risk of knowing how to, and actually doing this infrastructure stuff well? Just deploy your applications to the 'cloud', make sure you have a powerful and fault-tolerant Internet connection, and away you go.

Does this work for your critical business processes, perhaps run by a business process management (BPM), enterprise content management (ECM) or traditional imaging and workflow solution? AIIM talks about SaaS for ECM, and adds some nice commentary on the key tests for an organization selecting a SaaS solution: does SaaS meet the tests of speed, functionality, cost, flexibility and suitability?

Since BPM should be about running your differentiated processes, the cookie cutter approach to cost effective SaaS solutions may not be appealing. But when you have the ability to build exactly your solution and run it in the cloud on a common Windows operating system and database, BPM might become viable without complex infrastructure requirements (at least those that your boss can see).

A post from the Improving New Account Opening blog