Wednesday, March 31, 2010

Once upon a time we called it 'collaboration'

There has been a big rise by the business process management (BPM) software vendors to get in on the 'social' game. As people become more comfortable with using Twitter and Facebook than they do Word and Excel, a new tag has appeared: Social BPM. My argument is that, once upon a time this was called 'collaboration' and the big software vendors (Documentum/EMC, Vignette, etc) did it.

Collaboration software was about helping teams of people, typically on projects or with a need to share information to get a job done, to set up 'collaboration spaces' or 'team rooms'. There was not a lot of process enforcement, because the workflow capabilities of the products were limited and the aim was to focus on ad-hoc collaboration of people, rather than strict process flows.

This lack of process management was probably a shortcoming, since companies grew able to share information online easily, but still struggled to put some structure around the work that was done. And maybe collaboration's demise was due to the name 'collaboration'. A guy I worked with at Vignette, Casey Conner wrote a book with Christine Lambden called the Consulting Stance. It contains many thoughts about how consultants communicate, and if I have remembered well, there is a story about how a consultant failed for a day to communicate with a team in Europe about the value of collaboration - a 'collaborator' still carries negative connotations for many Europeans brought up around the time of the World War II.

Even if collaboration software just died because its marketing showed limiting insight, and its technology missed the mark a little on what businesses really needed to improve the way they work. Social BPM is a land-grab by BPM vendors to help them reclaim some flexibility in their otherwise fixed, rigid product sets. Companies wanting to improve should also search for solutions offering other capabilities such as workflow management, and rapid process improvement when going out to look for solutions on Google. Social BPM has appeared on my website, but its not there to try and charge companies enterprise software prices for common 'social' software capabilities.

A post from the Improving It blog

Let us help you improve your business today. Visit


Adam Deane said...

Hi Phil,

Saying that “collaboration” still carries negative connotations for many Europeans, is like saying people in Boston don’t like paying taxes because it reminds them of the British... ;->

I agree with you that Social BPM is a result of people becoming more comfortable with using Twitter and Facebook.
Additional communication techniques will also be added to Social BPM as people are already comfortable with voice messaging, SMS, wikis and blogging.

I don’t think that collaboration software has died. The opposite.
I think that there is more interest in collaboration today, mainly due to Microsoft’s heavy investment in the marketing of SharePoint, Microsoft’s collaboration platform.

Social BPM is still collaboration... but maybe more user friendly.


Phil Ayres said...

Adam, you made me laugh so hard I scared the home office cat. Hey, I'm a Brit, so the Boston thing always makes me smile (pleased they were rid of us, I say). And in my defence/defense, the 'collaborator' piece did certainly make me think about the number of times I'd got blank looks from my European work friends. Though I think that may just have been 'cos I was making no sense, rather than any other dated meanings.

You know, I forgot about Sharepoint. From my fairly extensive use of it (as an end user, rather than developer) I found it to be more a clunky portal than a true collaboration solution (compared with the ease and flexibility of the old Vignette product). But you are right, I suppose many more people are interested in the concept now because of the half billion MSFT invested in its marketing.

I certainly hope that Social BPM turns out to be easier to setup and use than Sharepoint. I know my company's attempt is...

Roger said...

In my company, there is currently a war between supporters of Lotus Quickr / Quickplace and Lotus Workflow (which we have been successfully using for years) and Sharepoint (which is being pushed by the new centralised-services-model IT department, which, BTW, provides vastly worse service than the previous model of one IT cell per department plus a coordinating office.)

The main difference seems to be that any bright person could set up a Quickr on their own but with Sharepoint that have to come cap in hand to IT.

However the best collaboration system I ever used, in terms of productivity enhancement, was a simple wiki (this one was Moin Moin, but it doesn't matter.) The simplicity and flexibility enabled it to grow into dozens of uses never envisaged, and to quickly atrophy and weed out the planned uses that turned out not to be productive. In effect it had all the power and flexibility of an office system built around paper, but with built-in security, automatic backups, powerful search features, intercontinental collaboration, hyperlinking, and less waste of paper ....

True, it didn't have software enforcement of the business rules, but wiki version control is adequate even for groups ten times the optimal team size. If you really need software enforcement of business rules then your teams are far, far too big.

Phil Ayres said...

Roger, you have definitely had some experience in this by the sound of it. Your thoughts around Sharepoint do seem to mirror mine.

Although I agree to a point around Wikis, and especially their usefulness in pure ad-hoc team work and projects, there are many back-office processes that a wiki could not handle with adequate control, even though there are few people involved. In general, anything that requires a repeatable flow of work or information, with some logic or rules in place to effect certain decisions, and finally an auditable trail of what happened to the work.

Really anything that requires formal reviews and approvals fits in this category in my experience. For example, managing expense claims through to payment, order through cash, employee recruiting through to on-boarding, account opening for banks, insurance claim processing, etc.

Maybe you are familiar with more collaborative operations, where I agree that a wiki is a great tool. Could you name some examples where you have used it as a substitute for a more formal workflow?

Roger said...

Hello Phil,

Sorry for the very, very long delay in replying ... I didn't realise you had asked some questions until I just found this again whilst doing some spring cleaning.

Hmm, my reply got pretty long, so I'll try to split it in two parts.



> Although I agree to a point around Wikis, and especially their usefulness in pure ad-hoc team work and projects, there are many back-office processes that a wiki could not handle with adequate control, even though there are few people involved.

We'll have to agree to disagree there. Certainly there are processes for which wikis are unsuitable, but in my experience they are a small minority of total work performed.

> In general, anything that requires a repeatable flow of work or information, with some logic or rules in place to effect certain decisions,

Just as we easily managed this with paper in the past, it works just as well today digitally with any system that can create some kind of a form -- and that includes wikis.

The automatic management of workflow is also helpful for keeping overburdened people up to date on their outstanding list of sign-offs. However, while such a feature is not built-in to any wiki that I know of, it is trivial to add: less than a dozen lines of custom code. (Without any custom code, it is also trivial to do it semi-manually in a wiki -- just change a Category when you submit your updates -- but in that case it's possible for someone to make an error.)

Where the addition of automatic management of the workflow helps in a way that wikis cannot handle at all, is in a workflow within a group that is so large it becomes difficult to manage "who is meant to do what next."

> and finally an auditable trail of what happened to the work.

All wikis are highly auditable. In fact most keep much more detailed audit trails any workflow tools I have examined.

> Really anything that requires formal reviews and approvals fits in this category in my experience.

No, not at all. I have managed a system where project approvals were managed in this way. A formal process, requiring submission and review of specific types of data, authorisation by specific stakeholders (some of them in a particular order), and finally, release of fairly large sums of money (usually in the high hundreds of K, sometimes in the low megabucks.)


Roger said...

There were up to about 11 people involved in the process (more usually 7), and it worked pretty well. Years back it started out as a paper based process, and the only thing really wrong with that was this one guy who kept losing the damn forms! Plus, if the system had ground to a halt, it was difficult for the project manager to figure out where his form was: had Mr. Junkyard-Desk lost it again, or was someone else sitting on it?

When it first became my job to manage this process, I fixed 90% of the problems with a simple procedural change: every time someone finished with a form, it had to come back through me rather than directly to the next guy on the list. And I kept a photocopy of the most up-to-date version.

This crude, 1970s technology solution actually fixed pretty well all of the problems that were workflow related.

I then later changed it to a very simple wiki based system. (It was actually on a QuickPlace, but didn't use any of Lotus' Workflow tools at this point.) While the original system didn't have any problems with malfeasance, the new system -- despite its simplicity -- was pretty close to watertight, and perhaps more importantly, it could be more easily reviewed by external auditors, and its integrity demonstrated.

The wiki system did away with the need for photocopies, and was easier to use if someone was off-site or travelling. But the most important advantage was that it made most of the steps parallelizable. The schedule, technical and budgetary reviews could all occur in parallel before the "sign-offs" phase started. In the sign-offs phase, all stakeholder sign-offs could also occur in parallel, except for the final approval from the big boss.

Finally, the workflow people got a hold of it, and workflow-ised it. They didn't have a workflow pattern that had stages of 1 in sequence (submission), 3 in parallel, sequential jump to n in parallel, then 1 in sequence. And "for a process only used by 11 people" they couldn't be bothered custom-coding the business logic like that, either. So instead it got one of the standard pattern templates; the closest was a purely sequential one.

The system still occasionally stalls at Mr. Junkyard-Desk. He now can't claim to have lost it, so he uses different excuses.

So how do these systems compare?


Roger said...

* In terms of accountability and auditability, the wiki system was the strongest, the workflow system next, and the paper system weakest, but all of them were adequate.

* In terms of level of maintenance, the paper system was definitely the worst; the other two systems were about the same, with a slight edge to the wiki system.

* In terms of workflow efficiency, there are two situations. If a project application hits a Mr. Junkyard-Desk stall, then it will take a long time, and the time is much the same in all three systems. If it doesn't hit such a stall, then the wiki system was by far the fastest. The workflow system is much slower -- slightly faster than the paper system, but not by much. In fact workflow really only handily this beats paper for the case of someone being offsite. This mediocre performance from workflow isn't an inevitable feature of the software, but a result of poor design decisions in the workflow template -- which, however, arose out of the centralised control, and relative inflexibility of the system, both of which tend to make customisation too much trouble.

* In terms of ensuring a quality process --- that good project submissions get approved, and bad ideas killed off quickly before wasting money on them -- none of the systems has been useful at all. The fact is, the only person who takes the time to really scrutinise weak proposals is ... Mr. Junkyard-Desk.

> For example, managing expense claims through to payment,

Yep, a wiki can't do that one, at least in our company, but probably not for the reason you are thinking. The first 3 stages are easy: Amex sends me my bill; I fill out our form, attach the bill and receipts, and forward it to my boss; he signs it off. So far, a wiki could easily handle that efficiently, with no risk of fraud (or no more risk than any other system, anyway.) But the next stage is impossible for a wiki: forward it to a (randomly assigned) person in Finance to make a ledger entry on the mainframe (which will result in a B2B direct payment around 11 pm that night.)

It would be easy in a wiki system to have a Caetgory tag that means "hey, someone in Finance, this one needs processing!" but that would potentially allow a confederate in Finance to always grab you claims, if you co-ordinated timings so no-one else could jump in. The business rule that the finance processor must be random -- that me and my boss are unable to influence the choice of processor, *and* the processor is unable to select our claims -- requires some other kind of system. However, even if you choose a workflow package, you will probably be doing some custom coding because I have never yet seen one that has a built-in for this sort of rule. [...]