Friday, June 02, 2006

Suitability determination in a vacuum

In a previous post I talked about using business rules software to enforce suitability determination for annuities – here.

Enforcing the rules in a vacuum is not enough. You need to be able to demonstrate that for a particular decision the rules were enforced as part of the broader business process, within the context of all of the customer information and documentation. This requires more than an audit trail provided by a rules engine, but real human or machine readable proof of what the rules were, and how the decision was made, at what point in the process, and based on what criteria. If you end up in court contesting a suitability claim, its going to be far more convincing if you don’t need a software forensics expert to demonstrate that the decision made did actually follow the rules.

To ensure legal weight, the suitability selection and decision proof should be captured and retained for the life of the associated customer documents. The customer documents (the customer file) should be in an electronic records management system. So this is where the suitability selection and decision proof should go. The problem is that the majority of organizations do not have a records management system.

Many vendors offer records management systems. Few target them effectively at financial services organizations, so the CIO has to work out whether a vendor’s interpretation of DoD 5015.2 certification really matters to them. More important than a specification written for defense organizations, and the de-facto standard for federal government, is the integration with the organization’s document management and business process management systems. This is where all the information is coming from, including suitability decisions, so having seamless integration with these core systems is essential to ensure the integrity of the stored information.

Unfortunately many of the big software vendors have only loosely integrated the records management software they acquired with their document management systems, and even less with their process management. IBM and Documentum are examples of this, and are often accused by analysts of the complexity of the end solution. Stellent is better, having packaged their products up well. Vignette has records, document and process management as a single seamless product, meaning that organizations can depend on all the components working as expected.

Enforcing suitability (or any business rules) in a vacuum is not good enough. Capture the results in the context of the business process alongside customer documents and an organization is more likely to stay out of court. If it comes to court, ensure that all the proof is retained throughout its life in a well integrated records management system and the software forensics experts will not be able to pick apart a defense on the stand.


James Taylor said...

While not disagreeing with your comment regarding documentation, business rules management systems do produce human-readable logs or at least can easily be made to do so. This is part of what makes them great for compliance - see the section on compliance on my blog for more.
It is also worth noting that any analytics you use must also be explainable - saying you rejected someone becuase "the risk model said to" does not cut it either! This is actually harder - there's a post on it here

Phil Ayres said...

(James, sorry for taking a little time to respond. After a great vacation in Europe, jetlag has kicked in!)

James' comments around human-readable logs are spot on. I'm sure it is dependent on the software used as to whether they really do communicate the decisions made clearly and accurately.

A complete system should treat any logs as formal records, directly accessible alongside all other records in a customer file (assuming the user has privileges to read them of course!). This ensures they are available as proof whenever needed.

Unfortunately I have seen too many systems combining many components that treat decision logging as an after thought.

As for explaining the results of your analytics, this is a complex area. Making, justifying and explaing analytical decisions is a domain for the experts. I would encourage interested readers to look at the links James provided for more information.