MIOsoft logo

Data first (part 3): Make the most of your fragments

Rob Rothwell

Customer 360, Single Pane of Glass--whatever you want to call it, it promises to integrate all of your data into a single, unified display. Everyone wants it, and with good reason.

Here’s what no one’s willing to admit: Customer 360 is easy when you have a single system full of pure, harmonious, complete data, but very few businesses have that. And getting Customer 360 out of messy, conflicting, fragmented systems is a lot harder.

But here’s something else: that’s okay.

It’s not that a multi-system environment is ideal, from a data quality standpoint. Far from it. It’s virtually inevitable that different attributes of primary business entities (like customers or products) will end up existing in multiple locations, inside a mass of databases, text files, spreadsheets, and more.

As more systems are added to the environment, it becomes more likely that an error in any one system will lead to performance issues across the entire system. Then the wrong customer is scheduled, the wrong service is performed, the wrong charges appear on the bill, and so on.

Even if everything works out in the end, the customer experience might be unsatisfactory, and the hidden costs of dealing with these errors can be substantial.

Some sources estimate that a single duplicate record costs a hospital $50 or $60. Even if the error percentage is very small, high volumes of customers and transactions can result in a large number of errors--with the associated costs.

To make matters worse, these errors can remain hidden in otherwise functional systems until cross-process interactions detect them. As they accumulate, overall system integrity can be highly challenged.

But let’s be realistic: this didn’t happen because businesses are run by people who wanted to store their data piecemeal and cause themselves a lot of problems.

It happened because businesses do not (generally) buy a database.

Businesses buy applications. They buy those applications because they need to do a specific job. Usually, doing that job means that the application has to store data, so a database and some hardware come along for the ride.

The business might do its best to ensure that all the components of its data ecosystem can talk to each other, but that’s probably secondary to business functionality.

Their need for that functionality isn’t going away. And in many cases--whether institutional inertia or a lack of alternative technology is to blame--that means that the systems can’t, either.

So what are businesses with multiple systems supposed to do?

Give up on the idea of having high-quality data and muddle along as best they can? Starve themselves of their essential systems so they can fit into a single-system vendor’s preconfigured box?

No way.

Here’s my proposal: take that conflicting data and turn it into something useful. Use it to get the benefits of Customer 360 without forcing your business to forgo the applications that have been effectively supporting it for so many years.

To imagine how this can happen, let’s back up and consider a real multi-system environment.

One of our recent healthcare customers had two locations. In the past, each location had installed its own imaging system. These separating imaging systems fed into a single archival system. A primary hospital information system, or HIS, glued them all together.

Our customer wanted to move both locations onto a single new imaging system; to maintain business continuity, the images needed to be migrated from the two existing systems into the new system.

In the past, the customer had made certain decisions about how to link the two independent imaging systems with the archive and the HIS. When they tried to migrate, they found that as a result of those decisions, only 5% of their 183,000 patient records--with a total of 1.5 million associated radiological studies--had a patient identifier (like a medical record number) that could be matched in the other systems.

But let’s just think about what this really means for a minute.

95% of the time, the computer couldn’t use primary IDs for cross-system patient record matching. So the computer couldn’t readily make cross-system patient information available to the people using these systems, either.

And yet: patients were being seen, images were being processed, and care was being provided!

It was all possible because human beings were doing what they’re good at. People would look for records based on criteria that wasn’t the “official” ID, like the patient name. Then they’d use their own judgment to identify records that belonged to the same person, distinguishing a record with a misspelled name from the record of a different individual.

Although this let our customer work around their data inconsistencies, they’d never developed a formal process for correcting them. This was another reason they needed our help for the migration.

For over 95% of the patient records present in our customer’s four systems, we were able to fuse a patient’s fragments of data from each individual system into a new and correct record. Our customer used this new record to correct their source systems. Once all the systems had corrected information, they were able to migrate successfully.

Our customer’s pre-migration situation is typical of any multi-system business. They had multiple systems that they’d bought for rational reasons. They’d made decisions about how to connect those systems according to their best judgment at the time.

In hindsight, those decision were less effective than they’d hoped they would be, but they still had to deal with the resulting data.

Even though our customer’s migration is complete, they still have multiple systems. It’s very possible that they always will. In healthcare, and in many other industries, there’s no single system right now--or even in the near future--that can truly meet all of the business’s needs.

You have multiple systems, and you can’t get rid of them; you have overlapping, fragmented data, and you can’t get rid of it; you have the possibility of conflicting data, and you always will.

This truth challenges the Customer 360 vision, which views a multi-source ecosystem and its changeability as inevitable frustrations.

But that’s not the right frame of mind, because fragmented isn’t synonymous with worthless. Ask any paleontologist.

Your complex systems mean that you can collect critical information at multiple points in the business process. This is an advantage, especially on an ongoing basis: the data, not the technology, becomes the centerpiece of Customer 360.

This is Customer 360 for customers who have complex data needs.

The techniques that we used to correct our customer’s static imaging data set are the same techniques that our customer’s users adopted to work around the broken cross-system identifiers. When you apply those techniques continuously, as data arrives from incoming systems, amazing things can happen:

Fragments of data can coalesce into that single, coherent view that Customer 360 and the single pane of glass promise.

So if your business functionality requires an ever-changing multi-system environment, then use one. But don’t give up on Customer 360. You don’t have to. You can still get that coveted single view.

It’s all in the fragments.