“I have never been in a project where data is not a problem” - and what to do about it
Data is always, an issue in digital business development. The lack of it. The quality of it. The access to it.
While attending WebSummit in Lisbon Portugal during end 2023 one speaker claimed that companies have been spending 20 years organizing data and are now ready to leverage it. For me this was a blow to the stomach. Really? Everyone is done with organizing their data, really? Most companies I work with struggle with this. It may be product data, customer data, or pricing data, or data of installed products. The list is endless and very few, if none, have no work left to do here.
Why is data important in digital business development?
Digital business development is mostly created via taking business processes and information from internal and manually managed to exposing and automating it in a manner that makes sense to customers. The data is essential. Without price data and product data you cannot do B2B e-commerce for example. Digital business development, more often than not, stumbles and cannot be realized due to issues with missing data or poor data quality.
How to approach the data issue?
There is a very basic process of three steps. Every time one evaluates an idea, or there is a suggestion of a UI of a new feature for the B2B site. Always look at the data displayed and assess.
Does this data even exist in the organization? If yes, move on to question 2.
Is the data of good enough quality? If yes, move on to question 3.
Is it possible to access the data?
If you managed to answer yes on all three questions that is great, just go for it. But often you get stuck on the way.
Question 1: Does data even exist?
Very often you do not even get past the first question. The filter you want to have in the search, the detailed drawing, they do not even exist.
Question 2: Quality?
Even when data exist the quality is often lacking. The data is just filled out in 30% of the cases. It is managed in different manners in different parts of the organization. It has only been managed for the last 2 years so any product older than 2 users does not have the data.
Question 3: Possible to access?
Even if you have data of good enough quality, it might not be enough. Can you access it? Not uncommonly it is in a system which will be replaced 2 years from now and hence no new development should be done on the old system. Or is it in an old ERP system where the data can be extracted one per week in a CSV format.
How to deal with it
So, if you cannot get passed all the three questions do you just stop and lie flat or what do you do? There are options.
Scope – work with what you have
Maybe one must accept that a feature only works for installations at least 2 years as the older data is un-reliable. Or we cannot show images of some products and thereby must have a solution that also works when there is no image. Work with what you have and scope the solution accordingly.
Take the bull by its horns
Recognize how the digital business we want to do requires us to have and maintain certain data. And fixing this might a project of its own. Before the development of a fancy customer interface for an automated quote request can be made, the first step is to define the process and what data is needed.
Why it is not so easy to just "fix" the data
People may often ask if we cannot just fix the data, for example by:
Creating the missing data
Cleaning out the faulty data
Correcting data
While it may seem easy at first glance this is usually a nightmare. Data reflects your people and processes. A data issue is most often also a people and process issue.
Let's have a fictional example: Color of a product is managed in a free text field and now this should be a pre-defined list of values to re-sue both for filters but also for targeted adds.
Firstly: What color do we even have. We must align the colors possible to choose from
Secondly: We need a process for how and when the color should be added
Thirdly: We need to train 10 different departments in 3 countries, so they know how to work with the new field
Fourth: We need to handle the legacy, what do we do with the 50 000 legacy items?
Fifth: … The list can go on.
So, now to some real examples:
At one client customer data needed to be cleaned up. The issue was that the non-aligned customer data was a representation of the non-aligned manners of working with sales and customers throughout the organization. This was not a data project; it was a process harmonization project. A task much bigger than just correcting data is cells. It was aligning customer offers, trainings cross departments. Hard decisions on what was a "correct" manner of doing something.
At another project there was a field in the B2B checkout called "sales rep " which should be removed. Why? It confused users in the check-out when placing an order and was a hurdle. The problem? This field was a remain from manually placed orders, and the field was used to calculate which sales representative was responsible for the order. This data was then used to calculate bonuses. The remove the field required essential changes in how bonuses was calculated. After a year of driving the bonus-calculation transformation, the field could be removed with a few hours of work.
Changing data is changing people and processes. Never underestimate the struggle and work needed.
Author: Helena Strahl, Consultant and partner at Waye. She’s always in project were data is a major issue.