Your organization is likely to have more data than it has ever had before. Customer data resides in your CRM, transactions are stored in databases, log data is sent to the cloud, and analytics teams run from data warehouses and data lakes.
More data doesn’t necessarily mean a more data-driven business.
The true challenge lies in making that information easily accessible, understandable, trusted, and usable. This is where data products come in.
What Are Data Products?
A data product is a data artifact that is reusable and could address a particular problem related to a business or analytical need.
This could be a curated dataset, dashboard, API, machine learning model, analytical output, or all of the above. The key is that you don’t just send out data. You package it with the context, quality, documentation, governance, and access required to be useful.
Consider a data set of customers.
Basic data could include names, email addresses and customer IDs. A customer data product could include CRM data, transactions, support conversations, customer identities, business definitions, data quality data, unlimited access controls, and documentation.
Now your analysts can use the same customer view without having to recreate it each time. That is where data products are most useful to me, because it is a way to transform repetitive data work into a reusable capability.
What Makes a Good Data Product?
Not all datasets can be considered a useful data product. Consider the users or applications that will be consuming it.
A good data product should be:
- Easy to discover: It can easily be located via a catalog or data platform.
- Interpretable: You can interpret the data and understand how to use it.
- Reliable: Quality checks can aid you in detecting incomplete or inaccurate information.
- Reusable: The trusted asset can be shared by multiple teams.
- Secure: A business information system will protect sensitive business information with access controls.
- Well owned: Someone owns the product and has a responsibility to take care of it.
- Interoperable: Can be used between various tools and systems.
This product mindset alters data teams’ ways of work. You’ll eliminate the “Where is the data?” question by having your users ask, “Which data product can solve my problem?
Types of Data Products
The type of data products you will need will be based on your business objectives and who will be using your data. Here are some of the most prevalent types.
1. Source Data Products
These products deliver operational system data from CRM platforms, applications, databases or APIs or outside sources.
Simple cleaning and governance can be performed and the source structure maintained close to the original data.
An e-commerce team might release a governed transaction data product to others, for instance, to be leveraged as insights for analytics and forecasting.
2. Master Data Products
Master data products create a consistent view of important business entities.
These could include:
- Customers
- Products
- Suppliers
- Employees
- Locations
For instance, you can have one trusted customer data product for both sales, marketing and support instead of keeping the customer definitions separate in each of these groups.
3. Analytic and Insight Data Products
These products transform data into information to aid in decision-making. Examples of common data products include:
- Sales performance dashboards
- Customer segmentation datasets
- Forecasting models
- Business intelligence reports
- Marketing analytics
- Predictive analytics solutions
These products are geared more towards data interpretation and providing answers to specific business questions.
4. AI & Machine Learning Data Products
Data products can be created to be used in the context of AI and machine learning, too.
These can be feature datasets, recommendation models, fraud detection models, training datasets, model outputs, etc.
For example, an e-commerce business might develop a recommendation data product that brings together factors including machine learning models, product details, and customer activity to yield individualized tips.
How do Data Products Work?
A common approach to a data product is as follows:
Data Sources → Data Engineering → Transformation → Quality & Governance → Data Product → Data Consumers
Let’s look at what happens at each stage.
1. Collect Your Data
Collect data from applications, databases, APIs, cloud storage, data lakes, data warehouses, or streaming sources.
2. Transform the Data
The data engineering team cleans, standardizes, joins and transforms the data for the purpose of the product. This is important because the users of the output will be impacted by the quality of the input.
3. Add Quality and Governance
The validation rules, the metadata, the lineage, the security policies, the business definitions and access controls are then applied. This layer is typically what makes a data product useful versus another undocumented data set when dealing with testing in a real environment.
4. Package and Publish It
You arrange the processed data into something that people can eat. This may be a dataset, a dashboard, an API, a model or multiple components that are interconnected.
Make the information easily accessible.
You need to make your users aware that the product exists. Describing ownership, documentation, quality indicators, and instructions for data usage can be facilitated via a data catalog or with an internal data marketplace.
6. Monitor and Improve
The work doesn’t come to a close when the product is published. Data quality, freshness, data usage, performance and business value should be monitored. Data products should change along with your business.
How Data Products Fit Into Data Mesh
It’s common to see data products pop up with data mesh!
Data mesh is a method of moving the ownership of data towards business domains. Rather than having one central data team be responsible for all data, domain teams can take ownership of their own data products, such as finance, sales, marketing, or customer service.
For instance, your sales team could have a sales performance data product, and your customer service team a support interaction data product.
This approach is a marriage of domain ownership, self-service data infrastructure and product thinking.
It is as easy as that – essentially, allowing the people most at home in the business context to decide what data is used by others.
Why Are Data Products Important?
A data usability issue is likely if your teams are constantly cleaning the same data sets, are searching through multiple systems or are debating which data metric to believe.
Data products offer opportunities to help you:
Eliminate the duplication of data preparation
- Improve data discoverability
- Create consistent business definitions
- Support self-service analytics
- Strengthen data governance
- Enhance cooperation between the business and data teams
- Provide trusted data to AI and machine learning teams
- Make data more re-usable across applications
The greatest value isn’t just another data asset. It’s providing a secure and readily available means for your customers to address a business challenge.
Data Products vs. Traditional Datasets
A traditional data set mostly provides you with data. A data product provides data and the context and controls you need to use it with confidence!
There may be customer data in your customer data set. Your customer data product might include standardized definitions, quality checks, lineage, owners, documentation, access policies and delivery methods.
That’s an important difference.
Data is contained in a data set. A data product is a piece of data that is useful in a specific context.
Final Thoughts
Just gathering more data doesn’t solve business issues, as they need data to solve them. The data must be what your teams need to use. To get that done, you need to leverage trusted data, business context, quality, governance, documentation, and delivery as reusable data products.
Data products can provide a more standardized approach to accessing and using data, regardless of the type of data architecture your teams are implementing—a modern data lake, data warehouse, data mesh architecture, or an AI platform.
In the era of data engineering and modern analytics, understanding what data products are is not only a technical question. It’s a component of creating a data environment that will meet your company’s needs as it expands.
