Review Data Management Concepts
Learning Objectives
After completing this unit, you’ll be able to:
- Explain key data management concepts.
- Apply data management concepts to real-world scenarios.
Reviewing Data Management Concepts
Northern Trail Outfitters’ rapid growth has led to the adoption of many technologies and applications, resulting in data silos, inconsistent data quality, and difficulties in accessing reliable information. NTO’s chief data officer recognizes that data inconsistencies cost organizations time and money. Business analysts might spend days reconciling inconsistent information from different systems. When data inconsistencies exist, user trust is often eroded, which can lead to adoption issues.
To address these challenges, NTO’s data needs to be managed more effectively and business users need to trust the data. Given that data is a shared asset, NTO needs a common language to help stakeholders communicate effectively.
Data management concepts can be broken down into three main stages: planning, delivery, and maintenance. This unit explores the essential data management concepts that guide NTO’s efforts to improve data reliability, simplify operations, and support strategic decision-making.
Planning Concepts
NTO relies on its data to make effective decisions, and data strategy drives its data management and technology investment needs. Let’s review the three main planning concepts related to organizing a data strategy, and see how NTO has applied them.
Planning Concept | NTO Example | ||||
|---|---|---|---|---|---|
Data strategy: A comprehensive plan that outlines how an organization collects, manages, analyzes, and uses data to achieve its business goal. | NTO defines prioritized business objectives and use cases to guide application development, enhancement, and data management investment decisions. | ||||
Data governance: Establishing and enforcing policies, standards, and procedures to ensure data is managed effectively and consistently across its lifecycle. | NTO’s data for the different lines of business have explicitly defined policies. Within the same brand, NTO explicitly allows order data sharing between retail stores and ecommerce to simplify customer returns. | ||||
Data architecture: The design and structure of how data flows across systems, ensuring it’s accessible, consistent, secure, and aligned with organizational goals. | NTO’s solution roadmap plans to integrate legacy systems with newer platforms, enabling seamless data flow and reducing data redundancy and the cost of maintaining data in multiple siloes. |
Application Development and Delivery Concepts
To ensure data is captured, integrated, and used properly, NTO application developers and architects use a common language to understand what purpose each technology serves and how they can work best together.
Let’s define the key components of application development and delivery as it relates to developing a data strategy.
Systems of Record, Reference, Intelligence, and Engagement
- A system of record is the authoritative source for specific data, typically where it’s created or recorded, and holds the official version of the data. It’s used for compliance, auditing, and historical data accuracy.
- The authoritative access point is called the system of reference, especially when there are different systems of record for different pieces of data.
- A system of intelligence supports decision-making through artificial intelligence (AI), machine learning, and analytics to reinforce business outcomes.
- Access points that interact directly with users, typically facilitating customer or employee interactions, are examples of a system of engagement.
Data Transformation and Standardization
- Data transformation (or data munging or wrangling) is the process of converting data from one format, structure, or level of detail into another to meet the requirements of a specific application or process.
- Data standardization ensures consistency by converting data into a common, predefined format or structure.
Data Integration and Enrichment
- Data comes from many sources. The process of combining and exchanging data from various sources is data integration.
- If additional, related data is brought in, this is known as data enrichment.
Data Validation and Identity Resolution
- During data validation, automated and manual processes ensure that data is accurate, complete, and conforms to predefined rules. Data validation prevents errors and ensures data integrity.
- Identity resolution is the part of validation focused on identifying, matching, and reconciling related records that may be intentional or unintentional duplicates.
Data Security
The practice of protecting data from unauthorized access, breaches, and corruption, ensuring its confidentiality, integrity, and availability throughout its lifecycle is data security.
Now that you’re familiar with the key components of application development and delivery, let’s review how NTO applies them to developing their data strategy.
Data Maintenance Concepts
Data flows constantly in a business. To ensure data remains reliable, NTO has ongoing data maintenance processes. This table reviews how NTO maintains its data.
Data Maintenance Concept | NTO Example |
|---|---|
Data operations: The management of the entire data lifecycle—from capture and processing to storage and delivery. | NTO seeks to ensure that all data used across the organization is reliable, relevant, and fit for purpose. |
Data quality: How well data meets expectations for accuracy, completeness, consistency, and reliability. | NTOs data stewards monitor data quality within systems of record and in systems of reference to ensure only reliable data is used in business processes. |
Data cleansing: Fixing or removing incorrect, corrupted, incorrectly formatted, duplicate, inconsistent, or incomplete data. Also known as Data Scrubbing. | NTO’s data stewards check for email addresses that occur at a disproportionately high ratio to detect potential invalid or irrelevant contact points, such as none@none.com. NTO stewards remove invalid data through data transformations or data cleansing tools. |
Data retention: Policies and processes that determine how long data is retained and the methods for its disposal, ensuring compliance with legal, regulatory, and business needs. | NTO must retain customer purchase history for 7 years to comply with legal requirements. However, only 3 years of order history is considered relevant to support warranty claims. To reduce storage cost, improve usability, and ensure data is maintained where it is relevant, NTO has in place a data retention policy that archives order details older than 3 years and purges order details that are older than 7 years. To make sure customer loyalty programs are not affected, all order date and amount data is retained. |
Data trend monitoring: The process of tracking and detecting changes in data patterns over time to identify anomalies, predict outcomes, and ensure data remains reliable. | NTO has automated data monitoring for any field used in identity resolution to ensure fake or invalid addresses don’t impact customer communication or lead to incorrect unified profiles. |
Data integration: The process of combining and exchanging data from various sources to provide the necessary information to business users. | To respond to customer order inquiries, NTO’s service agents require real-time order details from ecommerce, shipping, and payment systems. To increase upsell conversions, agents also need insights into customer sentiment and preferences, which can originate from different data sources and require tailored data transformations to provide actionable insights. |
Summary
In this module, you learned about key data management concepts, the components of a data strategy, methods for establishing information needs, and the various dimensions of data quality. You examined the processes for planning and developing effective data strategies, maintaining data quality, and ensuring organizational compliance and security.
To further your understanding of data management, consider applying these principles within your organization to foster data-driven success.