Data Lifecycle Management (DLM) has always been essential. Some people and organizations actually take the management of the data lifecycle stages for granted. Data management as a separate discipline with its own platform for managing all data and the data quality lifecycle is often not a strategic initiative for many organizations. These organization will actually manage their data relative to a particular practice such as IT Service Management (ITSM), security, or a specific application. This should not be the case; Data Lifecycle Management should be a defined holistic practice that supports consumers such as ITSM, security, or a particular application.
Data Lifecycle Management should be a foundational strategy for all organizations. Every organization should have a strategy for managing their data in addition to other strategies. The data lifecycle management strategy should be governed by the organization’s executives and taken very seriously, not just as something all the different disciplines have to manage based on general governance standards. Data is the foundation of every activity, process, and procedure done to support and deliver service and products to our customers. Understanding and using data wisely is the key to understanding our customers’ needs and the organization’s needs as a whole, including the constraints to delivery and support of better service and products.
What is Data Lifecycle Management?
Data Lifecycle Management is more than just a process to control the data from beginning to end within an organization but should be more of a practice that organizations should use holistically across all of the data they manage. Organization data is just that organizational data for use by the organization for specific decisions that each entity of the service or product value stream needs to function.
Data is the asset that drives the organization and should always be considered a strategic asset that supports all business functions. When visualizing an organization from an architecture perspective, data management should be seen as a foundation supporting all other areas of the organization.
The Data Lifecycle Management practice should consist of the following aspects:
People
This includes governance, managing risk, and making sure people are compliant with the safe use of data within the organization. Such as following regulatory compliance, security, data integrity, confidentiality needs for the organization and its customers.
Research
Research data lifecycle is the act of analyzing data to help with decision support. Many organizations have started automating data analysis when using Machine Learning or Artificial Intelligence methods.
Process
The organization should formally define the process relative to the stages of the data. Identify roles and responsibilities (RACI) for the data. Establish data workflows, translations, etc. The process should include process controls (governance, policies, standards, etc.), the activities, and process enablers such as organization assets (people, technology, etc.).
Governance, Risk, Compliance (GRC)
Data should be governed, risk-managed appropriately, and have compliance measures put in place. Stronger controls should be given for Personally Identifiable Information (PII) and information and sensitive data, such as data that is subject to regulations like HIPAA, CCPA, and GDPR.
Technology
To help manage the data in the most effective and efficient manner for all of the stages of the data. The technology should help enable a holistic approach to the management of data from all sources, such as the cloud.
Suppliers
Third-party suppliers, such as cloud vendors, can greatly help with Data Lifecycle Management. Almost everything, especially capabilities that the organization does not have, can be offered as a service from vendors to help an organization avoid the struggle and cost of acquiring its ability. This usually results in time savings and a faster time to market for their solutions. Organizations should be cautioned about the cost over time relative to the return on investment of the services and products offered.
Metrics
Data should have key performance indicators, critical success factors, and goals related to expected outcomes of the data, information, and knowledge itself. This helps organizations with improving their data and overall understand constraints to decisions in their organization.
Stages of Data Lifecycle Management
The Data Lifecycle Management stages are essential to understand. Each data lifecycle stage has value and particular outcomes that should be managed. Different parts of the organization interact with the stages at different moments in the life of the data. Each interaction should be in support of the value chain of data transportation across the organization. An organization should be careful of managing data in a silo and should be cautious of not collaborating on the use and transformation of data. If not, organizations may create inefficient data usage, information, and knowledge affecting overall service performance.
Data Lifecycle Management stages are – creation, storage, translation, usage, archival, destruction. Each stage is explained below.
- Creation – This is defined as any source or input for creating data, such as acquisition, data entry, and data capture by applications, Machine Learning (ML), sensors, Artificial Intelligence (AI), etc. A research data lifecycle can spin-off from the collection or creation of data as an innovation project.
- Storage – This is defined as where and how the data is recorded or stored, including backup, continuity, and recovery plans and procedures. All data is not digital, so concerns here should be for all data, including non-digital. Also, it should be included in data in transit that may be stored for smaller amounts of time using technology.
- Translation – Some data can be used as-is by the consumer of the data, and some have to be transformed or translated into information or knowledge for the organizational usage for decision support. This transformation of data or data processing can be considered information lifecycle management and knowledge lifecycle management. Data, information, and knowledge should be managed together and as a separate process to maximize the benefits and outcomes of each, helping to result in an overall data quality lifecycle.
- Usage – The consumption of the data, information, or knowledge transformed into use for viewing, processing, sharing, saving, and other activities. To do this effectively, the organization should determine what data, information, or knowledge is needed by each consumer of the data (application, person, or other technology) to ensure the data is managed from an enterprise perspective. Hence, all consumers interpret the data the same to support collaborative decisions. Data usage also includes managing the transfer and publication of the data.
- Archival – Some data cannot be immediately deleted but has value from a historical perspective or a compliance perspective and must be archived. Archived data is usually no longer active and is for long-term retention. Many organizations may use data warehousing capabilities for archival of data that is seldom used for performance reasons but may also use technology for faster access of their achieved data.
- Destruction – Data should be destroyed as needed based on archival needs and the needs of the organization to make decisions. Keeping too much data around increases the cost of managing the data, thus affecting the total cost of ownership and return on investment of the organization’s services and products.
During data translation, the data can go through a research data lifecycle. This may occur in the organization when doing a data analysis exercise for combining data or determining data relationships for the information and knowledge articulation of the data. Before archival of data, the organization may determine if any data can be reused by the organization for an advantage.
Data Lifecycle Management goals
The main goal of data lifecycle management is to ensure all data has value for the organization in an agile fashion when needed. The data should be available when required by the appropriate person or technology, be confidential, and have data integrity at all times.
In addition, the three main goals of data lifecycle management are:
- Data compliance and governance – Compliance and governance is not only an organizational goal but is also address with government regulations on the use of customer data. All organizations have to be compliant with various government regulations depending on the business and services that they supply to customers. Without this as a goal, no organization would be able to perform its business functions.
- Data protection – Data protection is everywhere. Data is sometimes labeled confidential, private, secret, and with other labels to help protect the data from those who should not use it for any reason except the intended reasons. Data protection should occur at all stages of the data management lifecycle. From collection to disposal, data has to be protected. Thieves are getting smarter and wiser every day, and data misuse, theft, and schemes are constantly evolving. There are standards and regulations such as General Data Protection Regulations (GDRP) and The Data Protection Act to help organizations in this area.
- Data usage for decision support – All data should be used for decision support. This happens when data is translated to information, information to knowledge to support decisions by people and technology. Any data that an organization captures or creates that does not serve this purpose is a waste of time and resources for the organization. An organization that sees its importance actually makes Decision Support Data Systems (DSDS) to help identify, collect, and analyze their data across the organization. This data is collected from various functions within the organization to help with technology usage for a better result in performance improvements and less manual efforts.
The Data Lifecycle Management three goals have to support the mission and vision of the organization. This is of strategic importance. The tactics and operational aspects of Data Lifecycle Management are supported by programs and projects for innovation, growth, competitive enhancements, and overall to keep the business running. All of these perspectives have to work together for the good of the organization and its customers. Disconnects from the goals with operations or strategic intent will create bad data lifecycle practices within the organization.
Data Lifecycle Management tools
Data should be managed from a strategic, tactical, and operational perspective until the data no longer exists or is destroyed by the organization. Data Lifecycle Management tools help with each stage of managing data. Tools can be categorized in many areas such as:
- Cloud management.
- Version control.
- Project management.
- Development.
- Release management.
- Change management.
- Database management.
- Storage management.
- Data quality.
- Product lifecycle.
- Any tool in the organization that touches, creates or manipulates data.
When selecting tools, it is best that the tools can be integrated with other tools and data sources outside of the native tools to enable data collaboration and coordination of the data across the organization. The organization should be able to map or architect the tools into an overall data lifecycle management tool perspective. When taking inventory of current data architecture and tool usage, the organization may find tool usage that they can save cost and do tool reconciliation of tools that perform the same tasks with the same data.
One tool, for example, that can be useful is Amazon (AWS) Data Lifecycle Manager. This tool uses automation to back up data stored on the AWS Elastic Block Store (EBS) volumes, including the creation and deletion of the EBS data. It uses policies that should come from your overall data lifecycle management strategy. Protection of value data, standardization of methods, and storage/retention/disaster recovery compliance are all critical product capabilities.
Always remember the tools cannot provide enough without them being a part of a more considerable practice to support the organization’s needs. The organization should work as one team. Data, information, and knowledge across the organization are the enabler to help with the coordination and collaboration of all organizational functions, teams, and roles, including automated technology.
Data Lifecycle Management conclusion
Data Lifecycle Management as a practice is vital for all organizations. Holistic data management should not be an afterthought but a critical strategic capability of an organization. Key roles that an organization may consider are data trustee, data custodian, data steward, data owner, and data manager.
Not managing data effectively is a considerable risk for the organization and its customers. Data is growing, and the management of it cannot be left behind. Open architectures and the ability to connect to anything anywhere give a better experience because of data accessibility and create more opportunities for data challenges, especially malicious use of data. Make Data Lifecycle Management a strategic initiative for your organization.
Actian’s customer 360 analytics solution puts you in control of your organization’s customer data, reducing the need for specialized expertise by making it easy to use the BI tools companies already rely on. Increasing revenue while reducing vendor lock-in from CRM and gaining deployment flexibility across multi-cloud or hybrid environments