Types of Data: Reference Data, Transactional Data, Warehouse Data and Business View Data21st November 2020
Reference data is data used to classify or categorize other data. Typically, they are static or slowly changing over time.
Examples of reference data include:
- Units of measurement
- Country codes
- Corporate codes
- Fixed conversion rates e.g., weight, temperature, and length
- Calendar structure and constraints
Reference data sets are sometimes alternatively referred to as a “controlled vocabulary” or “lookup” data.
Reference data should be distinguished from master data. While both provide context for business transactions, reference data is concerned with classification and categorisation, while master data is concerned with business entities. A further difference between reference data and master data is that a change to the reference data values may require an associated change in business process to support the change, while a change in master data will always be managed as part of existing business processes. For example, adding a new customer or sales product is part of the standard business process. However, adding a new product classification (e.g. “restricted sales item”) or a new customer type (e.g. “gold level customer”) will result in a modification to the business processes to manage those items.
Reference data management
Curating and managing reference data is key to ensuring its quality and thus fitness for purpose. All aspects of an organisation, operational and analytical, are greatly dependent on the quality of an organization’s reference data. Without consistency across business process or applications, for example, similar things may be described in quite different ways. Reference data gain in value when they are widely re-used and widely referenced.
Examples of good practice in reference data management include:
- Formalize the reference data management
- Use external reference data as much as possible
- Govern the reference data specific to your enterprise
- Manage reference data at enterprise level
- Version control your reference data
Transactional data are information directly derived as a result of transactions.
Unlike other sorts of data, transactional data contains a time dimension which means that there is timeliness to it and over time, it becomes less relevant.
Rather than being the object of transactions like the product being purchased or the identity of the customer, it is more of a reference data describing the time, place, prices, payment methods, discount values, and quantities related to that particular transaction, usually at the point of sale.
Transactional data describes an internal or external event which takes place as the organization conducts business and can be financial, logistical or any business-related process involving activities such as purchases, requests, insurance claims, deposits, withdraws, etc.
Transactional data support ongoing business operations and are included in the information and application systems that are used to automate an organization’s key business processes such as online transaction processing (OLTP) systems.
It is grouped with its associated and references master data such as product information and billing sources.
Transaction data is data describing an event (the change as a result of a transaction) and is usually described with verbs. Transaction data always has a time dimension, a numerical value and refers to one or more objects (i.e. the reference data).
Typical transactions are:
- Financial: orders, invoices, payments
- Work: plans, activity records
- Logistics: deliveries, storage records, travel records, etc.
Typical transaction processing systems (systems generating transactions) are SAP and Oracle Financials.
In computing, a data warehouse (DW or DWH), also known as an enterprise data warehouse (EDW), is a system used for reporting and data analysis, and is considered a core component of business intelligence. DWs are central repositories of integrated data from one or more disparate sources. They store current and historical data in one single place that are used for creating analytical reports for workers throughout the enterprise.
The data stored in the warehouse is uploaded from the operational systems (such as marketing or sales). The data may pass through an operational data store and may require data cleansing for additional operations to ensure data quality before it is used in the DW for reporting.
Extract, transform, load (ETL) and extract, load, transform (ELT) are the two main approaches used to build a data warehouse system.
Using Data Warehouse Information
There are decision support technologies that help utilize the data available in a data warehouse. These technologies help executives to use the warehouse quickly and effectively. They can gather data, analyze it, and take decisions based on the information present in the warehouse. The information gathered in a warehouse can be used in any of the following domains:
- Tuning Production Strategies: The product strategies can be well tuned by repositioning the products and managing the product portfolios by comparing the sales quarterly or yearly.
- Customer Analysis: Customer analysis is done by analyzing the customer’s buying preferences, buying time, budget cycles, etc.
- Operations Analysis: Data warehousing also helps in customer relationship management, and making environmental corrections. The information also allows us to analyze business operations.
Functions of Data Warehouse Tools and Utilities
- Data Extraction: Involves gathering data from multiple heterogeneous sources.
- Data Cleaning: Involves finding and correcting the errors in data.
- Data Transformation: Involves converting the data from legacy format to warehouse format.
- Data Loading: Involves sorting, summarizing, consolidating, checking integrity, and building indices and partitions.
- Refreshing: Involves updating from data sources to warehouse.
Business View Data
Business intelligence (BI) comprises the strategies and technologies used by enterprises for the data analysis of business information. BI technologies provide historical, current, and predictive views of business operations. Common functions of business intelligence technologies include reporting, online analytical processing, analytics, dashboard development, data mining, process mining, complex event processing, business performance management, benchmarking, text mining, predictive analytics, and prescriptive analytics. BI technologies can handle large amounts of structured and sometimes unstructured data to help identify, develop, and otherwise create new strategic business opportunities. They aim to allow for the easy interpretation of these big data. Identifying new opportunities and implementing an effective strategy based on insights can provide businesses with a competitive market advantage and long-term stability.
Business intelligence can be used by enterprises to support a wide range of business decisions ranging from operational to strategic. Basic operating decisions include product positioning or pricing. Strategic business decisions involve priorities, goals, and directions at the broadest level. In all cases, BI is most effective when it combines data derived from the market in which a company operates (external data) with data from company sources internal to the business such as financial and operations data (internal data). When combined, external and internal data can provide a complete picture which, in effect, creates an “intelligence” that cannot be derived from any singular set of data. Among myriad uses, business intelligence tools empower organizations to gain insight into new markets, to assess demand and suitability of products and services for different market segments, and to gauge the impact of marketing efforts.
BI applications use data gathered from a data warehouse (DW) or from a data mart, and the concepts of BI and DW combine as “BI/DW” or as “BIDW”. A data warehouse contains a copy of analytical data that facilitate decision support.
Business intelligence can be applied to the following business purposes:
- Performance metrics and benchmarking inform business leaders of progress towards business goals (business process management).
- Analytics quantify processes for a business to arrive at optimal decisions, and to perform business knowledge discovery. Analytics may variously involve data mining, process mining, statistical analysis, predictive analytics, predictive modeling, business process modeling, data lineage, complex event processing, and prescriptive analytics.
- Business reporting can use BI data to inform strategy. Business reporting may involve dashboards, data visualization, executive information system, and/or OLAP
- BI can facilitate collaboration both inside and outside the business by enabling data sharing and electronic data interchange
- Knowledge management is concerned with the creation, distribution, use, and management of business intelligence, and of business knowledge in general. Knowledge management leads to learning management and regulatory compliance.