Operational Performance Monitoring and the search for data

No video selected

Select a video type in the sidebar.

WHY PERFORMANCE MONITORING?

Transformation is about making significant changes to existing processes, structures and systems in order to achieve a desired improvement or outcome. It requires organizational commitment, courageous leadership and greater risk taking than incremental change. It often involves the whole system and may require difficult decisions that involve large investments of time, money and resources. Transformation usually seeks to make fundamental shifts in the way an organization operates and is often instigated by external forces such as competitive pressures or regulatory requirements. The ultimate goal of transformation is to create sustainable improvements in performance, customer service or competitiveness that cannot be realized through incremental change alone.

In the past, as today, businesses seek products, services, and offerings that allow them to remain ahead of their contenders. Continuous innovation, cost reduction and improved resource usage are just a few examples of better operational performance. One of the most well-known concepts of continuous improvement is Kaizen. Kaizen is based on the idea that small, ongoing positive changes can reap significant improvements. Typically, it is based on cooperation and commitment and stands in difference from approaches that use radical or top-down changes to achieve transformation.

In general, operational performance objectives are measures that we need to evaluate the efficiency of an entire process. We are looking to identify ways to improve operations, such as developing a product or service to the end of its life. This may include increasing quantitative output, lowering costs and waste, enhancing quality, mitigating safety risks, shortening innovation cycles and improving resource planning. Eventually, performance measures allow managers or teams to assess process inputs (the resources allocated) and outputs (direct results of process steps or service activities) to make an informed decision.

"What gets measured is what gets done" is still true today. Operational performance objectives describe and focus long term goals and strategic objectives; they also draw people's attention to what counts, what matters, and what is essential. Performance results are helpful for leadership and management because they are precise and actionable. Operational performance objectives have a directive function by focusing attention, interest, and eagerness on mission-relevant and goal-relevant actions. The connection between goals and performance has been demonstrated empirically: What gets measured gets attention. This can benefit the process already before operational improvements have been implemented or the measurement results are known (learn more about RTLS solution and how to improve operational performance).

Measurement helps our understanding of performance, both its intended and unintended aspects. Are things getting better or worse? What are the outcomes of a program or service, and how do the results stack up against the inputs? Replies to these simple questions are consequential for the manager/team willing to listen. Performance measures are also the most effective way of communicating with stakeholders about the success of products, services or improvement activities. The use of performance measures objectives that are easily understood, demystifies a process and allows them to know how well they perform. A good example is the tire industry and their quest for improvement - What sustainable development goals do they target?

BEFORE MONITORING: SMART OBJECTIVES

By establishing objectives, managers and teams give themselves a target to aim for. SMART is used to support and guide goal setting. Probably you know that SMART is an acronym that stands for Specific, Measurable, Achievable, Realistic, and Timely. Therefore, a SMART goal blends all of these criteria to help focus your efforts and increase the chances of achieving your goal.

SMART goals are:

Specific: Well specified, clear, and unambiguous (who-what-where-when-why?)
Measurable: With particular criteria that measure your advancement toward the accomplishment of the goal, including indicators, measure intervals and methods
Achievable: Attainable to achieve but still bold to make everyone feel challenged
Realistic: Within reach and relevant to the purpose
Timely: A starting and a target date give you a timeline to create urgency.

SMART goals set a group up for success by making goals specific, measurable, achievable, realistic, and timely. The SMART method provides direction and helps organize and reach the team's goals.

Working toward a SMART goal, it is time to look at measuring the progress in detail.

Dealing with legacy systems migration? Then continue reading here...

MEASURING OR HOW TO COLLECT DATA?

A SMART goal must be measurable to be comparable. Only when you can compare data you can detect discrepancies or, in other words: progress toward your goal. Comparison is possible when you use measures of quantitative assessment. Unfortunately, not all data are immediately available in this format. In science, we call this quantitative and qualitative research. When collecting and analyzing data, quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings. Both are important for gaining different knowledge, but the investigative team must transform qualitative research data first to be measurable.

Quantitative research is expressed in numbers and graphs. It is used to test or confirm theories and assumptions. This type of research can establish generalizable facts about a topic. Standard quantitative methods include experiments, observations recorded as numbers, and surveys with closed-ended questions.

Qualitative research is represented in words. It is used to understand concepts, thoughts or experiences. This type of research enables you to gather in-depth insights on poorly understood topics. Standard qualitative methods include interviews with open-ended questions, observations described in words, and literature reviews that examine concepts and theories.

Learn more about identifying assets with your smartphone through a handheld process

DATA COLLECTION METHODS - OVERVIEW

Quantitative and qualitative data can be collected using various approaches. It is important to note that every method will produce different results. Therefore, deciding when to use a specific data collection method is critical to answering your measurement question. A rule of thumb for determining whether to use qualitative research methods is if you want to understand something like a concept, thoughts, or experiences to formulate a hypothesis. Use quantitative research to confirm or test a theory or hypothesis.

You can choose a qualitative, quantitative or mixed methods approach for most research topics. Assume the following example: You want to know how satisfied your customers are with your services. Suppose you survey 300 customers and ask them questions such as: "on a scale from 1-5, how satisfied are you with our services?". In that case, this is a quantitative approach as you test a hypothesis (e.g. "Our customers are generally pleased with our services"). After collecting the data, you can immediately perform a statistical analysis and draw conclusions such as: "on average, customers rated our services 4.2".

Continuing the example, alternatively, you could conduct in-depth interviews with 15 customers and ask them open-ended questions such as: "How satisfied are you with our services?", "What is the most positive aspect of the customer success program?" and "What can be done to improve our online support?" Based on your answers, you can ask follow-up questions to clarify things. Before comparing the answers, you transcribe all interviews and try to find commonalities and patterns to indicate reasons for the outcome ("Customers like our services most due to the short answering time 24/5. They would love to have it 24/7.")

Mixing up methods is the preferred way to back-up data. For example, in a mixed-methods approach, you would conduct interviews to find out how satisfied customers are with your services, helping you get insights and formulating a hypothesis. Later, you use a survey to test these insights on a larger scale. It's also possible to start with a study to find out the overall trends, followed by interviews to understand the reasons behind the trends better.

Typically, quantitative data collection methods include

Surveys: List of closed or multiple-choice questions distributed to a sample (online, in person, or over the phone).
Experiments: Situation in which variables are controlled and manipulated to establish cause-and-effect relationships.
Observations: Observing subjects in a natural environment where variables can't be controlled.

Qualitative data collection methods use

Interviews: Asking open-ended questions verbally to respondents.
Focus groups: Discussion among a group of people about a topic to gather opinions that can be used for further research.
Ethnography: Participating in a community or organization for an extended period of time to closely observe culture and behaviour.
Literature review: Survey of published works by other authors.

Many data collection methods can be either qualitative or quantitative. For example, in surveys, observations or case studies, your data can be represented as numbers (e.g. using rating scales or counting frequencies) or words (e.g. with open-ended questions or descriptions of what you observe).

ANALYZING DATA

Qualitative or quantitative data alone can't prove or demonstrate anything, but it must be analyzed to show its meaning concerning the research questions. Therefore, the method of analysis differs for each type of data.

As quantitative data is founded on numbers, simple math or more advanced statistical analysis is used to uncover commonalities or patterns in the data. The results are often documented in graphs and tables. Applications such as Excel, SPSS, or R can be used to calculate things like:

Average scores
The number of times a distinct answer was given
The correlation or causation between two or more variables
The reliability and validity of the results

On the contrary, qualitative data is more challenging to analyze than quantitative data. This is because it consists of text, images or videos instead of numbers and must be made comparable. Nevertheless, some common approaches to analyzing qualitative data include:

Qualitative content analysis: Tracking the occurrence, position and meaning of words or phrases
Thematic analysis: Closely examining the data to identify the main themes and patterns
Discourse analysis: Studying how communication works in social contexts

MONITORING INDICATORS IN CONTEXT: UNDERSTAND OPERATIONAL PERFORMANCE

Once data is collected and analyzed, management and team still need one final step to gain the desired insights: the context. Data like a number is just a number. Its meaning (good or bad, important or trivial, expected or unexpected) is set externally. Its value is not inherent but rather depends on context. The context is what tells the story.

It is not enough to say that the customer satisfaction rate was high. The description "high" can mean any number of things. Your first reaction should be "how high?" and "high compared to what?" Focusing on an actual number only gets you part of the truth. Let's say the customer satisfaction was 92%. How does that compare to last year? It may be low if it was 96% last year, or it may be high if it was 82%. Or if in the same industry the value is below 80%, 92% looks good, whereas if the industry benchmark is above 95%, it looks less good.

So-called competitive benchmarking compares how well (or poorly) an organization is doing to the leading competition, especially for critically important attributes, functions, or values associated with the organization's products or services. For example, on a scale of one to four, four being best, how do customers rank your organization's customer care compared to the top competition?

Before an organization can achieve the full benefits of benchmarking, its processes must be comprehended. Because benchmarking requires significant investments of manpower and time, management must champion the process all the way through, including being ready and willing to make changes based on what is learned. Once data was collected as described above, your goal is to determine the gaps between your operational performance measurements and those of others or previous years. This practice should lead to the question of which differences in practices caused the gaps. The answers may be genuine insights or the starting point for further investigations: your next hypothesis.

TAKEAWAY

Monitoring operational performance and benchmarking should be a constant, automated task: You could compare historical and current data immediately, and on-demand based on your digital data feeds. But insights will be only valid if the database is accurate, representative and bias-free. Continuous collecting under these assumptions is tiring if you don't use sensors, machines, or computers to do the laborious work for you. Industry 4.0 applications offer a broad range of automated data collection in manufacturing.

If you are keen to uncover black boxes in your processes and generate monitoring data from them, you will have the chance to discover the true bottlenecks of your production line.