Talent Toolbox

There may be times when the quantitative results yield a different outcome than the qualitative results. For example, in a 360 survey, the competency communication may have a positive score, yet the open-ended comments for that same competency may have negative undertone. A survey that includes both quantitative and qualitative questions can yield different results due to the inherent differences in the types of data collected and the analysis methods used for each. 

Quantitative questions typically involve closed-ended responses where respondents choose from pre-determined options or provide numerical ratings. These questions focus on measurable aspects and generate numerical data. The analysis of quantitative data involves statistical techniques such as averages, percentages, correlations, and regression analysis. This type of analysis aims to identify patterns, relationships, and trends within the data.  

On the other hand, qualitative questions involve open-ended responses, allowing respondents to provide detailed and subjective information. These questions delve into individuals' experiences, opinions, beliefs, and perceptions. The analysis of qualitative data involves a process of coding and categorizing the responses, looking for recurring themes and patterns. It often involves techniques like content analysis, thematic analysis, or discourse analysis. 

The differences in results arise from several factors:  

Data representation: Quantitative data is typically summarized using numerical values and statistical measures, such as means, standard deviations, or percentages. This allows for easy comparison and generalization. In contrast, qualitative data is represented through descriptive narratives or quotes, providing richer and contextualized insights but making it challenging to summarize or generalize. 

Different data collection methods: Quantitative questions typically employ structured surveys with closed-ended response options, while qualitative questions often involve open-ended responses. These methods capture distinct types of information and may elicit different perspectives from respondents (Creswell, 2013).  

Analysis techniques: Quantitative data is often analyzed using statistical methods to identify patterns, correlations, and significant relationships (Bryman, 2016). On the other hand, qualitative data requires thematic or content analysis to identify recurring themes, codes, and categories (Creswell, 2013). The differences in analysis techniques can lead to distinct interpretations and findings. 

Perceptual differences: Quantitative responses in a 360 survey often involve rating scales or numerical assessments that provide a quantitative measure of leadership competencies. These ratings can be influenced by various factors, such as rater bias, differences in perception, or personal relationships between the leader and raters. On the other hand, qualitative responses in the form of open-ended comments allow raters to provide subjective insights, examples, or anecdotes about the leader's behaviors or performance. These qualitative responses can capture more nuanced and contextualized information, highlighting specific incidents or patterns that may not be apparent in numerical ratings. 

Due to these differences, quantitative and qualitative questions may generate contrasting findings. For instance, quantitative data might reveal statistical differences between groups or highlight numerical trends, while qualitative data may provide rich contextual details, individual perspectives, or unexpected insights that cannot be captured through numbers alone.  

What you can do if you see differences: 

• Dig deeper. Interview survey participants to gain a better understanding of why the difference in scores may have happened.  
• Understand that there may be survey takers that give the highest ratings across all questions that could cause that discrepancy between the quantitative and qualitative questions.  
• Focus on the big picture. For example, look for overall patterns.  

Citations 

Creswell, J. W. (2013). Qualitative Inquiry & Research Design: Choosing among Five Approaches (3rd ed.). Thousand Oaks, CA: SAGE. 

Bryman, A. (2016). Social Research Methods (5th ed.). London: Oxford University Press. 

About EDA, Inc. 

EDA, Inc., a world-class human capital firm with powerful Employee Experience platform, providing SaaS and AI solutions, premium executive development services and leading industry research. EDA’s services include.

• C-Suite & Executive Coaching,
• Custom Executive Development,
• Succession Planning,
• Assessment & Selection Services, and
• High-Potential Development.

It is our pleasure and passion to help organizational leaders create conducive workforce environments so that they can attract, retain, and develop top talent and advance their mission and strategy.