Types of question: Structured/Close-end, Unstructured/open-end

Structured questions take many forms and include:

• Single response with nominal or ordinal categories (e.g. From the following list please select the category which includes your household income)
• Multiple responses (e.g. From the following list of pizza toppings please any or all that you regularly use)
• Scaled questions (e.g. The President is doing a good job: Strongly Agree to Strongly Disagree), and
• Numerous variations on these primary types.

Unstructured questions are a bit more qualitative in feel. They do not require pre-defined categories and they allow the respondent to express their views openly. This is their blessing and their curse. Open-ended questions, as they are also known, produce a higher cognitive load in the sense that the respondent has to think harder to come to an answer. This can create a lower response rate and sometimes lesser quality data. On the other hand, they can produce rich insights that provide depth and color to the black and white of structured questions.

Open-ended questions require additional time on the part of the researcher to analyze and code the responses although text-mining software is making this easier. For best results on a survey, keep open-ended questions to a minimum and use them as sub-questions driven by critical responses to a structured question. For example, if someone selects a high or low response to the Net Promoter Score, you can follow up with unstructured questions asking the respondent to elaborate on their score.

Open-ended questions

Open-ended questions are exploratory in nature, and offer the researchers rich, qualitative data. In essence, they provide the researcher with an opportunity to gain insight on all the opinions on a topic they are not familiar with. However, being qualitative in nature makes these types of questions lack the statistical significance needed for conclusive research.

Nevertheless, open-ended questions are incredibly useful in several different ways:

• Expert interviews
• Small population studies
• Preliminary research
• A respondent outlet

1. Expert interviews

Since questions that are open-ended ask for the critical thinking and uncut opinion of the respondent, they are perfect for gaining information from specialists in a field that the researcher is less qualified in. Example: If I wanted to learn the history of Ancient China (something I know very little about), I could create my survey for a selected group of historians whose focus is Ancient China. My survey would then be filled with broad open-ended questions that are designed to receive large amounts of content and provide the freedom for the expert to demonstrate their knowledge.

2. Small population studies

Open-ended questions can be useful for surveys that are targeting a small group of people because there is no need for complex statistical analysis and the qualitative nature of the questions will give you more valuable input from each respondent. The rule here is the group must be small enough for the surveyor to be able to read each unique response and reflect on the information provided. Example: A supervisor who is looking for performance feedback from his/her team of six employees. The supervisor would benefit more from questions that allow the respondents to freely answer rather than forcing them into closed-ended questions that will limit their responses.

3. Preliminary research

As stated in the closed-ended questions section, conclusive research usually requires preliminary research to be conducted in order to design the appropriate research objects, survey structure and questions. Open-ended questions can reveal to the surveyor a variety of opinions and behaviours among the population that they never realized. It is therefore, incredibly useful to use open-ended questions to gain information for further quantitative research.

4. A respondent outlet

It is usually a good idea in any survey, no matter how large, to leave an open-ended comments question at the end. This is especially in the case of a survey asking closed-ended questions on attitudes, opinions, or behaviours. Forcing respondents to answer closed-ended questions asks them to fit in your box of options and can leave them with extra information or concerns that they want to share with you. Providing respondents with the outlet of a comment box is showing them the respect they deserve for taking the time to fill out your survey.

There are a few drawbacks to open-ended questions as well. Though respondent answers are almost always richer in quality, the amount of effort it takes to digest the information provided can sometimes be overwhelming. That is why open-ended questions work best in studies with smaller populations. Furthermore, if your survey sample is a fraction of the population you are studying, you will be looking to find data which can be inferred on the overall population as statistically significant. Unfortunately, open-ended questions cannot be used in this manner, as each response should be seen as a unique opinion.

Closed-ended questions

Closed-ended questions come in a multitude of forms, including: multiple choice, drop down, checkboxes, and ranking questions. Each question type doesn’t allow the respondent to provide unique or unanticipated answers, but rather, choose from a list of pre-selected options. It’s like being offered spaghetti or hamburgers for dinner, instead of being asked “What would you like for dinner?”

Use closed-ended questions for the following:

• When your audience isn’t particularly interested in your survey topic
• When you need quantifiable data
• To categorize respondents

1. When your audience isn’t particularly interested in your survey topic

Closed-ended questions are easier to complete than open-ended questions. Why? Because closed-ended questions lay out all of the possible answers, removing respondents’ task of coming up with their own responses.

So, when you find yourself surveying an audience who may not be excited about what you’re asking them, air on the side of using closed-ended questions. It’ll give them an easier survey-taking experience and, in the process, provide you with a higher completion rate.

2. When you need quantifiable data

If you’re looking for statistically significant stats, closed-ended questions are the way to go. Going back to our earlier example, using a closed-ended question can help us arrive at stats like: 70% of respondents want to eat spaghetti for dinner versus 30% who prefer hamburgers.

Questions that are closed-ended are conclusive in nature as they are designed to create data that is easily quantifiable. The fact that questions of this type are easy to code makes them particularly useful when trying to prove the statistical significance of a survey’s results. Furthermore, the information gained by closed-ended questions allows researchers to categorize respondents into groups based on the options they have selected.

3. To categorize respondents

In other words, they allow you to conduct demographic studies. Why is this valuable?

Imagine that the manager of a designer clothing store believes that certain types of people are more likely to visit their store and purchase their clothing than others. To decipher which segment groups are most likely to be their customers, the manager could design a survey for anyone who has been a visitor. This survey could include closed-ended questions on gender, age, employment status, and any other demographic information they’d like to know. Then, it would be followed by questions on how often they visit the store and the amount of money they spend annually. Since all the questions are closed-ended, the store manager could easily quantify the responses and determine the profile of their typical customer. In this case, the manager may learn that her most frequent customers are female students, ages 18-25. This knowledge would allow her to move forward with an action plan on how to cater to this niche better or break into other target demographics.

The major drawback to closed-ended questions is that a researcher must already have a clear understanding of the topic of his/her questions and how they tie into the overall research problem before they are created. Without this, closed-ended questions will lead to insufficient options for respondents to select from, questions that do not properly reflect the research’s purpose, and limited or erroneous information.

For example, if I asked the question, “do you get to work by driving, busing, or walking?” I would have accidentally omitted carpooling, biking, cartwheeling or any other form of transportation I am unaware of. Instead, it would have been better for me to ask the open-ended question of “how do you get to work?” to learn all the different types of answer before forcing the selection based on a list of several options.

Types of question: Structured/Close-end, Unstructured/open-end

Structured questions take many forms and include:

• Single response with nominal or ordinal categories (e.g. From the following list please select the category which includes your household income)
• Multiple responses (e.g. From the following list of pizza toppings please any or all that you regularly use)
• Scaled questions (e.g. The President is doing a good job: Strongly Agree to Strongly Disagree), and
• Numerous variations on these primary types.

Unstructured questions are a bit more qualitative in feel. They do not require pre-defined categories and they allow the respondent to express their views openly. This is their blessing and their curse. Open-ended questions, as they are also known, produce a higher cognitive load in the sense that the respondent has to think harder to come to an answer. This can create a lower response rate and sometimes lesser quality data. On the other hand, they can produce rich insights that provide depth and color to the black and white of structured questions.

Open-ended questions require additional time on the part of the researcher to analyze and code the responses although text-mining software is making this easier. For best results on a survey, keep open-ended questions to a minimum and use them as sub-questions driven by critical responses to a structured question. For example, if someone selects a high or low response to the Net Promoter Score, you can follow up with unstructured questions asking the respondent to elaborate on their score.

Open-ended questions

Open-ended questions are exploratory in nature, and offer the researchers rich, qualitative data. In essence, they provide the researcher with an opportunity to gain insight on all the opinions on a topic they are not familiar with. However, being qualitative in nature makes these types of questions lack the statistical significance needed for conclusive research.

Nevertheless, open-ended questions are incredibly useful in several different ways:

• Expert interviews
• Small population studies
• Preliminary research
• A respondent outlet

1. Expert interviews

Since questions that are open-ended ask for the critical thinking and uncut opinion of the respondent, they are perfect for gaining information from specialists in a field that the researcher is less qualified in. Example: If I wanted to learn the history of Ancient China (something I know very little about), I could create my survey for a selected group of historians whose focus is Ancient China. My survey would then be filled with broad open-ended questions that are designed to receive large amounts of content and provide the freedom for the expert to demonstrate their knowledge.

2. Small population studies

Open-ended questions can be useful for surveys that are targeting a small group of people because there is no need for complex statistical analysis and the qualitative nature of the questions will give you more valuable input from each respondent. The rule here is the group must be small enough for the surveyor to be able to read each unique response and reflect on the information provided. Example: A supervisor who is looking for performance feedback from his/her team of six employees. The supervisor would benefit more from questions that allow the respondents to freely answer rather than forcing them into closed-ended questions that will limit their responses.

3. Preliminary research

As stated in the closed-ended questions section, conclusive research usually requires preliminary research to be conducted in order to design the appropriate research objects, survey structure and questions. Open-ended questions can reveal to the surveyor a variety of opinions and behaviours among the population that they never realized. It is therefore, incredibly useful to use open-ended questions to gain information for further quantitative research.

4. A respondent outlet

It is usually a good idea in any survey, no matter how large, to leave an open-ended comments question at the end. This is especially in the case of a survey asking closed-ended questions on attitudes, opinions, or behaviours. Forcing respondents to answer closed-ended questions asks them to fit in your box of options and can leave them with extra information or concerns that they want to share with you. Providing respondents with the outlet of a comment box is showing them the respect they deserve for taking the time to fill out your survey.

There are a few drawbacks to open-ended questions as well. Though respondent answers are almost always richer in quality, the amount of effort it takes to digest the information provided can sometimes be overwhelming. That is why open-ended questions work best in studies with smaller populations. Furthermore, if your survey sample is a fraction of the population you are studying, you will be looking to find data which can be inferred on the overall population as statistically significant. Unfortunately, open-ended questions cannot be used in this manner, as each response should be seen as a unique opinion.

Closed-ended questions

Closed-ended questions come in a multitude of forms, including: multiple choice, drop down, checkboxes, and ranking questions. Each question type doesn’t allow the respondent to provide unique or unanticipated answers, but rather, choose from a list of pre-selected options. It’s like being offered spaghetti or hamburgers for dinner, instead of being asked “What would you like for dinner?”

Use closed-ended questions for the following:

• When your audience isn’t particularly interested in your survey topic
• When you need quantifiable data
• To categorize respondents

1. When your audience isn’t particularly interested in your survey topic

Closed-ended questions are easier to complete than open-ended questions. Why? Because closed-ended questions lay out all of the possible answers, removing respondents’ task of coming up with their own responses.

So, when you find yourself surveying an audience who may not be excited about what you’re asking them, air on the side of using closed-ended questions. It’ll give them an easier survey-taking experience and, in the process, provide you with a higher completion rate.

2. When you need quantifiable data

If you’re looking for statistically significant stats, closed-ended questions are the way to go. Going back to our earlier example, using a closed-ended question can help us arrive at stats like: 70% of respondents want to eat spaghetti for dinner versus 30% who prefer hamburgers.

Questions that are closed-ended are conclusive in nature as they are designed to create data that is easily quantifiable. The fact that questions of this type are easy to code makes them particularly useful when trying to prove the statistical significance of a survey’s results. Furthermore, the information gained by closed-ended questions allows researchers to categorize respondents into groups based on the options they have selected.

3. To categorize respondents

In other words, they allow you to conduct demographic studies. Why is this valuable?

Imagine that the manager of a designer clothing store believes that certain types of people are more likely to visit their store and purchase their clothing than others. To decipher which segment groups are most likely to be their customers, the manager could design a survey for anyone who has been a visitor. This survey could include closed-ended questions on gender, age, employment status, and any other demographic information they’d like to know. Then, it would be followed by questions on how often they visit the store and the amount of money they spend annually. Since all the questions are closed-ended, the store manager could easily quantify the responses and determine the profile of their typical customer. In this case, the manager may learn that her most frequent customers are female students, ages 18-25. This knowledge would allow her to move forward with an action plan on how to cater to this niche better or break into other target demographics.

The major drawback to closed-ended questions is that a researcher must already have a clear understanding of the topic of his/her questions and how they tie into the overall research problem before they are created. Without this, closed-ended questions will lead to insufficient options for respondents to select from, questions that do not properly reflect the research’s purpose, and limited or erroneous information.

For example, if I asked the question, “do you get to work by driving, busing, or walking?” I would have accidentally omitted carpooling, biking, cartwheeling or any other form of transportation I am unaware of. Instead, it would have been better for me to ask the open-ended question of “how do you get to work?” to learn all the different types of answer before forcing the selection based on a list of several options.