Sampling errors

A Sampling error is a statistical error that occurs when an analyst does not select a sample that represents the entire population of data and the results found in the sample do not represent the results that would be obtained from the entire population. Sampling is an analysis performed by selecting a number of observations from a larger population, and the selection can produce both sampling errors and non-sampling errors.

Sampling error can be eliminated when the sample size is increased and also by ensuring that the sample adequately represents the entire population. Assume, for example, that XYZ Company provides a subscription-based service that allows consumers to pay a monthly fee to stream videos and other programming over the web. The firm wants to survey homeowners who watch at least 10 hours of programming over the web each week and pay for an existing video streaming service. XYZ wants to determine what percentage of the population is interested in a lower-priced subscription service. If XYZ does not think carefully about the sampling process, several types of sampling errors may occur.

Examples of Sampling Error

A population specification error means that XYZ does not understand the specific types of consumers who should be included in the sample. If, for example, XYZ creates a population of people between the ages of 15 and 25 years old, many of those consumers do not make the purchasing decision about a video streaming service because they do not work full-time. On the other hand, if XYZ put together a sample of working adults who make purchase decisions, the consumers in this group may not watch 10 hours of video programming each week.

Selection error also causes distortions in the results of a sample, and a common example is a survey that only relies on a small portion of people who immediately respond. If XYZ makes an effort to follow up with consumers who don’t initially respond, the results of the survey may change. Furthermore, if XYZ excludes consumers who don’t respond right away, the sample results may not reflect the preferences of the entire population.

Sample Size and Sampling Error

Given two exactly the same studies, same sampling methods, same population, the study with a larger sample size will have less sampling process error compared to the study with smaller sample size. Keep in mind that as the sample size increases, it approaches the size of the entire population, therefore, it also approaches all the characteristics of the population, thus, decreasing sampling process error.

Non-Sampling Errors

A non-sampling error is an error that results during data collection, causing the data to differ from the true values. Non-sampling error differs from sampling error. A sampling error is limited to any differences between sample values and universe values that arise because the entire universe was not sampled. Sampling error can result even when no mistakes of any kind are made. The “errors” result from the mere fact that data in a sample is unlikely to perfectly match data in the universe from which the sample is taken. This “error” can be minimized by increasing the sample size. Non-sampling errors cover all other discrepancies, including those that arise from a poor sampling technique.

Non-sampling errors may be present in both samples and censuses in which an entire population is surveyed and may be random or systematic. Random errors are believed to offset each other and therefore are of little concern. Systematic errors, on the other hand, affect the entire sample and are therefore present a greater issue. Non-sampling errors can include but are not limited to, data entry errors, biased survey questions, biased processing/decision making, non-responses, inappropriate analysis conclusions and false information provided by respondents.

While increasing sample size will help minimize sampling error, it will not have any effect on reducing non-sampling error. Unfortunately, non-sampling errors are often difficult to detect, and it is virtually impossible to eliminate them entirely.

Methods to Reduce Sampling Error

Of the two types of errors, sampling error is easier to identify. The biggest techniques for reducing sampling error are:

(i) Increase the sample size.

A larger sample size leads to a more precise result because the study gets closer to the actual population size.

(ii) Divide the population into groups.

Instead of a random sample, test groups according to their size in the population. For example, if people of a certain demographic make up 35% of the population, make sure 35% of the study is made up of this variable.

(iii) Know your population.

The error of population specification is when a research team selects an inappropriate population to obtain data. Know who buys your product, uses it, works with you, and so forth. With basic socio-economic information, it is possible to reach a consistent sample of the population. In cases like marketing research, studies often relate to one specific population like Facebook users, Baby Boomers, or even homeowners.

Methods to Non- Reduce Sampling Error

(i) Thoroughly Pretest your Survey Mediums

As discussed in the example above, it is very important to ensure that your survey and its invites run smoothly through any medium or on any device your potential respondents might use. People are much more likely to ignore survey requests if loading times are long, questions do not fit properly on their screens, or they have to work to make the survey compatible with their device. The best advice is to acknowledge your sample`s different forms of communication software and devices and pre-test your surveys and invites on each, ensuring your survey runs smoothly for all your respondents.

(ii) Avoid Rushed or Short Data Collection Periods

One of the worst things a researcher can do is limit their data collection time in order to comply with a strict deadline. Your study’s level of nonresponse bias will climb dramatically if you are not flexible with the time frames respondents have to answer your survey. Fortunately, flexibility is one of the main advantages to online surveys since they do not require interviews (phone or in person) that must be completed at certain times of the day. However, keeping your survey live for only a few days can still severely limit a potential respondent’s ability to answer. Instead, it is recommended to extend a survey collection period to at least two weeks so that participants can choose any day of the week to respond according to their own busy schedule.

(iii) Send Reminders to Potential Respondents

Sending a few reminder emails throughout your data collection period has been shown to effectively gather more completed responses. It is best to send your first reminder email midway through the collection period and the second near the end of the collection period. Make sure you do not harass the people on your email list who have already completed your survey! You can manage your reminders and invites on FluidSurveys through the trigger options found in the invite tool.

(iv) Ensure Confidentiality

Any survey that requires information that is personal in nature should include reassurance to respondents that the data collected will be kept completely confidential. This is especially the case in surveys that are focused on sensitive issues. Make certain someone reading your invite understands that the information they provide will be viewed as part the whole sample and not individually scrutinized.

(v)  Use Incentives

Many people refuse to respond to surveys because they feel they do not have the time to spend answering questions. An incentive is usually necessary to motivate people into taking part in your study. Depending on the length of the survey, the difficulty in finding the correct respondents (ie: one-legged, 15th-century spoon collectors), and the information being asked, the incentive can range from minimal to substantial in value. Remember, most respondents won’t have an invested interest in your study and must feel that the survey is worth their time!