26 QUESTIONS TO HELP RESEARCH BUYERS OF ONLINE SAMPLES

This answer might help you to form an opinion about the relevant experience of the sample provider. How long has the sample provider been providing this service and do they have for example a market research, direct marketing or more technological background? Are the samples solely providedfor third party research, or does the company also conduct proprietary work using their panels?

The description of the type of source a provider uses for delivering an online sample might provide insight into the quality of the sample. An actively managed panel is one which contains only active panel members - see question 11. Note that not all online samples are based on online access panels.

The answer to this question may simplify the comparison of online sample providers in the market.

Combining panellists for different types of usage (like direct marketing) might cause survey effects.

The inclusion of hard-to-reach groups on the internet (like ethnic minority groups, young people, seniors etc.) might improve the quality of the sample provided.

The type of rewards and proposition could influence the type of people who agree to answer a questionnaire or join a specific panel and can therefore influence sample quality.

The panel attrition rate may be an indicator of panellists’ satisfaction and (therefore) panel management, but a high turnover could also be a result of placing surveys which are too long with poor question design. The method of calculation is important because it can have a significant impact on the rate quoted.

The opt-in process might indicate the respondents’ relationship with the panel provider. The market generally makes a distinction between single and double opt-in. Double opt-in describes the process by which a check is made to confirm that the person joining the panel wishes to be a member and understands what to expect.

Confirmation of identity might increase quality by decreasing multiple entries, fraudulent panellists, etc.

Extended and up-to-date profile data increases the effectiveness of low incidence sampling and reduces pre-screening of panellists.11

The size of the panel might give an indication of the capacity of a panel. In general terms, a panel’s capacity is a function of the availability of specific target groups and the actual completion rate. There is no agreed definition of an active panel member, so it is important to establish how this is defined. It is likely that the new ISO for access panels which is being discussed will propose that an active panel member is defined as a member that has participated in at least one survey, or updated his/her profile data, or registered to join the panel, within the last 12 months. The type and number of sources might be an indicator of source effects and source effects might influence the data quality. For example, if the sample is sourced from a loyalty programme (travel, shopping, etc.) respondents may be unrepresentatively high users of certain services or products.

The sampling processes for the sample sources used are a main factor in sample provision. A systematic approach based on market research fundamentals may increase sample quality.

Survey results can sometimes be influenced by the wording used in subject lines or in the body of an invitation.