|
26 QUESTIONS TO HELP RESEARCH BUYERS OF ONLINE SAMPLES
|
These questions, in combination with additional information, will help researchers consider issues which influence whether an online sampling approach is fit for purpose in relation to a particular set of objectives; for example whether an online sample will be sufficiently representative and unbiased. They will help the researcher ensure that they receive what they expect from an online sample provider.
These are the areas covered:
|
Company profile
1. What experience does your company have with providing online samples for market research?  |
|
This answer might help you to form an opinion about the relevant experience of the sample provider. How long has the sample provider been providing this service and do they have for example a market research, direct marketing or more technological background? Are the samples solely providedfor third party research, or does the company also conduct proprietary work using their panels?
|
Sample Source
Panel Recruitment
7. If the sample comes from a panel, what is your annual panel turnover/attrition/retention rate and how is it calculated?  |
|
The panel attrition rate may be an indicator of panellists’ satisfaction and (therefore) panel management, but a high turnover could also be a result of placing surveys which are too long with poor question design. The method of calculation is important because it can have a significant impact on the rate quoted.
|
8. Please describe the opt-in process.  |
|
The opt-in process might indicate the respondents’ relationship with the panel provider. The market generally makes a distinction between single and double opt-in. Double opt-in describes the process by which a check is made to confirm that the person joining the panel wishes to be a member and understands what to expect.
|
9. Do you have a confirmation of identity procedure? Do you have procedures to detect fraudulent respondents at the time of registration with the panel? If so, please describe.  |
|
Confirmation of identity might increase quality by decreasing multiple entries, fraudulent panellists, etc.
|
10. What profile data is kept on panel members? For how many members is this data collected and how often is this data updated?  |
|
Extended and up-to-date profile data increases the effectiveness of low incidence sampling and reduces pre-screening of panellists.11
|
11. What is the size and/or the capacity of the panel, based on active panel members on a given date? Can you provide an overview of active panellists by type of source?  |
|
The size of the panel might give an indication of the capacity of a panel. In general terms, a panel’s capacity is a function of the availability of specific target groups and the actual completion rate. There is no agreed definition of an active panel member, so it is important to establish how this is defined. It is likely that the new ISO for access panels which is being discussed will propose that an active panel member is defined as a member that has participated in at least one survey, or updated his/her profile data, or registered to join the panel, within the last 12 months. The type and number of sources might be an indicator of source effects and source effects might influence the data quality. For example, if the sample is sourced from a loyalty programme (travel, shopping, etc.) respondents may be unrepresentatively high users of certain services or products.
|
Panel and Sample Management