ONLINE RESEARCH 2009
ONLINE PANELS AND BEYOND
CHICAGO / 26 - 28 OCTOBER
Conference review: Window into 2010
by Ray Poynter
- Listening
- Communities
- LinkedIn and Facebook - the shape of the future?
- Quality and Compliance
- Mobile
- Hype or Transformation
- In conclusion
Set in Chicago, in an unusually mild October week, a well attended (over 230 people from 28 countries) ESOMAR Online Conference shone a spotlight on the changes we are likely to see in 2010 and beyond. Rather than run chronologically through the Conference, this report addresses the implications of the conference thematically.
Listening
The single biggest message from Chicago was that researchers need to Listen, which in this context means using observational techniques. The importance of listening was highlighted by an excellent presentation by Annelies Verhaeghe from Belgium and Emilie van den Berge from the Netherlands called “Getting answers without asking questions”, which was a case study looking at how web scraping technologies were used to extract over 70,000 comments from social networks about the Dutch version of the TV show X factor. One great aspect of the case study was that it showed how the information had been used to change the show.
The listening theme was further developed by Karina Besprosvan and David Oyarzun, from Chile, who presented the progress they have been making in the use of Twitter as a tool for research, in their paper “Tweetmiotics”. Their case study looked at collecting two million tweets, sampling 200,000 and analysing them to produce insight.
The listening topic raised a number of issues, including: what technologies are best? What are the ethics of extracting comments? How should we process millions of comments? What does representativity mean in this field? Indeed, is it qual, quant, or something else?
Twitter played other roles in the conference, beyond being a source of passive data collection. Many of the delegates tweeted their way through the conference, sharing thoughts and pictures. To get an idea of what it was like to be there, enter #esoc into the search field of Twitter (you don’t even need to be a member). In addition, the Tuesday night saw a Tweet-Up at the Luxbar, where over 40 researchers gathered to extend the social side of the conference, in an initiative organised by Brian LoCicero using twtvite. One of the interesting things about this use of Twitter (both during the day and at the Tweet-up) was the way that it managed to involve people who were not attending the conference.
Communities
Several of the presentations, and much of the discussion, made it clear that communities have become a mainstream technique, at least amongst the avant-garde. Over the last couple of years many conference papers have been looking at ‘exploring’ the idea of communities. However, this year the Conference saw papers that had moved on to more advanced issues, such as a taxonomy of communities from James Kennedy (“Online community platforms”), a cross-cultural review by Manila Austin (“Cultural differences: a draw or barrier?”), a study of community member’s views from Australia (“It works for us but does it work for them?”), and a Swarovski case study of innovation communities by Volker Bilgram (“How to be successful in co-creation research?”).
Although more than 90% of the delegates appeared to be fans of communities as a research tool (based on a highly scientific raising of hands and chanting of ‘I believe’) a very interesting query was raised by Bill Blyth, who asked whether there was, or could be, a business model that would generate sufficient revenue from Communities, at an industry level.
It is also worth noting that the two papers that were nominated for the ESOMAR Excellence Award were both about communities, i.e. "Optimizing engagement in multinational online communities" and “How online research communities work for consumers invited to participate".
LinkedIn and Facebook - the shape of the future?
Perhaps the most exciting and most scary contribution was the session with Daniel Shapero of LinkedIn and Sean Bruich of Facebook (who has the interesting and revealing job title of Monetization Analytics). The session was moderated by Tom Anderson and illustrated the amazing potential that Facebook and LinkedIn present in terms of being able to reach potential respondents, especially given the massive amount of information held by these social networks. This data could allow targeting researchers to reach tightly defined groups and avoid having to ask the traditional profiling type questions. However, the session made many delegates wonder what would happen if social networks decided to compete directly with market research?
Both speakers made the point that they could not imagine allowing researchers direct access to their members, because they were concerned to protect their members from abuse, a telling indictment on many research surveys and approaches.
Daniel Shapero of LinkedIn and Sean Bruich of Facebook panel discussion with Anderson Analytics’ Tom Anderson.
Quality and Compliance
Perhaps the most important session was the one that elicited the least amount of discussion, namely the issue of quality and compliance. There are a small, dedicated team of people who are working on behalf of the market research profession/industry on a range of compliance/quality issues. This work is not sexy, the results are often criticised by people who have not contributed to the process, but it is vital to the future of market research.
The Conference had important presentations from four people who have put in great work in this area. Kees de Jong talked about the ESOMAR Internet guidelines that are currently being updated (see presentation), Joel Rubinson talked about the ARF initiatives, Bill Blyth talked about the progress that has been made with ISO standards and the fact that clients are going to start requiring suppliers to be accredited, and Brian Fine presented findings illustrating how the validity of online panels can be improved.
The challenge for the people working on our behalf in this field, for ESOMAR, and indeed for all of us, is to determine how to disseminate this information and to make it accessible. One of the challenges is that most of the people attending the conference do not deal with the compliance /quality issues on a daily basis. Most of these decisions are made by COOs, CIOs, CTOs, procurement departments, or compliance teams. Researchers need to know about them, they need to lobby the decision makers in their companies to address these issues, but in order for this to happen we need to re-visit the way we communicate it to researchers. For example, the decision to go for ISO accreditation is not normally made by researchers, indeed even the choice of which panel company to use is not often left to the discretion of individual researchers.
Mobile
The discussion about mobile research is beginning to suffer from the fact that it has been billed as ‘the next big thing’ for several years now, without showing any great inroads into data collection. Steve Lavine took the Conference through what has happened to date, with his presentation “Mobile Interviewing”, highlighting the many niches where mobiles have made inroads, for example in exciting areas such as e-ethnography.
Chris Ferneyhough took a different approach to the discussion about mobiles, in his paper “Best practices in mobile research”, concentrating on how many online surveys are already being completed using smartphones, for example iPhones, and he presented a study showing very similar responses between those using PCs and those using smartphones.
It may well be that in the next few years the key importance of mobiles will be as the medium by which people connect to communities, to Twitter, as a tool of e-ethnography, and as a part of other observational research, rather than as a medium for collecting large amounts of survey data.
Hype or Transformation
The first day finished with an exciting and well executed Pecha Kucha session, with three contributions at the conference (Steve August, Anthony Hamelle, and Tom Ewing), plus a video contribution from Lee Ryan. The topic for the Pecha Kucha session was intended to query whether Web 2.0 was truly transformational, simply a change, or just hype. The problem with this title, and perhaps with the whole Conference, was a shortage of cynics. Of the nearly 240 delegates no more than a handful were prepared to express doubt about Web 2.0, communities, passive research, and other new initiatives.
Perhaps ESOMAR needs to challenge the regional conferences, for example APAC and Latin America, along with its Congress, to challenge the issue of transformation or hype, because at these less focused conferences there might be more sceptics?
In conclusion
ESOMAR and the Programme Committee (Reg Baker, Michael Bartl, Monique Morden, Martin Oxley, and Steve Schwartz) are to be congratulated on the way they have revamped the Online Panels Conference into something new and more relevant to the new decade. The Panel Conferences had a clearly quantitative focus; the new title and format addresses both quantitative and qualitative issues, along with the debate about whether processing over 70,000 comments or 200,000 tweets is qual, quant, or perhaps something deserving a new title.
I am lucky to get to a large number of events each year and, for my money, this was one of the best Conferences of the year.
The wrap up panel moderated by Market Strategies International’s Reg Baker (Committee Chair), together with Michael Bartl from HYVE Monique Morden from Angus Reid Strategies and Steve Schwartz from Microsoft.
Do you agree that 2010 will be all about Listening? Answer Ray on his blog.