by Josh Bernoff
Since the competition was so intense this year for the Forrester Groundswell Award winners, I wanted to create as much transparency as I could around the judging process. This is in response to questions from some of the entrants, who clearly put an enormous amount of effort into their submissions.
Q. What exactly was the judging process?
Three people at Forrester were involved in the judging: Zach Hofer-Shall (a researcher on our interactive marketing team), Jeremiah Owyang, and me. We took a first pass to separate the most promising applications in each category, then we spent more time on those promising applications, looking for those which we thought were good enough to win. For an application to win, Jeremiah and I had to agree it was the best in the category. We were looking for both innovation and proof of business (or non-profit) results -- the winners in each category were excellent on both counts.
Q. It looks like some of the entries ended up in a different category from where they were submitted.
In working with clients, we have found that deciding on objectives is often difficult. Successful applications often start off meeting one objective, and end up delivering on others.
In this process, we looked at each application and asked, in what category would this application be most likely to win? Usually, but not always this was the category in which it was entered. But we made some changes as well. In the case of the Brooklyn Museum, there were three entries in different categories, but taken together we thought they demonstrated excellence in the Social Impact category, which is the category in which they won. "In the Motherhood" by LiveWorld was submitted for Supporting, but we thought it was an excellent Embracing application, since the actors in it acted out scripts suggested by the groundswell, and that's why it was a finalist in the Embracing category.
Just like last year, there was one company that stood out for its transformation. We felt this year's entries from Intuit were best recognized together, rather than as separate applications, which is why we gave Intuit the Company Transformation award. I don't know if we could make this a regular category, it just seemed to emerge from the other entries.
Q. How much weight did you give the evaluations from the community?
We were pleased at how many entrants sent their followers to our site, and the reviews that were generated. We all have to acknowledge, though, that the people coming to the site through that kind of promotion were much more likely to say very positive things. Partly because of the potential for this sort of gaming, we weren't going to pick the winners based on a popular vote.
However, many of the comments added insights that we might not have noticed otherwise. So the main value of the reviews, from my perspective as a judge, is that they helped to enlighten me. Taken together with all the other information in the entries and the sites they linked to, this gave us a clearer picture of what was outstanding (or in some cases, flawed) about the entries.
Q. Does this process favor large companies?
Absolutely not. It favors excellence. A small company can deliver outsized results with a small budget, as the Common Wealth Credit Union of Alberta did, and as the Brooklyn Museum did. National Instruments was a medium sized company with an incredible Supporting application. On the other hand, big companies and their agencies or vendors, like Hershey's with House Party, Mattel with Communispace, Starbucks with Salesforce Ideas, and Accenture deserve to be recognized for excellent work, too. It's not about how much you spent; it's about how effective and creative you were.
Q. What exactly does it mean to be a "finalist"?
Finalists are basically applications that could have won, except that somebody else was even better in the category. The 44 finalists this year were a lot, but they reflect the number of excellent entries. However, there's a clear distinction; the winners deserve to be recognized as the best of an excellent group.
Q. Will you do this again? What will you change?
We will definitely do the awards again in 2009. One thing we changed from last year -- making all the entires publicly available -- was a big plus, I expect that to continue. I expect us to make changes based on what worked well and what didn't this year; for example, I hope to start the process a little sooner to give people a longer time to prepare entries and vote on them. I welcome your suggestions.