Today the Bones Blog brings you a short interview with Greg Farshtey on the subject of focus groups, an often-misunderstood concept lately. What are the facts? Bold is for important points. Blue is me, black is Greg.
I've noticed lately that one of the cookie-cutter arguments complainers are using has been the anti-focus groups one. [Someone] just claimed in a topic for example that focus groups just like whatever LEGO throws at them in thanks for being chosen. He also said younger fans buy "anything".
That is a very common misconception of people who have never attended a focus group. Trust me -- I attended the initial groups we did for Dino Attack when the theme was just going to be hunting and trapping dinos -- beautiful sets, and the kids hated every one of them. They wanted the heroes to be attacking the dinos, so they got redone as more aggressive sets and sold great. Same with BIONICLE -- the kids are forced to make choices in these groups, of which sets of the group they like best and why. These are professionally run groups, managed by market research professionals outside of LEGO Company, so a kid who just says, "I like everything!" won't be allowed to get away with that answer.
Why are focus groups important?
Focus groups are important because, with any product, you need to gauge reactions from your primary audience. This is why movie studios do previews and have people fill out comment cards .... this is why TV producers have people watch their shows and turn a dial up or down when they like something or they don't. Every manufactuer just about uses some form of focus group or focus testing, whether they do it themselves or they hire an outside firm to do it. And all those BIONICLE sets everyone on here loved in 2001 and 2002 all came out of focus groups too.
What's the basic process of choosing focus group members?
LEGO contacts our market research firm, tells them what we want to test, how many sessions we want, how many kids in what age range, and what level of enthusiasm we want them to have for the line (heavy users, medium users, light users, and sometimes even rejecters, though not often). They then recruit the kids, so we have nothing to do with the selections directly. They also conduct the groups and present a debrief when they're done, we just observe from another room.
How important are their reactions to the final product compared to other things like past sales or just set designer's instincts or whatnot?
Designer instincts and past sales tend to go into the initial design phase. And they do play a part in that you don't automatically change everything you hear a criticism about -- some things have to stay the same for manufacturing reasons, etc. Also, you pay more attention to comments you hear from multiple groups, not just comments from one or two. If you have, say, 8 or 12 sessions over three days (which is common) with different groups of kids each time, and they all pick the same option or they all hate the same thing, you have to pay attention to that.
Do they like "anything"? It would be pointless to bother with them if they just liked everything, right?
You're right, it would be pointless. But that's not the case. It's pretty easy to spot kids who aren't really into it or aren't thinking about their feedback, and less weight is given to their comments. Most of these kids take their job very seriously and they get pressed to defend their comments. If they like something, or they dislike something, they have to say why. If they think something could be better, they have to say how it could be better. The difference between a focus group and an online forum is that focus groups are pushed to be as constructive as possible in their comments, where online posters can just say, "That stinks!" and no more. Because of that, we get a lot more valuable feedback from the groups.
I would also add that focus groups are not just used for sets -- they are also used for TV ads, magazine redesigns, online and offline game testing, etc.
Are focus groups infallible? No, they're not. But what I see when I see people screaming about them is that they are basically saying, "They should listen to me and not them" -- in other words, the complainers don't object to focus groups, they just want to BE the focus group.
In my observation, this is more of a "grasping at straws" tactic that some complainers have started using. Some of it might be taste discrimination; implying that the focus groups and thus the tastes of most fans are somehow inferior to the complainer in question, or it might simply be an assumption that the focus groups aren't accurate.
If it's taste discrimination, I've already addressed this fallacious, but somewhat understandable mistake here: Taste Discrimination Fallacy, Equality of Taste.
If it's a question of accuracy, there's some huge problems with that.
1) Complainers don't have magical access to what the majority wants anymore than anybody else. Just because we might wish we were the focus group doesn't mean our tastes would be more accurate to the majority. In fact the opposite is often true; online fans are often very different from most fans, in any franchise.
2) Random sampling is a proven technique for accurate polling in all areas, including political polling and in fact any kind you can think of. Naturally, the majority will most often be the ones picked.
3) Many of the trends in focus group opinions were represented vocally on BZPower before several problems were fixed. Gear unpopularity, and more "violence/darkness" come to mind. The latter fits Greg's example from Dino Attack too -- so before the problem was fixed, the "complainers" basically were the same taste-group as the focus groups.
4) LEGO uses an independant firm, and the system inherently requires intelligent responses. It's not the simplistic "LEGO shows a few kids the prototypes and they all say 'I likey!'" that complainers seem to have in mind.
5) Most importantly, as LEGO has improved sets more and more, listening to focus groups included, sales have gone up.
Either way, I've seen the argument come up seemingly because almost all other illogical complaints have been so thoroughly debunked, while this one hasn't. Those using it display an almost shocking ignorance of what focus groups are exactly and how they work -- ignorance is not necessarily something to be ashamed of, but if you're going to propose that something is a problem, you should be familiar with its details. Yet those using it seemingly have made no effort to find out, so it seems more like a convenient excuse to keep complaining.
Hopefully, this entry will help clear up such confusion.
Ultimately, the important lesson is that NO illogical argument should be used to say "I don't like this; LEGO shouldn't have done it." We're all free to dislike sets, but that's a matter of taste, not a matter of what LEGO actually should do. Just because you don't enjoy something doesn't mean it's a "problem" -- because you're not the only fan.
Instead, we should seperate our personal tastes from our opinions about what LEGO should do. So if focus groups' tastes tend to be different from yours, you should be mature enough to say "I wish focus groups and most fans shared my tastes, but this is reality and I can deal with it. I agree LEGO should pay attention to focus group results."
Now, if you can show some problem with focus groups that none of us have thought of, by all means, let's hear it. Otherwise, that argument ain't gonna fly no mo'.