In our latest podcast episode, Market Research & Prototype Testing, our host is joined by our Research Director, Lang McGilp, as co-host to speak with episode guest, Ben Jacobson.
Ben is a partner at Conifer Research, a design research consultancy in Chicago that has worked in prototype testing with brands that include Samsung, Pepsi, Nestle Purina and BP.
In the episode, Duncan, Ben and Lang look into just what the practice to prototype testing is all about, Ben tells some stories and relates some of his experiences when it comes to the do’s and don’ts of prototype testing and looks forward to giving his ideas of where the practice of prototype testing, and market research in general, is going in the future.
If you’d like to learn more about Conifer Research and the work they do, check out their website. There, you’ll find a link to the Research RX tool that was described in the episode.
Insightrix Research releases new content regularly!
Check out the Insightrix blog roll for informative articles about everything and anything market research related.
You can follow Insightrix on social media on Facebook, Twitter and Instagram, and subscribe to our YouTube channel to make sure you never miss a video.
You can also access Stories of Market Research: The Insightrix Podcast on your favourite podcast player app, including Apple Podcasts, Stitcher and YouTube. Make sure to subscribe to be notified the moment a new episode is published!
Episode Transcript
[EPISODE INTRODUCTION] [DUNCAN] Prototype testing is a valuable step in the product or service development process.Whether it’s through co-ideation in development, or to align a product or service with the needs of their users, or to discover and define business opportunities for existing services or products, prototype testing can help companies learn a lot about how real users interact with products and services in real-world situations.
Hello, my name is Duncan McGregor, Marketing and Communications Coordinator at Insightrix Research in Saskatoon, Canada, and your podcast host.
On today’s episode, we’re joined by Ben Jacobson, partner at Conifer Research – a design research consultancy in Chicago, USA.
Conifer Research is a company that works heavily in prototype testing – having worked across a wide swath of industries with the design and innovation teams from companies like BP, McDonald’s, Nestle Purina, Pepsi and Samsung.
In the episode, Ben shares some of his experiences in prototype testing, how it has benefitted Conifer Research’s clients and the ways it has helped them – and – Ben relates some of the techniques his company has employed in the past to get their clients the insight they need.
Ben also talks about where he thinks the field of prototype testing, and market research in general, is heading and offers some suggestions for anyone looking to dive into the field.
And for this episode of the podcast, Insightrix own Research Director, Lang McGilp, sits in as co-host, lending his experience and research acumen to the discussion.
Do’s and don’ts of prototype research, observational research vs. big data and humanizing top box scores – all that and more, in this episode of Stories of Market Research: The Insightrix Podcast.
[THEME MUSIC] [INTERVIEW PT.1 BEGINNING] [DUNCAN] Hello. We’ve got Ben Jacobson, from Conifer Research from out of Chicago on the phone. How’s it going Ben? [BEN] I’m doing great, thanks. How are you? [DUNCAN] I’m doing great. Thanks for coming on. I’ve also got our own… [BEN] Thanks for having me. [DUNCAN] Oh no, it’s a pleasure. I’ve also got our own Research Director here at Insightrix, Lang McGilp, on the phone to co-host. So, thanks for going us.So, Ben is a partner at Conifer Research, a research agency that does a lot of work in prototype testing. This isn’t something we usually do a lot of here at Insightrix, so we wanted to have him on to talk about his company and the work you guys do. Is there anything you could tell us about Conifer Research and the types of prototype testing you do?
[BEN] Sure. My first question would be why don’t you do prototype testing up there? But besides that, my little company got started in 2001, and we really got started in all of this – I’m just going to give a shout out to one of our first patron clients, who back in the day was Steelcase Incorporated, run by a visionary CEO whose name is Jim Hackett. And Jim Hackett, of course, is now the CEO of Ford. And they have long sponsored observational research and ethnographic research both for new concept development but also testing all the iterations of the designs that they do. And so, we specialize in observational research, in-context interviewing, those sorts of things, which has a wide variety of uses. One of which is, certainly, prototype testing. And that’s really the first way that we got into it. [DUNCAN] Right on. How do you find that prototype testing benefits your clients? I know it’s a pretty broad question, but what are the most common or key areas that you help your clients with? [BEN] The first thing to kind of wrap our brains around as we talk about this is that testing, the word prototype can mean almost anything, right? [DUNCAN] Yeah. [BEN] It can literally mean some rubber bands and paper clips and pipe-cleaners and things that you’ve made in an ideation session. It can mean sketches and concepts. It could be a fully engineered and absolutely functioning piece of machinery, right? So, the range of what falls into that is quite large – and we love to get involved in all of those aspects of it. We don’t just specialize in one teeny, tiny bit.And, I guess the key thing to think about is it’s just dramatically cheaper, more interesting and insightful to identify the joys and pains that you’re about to set loose in the world at a small scale rather than at market scale after launch. So, that is probably the number one thing that people out there who are thinking about this should keep in mind, is that this is, sort of, something that you absolutely should be doing.
And I guess you said, the other one was – how does it help my clients? Well, so, I mentioned, we save them valuable time and money by getting things as close to right as possible before they sink a lot of money into engineering and development. And I can tell you some stories about that later. But, sometimes, it even becomes obvious that an idea should be radically transformed, repositioned or even killed outright based on what you can learn in a prototype test. And if you talk to people in business, this is probably the thing that doesn’t get accounted for very often but should be. Which is, how many times did you save yourself from embarrassment in the market? Or building something at scale and launching it, and then having to write it all off two quarters later? Things like that – and when you think about those epic fails, usually at the bottom of that is some kind of vacuum when it really came to understanding how this was going to function in the real world. And I guess the third thing that I would add in that sort of basket of topics is it’s really important to have a different set of eyes on the design and development as you move things from concepts into offerings out there in the world. If you let an internal design team or an external design firm who created the concept do the testing, it often, I think, introduces a whole lot more bias into the process than I personally think is healthy.
[LANG] Yeah, we do quite a bit of advertising/concept kind of testing, and I find a similar approach, where sometimes just being that extra party at the table, that’s not involved in creating or taking an idea and putting it into something for consumers to absorb, gives you that chance to be a little more objective and that kind of thing and ensure that what you’re showing people aligns with the intended outcome. [BEN] Oh yeah, Lang – I mean, that gets multiplied, right? I mean, the multiplier effects on this kind of this are so interesting, right? So, it’s at the table with your client and their stakeholders, etc., but even in the conduct of the research. If you’re, in any way shape or form, the people that are engaged in a test or a concept test, something like that, you’ll have an idea about who this is for, and those people are sitting in the room, you immediately introduce this whole sense of how you please the people who are paying you and feeding you M&Ms, right? And it’s really difficult to get out of that unless it’s really clear that you’re an unbiased third party. [LANG] Interesting. Do you, and I wonder if the barriers might be the same with some of the work that we do, but what are some of the challenges that you find with clients who maybe want to just take the phrase “down and dirty” – just something simple. Give me the basic. Or, maybe I’m struggling to convince people to even bother with this and just move things down – move the chains down the field a little bit more on our own. What are some of those barriers that you find that you need to convince your client or they need to convince their internal client this is worth taking the time and a little bit of money to invest and figure things out? [BEN] Well, when you have the formula for that, will you let me know? Because I think that would unlock a lot, right? So, I guess that there’s a couple of things. One is I like to think about it as really honest conversation with people. And it’s hard to do when you don’t even know them, right? When people send you an RFP through the email and they’re asking for something, and you don’t know them. And it’s hard, as well, when you know someone and you have multi-year relationships with people and they’ll say we want to do X, Y and Z, and we’ll say that seems like a good idea or a bad idea. But it’s hard to speak truth to the people who would write you a cheque. And I think we all have to keep that in mind and try to check our egos at the door and be as faithful as we can to what’s the good, right and true thing that should be happening. I like to remind people or even just ask them in the form of questions, how much money are you are you spending to take this from me, let’s say they use a staged gate or some crazy scrum model or whatever, right? Now how much money are you spending to go to move the chains down the field? And how many times have you already done that without checking your assumptions or without exposing this to the people who are going to use it? Because the further you go, the more expensive that really is. And there’s a ton of cost involved in that that people think isn’t real. And people will tell me all the time. They are like, Oh well, this is an innovation project and it has to come up with a new idea, a concept, a service, whatever – that has at least 100 million dollars a year in incremental income to the firm. And then I’m thinking, OK, 100 million dollars is your goalpost and you don’t want to spend $100,000 to figure out whether or not your moving in the right direction. And I think that they have their math. They went to business school, I didn’t. But in my mind, there’s definitely times and places along the way when you need to check things out and make sure that you’re not making some goofy, horrible mistake. And the other part of it is not just the mistakes, but the opportunity to make it better. I mean, not just not bad, right? You can learn things along the way by iterative processes that really amp up the quality and the experience of the final offering at the end. And if you think that you’ve got it all right, from the Post-It note you stuck on the conference room wall all the way through to launching something into the world, that’s arrogance. And that kind of arrogance usually ends badly. [DUNCAN] So, could you give us an example or two of how you’ve used your methodologies to assist your clients? [BEN] Oh God, yeah. We could talk all day about that. [DUNCAN] Awesome! [BEN] I know you don’t have that much time, but we’ll talk about a few things – and shut me up if I’m not making sense or if I’m causing you pain… [DUNCAN] Okee-dokie. [LANG] Oh no, we’re in. This is very interesting. [BEN] …or anything like that. So, here’s a fun one and it’s an oldie but a goodie. Right? So those are kind of fun. So, we helped a large, not-to-be-named department store brand do a test of some price-check machines. You guys know what I’m talking about? [DUNCAN] Yup, you bet. [LANG] Yeah. [BEN] Yeah, those little scanners. They’re all through the store and you know, you slide the barcode under there and this costs X, Y, Z, right? So, first you have to realize that price-check machines were not even close to a new thing in the world when we were doing this. All their competitors had them. And so, it was an internal, emotional problem they were having with making the decision to roll these out to all of their stores. And so, on the good side, it was really easy for them to put their hands on some functional prototypes and put them in a store, which they chose to do literally in a small town in Oklahoma so nobody would find out. And so we put these into five different formats of their stores, which at the time was like overkill, right? Why do you need to go to that length? But they were insistent that they had to do this. And they were worried that the machines would disrupt their shoppers and change their usual purchase behaviour or even cause confusion. So, we set up the machines in real store environments, using fixed video cameras to record behaviour and interaction with the machines. We hung around in the various departments. After people used the machines, we would approach them and interview them about the experience of using the machine, etc., etc. So, that was fairly straightforward, but it got really, really interesting when things started going down the road. So, the thing is, on these machines, they had, of course, a little, small screen that would display the price and some information. They had the scanner with the funky, little laser. And, oddly enough, they had a series of simple, little black buttons on the front – and the buttons had no function, but if you pushed them, they went beep. [LANG] No way. [DUNCAN] So classic. [BEN] I wish I could get the right beep, right? But it was a charming little beep and etc. So, but nothing would happen, right? It just went beep. So, when you present people with a button, many, many of them will push it. [LANG] My kids do that to me all the time. [BEN] Of course, right? That’s how they learn the power button on every device in your home before they’re one year old. They’re going to push the button. They’ve seen you push buttons; they want to push the buttons. You can imagine what the kids in the store – they were going crazy with the buttons. But, either way, when someone pushes a button, they have ideas in their head. They have expectations and assumptions that are connected to some interesting mental model about how they think things work – and even what they think might be possible. And if it beeps, then those expectations are heightened because they already got some kind of response. And so, you know, what we started to discover was that as we were watching the video, we’d see people push the buttons. We’d see them hear the beep. They would look at the screen kind of funny – you’d see little changes in their body language, etc. And then, they would walk off. Then we would interview them. And so, one of the things that we discovered was that what they asked us for, what our clients asked us for, which was people like price-check scanners, especially in stores where there’s always this rolling markdown of prices and that’s part of the brand of the store – so, it’s an unmitigated positive. That was the easy answer. But the really interesting part of this came with the ideas that people talked about when we started to dig around a bit in what they thought might happen when they pushed those buttons. And we wound up with a very forward-looking set of ideas that came out of this work. Literally, just what shoppers thought might be possible through this digital interaction that they were having in the middle of a retail environment. And so, that’s a bit of a goldmine – an unanticipated goldmine. And that’s why I like the story because it really shows you can get a variety of very tactical answers to questions you might have about a prototype, but also introducing something like a button that goes beep or does nothing or what have you, is an interesting way to tap into these future-oriented insights about what your customers are ready for, and maybe already expecting from your brand. [LANG] Interesting. I’ve got a quick question for you here. So, knowing that you’re primarily focused on this prototype testing, have you – like, what are some of the benefits that you often explain to your potential clients? Is this is the benefit of having something physical, hold in the hand or in the store or in the whatever kind of real environment as opposed to perhaps something more conceptual, maybe written on paper, maybe part of a survey. So, people aren’t really – it’s not confronted and it’s not really right in front of them and it’s… What kind of the thing that you often mention to tell people that this is an added value? [BEN] Well, I’ve two things. I think that anything that goes beyond a word or a set of words is incrementally better, and so, I would never pooh-pooh fun sketches on a napkin. Or even non-designer-type sketches that end users or consumers would do in a co-creation or an ideation session. There can be great things to learn from that, but you wouldn’t want to necessarily think about the big processes – thumbs up/thumbs down, right? This is not, “Do we go forward or not?” It’s, “What did we learn from this and how would we incorporate this into the next iteration?” So, that’s my caveat to what my answer is. On the other hand, leaving things only in words is a failure mode. And I do this with my clients all the time. I’ll say, “Everybody take out a Post-It note and write this word down on it.” And it’ll be a word like – here’s my favourite one is collaboration. OK, so, we put the word “collaboration” up on a whiteboard because that’s where all good ideas land – on a whiteboard with a Post-It note that says “collaboration”. And then, what’s fascinating is, what does that mean, right? And very often, people will go around and they distill and they distill and they come down with this vision of a thing and it’s going to be a collaboration zone or a collaboration technology, or whatever it is. It’s the new standard bearer for what your new innovation project is going to be all about, but everybody has a completely different idea of what that means. And they could all be right, but if it was your job to build that thing, you would be way off – probably most of the time. And when you ask people to write down what it means, for some people, it means, well, we shared writing a document. We passed that document back and forth through email until it was finished and then we collaborated and that was the result of our collaboration. For other people, it’s literally we need a bunch of Toobers and Zots and fun playthings on a table and lots of whiteboard space, and we’ll roll up our sleeves and have some fun music playing in the background – that’s collaboration. For other people, it’s a conference call. So, what are you making? And if you leave it in the word, they’re just certain words and people love those words because they’re kind of sexy and fun and interesting, but they hide more than they reveal. And so, the minute you go from a word that’s kind of fuzzy and fun to talk about to something tangible, people go, “Oh, that’s not what I thought you meant.” And it’s horrifying – I’ve seen teams walk out of a meeting and come back a month later and they’ve all been working on different things because they had one word in their head and they were taking it in different directions. So, visualization is key to success, and the more tangible something gets as you take it from a Post-It note to a napkin sketch to a concept story or a storyboard, all the way through to something that humans are going to interact with in some capacity, there are multiple opportunities – and they don’t have to all be big and expensive. You can do things that are quick and not dirty, as long as your expectations are equally adjusted to what you need to get out of it and what you might get out of it if you’re lucky. Anyway, probably a long answer to your very good, short question. [LANG] No, it’s really interesting. [DUNCAN] It’s a great answer. [LANG] You’d mentioned co-creation there a little bit and that’s something that, I think, there’s been some speaking of as well in our industry. [BEN] Yes. [LANG] Can you fill us in on co-creation. Like, how you actually harness people and get them engaged on something which may not necessarily be sexy or an interesting topic to the consumer, but it’s really obviously very important to your client, so that you actually – I guess my ultimate question is through co-creation, can you unlock the door to greater brilliance than more of a traditional approach? [BEN] I would say the answer is probably yes. And I think that what falls under the bucket of co-creation is also large. So, what I’ve found to be pretty successful in this regard is to get people together in a setting that is, if not real, then is at least making a solid, playful effort at creating something of an immersion for them to get them into the right cognitive and emotional state. And so, we often do co-creation activities with people towards the end of projects where we may have done a bunch of initial fairly open-ended research, we’ve come up with some general opportunity spaces for people, for our clients to innovate in. And then we might bring end users or consumers in, and what we’re probably going to do is – here, I’ll give you an example. We did one on cooking and mixing and measuring and like really basic stuff. Like, when you think about cooking and baking, mixing and measuring has not been innovated in – in what 75 years? [DUNCAN] If that… [BEN] Whatever – it’s been a long time, right? It’s been a very basic kind of a thing. And so, getting people to engage with that is an interesting challenge. But we have tremendous visual information because we were in people’s homes and we watched them cook. And so, we created a fairly life-size kitchen environment to bring people into, and we had video stories of people just like them who were talking about how they cooked. And being able to use those research materials as a way to get a new batch of participants into the topic and having some connection to the challenges or that being able to and then share their own stories and, therefore, be in a projective state of mind can be really very effective. I think that the hang-up that a lot of clients have is that they think that that’s going to produce the idea – the one idea. Or that there will be some sort of consensus at the end of these things that yields the brilliance, and I think that’s a mistaken expectation. We often follow co-creation events with co-design events. So, what you learn in co-creation is often how far your users or consumers are already able to imagine and push beyond. You can learn sometimes what they feel like are barriers in the sense of, “Oh my God, I would never want to go there,” kind of a thing. You can learn a lot in the way that they attempt to solve problems and tell stories and imagine things working and not working in their own kitchens or wherever that is. So, the ideas that they come up with are actually a set of interesting things that you should tear apart. Like, “Oh, so this is really telling us the story about we have permission to live on the countertop, but we don’t have permission to be plugged in. We have permission…”. So, you’re pulling out of a session like that what I would often call consumer or user criteria for something like success. And also, like drop-dead, bozo no-noes for failure. And so, what that does is that sets up a professional ideation session where you have designers, engineers, market strategy people, etc. who roll up their sleeves and say, “OK, we learned this from the primary research. We learned this from co-creation. No what can we do with that?” Because that is a tremendous place to start from, as opposed to starting from the Post-It note on the wall. [LANG] So, when you do those co-design sessions, is that also with the consumer or with the target audience, or is it that like what you said – with the engineers and the other folks… [BEN] It’s more internal, yeah exactly. And because you’re really working on more fine tuning of ideas. You’re mixing and matching ideas. You’re trying to hybridize things and take them in different directions without… You don’t want, in that scenario, a person who’s judging the outcome, right? At that point, you’re really in a moment of trying to push the boundaries in different directions. And then, that’s often best handled by people who are really comfortable with doing that and with the ambiguity involved and looking to drive that. And that can often unfold over weeks. You do one session, pull things together, crystalize them, come back again after you’ve let your brain marinate for a week and you do it again and you often you wind up with some much more refined concepts that have all different kinds of nuance baked into them because you gave it time. You took some time in the field to really learn primary things from end users and consumers. You took time with consumers to hear how they would project their own solutions into it, and then you’re combining that with, of course, with some of the company’s own criteria involved. It needs to make the magical hundred-million. It needs to keep a certain factory functioning. It needs buy supplies from wherever, right? So, there are all sorts of interesting business criteria – you don’t want to spray those all over your consumers and users, but there is an important moment for them to play that roll. [LANG] This might be slightly off topic, but I can sort of almost see the world of cars, with infotainment systems as a really fast-emerging that’s happened over the last 10 years or whatever. [BEN] Yeah. [LANG] Where even with like with where people put their smartphone when they’re driving – you can see the redesign of the consoles and dashboards. And the other interesting thing I find is it’s very fractured right now. Each company’s approach to the infotainment system and functionality and touch screen or a knob in the centre of the console – they’re quite different from each other so that we haven’t really got a standardized kind of thing. And I wonder if some of the techniques you talked about would have helped auto manufacturers to go along to where they’re at now. [BEN] I can only imagine that it has. There’s – I haven’t done any work for big car companies. Happy to do that if they ever want to reach out. But I do think that you’re spot on with that. People are innovating in different directions. I think that there’s an underlying notion in that industry that drivers are going away. And so, there’s this – so much of this technology is moving so quickly that the sensibility for making really big investments is probably small. [LANG] Yeah. [BEN] So, here’s a funny example of that. [DUNCAN] Awesome. [BEN] Right. We did a project with a fast food company. This is years ago. You remember when key fobs were all the rage? Where you would put your loyalty card from the supermarket that goes on your keychain – it’s got a barcode or what have you on it, right? So anyway, this fast food company, they do tons of drive-through business, etc., etc., and they wanted people to scan their loyalty card at the drive-through because that way they get their discount, they get their favourite meals already pre-loaded in there, etc., etc. – speed everything up, right? So, we do a lot of work with this company over the years and there was this brilliant moment where we’ve got the key fob. We’re giving it to people. We’re taking it through the drive-through. And it becomes obvious what a classic epic fail this would be because where is the key fob? [DUNCAN] On your keys. [BEN] Where are you keys? [DUNCAN] In your ignition. [BEN] Yeah, right? So, a massive investment in key fobs and all that kind of stuff would’ve been a horrible idea. And they were able to find that out pretty quickly. But you can push a good idea really, really far in your head without sometimes hitting that reality moment, right? Where you’re like, “God, that doesn’t make any sense.” And sometimes it can go much further than you think. It’s hard to invest in technology that changes that quickly because that went to RFID, RFID goes to smartphones. So, how much are you going to invest in the console of a car and an entire infotainment system if in five years, half the cars are going to be driverless. [LANG] Yeah. [INTERVIEW PT.1 END] [INTERSTIAL MUSIC] [ADVERTISEMENT] [DUNCAN] Before investing in a new advertising campaign, you need to know it will work for your brand.At Insightrix Research, we developed our ad testing solution to be the most comprehensive tool for assessing, predicting and optimizing advertising performance.
Insightrix Ad Check is a pre-testing research tool to assess advertising before it is launched via consumer testing to measure and maximize ad effectiveness and purchase intent.
Insightrix AdCheck will optimize and enhance your copy for the greatest impact and identify the most effective creative to produce and run, allowing you to get the most of your media budget by putting your spending behind proven, strong advertisements.
Using standardized copy testing measures based on a battery of questions, we can help to quickly execute your research project, meet tight timelines and build norms.
Insightrix AdCheck survey designs are flexible! And that means we can work with you to accommodate your individual and specific campaign requirements.
It costs the same to produce a bad ad as a good one. Early stage pre-testing can ensure your final production costs are invested in the very best creative.
To learn more about the Insightrix AdCheck ad pre-testing solution, visit offers.insightrix.com/insightrix-ad-check or contact insightrix at info@insightrix.com.
[INTERSTITIAL MUSIC] [INTERVIEW PT.2 BEGINNING] [LANG] So, that actually maybe hits our next question here. I’m just curious about pitfalls you’ve observed when clients are looking to do the kind of research that you do. What are some things to watch for? Obviously, you mentioned very fast-evolving technology can run you out of relevance. Any other key things that you’ve observed over the years? [BEN] Yeah, a few. I have two that are – sort of – seem contradictory but – well, they might be, but they’re still important. So, the first one is really not being focused enough on the criteria for success. And I talked about that a little bit. And I mean that is consumer criteria or user criteria for success. Often, companies are very focused on their own criteria for success, as they have to be because they’re businesses, right? But they’re often glossing over the user side of that. And so, having a very conscious way of articulating those at the end of research, I think, is really important. The second one is being overly focused on current functionality. And so, that’s the story that we talked about with the price-check machines. They were obsessed with the price-check machines and how they were going to impact today’s customers. Fast-forward 10 years, all of that is irrelevant. But, what they could have learned from that was what their customers – actual customers who are in their actual stores. So, not hypothetical customers, recruited in a survey or focus groups, but real humans in their stores – what did they, in Oklahoma! OK? So, this wasn’t in San Francisco Bay, OK? This is in Oklahoma in a small town, right? They had amazing ideas for what technology could help them to do in an in-store experience, and I’ll tell you right now, that company didn’t want to hear it. They just wanted the answer to the very limited question of, “Should we put these price-check machines in there or not?” So, being overly focused on that is bad. Assuming your customers or end users or stupid is another really bad one because they’re usually much more forward thinking, more curious than most of the folks out there that I work with give them credit for. And the last one is a lack of patience. You brought up quick and dirty. My usual answer to well we just want to do it quick and dirty is when a client says that or a potential client is that you’ve come to the wrong firm. Because we really don’t do quick and dirty. We can do quick things – relatively quick, but we don’t like the concept of quick and dirty. We like quick is good as long as you know how to train it and focus what you are trying to get out of it. [LANG] Yeah. Yeah, that makes total sense. Where do you see the future? I mean maybe it’s a two-part question, but have you seen a great deal of evolution since you started in this path in 2001? Perhaps new technology advancements, changing consumer communication patterns, social media, whatever it’s been over the last almost 20 years and what’s it look like going forward, I guess, for your business? [BEN] Yeah, well, dramatic change. I mean, it couldn’t be any more dramatic. When I first started doing this work, pretty much, people didn’t know what the heck you were talking about if you said ethnographic research or design research. Those were – they didn’t make any sense in a business context. What you refer to as traditional research was all the rage and got all the money and there were only a few companies and schools out there that were engaged in this notion of blending strategy, design and social science research from an observational standpoint or a cultural standpoint in that way. That was a very new thing back in the late 1990s/early 2000s. Nowadays, I get RFPs from people I’ve never met that say they want to do an ethnography. They want half a dozen unmet needs, and crisp insight statements, and etc., etc. And obviously it’s a bit shocking to see that, right? Because most of them don’t have the experience or else they would know you can’t guarantee finding a half-a-dozen unmet needs out there in the world. That seems pretty crazy, but that’s what I mean. It’s gone from something that nobody knew much of anything about – and I would have to do really a lot of explaining just to get in the door to now people asking for things by name and having a very particular set of expectations around them. So, that’s been huge. The other part of your question, the technology shift has also been dramatic, and does make other things possible, but my big fear is that it’s gone in a direction that is probably not healthy in the long run. And my sensibility is that this is a pendulum swing, but hopefully I’ll be around to see it swing back the other way. So, back in the old days, before there was all this technology, if we wanted to engage with people in what we call a self-documentary technique or approach, we would take disposable, actual cameras and create a little journal and put it in a Fed-Ex envelope with a return Few-Ex envelope and send it out to research participants and ask them to document something in their life. And then, they would send it back. And we might go out and visit them a week later and they knew a little bit more about us, and we knew a little bit about them, and it was a great way to get things started. And nowadays, of course, nobody does that. There’s all sorts of interesting research platforms where people can log on. They can take pictures and video and self-document almost much any aspect of their lives that they can, and I think that’s become something of a substitute for people. And that’s where, I think, there is a big mistake. It’s certainly faster, better and easier than sending out disposable cameras and sending them back and forth. Not doubt about it, but it doesn’t fulfil the really critical part of this work, which is about being present and having an experience alongside your end users or customers and having all that extra information you get from being in context. And, right now, I feel that people are just blowing that part of it off completely. They think, “Oh well, I can get 50 people on this research platform and they can tell me everything I need to know about this prototype or this set of sketches that I sent to them or what it was like last time they went to the doctor, or riding the bus – it doesn’t matter what it is. And they feel like that’s it. They’ve got everything they need to know. But they’ve only got like maybe half the story, and they don’t have any of the direct experience – meaning the clients don’t or the design team doesn’t or what have you. They don’t have that direct interaction so that they understand things at a more fundamental level, and I think that’s a mistake, but things are going that way. And the other really, I think, tragic thing that’s happening in our world is just this unrealistic notion that big data solves all, and that there’s some inherent truth in amassing all this information that seems to be swirling out there. [LANG] Do you find that talk of big data is being viewed as a potential substitute or threat to observational research? [BEN] Well, there’s a finite amount of money right now that people are willing to throw at anything, and big data’s the rage. There’s just wholesale budgets being shifted inside companies from one place to another. In the short term, this is certainly palpable. In the long term, I think it’s not going to be that different from some other experiences I’ve had with clients. Sometimes, I sit down with somebody and they’ll push literally a foot-and-a-half stack of documents across the desk to me and say, “We have all this quantitative data. We know all these things about these people in these segments or what have you. But we can’t do anything with it. We can’t see them as people anymore. We can’t talk to them. We can’t build something for them. We just know all this stuff about them and it’s decontextualized from anything real.” And then, you’re back out there in the world, trying to humanize an endless series of top box scores. And that’s – that’s our mission, right. I mean, that’s my mission in life: keep the human beings a part of all of this because otherwise, things just go off and I don’t know what they become but it’s certainly not pretty. [LANG] Yeah, for sure. And obviously we’re in a smaller market here in Canada but over the years, I’ve noticed a surge in sort of qualitative research and then periods where it cools off and that kind of thing. I’ve sort of noticed, as of late, there’s less, I think, appetite for even your basic focus group or something like that or even interviews with people. It’s more, I think I’m not sure if it’s just the investment of time or what it is, but it’s interesting to see who things kind of ebb and flow. [BEN] Yeah, it is, and I don’t know if there’s an exact, easy place to assign causality or anything to all of this. But like I said, what I think is that ultimately these are pendulum swings and there’s a point when people have different needs, and my anticipation is that some of that will come back when people realize that big data doesn’t solve all, or that maybe big data causes big problems. There’s a great book called – what is it – something of mass destruction? Weapons of Mass Destruction – great book, actually, which details a series of fairly horrifying stories about giant policy decisions that were made based on big data analytics. That can be an issue. [LANG] So, I mean, we talked a little bit about what are some of the trends and the pendulums and things like that. What do you sort of see as the future? You mentioned that you’d like to see perhaps some of this might sort of swing back, but are there any other alterations on the horizon that could make prototype testing a little – adapt a little bit different, or that kind of thing? [BEN] I would love to see some. And I think one of the big opportunities in the world that has not been fully realized is there is a notion, years ago when the dot com boom got started that’s probably discredited with its bust, but it was this idea of collaborating to compete, and I think that there are tremendous opportunities for big organizations that work together and interact with one another in very interesting ways to conduct research in a joint fashion so that you’re a) you’re not wasting money and time and b) you start to acknowledge that there are huge systemic interactions among and between the parts and pieces of our economy and of our behaviour and of our daily lives, and starting to knit those things together from a research standpoint would actually, I think, promote just a lot more well-being and creating more joy in the world and trimming probably atrocious amounts of waste. But people are pretty loath to do that. There’s a very few examples of it. We had a one really great example where we were lucky. They were, both companies were our clients and so – and they were each others’ partners. So, it worked out really, really well. One of them is a paper products company and the other’s a hospitality company. And so, the paper products company had cool new stuff they wanted to test and the hospitality company wanted to find out whether or not that was going to improve guest experiences. You know, you can’t really do testing of new paper products without a real-world hospitality situation to do that in. In the old days, that was – people would have to run two separate projects and try to sell each other on whether or not they were done right. But they end up talking together, creating a great research project where they share the insights that came out of it. What a concept, right? Sharing? It’s just insane, right? To share what they learned both about guest and experiences – about products and the prototypes and things like that. And then they had high confidence in the work because they were a part of conceptualizing it. And that’s just two companies, right? You think about how many different ways in which our institutions are interconnected, and yet we don’t take the time to think about, “How could we do something together and arrive at a better place?” Rather than everybody off and doing their own thing and trying to smash it together and hope it works. [DUNCAN] Wow. Well, we’ve kept you for a while here, so I want to just ask one more question though. If someone wanted to learn more about prototype testing and the techniques that you use to do it, what would be a good place to get started with that? [BEN] Yeah, you know there’s a couple of things that popped into my head. So, there are really a decent number of deign schools these days, such as – like here in Chicago it’s the Institute of Design in Chicago but there are others around the country and around the world, and many of them offer executive trainings and workshops in ways to get involved in these sorts of projects. And it’s really an inexpensive way for professionals to amp up their understanding and their knowledge about what’s possible. And, building on that, if that was exciting and they found it valuable, then you can hire interns and graduates from those programs who have skills in behavioural prototyping and understanding both the research and the design aspects of it, and so forth. So, I think that’s a great way. It’s low cost. It allows, I think, for an exposure to what’s new and what’s emerging from a technology standpoint, as well as from research and a real understanding of new product development and new service development, etc. And often, what I’ve found is that a number of my clients who’ve done these sorts of things, they wind up becoming more deeply engaged with that community because it is really a community of people who are interesting in innovation and design research and things like that. So, I think that’s probably the best way. Obviously, there’s books and things like but, as you can tell, I’m a bit of a freak for getting involved. And so, I think that, ultimately, if you really want to get something into your organization, you’re going to have to go there and try it out. Obviously, people could hire us, but I think that wasn’t your question. [DUNCAN] Yeah. Well, speaking of trying things out, I was actually on your website just recently and I saw that you’ve got an new thing called – well, it’s a bit of a quiz. I’m wondering if you could tell me about that really quick. [BEN] Oh yeah. Well, thanks for that. [DUNCAN] No, it’s great. It’s a great resource. [BEN] We have a little tab on our website – coniferresearch.com – called Research RX. And really, it’s a short, semi-fun quiz that helps organizations, companies, what have you figure out what’s the kind of right type of project to support what they’re trying to get done. The way we’ve been looking at it over the years, we really see that there’s basically three different kinds of projects. There’s ones that are sort of a big, fuzzy exploration. Another is more about kind of growing what you’ve got, and the third is really about refinement of concepts or prototypes or things like that. And there’s more to it than that. I don’t want to bore people to death with it, but the basic idea is that once you have a way to have a conversation between your agency of choice your company, your organization, a lot of times there’s just so much that is hidden in terms of what’s needed, how broadly does that need to be shared – there’s just all of these different kinds of fascinating questions, that rarely get packed into an RFP. And a lot of times, by the time an RFP is selected, a person is selected to do the work, it’s – you’re already a week late and running down the road. And so, this is an interesting way to begin the dialogue by trying to learn a little bit more about each other and how to think about the goals of a project and how that might flow through into the exact methodology that you would choose and what you should expect to come out of the results. [DUNCAN] Well, that’s a really, really handy resource and I’ll make sure that we’ve got some information and a link to it so people can check it out on the podcast website. [BEN] Super. [DUNCAN] Yeah. Well, I really appreciate you coming on our little podcast. We really had a good time. [BEN] Yeah, I did too. I think your efforts and, certainly, what we try to do in the world is to be as focused as we can on bringing more joy into the world and encouraging our clients to do the same thing. And, I guess it’s one of my favourite sayings is more joy, less BS. And it plays out and it’s so puritan in its approach. There’s an obsession – I’m sure you guys have run into it, right? You go do research and all your clients want to know where the pain points are and it literally is masochistic. It’s like, “Tell me all the ways in which we’re hurting people.” And I can’t help think that, well, don’t you want to know when they smile? Or laugh? Or say that was good? Because, really, that’s the good stuff, right? And I can’t tell you – yes, I can. I can tell you I have never, ever, ever – in 20+ years of doing this kind of work – had anyone tell me, “I just want to buy the thing that sucks less. Of all the things on the shelf or in the display, which one sucks the least because that’s the one I want.” No. You want to make people happy. So, if you only think of those pain points, you’re never going to get to the joy. So, I try to tell people, start with the joy. What makes people happy? Focus on that like a laser and do your best to deliver on that because that, I think, is the path to success. [DUNCAN] Well, right on. We really appreciate you coming on and really look forward to talking again in the future. [BEN] Yeah, thanks gentlemen. [DUNCAN] Any time. [BEN] Talk to you guys soon, Duncan and Lang. [END INTERVIEW SEGMENT 2] [INTERSTITIAL MUSIC] [DUNCAN] And there you have it. I’d like to thank our research director, Lang McGilp, for taking the time out of a busy schedule to lend his expertise to the conversation. And I’d really like to thank Ben Jacobson for coming on the podcast and sharing his thoughts, ideas and experience to the episode. While the episode interview was one of the longest we’ve done yet, I think we could have easily gone another hour it was so interesting. If you’d like to learn more about Conifer Research and the prototype testing they do, you can find them at www.coniferresearch.com. On their website, you can also check out the Research RX tool that we spoke about in this episode. If you didn’t get the URL, we’ve posted the information on our website at insightrix.com/podcast. And of course, we’d like to thank you – our awesome and loyal listeners. Without you, none of this would be possible. If you enjoy Stories of Market Research: The Insightrix Podcast, why not take a minute to rate and review it on Apple Podcasts, Stitcher or anywhere else you access your podcasts. It would mean a lot to us, and it helps get the show in front of more people like you – folks with an interest in research. And if you’d like to know more about Insightrix Research, you can check out our website at Insightrix dot com, or follow us on social media on Facebook, Twitter or Instagram. We’ll be back in another few weeks with another episode of Stories of Market Research: The Insightrix Podcast.
