Key Points
- Being a PM for complex, enterprise products
- How to find problem areas and new opportunities, even for well-established products
- Figuring out if a problem applies to the broader customer base
- How to prioritize opportunities that you identify through research and feedback
- Tools and process to validate new features with a lower risk, as they’re built and launched
- Identifying features to kill
- The importance of capturing context from the customers we talk to
- The nuances of segmentation in enterprise products
- Collaborating with Sales teams to improve their communication towards customers and sell benefits, not features
- The PM mindset of constant questioning and validation of assumptions
Transcript
Daniel:
Just to get started, and for those that may not know you, can you tell me a bit about yourself and your background in Product Management?
Nils:
Right, so I’ve been a Product Manager for umpteen million years… since the mid 90s, I think. Which nobody really knew what it was back in those days. There was a thing, but nobody really knew what Product Managers did. And the most interesting thing in that period was for 7 years I was the Product Manager for a tool for Product Managers, called Accept360. So that was a very interesting experience and a good learning experience because I learned a lot about what Product Management was all about.
One of the really key things that I learned, which really applies to this whole Customer Feedback thing was, even though I was a Product Manager and I’d use my own tool, I still didn’t know what my customers did. Their experience was totally different than mine, in terms of being Product Managers. It just really drove home the idea that even though I’m the same job title as my customers, I’m still not my customer, right?
There’s two different dimensions to that. One is that they’re having different experiences as Product Managers. The other is that they’re not interested in my tool; they’re interested in getting a job done. So my tool is just a tool for them, is not something that they’re focused on. And so, it was two levels of learning about that.
Daniel:
That definitely resonates with a lot of the issues that we face. I would say… Can you tell me a little bit about what you’re doing right now and what kind of role are you playing. I know that you’re a Product Management consultant and you’re working for an enterprise firm, right?
Nils:
That’s right. At this moment I’m working for a company called Innotas which makes Project Management software, not Product, and I’m a Product Manager there. I did a year of consulting in 2014, I enjoyed that, but felt I needed to get back into having my own Product, for a variety of reasons. I still think I might go back to the consulting but I was ready to do it.
Daniel:
Right, so I didn’t know about that switch. So what kind of role do you play at Innotas? What’s your main responsibility and what are you working on?
Nils:
I think of myself as being responsible for the whole product, although there’s another Product Manager and the VP of Product, who actually runs development as well as doing some Product Management. I’ve been there about a year, so I now know a lot. It’s taken me a long time to ramp up on this product, which is pretty complex, a lot of moving parts. It’s actually very much like my old Product Management tool, in the sense that it has many of the same components. They work together a little bit differently, and of course Projects are very different from Products. In particular, if you think about Projects, they don’t have customers; they might have users but they don’t have customers. You don’t have to sell them. They don’t have competitors and they don’t have nearly as many users, typically, as a Product would have. So, it’s been interesting to see the similarities and the differences. So I’ve been there about a year, and you know, as I say, I consider myself to be responsible for the whole thing even though I’m only technically responsible for only parts of it. I’m also looking at this as an opportunity to really apply a lot of Product Management theory, right? To see how I can help the company accelerate its growth and address a lot of the things that a lot of companies that are at this point in their point in their life are struggling with. So, about 110 people, and companies at that size can start to struggle in certain ways and so, some of those things are because of Product Management not doing the right things. So, my goal is to do the right things.
Daniel:
Great. So, talking about doing the right thing, when you’re trying to find what’s important to the users and to the customer base, when you’re out and doing interviews… what kind of questions do you ask them? What are you looking for, in order to understand what’s important for them?
Nils:
So it’s a well-known domain, in fact, this is true of all the products, not just mine. But if somebody from 60 years ago, who was running a big Apollo project maybe, were to come to look at all our tools, they’d say “I don’t know any of this other stuff about mobile phones and about the Internet, but I totally recognize what you’re doing with this product”, right? 'Cause it has many of the same concepts. So that’s an interesting set point for all the products, they all have, at the core, very similar concepts. Now, that may not be a good thing, but it is reality at this point. There’s a couple of different things that I’m looking for specifically, at the moment, in terms of my product.
One is, obviously how to make it do the things that it currently does, how to make it do those things better. But the other thing that I’m really looking for is the gaps. And I think there’s a lot of gaps still in the product—in the project management world—just as there are in any process. And I’m trying to figure out what those gaps are and whether those gaps are worth filling.
And so for example, if you think about Project Management tools, and I’ll use Project Management as an example of things, the same approach can be taken to lots of other things. But with Project Management, you know, it’s got, there’s Projects, and there’s Tasks, and there’s Issues, and there’s a Timeline, and a work breakdown structure, and a Gantt chart, and things like that. But, if you think about what Project Management really is, it’s about organizing people to get some work done. We can represent all that in the tool, but how much value can we provide around making that process of getting the work done more efficient? For example, collaboration. Projects are a lot about collaboration, but Project Management tools have not been that much about collaboration. A few do, but they all could do more.
So those are some examples from Project Management. And then the point is, how can I use my skills for getting Customer Feedback to go and explore where there’s still pains for Project Managers, even if they have a tool. If they have a tool like ours… What are the things that are still painful for them?
And there’s a variety of reasons for doing this. One is, it may be that we actually already solved those problems, or some of them, in which case we can then use the expression of what customers tell us is a pain and we can turn that around and say: “Hey, here’s a benefit that you’ll get from our product, that you should know about”. We can do a better job of solution-selling, basically, if we understand the pains better. And then of course, if we figure out pains that people have that we don’t address, and in particular if competitors don’t address them either, which is not that unlikely to find, then we can add those to the product.
That’s sort of the big picture thing that I do when I’m trying to get Customer Feedback. I’m trying to go and talk to Project Managers, and find out what problems they’re still facing, even though they have a tool.
Daniel:
I guess the big issue there is that it’s a very well known domain, the tool have been very well defined for a long time and maybe Project Managers don’t feel as if there’s a problem there. Or maybe just used to doing thing in a certain way and they think “that’s just the way of life”. How can you try to derive, the hidden needs that might be there? What sort of question do you dig into in order to really find what they don’t believe to be a real issue?
Nils:
Right, that’s an excellent question. The way that I do this, and I don’t really have a system for it, particularly. But I listen for things where they talk about pain, or they talk about frustration, or they talk about, things they’re worried about, or things that cause them to have a bad day, or that cause them to have a good day. And so, for example, I’ll actually ask them. Well, for one thing, when you talk to a Project Manager, you’re gonna say “how are things going?” and they’re gonna say: “well, I’m really busy.” You can ask them “what has gone wrong recently?” and everything goes wrong, if you’re a Project Manager. So, you can start to get a sense that it’s not perfect, right? And then you start to drill down on, and so then you start to drill down on “in what way was it not perfect?”. And so that would be one way that I would start.
The other thing I’ve done is “what is I win/I rule sort of situation for you as a Project Manager?”, “Give me an example where you had a ‘I rule’ feeling as a Project Manager”. And then maybe go into that to determine what were the things that came together that made that happen, and why does that not happen more. So that’s another path that I sometimes take.
And of course there’s the opposite as well, which is, “I suck”. “Give an example of an ‘I suck’ situation. A time when something went wrong and characterize that.” And then you can drill down and in our domain, it might be a collaboration problem, it might be an expectation-setting problem, it might be a scheduling problem. You know, so you drill down and you find out what could’ve happened, what could’ve been done to make that not be an “I suck” situation. Another one could be “I suck because I did not make it to my kid’s baseball game, and that made me feel terrible.” Well, how can we help them make sure that they get to their kid’s baseball game? I often try to look at whether personal goals that either achieving or not achieving. And this is actually on both sides… I love to find out when my customers are able to achieve personal goals that they couldn’t, before they had our product. So it’s a flip side, right? If I can talk to a customer, and they say “you know what? Before we got Innotas, I was missing all my kid’s baseball games. But now that I’ve got it, we’re so much more efficient, I make it to all the games.” I love that kind of story, and I search for that. And they’re not that common, but if you can find them, they’re very powerful from a Marketing standpoint. Because part of the Product Manager role is to enable go-to-market and one of the best ways to go-to-market of course, is to have really great stories from customers about how they’re lives are better because of our product.
Daniel:
We usually resort to doing interviews because they’re open-ended and we can find out a bunch of things and they’re really flexible. Are there any other discovery methods that you try to go to get this kind of insight?
Nils:
Well, primarily, I use interviews. The way that I often do this—and I don’t do as many as I should, this is an area where I could be doing much more than I am, sadly, even though I’m a very big proponent of them; I think that’s true of everyone. But you know I’ve gone out to LinkedIn and I’ve looked at who is in my network who’s a Project Manager and who’s not a customer? And I say, “Hey, I’d like to talk to you about doing Project Management”. And I’ve done that a few times. Obviously, I have existing customers and over time I’ve built up relationships that I’ve found are good for giving me feedback. You know, who either can already start to take a little bit of a step of abstraction, which can be very helpful, or are just noisy and have lots to say, and that’s also useful. So that’s a lot of what I do there.
I’ve never used surveys, for example, or questionnaires. There are some things that I can imagine doing like, putting instrumentation into the product such that if somebody used a particular feature, where I could pop up a little thing that said: “Hey, we’d love to talk to you about how you’re using this feature, could we set up a phone call?”. But I don’t do any of that
😀
Daniel:
Say you understand there’s a valuable opportunity and you found it and maybe realized that you have one or a few customers that think that’s valuable. How can you try to validate if it applies to the broader customer base or prospective clients?
Nils:
Right, so I do a couple different things there. One, is I would talk then to these customers I know of and are sort of my pool of people that I can go talk to and I would obliquely try to get them engaged in this topic. I might go and say, depending on what type of feature, what type of problem is that I think I’ve found, I might go and ask them about “how often does this thing happen to you?” or “when you think about this area, what are some of the issues you think of?”. You know, it’s hard without having an specific example to give you the sort of the questions that I might try to ask. But the goal is to not lead, but have them start at least somewhere near the area of what you’re trying to get to.
Daniel:
So that’s a big issue there… How can you make sure, what kind of formulation for the question do you use, in order to make sure that you’re not leading them towards some already preconceived idea that you might have, or maybe on the problem space? How can you structure your question set in a way that isn’t conditioning the other person?
Nils:
Some of the ways that I’ve heard to do this, and that I might try, or something like this, would be to ask them about, “what’s the biggest issue in [a very general area that includes my area]?”. And I might listen to “what are the top 3 issues you have?”. And if I don’t hear my problem, then maybe I already decide “this obviously isn’t very important”.
😀
So setting the context right is important, because if you set it too big, then your thing might be in the top 10, but not in the top 3. Whereas if you were to set it somewhat smaller, you’d be in the top 3. And so that’s a challenge and an art. That’s why you have to talk to multiple people.
So the second piece of that is that I would try to talk to as many people as I could so I wasn’t just hearing what one’s person experience was; that’s one reason. The other would be that I can sort of fine tune the way I ask, and make sure my questions are focused on the right thing.
I think one of the things you wanna do in that process is you wanna sort of have, you want to go into the hypothesis that says: “I hypothesize that 3 out of 10 people that I talk to express this as one of their problems.” And you might want to say: “If out of 10 people I don’t hear it at all, then it’s not a good idea”. So you might want to have a set of rough qualitative measures around your research project to help you know when to stop, basically.
Daniel:
When you get to a point where you a have a set of validated opportunities, and you know there’s two, three, or four things that you can work on at the same moment… How do you take your pick? How can you decide which to pursue, based on your current product strategy? Sometimes, they’re not really comparable and they may be shooting to different objectives, and it gets hard for people to know where to funnel down and move forward.
Nils:
Right. Well, it partly depends on how good the product strategy is. If the product strategy is well defined, then some things are going to fit better with it than others. That’s one thing. Obviously, you should have some sense of how big each of these opportunities is as well. Minor problem that a lot of problems have vs Major problem that a few customers have or “major problem that a lot of customers have.” You should have a sense of what it’s going to be worth and you’re going to turn that into money. Because you’ll be able to sell more, sell faster or sell an add-on or get more renewals or whatever it might be. You should have some kind of a hypothesis of how’s that going to work, for each of these potential things you could do. And of course you also probably want to have some kind of sizing, rough sizing, of how much effort it is. Because, you’ll have to trade things off.
Assuming we have those things: we have a decent strategy that we can compare them to, we know how much each one will make us and we know how each one will cost, then hopefully you’ll actually have weeded out a lot of possibilities with that combination.
And then at some point you may have three things that are all equally desirable. And in that case you just have to make a decision.
😀
I mean, I would actually at that point probably start to use things like “how marketable is this?”, “how exciting is it?”, “how much of a splash can I make?”, and I’d also take a look at “is there a natural ordering?” If I do one of these first, does that enable me to do another one later or does it preclude from doing another one later. So there’s a lot of different things I would bring into the prioritization picture.
Daniel:
And as you move forward and you start to work on these things and you start to build them. What kind of customer input do you seek at that stage? Do you try to validate things before shipping them? How does that work for Innotas and your current work?
Nils:
Ideally what we would be doing is throughout this process… So first of all, in terms of getting the process of creating the solution, part of what I would do is make sure that my requirements really talked about the problem and the customer experience that they’re having, without the solution. The customer experience when the customer has the solution. Very much problem focused, with the idea that my Dev team is supposed to be good at solving problem. I want to give them the right kind of information so they can use their skills to come up with a good solution.
Obviously, I have to guide that. Partly because they’re engineers, but partly because it’ll be a constant set of feedback throughout that process.
But also, what I’ll also be doing, probably before any code is cut, is working with the designer. We have a designer, essentially a UI designer. She’s not a visual designer, but she’s a UI/UX designer to create mockups and we will a lot of testing of those mockups with customers and internally.
Daniel:
Ok, so what kind of testing is that? Is it usability testing, is it having a prototype and observing what comes out? Do you set goals for what you’re trying to test? Or is something more observational?
Nils:
Unfortunately we’re not doing probably as well as we could. A lot of what we’re doing is we’re sharing designs over a webinar (Webex or Gotomeeting) with the customers who have expressed interest in having this problem solved and showing our potential solution. What we’re not doing yet, and I would like to move toward this, is giving customers clickable prototypes. Instead, what we’re doing is, after we’ve reviewed mockups with customers, we then build the stuff. And then we will start opening up very early versions to customers.
Daniel:
Is it to a segment or to specific customers? Or to the entire…
Nils:
Well, it’s a combination. Sometimes is specific customers, particularly earlier on. And then, later on, we will release things as beta or we have what we call “Sandbox servers”, so customers can put their data onto a sandbox server and then try the new features. Most of the features we do we have on a switch, so we can switch them on and off, and we’ll switch them on in a sandbox, even very early, to get feedback.
This is an area where we could do much better than we’re doing, but I think we’re doing it much better than a lot of people do. So, I feel okay about it.
Daniel:
Let me ask you something… do you think there’s room for quantitative methods applied during the discovery and delivery stages? Usually we end up using mostly qualitative, because we’re trying to learn, but what kind of value or what kind of things can you take from actually trying to quantify objectives during these stages?
Nils:
Well, I think, as I mentioned earlier, hypothesis and sort of thresholds are good, even in the qualitative testing, but we’re enterprise software. Which means we don’t have that many customers. We have hundreds of customers. Which means that is really hard for us to get a statistically significant sample. I mean, we can do that a little bit in terms of usage of existing features.
We can look at the logs and figure out how many people are using this particular feature or that particular feature. Have this thing turned on or have created this kind of a custom field or things like that. And we do use that. We do look at, for existing features, how they’re being used. This is useful to deciding whether to end of life something or to deprecate a feature, as we move forward. But for finding, validating new features, that’s pretty difficult. We just don’t have a statistically big enough sample. Now, if we were consumer software, I think it would be a much different story. That’s sort of where I come from and that’s why so much of it is qualitative.
Daniel:
Yeah, sure. But, you touched on something there that, sadly, isn’t more common, which is end of life and cutting features from the product. How does that work for you guys? Do you have a certain maximum time period where you evaluate if something can stay on or is it more of as you realize that something’s not working out, you take it out?
Nils:
We don’t have a perfect solution to this, at the moment. But the simplest thing is if we have a feature that no one is using. It’s fairly easy to remove it.
😀
If we have a feature that we have replaced with another feature that’s better, and we’re actually going through that process right now. We have a feature that wasn’t used by very many customers, we haven’t sold it much and we have a new feature that essentially replaces it, it’s a little different. And our goal there is to be able to calendar the end of life of the original feature. What we’ll be doing there is we’re letting the customers know that this is happening and we’re gonna help them migrate, if they haven’t migrated yet. In fact I’m having some trouble getting them to call me back, basically, but that’s a whole other issue.
Assuming we successfully get them migrated we will then be able to end of life it, because we know exactly who’s using it. That’s one of the nice things about being an enterprise SaaS company. We really know who’s using what. We can end of life stuff pretty effectively if no one’s using it.
Daniel:
Cool. I wanted to start maybe digging into I know some things that you also talk about, which are how do you manage all of this information, as you move forward with your product work, in a way that you can maybe go back to and reflect on it and decide upon it in the future. And know why you took some decisions along the way. You call it the Product Management system of record. All these bits of information come from interviews, maybe some usage metrics, maybe some tests, experiments that you’ve ran… How do you put all of this together in a way that lets you make decision both right now, and in the future?
Nils:
Well, I don’t do a very good job of it, unfortunately, at the moment. I’m like the cobbler’s children. What we have at Innotas is a Wiki that we use to capture a lot of this stuff. One of the things that I’ve been doing over the year that I’ve been there is trying to make much more use of this. It really wasn’t being used very much as a central system of record. So that’s one thing.
I would like to, now that I’ve been there a year, start to look at tools, and I probably most likely look at ProductBoard, the product from Hubert Palan, because I think it’s the closest to capturing the things I want. Which is, how do I take this qualitative research that I’ve done and tease out the information, and find the signal in the noise as I’ve said on my blog a couple times. So that’s what I’d do.
In the meantime what I do is I try to take good notes and I try to put them in a central place. Some place where I can do some searching, and use what tool there are in a Wiki to do some highlighting and do some capturing of that. It’s very difficult to do it without purpose, with tools that are ad-hoc, as I’ve talked about.
Daniel:
Right, I mean when you talk about trying to separate the signal from the noise… What exactly is signal, and what is noise? What kind of things do you know that you need to be looking for and what kind of things do you try to ignore willfully?
Nils:
I think my take on this is I don’t know a priori what to ignore. So what I try to do, as a practice, is to notice if I’ve heard something before. So that’s one key thing. Have I heard someone else say something like this before? Or say this exact thing?
An example that I use is, for a while I was doing some research for creating a Product Management guide. So I talked to several people about how they learned to do Product Management, and one thing that really surprised me, but it came up multiple times was… people said “well I go to LinkedIn to find out what people are saying about Product Management.” The first time I heard that I thought “well, that’s very strange, that’s not a place that I would go”. The second time I heard it I said “Oh, that’s a signal”. I don’t know what it means, but I know that it’s important, at this point. So that’s an example of the type of thing. In some of the features that I’m working on, when I hear two people ask for the same thing that it’s not there, I start to think that’s not just this one person’s idea about how it should work. But there’s probably an underlying need. Now, on the other hand, I don’t then say “the need is to be LinkedIn” or “to interface with LinkedIn” or whatever. The point there is you then start to drill in. Why is LinkedIn where people look? Are there other things like LinkedIn? What it is about Linked that is attracting people? Is LinkedIn a Product Management toolkit and I didn’t know it? You know, all these questions that then arise based on the signal that came in. I don’t know what it means by the time I’ve detected it.
The other thing that I often find is that things that were not a signal 6 months ago, when I look at them again, are now a signal. It might be that six months ago I did my first interview and never really hear about LinkedIn again and then this month I do another interview and then there’s LinkedIn again. I’d like to be able to go back and review things and see “Oh, what did I maybe miss in a previous conversation when I first read these notes, that now seems to be more interesting?” And there’s a lot of different ways that can happen. It can also happen because maybe I now understand what somebody said. Maybe somebody said something that I didn’t understand really. And now I know more and I can now understand it. Or maybe asked for something that just wasn’t feasible a year ago and now we’ve got enough product infrastructure that suddenly that thing that wasn’t feasible is now feasible. And I should make sure to make that input, even though it’s a year old to make sure that the thing I build in response to it is good, or addresses enough of the problem. So those are the ways that I do it. The biggest challenge that I have, and this is kind of terrible to admit, so I have this new feature that I’ve been working on, I’ve been putting a lot of my effort into it… Basically a big new set of capabilities, and so I’m demoing that all the time to new customers or to existing customers and the problem is that I can’t, I’m not good at taking notes while I demo and so I’m having some conversations, and I’m not doing a good job of capturing them. And that’s very frustrating to me right now. I need to come up with a solution to that.
Daniel:
The visual cues and all that kind of context that goes on, right?
Nils:
Right, exactly. I just need to just improve it. Either by taking notes by hand next to my computer, getting a second monitor and having some place to type. Because what usually happens is I’m demoing and somebody’s telling me something, and I can’t really type that in, because they’re seeing my screen, right?
So I’ve done a couple things. I’ve actually have asked to other team members and take notes for me. Their notes are not as good as mine, I don’t think. But, I mean they always capture the thing that I want to have captured. You know often times is actually what I say that’s interesting in response to what they say. So we’ll talk about a thing and I’ll say, “do you mean this?”, “tell me more about that” and they’ll say more and then I’ll say “so what I’m hearing is XYZ”, and I’d really like for that XYZ part to be captured. You know, it’s just a process that I’m gonna go through and try to improve.
Daniel:
That’s interesting, in a way there’s a lot of context that we need to be looking at, and capturing the context is always really hard. And I think a big part of the context is in which we look a given piece of feedback and customer information is “to which segment does that user or customer belong?” What kind of segmentation do you use in order to filter the things that you’re looking at in that light?
Nils:
We have a pretty rough level of segmentation that we think about. Partly, this is because we don’t have that many customers, we have hundreds of customers, not thousands or millions. And we pretty much know that almost all of our customers and users are in IT. They are either people that are working on projects or that are managing projects, or people that own the Project Management function, the PMO.
In that sense, we know a lot about our customers already. The segmentations that we use are things like how big is the customer (in terms of number of seats), which usually maps to how big they are in terms of their revenue. There’s definitely a difference in our customer base between the lower half, in terms of their needs, and the upper 20%. The upper 20% are essentially big enterprises, so they have big enterprise needs. The lower 50% are generally pretty small and they don’t necessarily need the enterprise features. And when I say “enterprise features”, I mean the things that big companies need, because it’s all enterprise software even if it’s used by a small group. So that’s one of the big segmentations that we use.
The other is, we segment the users a little bit, between the ones that are—in fact we have two tiers of user license—one is for people who essentially only enter their time and then one tier for people that are spending a lot of time in the product and are actually creating projects or adding tasks or things like that. And then of course there’s an admin tier as well. So we also segment along those lines.
And then a little bit, even among the users we segment somewhat between Project Managers and managers of Project Managers or IT management (directors and CIOs and that sort of thing.)
Daniel:
So it’s mostly either a business involvement with the product or a functional role or job that they’re trying to do through the product, right?
Nils:
That’s right, yeah. Actually, I should say a little bit more. Because that’s in terms of what we’ve done with the product at the moment. We definitely have sort of different segments, different verticals that we could and probably will start to focus on. So we have a segment of Education, we have a segment of City Government, we have a segment of Healthcare, and we have a segment of “Everybody else”, pretty much. Most other customers fit under the “everybody else” but we’re not doing anything specifically with those segments at the moment. We do know that they exist and that would probably be as we grow, because we’re at this point where we need to think about how we can make use of the fact that we have these different customer segments, and if there’s anything that we can use to differentiate our product within those segments. Either by adding new features or just by talking in different ways, or pre-configuring different vertical presets, for example.
Daniel:
You talked about looking at customers in a business sense. How big they are as a customer. I would guess that probably means that the bigger customers have a bigger voice of asking in terms of asking for features or giving you some sort of feedback, on one end. And also, going into the next topic that I wanted to talk about which is a lot of Product Managers feel like the process of working with other customer-facing teams like support or sales is full of “hardship”. Sales maybe commits to a certain loud or highly important customer or new customer, or maybe Support is coming down with information about some issue that is filtered through their own view and maybe doesn’t dig into the why of the problem, as much as you would like to. So I wanted to ask you about how do you see this inter-team collaboration in a way that leads to a more productive and healthy relationship between Sales, Support and the Product team?
Nils:
Great question. You know, something that is a challenge. I think at Innotas we do pretty well at not selling features we don’t have. We’re not perfect, but we do a pretty good job, so we don’t have that many commitments that we have to do, based on a Sales contract. It’s just such a bad practice to do that. For one thing, it really impacts revenue recognition, so we avoid that, pretty much.
But there are a lot of cases where Sales people come up to me and ask, “when are we going to do this? When are we going to do X?” And one of the things that I do, you know, my usual answer to that is “let me talk to the customer” I don’t make a commitment, I may say that’s something we’re thinking about and I’ll always say “let me talk to the customer”. To some degree I do that when I’m talking to Services as well. Basically, my rule of thumb is I can’t trust anything anybody says.
If somebody says “this customer needs this button to be green”, usually that doesn’t mean that the button should be green, it means something else. My job as a Product Manager is to figure out why they want that button to be green. And whether the fact they asked for that button to be green means they don’t understand the product very well, the product doesn’t work very well, they don’t know what they’re doing, the implementation was bad, there are so many different things that can be behind that request and so my job is to figure that out. Obviously I don’t do that, I don’t drill down on every single enhancement request, because sometimes it’s like “yeah, that’s a really good enhancement request, there’s no question we will do that, if we can prioritize it.” But sometimes “the button is green” is because there’s a bunch of buttons on the screen and the person always needs to push the green one, so they want to make it easier to find. Well, maybe that means the button shouldn’t even be there or it should always do that thing. I’m never just going to turn the button green.
So that’s the #1 thing — basically don’t trust what anybody tells you, because even if they’re right in some sense, and they’re well meaning, and they’re not trying to jerk you around, the thing that they’re asking for is probably not the thing that should be done.
Daniel:
Right. I mean, this also can be a two way street, right? In the sense that they also need informed of what’s going on in the product and maybe even get some of that “Product Manager spirit” or sensibility, in the sense of being able to ask the right questions to the customer as they’re already in front of them. Or maybe they have a more day-to-day relationship than the PM might have. What’s your experience on that front? Do you think that’s viable, and you can make that work? How can you keep them in the loop, keep them involved throughout the process?
Nils:
I think that’s an area where we could make a lot of improvements. At Innotas and many companies of course, I think this is true. One of the things that I’ve been doing over the past years is working with the Sales Engineers, the people that do the demos, to do a couple things. One is to help them make their demos much more about benefits rather than features. So that’s one aspect. But the other thing is to push for them to start pushing back on the Account Managers to do better discovery. So that they know what problems the prospect is concerned about, and so they can make their demos much more aligned with what the prospect is interested in. And that’s sort of like, I’m trying to get them to be more “Product Manager-like”.
And think that’s definitely incumbent on the Product Managers to help make that process happen. The other side of that is, though, we need to give them the right tools. For example, we need to give them good questions for doing discovery with, and good techniques for doing objection handling, if it’s things about the product. Some of this is Sales 101 but some of it is really things that the Sales organization needs guidance and help from the product team, in order to have good answers or to have good questions. I thinks that’s an important thing, part of our role. I’d like to be able to do more of it. At Innotas will continue to do more of that this year.
Same for the Marketing organization. The way that I look at the interaction between Product Marketing and Product Management, is that Product Management is responsible for the value proposition. This is a product that does X, that solves problem X, it solves it for segment Y and it’s better than the competitors for reasons A, B, C. Marketing’s job is to take that information and turn into getting prospects in the door, and telling them the right story so they’ll click on the next thing, or so they’ll call us up, or so that we can call them. It’s really not Marketing’s job, in my opinion, to develop the value proposition. It’s their job to take what the Product Managers gives them and make it beautiful. I don’t consider my job to be making anything beautiful, really. So that’s something that again, I feel like because Product Management has all these responsibilities, it’s sort of this “CEO of the Product” concept, there’s a lot of things that we need to provide everyone else, so that they can do their jobs effectively. And if we don’t give them that, then they make stuff up that’s probably wrong, through no fault of their own, and doesn’t help anybody win.
Daniel:
Exactly, I think that we probably feel a lot of pain, in the sense of, looking at the symptom, but we fail to correct the root cause of the things that we are facing are, and I see it over and over again. So I wanted to, now that we’re maybe hitting on our time schedule here, I wanted to maybe try to dig in, sort of wrap up of what we’ve been talking about. As an industry overall we tend to talk a lot about the positives and the things that work well. Can you tell me about bad experiences that you’ve had with some methods and some customer feedback channels, a few Don’ts when it comes to gathering and using customer feedback and data?
Nils:
My best example is one that I didn’t do, but I was involved with the company that did it. And this is something that is a bad way to get customer feedback… is you say to your friends, who are maybe CEOs of companies, you say “I have this idea for a product, here it is, XYZ. Would you buy it if we made it?” And so our CEO at the time did that, got a lot of Yes’s, built the product at a cost of a great deal, you know, multi-millions, and sold zero. The point is, that’s a very terrible way to get customer feedback. You don’t show the product, or talk about the product, until you’ve validated there’s a problem. And you don’t ask people if they’ll buy it. You either get them to give you money or you get them to tell you, in some other way, how valuable it is to them. That happens a lot too. That didn’t just happen to my company, that happens to a lot of companies that people do market research that way. And it’s just, it’s a failure. You cannot trust the results that you get. They’re not useful for making decisions. And if you make decisions based on them, then you’re going to fail. All of us struggle with that, though. It’s really nice to be able to go to somebody and say “Hey, I’m building this thing, what do you think?” And you know, they’re gonna say, “That’s cool”. And then you say “Would you buy it?” and they’ll say “Sure, call me when it’s ready”. It’s very seductive to fall into that pattern, but you just have to know that’s a failure pattern.
Daniel:
Right. In a sort of summary or as a way to include any more insights that you might have on this topic, what would your top recommendations be for Product Managers that are trying to use and make sense of all this customer data and customer feedback that we’re getting, in order to try to make good decisions?
Nils:
It’s very interesting that you ask that, because Rob and I just did a podcast yesterday. We recorded it, we haven’t published it yet. That actually has a list, and I’ll will just tell you the list, the key takeaways on the same topic.
You are not your customer, we talked about that.
You’re customer does not know best, and you can’t trust what they say. This is the whole thing about asking people directly. You can’t really ask people directly, but you can use the techniques that we talked about, open ended questioning, setting a context, asking about problems. You can use those techniques to improve the value of the information that you get. We didn’t talk about the “asking Why 5 times”, but that is one of those techniques. You say, “I want the button green”, “why do you want the button green?”. “Well, because I have to click it all the time, and I can’t find it amongst all those other buttons”, and you keep asking why. A very useful technique, and it conceptually helps you understand the idea that the first thing you hear is just a surface symptom of what the real problem is, which is often far below.
Obviously, the last point, if you show before you’ve done discovery, or if you ask close-ended questions, you’ll get answers that aren’t really usable. They will seem usable, like “all these people told me they’d buy it”, but it’s not good to act on that information if you got it that way.
And then, I think the other thing is, it’s really important to understand why you’re trying to get the feedback. What’s exactly your goal? What’s your hypothesis? What are the constraints around? What are the thresholds around your hypothesis? So if you’re trying to validate that a problem that you’ve come up with is actually important, how many people do you have to hear that problem from, without you prompting, before you decide to not go ahead with it? Or, if you’re trying to figure out how to improve the usability of something. Understand what question you’re trying to answer. And sometimes, realistically, you’re sort of casting the net, you’re trying to see “well, what can I do around here? So let me go explore and see if there’s any problems”. But even then, you should know in advance, that that’s what you’re doing. Not that you’re trying to find a specific thing, but you’re trying to find indications that there might be a problem that you can solve at that point. So, understanding the context that you’re trying to work in. Because that also drives what type of testing you want to do. What type of customer interactions. There are some things where you can use a survey to get good information. But typically a survey is not going to be that good for finding problems. It might be OK for validating that people have a problem, but for finding a problem is going to be terrible.
Daniel:
Great! That’s a very useful list there!