In the last couple of modules, we talked about the concept of error or ways that your surveys might be wrong and lead you to make bad decisions based off bad data. One of the primary ways that I think you can reduce error and to help have really constructive surveys that help inform your decision making process, is to be very clear about how you define your population. So, in this module, what we're going to talk about is defining the actual group of users that you're interested in representing with your survey. I think the more clear you can be about how you define these users, the better actionable results you're going to get for your UX goals. Part of that of course is matching the populations that you have to these overall user experience goals that you have. So, I've been defining population as the users you are interested in. I think professional survey researchers define population as what we would colloquially think of as the US or world population. For us, as UX researchers, population can often mean different things. As long as you're clear about the types of people you're trying to represent, I think that you can say some very clear stories based off your survey results. That's why it's so important to define your population. If you can really be clear about this set of users, you get to craft questions that address more meaningfully and in a targeted way what you're actually trying to get from them. You avoid wasted effort for you and your survey respondents. We're going to talk later about the ethics of survey research, but one prior concern we should always have is, are we asking people to contribute effort to our survey in a way that's wasteful of their time and respectful of their efforts on our behalf? It also makes the analysis much easier on the back-end if you're more targeted about the population that you're interested in. You become less floundering in the story you create from your survey data if you're clear about that story from the get-go. So, defining your survey goals is a matter of defining your UX goal for a particular survey, and using that to help you define your population of interest. Not every person is always going to be of interest to you when you have a UX goal. Having more specific goals and more specific populations will lead to more specific data collection efforts, and help you to maximize your efficiency and data-collection quality while minimizing your cost and time. Previous courses in this many series on UX that had a lot of information about how you set those UX goals, so I'm going to point you back to those, and how to think about these actual UX goals that you might have. I am going to talk about, however, some UX survey goals that you might have. So, are older users more or less satisfied with the latest version of our application? I think this is a great research question to think about, because it defines basically what are you going to do with your survey. Here, your population of interest is older users. How do you define older? That's going to be essential for who you access in your population. Defining older as, for instance, over 30, is an entirely different context than defining older as older than 65. A lot of research has shown that the difference between 65 and 85 is hugely different than 85 and older. How you define older is really defining your population at that point. Here's another example. Are people satisfied with placement of ads on this page? So, in this case, people is a very broad category, but you've defined it further with placement of ads on this page. So, they have to have experienced the placement of ads on that page in order to have a preference for that. So, while people can be a very broad category, having that population so broad means that you want them to experience the actual placement, well, before eliciting their preferences. So, that could be something that they've done in the while where they've actually experienced placement of ads on a page as they use a site, or it could be in a lab setting where you're setting up a wire frame with ad placement and they experienced that. In this case however, people so broad that you might be losing a lot of detail in the breadth of that population definition. So, here's another example. How are expert users of this service going to react to this design? This is a similar set of problems that we saw with the population being defined by older users. How do we define expert users? Is it that there are people who have been on the site longer? Is that they posted more things? Is it that they have a higher rating by other users? In the online community user experience work that I've done, this is a consistent problem that we have. How do you define core members or how do you define expert members? Defining the parameters of that population, or how you operationalize, what expert means, in this case, is essential, because your UX goals are based off that operationalization. If you do not get that correct, you might be misrepresenting the very group of users you're actually interested in. Then, once you've defined that population, how do you access those users? That becomes an entirely separate set of issues. Finally, we have a example here. How did that user find their online shopping experience? So, this is a different question overall, because here the population is experiential. That user is a particular persons. What we're doing is we're defining a set of users by a purposive experience that they've had. In this case, a shopping experience in an online setting. So, that becomes a different list of population that we have access to. In this case, the population can be defined by a set of actions that they take within a context that we want. That creates an entirely different contexts, of course, than a more broader people, or older people, or anything like that. So, defining and characterizing a particular population becomes hugely important. So, here's some things to consider when you're defining your populations. One is to think about demographics. Demographics of course include personal characteristics like education level, or income, or gender, all of those things that we commonly think about, but they can also include experiences. One of the most challenging questions that we've seen in survey research is, how do you get a person to characterize their Internet expertise? How good are you at being online? As a deceptively hard question to ask, and it's an important demographic that shapes a lot of people's experiences with online tools. A lot of people, I think have the tendency to collect these demographics through post hoc data collection. They throw up a big block of questions at the end of their survey that asked demos that they might be interested in, and use that to retroactively see what different populations do. That's a tactic, and it sometimes works. But I think it's better to think about your demographics at the front end of your survey, and think about really what types of experiences and personal characteristics are you interested in representing through this UX work. Another thing to really think about with your population is the survey mode that you're going to use. So, how are you delivering that survey is going to shape the types of population that you get access to. While in the United States at least, the digital divide has been closing for a number of years, there are still vast differences in Internet literacy and access between people in different parts of the world and different parts of the country. So, for instance, older Americans are less likely to be on the Internet, they're also less likely to have Internet literacy, though of course that's a general trend, there are definitely older people who are incredibly Internet literate. Rural areas differ from urban areas, different areas of the country, different from other areas of the country. So, thinking about how you deliver a survey is an important way to think about how you're going to access people. An increasingly popular way to ask survey questions, for instance, is through text messaging or SMS. That, of course, is going to limit the types of people you have access to. For people who are comfortable accessing survey research through that mode. Finally, think about that person's relationship to the product. A lot of us, I think do a UX research where we're interested in a person who has a particular relationship with a product or service and understanding more about their experience. So, is the person a new user? Are they an experienced user? Are they user who suddenly has dropped off in the types of experiences they've had? Are they user who had a particular experience that we want to represent? Are they somebody who has no experience with the product and we want to see how they experienced that new product? All these are different ways that we can define this population, and help us to craft more targeted, less error-prone surveys in the long run. So, defining the goals of your survey for UX and the target users that you want to represent will help you to sharpen the questions you ask them, and refine the story you're going to tell to other members of your team later in the process. Being crystal-clear about the users you're trying to access through your survey tools really helps to create a high-quality survey while mitigating cost and time.
