Front Porch 2015 – Kelly Moran, Learning from your users: Ethnography in Software

hi everyone okay good all right uh to anthropologists walk into a bar no actually to anthropologists walk into an island I forgot so to anthropologists walk into an island and they decide that they’re going to study different sides of the island and then at about six months come back together and compare notes so they go off and they come back and they mean the center of the island and the first anthropologist says this is so exciting I learned the most amazing thing I pointed at a rock and I said what is that called and the islanders said Unga Bunga and then I pointed at a mountain and I said what is that and the Islanders said Unga Bunga this is amazing this has deep cultural implications for their thoughts of perspective and time and space and sighs this is just incredible I’ve on earth something amazing and the second anthropologist said that’s very interesting yes on my side of the island unga-bunga means pointing your finger so a little lesson in not getting too carried away with yourself sort of basing yourself in reality when you are trying to learn from people so we’re going to talk today about learning from your users everyone here is in some way involved with building things right creating and making things and you work really hard at that right most of you have figured out ways to basically make things work right users can use them but your users don’t always appreciate what you build and that’s frustrating right I mean why don’t they realize how hard you’re working why don’t they get it right how many people have users that just don’t get it no one wants to raise unhandcuff will raise your hand good free right they’re not getting it but the problem is did you try and get them first right you don’t know the insights to their lives you don’t know the little details to what they’re really trying to do so while you can make something usable and functional something that people can use to get things done what you really want to do is something that people love to interact with and I want to give you a few pointers on how to do that today so first let’s consider this a little bit of a lack of understanding there on both of those levels right so what do we do about that well first of all my name is Kelly Moran I am a design research or a project to a2 and i’m also an anthropologist has anyone here ever been able to work with a researcher before has anyone observed research happening before has anyone ever met an anthropologist before all right all right not very many and I think half of you I know great so Who am I where do my people come from or what island in my friend well traditionally we come from a university and I do have the fancy book-learning behind me I’ve also done some work out in the wild i served in the u.s. Peace Corps in Uzbekistan for 27 months and I did a study abroad in Saudi Arabia that bottom circle there I always forget to point out what that is those are people one of those as me i’m the tall one and we’re wearing the traditional women’s garments in saudi arabia the hat is not traditional but someone gave it to me and i thought i should wear it so not a traditional part of the saudi woman swear after all of that I found a couple of jobs work with some companies eventually ended up at project 202 a project to a 2 we like to uncover user needs ooh that totally got lost a cover user needs design great solutions and build solutions out to launch and at project 22 I think we have a really unique situation because what we do is we have researchers designers and developers and each one of those is allowed to evolve and build their practice and we draw on the strengths of out of all of those to do that really well though we have to understand one another we have to break down some of the some of the silos some of the gaps we have between us so to do that here today lets you and I get a few things out on the table let’s talk about stereotypes and preconceived notions okay so who am I am an anthropologist what is that like am i studying you right now no all right do you do your job all the time no I’m not interested in studying individuals on the sly sneakily I do it for work I study groups of people I like to see how they do things with their consent another one i watch the show bones you must like dead people I do that this one’s fun you must wear a lot of turquoise jewelry sing Kumbaya a new

drum circles I seriously get this that’s hippies it’s fine it’s not mutually exclusive but not necessarily anthropologists okay your turn you’re not very good with people you don’t really like people and you avoid people at all costs and yet here you all are at this conference in this big room together this is a really unhealthy way to think of the people that you work with this is a terrible thing to think about people you totally want to build an app for me I have this really good idea it’s gonna be a start-up you’re gonna just like build it for free because then you’ll get like some of the profits or whatever um and you would love to do that for me right because I’m studying you right now because I never stopped doing my job and you never stopped doing your job and you would do it for free um this one is I’m really prevalent um you make stuff like you make stuff that’s that from nothing I don’t understand how that’s not creative but I hear this a lot I hear this from developers I did a poll in my office what are the top three stereotypes that you get and this one came back from everyone so not cool I don’t know what like a creative is they say like oh the developers aren’t part of the creatives like it like do they go and packs what our creatives what do they do I don’t know so now that we have it on the table and we know that those things were sort of silly through our powers combined we can start to understand the users better now that we’re understanding one another a little better there is another piece of this puzzle right users don’t know what they want appreciate us and they’re just doing it wrong how do we feed these stereotypes what do we do well according to project 22 we do design research we direct the strategy in design for software projects we understand our users my favorite is Margaret Mead I’m sorry the yellow is not showing what it says is that anthropology demands the open-mindedness with which one must look and listen record an astonishment and wonder that which one would not have been able to guess not have been able to guess right don’t use those stereotypes don’t rely on those you want to actually observe people talk to them don’t guess right figure out what your users are doing by involving them have another example of some Islanders these are these are two villagers and the different groups on the different sides of the island where different color lenses one group is the blue lens tribe they run around seeing everything as blue the other group of villagers is the yellow lenses and they run around and say everything is yellow and they don’t exactly see eye to eye because they’re seeing these different colors all the time so one of the villagers decides that she’s going to go over to the yellow side and put on a pair of yellow lenses and she’s going to spend some time with them this is a great ideas a great start so she goes and she spend some time with yellow lenses on then she comes back to her blue lens people and they say what did you learn and she said it was so amazing because everything was Green right so part of understanding other people is understanding that everyone has their own unique lenses everybody has a different perspective in a different view on which they look at the world and so part of understanding what your users are going to need is to try and step into that to understand that you have your lens and they have theirs and figure out where those differences are so what is ethnography anyways right that’s a that’s a word that’s been going around it’s been used in business lately you see it all over the place I see it all over the place who has heard about ethnography before not many right who’s like Reddit and like harvest business review or seen it like come up on some big fancy business page on the yeah couple yeah so so what is it all right there’s a lot of misconception out with ethnography is well first of all ethnography is a descriptive study of peoples and cultures that’s the basic definition you’ll hear all kinds of definitions for what it is but this is what it is a descriptive study of people and cultures ethnographic research importantly is an approach not a method it’s not a specific exact prescriptive way of doing things it’s the way of approaching a problem it’s a philosophy it’s a way of looking at things as such it favors qualitative methods / quantitative so describing things instead of counting them and a famous anthropologist clifford geertz has said that ethnography should seek thick description so thick description meaning providing context providing information that your audience wouldn’t have if you don’t provide it to them right so his classic example is a wink if you describe a wink without using thick description if you describe it thinly a

wink is the closing and opening of an eye right very straightforward but people can do that for a lot of different reasons you might wink because you’re flirting or because you’re letting someone in on a joke or you might just have something stuck in your contact lens so if you are describing a situation and you say Jenny winked at Joe and you don’t provide any thick description you’re irresponsibly allowing your audience to interpret that however they need to and you might get some pretty bad rumors about Jenny and Joe going on when she just had dust in her eye right so you need to provide a thick description what was happening before the wink and after the wink who else saw it how did they respond what is Jenny and Joe’s relationship so when you’re studying your users you need to understand a little bit more about what they’re doing and how they’re using things and that wider environment around it ethnographic research is also conducted in context where things happen lab studies are a great way to learn things in a controlled environment there for a specific purpose they’re not ethnography they’re just a different thing we say that if you want to understand the pen you must understand the paper right if you were building the perf building a pen and you didn’t think what kind of paper it was going to be used on you’re not going to be providing a very good solution to that ethnographic research is also systematic but responsive so this is sort of tackling two myths that I hear about ethnography one is that ethnography is just kind of like hanging out with people it’s a little bit more systematic and orderly than that you have guidelines you have things you want to find when you go out to study your users you have an endpoint in mind you have some thoughts about what’s going on but it’s responsive to emerging trends and themes right so what that means is you might have to change you might have to be a little bit flexible in your approach when you’re going out and you’re working with users you might find they’re doing something completely unexpected and if you just try and go on with the questions you’ve written down to ask them you’ll find that you’re completely off base so you need to be responsive to what’s going on ethnographic research also uses key informants who act as guides and help provide access to the community so that’s often things like stakeholders or subject matter experts when we apply to software so these are people who can give you those good reality checks right unga-bunga does not mean rock right you have someone who can help you tease that apart ethnographic research most importantly seeks out the insider perspective and anthropology called the emic perspective it seeks to see how the users themselves are viewing something and then it leger’s in your outsider perspective you’re edik I know those are super similar anthropology students mess these up all the time and undergrad I think of emic like me the M you’re looking at my perspective so here’s a really good example of that not in use Superman right so Lois Lane says what’s the S stand for Superman says it’s not an S in my world that means hope Lois Lane being a little bit snarky you will hear it’s an s right not really considering the emic perspective she’s a journalist not an ethnographer she is a different approach and it’s funnier this way ethnographic research is also generative so you’re doing it to discover new things if you go out there thinking you already know exactly what your users are doing you’re doing that wrong Sherlock Holmes via Arthur Conan Doyle never theorize before you have data invariably you end up twisting facts to suit theories instead of theories to suit facts ethnographic research also looks to find that implicit or that not typically stated feature of a group what this means is that a survey is not going to work right it’s not the things that people will display tinley tell you it’s the sort of deeper more secretive things I have an example here oops sorry so let’s pretend then aliens come down right and they ask you some questions about how your day goes and what you do in your life so what kinds of things might you tell the aliens that you do get dressed take a shower I hope you take a shower before you get dressed but if you don’t tell them that right you might eat right most people don’t answer that they breathe but they do that all day long so when the aliens take you back and they’ve created a habitat for you and they didn’t include air and then they say well the user didn’t tell me that whose fault is that really right so what is that implicit thing right what is that thing that you only find by observing and watching right we call that making the familiar strange and the strange familiar it’s just sort of twisting your perspective right what color is my lens what color is their lens right what happens when you combine those this is another really good example from the anthropologist horace

minor there’s was a group called the nacirema and the nacirema were an interesting tribe they had some very sort of complex rituals surrounding what they cut thought of as mouth Devils or mouth demons and they had a lot of sort of processes that they would involve to to handle those and to eliminate those and they would they would visit these like holy mouth men all the time very strange but if you flip the word brushing your teeth and going to the dentist right everything can be strange if you’re not understanding it right so for that reason ethnographic research is inclusive there’s a participatory component you want to involve your users right let them know what you’re doing while you’re doing it let them explain to you what they’re doing come back to them say I think when you did this this will help what do you think about that oh no no let’s try it this way right you want to include them so when we apply this to it to software a lot of times it’s called design ethnography and clearly doing a large ethnographic study to build software is expensive and time-consuming so there are a couple of things you can do to make that easier and more feasible one of these is looking at design ethnography as that way to reduce the probability of failure by understanding what’s going on with your users all right harvest business review they talk about ethnography sometimes why proposed product launches fail we didn’t do the research but we’re pretty sure this will work also when you’re doing ethnography are learning from your users you don’t have to call if nog refer you can just call it learning I’m you want to avoid that reliance on self-reported data right those surveys I have an example here we did a project where this was when the term green and eco-friendly it was still really new and this client wanted to understand how to reach those green consumers and they thought well what does green even mean what’s that what’s that mean to be a green person what is that so we handed out disposable cameras because people will still using those back then for more than just weddings and we gave them the cameras for a week said take pictures around your home of things that are green we collected those develop the photos brought them back to them a lot of what we saw were these 7th generation brand cleaners is anyone familiar with these ok the idea behind them is that allegedly they’re based on a Native American belief that every action you take should impact you should consider the implications for the next seven generations right so thinking far ahead for what you do right and so we asked people we said we said do you use these for for all of your cleaning needs your to use a variety of products and they said oh god no I have children I don’t use chemicals in my home I only use these safe non-toxic chemical free products right that’s all that I’d use so seeking to understand a little bit deeper we said can you kind of show us how you use these maybe demonstrate for us and so they took us to the kitchen cabinet and open it up and right next to their seventh generation eco-friendly chemical-free non-toxic cleansers so we said oh what is the bleach for and they said I only use that for the really tough jobs they weren’t lying they were telling us about their aspirational selves right they wanted to be using just those eco-friendly cleaners but they weren’t quite getting the job done so they were doing other things if you had just taken them on their word and tried to build a product for them you might find that they lack confidence in that product because from their experience green things don’t quite work for everything right so we need to understand those things and finally when you’re learning from your users you really want to think qualitatively about what you’re doing so descriptively understanding seeing the context and understanding their problems from their point of view there emic perspective what is it they’re trying to do and then you apply your viewpoint on top of that so let’s look an example of integrating this so what we like to do is we send a research or in a designer together out into the field I would love to start bringing developers out as well I think that’s a great way to to make sure that they’re that y’all are exactly seeing what’s happening right now that’s not quite feasible we do bring them into the lab to observe and we’ve gotten good feedback from that but we send the designer and the researcher out and then they switch roles and the designer begins to design and the researcher sort of consults on that and then we loop in the developers and the researcher comes back in to do test

so researcher gets to follow along a little bit throughout the way I would love it if the developer got to kind of start from the beginning as well and in this way you have that user focused innovation right from the start of the project I have some examples I want to go through with you we did some work on some enterprise software for accountants so professional accountants doing account reconciliations this was a large piece of software customers were saying that there were too many clicks and it’s hard to use most importantly for the client the sales were lagging but they said you know this software needs to be customizable right who’s ever not said that who’s never said that needs to be like flexible and it needs to fit within a suite of other enterprise products for this this is a large project there’s a big product so we need to talk to more people and we talked to 19 users at six sites I think we were in four different states so a large project or not all like that and from and then we talked to three people inside the organization so from the total user count we had 800 unique insights and observations let’s look at a couple of them in context observation the first thing that will happen you try and do this we have everything ready for in a conference room please just go off into this corner we will bring people into you just stay here right you want to mitigate that let’s just see where they’re sitting can we just look at their workstation can we take up a couple pictures oh look let me pull up a chair right be a little bit creative get into that environment don’t let them push you off into the conference room let’s see why why not the conference room what are some things that you see of this user in context two monitors and a calculator yeah still using a calculator two monitors what else this is the first time no one’s pointed out the banana good for you all thank you other people there are other people involved in this files papers sticky notes it’s got a ruler some point he’s going to be doing something with that right this is just what we’re seeing from being in the environment we saw that five out of six of them were using two monitors they were still using adding machines they were too Qbert open space the lots of stuff happening around them paper paper paper and this so sad the system had the ability to look up reference numbers but the way the results returned it was like pages and pages and pages of 10 at a time not really able to filter so you just had to click through and so user who had a broken ankle would rather get up from her desk hobble down the hallway to a locked room open it up get the correct binder come out of the room relock the dorks it was supposed to be secure at all times hobble back down to her desk look through the des flip through the pages find the number she wants write it down flows the book hobble back over to the door unlock and put the book and could take the book I mean this is ridiculous she would rather do that than use the system it was literally at her fingertips but she couldn’t find it right serious problem with this filtering and sorting that’s just what we learned just from going and being in the environment or watching people so what are some things that we learn only actually sat down with them observed and asked a couple of questions so we learned that they really can’t break away mentally from Excel is the most amazing software ever interesting thing they were importing data of multiple types into excel as images because they just love excel so they would take PDFs and put those into an Excel spreadsheet they would take screenshots they were using snippet and taking screenshots of things and putting those into Excel and then they would use the Excel features to highlight and also circle the same piece of relevant data we said wire doing that and they said well it’s better than my manager not seeing it rejecting this because you ruin their numbers for the whole month if their manager kicked something back so they were just making this atrocious like if they knew how to like make things flash they would have put that on there too attachments and cover sheets so inside the software they needed to attach supporting documents and the software allows you to attach almost a limited number of attachments but it took so long for the managers to open each one because there was such a problem with navigating to each attachment and opening and viewing it that they just wanted one attachment so that’s also why they were doing that excel sheet with like PDFs inside it so that you could just the manager could just click once and open the excel sheet and then go through the tabs of the excel sheet they would rather do that than have things organized inside the software this is

really sad they were printing out documents for multiple pdfs that they could physically organize the papers and then rescan them in into the order that they wanted them because they didn’t know how to reorganize inside a PF and they were making these cover sheets in Excel so that in that single attachment of Excel the very first tab they would title cover sheet and they would summarize all the account data and activity for the month and put that on the cover sheet that is exactly what the software is for exactly what the software was for and they are putting it into Excel and attaching it inside the software and pretty much just using that as a holding station the really expensive file cabinet we also saw that their roles weren’t matching up very well the software you could log in as as basic a do or a reconciler or as a reviewer or as an administrator but many companies were doubling up there Dewar’s with their administrators and so the assumed workflow is that you’ve log in as your doer and you would start and then you would go and then you would be done but really what happened is that you would log in as your doer and you would start and then someone would say oh my god I need you to stop right now I have to have you add me as a new account manager on this thing I have to do it now so they have to stop log out log back in as the administrator do that work go back in and the system didn’t really allow for those switching of the roles a lot of switching gear and then they were sort of sad they’ve gone to school to be accountants and they were spending all this time trying to make the software work properly for them and then we saw right all that work was actually being done in excel in spreadsheets and email communications back and forth and then they were just using the system to hold it all so we did several things which we can probably sort of skip over for large part 1 was bringing in Excel patterns of course bringing and here’s my favorite one in the center the software had something called an account homepage that’s something that the the people who developed the software had created and it was basically the cover sheet so we renamed it the cover sheet right don’t make them use your words they have their own words they’re professional accountants users are pros at what they do if they have words for something work that in cleaned up of course the viewing of attachments and loading of attachments that they weren’t making these these horribly ugly terrible Excel sheets with you know six or seven tabs with the pdfs attached inside them and the lights and the circles and the highlights right make that just easier to get in there and then we transition to design and qualified people did things and then you go into testing I just want to have a couple of notes on testing people sometimes get frightened of it but it can be very low-key very scrappy it depends on what you’re trying to do I don’t want people to be afraid of it I don’t want you to be afraid to put your software in front of a user and just say try some things out right and just watch them you will learn a ton just by watching them do what they do so how can you bring ethnography to your projects how can you learn about your users so I have a question for you do you need a researcher to go with you to talk to users no don’t be stereotyped number one you’re not afraid of people go talk to them right if you’re going to go talk to 22 people across four states you might want to have a researcher involved this is where we come in handy but but get started right put your boots on and get out there talk to two people next week a couple weeks later talk to two more i’m going to offer you skype as an alternative if you have that multi-state scope right otherwise if they’re around you you should go to them and what’s not showing up is don’t go in with solutions in mind right if you’re still building make sure you’re figuring out what the problems are first with the users before you go in with solutions brief pause here to talk about informed consent it’s very important that people know when you’re studying them that’s why I’m not studying you right now that was my stereotype number one if you’re doing something low-key like just meeting someone at their office and ask them some questions showing them a few things you can make this very casual you can say by the way I’m going to take some of this information back and work on it with my team is that okay with you

that’s good that’s informed consent you’ve let them know what you’re doing and you’ve given them a chance to say um that’s creepy and weird no and they really rarely say that but you have to give them that chance if you are taking photographs video audio you must tell them before you turn those devices on and you would probably want to go ahead and get written consent for that to make sure that everything is above the board if anyone asks any questions later and so you need to get a written form if you’re going to take that kind of stuff away audio images things like that some tips on observing um remember the physical environment that’s what you’re going in context do you want to see what’s around them are they using two monitors are they still using a calculator you want to see the lighting and the noise this will help you decide if the you I should be dark or light if there’s a lot of distractions they’re going to need help focusing if there are other people involved that they’re talking to what are those roles what is that workflow what artifacts are around your eyes still using paper and binders can you help bring that into the system or do they need that stuff and then documenting it right as nog raphy is the descriptive discipline you want to write things down take the photographs in the audio if you have permission some tips on questions you are not an expert in other people’s lives even if you’re building an app for developers you are only an expert and how you develop you don’t know how they do things so go in with that open mind of i want to learn from you a really good tip is to rephrase the things that they say and ask them if you got it right people love to hear their words brought back to them when you’re having a conversation with a friend pay attention next time you’re probably not listening to like seventy percent of what they say because you’re crafting your own response and you’re trying to remember that you want to tell them about the movie that you saw or the podcast that you listened to you you’re not listening that well this is very natural you want to prove that you are listening so repeat some things that they say so if I heard you right I think what you said was and then let them correct you if they need to sometimes people kind of want like a second chance they want to kind of go back a little bit that’s fine avoid the leading questions so things like this page isn’t really working for you is it right don’t lead them right say things like tell me what’s good on this page and what’s not so good on this page right ask them to describe for you right and then finally take note of their ideas and ask what problem does this solve right we hear a lot my users have really bad ideas my users don’t understand how things are working on the back end so they’re trying to do this really stupid thing all right well they’re not experts in building stuff i like to use Homer Simpson for this is everyone remember the episode where homer got to design his own car right so Homer got to design his own car and it looked at rochas and the whole company fail and his secret brother was involved and he’s terrible Homer’s not an expert at car design Homer had problems and he solved them the best way he could if the designers and the engineers had instead asked him what are you trying to solve what’s bothering you with the way the car is today they could have provided expert opinion so by getting his emic internal perspective they could apply there edik outsider expertise and provided a proper solution two homers car problem don’t expect your users to do your job for you and finally when you’re learning from your users honor the idea of reciprocity they’re giving you something give them something back starbucks gift card something like that let them know that you are really appreciative and that you’ve helped that they have helped you I have a quick recommended reading list for you I am a firm believer that if you’re going to do anything that you’re going to remotely call ethnography or qualitative research you should have read a little bit of that before so as I said ethnographies both a process and it’s also a written product so I have a couple ethnographies for you and they’re kind of fun this one guests of the Sheik this woman um this was back in the 50s she married an anthropologist and for their honeymoon he took her to Iraq so he want to get his thesis work done and so they went off and he very quickly realized that fifty percent of the population was entirely inaccessible to him because he was not allowed to talk to any of the women so he said could you do a little research for me so she went out into the into the village and she did work for him and helped him out and she was really inspired by that and when they came back she can she went back to school and got her PhD as an anthropologist and her stories just just really nice it’s a really interesting read I think this next one is super fun gang leader for a day this was featured

in freakonomics so this is a sociologist who originally was going out and doing survey data and he wanted to study the economics of gangs and he went to one of the worst areas of Chicago with his clipboard and he went up to man and said what is it like to be black in America didn’t go well they took him to a stairwell in a practically abandoned building and held him there didn’t hurt him but held him there for hours until they’re legitimate gang leader arrived and said what do we do with this guy and he said if you want to know what it’s like to be black in America come back tomorrow and to his surprise Sudhir went back and he ended up studying this gang for about a decade and doing some really interesting work on how how economics works inside gangs in the United States so really fun read this one is the heaviest of them all both in content and in style st scholars and schizophrenics mental illness in rural Ireland it’s a heavy read but for those of you who wanted something of it beefier I want to recommend that the final thing I want to leave you with is this quote from another ethnographer bronislaw malinowski ethnography has a goal an ethnographer should never lose sight of this the goal is briefly to grasp the users point of view his relation to life and his vision of his world thank you so much you