Tommaso Colombino

INTERVIEWER

Tommaso is on the phone in the [Xerox Europe] Research Centre in Grenoble, [France]. Hi, Tommaso. Can you tell us a bit about you and your role?

TOMMASO

Sure. I’m a senior scientist in a group called the Work Practice Technology here in Grenoble. We are a multidisciplinary team comprised of social scientists and computer scientists, and our research activities are mainly based around conducting detailed qualitative studies of the organization of work. And we use the findings of those studies to design and propose new technologies and better configurations of existing technologies in organizational processes.

INTERVIEWER

I guess a good point to start with would be talking about the techniques that you use for ethnography.

TOMMASO

The main technique is observation. So, it’s field studies. We go on site. We spend a period that can vary from a few days to two weeks, three weeks. We sit down with people. We observe their work. We shadow them, which means that we follow them around as they do their work, and mostly we take notes. Occasionally we use video as well, and audio recordings, but that is more intrusive and so I prefer to rely more on field notes.

INTERVIEWER

What sort of situations would you use videos as opposed to field notes?

TOMMASO

I would use video when I’m observing something which is difficult to record, because it’s too complex or multifaceted. So, I like to record what people do on their machines or on computer screens, things like that. Then it helps, because then I can play the video back and I can look at all the detail of what’s happening, which I wouldn’t be able to capture in real time.

INTERVIEWER

And what are the differences in ethnography as opposed to, say, a focus group or a one-on-one interview, or even something like market research?

TOMMASO

The difference is that when you use those types of qualitative methodologies, what you rely is on what people think to tell you about their work, but the reality of work and work practice is that people know how to do a lot of things that they wouldn’t think to talk about when you have them in an interview or in a focus group. They usually won’t talk about things like how they handle exceptions and contingencies, what their informal networks are, what the, sort of, on-the-job learning is like. All the things that are undocumented. And the goal of an ethnographic study is precisely to uncover those things.

INTERVIEWER

Can you talk about some of your work specifically? So, a project you’ve worked on and used your ethnographic approach, so we can get a sense of how it actually happens on the ground?

TOMMASO

Sure. One of the parts of the business that we’ve been collaborating extensively in the past few years is customer care and, more specifically, outsourced customer contact centers, so call centers. So, we went in with a remit to look at the works of the call center and their management and, in particular, to look at those parts of the job like compensation mechanisms and performance management practices that were likely to be drivers of attrition.

INTERVIEWER

Okay. And so what did you do?

TOMMASO

Well, what we did was, we sat with the agents and we sat with the supervisors, and we tried to sit through all those parts of the work that are relevant to performance management in the call center. So, we observed things like coaching sessions between agents and supervisors, training, and we also observed the routine work of the agents on the phone with the customers in order to understand what their routine was like.

INTERVIEWER

And what kind of insights did you get from that?

TOMMASO

The way performance management is generally handled, like I said, is through coaching sessions between agents and their supervisors, and these typically take place every week. And during a coaching session, supervisors review the ongoing performance of an agent, both with respect to the qualitative and quantitative metrics they are assessed on and may provide specific goals and objectives for the agents to attain as well as practical advice on how to get there. So, the goal of that is to provide agents with a right level of awareness and understanding of their ongoing performance within the broader context of the call center operations, and to provide motivation and performance-related incentives to employees.

Now, when we observed the process we noticed two issues, which we thought that needed to be addressed. The first issue was that there was no one place that the supervisors could go to gather up-to-date information on the performance metrics of their agents. What they actually had to do was pull information from multiple sources and people in order to create a static snapshot of the data that they could then share with the agents. And this is a process that consumed a lot of time that the supervisors felt would be better spent actually helping the agents with their routine work. The second issue was that the agents themselves had very limited independent access to up-to-date information about their own performance, meaning that between coaching sessions which would take place every week or every other week, it was difficult for them to keep track of where they actually were with respect to their goals.

INTERVIEWER

Let’s move on to talking about the make-up of your team and how it’s different from other teams of ethnographers. So, some ethnographers work independently and then hand their work onto developers, but your team’s set up differently, isn’t it?

TOMMASO

We are set up slightly differently, because we have computer scientists which are actually part of our team. So, what that means is that we can take our studies and the outcome of the studies and actually turn those into a concrete proposal for a technology. What that means is, we don’t actually develop the technology itself. We don’t have the resources to do that. But we can go as far as producing either a mock-up, a concept design or, if given the time, a, sort of, working prototype.

INTERVIEWER

And how does that differ from the other way of doing it? So, ethnography working independently of computer scientists or developers?

TOMMASO

If you have a team that’s made up entirely of ethnographers, then you’re more likely to work on more, sort of, short-term consultancy-type projects and get involved with technology development when some other group comes and asks for your help or your involvement, i.e. some other group that has the technology competences or is doing research on some core technology. We have the option of planning our own projects, long-term projects, which start with ethnographic studies and end with some type of technology proposal.

INTERVIEWER

Okay. Are there any disadvantages to working that way?

TOMMASO

I would not say that there are disadvantages, but there is a challenge involved in finding a good way to communicate your findings to the technology part of the team, and there is also a challenge involved in scheduling.

INTERVIEWER

What is the challenge in communication? And how do you overcome that?

TOMMASO

The challenge in communication is in making what is essentially a description of how people do their work relevant to someone who hasn’t actually observed their work like you have. So it’s finding the right amount of detail so that you don’t overwhelm people with too much detail. And it’s being able to communicate that effectively without wasting too much time going through, you know, hours and hours of material, which is what we collect. I mean, we have sometimes hours of recording, you know, hundreds of pages of field notes. And we need to be able to condense that into something interesting and into something that speaks to someone who has an expertise in computer science and technology design so that it can help them come up with a potential solution or innovative idea. That’s not always easy to do. We have the advantage of having worked together for a long time so that we know each other and we’ve built a relationship that allows us to do that. But there’s no sure-fire method of ensuring success in these types of projects.

INTERVIEWER

In terms of the actual practicalities, do you give a presentation? Do you give a report? Do you use visuals? What sort of methods do you use for communicating your findings?

TOMMASO

We use all of those methods. It, sort of, depends on how people prefer to receive the information. My preference is to explain things to people face to face. So we have sessions where we go through perhaps a presentation that I prepared, but it’s very conversational, it’s iterative. And so that I have a chance to really walk people through a scenario or examples from the field work that I think illustrate things that are interesting about the work that we observed. My least preferred solution is to just write a report, package it up and hand it off.

INTERVIEWER

In terms of selling the business case for ethnography, is there any way in which ethnography can fail or are there any ways in which it’s frustrating to you?

TOMMASO

Well, there’s always a risk that when you do an ethnographic study, you won’t learn anything that is particularly interesting and useful. That does not happen to me very often, because usually very few people bother in an organization to take the time to go actually observe the work as it happens that there are very good chances that you will see or learn something that even the management or the people involved in the work themselves hadn’t thought of. But there’s always the risk that you’re not going to be able to address the issue that you were asked to look at in the first place.

INTERVIEWER

Is there any way of mitigating that upfront in terms of, you’ve made the mistake where you’ve gone into a study and you haven’t got anything out of it? Have you learned, as an ethnographer, how to avoid doing that again?

TOMMASO

It’s a matter of making, you know, clear what it is that you can deliver and what expectations people should have. The thing that we don’t do is promise that we will lower attrition by 10% within the next six months or something like that, just to get them to agree to the study. If we do that, then we’re setting ourselves up for failure. So we try to, sort of, be realistic when we explain to people what we’re gonna do and what the outcome is going to be.

INTERVIEWER

When you talk about metrics and quantitative assessments of success or failure, it brings up the question of the big data age, and people want facts and figures, sometimes not the more qualitative findings that ethnography will produce. Is that an issue that you see? Or, do you have to find a way to work alongside big data?

TOMMASO

We are actually often very keen to work alongside big data, because I think there is a potential complementarity between the methods. To give you an example, with the, again, coming from the call centers, we have done-, not “we” in our group, but “we” as a research center, big analytic studies of the, sort of, agent performance data and the metrics. And one of the things that we saw was that there was a, sort of, non-linear correlation between average handle time, that is to say, the amount of time that agents spend on their phone with the customer, and customer satisfaction surveys, or positive customer satisfaction surveys. Meaning that up to a point the more time you spend on the phone with a customer, the likelier it is that the customer is going to be happy with the interaction with the call center, but beyond the certain point, that relationship starts to drop.

And so that was an interesting finding that came from the analytics, but there was no explanation attached to that finding. Whereas when we, sort of, went into the call center, we were also able to -- because we witnessed some particularly long phone calls -- we were able to tell them that when phone calls last beyond a certain amount of time, it’s most likely indicative of the fact that there’s a very serious problem. And that the outcome of that interaction beyond a certain period of time is not likely to be a good one for the customer, which explains that pattern. So in that case we were able to work together quite effectively, and this is something that we like to do. You know, we think that we can add a lot of the insight into what big data and analytics can discover.

INTERVIEWER

Thanks so much for speaking to me, Tommaso.