Episode Transcript

Ethan:

Hi this is a Responsive Web Design Podcast, where we interview the people who make responsive designs happen. I’m your host, Ethan Marcotte.

Karen:

And I’m your other host, Karen McGrane.

Ethan:

And this week, well, literal backflips of delight are happening here at the Responsive Web Design Podcasting Studio, because we have with us today Amber Reed and Jeremia Kimelman, who are here to tell us about the FBI’s Crime Data Explorer. Amber, Jeremia, thank you so much for joining us.

Jeremia:

Thank you for having us.

Amber:

Thank you.

Ethan:

So Amber, Jeremia, maybe you could introduce yourselves to our listeners, tell us a little bit about what you do, and tell us a little bit about the new Crime Data Explorer for the FBI.

Amber:

Hi, my name’s Amber Reed, I’m very excited to be here today. I’m a designer formally with 18F’s experience design team. Jeremia and I, and a team of super talented people from the 18F and the FBI worked on the Crime Data Explorer over the past year. My background includes work as a visual designer, experience designer, researcher, and a service designer, all of which were super useful on this project.

Jeremia:

Hey, I’m Jeremia, long time listener, first time guest. Excited to be here. I am a website developer at 18F, and like Amber said, we built this tool in collaboration with the FBI over the last year. My background is in web development, and before 18F I did a program called Code For America, so I’ve kind of been in this government website space for a couple of years now. The goal with the Crime Data Explorer was to take this national data collection effort that the FBI has been doing for over eighty years and update it, modernize it to be really useful to folks in 2017.

Ethan:

Well, that’s great. Maybe we could talk a little bit about how the project got started. Specifically, one of the things that struck me while I was browsing through the Crime Data Explorer is how much stuff is on the website. It’s an incredibly dense amount of information that’s being published. So I’d love to hear, when you were getting this responsive redesign started, were there any questions from your partners at the FBI about responsive design as a methodology? Did they feel like they might have to compromise on the scope of what they were trying to do when they were thinking about multiple devices?

Jeremia:

From my perspective, it was not the most challenging conversation I’ve had about responsive web design. This is a federal government website; there’s a really cool project that 18F worked on called Analytics.USA.gov, where you can see real time analytics from across all of the United States government websites. Over two and a half billion visits in the last couple of months, forty percent are mobile. So, it’s not so much a conversation of do we need to this, it’s we do need to do this. We work for the government, we need to build services for the public, so we need to make sure we can meet people where they are. To their credit, the FBI, when they came to us, already knew this and had an idea that it would work on phones and work on desktop displays as well.

Karen:

Let me follow up on that quickly and ask if you had any scenarios of use in mind where people might need access to this data on a mobile device or in a mobile context, and if you imagined that there might be other use cases that would be different for people coming in on a desktop computer.

Amber:

One thing that we did is we started off this project with a lot of research, discovery research, talking to different people to find out what were their needs, and one of the things that we started off with in the beginning was that we understood that there’s a range of needs, going from novice users, people interested in the data but not super familiar with it, to expert users. Our hypothesis was that, depending on what people plan to do with the data, they might be interested in it on different devices. So, we made some assumptions in the beginning that, depending on the platform somebody was accessing the data with, they might have different end goals in mind.

Jeremia:

One of the examples that I can give to back up Amber’s point is we assume that if you’re on the phone that you’re not interested in downloading CSVs and starting to actually crunch the raw data. That’s just not a behavior that we assume people would use their phones for, and in our research that turned out to be pretty true. So, on the desktop there’s a really rich experience where you download the CSVs and you can pull those into Excel or Google Sheets or whatever your tool of choice is. On the phone, we really focused more on the kind of persona Amber was talking about: someone who is not as familiar with the program and is kind of trying to get their head around the question, “What is crime in my area?”

Ethan:

I’d love to hear a little bit about how you fed some of that research and some of those requirements into the design process. When you’re actually starting to think about some of these different classes of users, what kind of tools were you using to start sketching out the layout for this? How did you actually start talking about design with your partners at the FBI? Basically, I’d love to just hear a little bit more about how that design process got started.

Amber:

So, moving from that discovery research into initial design directions was an interesting process. What we did was we held some workshops with our partners at the FBI, talked to them about the things that we had heard, and then we started sketching. So, we did a lot of hand sketching, we focused on co-creation and collaboration through this process; and moved from a storyboard format into initial prototypes. So, we went from hand sketching to Sketch files with low fidelity and minimal interactivity to validate some of our ideas.

Karen:

Let me ask a little bit about how you understood the format and the structure of the data that you were going to be working with. One of the things that I encounter a lot in working with organizations that are shifting to a responsive design is that they need to look a lot more carefully at the sizes and formats of things, and to make sure that everything from a block of text all the way down to various labels will work appropriately across a variety of smaller sizes of screens. How did you get your arms around that, and did you run into any problems with making sure that all of the content fit?

Jeremia:

Yeah, we definitely ran into some problems and challenges trying to do that. There’s two things with this site: there is a ton of raw data, and then there’s a ton of content that is generated from that data—content such as charts and descriptive sentences that describe the trend line that you see in those charts, or things like that. First, to understand what we were working with, we really had to explore the data. As I said before, the FBI has been collecting this data for a number of decades, and it’s one of the larger national data sets related to police activity and crime in the country. When we tried to load it, it was well over a terabyte. Just getting our hands around what that meant was really hard—and actually, when Amber just described our design process, we use that for our internal team communication as well, because it was really hard to understand if we wanted to show it in this way, what does that actually mean. Not necessarily what does that mean on a small screen, but what does that mean with our data. Is it going to look broken because it’s going to be so empty?

And then on top of that, once we tried to take the data and turn it into paragraphs, there’s kind of this feature of the site where a lot of the data you can download is described in prose. Something like “Connecticut’s reported homicide rate increased this much from the last year.” And so as we were starting to generate that content and starting to play around with the fact that there’s over 18,000 law enforcement agencies all with varying name length, our design started to really meet that friction point, where it’s like, okay, on a small phone, some really long police department name is going to look totally different than on a desktop display, and, in addition, that chart might look totally different because it needs to be much smaller. So, we did run into those problems.

What was really nice, in my opinion, was that there was kind of a relentless focus on that it would work on phones and that it would work on desktop displays, and that it would work on some number of devices in between. And so we always had to make trade-offs, but they always felt relatively appropriate because we were still trying to get users the interface on their device.

Amber:

Something that I think is interesting there—because, as Jeremia was saying, there’s so much data and there’s so many different things that we could have shown or revealed in the interface, we really let the user needs help guide us to what would be the most valuable information to display in the interface. So, we were working not only with the technical challenges of the data, trying to figure out what was the most interesting or applicable or relevant data to display, and then understanding what were our partners goals with this product, too. So, trying to bring the three together was a challenge, but great.

Ethan:

That’s fantastic. I’d love to hear a little bit more about how you reviewed the responsive prototypes with the FBI and other members of the organization. We talk to a lot of organizations that are still trying to get their hands around actually conducting review sessions and gathering feedback on actual prototypes rather than traditional design assets. Did you have any challenges there? Or if you want to share some lessons about how you managed that process, I’d love to hear how that went.

Jeremia:

My lesson learned from this project and from working in the style that Amber outlined is that prototypes are one of the more valuable tools you can use for communication. When we would talk with the FBI about what the site might look like with static designs, we quickly noticed that people, ourselves included, project onto static designs what they expect things to do. That’s kind of a communication breakdown, because now you don’t have a shared set of expectations. So, what we would do is we would use these prototypes to communicate that. There’s a really nice side benefit—I would even call it maybe a primary benefit—of using prototypes, which is that you can do real user interactions. Under Amber’s leadership, we engaged in a grueling testing regime, doing it every two weeks. Which, I say “grueling,” but it was really nice. That’s a fast pace to build a large data application and test it every two weeks. We would be able to show that to users, and a part of the technical challenge was that we had to have them show their screen. So, we ended up seeing the site on a number of different browsers and a number of different devices, on a number of different displays, and that helped us to understand a little bit more about what kind of aspects were not working responsively.

If I can just highlight a challenge and something I might do better and work on in the future, is that we didn’t really do a whole lot of user testing on phones. We tested it ourselves; our colleagues would test it and we would, as we were making it, try it out on different devices and such. But because of the technology for screen-sharing, it was pretty hard to do user testing for phones. So on my next project that has a similar responsive demand or similar responsive aspect, I would definitely test with real users on phones.

Karen:

That might be a good lead-in to my next question, which is about how you measured or evaluated this site in general. So, what kind of research or data or analytics are you looking at to know if the site is working and know if it’s meeting the mark with your various audiences?

Amber:

I would say that the larger question for us has been how do we measure the success of design in general. At 18F, we rely on getting feedback from users throughout the process. So, we’re working iteratively to refine as we go. We don’t necessarily have quantitative indexes, but often our partners do. We’re ensuring that our design is meeting our partners’ staff and users’ needs, and that’s our success criteria.

Jeremia:

I one-hundred percent agree with Amber’s expression that we use this qualitative metric of user feedback to judge success. We also use analytics, and we do measure things, but we just find that sometimes success is a little bit hard to measure by proxy. So, we use quantitative metrics for more granular things, like performance. We are concerned with performance and the size of the page, how fast it is to use for users. So, we did care about those metrics, but for the success of the site, generally it was really helped by this every two-week user testing session, making sure that we were building a thing that people wanted to use.

Ethan:

Well Jeremia, I have to admit, when you said the word “performance,” my ears perked up a little bit. I just have to ask about that, because that’s something that I think a lot of organizations are also trying to figure out how best to design for. Not just creating these flexible sites, but also making sure they’re as fast as possible. Can you tell me a little bit about how you actually tested for performance, or what kind of metrics you were using to ensure the Crime Data Explorer was as fast as possible?

Jeremia:

Absolutely. So, performance makes sense in the abstract, but it’s kind of hard to measure sometimes unless you get a little bit more granular about it. In website building land, there’s a ton of different metrics for performance. So as a team, we kind of sat down and looked at what is the value of each of these metrics and what do we think we can use really well to make sure that the site meets users where they are and works for them. The two we came up with were the total size of the site, so the amount of information that’s transmitted across the internet… This was really important for us for mobile phones, because mobile phones are generally on cellular connections, and so we really wanted to minimize the amount of data that needed to go across the wire in order for the user to have a workable site.

And then right after that, what we would measure is the time-to-first-interaction. So, there’s a couple different tools that are out there that can do this in an automated way. I think Google just put one out called Lighthouse. There’s a number of them and they’re great, because what they can help you do is run a couple easy user scenarios and see how many seconds does someone generally have to wait on the 3G connection before they can actually use the site. For us, as soon as you get up into the four or five-second range, that starts to get really unacceptable—and probably for a lot of people that’s even too long. So, we try and cut that down as much as possible, in addition to the number of bytes for the website.

Karen:

Well, this has been a really interesting interview, and I think it reflects the quality of the site. I’d love to know from each one of you, if you have any advice for other people, other organizations that are thinking about doing a responsive web design, maybe a responsive design that has a lot of heavy data, what would you tell those people?

Jeremia:

I think my biggest takeaway is that sometimes I hear people talk about responsive redesigns and it sounds like they think it’s just going to be a ton of work. And it is—I don’t want to sugar coat that. But the fact of the matter is it’s really exciting work, because what we’re doing is making sites work better for users on a whole bunch of different devices. As someone who makes websites, the joy that I get from this profession is really building a thing that people like to use. So, when something is responsive and is better to use, that makes me more excited about building it. So, that’s kind of fuzzy advice, but basically I would just say remember that you’re doing this for the users for a reason, and that’s a reason to be excited.

Amber:

What I’ve learned from this process and the advice that I would give to people is to really start by understanding users’ needs, and to think about their needs across platforms and devices and how they might be different. In addition to that, the next thing that I found to be super helpful and successful on this project was to work iteratively. So, to continue to develop designs, put them in front of people, get feedback, and then go back to the drawing board and refine as you go.

Ethan:

Well, that was some wonderful advice to close things out on. Amber, Jeremia, thank you so much for coming on the show. The new site is looking beautiful. I can’t thank you enough for coming on. Thanks very much.

Jeremia:

Thank you.

Amber:

Thank you.

Karen:

Thanks to everyone for listening to this episode of a responsive web design podcast.

If your company wants to go responsive but you need help getting started, we offer a two-day onsite workshop to help you make it happen. We also offer these workshops to the public, so please go to responsivewebdesign.com and see when we’ll be in a city near you.

If you want even more from us, you can sign up for our newsletter, subscribe to this podcast, and read full transcripts of every podcast episode at responsivewebdesign.com.

Thanks for listening, and we’ll be back next week.