Episode Transcript

Ethan:

Hi, this is a Responsive Web Design Podcast, where we interview the people who make responsive designs happen. I’m your host, Ethan Marcotte.

Karen:

And I’m your other host, Karen McGrane.

Ethan:

And this week, well, we’ve got a really special episode for you because we’ve got Dan Chilton and Guillermo Esteves coming to us from Vox Media. We’re going to be talking specifically about the work that they’re doing on performance throughout the entire Vox brand. Dan, Guillermo, thanks so much for joining us.

Guillermo:

Thank you for having us.

Dan:

Yes, thank you. It’s an honor.

Karen:

But before we get on with it, I’d like to welcome back our sponsor, Gather Content. I’m thrilled that they’ll be sponsoring this podcast for the rest of the year. I recommend Gather Content to my own clients who are going through a website redesign. Gather Content provides some much needed structure and editorial workflow to help manage a large-scale content creation process. Because Gather Content works with so many organizations going through a website redesign, they have unique insight into how content fits into a web project. And because they are such great people, they wrote down their advice for you! They’ve put together a 41-page guide to Content Strategy for Website Projects. Head on over to gathercontent.com/RWD to read their advice or download a PDF to share with your team. Thanks, Gather Content!

Ethan:

So, like I said, this is a little bit of a different format for us because we usually have a brand on to kind of talk about a site-wide redesign, whereas I think Vox Media has generally been speaking a little bit about performance and how it impacts their sites across all of their brands. Could you maybe tell our listeners a little bit about that process and some of the work specifically that you’re doing?

Dan:

So, Guillermo and I are both part of the performance team at Vox—and just to give you a little bit of historical context, we officially spun this team up at the beginning of 2015, when it was fully resourced and fully operational. We did a little bit of research and proposals prior to that, leading back to about mid 2014, but the makeup of the team is myself and Guillermo, who act sort of as directors and we communicate out the goals and big wins of the team to the rest of the company. And then we have two performance engineers, Jason Ormand and Ian Carrico, and they are just full-time working on optimizations, giving recommendations to other parts of the company surrounding performance, and evangelizing performance in the community at large; they actually just spoke at Velocity Conf last week. So, that’s the general background and context of the performance team here at Vox.

When we do our reporting out to the rest of the company and we say we knocked off fifty percent, it’s always with the mobile numbers that we do that reporting with.

Ethan:

We like to start our interviews by asking our guests a little bit about how their organization thinks about mobile users and desktop users. At least when we’re talking specifically around performance, those terms tend to get kind of fraught. They tend to think about mobile users as exclusively low bandwidth, or desktop users—wide-screen devices that are exclusively on reliable, fast network speeds. Can you tell me a little bit about how different devices inform how Vox Media talks about performance?

Dan:

So at Vox, we definitely do differentiate between mobile and desktop. We’ve been a mobile-first company for several years now; our traffic has kind of dictated that we do so, since we’re seeing the majority of it come across from mobile devices. So to reflect that on the performance team, when we do our site monitoring, which we use a great tool called SpeedCurve, we actually set it up with a mobile profile and a desktop profile, and when we do our reporting out to the rest of the company and we say we knocked off fifty percent, it’s always with the mobile numbers that we do that reporting with. And that’s set up to be a 3G connection, and the reason why we do that is we want to make sure that the optimizations that we’re making are making the biggest impacts on our readers who would have the slowest connection. There are still people who are using dial-up and such, but 3G is going to be consistently probably the layer of our readership that has the slowest connection, so we want to make sure that they have as quick and performant an experience as possible. So, not only when we’re designing our sites are we mobile-first, but also when we’re monitoring our sites and directing our optimizations you could also consider us mobile-first in that arena, as well.

Karen:

Could you talk a little bit about how the performance team works with the rest of the organization? What kind of people do you have working on actual product performance, and who else do you have to work with, or how do you work with all of the different brands?

Guillermo:

Like Dan mentioned, our team is composed of Ian Carrico and Jason Ormand, they are part of our performance team. But the way that we work, we embed these engineers into different teams in the organization. At the beginning when we started the performance team, we did a lot of work to tackle a lot of the low-hanging fruit that we had in our codebase. But as we started fixing all the stuff that we had leftover over years of building our sites, we moved towards embedding our engineers into other teams. For example, Ian is embedded into a revenue team to help that team improve performance of all of our ad products. Meanwhile, Jason is embedded into the team that’s building the next generation of our Chorus platform. This is a platform in which we’re building Curbed and Recode, and then we’re going to move over the rest of our sites very soon to this platform. And he’s helping make sure that performance is a consideration in the way we build that platform. For example, building responsive images, replacing the old way that we used to lazy load images into our sites, and moving towards responsive images.

Ethan:

I think that sounds great. One of the conversations I’ve been really interested in following over the last few years is how, for performance to be really successfully adopted organization-wide, it has to be seen not just as a development effort but it also has to be considered from a design perspective as well, and when we had Yesenia on the podcast, she spoke really eloquently about that. As you’ve been partnering with different publications and brands at Vox Media, have you seen the design process or the editorial process change as you’re advocating for thinking about faster experiences and slower networks?

Dan:

Yeah, we have seen that change. One of the first things that we set up as a performance team when we were originally bootstrapping the team was a performance budget, and I believe Yesenia mentioned that a little bit during the time that she was on this podcast. But we do set up a budget for them to work within and to try to hit, and so design is very cognizant of that as they’ve been creating new mock-ups and new experiences as we continue to build new sites or refactor other ones that we’ve acquired. So that’s been a really good reference for everyone to work from.

We have started to see over just this last year, 2015 specifically, that teams have started to make sure that performance is a priority.

Ethan:

There’s that tweet that I loved, Mat Marquis who said on Twitter once, “Small screen users definitely want just the core information on a web page and desktop users just want this twelve-megabyte picture of a salad.” I think that comes back to how do you produce artwork, how do you actually think about the design of editorial features, and consider network speed. Have you seen that change or is it still like, “We’re going to think about the design as aesthetics only” and then the performance team kind of comes in at the end and makes it as fast as possible? Or, have you started to see that kind of come in earlier in the design process?

Dan:

When the performance team originally was spun up, I think it was more of them kind of coming in at the end of the process. So design would be done and we would make recommendations and say, “This could work,” or, “Maybe we need to scale back this certain feature because it’s going to be putting an undue amount of page size on our readers; we definitely don’t want that on mobile.”

But as we’ve made documents and we’ve made a lot of reference material available to the team, designers and product managers and everyone, about performance, and we’ve been evangelizing it throughout the entire company, we have started to see over just this last year, 2015 specifically, that teams have started to make sure that performance is a priority. It’s not an afterthought or something that we go in and fix at the tail end of the process, but rather we’re thinking about it from inception all the way through.

There’s some work that we’re doing, and Lauren and Yesenia touched on it during their interview, where Chorus in general—Chorus being our content management system, or “modern media stack,” as we like to call it—it had been segmented in the past, where every site was a bespoke, custom version. So when we had to make changes, it was very specific and it really would snowball into making a lot more work for designers and for engineers than was necessary. So, as we’ve moved towards centralizing all of that code, performance is baked in from the get-go. So while, yes, designers and other members of the company at large have been showing that performance is important, we’ve also made sure to bake that into the actual code and the process itself, so it becomes more of a foregone conclusion and all the stuff that we’re putting out is as performant as possible.

Performance is a more important priority when building internal features and having performance budgets for things like typography and images. I think that’s one of the great consequences of having a performance team.

Guillermo:

And just to add on to that a little bit, just to give you a little bit of background: when I joined Vox back in 2012, one of the things that I used to do was help with some of the high-touch internal features on The Verge. And I can speak from experience that back then when we were a much smaller team and performance wasn’t that much of a priority, we would prioritize the look and the artwork of the features, not so much the performance. It got to a point when we were ready to publish and we would realize, “Oh my god, there are fifty megabytes of images in this feature, this is not great.” Since we’ve started the performance team and we’ve gotten more of the team invested in this and we have great designers, like Yesenia, who are heavily invested in performance, we’ve seen that that has stopped being the case and now performance is a more important priority when building internal features and having performance budgets for things like typography and images. I think that’s one of the great consequences of having a performance team in Vox Media.

The two metrics that are most important to the company at large when we’re talking about performance is our start render time and our page load time.

Ethan:

I think that’s fantastic. One thing you’ve both mentioned so far is one of my favorite phrases, which is a performance budget. Could you tell me a little bit more about how budgets are defined? Are they defined on a per-project basis? Are you setting one that’s sort of the measuring stick for all of Vox Media? I’d love to hear a little bit more about that process and how you use it in your daily work.

Dan:

So, when we originally set out to create a budget, there was a lot of different ideas around whether a budget should be a one-size-fits-all that all pages have to conform to—your home page, an article page, a video page, a gallery page. And then there was the other train of thought that suggested that you should have a custom profile for your budget depending on the content type. We ended up at the beginning going with just the kind of one-size-fits-all, which, again, I’ll mention SpeedCurve, you can set the performance budget goals in there, and then as it monitors your site it’ll tell you whether it’s within the range of the goal or not. We’ve also built some internal tools that track that and we’re continuing to build more tools into the development environment, so when our engineers are working on their local machine, they don’t have to wait to push something into production and then see it be performance regressed at that point. We want them to be able to make sure that everything is within the realm of that performance budget while they’re actually building it.

So, we still have this kind of one-size-fits-all that we’ve broken into our time-based performance budget, where we look at—on the performance team specifically, we get a little more fine-grained with the metrics that we pay attention to, like time to first byte, start render, page load, fully loaded, Speed Index, all that stuff. But when we report out, the two metrics that are most important to the company at large when we’re talking about performance is our start render time and our page load time. We also have custom metrics for the revenue side of things, so we can keep track of when our ads are being rendered and how many are being shown on the page, stuff like that, business concerns, and that’s also handled through SpeedCurve. And then we also have some quantity-based performance budgets, like page requests, we want to keep those as low as possible; with ads included, those can sometimes get pretty bloated, so we want to keep an eye on that. Also the page size, which we’ve already kind of touched on, and it’s been a helpful constraint with the design teams.

They see a little rocket that goes from red to yellow to green to let them know if the performance of the ad that they’re setting up is within the accepted parameters of our budget.

So yeah, in the end, I think this one-size-fits-all budget has been helpful for us. The biggest challenge with it is just making sure that the numbers surrounding that are in front of our developers and in front of our designers as much as possible so they can keep that at the front of their mind while they’re developing or designing stuff. I think we’ve been pretty successful up to this point, but there’s still some work that can be done as far as exposing that.

Guillermo:

Yeah, we’ve done a really good job of exposing this budget and these metrics. One example that I really like is in our custom ad authoring tool—which our designers put together, the custom high-touch ads that you see on our sites—we’ve set up some custom web page testing so that as they’re putting together these ads with these images and these effects, they see a little rocket that goes from red to yellow to green to let them know if the performance of the ad that they’re setting up is within the accepted parameters of our budget.

Dan:

I also want to plug one more thing real quick. One of our performance engineers, Jason Ormand, actually built an open source tool which you can check out, called Justice.js, and it’s a little dash bar that basically embeds itself at the bottom of the screen and it will show you, in a real time graph, the frames-per-second when you’re scrolling around, so you can pick up on scroll jank really easily. It’s a really neat tool, and that’s available on all of our sites for all of our engineers as they’re developing.

Guillermo:

Yeah, it’s easy to forget that performance is not just how fast the page loads, how fast you see everything on the site, but also how it scrolls as you’re trying to read the content. You don’t want jank and having it jittery as you scroll up and down. So, that’s a really neat tool that we use when we’re putting together especially editorial features, to make sure that all the effects and all the artwork and all the images that we’re putting in are not making a terrible experience for the readers.

One of the original directives of the performance team was we weren’t going to set ourselves up to be performance cops, we weren’t going to go around slapping people on the wrist.

Karen:

How does having this data available for your designers, and developers, and all of the different brands change the conversation? I know there’s lots of organizations out there probably listening to this and thinking about the conversations they have with their stakeholders, where everybody’s got to have the full bleed images, and they want all the fonts, and they think a big carousel is going to be amazing… Does the data really help persuade people that they can’t have all the features they want, they can’t have every single bell and whistle they’ve ever wanted on the page?

Dan:

That’s a really interesting question. So, what we’ve found over this time, and one of the original directives of the performance team was we weren’t going to set ourselves up to be performance cops, we weren’t going to go around slapping people on the wrist, saying, “You built an article that broke the page size budget! You have to take that down or change that immediately!” Our goal setting out was to set up best practices basically, like the budget, make recommendations, be a resource within the company that people can turn to when they have to make performance-related decisions. But ultimately, like anything else, it’s a compromise, and sometimes there’s large high-touch pieces that will have to break the performance budget, and we’re okay with that, we’re not going to shut things down and not let them deploy if that’s the case. So we want to make sure that data is available and if someone does start to complain about performance size, we can point to that and say, well, you know, this was bigger than what we would’ve liked and that’s the reason it slowed down, or you have this weird scroll detector that made a bad experience for the users. But ultimately, we leave that decision up to editorial, we’re not going to intervene there. It’s our job just to make recommendations; we make optimizations ourselves on a larger, kind of network-wide basis and the smaller article-by-article decisions are up to the best judgment of editorial. We would love, yes, all of them to be as small as possible, using as few fonts as possible, and images and all that stuff, and that’s the engineer in us talking. But we understand that it’s a give and take, and our designers are great about that and they haven’t gone off the rails yet, so I think we’re good in that respect.

Performance in advertising is probably one of our biggest priorities on the performance team, that’s why we have an engineer embedded in the revenue team to make things faster.

Karen:

How do you deal with advertising? There’s been a lot of coverage lately about the fact that ads are really the biggest performance drain out there. But they’re also what drives your whole business, so how do you make those trade-offs?

Guillermo:

Well, you know, we also like to read our own sites and it’s not great when our sites load slow. We are the very first people who realize that this is not working great when things load very slowly. So, performance in advertising is probably one of our biggest priorities on the performance team, that’s why we have an engineer embedded in the revenue team to make things faster. And since we’ve had Ian on the team, I think we’ve made great strides to improve the performance of our advertising. We’ve done things like enable async ads so that content loads as fast as possible and isn’t held up by the ads. We’ve done things like optimizing our loading scripts, moving stuff to the bottom of the page, inlining the scripts to remove all the render-blocking scripts from the head of the page. It’s a constant battle. We’re working very hard to make things as fast as possible. I mentioned that tool before, where we tried to expose the performance, that two of the people who are creating those ads are making sure that they’re not making ads that are going to slow our sites down too much.

Dan:

To ad a quick plug to our product blog, the work that Guillermo just mentioned about inlining our JavaScript calls for our ads, Ian actually wrote a great article about that earlier this month on our product blog about how that reduced our start render times by up to forty percent in some cases. So, definitely check that out if you want to get into the technical nitty gritty about it.

Think about your long-term goals of how you’re going to measure performance, how you’re going to keep performance at a perfect level, and how to get the whole team invested in it.

Ethan:

As we come to the end of our time, Dan, Guillermo, I would just love to ask, if there are any listeners of this podcast out there who are trying to advocate for better performance practices in their organization or they’re just starting to think about page speed as a design factor, do you have any advice for them about how they could get started?

Guillermo:

I think, first and foremost, you need to start monitoring performance on your site. It could be running WebPagetest, it could be using a tool like SpeedCurve, but the important thing is to get some numbers and get a baseline where you can start working from. Also, read up on studies about the effects of performance to business goals; it’s even better if you can do your own studies. One of our engineers, Harold Neal, did an interesting study that contrasted the effects of page load on bounce rate and found that there’s a correlation. So that’s a great case to be put together to the business that performance matters and it’s very important.

But I think my biggest recommendation is to think about the long game, because once you start working on performance and once you set up your performance team, you’re going to have a lot of low-hanging fruit that you can fix, a very concrete set of tasks that you can do to fix the performance of your site. But after that, what are you going to do? What are you going to do to prevent those things from regressing, to prevent these issues from coming back? How do you encourage the rest of your team to believe in performance, to make sure that it’s a priority for them? I think that’s the biggest advice I would give, is to think about your long-term goals of how you’re going to measure performance, how you’re going to keep performance at a perfect level, and how to get the whole team invested in it.

Dan:

Yeah, Guillermo laid that out really well. Just to add to that, I think from the onset, when you’re thinking about a team or whether it’s just one person that you want to set aside to work full-time on performance, you want to make sure that you frame that where it’s not their sole responsibility to be, like I mentioned earlier, a “performance cop.” It should really be the responsibility of the whole team and the whole company at large, hopefully, to be on board with performance. And whether you’re starting a team or it’s just one certain person who’s going to step in to be monitoring performance, they really should be a resource for knowledge around performance in the entire company rather than the person who’s supposed to go in and remove all the cruft and do all these refactors and things like that, because that could burn someone out really quickly.

And also, on the point of monitoring, Guillermo mentioned that the first thing you should do is start monitoring so you can set up that baseline. But also potentially a big business motivating factor is if you can do benchmarking and you can show, okay, our two closest competitors have a site that loads three times faster than us. Yeah, there’s business reasons why that’s important, but also you can kind of create a little artificial competition and say, “We’ve got to be faster than competitor X,” and that can be more powerful than a lot of people may give it credit for. So, monitoring is important for that as well. But ultimately performance should be something that everyone on the team cares about and not just the responsibility of just one person or one team.

Karen:

Well Dan, Guillermo, I couldn’t have said it better myself. I think that’s a fantastic message to leave our listeners with, so thanks so much for coming on the show today.

Guillermo:

Thank you for having us.

Dan:

Thank you for having us.

Ethan:

Thanks to everyone for listening to this episode of a Responsive Web Design podcast. And thanks also to our sponsor, GatherContent. Go to gathercontent.com and take control of your content production process today.

If your company wants to go responsive but you need a little help getting started, Karen and I offer a two-day onsite workshop to help you make your responsive design happen. Visit responsivewebdesign.com/workshop to drop us a line—we’d love to hear from you.

If you want even more from us, you can sign up for our newsletter, subscribe to this podcast, and read full transcripts of every episode at responsivewebdesign.com.

Thanks so much for listening, and we’ll be back next week.