Episode Transcript

Ethan:

Hi. This is a Responsive Web Design podcast, where we interview the people who make responsive designs happen. I’m your host Ethan Marcotte.

Karen:

And I’m your other host, Karen McGrane.

Ethan:

And this week we’re really excited to speak with Rob Huddleston and Brian Dillon from Capital One. Welcome, guys.

Rob:

Hey, how are you?

Brian:

Thank you.

Ethan:

So, for those of you who were here last week, this is Part Two of this little series where we’re just going to dive into some more of the design and execution details on the Capital One responsive redesign. So, Rob, Brian, thanks so much for coming out. Why don’t you tell everyone a little bit about yourselves?

Rob:

Sure. So, I am Rob Huddleston. I’m one of the creative directors on the Capital One digital design team and I focus on, really, UX standards and governance, and I support all things on the capitalone.com platform.

Brian:

And, I’m Brian Dillon. I manage our UI engineering team and IT. We make sure that everything that the design team delivers to us makes it out to the web in a performance and responsive and accessible way.

Ethan:

Fantastic. So last week was a bit of a massive episode for us. We just really kind of dove into a lot of the strategy and a lot of the narrative behind the capitalone.com redesign. Brian, I hear you’re the guy to talk to, though, when it comes to a lot of the execution details. Why don’t you tell us a little bit about your role in the project, specifically.

He’s being incredibly modest. Brian did single-handedly, in roughly four weeks to a month, write every line of code required to pull off this responsive retrofit for the entire website.

Brian:

Yes, I guess that’s true. I started out by reading a bunch of your guys’ blogs, actually, and decided that Capital One should be responsive. So I started prototyping some stuff a couple of years ago. And got to a point where I was like, “Hey we can actually deliver this on the platform.” I was familiar with the platform and helped build our marketing and acquisition site from the beginning. So I was very intimate with it. So I started putting together a prototype and shopping it around to a lot of our line-of-business customers, and try to get some headway with that. And when we finally got buy-in, I was the guy who sat on the team and wrote essentially every line of Javascript and CSS and everything to make it happen.

Rob:

I think he’s being incredibly modest. We talked about it a little bit in the last interview, but Brian did single-handedly, in roughly four weeks to a month, write every line of code required to pull off this responsive retrofit for the entire website. So it’s a pretty outstanding feat. He deserves some kudos.

Brian:

It was a great opportunity for us, too, because not only were we making a better user experience for mobile devices and tablets and everything, but we also got a chance to redo all the accessibility on the site, and because we had re-platformed a few years earlier, we decided to kind of scratch everything we had from then, the legacy platform, and focus more on standards.

Ethan:

Well, that’s great.

Brian:

Yep.

Not only were we making a better user experience for mobile devices and tablets and everything, but we also got a chance to redo all the accessibility on the site, and focus more on standards.

Ethan:

So, since you guys have thought about this as kind of a responsive retrofit, you sort of have an established design direction for the most part, it sounds like. So how did you approach… what was your first point of entry to make the site responsive?

Brian:

I looked at a few libraries and frameworks that are out there like Skeleton and Bootstrap and then I think Gumby was one of the ones I looked at. And Skeleton really appealed to be because it was already really lightweight and the classes and the grid structure already kind of matched what we had, like leveraging of the 960 grid system. So I took that and kind of stripped it down to its barebones. I started mobile first, made it all em-based and went from there.

Ethan:

Very cool. Was there a specific section of the site you were focused on or was it just a couple representative pages?

Brian:

It was the entire site. So the scope of capitalone.com is roughly about 2,500 pages.

Ethan:

Right.

Brian:

And so, but it’s broken up, not by page templates, more it’s really like a true grid structure. So we were able to contribute things at an individual component level. And so we focused on each component and how those scaled, and then dropping it into a grid that was already pretty solid made it much easier.

We had the ability to really drive this thing at such a large scope and large scale, and with one release launch the entire website, responsively so. If this is something where we want to be ahead of the curve, we want to be the first to do this, let’s not be afraid to just go big.

Rob:

Yeah. So one thing that we did early on, we were asked to by executives since we started to really push this idea. Really, we thought about it from the start, the same way we think that every other large-scale organization that we were keeping our eyes on was also thinking about it. And that was, prototyping or having a guinea pig, if you will. Do we test a responsive micro-site? We had seen other financial institutions doing that. I think Bank of America had a recruiting site that was responsive at the time. We didn’t see a lot of value in that. We thought, okay it’s a good test run. It gets us into the code. It gets us actually creating something responsive. But at the end of the day, there’s not a huge amount of… there’s not a lot of bang there. We thought, let’s test a responsive homepage, but we saw that very quickly as just a bait-and-switch. We also saw other websites and companies doing the same thing, testing the homepage first, and we just thought that set our users up for a really bad experience at the end of the day, having a nice, elegant, responsive, mobile-friendly homepage and then one click away you’re in this nasty pinch and zoom experience.

So, because of the decisions we had made with the re-platforming, and a lot of that we discussed in the last interview with the design system and the scalable components and the modules and what-not, we were really set up for a huge success. We had the ability to really drive this thing at such a large scope and large scale, and with one release launch the entire website, responsively so. That’s really what we focused on when we started being asked the question, “Can you do something smaller?” We said, “Yeah we can, but there’s not a lot of value in that.” If this is something where we want to be ahead of the curve, we want to be the first to do this, let’s not be afraid to just go big. So, it was a bit of a hard sell, but once we got people behind that, we got people really excited about the opportunity that was, we were being faced with this effort. We got a lot of people excited and we had the thumbs-up and the green light to actually go site-wide with the effort.

Brian:

And with the way that the platform’s built and pages are contributed, we were asked if we could just do a section on the site, like one line of business, like auto loans or something. But it would have been much more difficult just to focus on that and separate out that codebase from the rest of it, than to just do the whole thing as a big bang.

Karen:

One of things that really interested me in our last conversation was your discussion of how you re-platformed your CMS a few years back. And what it sounds like to me is you essentially built your own component-based CMS. And, when I talk to people about their responsive retrofits, some times they work and sometimes they don’t.

Brian:

Mm-hm.

Karen:

And what strikes me, in talking to you, is that having this very flexible, modular component-based CMS, where you weren’t operating around page templates, but rather a flexible grid system that different components could fit into, sounds like that was a really important part of being able to execute this responsive retrofit so well. Can you talk a little bit more about that? I think this is something that I’m very interested in and I think it’s something that a lot of organizations should start adopting now.

When we started that re-platforming effort a couple of years back, we went through and looked at all the designs that are in front of us and kind of chopped up everything into components and then we were able to create a common component structure.

Brian:

Yeah, definitely. So we have, I think our internal CMS is Interwoven TeamSite but we’ve kind of chopped it up so much that it’s no longer what it was out the box. And so when we started that re-platforming effort a couple of years back, we went through and looked at all the designs that are in front of us and kind of chopped up everything into components and then we were able to create a common component structure and so, content, templates within that CMS. And so as our content editors are entering stuff all they have to do is put a headline, the body article, what the call to action may be, a link. And then our presentation templates, on the backend, takes all that from a RESTful service and puts it all together, essentially.

Karen:

That’s really cool.

Brian:

Yeah.

Rob:

I think that was key to the success of this and other efforts that we’re in-flight with. And it was just really thoughtful and again, I think I pointed this out in the other conversation, but it wasn’t a happy accident. It wasn’t like a Bob Ross moment that we had here. This was a lot of like, early on in 2010 we were re-platforming, this was a lot of months of thought put into how we were going to re-platform, what decisions we were going to make, so separating code and content, modularizing, and making those modules not only scalable within the grid so the same component type could be positioned within the page at four columns, six columns, ten, twelve, what have you—but even within those components there could be a series of elements, so a heading, a sub-heading, an image, a body copy, a button or a call to action, an icon, depending on the component. Then any of those elements can be turned on or turned off, so that there’s this huge amount of flexibility both within the content module itself but then within the presentation of the content.

Brian:

We really did our due diligence there when we were building those content templates. I mean there’s still probably XML nodes within them that we don’t use because we try to make them as scalable as possible for future implementation. We’ve even moved in a direction now where we’re kind of bringing them together as one content template. We’re calling it an uber template. So no matter what your content is you can use this one content type and be able to assign a presentation template to it to output what you need. And this is really cool too because it also kind of paves the way for some of our adaptive exercises that we’re doing where we’re able to assign three different banners in the content management system for mobile, tablet, desktop, and then on the front end use logic to be able to pull which one of those banners we want based on the device type that’s being rendered.

I think design systems and patterns are probably the best place to start and give you the most flexibility moving forward in terms of scalability and future-forward thinking.

Rob:

And I think too, I recently met someone at a conference that we were speaking at in DC, and he works in the design and UX team at the IRS, of all places. Over conversation he informed me that they have been working for quite some time to try to deliver a responsive site and were having huge amounts of trouble doing it. Throughout the conversation he was interested in the design system and how we had componentized and modularized the site and the steps that we had taken years before that set us up for this success. After the conversation he says to me “I think that’s what we’re going to do. That’s our next step.” Let’s take a step back and really think about content first and componentizing and modularizing the site, making it scalable from a design system perspective, and then responsive being step two or something after that. I was really interested just having that conversation and having someone, obviously his work is a different industry but similar to what we were dealing with years prior. Having someone react to the thought of thinking about the design system first and then all the benefits that can come out of that it was really interesting. I know I haven’t spoken with him since but I know he was planning on taking that back to his office and his team and saying “Let’s take a step back.” Let’s think design system first and then what are the benefits and wins of establishing a strong system. So, Karen, I think you’re right. I think it’s definitely something that, especially large organizations should be thinking about. Smaller as well. I think design systems and patterns are probably the best place to start and give you the most flexibility moving forward in terms of scalability and future-forward thinking.

Ethan:

That’s great. I mean I know for me when I’m building a design system there’s always, I always need to revisit some earlier assumptions. I’d be curious given the timelines you guys are working under, 2500 pages, roughly two months, you’ve done this sort of massive replatforming project back in 2010, was there anything that you had to revisit from the system when you were actually working? Anything in particular you had to kind of rethink when you were actually making it responsive? “No” is an acceptable answer, but…

Rob:

Well, I don’t think no is the answer. I think we knew going into it that we were going to have some design elements, large format tables. We’ve got an over-abundance of navigation on the site everything from super-menus and primary navigation, secondary left-hand navigation, tertiary tab navigation, utility navigation, I mean we’ve got this abundance of navigation. We knew there were things that we were going to have problems with and we tended to focus on how to solve for those specific things.

Brian:

A lot of the global elements we knew we were going to have issues with the header, the footer, the navigation like you said, so we took a look at those, we did the best we could for the first implementation knowing that we were going to go back and build usability with them and test new implementations. In fact I think we’ve redone our primary nav since we launched responsive probably three times, and we’re in the process of doing it again right now because it’s always a test and learn environment and there’s always a better way to do it.

Rob:

Yeah, so we opted, let’s identify the problem areas, let’s do the best we can with what we’ve got in the time we’ve got and then let’s focus on the future. What does the next year look like as far as testing, whether it’s production tests, A/B tests, or if it’s taking new design concepts and new patterns in front of users and doing a variety of testing and landing on something that we can put into production. Again to Brian’s point, just iteratively enhancing and improving.

One thing we did too very early on, and I think this helped us out a lot was, and I’m not sure if we mentioned this the other day, was we brought an expert. So we had R/GA as an agency that Capital One was using at the time quite heavily with a lot of our design exercises, and lucky for us Brad Frost was working at R/GA at the time. So having seen him speak a few times I just took a chance and sent him an email to ask if he had time to just come sit with us. We were of course on a tight deadline and a schedule but he replied back, he made it work, he came and sat with us for a couple days and really went over not only the work we were doing but kind of our plan to get this thing done. One thing that I think we got out of that—not only did we get a little bit of validation—but we also had a better understanding that the same things that we were tackling, that we were having problems with, large data tables and navigation and what have you, these were the same problems that everyone was having at the time and still is to some degree with responsive design. So just the understanding that we weren’t alone in this was pretty liberating. It gave us the opportunity to just move forward and know that, again, it’s not going to be the perfect solution when we go live with this, but we at least had a plan and we had opportunity to improve. And again it gave us an opportunity to maybe solve for something that others hadn’t yet solved for in a way that was accessible, it was responsive, it was everything that it needed to be.

I think large format tables are still one of the problems that we’re dealing with and to Brian’s point about testing navigation and redesigning it three or four different times in different ways, we’re doing the same thing with our large data tables, comparison tables. At least understanding that we weren’t alone in that was pretty liberating for us and it just gave us the road to move, what’s the saying? Gave us the runway? I don’t know. But it at least gave us kind of an understanding that we weren’t going to fail with these three things. We were going to make it better but there was always going to be opportunity to make it even better beyond that, and that’s what we’re focusing on now is really how to make those things better than what they are and we’ll continue to do that. That’s just kind of our way here at least right now.

It wasn’t the right content for the end user but once the business customer started seeing what that looked like on a mobile device they were like “Okay, yeah, we need to move away from this.” And so we really changed their mindset on that and had them start thinking in terms of what the user’s seeing on a mobile device.

Brian:

It was a mindset change too for the business customer. One of the biggest content problems that we had at the time was the use of Flash, and so we had a bunch of interactive calculators and animated banners and whatever it may be on the site and we knew we weren’t going to solve for that from the start. But we had a relative fall-back image in place. It wasn’t the right content for the end user but once the business customer started seeing what that looked like on a mobile device they were like “Okay, yeah, we need to move away from this.” And so we really changed their mindset on that and had them start thinking in terms of what the user’s seeing on a mobile device and on a tablet and whatever it may be.

Ethan:

Well that’s great. I like that test and learn model. I think that’s really pretty powerful. So when you’re looking at something like complex navigation systems or data-heavy tables, how do you… I mean there’s obviously a gut check to see whether or not something’s working, but how do you guys test and learn from something you ship initially?

Brian:

Yeah, so with the tables it’s always a struggle because we’ve given that flexibility to the content editors to build whatever they want, and they have the template to output whatever kind of table they want. So they can really fill it, put as many columns in it and that’s where it starts to become problematic. With the bigger tables on the site, our browse and compare stuff, those are handled by dev, we don’t really give those, the control to the content editors the way they would probably like. So we’re able to thoughtfully go through and eliminate columns where we need to or even start to rethink the table as a whole. Maybe it doesn’t need to be a table. Maybe we can do it like a baseball card style or something like that where the content will scale nicely. But yeah, for some of the bigger tables the content editors are contributing we try to push back on them. Rob and I kind of sit on what we call our Online Review Board and so we see a lot of everything, we’re supposed to see everything that comes through the site before it goes to production, so that we can provide feedback and say this is going to work or it’s not going to work. So we try to get ahead of those as much as possible before they go out the door.

Rob:

And I think, one thing we did do with those large format tables with our phase one approach, unfortunately I think Karen, you’re going to hate to hear this but we did hide quite a bit of content. So for the large one thing we did, and we did this for a couple of reasons. So we had a lot of issues. I think we discussed this a little bit before, just the idea that the M dot might go away. So we had entire teams and departments really fighting against this work. Even though what we were trying to do was leverage some of the work and the existing patterns they had already created in that mobile web solution. Our browse and compare table, so a table where it’s 20 different credit card options and you’re browsing AP-wise and different product features…

Brian:

APR’s, annual fees, all that stuff, yeah.

Rob:

So, one thing we did do was, on mobile, we just decided to match the presentation of the mobile web solution, which was just showing the card art and the primary feature, and the rate, I believe. So we really reduced, we got rid of a majority of that table’s content for mobile, which was not something we wanted to do. By no means are we trying to hide content or remove content on the mobile experiences. But again, for phase one this was the decision we made where we could align closer and maybe make some peace with the M dot team by doing this. We could also, again, just hit our deadline and get this out the door knowing that within a few months, within a period of time, we were gonna incrementally improve on that. And we’ve tested, to Brian’s comment earlier, we’ve tested everything from what we call the baseball cards, using a series of unordered lists really to present on desktop as if they’re a table, with columns and rows. But when you break them down they’re individual content components, so on mobile and on tablet, each product is presented as a tile or a baseball card and you can swipe and scroll. It’s less of a tabular compare, but it’s more of a mobile-friendly solution. So, we’re still playing with that as well.

Brian:

It’s a mobile-friendly and accessible solution. It allows us to maintain that semantic value of each card or product without having to leverage tabular data. But yeah, like he was describing with that card compare and browse table, we hit the low rates and we hit the fees, and the transfer fees and all that, but we also hit the apply app buttons. So we weren’t allowing people to just make a decision based on what we were giving them, we forced them to go through to the product detail page which then presented them with all that information that they needed to make a decision and apply.

Rob:

Yeah, so even in hiding the content we were trying to be as thoughtful as possible to make that content available to them, and as Brian pointed out, they couldn’t apply for the product until they had that information in front of them. Again, I was just being thoughtful in the decisions we were making, not doing anything haphazardly or just with the goal of getting it out the door. We were being as thoughtful as we possibly could.

Karen:

So, as you probably know, my whole schtick is that I think mobile should be a forcing function to get organizations to clean up their crappy content.

Rob:

Yeah. [laugh]

Karen:

Do your future plans include going beyond looking at the content format, like large data table formats or Flash? Are you exploring content quality and content substance? Is it going to change what you publish as a result of asking people to say, “Oh wait, how is this going to work in all of these different formats?”

It took almost a year, but they’re finally coming to us saying, “Okay we need to redesign our site.” But what they mean is it’s less of a visual thing and it’s more of a, we need to rethink the architecture, we need to rethink the content strategy. We need to think mobile first and content first.

Rob:

Yeah, so it’s taken near a year to get our line-of-business partners and our customers really focusing on that, but it’s absolutely happening, which is great for us because we’ve been preaching that same thing for many years before responsive. Yeah, one thing we did—and we knew this was something that not modifying the desktop experience gave us an opportunity to do with responsive was—once on mobile our line of business customers, I mean it was a huge bright spotlight shining down on all of the content issues, all of the design decisions that weren’t considering mobile at the time. So these things, now that they’re in the hands of our business customers and they’re seeing them on a day-to-day basis, they’re now gathering feedback from user inputs. They’re now coming to us, granted it took almost a year, but they’re finally coming to us saying, “Okay we need to redesign our site.” But what they mean is it’s less of a visual thing and it’s more of a, we need to rethink the architecture, we need to rethink the content strategy. We need to think mobile first and content first. So, they’re coming around. It took a while but it was very obvious days after the release we started getting the questions asked, “Oh why does it look this way on the mobile device?” And it all came down to content or design decisions that were, when you start stacking content on top of each other in more of a vertical perspective on the mobile device, and they start saying, “Well we wanted this to be the, this is the most important piece of content but it’s all the way at the bottom of the page, now.” Well, that’s what we’ve been trying to have you focus on for quite a while. So, we start getting the questions, we start actually training and educating, and really having more of a collaborative conversation and that has helped, I think. Not only between our team and the business partners but, it’s just opened up a lot of eyes on the business side of the house to really say, okay mobile first and content first is important and we should be rethinking everything we’ve done up until now. So, to answer your question shortly, yes it’s finally happening but it’s taken a while.

Karen:

That’s great to hear.

Ethan:

That’s fantastic. One word that popped up earlier that’s kind of near and dear to my heart was performance. Was this a priority for you guys in the redesign or is this something that’s kind of on your roadmap in general?

Every department within Capital One defines its goals at the beginning of the year and I think IT’s goals for this year are performance, performance, and performance.

Brian:

It is definitely on the roadmap. Every department within Capital One defines its goals at the beginning of the year and I think IT’s goals for this year are performance, performance, and performance. So it’s all about making all of our sites much faster. When we went live with the responsive redesign, we started tracking page load times and http requests and everything on the page. And we really started scaling down as much of those requests as we could and forcing the design team to deliver images that were compressed and progressive and whatever. We have also employed a CDN since then and are still working on chipping away where we can. I think when we launched responsive, the mobile site was loading at about ten to twelve seconds, which is awful. I don’t think that was reality, that’s like fully loaded on time. But downloaded it was probably within like three or four seconds. Now, fully loaded we have it down to, I think it’s sub two seconds, which is fantastic. Our organization has made a statement that we want to be within the top five of our competitors as far as performance goes, and I think right now we’re number two.

Ethan:

Oh, great. Okay, so when you’re sort of framing what counts as a successful performance-friendly experience, it’s framed in terms of competitive analysis?

Brian:

Yeah, I guess it depends on who you’re talking to. If you’re talking to me, I think it’s customer experience, right?

Ethan:

Oh, sure.

Brian:

We always want to deliver the best customer experience we can for whoever’s using whatever device that they’re on. But yeah, when we try to measure it on paper we are looking at all of our competitors, and even like kind of across the board like some of the other larger Fortune 200 companies. That’s just like a baseline to see where Amazon or where Starbucks and where all these people are at to put us into perspective.

Ethan:

I think that’s super-valuable. That’s great.

Brian:

Yeah, and adaptive has helped us with that a lot, too. Being able to really deliver content that is necessary for the device that it’s being displayed on so we’re not delivering a large banner and then scaling it down. We’re delivering the banner that’s targeted for that resolution.

Ethan:

Mm. Mm-hm.

Karen:

Did you do any usability testing?

Brian:

Oh, we do a ton of usability testing. We have an in-house usability lab that we built where we bring people in, it has the two-way mirror and everything. And then we go to usability testing research facilities all over the place.

Karen:

Any interesting insights about what people understood or didn’t understand in the responsive format, especially on smartphones?

Rob:

We did quite a bit of testing last summer, a few months after the release, where we really focused primarily on navigation and, again, those large data tables. And we tested a cross-section of users. A variety of ages, income brackets, you name it. We tested a pretty wide range of people over, I think, three days in New York City. It was really interesting.

As a financial service provider, we’ve got a pretty wide audience group. And I think one of the biggest problems we knew we were gonna see going into it but, what is the biggest problem that we’re faced with, I think, is just coming up with design patterns and design that meets the needs of all, or the 80% target market. So, of course, we saw within an older audience group, let’s just say 50-plus, they weren’t comfortable with the hamburger icon or the lack of the word “menu” or “navigation.” Whereas obviously the 30 and under group who’s on their mobile device all the time already, they just navigate it without any thought, without any issue. It wasn’t necessarily something that we were surprised by but it’s again, just another challenge that we have to solve for. How do we thoughtfully come up with a new navigation icon pattern or navigation paradigm for mobile or for tablet that will not only fit well within the younger market but will not scare away or be problematic for an older market because we’ve got to cover both ends of the spectrum.

Brian:

Yeah, we’ve followed a lot of the conversations about the hamburger icon and you know, what to do with it. I think the problem that we have with that now is going back to that navigation structure and how much navigation we have. Just because we have to accommodate for every line of business that we have and the different funnels that you can take from any page. So we don’t really have the option of limiting ourselves to four or five buckets across the screen and take up that real estate. We have to kind of hide it at this point until we start to pare down that navigation into something that’s much more usable.

There’s a huge project going on right now that we’re both involved in with—call it site redesign if you will—but it’s really site rearchitecting. And at the focus of that is navigation and how do we pare the navigation down so that we can make it more simplified and also less overwhelming.

Rob:

And that’s something we’re working on right now. There’s a huge project going on right now that we’re both involved in with—call it site redesign if you will—but it’s really site rearchitecting. And at the focus of that is navigation and how do we pare the navigation down so that we can make it more simplified and also less overwhelming and just finding better patterns and better ways of navigating the site through all the different device types. So that’s pretty exciting too.

Brian:

Yeah, our navigation has never really been where we wanted it to even before we went responsive. It’s always been driven by the line-of-businesses like they all want their relevance and to be prominent at the top of the page, right? And so it’s not really taking into account the user and how the user is shopping for products. Because you know we have business cards and personal cards all kind of intermingled together instead of separating them out by audience type. So that’s something we did a card sorting exercise and went through that and now have really pared down to what we think is a better user experience. We still struggle with how to design and deliver that but at least we have something that makes more sense I think as people are shopping.

Ethan:

That’s great. So how much, if you could tell me, are devices a part of the design process currently? Do you guys think about different form factors when you’re approaching specific content problem like navigation or tables? Or is it just mainly in the QA side of things?

One of the things that this responsive effort really did was changed the way that we test everything. So we had to go out and buy a ton of devices and our testing team had to learn how to test and write new test cases on each device type and each operating system.

Rob:

Yeah, that’s a good question. So it’s absolutely on the testing and the QA side of things, so one thing we did, that was one of the things that this responsive effort really did was changed the way that we test everything. So we had to go out and buy a ton of devices and our testing team had to learn how to test and write new test cases on each device type and each operating system and what have you. so there’s testing of course, and here’s maybe one of the—I don’t want to say the drawbacks or the downsides—we saved so much on the cost of having two sites and maintaining two sites but then we had to increase so much the testing cycle. That would be the one, I wouldn’t call it a negative necessarily, but that was maybe the one thing that we didn’t consider going into this at the early stages that we were going to be faced with was how much more we have to test and how much longer the testing cycle now is. But the testers have adapted and they’ve gotten really great at it.

Now the design team, that was the other thing too where we’ve got a mobile team, we’ve got a tablet team, we’ve got the web team. These teams are now becoming a little bit more integrated now that we are working responsively and we’re pushing responsive out to other platforms and other sites across the organization. So our teams are actually leveraging each other’s work a lot more. I’m working hand-in-hand with the mobile team to adopt the patterns that they’ve already created and the elements that they’ve already tested, and same thing with the tablets. So we’re pulling that into the design. But because we had to create a device library for the testers we also had to do the same for our design team. So we’ve been pretty fortunate in that respect that our leadership within the design community is fine with us spending money and buying new toys and new devices. And we’ve got a locker full of them that we all have access to, so that’s been pretty nice.

Brian:

Yeah, I think between Rob and I we have probably upwards of 20 tablets and mobile devices to play with.

Ethan:

That’s great.

Brian:

Yeah.

Ethan:

That’s fantastic. So, I guess the follow-up question to that is, how do you guys talk about support internally? If you’re looking at the layout in Android 2.3 or something versus the latest iOS, how do they talk about regressions or the quality of the experience?

Brian:

As far as testing goes, we have our top, I think it’s like ten devices that we target based on traffic patterns. So, those are always in our library and we update the operating systems as needed, or go out and purchase a new device when needed. And with every new release we do, we full regression testing on the side. So we have automated regression and then people actually going through and clicking on the devices and going through experiences. The great thing is with the way that our site is designed and developed, being so modular, if a story (and we’re all agile, too) if an agile story is just focusing on updating our FAQ template or something like that, all they have to do is test that template on one or two pages and that meets the needs of our full regression testing, because we don’t have to go through all 2,500 pages. We know that that module is going to work across the board. So, it’s kind of a limited scope and it makes it a lot easier to quickly go through all those devices and test what they need to.

Ethan:

Sure. That’s great.

That’s just another bonus of having that flexible design system and that modular approach that we’ve taken. While we are testing on more devices, we’re testing fewer pages or specific components only because they’re cascading site-wide.

Rob:

Yeah. So I think that’s just another bonus of having that flexible design system and that modular approach that we’ve taken. While we are testing on more devices, we’re testing fewer pages or specific components only because they’re cascading site-wide, and if we test one or two—to Brian’s points—then we can trust that they are performing equally on all instances where they’re used.

Brian:

And when we scope out new stories and development features we really focus on getting what the test strategy is gonna be. And so I’ll work with the design team or I’ll work with the testing team, kind of play the liaison between and say, “Alright this is what’s changing, this is what we need to test.” And so, sometimes they don’t need to test something on a mobile device because of what the change is and then other times they do. It just really depends on what that feature is.

Karen:

I feel like I could talk to you guys about this all day.

Ethan:

I know, this is fantastic. I’m writing stuff down furiously, like last week, so. Great. Well Brian, Rob, thank you so much for taking some time today to talk about this.

Brian:

Not a problem.

Ethan:

This is really great to spend two episodes hearing about the redesign.

Rob:

Thank you guys. It’s been really fun. It’s been nice to be invited to be part of these conversations and it’s been really fun to be part of them, so thank you guys very much.

Karen:

Yeah. Congratulations on everything you’ve accomplished to date and I will look forward to hearing more about what you guys are doing.