Select Page

TD Bank IT 3.0 Cloud Marketplace

Featured Project - Case Study

The IT 3.0 Developer Marketplace is an enterprise self-serve cloud portal where software teams can access and use technical components automatically.

How We Did It


Like the Amazon Web Services (AWS) portal, it is an online catalog to discover, manage and “own” approved technical components for enterprise software development that integrate with a continuous delivery production pipeline.

Goal – ​Create, evaluate the design, information architectures, workflow and menu structure of a new application to create navigation, increase ease of learnability and strengthen users’ confidence in a new enterprise platform.

My role – Lead user research activities and design strategies. Setting goals and objectives, stakeholder interviews, planning and executing online card sorting, quantitative data analysis and synthesis, usability testing, recommendations and insight communication, wireframing, iterative prototyping.

Stakeholders and collaborators – Developers, tech managers, business analysts, product owner, project managers, service delivery managers.

Techniques Used
A/B Testing
Data Analytics
Design Patterns
HTA – Hierarchical Task Analysis
Hackathon
Information Architecture
Personas
Screen recordings
Sitemaps

Socialization
Stakeholder Analysis
Style Guide
Surveys
Usability Testing
User Flows
User Interviews
UX Interaction Documentation
Wireframes

Tools Used
Anguar.js
Axure
Conferencing Phone
Excel
Fabric UI
iPad Tablet
Lync
Meeting Rooms

 
Morea Recorder
OBS – Open Broadcast Studio
Outlook Email
PC Laptop
Photoshop
PowerPoint
Portable Projector
Sketch

The Problem


The problem at first glance was easy to define: create an online application where Developers can get cloud based resources. The challenge was how to build this application from scratch.

It was decided early on by the CIO of enterprise architecture that the user experience will be built first to guide the technical infrastructure, instead of the other way around.

In the initial meeting with CIO and his key stakeholder executive team, I asked “If you can today, in one sentence, what problem are we trying to solve?” After a few moments of silence, I proposed that we re-convene in a few days where I lead a Product Strategy workshop to help plan and align the UX and Business Goals, with the key stakeholder team.

The Process


Step 1: UX Strategic & Business Case Alignment Workshop

What’s the size of the “breadbox”?

Rather than rushing to create wireframes and prototypes, I’ve found success by creating and confirming a set of guidelines. Over the past decade plus, I’ve compiled a simple set of “Key Starting Questions”, based on insights from Apple’s Steve Jobs, Harvard’s Michael Porter, Management Guru Peter Drucker, and many more.

Once these questions were answered and approved, the “Key Starting Questions” help guide the general direction of the UX work, as well as defining “where we don’t want to go”.

Sample Key Starting Questions


Step 2: User Research Product Survey
Who are the target users?

Sample UX User Survey

We started by creating a User Survey to identify target users and what best practices they want to use. We sent out surveys to staff in 5 groups which included Architects, IT Managers, Product Owners, Project Managers and Software Developers.

 

  • One way we were able to get a 90% rate was that the email survey was sent by the CIO.
  • Over 1000 points of quantitative and qualitative feedback were analyzed summarized, to help group areas of focus for the UX design process.
  • These pieces of feedback were also used to guide UX Personas, User Stories and Visual Wireframing.

Sample Executive Summary

Sample UX Survey Results


Step 3: Personas

Target users identified

It was clear from the User Research Survey that there were 3 specific end users, not the original 5 that were originally assumed. These users had specific needs, some distinct and some that overlapped.

 To aid in focussing features and functions for these users, UX Personas were created. UX Personas, a user centred design tool, are a representation of end users that show how they use a site or an app in similar ways.

Sample Personas

Personas, based on data collected from users, illustrate attributes such as attitudes, behavioral patterns, goals, needs, skills and work responsibilities.

The 3 UX Personas identified were:

  • Product Architect
  • Software Developer
  • IT Manager

Personas were useful to help guide the project team to understand how to plan phases of development, as well as to report progress and get approval from project funders to continue the project as they have confidence that the work is headed in the right direction.


Step 4: User Stories

Product Features Documentation

 

The next step was to create User Stories. These are are short, simple descriptions of a feature, from the viewpoint of the user’s requirements.

 They typically follow a simple template:

As a < type of user >, I want < some goal > so that < some reason >.

Sample UX User Stories

The User Stories helped the Agile team prioritize feature development planning.

Originally captured and categorized in Microsoft Excel, they were eventually transferred into JIRA and a KANBAN board for the product backlog.


Step 5: Information Architecture

Understanding the Technical Contraints

 

One problem we realized is that we didn’t know which technical pieces existed to support the UX of the product. Like many large enterprises, there are many pieces of infrastructure in many silos across the organization and no central place to find easily find them.

To be able to break down User Stories to match with infrastructure integration and development, it was essential to have an Information Architecture diagram mapped to content sources.

Sample Information Architecture with Content Mapping

Together with a Lead Developer and IT Architect, we were able to find and document pieces that were needed to be able to manage the product backlog as well as have validate which infrastructure exists and what needs to be developed.

This was a big time saving opportunity in our planning sessions and helped manage the Agile Sprints. As a living document, we added to the diagram documentation during Sprints so the team members had a main visual reference, reducing the need and time to find and contact members outside our team.


Step 6: Sitemap and User Flows

Defining the Structure and Visualizing How Users Interact

 

Sitemaps and User Flows are a standard UX Tool. 

Sitemaps are a visual systematic hierarchy of screens that are presented to the user. Starting with the home screen, it shows a navigation representation of how screens are related to each other.

User Flows represent how a user navigates, from start to finish to complete a task. These were gleaned from User Stories as we categorized different ways and requirements that users need to do their jobs.

 

Sample Sitemap and Single User Flow

 

 

For example we found groups of users that navigate in different ways:

  • An IT Architect wants to start from the home screen, go to a category to see the available options and sub-categories, then go to a product page to view details to see if that is what they are looking for. Once satisfied, then they want go through the checkout and confirmation process.
  • A Developer wants to use a search function to get to relevant search results as they know the specific item they are looking and then go through the checkout and confirmation process.

In the above example, both are valid, as they do work in different ways.

  • As a real world example, sometimes people go to the CNN.com homepage to get a view of the most current news; alternatively, sometimes people go to Google.com to find something information they want.

Another benefit of Sitemaps and User Flows are to validate, consolidate and remove duplicate ways that users navigate and complete tasks.

 


Step 7: Wireframes

Defining Elements on the Screen

 

One of the most visible UX tools are Wireframes. While traditionally UX work started with Wireframes, seasoned UX professionals have found benefits and overall project efficiency by doing up front analysis work before getting to Wireframes, as online applications have become more complex.

 Wireframes are like a blueprint and prototype of the elements represented on a screen, arranged in an order and layout of what the final product will eventually look like. The purpose of Wireframes are to get early feedback from the product owner, developers, graphic designers and end users. It is especially important to both front-end and back-end Developers so they can plan out their development process.

Sample Annotated High Level Wireframe

For the project, multiple versions of Wireframes were constantly created using UX Wireframing tools. During development of Wireframes, it’s useful to ask a neighbour by showing the screen, via quick online screensharing as well as walking over to a colleague with screenshots on a tablet, to validate and ask “What do you think this screen is for and what can you do from it?”. This guerilla technique was useful to help validate which elements “make sense” and “where don’t we want to go” to iterate very quickly.

Wireframes were published to Confluence with annotations, so the team can give feedback and ask questions to balance UX and technical issues.


Step 8: Interactive Prototype and Usability Testing

Confirming Assumptions with Real User Feedback

The next step was to create an Interactive Prototype that we could use for rounds of Usability Testing. We created an Interactive Prototype, first based on output from the Wireframing tools and then a functional prototype was created by the development team in Angular.js, so that we could make more rapid updates to ongoing code.

A Usability Testing plan and system was created so that we could get direct input and feedback from users as we iterated through different versions during Sprints. This was paramount, so we could get empirical information to guide, optimize and update Wireframes, User Flows and Development coding.

Using experienced Usability Testing facilitation techniques, we created a testing script, as well asking users to talk “out loud” of what they are doing and thinking, explain to them that it is useful to do this as “we can’t see inside your head, which they were happy to comply. These are some of the questions and tasks we asked:

  • What do you think this is?
  • Task: Please show how to buy a cloud server.
  • Task: Please show how to see which services you own. 

For each of the 3 user groups, we performed 10 one on one half hour sessions, for a total of 30 Usability Testing sessions (15 hours in total). The one on one sessions were more useful than group sessions or workshops, as we were able to remove group bias influences and “group think”.

We used Morae Screen Recorder (there are others such as OBS, UserZoom, etc.) as it was approved Enterprise software, so we could record video of where the users are clicking as well as seeing where their “pain points” were, ability to track their navigation movements and have an audio recording so we could use their narrative of what they liked and didn’t like. Using a tool that also recorded the user’s face via webcam was incredibly important, as we could see and analyze facial and body language to help validate their comfort with the system we were developing.

This results were then analyzed to help evaluate and update iterations during development. We updated Wireframes, User Stories, Prototypes and development features to guide our end product and report back to stakeholders, based on empirical data.

Doing these testing sessions had additional ongoing benefits. Testers had a positive experience, often stating that this is the first time they have been asked for their input on the tools they use to do their jobs. They helped recruit more testers, as well as giving positive feedback socialization to their peers, which helped adoption as we rolled out the final solution in later stages. 


Step 9: UX Documentation and Development

Supporting the Development Process

 

The UX process is not done when the wireframes and prototype being approved and handed over to the developers. To aid developers, we created UX documentation to create a design pattern library in a Confluence Documentation portal, based on Brand guidelines. It explains how each component is to be laid out and how a user interacts with it.

Example
Info Tile: This is a call to action hyperlink component.

 

  • It is a combination of a button and an icon, which together provides rich information so a user can quickly evaluate if it fits their needs.
  • It has a static state, mouse over state and mouse down state.
  • As a user moves their mouse over the tile, it goes from a static state to a mouse over state; when clicking on the tile, it goes into a mouse down state.

CSS Static State

  • Pixel size:
  • Hex Colur:
  • Title hex color:
  • Title text position:
  • Title text size:
  • Description text hex colour:
  • Description text position:
  • Description textsize:
  • Icon size:
  • Icon position:
  • Shadow position:

CSS Mouse Over State

  • Pixel size:
  • Hex Colur:
  • Title hex color:
  • Title text position:
  • Title text size:
  • Description text hex colour:
  • Description text position:
  • Description textsize:
  • Icon size:
  • Icon position:
  • Shadow position:

CSS Mouse Down State

  • Pixel size:
  • Hex Colur:
  • Title hex color:
  • Title text position:
  • Title text size:
  • Description text hex colour:
  • Description text position:
  • Description textsize:
  • Icon size:
  • Icon position:
  • Shadow position:

As developers would create screens, they would come across technical constraints. We would make iterative updates to wireframes, prototype screens and UX documentation. This was essential to ensure that the end product would be on target as some members of the team worked remotely.

« Gallery Carousel – Prototype Updates »


Step 10: QA and Deployment

Socialization and Hackathon for Change Management

Any new product launch needs to be socialized to understand how it will impact users. To manage this, we held training session and presented the new product at “Town Halls” where we used the feedback to make changes.

As this would impact users’ work, we created a 30 day Hackathon where we asked developers to create applications, based on components they could get from the Marketplace.

Using gamification techniques to increase product engagement, we create a Hackathon portal. Users could participate in 3 ways:

  1. Login and register in the Marketplace to get free custom T-shirt.
  2. Answer daily technical questions to earn points to be on the leaderboard
  3. Create and submit an application from Marketplace components, to win one of 5 Gold iPhones.

We had a dedicated team member to answer any questions or issues that users had. We logged all this information and feedback in JIRA so we could make updates to the site.

This was a useful QA technique as we got real world feedback from users before deployment.

The Hackathon was a useful UX technique as it introduced the product in a fun, engaging way.

Ultimately, we registered and trained over one hundred users, which cut training costs by hundreds of thousands of dollars and ensured a smooth adoption after the product was deployed.