Using design thinking process to improve the email newsletter experience for constituents of Boston

Defining project goals amidst uncertainty

Kickoff meeting with the client left us with a list of questions and no clear project goal. The team from the City wanted to improve the user experience for constituents of Boston but didn't know where to start. Some of the requests were:

• What's the readers' experience across the City's newsletters?
• Do readers want information coming from the City at large or from specific departments?
• What are the best practices/approaches for newsletter creation?
• Is the current newsletter system/process efficient?
  ‍

So, to better understand and define the problem we did another round of interviews separately with department management and newsletter creators. This helped us to clarify that we have 3 types of users in this project, each with their own goals, needs, and experiences:

‍

‍

Readers are feeling overwhelmed and confused

As the priority for the team was the experience of newsletter readers, we concentrated on diving deeper into that and conducted interviews with 5 e-newsletter readers. Additionally, we processed the data from interviews with newsletter creators as we saw some patterns emerging.

“I thought this was going to be all artists resources but then actually they are talking about voting and then the census and nothing about artists unless you scroll down to the end of the email.”
                                                                                                                         - interviewed Constituent of Boston

What does research say?

In fact, the Neilson Norman group's research into the usability of marketing emails and newsletter design supports our findings:

• Messages from recognizable senders, purposeful and concise subject lines, and informational pre header text are most effective
• Number of topics covered in the newsletter should be limited not to overwhelm the user
• Users judge an email based on the imagery used; it should be big, high-quality, and relevant
• Content should be relevant and preferably personalized
• Users are skimming the newsletters, so they should be designed to facilitate this way of reading

‍

‍

Constituents are interested in content from the City

To better define the problem we also needed to evaluate current state of things, so we analyzed 20 latest newsletters from the City. We noticed some common patterns: inconsistency in header and iconography use, and layout, overuse of hyperlinks in text, drastic variation in newsletter sizes, text becoming too small to read on mobile devices, a lot of duplicated information, presented in different ways

Extracting analytics on past newsletter performance has allowed us to observe that for the majority of City newsletters, performance was consistently strong. With open rates often exceeding 30%, constituents demonstrate high engagement with the content the city provides. Furthermore, readers frequently click on content within newsletters. Consequently, our design challenge became to enhance the accessibility and implement best practices to engage users even more, while also respecting the success achieved by previous emails.

‍

‍

‍

Heat maps for us have proven to be unreliable tool for analyzing user behavior, with some instances yielding empty or inaccurate results. While certain heat maps have confirmed our hypotheses – for instance, users clicking on images rather than embedded links, and showing a preference for a one-column layout – we have chosen to approach these insights with caution and use best practices as our guiding principles.

‍

‍

What other cities are doing right?

To get better understanding of the industry we subscribed to major US cities' newsletters and monitored what we received for a month. This gave us insights into what other cities are doing and gave some context:

• Boston has the largest number of newsletters (65 publications for a US average of 20)
• Most newsletters limit the content to less than 7 different topics, creating one or 2 pages of content maximum
• 65% of the cities are including less than 8 links

‍

‍

Defining project objectives, deliverables, and scope

Another venue we explored was opportunities for consolidation and reducing the number of newsletters. For this, we conducted a card sort to see how users were grouping newsletters together. This way we discovered that users were confused by newsletter names (ex "Slow Streets" newsletter) and placed them in different categories. Unfortunately, we realized that this would be out of scope for our current project. However, exploring this topic will be recommended for the team in the long term.

Ultimately our objective became to enhance user experience (UX) by creating a template that adheres to best practices, accessibility standards, and brand design guidelines.

‍

Translating insights into design decisions

Keeping in mind our research insights:

‍

‍

Our team conducted design studio, where all three of us hand sketched our ideas on paper to start ideating on a solution.

‍

‍

Uncovering usability issues

To find out how users interact with our design we tested all three versions of the new template with 5 users. To create real-life experience for our users we did usability testing on high fidelity wireframes. Here are our big findings:

• Usability problem: icons in city announcements banner (that was designed to consolidate important City messaging at the top of the newsletter, without taking valuable space away from each department and their content). Why? Testing showed that icons housed in squares were not getting any clicks on them, when blue buttons were diverting user attention in the top banner from the buttons in the email.
• Solution: taking into account that the iconography is part of the city brand, we recommended positioning the icon in close proximity with the city logo and department name, so it’s easier to associate them together; and use it consistently across all departments or to avoid using it

‍

• Usability problem: department icon in the header. Why? When it was positioned on the right users were not sure what these icons meant or what they were for. In addition to this, the icon when positioned on the right is disappearing on mobile devices.
• Solution: taking into account that the iconography is part of the city brand, we recommended positioning the icon in close proximity with the city logo and department name, so it’s easier to associate them together; and use it consistently across all departments or to avoid using it.

‍

• Usability problem: text positioned in the image. Why? Users had hard time to read it, even though it looked appealing to them. Secondly, it’s not ADA compliant, because screen reader could not read text on the image.
• Solution: avoid putting text on the imagery and use a simple stackable layout, which is responsive when on mobile screens.

‍

‍

Before and after look

‍

‍

Reflecting on the project

This project was grounded in research and showed our team that UX problems are not always that obvious. It took us a while to narrow down the problem and stay focused on it. If we had more time, I would like to conduct A/B testing with a new template and see how it performs in comparison with previous ones. Additionally, I would also like to dive deeper into the issue of e-newsletter frequency and possibility of their consolidation, and gain more insight into our users' expectations regarding e-newsletter content. 
At the time of the project handoff, the city is reviewing our findings and is in the process of implementing them. In the meantime I was happy to continue working with my team on the next project for the city: emergency e-newsletters as a part of emergency communication plan.

‍