User research: online reading behaviors for a literary magazine

How do people read online?

Project metadata
Project title User research for terrain.org: A Journal of the Built + Natural Environment.
Project tagline Improving engaged reading online through design
Project summary Usability interviews and a heuristic review
Company/client name Publisher, Terrain.org
Project dates/time frame January-April 2016
My major responsibilities In -person user interviews in context, literature review on readability
Platforms Web
Tools used
Key performance metrics Increased readership, time on page, more donations
Team UX team of one
Link to final project Terrain.org

As a writer, you fight a war against indifference. You have to force people to care enough to click through to your story. You have to convince them to take a chance on you…
—Quincy Larson, Free Code Camp

Project summary

Terrain.org is an online literary magazine that publishes high quality essays, poetry, and creative nonfiction about the natural and urban environments. This project aimed at increasing readership,  encouraging more engaged reading of longer essays, and maybe there by increasing donations. I conducted usability interviews, and an expert review with four other designers. My role was user researcher. I interviewed people in person and interpreted the results. In collaboration with the publisher and site owner.

 

Terrain.org, home screens
People choose a story to read based on the featured image, title, and a short excerpt.

Executive Summary

A casual glance shows Terrain.org looks beautiful and is doing well. It has lots of visits ( 200,000–250,000 in 2015) and a large list of subscribers. But, there are some pain points. The number of subscribers had leveled off. Analytics showed the bounce rate is high and average time spent on site is low–a major concern for a literary magazine featuring long essays.

Pain Points

Site Visits

In 2015 were down 4.75% from the previous year

Returns

Only 20% of the total viewers were returning visitors.

Bounce Rate

85% of visitors exit from the home page.

Pages Viewed

The average number of pages viewed is 1.5.

The number of new subscribers has leveled off and few people use the “donations” link (less than 1% of site traffic).

Research overview

Since Terrain is an all online journal and there are a lot of challenges to reading longer articles on screens, we started comparing how people read online as compared to print magazines. After a review of the literature on readability and surveying similar text-heavy sites, we conducted an expert review of the site usability. Next we conducted live interviews and walk-throughs of the site with both current Terrain readers and potential readers.

Research questions

We wanted to understand the goals of new visitors and things like:

  • How does site usability and typography influence engagement with longer literary texts?
  • How do most Terrain readers arrive at the site?
  • What are their motivations for coming there?
  • How do they search for content they want to read.

In addition, we asked people how they thought about donations, to this and other sites. Is the Donate button too small to easily find?

Data analysis

I created a matrix of results from the expert usability review (also called a heuristic evaluation) showing the scores, and comments for each check point, as well as an overall score for the site (54%).

Expert review

A radar chart of the expert review, showing categories and detail questions asked.
A radar chart of the expert review, showing categories and detail questions asked.
Ideally you want your site to score around 75% on an expert review, or heuristic evaluation such as this (Balatbat, 2014). The lower scores are because we were very picky about readability. Note the low score on find-ability–we flagged find-ability as an issue.

User interviews

I recorded user interviews using audio and screen capture videos. From our notes and transcripts, I organized the rich qualitative data obtained from the interviews by research question, using affinity mapping with sticky notes. I also noted new themes and ideas coming out of the research. I then organized this data into a Findings and Recommendations Matrix, featuring quotes, summaries and key insights from each participant.

Affinity mapping
Affinity mapping is a business tool used to organize free wheeling qualitative data like interviews. It allows large numbers of ideas to be sorted based on their natural relationships, for review and analysis.

Key findings

Most users commented positively on the site design, admired the images, and eventually read something interesting. Yet, the site’s overall usability score is below average. Users in the interviews were more forgiving than the experts, and combining both methods we found key design edits that would increase readability without changing the site’s WordPress theme.

Most people interviewed preferred print on paper for personal, literary, and in-depth reading over digital screens. Computers were seen as better for research, email, and finding facts. In keeping with this, participants all scanned the pages first, looking for cues to content before committing to an article. Everyone got stuck on the featured images, trying to use them to decide what to read, which was not reliable. The photos were either a beautiful distraction, or visual overload.

Everyone interviewed did finally enjoy some longer reading that engaged their interest. This supports our idea that providing the right visual cues, and clearing the critical path to interesting content would help people reach this desired state of engagement with text.

After watching people choose a story from the home page, I believe most of the page views come from links in the email newsletter to subscribers. People usually read only the page they clicked on, and don’t browse the rest of the site. That could be due to browsing usability. My theory is that a user is confronted with too many choices, however enticing, and has a hard time choosing just one to read. This is knowh as Hick’s Law: the more stimuli people have to choose from, the longer it takes the user to make a decision. In effect too much cognitive load and likely abandonment of the page.

 a drop-down secondary menu appears, (blue outline is mine) showing a curated view of articles in that category.
When a user clicks on a top menu item, like “Nonfiction,” a drop-down secondary menu appears, (blue outline is mine) showing a curated view of articles in that category.

Few people found the Donate and Subscribe links. People wanted to share articles, but there is no share link to make that easier.

Key recommendations. Usability improvements will help keep users on the site, but that alone will not increase readership. Since most people arrive at the site from external links to a particular article, Terrain should maximize its presence in the external places where likely readers readers would find the site, like links and banners on other relevant sites, content aggregators, social media, email newsletters, etc.

Given a longer-term trend away from home pages and towards apps and social sites, Terrain should look outward to publishing on other social sites like medium.com, Apple News app, and Facebook’s new Instant Articles feature which allows people to read without leaving that site. A longer-term goal might be to create a phone app, as more people are reading that way as well.

 


A Readability Reading List

Balatbat, C. (2014, January 27). A Beginner’s Guide to Heuristic Evaluation (Part 2). Retrieved April 24, 2016, from http://blog.foolprooflabs.com/2014/01/beginners-guide-heuristic-evaluation-part-2/

Jabr, F. (2013, April 11). The Reading Brain in the Digital Age: The Science of Paper versus Screens. Retrieved April 25, 2016, from http://www.scientificamerican.com/article/reading-paper-screens/

Goodman, Elizabeth; Kuniavsky, Mike; Moed, Andrea (2012–09–01). Chapter 11. Observing the User Experience: A Practitioner’s Guide to User Research (Kindle Locations 4818–4819). Elsevier Science.

Herrman, J. (2016, April 17). Media Websites Battle Faltering Ad Revenue and Traffic. Retrieved May 02, 2016, from http://www.nytimes.com/2016/04/18/business/media-websites-battle-falteringad-revenue-and-traffic.html smid=tw-nytimes

Holst, C. (2010, November 01). Readability: The Optimal Line Length. Retrieved May 21, 2016, from http://baymard.com/blog/line-length-readability

Krug, S. (2006). Don’t make me think!: A common sense approach to Web usability. Chapter 2. Berkeley, Calif: New Riders Pub.

Marjoo, F. (2013, June 6). You Won’t Finish This Article: Why people online don’t read to the end. Retrieved March 31, 2016, from http://www.slate.com/articles/technology/technology/2013/06/how_people_read_online_why_you_won_t_finish_this_article.single.html

Nielsen, J. (2003, November 10). The Ten Most Violated Homepage Design Guidelines. Retrieved May 09, 2016, from https://www.nngroup.com/articles/most-violated-homepage-guidelines/

Nielsen, J. (1995, January 1). 10 Heuristics for User Interface Design: Article by Jakob Nielsen. Retrieved April 01, 2016, from https://www.nngroup.com/articles/ten-usability-heuristics/

Nielsen, J. (1995, January 1). How to Conduct a Heuristic Evaluation. Retrieved April 2, 2016, from Nielsen Norman Group, https://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/

Nielsen, J. (2008, May 6). How Little Do Users Read? Retrieved March 30, 2016, from https://www.nngroup.com/articles/how-little-do-users-read/

Nielsen, J. (2007, July 9). Write Articles, Not Blog Postings. Retrieved March 30, 2016, from https://www.nngroup.com/articles/write-articles-not-blogs/

Nell, V. (1988). The Psychology of Reading for Pleasure: Needs and Gratifications. Reading Research Quarterly, 23(1), winter, 6–50.

O’Hara, K. (1997). A Comparison of Reading Paper and On-Line Documents. Retrieved July 26, 2016, from http://www.sigchi.org/chi97/proceedings/paper/koh.htm

Sharon, Tomer (2012–03–21). It’s Our Research: Getting Stakeholder Buy-in for User Experience Research Projects (p. 97). Elsevier Science. Chapter 3.

Travis, D. (2016, April 12). 247 web usability guidelines. Retrieved May 01, 2016, from http://www.userfocus.co.uk/resources/guidelines.html

uk.gov, Government Service Manual. (2014, June 4) Expert reviews: Getting input into products and services. Retrieved April 1, 2016, from https://www.gov.uk/service-manual/user-centred-design/user-research/expert-review.html

 

View the entire report

Leave A Comment

Your email address will not be published. Required fields are marked *