Redesigning Search on GlobalGiving To Help You Raise More Money

GlobalGiving is constantly working to improve our site and drive more donations to projects and organizations around the world. One of the ways that we do this is by continuously testing and optimizing our website to increase our conversion rate (a term used to describe how many visitors to our website become donors).

As we mentioned a few months ago, we’ve been testing a major redesign of our search pages and today I’m happy to share the results.

Summary

Based on the Money for Good initiative, GlobalGiving designed and tested a new version of our search/find feature. We designed two radically different versions (one providing a comparison view and one providing a visual view) and conducted a multi-celled A/B test to determine which had the biggest impact on conversion rates.

We found that both of the redesigns did better than our existing search design (the control cell), and within each design variation found promising directions for additional refinement.

Our Initial Hypothesis

By providing better visual designs of our search/find feature, we hypothesized that we could increase the conversion rate of donors. Furthermore, by offering better information within these designs we believed we could further increase the conversion rate. “Money for Good II” suggests that a detailed comparison of projects will produce the largest impact on conversion rates.

Design

GlobalGiving designed two new visual designs for our search – one based on a “Consumer Reports” listing to allow for comparison between projects and another with a heavy visual design inspirited by Pinterest.com to leverage donors’ natural propensity to respond to images.

Within each major design direction we identified several sub-variations. In these variations we altered the information that was displayed to visitors to determine what effect this information had on the conversion rate in comparison to the control cell for that group.

Test Cells

Treatment Treatment Description
A 0 Control (current state)
B “Consumer Reports” 0 Control
1 Organization Level (Superstar, Partner, Leader)
2 Project Grade (A, B, C)
3 Project Statistics (Grade, Donations, Reports, Funding Level (raised vs. remaining))
4 Long-Term Impact Qualitative Statement
C “Visual” 0 Control
1 Organization Level (Superstar, Partner, Leader)
2 Project Grade (A, B, C)
3 Funding Level (Amount Raised and Amount Remaining)

 

The organization level is a ranking system that GlobalGiving developed several years ago to incentivize organization and project leader behavior on our website. It rewards projects that report on their progress, raise money regularly, and have a large number of donors. Superstar is the highest ranking and Leader the second highest. Partner is the third highest and the minimum level an organization must obtain to post a project on the website. This is the first time that we have used this ranking direct to attempt to influence donor behavior.

The project grade was designed as a method to provide a clear, easily understood ranking system for donors. The Money for Good research suggests that a simple ranking system could increase donations. After much internal debate, we decide to map the organization status to a letter grade. Superstar organizations got an A, Leader organizations got a B, and Partner organizations got a C. We feel that this is an imperfect system that doesn’t fully reflect the nuances of real world performance and may suggest that a C project is less worthy of donations than an A project, which may not be true. However, for the purposes of testing the effect that a simple grading system would have on donors, we decided to use project grades for this round of testing.

The project statistics were a combination of the grade, the number of donations, number of project reports, and the funding level.

The funding level is a green progress bar indicating the fundraising progress of the project.

 

A – Control

B0 – Comparison (Control)

B1 – Comparison (Org Level Rewards)

B2 – Comparison (Project Grade)

B3 – Comparison (Project Stats)

B4 – Comparison (Long Term Impact)

C0 – Visual (Control)

C1 – Visual (Org Reward Level)

C2 – Visual (Project Grade)

C3 – Visual (Funding Level)

Methodology

We ran this test from November 11, 2012 through January 21, 2013. During this time, each visitor that went to a search/find page on GlobalGiving was randomly assigned to one of the ten test cells. A cookie was placed on the user’s browser indicating which cell they were assigned. If the visitor came back to the site during this test they would see the same cell. Additionally, this cookie was used to track the donations that were made.

Results

Here are the results from the test:

  
Event Label

Visits

Revenue

Transactions

Ecommerce
Conversion
Rate

Delta

A – Control

4,617

$39,878

893

19.34%

0.00%

B0 – Comparison – Control

5,449

$66,076

1,132

20.77%

1.43%

B1/C – Comparison – Org Reward Level

5,573

$55,237

1,149

20.62%

1.28%

B2/D – Comparison – Project Grade

5,458

$55,132

1,255

22.99%

3.65%

B3/E – Comparison – Project Stats

5,516

$67,157

1,123

20.36%

1.02%

B4/F – Comparison – Long Term Impact

5,530

$50,865

1,139

20.60%

1.26%

C0/G – Visual – Control

5,403

$61,027

1,135

21.01%

1.67%

C1/H – Visual – Org Reward Level

5,443

$47,493

1,111

20.41%

1.07%

C2/I – Visual – Project Grade

5,603

$54,976

1,153

20.58%

1.24%

C2/J – Visual – Funding Level

5,638

$64,988

1,167

20.70%

1.36%

 

As you can see, all of the test cells improved the conversion rate over the control cell.

Contrary to our hypothesis, the detailed project comparison test cell (B3) proved to be the least effective of the comparison variations. The most effective variation of the comparison cells was the one using a project grade (B2).

Of the visual designs, the most effective was the control version that simply has the project picture and the name of the project without additional information. The funding level is close behind in terms of conversion rate but was able to garner slightly higher donations on average.

In addition to this raw visit and transaction data, we also got qualitative feedback from users via customer service. Generally users seemed to respond well to the new designs with one major exception – all of our project partners disliked the “project grades.”

Based on these results and the feedback from project partners, we’ve decided to remove project grades from the next round of testing. Although the grades provided a modest boost to conversion rates, we believe that with additional user interface refinement that we can boost the conversion rate by several percentage points without using a grade system.

Conclusions and Next Steps

Based on these results, GlobalGiving has designed a new round of tests that we are currently testing on our site.

Since both of the new design directions out performed the control (old) design, we stopped using the old design and instead focused on refining our two new designs. We will attempt to see if adding donation options to the comparison view has a positive impact. Additionally, we’ll see if the project statistics view performs well without the project grades. We will compare this against the two winning cells in the visual designs – the control and the funding level.

In summary, the test cells for the next round are:

Treatment Treatment Description

A “Consumer Reports”

0

Control

1

Donation Options with Project Summary

2

Donation Options without Project Summary

3

Project Statistics (Donations, Reports, Funding Level (raised vs. remaining))

B
“Visual”

0

Control

1

Funding Level (Amount Raised and Amount Remaining)

2

Funding Level (Amount Raised and Amount Remaining) + Key Project Statistics

 

Initial results are showing that these test cells are performing very well and may hold the key to an additional boost in conversion rates for all of the projects on our site.

Future Plans

Once the test has concluded and we’ve analyzed the results we’ll share them with you here.

Regardless of the results, we anticipate keeping both the visual and comparison designs on the site. After the test has concluded we will add the ability for donors to toggle between the views. The best performing cell will become the default option for new visitors and the other will be available for users that click the button to toggle to that view. This provides advanced tools for power users and the best conversion for the average visitor.

 

No Comments

Comments Closed