A/B testing of your online donation forms can be a powerful practice for optimizing their performance and maximizing the funds your nonprofit raises online. Analyzing relative conversion rates for your A/B tests within a tool like Google Analytics is just the beginning of the insights you can gain in your A/B tests, though. By integrating your A/B test with the Salesforce Nonprofit Success Pack, you can move beyond simple conversion analysis and compare performance across a host of metrics.

In this step-by-step recipe, we'll create a report in Salesforce that will allow you to quickly see which form in your A/B test is performing best with respect to new vs. existing donors.

Objective

Create a custom Report in the Salesforce Nonprofit Success Pack that compares online donation form A/B test results for new and existing donors

Estimated Time

30 - 45 minutes

Requirements

• Salesforce.org Nonprofit Success Pack
• Use of the Donations (aka Opportunities) object to track donations
• Use of an online donation service that allows for the creation of multiple donation pages and associate each with a Campaign in Salesforce
• Use of Campaign Hierarchies to organize A/B testing Campaigns
• Use of Google Analytics or a similar service for the creation of Experiments to setup the A/B test online
• Rudimentary familiarity with creating custom fields and Reports in Salesforce

Desired Outcomes

Easily view and compare the performance of individual online donation forms in an A/B test across different metrics. This how to focuses on evaluating performance of A/B test donation forms with respect to new and existing donors. 

The End Result

The charts attached to the first comment below are based on one custom Report of the Report Type "Campaigns with Opportunities." Depending on the chart, the light blue bar shows either the number or percentage of existing donors per campaign. The dark blue bar shows either the number or percentage of new donors per campaign. One campaign represents the original (control) donation form and the other campaign represents the slightly modified (variant) donation form.

These charts show the modification in the variant donation form brought in more new donors and less existing donors than the control donation form for our A/B test, suggesting that the slight modification in the variant donation form for our example test could be effective in bringing in new donors.

Step One: Create custom field on the Opportunity object

For the Report we'll create, we need to determine which Opportunity records represent first donations from new donors. Fortunately, the date of a Contact's first gift already exists in the Salesforce Nonprofit Success Pack in the First Gift Date field. We'll use that date in a new formula field on the Opportunity to determine if a given Opportunity record is the first. We'll also add in some logic so the field is only populated if the Opportunity is a donation. The formula below is specific to an NPSP instance using the NPSP Default record type and Soapbox Engage's Donations app for donation records. You can modify those record types to meet your needs.

To create this field in Salesforce Classic:

• Go to Setup
• In the left column, click Customize
• Click Opportunities
• Click Fields
• In the Opportunity Custom Fields & Relationships section, click New
• For each step in the process, use the values in the Opportunity Object Custom Fields chart below to create the fields

To create this field in Salesforce Lightning:

• Go to Setup
• Enter "Object" in the Quick Find box in left column
• Select Object Manager under the Objects and Fields menu
• Scroll down the list and click Opportunities
• Click Fields
• In the Opportunity Custom Fields & Relationships section, click New
• In the Fields & Relationship section, click New
• For each step in the process, use the values in the Opportunity Object Custom Fields chart below to create the fields

Opportunity Object Custom Field

Field Label: Is First Gift Ever

API Name: Is_First_Gift_Ever__c

Data Type: Formula (Text)

Formula: IF ( CloseDate = npsp__Primary_Contact__r.npo02__FirstCloseDate__c, IF ( OR (RecordType.Name = "Soapbox Donation", RecordType.Name = "Soapbox Recurring Donation", RecordType.Name = "NPSP Default"), "true", "false"), "false")

Step Two: Create a Custom Report

The custom Report with the chart we want needs to include all Won Opportunities from new and existing donors in each Campaign in an A/B test. Each Campaign should share a Parent Campaign. To create our Report:

• Go to the Reports tab
• Click New Report
• For Select Report Type, choose Campaigns with Opportunities
• Click Create
• For Show, confirm "All Opportunities" is selected
• For Date Field > Range, select "All Time"
• Click Add next to Filters. In the new Filter form that appears:
  • Choose "Won" from the field drop down
  • Select "Equals" for the operator
  • Enter "True" in the text field
• Click Add next to Filters again:
  • Choose Parent Campaign Name from the field drop down
  • Select Equals for the operator
  • Enter the name of the parent campaign containing each sub-campaign in your A/B test.
  • Click OK 
• Next to the Preview Header, click Summary Format 
• Drag and drop the Campaign Name field to where it says "Drop a field here to create a grouping"
• Drag and drop the Is First Gift Ever custom field to where it says "Drop a field here to create a grouping" in the area right below the Campaign Name grouping.
•  Click Save and enter in the following values in the popup window
  • Report Name: New vs Existing Donors: Control vs Variant
  • Report Unique Name: New_vs_Existing_Donors_Control_vs_Variant
  • Report Description: Number and percentage of new and existing donors per Campaign
  • For report Folder, select the folder of your choice
  • Click Save

Step Three: Change your report to a Matrix Report and add in some trimmings

• Next to the Preview header, click Summary Format and Select Matrix
• Ensure that the "Is First Gift Ever" and the Campaign Name fields are grouped in each axis of the matrix. 
• Drag and Drop Add Formula to where it says "Drop Summarizable fields into the matrix."
• In the Custom Summary Formula Field popup, enter the values noted in the Report Formula Field section below. 
• Click OK
• Click Save

Report Formula Field

Column Name: % Existing vs New Donors

Description: Percentage of existing and new donors per Campaign

Format: Percent, 2 Decimals

Where will...?: At a specific row/column grouping level...

                       A: Campaign Name

                       B: Is First Gift Ever

Formula: RowCount / PARENTGROUPVAL(RowCount, CAMPAIGN_NAME, COLUMN_GRAND_SUMMARY)

Step Four: Modify the Report's Chart

To create the chart noted above in Lightning, edit the default chart as follows:

• Select Type: Vertical Bar Chart
• Y-Axis: % Existing vs New Donors
• X-Axis: Campaign Name
• Group By: Is First Gift Ever
• Formatting > Chart Title: Existing vs New Donors
• Click OK
• Click Run Report

The chart displayed should parallel the chart above to show the percentage of existing and new Donors in each A/B Test Campaign. To view the number as opposed to percentage of new and old donors, simply change the Y-Axis value to Record Count.