At eduWeb I chimed in about A/B testing on emails during the Q&A Session of Kyle James’ presentation (that’s me around 27:40). I just wanted to share an example of a recent test that I did.
The email was to announce that our online app was available. I knew from last year’s send that the subject line was fairly effective (“Butler’s Online App Now Available!”), so I wanted to take a look at the content and see where I could actually push more students to click through and take action.
For the 2007 send, it went to 14,650 students, and the results are below.
For 2008, we have 17,566 students to email. Rather than do it the same as last year, I first ran an A/B Test. I left the first email the same as last year, and for the 2nd one I used a button graphic to see if it would help clickthrough rates increase.
Test A: Same email as last year.
Test B: Added a visual clickthrough.
Results: Each test was sent to 3,500 random students on 8/7/08. The winning test after 36 hours would then be sent to the remaining 10,566 students.
Test A: 3,282 successful. 339 opens (10.3%), 72 click throughs (21.2%) as of 8/11
Test B: 3,292 successful. 719 opens (21.8%), 273 click throughs (38.0%) as of 8/11
2008 Send: 9,454 successful. 1,378 opens (14.6%), 449 click throughs (32.6%) as of 8/11
And for comparison,
2007 (1): 14,650 successful. 5,137 opens (35%), 853 click throughs (16%) after 1 month
2007 (2): 9,513 successful. 1,232 opens (13%), 270 click throughs (22%) after 1 month
So in first 4 days, we have had 47% of the opens as last year (2,436), and 93% of the click throughs (794). These numbers will continue to rise as the days and weeks go on. Based on early #’s, I can say that Test B has been a success, 34.4% click through rate to date compared to 17.6% click through rate over the course of last year’s entire campaign. Those are results.
Even if we don’t do a 2nd send this year, I think we’ll get close to the amount of opens we had last year cumulatively. I might do a 2nd send with a different title to see how that affects open rates, using the content from Test B to continue to push click throughs higher.
I started thinking about the button after reading Designing The Obvious on a flight last week. It was a really good book and made me think more about how I can incorporate more design-friendly aspects into emails.
I’d encourage you to consider an A/B test in the future and see how you can make the most of your email campaigns. This isn’t a new technique, but it’s usually overlooked.
Posted by Brad J. Ward | Posted in Analytics, Email, Facebook, Free, Higher Education, Marketing, Recruitment, Research, Social Media, Technology, Thoughts, Twitter, Web | Posted on 24-06-2008-05-2008
With all of the tools readily available at your fingertips, how could you afford to not use them?
Yesterday I had a meeting with BUMegan about some communication for incoming freshman. Previously, the ‘welcome week newsletter’ has been a word document around 5 pages long. Gross. So we started brainstorming about what to do this year. Email? eNewsletter? Welcome Week Blog? Post the info to the Facebook Class of 2012 group and the BUForums?
So I went to my network of higher ed professionals on Twitter for advice.
Posted by Brad J. Ward | Posted in Email, Higher Education, Management, Marketing, Recruitment, Research, Thoughts | Posted on 07-04-2008-05-2008
I’m back from a very refreshing week of fun and sun in Florida with my wife, and am starting to get settled back in the office and get caught up. While I was out I tried my hardest to avoid work email/RSS/Twitter, but it’s just impossible. To my credit, I left 75 work emails unread and 700 posts in my Google Reader, and did pretty well at avoiding Twitter.
Vacation also allowed me to take a step back and look my profession as a whole. I read or re-read a lot of great books:
- Fat Envelope Frenzy
- All Marketers are Liars
- The Price of Admission
- Cluetrain Manifesto
- Made to Stick
- Join the Conversation
Right now is a ‘season of change’ for me both personally and professionally. Vacation couldn’t have come at a better time; in the week leading up to my trip our Director of Admission resigned, our Assistant Director had a baby and went on maternity leave, and our print coordinator (my counterpart) and main web designer who did the butler.edu redesign put in their 2 weeks. Needless to say, change is in the air. With new positions come new opportunities, both for those leaving and those arriving. Change can bring more change, good and bad. And while we are filling positions, there are a few things I can look forward to/lobby for.
For example, our current CMS only allows me to change content on sites within go.butler.edu. I can’t control anything on the homepage, navigation, etc. and only recently got access to the callouts in the margins. (A post on that and web usability has been sitting in my drafts for months. I’ll get it out in after I have a little more data.) With the new web designer vacancy, I am going to lobby for access to the ‘ArtApp’, aptly named after the guy leaving. It is the ‘CMS backdoor’ that allows access to these sorts of things. No better time than now to cut red tape. I was hoping Art would give me the keys before he left, he has very similar feelings as me about the CMS limitations.
Another opportunity will be revamping emails, etc. Currently, I design emails and the copy comes to me. We’ve really worked over the past year at refining the copy into an ‘email-compatible’ format. I kid you not, previous emails have been more than 1.5 pages long in Word… imagine that in a 550-600px box. *shudder* The person leaving the position has been great at recognizing this need and helping to cut text before passing it on, and she has also been a wonderful liason for me to the print department for getting photos for emails. With the absence of this position, I am going to try and get access (finally) to the campus photo library for emails, and start working more on text edits and getting our electronic materials to match the print versions better.
All of these positions will be hard to fill; our team works so well together and hopefully we can find some people to step in and hit the ground running, but still be able to bring us some fresh ideas and thoughts on what we’re doing here. I haven’t even been here a year, and at times I feel myself slipping towards the dreaded rut of moving along with business as usual year after year using previous materials and methods.
In all, it’s good to be back. Do yourself a favor and take some time off if you haven’t recently. It’s healthy for you. Winter is pretty much gone, so get out and enjoy the weather. You don’t even have to go anywhere far; just enjoy a day to yourself. Take a photo walk around your hometown, read some books, play with your kids, work in the yard, wash the cars, clean out the garage, go to a presidential rally, or just do absolutely nothing. But take a day off. There is so much more out in the world other than work and keeping up with the 9-5. The blogs will be here when you return. The emails really aren’t that urgent. The project can wait a few more days. And when you get back, you’ll be refreshed and ready to start back up again.
Posted by Brad J. Ward | Posted in Email, Google, Higher Education, How-To Tuesday, Technology | Posted on 26-02-2008-05-2008
It’s Tuesday already? Just another snowy day here in Indy. Today we talk about creating a form in Google Docs. I’ve been using this because I really like how it instantly drops the form’s results into a spreadsheet. No FormMail to worry about, etc. I used it on an email that I sent out on Friday, and so far there have been about 40 responses. Others wanted to see it, so I just copy/pasted into an Excel file and mailed it back. If they had Google accounts, it would have been even easier. And if there weren’t prospective student emails involved, I would have just made it public for them to see.
And here is the link for the form that is shown: Google Form
Posted by Brad J. Ward | Posted in Email, Google, Higher Education, Technology | Posted on 05-02-2008-05-2008
Sometimes you just happen to ‘StumbleUpon’ things at work. Today, for me, it was this:
I wish that I knew what was going on, and I wish I could get an answer more than “ummm I think maybe some professors are playing around with it?” Do you honestly think a school would let a few absent-minded professors “play around” with a full google solution? Maybe I’m naive, but something is gone on here. Yep, I’m excited. I also just wish it wasn’t some big secret on campus. EMBRACE technology, people.