<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
		>
<channel>
	<title>Comments on: Split Test Your Emails: A or B?</title>
	<atom:link href="http://squaredpeg.com/index.php/2008/08/11/split-test-your-emails-a-or-b/feed/" rel="self" type="application/rss+xml" />
	<link>http://squaredpeg.com/index.php/2008/08/11/split-test-your-emails-a-or-b/</link>
	<description>Brad J Ward's Thoughts on Higher Education Recruitment</description>
	<lastBuildDate>Thu, 15 Sep 2011 07:23:52 +0000</lastBuildDate>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.2.1</generator>
	<item>
		<title>By: Wednesday Links &#187; Reading, Writing and Big Ideas</title>
		<link>http://squaredpeg.com/index.php/2008/08/11/split-test-your-emails-a-or-b/comment-page-1/#comment-883</link>
		<dc:creator>Wednesday Links &#187; Reading, Writing and Big Ideas</dc:creator>
		<pubDate>Sat, 06 Sep 2008 22:27:29 +0000</pubDate>
		<guid isPermaLink="false">http://squaredpeg.com/?p=182#comment-883</guid>
		<description>[...] All of Your Other Priorities Living Up To Your Potential Is B.S. The Allure of Junk Mail From India Split Test Your Emails: A or B? The Ultimate Guide to Achieving Total Happiness Be Here Now Marketing: How United Airlines Can Take [...]</description>
		<content:encoded><![CDATA[<p>[...] All of Your Other Priorities Living Up To Your Potential Is B.S. The Allure of Junk Mail From India Split Test Your Emails: A or B? The Ultimate Guide to Achieving Total Happiness Be Here Now Marketing: How United Airlines Can Take [...]</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Rob S.</title>
		<link>http://squaredpeg.com/index.php/2008/08/11/split-test-your-emails-a-or-b/comment-page-1/#comment-800</link>
		<dc:creator>Rob S.</dc:creator>
		<pubDate>Mon, 18 Aug 2008 14:05:48 +0000</pubDate>
		<guid isPermaLink="false">http://squaredpeg.com/?p=182#comment-800</guid>
		<description>@Kyle, I checked out the &quot;Top 14 power words&quot; link. Definitely some powerful (perhaps overused, though still effective?) words - I felt like I was reading the Sunday sales fliers.

Some of the sentences used on the page are enough to make an English major cringe: &quot;Your emails or advertising is missing to get answered ?&quot; It appears to be a European company. Either English isn&#039;t the native tongue of the person who wrote that content, or it is a case of SEO gone very wrong!

Hmm. All 14 words in one or two sentences... that could be entertaining.</description>
		<content:encoded><![CDATA[<p>@Kyle, I checked out the &#8220;Top 14 power words&#8221; link. Definitely some powerful (perhaps overused, though still effective?) words &#8211; I felt like I was reading the Sunday sales fliers.</p>
<p>Some of the sentences used on the page are enough to make an English major cringe: &#8220;Your emails or advertising is missing to get answered ?&#8221; It appears to be a European company. Either English isn&#8217;t the native tongue of the person who wrote that content, or it is a case of SEO gone very wrong!</p>
<p>Hmm. All 14 words in one or two sentences&#8230; that could be entertaining.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Rob S.</title>
		<link>http://squaredpeg.com/index.php/2008/08/11/split-test-your-emails-a-or-b/comment-page-1/#comment-795</link>
		<dc:creator>Rob S.</dc:creator>
		<pubDate>Sat, 16 Aug 2008 15:54:09 +0000</pubDate>
		<guid isPermaLink="false">http://squaredpeg.com/?p=182#comment-795</guid>
		<description>I don&#039;t blog and rarely post comments - probably explains the length of those rare comments.</description>
		<content:encoded><![CDATA[<p>I don&#8217;t blog and rarely post comments &#8211; probably explains the length of those rare comments.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Kyle James</title>
		<link>http://squaredpeg.com/index.php/2008/08/11/split-test-your-emails-a-or-b/comment-page-1/#comment-794</link>
		<dc:creator>Kyle James</dc:creator>
		<pubDate>Sat, 16 Aug 2008 13:28:43 +0000</pubDate>
		<guid isPermaLink="false">http://squaredpeg.com/?p=182#comment-794</guid>
		<description>Rob&#039;s back!  Welcome back from vacation buddy!  It&#039;s good to have your insight back in the conversation!

I don&#039;t have anything that strong to contribute, but I&#039;ve been enjoying vacation myself and haven&#039;t gotten a links of the week out for a while and here&#039;s one of those valuable posts that fits right into this conversation.  

&lt;a href=&quot;http://www.leadsexplorer.com/blog/117/top-14-power-words-for-email-advertising-and-communications/&quot; rel=&quot;nofollow&quot;&gt;Top 14 power words for email, advertising and communications&lt;/a&gt;.</description>
		<content:encoded><![CDATA[<p>Rob&#8217;s back!  Welcome back from vacation buddy!  It&#8217;s good to have your insight back in the conversation!</p>
<p>I don&#8217;t have anything that strong to contribute, but I&#8217;ve been enjoying vacation myself and haven&#8217;t gotten a links of the week out for a while and here&#8217;s one of those valuable posts that fits right into this conversation.  </p>
<p><a href="http://www.leadsexplorer.com/blog/117/top-14-power-words-for-email-advertising-and-communications/" rel="nofollow">Top 14 power words for email, advertising and communications</a>.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Karlyn Morissette</title>
		<link>http://squaredpeg.com/index.php/2008/08/11/split-test-your-emails-a-or-b/comment-page-1/#comment-788</link>
		<dc:creator>Karlyn Morissette</dc:creator>
		<pubDate>Fri, 15 Aug 2008 22:20:28 +0000</pubDate>
		<guid isPermaLink="false">http://squaredpeg.com/?p=182#comment-788</guid>
		<description>Forget TargetX....*I&#039;M* taking notes on this ;-)</description>
		<content:encoded><![CDATA[<p>Forget TargetX&#8230;.*I&#8217;M* taking notes on this <img src='http://squaredpeg.com/wp-includes/images/smilies/icon_wink.gif' alt=';-)' class='wp-smiley' /> </p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Brad J. Ward</title>
		<link>http://squaredpeg.com/index.php/2008/08/11/split-test-your-emails-a-or-b/comment-page-1/#comment-785</link>
		<dc:creator>Brad J. Ward</dc:creator>
		<pubDate>Fri, 15 Aug 2008 22:09:19 +0000</pubDate>
		<guid isPermaLink="false">http://squaredpeg.com/?p=182#comment-785</guid>
		<description>&quot;Plus I think my reply is longer than the original post lol&quot;

I think that&#039;s usually the case ;) Thanks for chiming in.</description>
		<content:encoded><![CDATA[<p>&#8220;Plus I think my reply is longer than the original post lol&#8221;</p>
<p>I think that&#8217;s usually the case <img src='http://squaredpeg.com/wp-includes/images/smilies/icon_wink.gif' alt=';)' class='wp-smiley' />  Thanks for chiming in.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Rob S.</title>
		<link>http://squaredpeg.com/index.php/2008/08/11/split-test-your-emails-a-or-b/comment-page-1/#comment-783</link>
		<dc:creator>Rob S.</dc:creator>
		<pubDate>Fri, 15 Aug 2008 21:54:48 +0000</pubDate>
		<guid isPermaLink="false">http://squaredpeg.com/?p=182#comment-783</guid>
		<description>&quot;Delivra is pretty lackluster when it comes to reporting. Any email past ~6 months disappears, so I have to keep all data on my own.&quot; Yikes!

Total click-throughs or unique recipient click-throughs? There is a difference. The distinction? Some recipients may have clicked on a link multiple times. (They do this - I see it in the metrics.) Some recipients may have clicked on every link in the message...

Also, which version did you send first? You may have tipped a spam filter at some point, thus causing some of the first and much/all of the second message to be filtered to the bulk folder. That obviously would skew the results.

In terms of last year vs. this year comparisons, was the message sent at about the same date as last year? Same day of the week? Were there more messages (print and/or e-mail) sent before this message this year than last year? These can have huge impacts. Multivariate testing extends beyond a singe e-mail send or Web page, of course.

Karlyn is correct about the stats being an open-to-click ratio, but they likely also reflect how the e-mail service provider (ESP) is recording opens.

Some e-mail tracking 101:
- Opens rely on display of 1x1 pixel tracking image
- Images in e-mail are often blocked by default (Yahoo, Gmail, Outlook, etc., etc.,)
- Many recipients that open a message are not recorded as opens since the tracking image does not display. (Conversely, preview panes might record an open even if the person merely clicked on the message to delete it.)

To combat this, some (but not all) ESPs count every message that has a click as an open even if the tracking image does not display. (The assumption: if the recipient clicked on a link, they must have opened the message.) If your ESP does this, it is little wonder why the version with more clicks also has more opens. The higher number of opens is, at least in part, because of the higher number of clicks. (Some spam filters follow links before the recipient even receives the message. When that is true, an open is recorded. E-mail metrics are a murky field.)

On the subject of image blocking:
- Picture the above message with all images blocked. Not so pretty, but thankfully the text is able to get the message through and the app link also exists apart from the button. (You may have seen worse results if the only app link were the button, since that button wouldn&#039;t display by default for many recipients.)
- Alt text for images is extremely important. Given the prevalence of image blocking with e-mail clients, what you had for alt text for the apply button might have been as or more helpful than the button itself. It is a good idea to always use image alt text to help give some context to blocked images. BIGGER TIP #1. If you use a lead-off banner image as this message appears to do (assuming it doesn&#039;t have the &quot;if you prefer to view this message in a browser&quot; spiel, wasting valuable preview pane space imho), the alt text for that image functions as the snippet that follows the subject line in the Gmail inbox, as well as some other e-mail clients. Further, if images are blocked but alt text is displayed, it functions as teaser text within the message. If you don&#039;t use a lead off banner image, you can use a 1x1 &quot;invisible&quot; image as the lead off and give it alt text, though some argue this looks spammy to filters.

Other reasons why version B may have had more clicks:
- The stats may reflect total clicks and not unique recipient clicks. The second message contained two visible links, including the button. (You have the answer to this, obviously.)
- There were two places to click through to the app page, and that alone can increase the likelihood of clicks (even unique recipient clicks).
- The first link is buried within a sentence/paragraph BIGGER TIP #2. Don&#039;t overlook how big this tip is, because it is huge. A link that stands on it&#039;s own in an e-mail (i.e. it isn&#039;t part of a sentence or paragraph) is more likely to be clicked. Your button stands on it&#039;s own. You honestly may have seen similar results with version B if the split test had one link in the paragraph as it is in version A, and the second link as a standalone but not in button format.

(And by now the TargetX folks reading this are taking notes to make a marketing tip e-mail out of it...)

@Melissa, there is a lot potentially spammy about the message:
- Yes, the exclamation point in the subject line
- The high ratio of text within the images
- The high image-to-text ratio
- The high html-to-text ratio
- The use of the word free in the body of the e-mail, particularly since it is in bold. Free is a powerful word, so using it can be good. Not sure about the risk of bolding, though. You can verbally work around it, too, and still bold your point.
- Having a url in a message that doesn&#039;t matched the hyperlinked url underneath it (tracking was on, so the url underneath it would have been a tracking url generated by the ESP, not http://go.butler.edu/apply). It is safer to make the link text, i.e. apply online for admission.
- Simply having open and click tracking turned on ups the spam score.

All of these things, even in combination, are not necessarily fatal in getting the message filtered as spam, depending on the sensitivity of the spam filter doing the filtering. More important is the reputation of the sender, in this case the particular sending IP of the ESP being used, not the Butler e-mail address the message appeared to come from. The ESPs sending infrastructure (sends per connection, connections per hour, etc.) can have a major impact, too, as well as other factors controlled by the ESP (use of authentication and a whole lot more).

That said, I think too many e-mail marketing pros downplay the impact content can still have on spam filtering nowadays. There was some credible research done in the last year to support this. Sorry I don&#039;t have a reference - I have it filed away somewhere.

It is a good idea to include a seed list of Yahoo, Hotmail, etc., accounts and add them to every send. While you can&#039;t always be assured a message didn&#039;t get filtered as spam because it made it to the inbox of your particular seed account (Yahoo may have filtered the send further down the recipient list, for instance), you can at least see the red flag when it does hit the spam folder of your seed account.

Brad, I have some ideas improve the message in your example, but I think my employer prohibits moonlighting for a competitor ;) Plus I think my reply is longer than the original post lol</description>
		<content:encoded><![CDATA[<p>&#8220;Delivra is pretty lackluster when it comes to reporting. Any email past ~6 months disappears, so I have to keep all data on my own.&#8221; Yikes!</p>
<p>Total click-throughs or unique recipient click-throughs? There is a difference. The distinction? Some recipients may have clicked on a link multiple times. (They do this &#8211; I see it in the metrics.) Some recipients may have clicked on every link in the message&#8230;</p>
<p>Also, which version did you send first? You may have tipped a spam filter at some point, thus causing some of the first and much/all of the second message to be filtered to the bulk folder. That obviously would skew the results.</p>
<p>In terms of last year vs. this year comparisons, was the message sent at about the same date as last year? Same day of the week? Were there more messages (print and/or e-mail) sent before this message this year than last year? These can have huge impacts. Multivariate testing extends beyond a singe e-mail send or Web page, of course.</p>
<p>Karlyn is correct about the stats being an open-to-click ratio, but they likely also reflect how the e-mail service provider (ESP) is recording opens.</p>
<p>Some e-mail tracking 101:<br />
- Opens rely on display of 1&#215;1 pixel tracking image<br />
- Images in e-mail are often blocked by default (Yahoo, Gmail, Outlook, etc., etc.,)<br />
- Many recipients that open a message are not recorded as opens since the tracking image does not display. (Conversely, preview panes might record an open even if the person merely clicked on the message to delete it.)</p>
<p>To combat this, some (but not all) ESPs count every message that has a click as an open even if the tracking image does not display. (The assumption: if the recipient clicked on a link, they must have opened the message.) If your ESP does this, it is little wonder why the version with more clicks also has more opens. The higher number of opens is, at least in part, because of the higher number of clicks. (Some spam filters follow links before the recipient even receives the message. When that is true, an open is recorded. E-mail metrics are a murky field.)</p>
<p>On the subject of image blocking:<br />
- Picture the above message with all images blocked. Not so pretty, but thankfully the text is able to get the message through and the app link also exists apart from the button. (You may have seen worse results if the only app link were the button, since that button wouldn&#8217;t display by default for many recipients.)<br />
- Alt text for images is extremely important. Given the prevalence of image blocking with e-mail clients, what you had for alt text for the apply button might have been as or more helpful than the button itself. It is a good idea to always use image alt text to help give some context to blocked images. BIGGER TIP #1. If you use a lead-off banner image as this message appears to do (assuming it doesn&#8217;t have the &#8220;if you prefer to view this message in a browser&#8221; spiel, wasting valuable preview pane space imho), the alt text for that image functions as the snippet that follows the subject line in the Gmail inbox, as well as some other e-mail clients. Further, if images are blocked but alt text is displayed, it functions as teaser text within the message. If you don&#8217;t use a lead off banner image, you can use a 1&#215;1 &#8220;invisible&#8221; image as the lead off and give it alt text, though some argue this looks spammy to filters.</p>
<p>Other reasons why version B may have had more clicks:<br />
- The stats may reflect total clicks and not unique recipient clicks. The second message contained two visible links, including the button. (You have the answer to this, obviously.)<br />
- There were two places to click through to the app page, and that alone can increase the likelihood of clicks (even unique recipient clicks).<br />
- The first link is buried within a sentence/paragraph BIGGER TIP #2. Don&#8217;t overlook how big this tip is, because it is huge. A link that stands on it&#8217;s own in an e-mail (i.e. it isn&#8217;t part of a sentence or paragraph) is more likely to be clicked. Your button stands on it&#8217;s own. You honestly may have seen similar results with version B if the split test had one link in the paragraph as it is in version A, and the second link as a standalone but not in button format.</p>
<p>(And by now the TargetX folks reading this are taking notes to make a marketing tip e-mail out of it&#8230;)</p>
<p>@Melissa, there is a lot potentially spammy about the message:<br />
- Yes, the exclamation point in the subject line<br />
- The high ratio of text within the images<br />
- The high image-to-text ratio<br />
- The high html-to-text ratio<br />
- The use of the word free in the body of the e-mail, particularly since it is in bold. Free is a powerful word, so using it can be good. Not sure about the risk of bolding, though. You can verbally work around it, too, and still bold your point.<br />
- Having a url in a message that doesn&#8217;t matched the hyperlinked url underneath it (tracking was on, so the url underneath it would have been a tracking url generated by the ESP, not <a href="http://go.butler.edu/apply" rel="nofollow">http://go.butler.edu/apply</a>). It is safer to make the link text, i.e. apply online for admission.<br />
- Simply having open and click tracking turned on ups the spam score.</p>
<p>All of these things, even in combination, are not necessarily fatal in getting the message filtered as spam, depending on the sensitivity of the spam filter doing the filtering. More important is the reputation of the sender, in this case the particular sending IP of the ESP being used, not the Butler e-mail address the message appeared to come from. The ESPs sending infrastructure (sends per connection, connections per hour, etc.) can have a major impact, too, as well as other factors controlled by the ESP (use of authentication and a whole lot more).</p>
<p>That said, I think too many e-mail marketing pros downplay the impact content can still have on spam filtering nowadays. There was some credible research done in the last year to support this. Sorry I don&#8217;t have a reference &#8211; I have it filed away somewhere.</p>
<p>It is a good idea to include a seed list of Yahoo, Hotmail, etc., accounts and add them to every send. While you can&#8217;t always be assured a message didn&#8217;t get filtered as spam because it made it to the inbox of your particular seed account (Yahoo may have filtered the send further down the recipient list, for instance), you can at least see the red flag when it does hit the spam folder of your seed account.</p>
<p>Brad, I have some ideas improve the message in your example, but I think my employer prohibits moonlighting for a competitor <img src='http://squaredpeg.com/wp-includes/images/smilies/icon_wink.gif' alt=';)' class='wp-smiley' />  Plus I think my reply is longer than the original post lol</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Brad J. Ward</title>
		<link>http://squaredpeg.com/index.php/2008/08/11/split-test-your-emails-a-or-b/comment-page-1/#comment-755</link>
		<dc:creator>Brad J. Ward</dc:creator>
		<pubDate>Wed, 13 Aug 2008 13:44:27 +0000</pubDate>
		<guid isPermaLink="false">http://squaredpeg.com/?p=182#comment-755</guid>
		<description>I don&#039;t think it&#039;s a problem.  My emails never suffer as a result.  I would avoid using &#039;spammy&#039; words in conjunction with !, such as ending a subject with free!, scholarships!, money!, etc.

A lot might depend on who you&#039;re sending through and if they are whitelisted with most email providers.  Delivra seems to be pretty good at this.

I&#039;m sure there is research for both sides, it just depends on which side you want to be on.  A lot of times I think people think way too much when it comes to all of this.</description>
		<content:encoded><![CDATA[<p>I don&#8217;t think it&#8217;s a problem.  My emails never suffer as a result.  I would avoid using &#8216;spammy&#8217; words in conjunction with !, such as ending a subject with free!, scholarships!, money!, etc.</p>
<p>A lot might depend on who you&#8217;re sending through and if they are whitelisted with most email providers.  Delivra seems to be pretty good at this.</p>
<p>I&#8217;m sure there is research for both sides, it just depends on which side you want to be on.  A lot of times I think people think way too much when it comes to all of this.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Melissa Cheater</title>
		<link>http://squaredpeg.com/index.php/2008/08/11/split-test-your-emails-a-or-b/comment-page-1/#comment-754</link>
		<dc:creator>Melissa Cheater</dc:creator>
		<pubDate>Wed, 13 Aug 2008 13:37:28 +0000</pubDate>
		<guid isPermaLink="false">http://squaredpeg.com/?p=182#comment-754</guid>
		<description>Very cool - thanks for posting the results!  I&#039;m a total novice in email marketing: so it&#039;s OK to have a &quot;!&quot; in the subject line? I&#039;ve always avoided !?* but, again, novice!</description>
		<content:encoded><![CDATA[<p>Very cool &#8211; thanks for posting the results!  I&#8217;m a total novice in email marketing: so it&#8217;s OK to have a &#8220;!&#8221; in the subject line? I&#8217;ve always avoided !?* but, again, novice!</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Kyle James</title>
		<link>http://squaredpeg.com/index.php/2008/08/11/split-test-your-emails-a-or-b/comment-page-1/#comment-745</link>
		<dc:creator>Kyle James</dc:creator>
		<pubDate>Tue, 12 Aug 2008 14:26:25 +0000</pubDate>
		<guid isPermaLink="false">http://squaredpeg.com/?p=182#comment-745</guid>
		<description>Coolness.  We need to get more people thinking about A/B testing and other things.  I&#039;ll admit I&#039;m bad about it too, because we just don&#039;t have time.  It&#039;s free to do and  can be very valuable for website testing also.  You can get into multivariable testing, but if you have the time (meaning months and years to test) you might as well just keep it simple w/ A/B and keep tweaking the loser.</description>
		<content:encoded><![CDATA[<p>Coolness.  We need to get more people thinking about A/B testing and other things.  I&#8217;ll admit I&#8217;m bad about it too, because we just don&#8217;t have time.  It&#8217;s free to do and  can be very valuable for website testing also.  You can get into multivariable testing, but if you have the time (meaning months and years to test) you might as well just keep it simple w/ A/B and keep tweaking the loser.</p>
]]></content:encoded>
	</item>
</channel>
</rss>

