A/B Marketing Test on 2 (4) Self-pubbed Books

On my birthday, my son mentioned to me that the mom of one of his friends, Tawdra Kandle, had also self-published a book and that, coincidentally, it was also her birthday. I came home that night and looked for her book. Turns out, we'd both set our books to be free via KDP Select for our birthday and amazingly, we were at #1 (hers) and #2 (mine) on the Contemporary Fantasy Free list. How cool is that? Then I looked at her book -- she'd also written a paranormal romance, she'd also set it in Florida, she'd also based her setting on the town of Cassadaga, and she'd released her first book on the very same day, December 9th, that I'd released mine! We'd even priced the same, at $3.99.

I left a note on her blog, tweeted her and followed her on Twitter, but it took me another month to realize that this was a perfect opportunity for an A/B marketing test. An A/B test, if you aren't familiar with the term, compares two samples. In Web design, a company posts two pages, and filters half their users to one page, the other half to the other page, and sees which gets a better response for their goals.

Tawdra, author of Fearless, Breathless, and now Restless, takes marketing and promotion seriously. She's following all the advice of the self-publishing community that exhorts authors to promote and sell and build a community. She's working hard at doing the right things, even in the right way. For example, she tweets a lot, but only rarely does she try to promote her own book and only when appropriate, for example when she releases a new book or has a blog post somewhere. She's even following the advice of writing fast -- she released two new books in roughly the time it took me to release one.

I am not working hard at doing all the right things. In fact, I'm not working on promotion at all. I figure maybe when I have three books out I'll start doing that. Or maybe five. Well, or maybe never.

So if she's doing everything right and I'm doing....well, mostly nothing...how much better could I do if I worked as hard as she did? It seemed worth trying to measure.

Novel Rank is a site that tracks the Amazon ranking of books. It's terrible for providing real sales numbers, because it actually has no way to measure them, but it shows the movement of a book's ranking on Amazon. It's an imperfect form of measurement, because it doesn't show sales outside Amazon. That said, both of us had enrolled our books in KDP Select (which doesn't allow you to sell ebooks outside Amazon), so it seemed likely that the results would be reasonably close to accurate. Besides, it was the best tool that I was going to get.

So -- book comparisons first.

Her first book, Fearless, is currently averaging a 4.5 rating on Amazon. A Gift of Ghosts is at 4.6. I'd say that's close enough to a tie. Her second book, Breathless, has a 4.5. A Gift of Thought currently is at 4.7. But Breathless has been out since March and Thought has only been out for a few weeks, and the earliest reviewers always seem to be the most enthusiastic. Rating-wise, I'd say we're equivalent.

Tawdra had professional covers made for her titles, I did my own. Point for her. Many people  in the self-publishing community insist that a professionally-designed cover is essential for success.

Tawdra had her books professionally edited, I didn't.Technically, a point for her (although someday I'll write a rant about the self-published attitude toward editing, and meanwhile, I will mention that more than one of my reviews comment on how well Ghosts was edited. Just saying...)

Create a website. Tawdra has her own domain name at tawdrakandle.com and a pretty typical author website with pages about her books, links to other sites, and a blog. Obviously, I do have a blog, and there are a couple pages for the books, but I’ve had it for years and it’s mostly personal. I don't treat it as a sales tool. I write about my kid and my dog as much as anything else, I think. Definite point for her. 

Facebook -- at the time I'm writing this, Tawdra has 1912 likes on Facebook, with 488 people talking about it. I have 6 likes, one person talking. (I guess that person is talking to herself. Maybe it's me? I don't really know.) Is it too obvious to say she definitely wins on Facebook?

Twitter -- again, at the time that I'm writing, Tawdra has 2500 followers on Twitter. I have 204. She has tweeted 5,475 times: I have tweeted 705 times. I’d actually been on Twitter for 2 years longer she had – since Feb 2007 compared to her Feb 2009, so effectively she tweeted more than ten times as much as I did. Like Facebook, Twitter is a clear win for Tawdra. 

Real-life book events. This is actually one that I think would be fun to do, but I haven't. She's done local book signings and even some long-distance book signings and readings, some here in Florida and some in New Jersey. (I should point out that if she sold paper books during the signings, those sales wouldn't be reflected in Novel Rank, so it's possible that those events were more rewarding than was measurable.)

Blog tours and events and exchanges and reviews. I wrote a guest post for Kindle-aholic back in May, but that's the only thing I've done. As you might expect, she seems to have done quite a bit. I'm drawing the line at finding links to everything, but if you read her tweets, you'll find plenty of links, including a Facebook party for the release of her third book. 

In terms of effort, it's clear that she's working really hard to promote and sell her books. She is doing everything that you're supposed to do. So how is it paying off and is it worth the effort?

The following chart shows our Amazon sales ranks for our first books from the day I started tracking in May, until the day I set A Gift of Ghosts to be free in June. (Novel Rank doesn't track free books, so I had to stop there.)

The lower line belongs to A Gift of Ghosts and it is the higher sales rank on Amazon. The numbers for both of us are...well, nowhere near life-changing. In the time measured, I sold 64 copies of A Gift of Ghosts.The dips at the end of the month where Fearless outsells Ghosts are when Tawdra did her real-life events, so those dips might actually have been even more pronounced (in other words, she might have sold more copies than are shown, so improved her sales ranking even more, but in a way that's not tracked.) 

The following graph shows the releases of my second book, her third (both,conveniently, in June.)

To promote my release of A Gift of Thought I posted a message on my Facebook pages and set Ghosts free for four days. She held a Facebook party, did giveways, tweeted -- I probably should have paid more attention, but I wasn't planning to write this blog post, so I didn't. Suffice to say, I didn't do much work at all, and she did plenty. (I should also mention that A Gift of Thought  is priced at $4.99 and she kept her price at $3.99.)

There are undoubtedly dozens of ways to argue with this test. I don't think it's possible to do a true A/B test in any real sense of the word, because there are too many variables that are outside our control. Our books are different. She does have professionally designed covers, but one could argue that mine are stronger. I've gotten a lot more reviews on Ghosts, probably mostly because of my original fanfiction audience, so maybe that's what makes the difference.

I do, however, feel pretty comfortable with my decision not to worry about Facebook or tweeting or a professionally-designed website or, really, spending hours working on marketing. It may be true that 90% of marketing is useless and you have to do it for the 10% that matters, but until I have some evidence about what that 10% might be, I'm not going to bother.So far, my not bothering has done just as well (okay, slightly better) than all of Tawdra's hard work.

By the way, Tawdra seems to be a very nice person -- you should follow her on Twitter at @tawdra, and if you like paranormal YA, you should give her books a try.