Sunday, July 1, 2012

A/B Marketing Test on 2 (4) Self-pubbed Books

On my birthday, my son mentioned to me that the mom of one of his friends, Tawdra Kandle, had also self-published a book and that, coincidentally, it was also her birthday. I came home that night and looked for her book. Turns out, we'd both set our books to be free via KDP Select for our birthday and amazingly, we were at #1 (hers) and #2 (mine) on the Contemporary Fantasy Free list. How cool is that? Then I looked at her book -- she'd also written a paranormal romance, she'd also set it in Florida, she'd also based her setting on the town of Cassadaga, and she'd released her first book on the very same day, December 9th, that I'd released mine! We'd even priced the same, at $3.99.

I left a note on her blog, tweeted her and followed her on Twitter, but it took me another month to realize that this was a perfect opportunity for an A/B marketing test. An A/B test, if you aren't familiar with the term, compares two samples. In Web design, a company posts two pages, and filters half their users to one page, the other half to the other page, and sees which gets a better response for their goals.

Tawdra, author of Fearless, Breathless, and now Restless, takes marketing and promotion seriously. She's following all the advice of the self-publishing community that exhorts authors to promote and sell and build a community. She's working hard at doing the right things, even in the right way. For example, she tweets a lot, but only rarely does she try to promote her own book and only when appropriate, for example when she releases a new book or has a blog post somewhere. She's even following the advice of writing fast -- she released two new books in roughly the time it took me to release one.

I am not working hard at doing all the right things. In fact, I'm not working on promotion at all. I figure maybe when I have three books out I'll start doing that. Or maybe five. Well, or maybe never.

So if she's doing everything right and I'm doing....well, mostly much better could I do if I worked as hard as she did? It seemed worth trying to measure.

Novel Rank is a site that tracks the Amazon ranking of books. It's terrible for providing real sales numbers, because it actually has no way to measure them, but it shows the movement of a book's ranking on Amazon. It's an imperfect form of measurement, because it doesn't show sales outside Amazon. That said, both of us had enrolled our books in KDP Select (which doesn't allow you to sell ebooks outside Amazon), so it seemed likely that the results would be reasonably close to accurate. Besides, it was the best tool that I was going to get.

So -- book comparisons first.

Her first book, Fearless, is currently averaging a 4.5 rating on Amazon. A Gift of Ghosts is at 4.6. I'd say that's close enough to a tie. Her second book, Breathless, has a 4.5. A Gift of Thought currently is at 4.7. But Breathless has been out since March and Thought has only been out for a few weeks, and the earliest reviewers always seem to be the most enthusiastic. Rating-wise, I'd say we're equivalent.

Tawdra had professional covers made for her titles, I did my own. Point for her. Many people  in the self-publishing community insist that a professionally-designed cover is essential for success.

Tawdra had her books professionally edited, I didn't.Technically, a point for her (although someday I'll write a rant about the self-published attitude toward editing, and meanwhile, I will mention that more than one of my reviews comment on how well Ghosts was edited. Just saying...)

Create a website. Tawdra has her own domain name at and a pretty typical author website with pages about her books, links to other sites, and a blog. Obviously, I do have a blog, and there are a couple pages for the books, but I’ve had it for years and it’s mostly personal. I don't treat it as a sales tool. I write about my kid and my dog as much as anything else, I think. Definite point for her. 

Facebook -- at the time I'm writing this, Tawdra has 1912 likes on Facebook, with 488 people talking about it. I have 6 likes, one person talking. (I guess that person is talking to herself. Maybe it's me? I don't really know.) Is it too obvious to say she definitely wins on Facebook?

Twitter -- again, at the time that I'm writing, Tawdra has 2500 followers on Twitter. I have 204. She has tweeted 5,475 times: I have tweeted 705 times. I’d actually been on Twitter for 2 years longer she had – since Feb 2007 compared to her Feb 2009, so effectively she tweeted more than ten times as much as I did. Like Facebook, Twitter is a clear win for Tawdra. 

Real-life book events. This is actually one that I think would be fun to do, but I haven't. She's done local book signings and even some long-distance book signings and readings, some here in Florida and some in New Jersey. (I should point out that if she sold paper books during the signings, those sales wouldn't be reflected in Novel Rank, so it's possible that those events were more rewarding than was measurable.)

Blog tours and events and exchanges and reviews. I wrote a guest post for Kindle-aholic back in May, but that's the only thing I've done. As you might expect, she seems to have done quite a bit. I'm drawing the line at finding links to everything, but if you read her tweets, you'll find plenty of links, including a Facebook party for the release of her third book. 

In terms of effort, it's clear that she's working really hard to promote and sell her books. She is doing everything that you're supposed to do. So how is it paying off and is it worth the effort?

The following chart shows our Amazon sales ranks for our first books from the day I started tracking in May, until the day I set A Gift of Ghosts to be free in June. (Novel Rank doesn't track free books, so I had to stop there.)

The lower line belongs to A Gift of Ghosts and it is the higher sales rank on Amazon. The numbers for both of us are...well, nowhere near life-changing. In the time measured, I sold 64 copies of A Gift of Ghosts.The dips at the end of the month where Fearless outsells Ghosts are when Tawdra did her real-life events, so those dips might actually have been even more pronounced (in other words, she might have sold more copies than are shown, so improved her sales ranking even more, but in a way that's not tracked.) 

The following graph shows the releases of my second book, her third (both,conveniently, in June.)

To promote my release of A Gift of Thought I posted a message on my Facebook pages and set Ghosts free for four days. She held a Facebook party, did giveways, tweeted -- I probably should have paid more attention, but I wasn't planning to write this blog post, so I didn't. Suffice to say, I didn't do much work at all, and she did plenty. (I should also mention that A Gift of Thought  is priced at $4.99 and she kept her price at $3.99.)

There are undoubtedly dozens of ways to argue with this test. I don't think it's possible to do a true A/B test in any real sense of the word, because there are too many variables that are outside our control. Our books are different. She does have professionally designed covers, but one could argue that mine are stronger. I've gotten a lot more reviews on Ghosts, probably mostly because of my original fanfiction audience, so maybe that's what makes the difference.

I do, however, feel pretty comfortable with my decision not to worry about Facebook or tweeting or a professionally-designed website or, really, spending hours working on marketing. It may be true that 90% of marketing is useless and you have to do it for the 10% that matters, but until I have some evidence about what that 10% might be, I'm not going to bother.So far, my not bothering has done just as well (okay, slightly better) than all of Tawdra's hard work.

By the way, Tawdra seems to be a very nice person -- you should follow her on Twitter at @tawdra, and if you like paranormal YA, you should give her books a try.


  1. Nice job Sarah! As a kindle reader, I don't do more than glance at the cover. As an older person (55yrs) I don't tweet. I have my facebook setup for work, so I don't use it for personal stuff. The first thing I do when I find an author I like is look for their website so that I can send them an email to get on their mailing list so that I know when they put out a new book. Somehow I found your blog which is serving the same purpose as I was able to attach it to my chrome home page which I have never figured out how to do for anyone else.
    BTW a self done website is really easy, unless you are making heaps, there are a lot of do it yourself website tools out there.
    Carol from New Zealand

  2. A mailing list is probably a really good idea. Twitter is just silly -- I told one person that I didn't follow people back if their tweets didn't interest me and she told me I should follow them but make a list of the people I was interested in reading and only read tweets from those people. So, okay, everybody's tweeting and no one's listening? Why does that make any sense at all?

    As for the website, I've thought about it -- even played with some software -- but domain names cost money, and I've never really seen the point. Maybe someday I'll get inspired!

    1. Hi! I found you through a comment you left on Lindsay's blog and I was intrigued by your physicist heroine. (I've got a marine biologist in my book, but he's not the main character.)

      I loved your analysis. You were able to show real data for something I suspected - I follow the ranking of a couple of writers who do *lots* of promotion, but their books don't seem to be rising.

      So I've got my first book up and I've decided not to even tell many people until I get my second or third book out. But I did want to set up a website and/or blog, so that has me sidetracked for a while.

      So the reason I posted a reply in this part of the thread? :)You can get a domain name for $10 for a year (I got mine through GoDaddy) and Google will host the site for free under Google sites. I'm still working on setting mine up so admittedly the learning process does take a lot of writing time.

    2. Wow, it's gotten cheaper since I last looked (which was probably years ago -- I thought it was $50/year or so). I wasn't familiar with Google sites -- it looks like fun and also like a huge, gigantic time-sink. I suspect if I get started, I'll spend days playing with templates and fonts and colors.

      On the book promotion, I did tell lots of people about my first book on its first free day, which was in its first couple of weeks. Not via twitter, but I posted a message on a mom's board that I'd been a member of for years, mentioned it on critique circle, told my WOW guild, and -- most importantly, I think -- wrote a fanfic story that included a note at the end that they could download my first book for free at Amazon and that it would be nice if they'd leave a review there. I suspect that bit of promotion -- which, really, is the only thing I've ever done, besides submit my free days to free sites -- was hugely important. I have no idea how many of my reviews are from fanfic readers but I'd guess a lot of the first thirty or so are, and I'm pretty sure that solid block of reviews is what has kept Ghosts selling. So I'm not advising not telling people about your book -- I definitely did that! -- I'm just not convinced that follow-up promotion makes a difference. I haven't done anything at all for A Gift of Thought, besides the one mention on Facebook, and it's been much slower to get reviews. It's also never been free, which probably makes a difference, too, of course!

    3. Thanks for the reply - It's helpful to know exactly what you *did* do. And I do plan to do something when I release my second book. But I hadn't considered writing a fanfic story. I'll have to give that some thought.

    4. I think the fanfic thing probably only works if you're a regular writer of fanfiction. I'd written a lot of stories (albeit for a teeny-tiny fandom), so many of my readers weren't total strangers. (Well, they were total strangers in the sense that I'd never actually met any of them, but they knew my name.)

    5. Agreed. Even as I wrote that I was thinking that it would take a while to build up a reader base at a fanfic site. And I'm not sure how much fanfic there is for MG other than Harry Potter (and probably Rick Riordan.)

  3. Sarah, you should think about a Facebook Author's Fan Page. For an example of one that really works, check out Debora Geary on Facebook.

    I like your practical, laid-back attitude toward marketing your books. I've heard others say pretty much the same thing: a lot of work for very little payback.

    Oh, and graphs!

  4. I have an author's fan page. (I think, anyway.) That's the one with 6 likes and 1 person talking. And I'm ... um, a member? a fan? whatever it's called ... of Debora Geary's page, already. I love some Facebook games and there's a Eureka group on there that I enjoy, but I mostly just find it really confusing. Although not nearly as confusing as Google+!

  5. Interesting experiment Wyndes. I've tried to dig empirical evidence out of marketers whenever they pop their heads up looking to sell their services. I've yet to get any evidence. As far as I can tell, all marketers sell is marketing.

    Anyway, congratualtions on your success, especially the non-marketed kind.