BM roadmap for the next 2 months

November 6, 2007

Future6-1
Here is what I’m currently working on for BookMooch:

– The next update (in a few days) will be a big back-end database change, and I’m hoping to get all the bugs worked out, and I’ll only do the update right after a backup in case there’s a problem.

– The “search engine” had to be completely rewritten, as my measurements found that it was responsible for 80% of the disk writes. I also found and fixed some big bugs with the current search engine in the process so the search results should be much improved.

– BookMooch is currently spending about 15 seconds writing to the disk for every 60 seconds elapsed. What that means is that if I don’t really improve disk writing efficiency, I’ll have big problems when the site grows more as I won’t be able to keep up.

– Next after the database change is finishing up the new forums feature & redoing tech support so that anyone who wants to can help answer questions.

– Once that’s all done, I’m writing BookMooch plugins to Facebook and Open Social and adding BookMooch to Google Books, and I expect those changes will bring it a lot more new users. I want to make sure we’re ready for the growth before I cause it.

– I’ve also had to change a lot of the code in the “mooching” and “pending” process, to build an API on top of it now that I have my first real user. http://www.bookswim.com/ wants to automate their use of BookMooch, to offer our users the books they have no demand for, and get books that they need. Again, I’m testing a lot, but there’s a possibility I’ll introduce some bugs, and I’ll fix those right away.

– I will probably soon have to institute a “reservation” system for popular books, where you can’t mooch a book that many people want until you’ve received their wishlist notification, or all the notifications have gone out. The reason, is that with the new API, it’ll be possible to write an “auction sniper” to always obtain every book by constantly checking to see if a book is now available. One can do that with RSS too, by tying an RSS feed to an auto-mooch program (once I speed up the refresh rate on the RSS feed, they’re once-per-hour now). Probably, the way this will work is that you’ll be able to mooch a book once you receive your wishlist notification email, but not before, and if the book isn’t on your wishlist, you won’t be able to mooch it until everyone who has that book on their wishlist has been notified. I’m still thinking this one through, and will bring it up for full discussion when I get closer to having the time to implement it.

ps: the next update will let you opt-out of the new “related books wishlist notification” feature.

35 Responses to “BM roadmap for the next 2 months”

  1. fluteflute said

    Is the new forum system going to ‘go live’ anytime soon?

  2. re: forum

    I have about 2 week’s work to finish the forum, and that, combined with the new tech support system (which will sit on top of the new forum software) probably means that it’s a month away, probably December.

  3. Erin C said

    I think I will like your new wish list ‘reservation’ format. I don’t know what RSS feed is or if I could even do it LOL! This new way sounds like everyone will be on a more even playing field. It will hopefully also be easier getting those high demand books when your all set for the new rush of users:)
    Thank you for all you do John!

  4. Erin said

    Just wanted to say thanks again for all your hard work-

    I don’t understand most of what you wrote about today, but I know that I am sending a book out today or tomorrow, and I should be receiving a book today or tomorrow, so I am a very happy camper.

    And for that, you deserve many, many thanks every day 🙂

  5. FergHova said

    It must be tough being the man 24/7. Teach me, won’t you?

  6. teachie said

    This all sounds very complicated. Thanks so much for all the work you are putting into this.
    From a very happy BookCrosser.

  7. Andrew said

    A possible approach to the ‘reservation’ thing, to allow for the fact that you can’t reasonably expect people to check their email instantly – a flat waiting period.

    You add a book, the mails are sent out, and people with it wishlisted have, say, twelve hours to claim it in; after that, it’s open season.

  8. The reality, unfortunately, is that automated mooching is possible now, and not even that hard to do. It’s a technique called “screen scraping” where you write a program to look at the html from a page, and act like a human being would.

    Initially, the ebay “auction snipers” had no access to an ebay API, and they still had no problem implementing it.

    We’ve also seen a lot of people threatening to use the RSS feed feature on BookMooch to do “sniping” as well — this was threatened several times on this forum by various members, in response to their dislike of the (then) new random shuffle of wishlist notifications.

    I suspect that people don’t understand what I mean by a “reservation” system.

    All I mean is that if there are 3 people who are wishlisted for a book, say:

    – john (1st)
    – allan (2nd)
    – rob (3rd)

    BookMooch currently sends each notification one hour apart.

    What I’m proposing is that when Allan gets his email telling him the book is available, he can mooch it, but not before he gets his notification. And, someone who doesn’t have this book on their wishlist can’t grab the book before the others have had their chance.

    What I want to prevent is having Allan sit all day on his web browser, reloading the “book details” page to see when the book is available, and grab the book as soon as it’s available.

    That’s completely possible now, either by:
    a) reloading the book details page over and over
    b) using RSS
    c) writing a program to watch a page

    Make sense?

    -john

  9. Andrew said

    Aha! I didn’t realise that there were currently staggered notifications. Yes, this makes limiting it to “when you get your email” make a lot more sense… I think I had assumed they all went out in a batch.

  10. Mary Ann said

    I think the way PBS does it, the first person in line for a book (ie the oldest wishlist entry) gets and email, and 48 hours to respond. No one else can even see that the book has been listed. Then, if person #1 declined, or did not respond, person #2 gets 48 hours. A book doesn’t get put up for everyone to mooch until after all wishlisters have had a chance to decline.

  11. re: PBS

    So, my suggestion is a bit similar to PBS, except that:

    1) the order of the BM notifications is a random shuffle, not based on seniority

    2) the BM notifications go out every hour, not every two days, so there isn’t nearly as long to wait

  12. Jimmy said

    I think you should stop trying to make everything fair. You’re just making things more complicated.

  13. Zillah said

    Thanks John for all the great work you do on this site- it’s the greatest thing since virtual sliced bread!

    Amongst all the wishlist notifications, are there any plans to say, add something like a ‘hold’ or ‘reserve’ button for a small period of time, so that for example, people outside the country of the potential mooch have time to have an ‘ask if not to my country’ email replied to?

    Or perhaps there could be an feature for members to be able to mark individual books with ‘will send overseas’ or ‘only to my country’ to solve the problem of people wanting to be ‘worldwide’ but not wanting to send heavy books, so having to be ‘ask if not to my country’ just in case. In effect, people wouldn’t need to be ‘ask if not to my country’ anymore- they could just use the delay feature if constrained by finances for example.

    Does that make any sense? LOL

    Thanks for the great site!

  14. Regen said

    First! I love Book Mooch, and I appreciate everything you do. However, since you started the current “random shuffle”, I have never received an e-mail that a wish list book was available- to me. The only way I can catch a book is to see if it’s currently available online because I can’t access home e-mail from work. If you take that capability away, it doesn’t look like I will ever again get a wishlist book!

  15. re: “hold for overseas”

    It’s my plan to allow people to mooch from others who have a “ask me if you want me to send to another country” setting, and then the book owner can say “no” to the request, without penalty, and the book is relisted.

    In this way, overseas members will be on the same footing as other members.

  16. Also, I don’t think any of us understand what sort of automated mooching you’re coding to be available from other sites. How will that affect us?

    The people at BookSwim approached me with this idea:

    1) they let people “rent” 3 books at a time, they’re like a netflix for books

    2) as such, they have lots of books that they have too many copies of

    3) could they “give away” the books they don’t have a need for, through BookMooch?

    4) in return, could they get books that they have an audience for

    This seemed quite reasonable to me, as they would only succeed if they offered books that people actually wanted to mooch, and it’d be fine by me if they got some books they had a purpose for, in return.

    My concern is that they immediately thought of implementing their own wishlist system, where they’d check every 5 minutes to see if a book they wanted was available, and thus they’d likely always be the 1st to get any highly-sought after book. That I *didn’t* like.

    So, I wanted to see if there was a way to encourage their fair use of the bookmooch system, since it would enrich us all to have more books available, but in a way that was fair to everyone else.

    -john

  17. Felicity said

    John you have my deepest sympathies spending half your time fixing glitches to make this wonderful site better and the other half explaining it to everybody. Thankyou for all yor efforts.

  18. Stephanie said

    I must have missed something. I thought the wishlist notifications went out *4* hours apart, not 1 hour apart?

    I love all the new stuff, and I think the random shuffle you have created is much better than the way PBS does it, so thank you so much.

  19. Michael said

    Let me see if I correctly understand the reservation system you are proposing. Here is my attempt to precisely describe the rules, and I hope that you will please correct me as needed:

    Each book has a “status,” that is either “on-hold” or “off-hold.”

    When a user X attempts to mooch a book B, the system checks whether B is currently on-hold. If so, then user X may not mooch B until either (1) X is notified that B is available for mooching, or (2) B goes off-hold.

    Books are usually off-hold. A book goes on-hold under two circumstances: (1) When B is first entered, it is found to already be on one or more wishlists. (2) While B is off-hold, one or more users add B to their wishlist.

    Once a book B is on-hold, it remains on-hold until one of the following is true: (1) There is no user X who has B on her wishlist, who has not received a notification for B since B last went on hold; or (2) B is removed from the system (either the owner deletes it, or the book is mooched).

    Once each hour, starting from the time B goes on-hold and ending when B goes off-hold, the system does as follows:
    (1) Choose a user X who has B on his wishlist, but who has not yet been notified that B is available;
    (2) Notify X that B is available.

    When a book B goes off-hold or is removed from the system, all notifications for B are “cleared” from the system.

  20. Yes Michael, your logic diagram (grin) of the proposal is exactly right, with the only mistake being that I mis-wrote in specifying that the delay between notifications is 1 hour, as it actually is 4 hours.

    And there’s one other tiny wrinkle, which is that the first notification goes out 2 minutes after the book is added, so that the person adding the book has 2 minutes to hit “whoops, remove that book” if they accidentally added it without meaning to.

  21. Michael said

    When a book is “on-hold,” how would it appear to users who do not have it on their wishlist? Would it be like happens now between the time when you click “Mooch” and the book’s owner accepts your request, that the book is effectively invisible to the rest of the world?

  22. DubaiReader said

    One small problem that occurs to me is that I can currently pick up the odd book via my wish list and get it mooched by an Angel. Once the new system is in place I will not be notified if a book on my wish list becomes available and the swapper does not swap overseas.
    Is there any way around this?
    As I live in Dubai and all my mooches are overseas, this is a significant loss. And no, I don’t search with a fancy programme!

  23. rachelsmdai said

    Hey, John – aren’t you on vacation??!!

  24. Barb said

    I like all of this. I agree that being too “fair” is counter productive. On the other hand, sometimes I don’t check my e-mail for 2 days and certainly no usually more than once every 24 hours. I would MUCH prefer a system that gave me 48 hours to respond to a mooch notification. But that is just me. Lets see what happens, most of my wish list is NOT highly sought after stuff… for now.

  25. Teresa said

    Please, please do implement the feature about ask-first features. It´s a great source of frustration, and there is no way to do the “right” thing.

    About bookswim, they would be held out also to book mooching ratios? Because if they list thousands of books they got no audience for (say ten thousand copies of the da vinci code), they would have hundreds of points and could conceivably pick a lot of “desirable” books.

    about rss feeds, any chance if you put some sort of delay ( say 24, 48 hours) you could put them working again? Because rss feeds are the only way to be warned of “foreign” books not listed on any amazon ( say italian, or portuguese books), plus for some very prolific authors ( somebody on the forum wanted asterix books, for example. Or if somebody wants more wodehouse extremely useful the best chance, if you got dozens and dozens of titles and hundreds of editions. I suppose the alternative is building huge wishlists.

  26. chunnie said

    All the work you have put in to improve this site is super. Searching for books in my country is so much easier than it used to be. All these new ideas you keep working on are fantastic. How you spoil us! Your efforts are moochly appreciated and we love you :p

  27. re: RSS feeds

    RSS feeds *are* currently working, and are on a 1h refresh cycle. Only the RSS feeds for full text searches are permanently gone.

    -john

  28. Teresa said

    I am not sure I understand, if you mean wishlist rss feeds working yes, but that a person can check inventory daily or wait for emails. Same thing for saved for later.

    If you mean rss feeds for search results gone for good, that is what I meant when talking of non-amazon books and foreign books. We can only add to wishlist books which are on *one* of those amazons (or which somebody manually entered before, which is still not ideal for all lot of reasons), and right now that means mostly english language+french+german+japanese. A lot of books but not all books in the world. If you are going to install a wishlist reserve book system ( in which case, likely a lot of wishlists will grow very much in size) so nobody gets an advantage of rss feeds and if text feeds are not too heavy, bringing back rss feeds for text searches would be very useful for those things you can not wishlist at all or which are just too general to wishlist properly.

  29. Delania said

    John, i love this site and while i hated the new wishlist system at first i’ve gradually adjusted to the new format. I’m am asking you to now leave it alone! If your going to change it yet again then just go ahead and do what PBS does, just shorten the hold time to 12 hours or something. Part of the reason i fell in love with this site was because of the way the wishlist system worked but as changes keep taking place some of the magic is fading. While this may also apply to me i want to quote my mother “Life isn’t fair!”.

  30. Anke said

    I don’t know how close attention you pay to the forum/mailinglist, but I think that thread about BookSwim might be worth a look.

  31. Brian said

    FrugalReader also tracks when a user first put a book on their wish list and an avaiable book goes to the first person who requested it (i.e. whoever has the oldest pending unfulfilled wish).

    FR also lets you set “Notify” or “Request”. Notify sends and e-mail, and then presumably waits a while (I’m not sure). Request just automatically requests the book for you if you’re up next in the queue without waiting for confirmation.

    What I’m not sure about on FR is what happens when someone with “Notify” is ahead of me if my wish type is “Request”.

    At any rate, a system that allows the user with the oldest wish to automatically mooch a wish list book, without waiting for a notification or response seems worth thinking about.

  32. Aliera said

    I’m just curious, but how many wishlisters does it take to be considered popular? 5, 10, 15, 20+?

  33. Diane said

    I have had trouble twice now with not being able to connect to bookmooch to mooch a book after I recieved my e-mail notification. On both occasions I have missed the book because I couldnt connect for a couple of hours.Are other people having problems or is it just me?

  34. Hope said

    I really appreciate all the work you put into BookMooch 🙂 Thank you!

  35. Martin said

    If there are 0 hits for an item, you get the listing of matching titles that are not available (LOVE THIS), and then you get a list of movies underneath. I really dislike seeing those movies there, it seriously undermines the identity of the site. I come to the site thinking about Paul Auster and Lorrie Moore, and the site gets me thinking about Bruce Willis and Bette Midler. That’s a real problem. Protect the brand, or at least let those of us who don’t want to see “A Nightmare on Elm Street 5: The Dream Child” — this happened on my last search — opt out of it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: