Outsourcing success tidbits

Last year we did a pretty cool “local O2O” (Online-to-Offline) project with a local TV station. The budget and timeframe were tight. And this project simply *begged *to be done with Ruby on Rails for a host of reasons.

We came in on-time, on-budget, with an exceptional end result after using elance.com to find a team in India.

Aside from insisting on and talking to references and seeing sample work, here’s a few tidbits we learned about making that work. To get a good bid (always go fixed-price) you have to have a good spec. Some keys include breaking the project into Phases, starting with a clear spec for Phase I, breaking that into at least 10 milestones. Some ingredients for a good spec are:

  • A full database schema. A fuzzy idea of the database, and/or turning the schema design over to anyone who’s not proven to be exceptional at it, is a train wreck. I repeatedly heard from good developers that a well-designed database lets them bid one-third to one-half of what it would cost otherwise. Take the money you’ll save and invest it in the schema design.
  • **Mock up screens. **You can not rely upon text, you have to show the fields, the buttons, etc. (It’ll help solidify your thinking, too.) Let me say this. The only three spots where we ran into ambiguity issues were the three places where we played a little loose on the sample screens, thinking it was “obvious” what those screens would look like.
  • Spec the environment. If it has to run on Heroku, make that clear.
  • Spec the unit tests. Contract developers in my experience do lousy job at this unless unambiguously required to do it right. And it’ll cost a lot less if part of the original bid than if you say later “oh hey, what about hitting X% unit test coverage.”
  • Include 2 or 3 major “gotchas”. Purposefully put at least 2 or 3 things into your spec that don’t make a lot of sense or contradict other elements of the spec. See who catches that. It’s instructive in picking a development team. As a trivial example, suppose part of your spec you’re going to run at Heroku. Put some aspect in your spec about storing files in some folder on the website. If they know their stuff they’ll immediately point out Heroku has no local file system, that you’ll probably want to push files over to S3. It’s a fair test IMO, because those sort of gotchas will happen anyway, so you’re entitled to ‘test’ developer’s ability to catch them before they spend you money writing code that won’t work.
  • Daily involvement. Generally, the closer you can come to full time co-involvement, the better the results. We setup a testing tag-team to test every feature as it was implemented, and actually designed the spec for incremental development and testing.
  • Spec comments and readability. I’m all about comments (lots) and code clarity (not using every cool shortcut the language offers, favoring understandability by humans over saving a few CPU cycles.) I provide examples of obtuse but fun little shortcuts that I don’t want to see, and the old fashioned way I do want to see it done instead. If I really wanted uber-dense tight code I’d go back to doing it all in APL or Forth. What I want is code anyone can look at and understand months from now.
  • Git / github. If you aren’t using it, do.
  • **Daily code review. **Review code as it’s checked in. Catch bad habits early.

Obviously that’s the tip of the iceberg, but those simple tidbits saves us a lot of time and hassle.

IMO Tropo not ready for prime time. Went with Twilio.

Subtitle: “When you are betting your ass on a platform, if you have a choice of platforms, bet on the company that *has *to deliver an actual working solution to survive, not on a company for whom the platform is an interesting diversion from their core business.”

I like the folks at Voxeo a lot. But I had to shut down an earlier business because their business model of paying per-port per-year is unworkable for a business with significant peaks and valleys in usage. (The business was a voice message broadcast system for schools, etc., where virtually all of the user activity happens within fairly narrow time frames. But after a short eval period we had to pay for ports 24x7x365 even though our customers’ usage occurred during perhaps 10% of a typical week. And since users expect messages to go out in a timely way, simply having 1/10 as many available channels was not an option.)

So I was very excited a year or two later when they introduced Tropo, cloud based telecom with no per-port fees, pure pay-for-use business model. even though it was too late for the prior business.

When a new project that required voice and SMS came along, I thought Tropo would be perfect. But Twilio looked interesting as well. We built a fairly significant app that handles inbound and outbound voice calls and SMS, and looking at the claimed features, Tropo looked like a hands down winner. Nonetheless, we architected the product to be relatively platform neutral “just in case.”

Once again, my belt-and-suspenders approach to app architecture paid off because Tropo was a lot of cool things, but “ready for prime time” is not among them.

Going into the project, here’s what initially convinced us to incorporate the Tropo platform:

Supposed advantages of using Tropo over Twilio:

  1. Tropo is based on Voxeo, a mature and highly profitable company/technology, Twilio is a startup
  2. Tropo has voice recognition, Twilio doesn’t
  3. Tropo apps can (supposedly) send/receive tweets via an associated Twitter account, allowing you to (supposedly) build very cool automated Twitter apps. Twilio doesn’t. In fact, when you go to tropo.com their headline reads “Add Voice, SMS, Twitter, and IM to Your Applications”
  4. Tropo has 24×7 tech support, Twilio doesn’t.
  5. Tropo is (supposedly) more robust, built as it is upon the Voxeo platform.
  6. Tropo has a very generous “free for development” policy so you can get up and running without paying for phone numbers and messages.

We dabbled with both at first, to get our feet wet, then proceed to build our app on Tropo. Addressing each of those (supposed) Tropo advantages in order, here’s why we were absolutely compelled to switch to Twilio.

  1. The overall impression I got was that Tropo being a disruptive pay-for-use business model offered by hugely successful company is that, well, they’re just dabbling with this… that Voxeo has no current profit motive to make it work right… their mainline business is (a) doing just fine thank you and (b) might even be disrupted if Tropo works *too *well. That’s just my impression, but it’s based on the following:
  2. Voice recognition. This claim is actually true, although Tropo does not offer (built-in) the full range of VXML grammars that make voice recognition practical. And we found that ambient noise makes it unusable for Real World Apps if your users are commonly on cells, in restaurants, or cars.
  3. Twitter. UPDATE: This seems to have been fixed recently. So far so good. In my experience, attaching a Twitter account to Tropo app works for maybe one day. Then the connection to Twitter silently dies, and there is NO way to tell it’s happened. YouR app just stops receiving tweets. There is no API to verify or restore the connection. You have to remove the Twitter account, then re-add it, manually. But remember… you have no idea when your Tropo/Twitter connection is broken. We tested several Tropo apps, with several Twitter accounts, over several weeks and the results were absolutely repeatable… Tropo apps fail to respond to Tweets within a day or two. It just plain doesn’t work. And it’s not Twitter’s fault… another thing we tested was whether an app like Hootsuite would ‘keep’ it’s connection to Twitter persistently and during the same several weeks of testing, had zero problems with Hootsuite. Now, I’m all about ‘things will get better over time’ except… well… when the company’s *headline *says it works… it kinda needs to work.
  4. **Tech support. **Actually this was a tie. We’d send support requests at midnight and get a prompt “I’ll have someone look at this” reply from a human at Tropo, but the actual answers came faster from Twilio about half the time. And on some major issues (Twitter, ‘lost’ SMS messages) it took Tropo a week, even two to respond, whereas we never waited longer than a working day from Twilio.
  5. In our experience, Twilio is more robust, period. Let me count the ways: – UPDATE: this ‘feature’ was finally removed so they no longer opt-out your users for you. If your user texts the word STOP to your Tropo app, your app does not get to handle the opt-out… Tropo does it automatically, and silently throws away all traffic to that user. Your app never knows about it (neither does the user!), and for all you know your messages are going through. This is wrong. A telecom platform should *not *silently throw away messages. (Twilio doesn’t presume to opt-out your customers, they let your app do it so you can keep track of it!)

Tropo silently fails to deliver SMS messages if your app sends them too fast. And there’s no specific rate, just general guidelines of ten per minute. Twilio lets your blast them out as fast as it wants, then Twilio handles the rate limiting to 1 per second. In other words, Twilio is designed for reliable message delivery, Tropo is “best efforts” and (even worse) does not **inform your app when something went wrong.
– To simulate modest concurrent user loads, we wrote a two very simple tester apps: one app sends 10 SMS messages to another app. **We never, not once, got all 10 SMSs to make it on Tropo.
Twilio never missed a message. Tropo support said that’s strictly an artifact of apps-talking-to-apps related to their gateway, that it would not happen if multiple cell phone users were talking to, but…. would you bet your ass on that? (Me neither.)
– Tropo doesn’t have the very helpful callbacks Twilio offers that let you do post-sessions.
– There’s five more items on my list, but candidly, IMO the issues of Tropo discarding or losing messages, silently, under multiple circumstances, should be enough to cross them off any developer’s list.
6. Free vs pay-for-use during development. Tropo is free. Twilio isn’t. At Twilio a phone number costs $1/month at Twilio. Text messages or phone-minutes are 2 or 3 cents each. *Dude, if you’re building a business do you even **care **about $20 to $40 in telecom charges during a month of testing?* (Me neither.) What you do care about is building something that works reliably when your users use it.

So that’s why we are deploying based on Twilio, not Tropo. (Though we left our Tropo code in place for when they get it together on the message-delivery-reliability front, and get Twitter working, we’d love to use that feature.)

But the moral of the story is clear to me: if your providers aren’t betting their asses on doing a great job for you, you may not want to bet your ass on their services. Ditto hosting, web design, you name it.

What’s in a name?

“… as in … ‘howdy pardner’ … my parents had a real fine sense of humor” is a common refrain when meeting new people. True enough.

There’s at least one more… a gentleman in the Philippines, whose parents saw the Clint Eastwood movie Paint Your Wagon and naturally assumed Pardner was a perfectly normal American name. Who am I to throw stones? I named my second daughter Jemima after I first saw the name in the credits of a movie filmed in England. (When I say ‘first saw’, syrup bottles don’t count.)

And there’s an “almost” … a gentleman in southern Cal who goes by Pardner since that’s what he was called from birth, including by his godfather, Bing Cosby, who attended his baptism.

Befluent, a complete phone-based learning system, for grins

I’ve built a couple of pretty substantial learning management systems, including what was perhaps the first web-based authoring and publishing system (one that eventually became docent.com now sumtotalsystems.com).

But the most challenging learning system I have tackled was “Befluent” a system I built a while back to test the viability of a pure audio phone-based system. The initial market I had in mind was the large numbers of earnest immigrants (like my friend Khwaja from Afghanistan) who would love to get off state welfare and get a job but whose English is holding them back. The state pays for classes, but without a regular conversation partner, class-time alone simply does not work for learning English as an adult.

My design goals were somewhat ambitious:

  • 100% phone-based deployment for students and graders
  • Fully functional learning management system, students and graders have IDs
  • Chatroom capability for students to collaborate with or without moderators
  • Multi-lingual prompts that adapt to learner (start with native language prompts, graduate to verbose, slow English prompts, then graduate to faster/shorter english prompts
  • Self-paced exercises with multiple-choice, fill-in, and essay responses
  • Graded quizzes, with helpful comments from graders
  • “Assembly line” grading, where particular questions can be assigned to particular graders

These recordings are ‘real’ — they are recorded off of actual phone
calls to a custom-developed phone learning system, not computer simulations, and not hacked-together audio files. (So the sound quality is so-so on the recordings.)

Notice the prompts seem painfully slow to native English speakers — because these are real prompts, intended for real learners.

The demo recordings are located at:

About 7 minutes and 7 MB.
This is what a student experiences
when they call into the phone number, choose a quiz (from many),
and answer different types of questions, including ‘voice answers’

About 2 minutes 30 seconds and 2 MB.
This is how a real teacher can call into the system, choose
a question to grade, then hear each student’s answers to that
question, assign a grade, and record a personal comment.
The grading could be done via internet, or phone.

About 1 minute 30 seconds.
When a student ‘calls back’ after a teacher has graded
their voice answers, the system tells them which quizzes
have graded answers. The student hears their grades,
plus whatever personal comments the teacher recorded.

Anyone with an interest in phone-based learning systems, I’d love to hear from you!

NY Times today outs me as founder of the nation’s largest street corner begging franchise

I guess I can admit it now that that damned NY Times article today “outs” me as the ultra-secretive founder of the “will work for food cardboard sign” franchise operation with over 37,000 well-heeled franchises capitalizing on street corners from coast to coast. The article applauds my innovative use of vertical manufacturing to supply durable plastic “pre-distressed, torn” signs indistinguishable from cardboard signs, and the patented “authentique homage de homeless” fragrance spray most franchisees opt to use, but is critical of our average franchise fee of $57,000, saying that “Wynn has made street corner begging the province of the well-heeled, pricing many wanna-be beggars out of the market.” I won’t apologize, however, since I think the flip side of that coin is that my organization brings a new professionalism to an industry formerly populated with less savory types. The NY Times article touches on that briefly, noting a typical beggar-franchisee has an MBA and an hourly income of $317 per hour.

My favorite quote from the article is:

Asked whether he regretted any aspects of his corner-begging-empire, Wynn wistfully sighed. “Dogs. Those damn dogs. I wish I’d been the one to realize how much money there is in renting those damn dogs to my franchisees. By the time we figured it out, that Vicks guy out east had locked up the rental-pitiful-dog market with long-term contracts. What can I say, he beat me to the punch, fair and square.”

Even before reading the NY Times article, deep inside, every time you saw one of our franchisees, you knew it was something like that.

Didn’t you?