Chris Meller

So I was recently linked to Paul Krugman’s opinion piece from the NYTimes entitled Prostates and Prejudices and I just can’t ignore it… I mean, you guys actually pay this man to put out hypocritical crap like this? Wow… I need to go to work for the NYTimes.

The third paragraph is where we really start to hit Rudy Giuliani hard:

Let’s start with the facts: Mr. Giuliani’s claim is wrong on multiple levels - bogus numbers wrapped in an invalid comparison embedded in a smear.

So a politician didn’t get his facts right or took something out of context to further his own point? Well, that’s not exactly news, but alright, let’s hear what you’ve got to say…

Mr. Giuliani got his numbers from a recent article in City Journal, a publication of the conservative Manhattan Institute. The author gave no source for his numbers on five-year survival rates - the probability that someone diagnosed with prostate cancer would still be alive five years after the diagnosis. And they’re just wrong.

Wrong you say? Ok, I can believe that. I mean, who’s ever heard of this City Journal and why would we believe a conservative Manhattan Institute anyway? Carry on…

You see, the actual survival rate in Britain is 74.4 percent. That still looks a bit lower than the U.S. rate, but the difference turns out to be mainly a statistical illusion. The details are technical, but the bottom line is that a man’s chance of dying from prostate cancer is about the same in Britain as it is in America.

Wa… Wait a minute. So you’re saying the survival rate in England is actually 74.4% but that it’s actually the same as the claimed 82% survival rate in the United States? I’m not sure my high school Algebra teacher would agree with you on that one, Mr. Krugman.

Oh, by the way, where did you get your claim of 74.4%? Forget to cite a source, did we? And how exactly does 74.4% end up equaling 82%? Oh, right, just take your word for it…

So that’s bogus numbers¹? Check. Invalid comparison²? Check. Smear campaign against Giuliani³? Check.

Hey, I hate the man just as much as the 9/11 next 9/11 guy⁴, but come on. If you’re going to call the pot black, at least make sure you’re not the kettle first.

1. Unless you can cite me a more reliable source than the City Journal. ↩
2. Unless in the universe in which the NYTimes resides 74.4 does in fact equal 82. ↩
3. Unless somehow this is supposed to be flattering to your former El Presidente. ↩
4. 9/11… ↩

So much of what we call management consists in making it difficult for people to work. - Peter Drucker

Boy, isn’t it the truth? My boss is “managing” me right into a hole of utter un-productivity right now.

He apparently doesn’t understand that only 10% of the project is actually the layout of the interface (ie: stuff he can “see”). The other 90% is where you’re working in the background making sure that pretty interface actually does something.

Two to three times a day, he’ll come by, call me, or email me asking if there’s anything new for him to see¹.

No, there’s nothing new for you to see, I’ll tell you when I’m done with something so you can see it. The more time we spend talking about how there’s nothing for you to see, the less time I get to spend actually working on something for you to see.

Add to it that it reinforces a bitter feeling that I’m not getting anything done nearly fast enough even though I’m working my ass off, and you’ve got one thoroughly unhappy developer.

Time to call those two job recruiters back, me thinks…

1. He’s supposedly on vacation this week, but at 8:24 this morning he still found the time to email me asking if there’s anything else he can see. ↩

The Idea
I had a brilliant idea yesterday, and couldn’t wait to get home so I could start coding it. Things were moving along great. I had my database created, I’d gotten Code Igniter configured and running with my standard set of libraries, helpers, etc. and I was hammering away at some code.

Damnation
And then it happened. I got to the heart and soul of the application: the part that interacted with the NewsGator API. Almost instantly, my entire world crashed in around me.

What’s that? You’ve never dealt with the NewsGator API? Consider yourself lucky. Only murderers and rapists should be doomed to such a fate. The NewsGator API is by far, and without a doubt, the worst API I have ever dealt with - and I’ve dealt with quite a few in the past.

The Spectrum
Think of the overall web API spectrum as a scale.

On the one end, you’ve got Yahoo!. Yahoo! offers what could be considered the Utopia of web APIs. They do your “standard” XML and SOAP interfaces, but they also offer a plethora of other options for many of their services: various REST APIs, JSON, and even plain old serialized PHP are all readily available for use. Whatever the language, whatever the platform, you can get data out of Yahoo! with a minimum of pain.

On the far, extreme, totally opposite end of this vast, gigantic, absolutely humongous spectrum of available APIs is NewsGator. You name it, they’ve probably done it wrong with their API. First problem? SOAP.

Ok, SOAP sounds great, and I have no doubt that it has its purposes, but for the love of god people… why? There are so many other alternatives available that could greatly simplify the process we really care about here: getting bits back and forth. I won’t say I have to end-all of answers as to the “perfect” format, but I can readily recommend any number of alternatives to SOAP.

REST? When? I’m working around bugs 24x7!
What? NewsGator has a REST API?

Well, you’d be partially right, at least. You POST your request and get back the same gobbly-gook XML data that you’d have gotten from the SOAP interface. This seems a bit backwards to me, since SOAP is XML-based and a SOAP interpreter would therefore also interpret the XML you’re receiving back (well, most of it). Tacking an XML parser on top of your REST API just seems perverted.

Oh yeah, there’s also no documentation for the REST API… None… Notta. The only thing you’ll find of help is a poorly-formatted PDF document that seems to kinda, maybe, sorta, give you a general idea of what’s going on… To top it off, apparently there’s no equivalent REST-based API for the SOAP-based interface called “PostItem”, according to forum posts.

Sample, Shmample!
Finally, the only code samples you’ll get from NewsGator are in .NET (C# to be precise). Talk about a long way migrating that over to PHP or, god save you, something like Ruby… Yahoo!, on the other hand, provides examples for most of their APIs in .NET, Java, Perl, PHP (4 and 5), Python, etc…

The Adventure Begins…
In my journeys over the past two days, I’ve hit more bumps in the road than I do on my way to work… In the mountains… In the middle of the ghetto… Through a huge swamp… Infested with mutant turtle speed bumps. It’s like a huge real-life version of Frogger, all wrapped in disgusting XML nastiness.

First, and not the least of my problems, has been a bug in PHP’s SOAP implementation that apparently has existed since September of 2004 and which was supposedly fixed in CVS in March of 2005. Here it is July of 2007, and I’m still hitting a problem parsing out a WSDL file that contains a ref attribute in it.

Alright, I know, we’re here to talk about NewsGator’s crappy API, not PHP’s crappy SOAP implementation¹.

NameThatTune
As for NewsGator, let’s start small, shall we?

Ever noticed a difference in naming styles between .NET developers and those developers more commonly found on *nix platforms (Perl, Python, etc.)? Of course I have no empirical data to back me up here, but it seems like .NET developers commonly use functions that FollowACamelCasedNamingPattern, while *nix developers_prefer_to_space_things_out_with_underscores.

I’m sure it’s just personal preference, but I prefer the underscored version, which just seems more easily read to me. That said, NewsGator has a serious case of .NET-itis².

Now it’s time for some real complaints. Let’s dive in.

You’re Lucky Number 17!
One of the first things you’ll obviously want to do when dealing with RSS feeds is get a list of the user’s subscriptions, right? For that you’d probably want the GetSubscriptionsList call… Or, as it’s called in the WSDL declaration, just GetSubscriptions (remember that…). What does the documentation say this call returns?

The result of this call is an OPML document with NewsGator extensions.

Yes, the only link about these extensions points to the raw XSD definition. No further documentation on what exactly each of the returned fields means is forthcoming. To top it off, we’ve taken SOAP and XML to another level entirely. Stick that OPML in your PHP pipe and try to smoke it, fool… You’ll end up using the built in XML parser and looping through looking for tags of the type OUTLINE, if that’s any hint.

So what does the call actually spit back? Well, it’s a huge list of feeds (which I parsed out into an array). Each feed has a number of detail items (16, in fact - and remember, none of them are documented). Some of these items are self-explanatory, such as “TITLE”, “NG:ID”, and “XMLURL”. Others aren’t quite so helpful: “DESCRIPTION” (My blog’s tagline, for instance, is stored in “TEXT”), “NG:SYNCXMLURL”, “NG:UNSEEN”, and “NG:USEDEFAULTCREDENTIALS” among them.

Alright, great. Now I’ve got a list of subscriptions for the user. How many unread items are in each? Well, out of those 16 fields I just got back for each feed in GetSubscriptions, one of them is “NG:UNREAD”. It’s a boolean value.

Apparently we’re meant to use the call GetSubscriptionCounts for this purpose. It returns a similar OPML document as its previous counterpart, but only contains 9 fields for each post this time. Out of those 9 fields, 8 of them are exactly identical to those found in GetSubscriptions. The 9th? “NG:UNREADCOUNT”. And why couldn’t we just make it the 17th field for GetSubscriptions and used a single call? Better yet, how about eliminating “NG:UNREAD” and simply using “NG:UNREADCOUNT” for the same purpose? Guess what, if it’s 0, there aren’t any unread items!

Now the fun really starts!

Spin the Wheel and Place Your Bets! Your Parameter could be the lucky winner!
They say a picture’s worth a thousand words, so we’ll let them do some of the talking here.

Remember our good friend GetSubscriptions? Take a look at its parameters, as per the online documentation:

Now that we’ve got a list of subscriptions, let’s pull in all the items for each feed. For that, we’d use GetNews. Check out its documented parameters:

Wait, what? Is it “token”, or “syncToken”? Make up your minds! With all this SOAP-based nonsense, I’m copy-and-paste’ing my brains out, and suddenly you throw a wrench into the clockwork!

Far from a unique incident, however, we can’t let GetSubscriptions and GetNews take all the blame… GetNews appears to be a bad influence, since it doesn’t play well with other calls either. Another GetNews highlight:

As you can see, our redundant friend GetSubscriptionCounts doesn’t quite match either:

Think you’ve seen enough now? Well, we’re not quite done yet… Just in case format blunders, missing documentation, and parameter SNAFUs weren’t your style, how about “undocumented features”³?

Presto! Watch Me Pull a Parameter out of My Hat!
Not to leave well enough alone, let’s make one more last jab at good old GetNews. I realized my parameter naming problem and corrected it, only to be met with a remarkably similar error telling me that I had an invalid value for the “stripMarkup” parameter, that it couldn’t be left blank.

The stripMarkup parameter? Am I the only one who missed that? Did you see that in any of the screenshots I just posted of GetNews parameters? Let’s look again at the extent of the documentation for this API call:

No, apparently it’s not just me. If you click on through to the SOAP Endpoint, NewsGator provides a helpful test mechanism for each of their API calls. Along with SOAP 1.1, 1.2 and REST mockup requests, they provide a simple HTML form that does an HTTP POST to the REST API and spits back the raw XML you’d get as a response.

Loading up this test form, I see where my error comes from:

Wait a minute! Where did all those extra fields come from? Those aren’t mentioned anywhere else! The only way you could find those is by looking at the raw XML of the WSDL file⁴. Whereas at least some of the other non-mentioned parameters we’ve seen have been slightly user-intuitive, some of these are absolutely meaningless to me. What the hell is the difference between “stripMarkup” and “useScrubber” anyway?

Try… Catch… Finally…
So that’s it. Everything I hate about the NewsGator API, all condensed down into one single (albeit lengthy) rant. It’s all laid out there on the line for anyone to see. Do what you will with it. With any luck, NewsGator will track down my post and (hopefully) fix some of these problems. Even if you just want to use these points as examples of what not to do when building your own web API, at least some good will have come from it all.

And to be perfectly clear, I want to make sure no one walks away from this thinking the API is a total loss. To be fair, there are a lot of good points about it. I think their sync token concept has some really strong points (even if it is confusing at first), and really strengthens their process of syncing so much data back and forth without totally bogging down everyone’s bandwidth in between.

I also don’t want to be thought of as making a personal attack here. As I recall, Gordon at NewsGator was their head API guy a while back, and he was always willing to jump in and help. The first few times I had trouble with the API, oh so long ago, he found my blog and did everything he could to help me through my problems, even though he knew nothing about PHP. He was a great guy, and for all we know this entire API was inherited in some bastardized form from a previous developer.

I also got a comment from Greg Reinacker on one of my other blog posts on the subject, and he too was very eager to help. It’s clear people aren’t the problem here (well, at least on the intentions and motivational sides of the equation), because NewsGator obviously has some amazing talent and passion working for them.

The one thing I can’t figure out through all of this, though, is how the hell NewsGator has managed to build so many products on top of this API. I’m one person and I can’t figure this out - how have they managed to coordinate an entire team of developers, geographically dispersed, and get them all on the same page with this, across so many different programming languages (.NET to Cocoa to Java) and platforms (Windows to OS X to Mobile)? It boggles the mind…

1. For the record, I also attempted to use NuSOAP, as I’ve tried in the past. Even after renaming all the “SoapClient” class declarations to “NuSoapClient” to avoid conflicts with the PHP5 native implementation, there are several hurdles to overcome. Eventually, I gave up and went with the PHP-native version, expecting it to have better support than NuSOAP, which appears to have been abandoned for several years. ↩
2. Which really makes sense, considering it’s coded in ASP… ↩
3. The “politically correct” phrase for a “bug” these days… ↩
4. Which I suspect is used to auto-generate these forms, as a matter of fact. ↩

Over on Digg, I saw the story How Scoble Reads 622 RSS Feeds Each Morning.

Now, without even reading the actual story, I can tell you (cynically, sure) exactly how he does it. Are you ready for this? Alright then… Here’s the answer:

How Scoble Really Reads 622 RSS Feeds Each Morning:
He doesn’t have a real job.

Now go find me someone with a real 8-5 job that actually expects them to get real work done and produce real results or they really get fired and who still has time to read 622 RSS feeds each morning and I’ll be impressed.

Over on Reddit, I found this story, titled New Microsoft Ad Features Girl’s Orgasm.

Now before we get into this, please, go watch the video (pick the first guy on the left, that’s the dad). Don’t worry, we’re not going anywhere, I’ll be here when you’re done.

Taps foot semi-impatiently…

Ok, done? That’s great…

So this girl’s “orgasm” is one quick gasp as the father imagines the worst possible things that could happen to his (hot) daughter while she’s out on a date with this loser, spurning him to change the curfew they’d just finished negotiating. The question being posed is, is it appropriate for Microsoft to use the image of teenage sex (in conjunction with the stereotype of the loser boyfriend, no less) to help sell their Suspicious “Male”-detection module of the new Hotmail service?

Personally, I don’t think there’s anything scandalous about this ad. I find it creative and a fresh perspective from a company (Microsoft) long known for being the stereotypical hard-core straight-laced business user (much like ‘PC’ in the Apple Mac vs. PC ads). I think the real question here is much broader: How Prudish is American Society?

Ever seen the movie Eurotrip? Hey, it had Michelle Trachtenberg in it, I couldn’t not watch… Well, in one scene, they’re hitchhiking along and she decides she’ll get a ride… by taking off her shirt.

No one stops.

The point is, this straight-laced American highschooler thought taking her shirt off would get any number of creepy guys to pull over and give them all rides. Unfortunately, they find out that in Europe, seeing a chick with her shirt off isn’t a new thing. Their example was an orange juice ad with lesbians in it, which certainly hammered home the point to me (perverts!). Elsewhere in the movie, the head mistress at a brothel claims that America was founded by prudes, which seems to be an accurate claim given the rest of the evidence we see throughout.

So I ask you again: Why is this ad groundbreaking? Are we that delusional to think that our children aren’t finding out about sex elsewhere, and that a 2-second gasp in an advertisement from the 800lb gorilla of software is going to turn them all into hormone-crazy sex heathens?

Let’s grow up, America. Stop blaming the advertising for turning your children into sex-starved homicidal maniacs and start taking a little extra responsibility for raising a child that’s broken, alright?

Hiding the truth from our society as a whole doesn’t make it go away. Pretending sex doesn’t happen doesn’t mean your children aren’t having it. This isn’t a nightmare, where it will go away if we just ignore it long enough, this is real life… It’s here to stay.