Archive for the ‘Web Tech’ Category

WebException and the HttpWebResponse

February 21, 2009

The following code is used to make a request and get the results:

HttpWebRequest req = (HttpWebRequest)WebRequest.Create("");
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
StreamReader reader = new StreamReader(resp.GetResponseStream());
string contents = reader.ReadToEnd();

contents will contain the HTML of this blog if the server gives a 200 OK response. Anything else will throw a WebException. You can wrap the snippet above in a try-catch to handle a non-200, but the exception is thrown in the GetResponse call so you get nothing from the actual response. 404? May as well be a 500.

Today I discovered that the WebException itself has two properties: Response and Status. This Response is the same as the resp above so you can extract out the server response in the catch.

This whole behavior of HttpWebRequest is counterintuitive in the sense that a non-200 is not an exceptional circumstance; I would have expected the response to be accessible and the status code to be populated.


How to Parse DateTimes from the Twitter API

September 11, 2008

If you were wanting to interact with the Twitter API under .NET, you might find yourself trying to convert a date value from the XML results over to a DateTime in your code. Twitter uses a weird format for their dates—Thu Sep 04 11:09:28 +0000 2008—that doesn’t get converted properly with just a good ol’ DateTime.Parse. You could spend a lot of time iteratively trying to figure out the correct format to use. Or, you could just use the following (my free gift to you, dear Twitter-API-consuming reader):

DateTime.ParseExact((directMessage.SelectSingleNode("//created_at").InnerText), "ddd MMM dd HH:mm:ss zzzz yyyy", CultureInfo.InvariantCulture, DateTimeStyles.AdjustToUniversal);

directMessage here is an XmlNode representing a single direct message extracted from a list of direct messages. I found this out after many iterations since all of the .NET libraries for Twitter just punt on this conversion.

The Land of Unicorns and Rainbows

September 2, 2008

“This shit is too pedantic, too convoluted, and violates too many preconceived notions of how authentication works. Instead of trying to figure out your bullshit, a user will just use the same username and password that he uses for everything. Problem solved.” — Ted Dziuba, “OpenID is Why I Hate the Internet”

Polishing the Chrome

September 2, 2008

I’ve just finished reading Scott McCloud’s comic book introduction to Google Chrome, Google’s new browser that will be released tomorrow in some fashion. I am very intrigued by this new player and I think McCloud’s work is an accessible introduction to the basic ideas behind the design. (Of course, it could have been summarized in a single page or blog entry much more efficiently than being spread over 40 pages but McCloud did a great job in explaining things.)

When Jason Kottke first began speculating that Google was working on its own browser, I dismissed it just like I did his notion of Google OS—a Linux distro with Google branding. (I’m not sure Kottke can count this as a win, by the way, since five years worth of bupkis does not make for a successful prediction.) It seemed like an overreach for Google, well outside its focus. The intervening years have proven that Google views its mission as pretty much everything on or related to the Internet.

But I think Google Chrome makes a lot of sense. For one thing, Google now has some skin in the game. It can act in such a way that Microsoft and Apple have for so long: if there’s some special feature that it would like to be prevalent, it can implement it in its browser and propose it as a standard to some friendly standards body. Microsoft got XMLHttpRequest in there early (among many other examples) and Apple got the canvas element accepted in exactly that way. Google, previously, had to work within the Microsoft-Apple-Mozilla constraints and now it gets to be the interloper, the mole. I think that’s a powerful position and likely well worth whatever money Google is throwing at this project.

Also, it clearly has some innovations to bring forth. Putting each tab in its own process is genius. At once it solves the issue of memory leaks and the expansive memory footprint that afflicts every modern browser as well as protecting the user from malicious code in a very effective way. Google also developed its own Javascript virtual machine called V8 to couple to the open-source WebKit. Refactoring the Javascript engine has become a hobby with the browser makers: SquirrelFish from Apple, TraceMonkey from Mozilla, and excuses from Microsoft (paraphrased thusly: “Oh, Javascript isn’t really that important. We focused our performance improvements in IE8 on everything else that stunk about IE.”) The neat thing about Google’s Javascript optimizations is that they took a completely different approach to speeding things up, which means the other browser makers will likely copy them. Innovation in the browser space is an unmitigated good thing.

Finally, as the most popular set of properties on the Internet, Google can drive adoption of its browser by offering improved performance and heightened functionality when visited using Chrome. It can tie its sites into the browser in a way that it never could if it weren’t a browser maker itself.

In the end, each of these reasons for developing its own browser also benefit the user so I applaud Google’s decision and hope that they can succeed in following through—which has always been a problem in Google’s application ADD environment.

[UPDATE (9/2/2008): Google Books has a better version, so I’ve updated the Google Blogoscoped link to point there.]

Feed and Social Distribution Session

July 24, 2008

Talk with Jerry Cain, Ari Steinberg (Manager of News Feed Team), and Tom Whitnah.

  • Use the right channel to deliver the right message. Overview of the old ways of communication with users. Requests: when one user wants another user to take a specific action. Notifications: user-to-user to tell a user an action has been taken towards her, app-to-user to tell a user about something important to her from the application. Feed story: when a user wants to share actions they’ve taken.
  • Feed is the center of communication on Facebook. Old: single stream, not interactive, only 25 slots. New: top stories as highlights, multiple streams of news, more room for application stories to appear, commenting on stories.
  • feed.publishTemplatizedAction: disaster, user-specific token sets made aggregation very difficult.
  • New methods: feed.registerTemplateBundle and feed.publishUserAction
  • Allow template bundles to include several templates per story size, ranging from user-specific to more general. You can define different templates within a bundle to handle 1, 2, and many aggregated stories.
  • Before you launch your application, think about the sort of templates you’re going to be using. Calling the API method only allows for one-line stories. Feed forms: FB.Integration.showFeedDialog()
  • Great communications = happy users. Respect users’ attention and their friends’ attention.
  • Higher acceptance, lower ignore = more requests allocated.
  • Notifications: user-to-user, can be sent to any non-friends who are also users of the app; app-to-user, allocation is approximately seven per week per user.
  • More read, fewer hidden / spam = more notifications allocated.


  • Allow access to News Feed? Nope.
  • Handle instead of a bundle ID? Not yet.
  • Facebook applications versus Facebook Connect? No differentiation.
  • (me) Disclose allocations per user instead of aggregate? Can put it on the short-term roadmap, but wary of disclosing who is a tattletale.
  • Statistics data available via an API call? Someone was working on that but he didn’t know the status of that work.
  • Set privacy in News Feeds? Not at this point but hopefully someday.
  • Timeline for new statistics to appear? In the next week or so.
  • Give visibility into an app’s spamminess? Talked about it with the Reviews application but have found some spamminess within the Reviews application itself.
  • What kind of history for allocation reductions? Spamminess metrics last about a month.
  • Expand News Feed on friends to see more of a story? Haven’t really thought it.
  • In stories themselves, possible to see how many times one was read in News Feed? Would like to, but not a high priority.
  • Add or view comments, available through API? Probably through fb:comments, not allow to add comments programmatically due to spam concerns.

Introducing the New Facebook Profile & More Session

July 23, 2008

Talk by Ruchi Sanghvi and Josh Elman. Here are my rough notes on the presentation:

  • The never-ending profile was the motivation behind the redesign.
  • Emphasizing feeds: increases engagement and encourage content creation
  • Simpler, cleaner profiles: easier profile navigation, clearer identity, more control
  • More control over profile: users decide about tabs, when to publish, and look of stories
  • “It all starts with the Wall”: feed, publisher, profile boxes
  • Stories: one-line, short, or full (up to 500×700). Done with feed forms, using FBML or Javascript.
  • Publisher: different versions for user and friends,
  • Info tab: deep integration with structured information, should represent information about what the user’s done. Info stuff is enabled through canvas page button using FBML or Javascript.
  • Tabs: provides the richest expression, hybrid of a profile box and canvas page (solely FBML, no advertising), no caching, 760 pixels wide, no autoplay, fb:visible-to-owner
  • Profile boxes: profile_main -> narrow, on Wall, wide and narrow appear on Boxes tab, Bookmarks will be migrated, must manually add a bookmark otherwise.
  • New permissions/lack of adding: reduces friction. First access provides user ID, friends, pic and names, publish feed stories, send requests. Add require_login to links to trigger permissions solicitation.


  • How do users first get into the app if they’re not adding?: About page much more static.
  • Should apps offer all integration points and let users decide or pick some?: Both, but mostly the latter.
  • Is user info going to be available to all? Still limited to privacy settings.
  • What can the Publisher do?: Rehashed already shown options
  • Smiley app? Uses shared preferences, which has been available for 9 months
  • Can you detect whether a user is in new profile? Yes, part of API.
  • Widened Wall, is it final? Yes.
  • (from me) Extended permissions, is the Wiki list definitive? Yes.
  • Engagement metrics, what are they? Bunch of them. User can reorder boxes.
  • What are these info sections? Not a mini-feed, opposite of one. More for static information. Button on canvas page shows example, the user allows it, adds it to profile, and edits inline.
  • What new stats will be available, users who have added tabs? Yes.
  • Logged in user for tabs is viewer or user? Viewer. Tabs focus should be on the user.

Off to F8

July 23, 2008

I’m getting ready to fly out to San Francisco for the F8 Conference put on by Facebook. We’re leaving Phoenix at 7:20 AM and returning to Sky Harbor by 10:30 PM, so it’s going to be a long day but very much worth it—I hope. I plan to blog my raw notes for each session throughout the conference for my colleagues that couldn’t attend as well as posterity.

Armchair Architects

May 24, 2008

In case you haven’t heard, Twitter‘s been having some alarming downtime due to scaling issues. It’s become quite popular and continues to grow significantly in both traffic and users every month. As more and more people come to rely on its unique service, these outages have grown increasingly frustrating and that has lead to a minor cottage industry in the blogosphere: complaining about Twitter and ponying up solutions to help them out of this situation.

So here’s what they need to do: shut up and realize that, by and large, they’ve got no idea what problems the Twitter team is having and no credibility in offering advice. Oh, you thought I was going to join in the chorus. While I have some experience with scaling in working in online banking and then a popular hosted blogging engine, I won’t pretend to have any special insight into the problem. Unfortunately, many of my fellow bloggers don’t share my restraint.

Just to give you an idea of the scope of the problem, Dare Obasanjo’s entry details some of the complications that become obvious after more than a superficial rumination. Some might say that attacking this is easy, but they’re dead wrong. What’s even worse, the application that the Twitter developers originally built wasn’t what it has become.

Second guessing and judging based on insufficient (or absolutely no) evidence is practically the coin of the blogosphere. Twitter needs to fix their problems, but I guarantee that their team cares more about doing so than you ever will.

[UPDATE (5/30/2008): More details.]


April 9, 2008

Google recently announced its AppEngine initiative and I can’t say I get who would want it. It strikes me as too inextricable from Google.

Amazon Web Services operates in a similar fashion but it is clearly serving as an infrastructure provider rather than a platform. While it’d be hard to migrate off of AWS if you ever chose to do that, it’s not as if you’re promoting Amazon by virtue of creating and running your application. At every turn, the AppEngine application uses Google products like Google Checkout and Google Accounts. Building a business so closely associated with the largest Internet company in the world strikes me as perilous.

AppEngine aspires to be a platform like Facebook has become. But it lacks the social aspects that make Facebook so attractive as an application platform. So, ultimately, I think AppEngine’s main competitor is not Amazon, Facebook, or even Microsoft (which has its own cloud initiative in development) but Ning. Who’s Ning? Exactly. I just don’t see this market as compelling so I don’t understand why Google’s entered it.

Rick, Rick, He’s Our Man

April 1, 2008

Over at Found on the Web, I’ve made my first April Fool’s joke. As you might expect, it’s Rick-Roll related.

Here’s the Javascript I used:

<script type="text/javascript" language="javascript">
var links = document.getElementsByTagName("a");
for (var i=0; i < links.length; i++)
links[i].onclick = function()
location.href="";return false;

It gets every hyperlink on the page, loops over them, and adds an onclick event handler that performs the redirect.