Archive for the ‘Technology’ Category

SQLcached

November 10, 2008

I have to agree with Dare Obasanjo’s latest blog entry about in-memory caching. After working on high-transaction, heavy database-using Web applications for the last nine years, there is one thing above all else that I have learned and taken to heart: a Web application is only as good as its caching strategy. My career has seen a progression from light to heavy cache usage and each new application has benefitted in scalability from that.

Dare’s entry got me thinking: why couldn’t the RDBMS itself incorporate a distributed, in-memory cache like memcached or Project Velocity? What if a Web application could basically eliminate the need for its own caching layer by relying solely on the database, which would then aggressively and algorithmically use one of the caching services to expand its memory-based caching?

If the problem with query caching in MySQL or SQL Server is the amount of server RAM that can be installed, then distributed caching seems like the perfect solution. It’s what the Web server layer uses: why not bring it down to the data layer. Moreover, given the common replication and clustering scenarios, there are likely idle database servers whose memory is already going unused for the most part. Putting a distributed caching system in place would put them in action while still keeping them ready for failovers.

The main objections I can see is that going to the database might cause an increase in network usage since some cache calls in the Web server layer would never leave the server and that the database would have to work to decide between file-level and cache-level access. But that would be minimal and the simplification it would engender on the Web application level would make the costs even less objectionable.

It’s entirely possible that Project Velocity is being undertaken with exactly this thought in mind. (It’s not clear that there’s any movement afoot in MySQL AB towards this end—at least from my cursory searches.) This idea would have to be implemented at the RDBMS level.

Advertisements

Thoughts on Android

September 30, 2008

Last week saw the introduction of the first Android phone, the T-Mobile G1. I’ve been following Android’s progress with interest because it seems to be the most compelling competitor to Apple’s iPhone so far.

This video by Engadget really helped me to understand the phone and operating system in a way that all of the specs and press releases have not. This particular video was better than most of the other ones I’ve come across because the phone’s operator was quite familiar with its features.

Here are the things from the video that I really liked:

  • That little drop-down panel notification that appears and disappears after a few seconds. It also appears to be able to be recalled at any time. It’s especially handy for background processes and applications, neither of which are possible on the iPhone.
  • The compass rose on the Google Maps application. This is a third-party integration, like a plugin or Greasemonkey script, that provides additional functionality not originally conceived by the app developers. This sort of customization is impossible on the iPhone and could be the basis for a much richer experience.
  • The Street View responds to movement on all axes by changing the view accordingly. This is pretty sophisticated positional analysis. Like the compass rose, it appears that Android can tell the application not only the phone’s coordinates but also its orientation on all axes. That’s not available on the iPhone and could be very useful.

That being said, I believe that Android is doomed to failure. First, it has forsaken multitouch ubiquity. After the pioneering efforts of Jeff Han, Apple and Microsoft have clearly embraced multitouch as the user interface of the future. By not requiring hardware manufacturers to support multitouch (or touch at all, really), Google has seriously limited application developers. If a developer wants to do a multitouch application once Android supports that, he is either limited to a subset of the customer base or he has to make it degrade gracefully on phones that don’t support multitouch—neither is an appealing option.

And openness has proven time and again to not be a huge selling point to the average consumer. There are already open mobile operating systems but people are clamoring for iPhones. There’s a certain abstract benefit to openness that is hard to communicate to users. The freetards may whine but the average person just looks at the iPhone and drools. They don’t particularly care that their phone isn’t open. Why? Because most phones in the past never were.

I predict that Android will linger forever on cheaper phones where a free operating system could make for increased profits. The Big Three—Apple, Microsoft, and Nokia—will barely notice its share and Google will mostly abandon the project due to its corporate ADD.

What to Expect That I’m Expecting

June 5, 2008

This iPhone-related patent application just surfaced today. Combined with the rumor that Apple will unveil the iPhone App Store at the Worldwide Developer’s Conference starting on Monday, it offers some tantalizing insights into what we might expect from the normally secretive company.

Here’s two excerpts from the patent application:

The device supports a variety of applications, such as one or more of the following: a telephone application, a video conferencing application, an e-mail application, an instant messaging application, a blogging application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and/or a digital video player application.

And also:

The wireless communication may use any of a plurality of communications standards, protocols and technologies, including but not limited to … voice over Internet Protocol (VoIP), Wi-MAX, … instant messaging (e.g., extensible messaging and presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), and/or Instant Messaging and Presence Service (IMPS)), and/or Short Message Service (SMS)), or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document.

Video conferencing? IM? Blogging? Digital video camera? SIP/VoIP? XMPP? These are all technologies that aren’t in the current iPhone. So the question here is whether this is a carte blanche, catch-all sort of patent application or a revelation of Apple’s upcoming intentions.

I am very conflicted about this report. On the one hand, having these applications built by the mothership means that they’ll be more widely used and available. For applications like instant messaging and video conferencing, ubiquity makes them useful. On the other hand, by including them for free Apple would short-circuit developers interested in selling such applications at a time when there are exactly zero sanctioned, third-party applications. This, of course, is nothing new for Apple.

It’s within their rights to offer whatever software they want on whatever platform they create. But if they’re looking to grow their third-party application catalog (and make money off the App Store), they’d do well to emulate the Facebook platform or even their own operating system. Competition with their constellation of developers has a chilling effect and should only be engaged in rarely.

[UPDATE (6/9/2008): Apple appears to not be competing with iPhone developers. From the keynote, it looks like they’re just adding features to their existing stable of apps and leaving IM et al. to third parties. I think it’s the right decision.]

Still Missing

May 29, 2008

Jim’s comment on my screed about Netflix reminded me that I left an important requirement out. “All I know is that I want to be able to watch streaming video from the network sites on my television” doesn’t fully capture what I’m looking for in the perfect Web-enabled TV assistant box.

I also need to rip every one of my DVDs to disk and be able to watch them on my television. Whereas I believe Jim’s frustration stems from aesthetic or possibly browsing concerns, mine arises out of the total obliviousness to proper DVD handling and care that my three daughters exhibit. Every disc they touch is smudgy and scratched. If I could digitize the movies before they got to them, they could enjoy skip-free entertainment and I wouldn’t have to repeatedly buy new copies. (I’ve only done that twice and I hated to do it.)

Being able to stream from another computer or server would adequately cover this need. I have no problem attaching an external hard drive to one of my Macs to accomplish this. I believe that existing devices already address this issue, but again they lack the Web-level streaming that I’d like.

Missed It By That Much

May 21, 2008

I’m a big fan of Netflix. One of the few reasons why I even bothered with dual booting via Boot Camp was so that I could “Watch Now.” So purchasing their new Netflix Player would seem like an easy decision. But it’s not.

This player is really just the opening salvo for what promises to be a protracted battle. The price is compelling, but there’s something missing. I can’t decide if it’s because it’s too tied in to Netflix or that it’s too limited in content—two sides of the same coin.

All I know is that I want to be able to watch streaming video from the network sites on my television. And I want to watch the Netflix instant video as well. At this point, the option that fits my requirements is the HTPC but I don’t want the expense, unsightliness, and noise of a full-fledged computer in my living room. The Netflix Player and the AppleTV are the perfect size and form, but they’re not powerful enough.

Unfortunately, I think I’m going to have to pass on the Netflix Player. Maybe version 2 will wow me.

Memcached Developments

April 24, 2008

Today I discovered that there’s a new port of memcached for Windows! This one reflects memcached v1.2.4, which added a bunch of new features like multi-get, append/prepend, and check and set. This is a huge release even though it’s in beta and is the first pass from the new maintainer. I am especially happy because I was considering putting in some time to do the port myself; I wasn’t looking forward to it since I’d have to learn C++ and it’d take me awhile.

I also found the BeIT memcached client library, which purports to improve on Enyim. I haven’t had a chance to try it out yet, but I like the embeddable, lightweight nature of its code. It supports a few commands that Enyim doesn’t so I’m going to give it a try. Enyim had a couple quirks that we found and worked around so I’m always willing to find a more straightforward framework if possible. I don’t understand the hashing issues enough to ascertain which is using the better-performing or more consistent algorithm so I don’t have a preference on that front.

All in all, these were two very welcome developments. It’s never been a better time to be a memcached fan in the .NET world!

A Future of Architecture

March 28, 2008

Tonight I attended a lecture on “Algorithmic Architecture” by Kostas Terzidis at Taliesin West. Terzidis, a Harvard professor and former software engineer, is interested in the intersection of architecture and technology—mainly the role that computers play in automating much of the design aspects of architecture.

He spent much of his lecture philosophizing about this crossroads we find ourselves approaching. In contradictory fashion, he opined that there is nothing new, only recycled ideas and forgotten truths, while arguing that computers have enabled us to realize designs that were impossible just decades ago. That increase reliance on computer power has divorced us from the practice of architecture while bubbling over the inevitable supplanting of the creative process. Honestly, it wasn’t terribly different from some of the incoherent rhapsodies of Frank Lloyd Wright.

The idea of automating the creative process intrigued me greatly though. At first, I was troubled by the mechanization of design. On the surface, it seems to obviate the individualistic architect, the creator of greatness. But upon pondering it more, I think that there’s no reason why the stamp of the architect has to go away.

The future of architecture as programming is one where a program is written that takes all the variables and constraints of the architectural problem and devises thousands of buildings that solve it. Site topography, materials, budget, number of rooms, central plant, and so on are just inputs to be considered. While this seemingly diminishes the role of the architect, I think it could easily maintain his part.

Perhaps each programmer qua architect would codify his aesthetic sense in different ways or assign different weights to the various considerations. In so doing, the designs his program generates might look all of a same piece and his particular theme shines through. His job, then, becomes one of practiced selection of the best design combination.

It’s going to be awhile before such a transformation of the trade takes place due to the massive increase in computing power necessary to effect it. But it seems inevitable and inexorable.

The Shotgun Approach to Recommendations

March 27, 2008

Netflix recommendation

I went to Netflix just now to rate a movie I watched. On the front page, the image at right greeted me.

You may or may not know, but Netflix is offering a $1 million prize for a 10% improvement in its recommendation algorithm. I honestly don’t know if the recommendation at right is what they’re trying to improve but if it is then 10% better shouldn’t be too difficult.

I mean, because I liked a documentary about capital punishment, a 80s puppet children’s show, and my children liked an insipid phonics teaching program they think that I’d really dig a pseudo-martial arts exercise video. Sorry, but WTF?!

They just completed a big ol’ upgrade but I think they missed a spot. Yeah, over there. See that recommendation engine-sized dark spot over there. There you go, you’ve found it.

VoIP Follow Up

October 2, 2006

Last May, I discussed my voice over IP experience. A lot has happened since then. Actually, a lot has happened in the last two weeks.

I finally activated my Packet8 account and promptly started the switch to Vonage. I can’t believe how horrible Packet8 service was. First, you have to dial a 1 before every phone number but the caller ID doesn’t automatically prepend that 1. That makes caller ID as an easy redial impossible. So you instead have to remember the area code plus phone number and then re-enter it in. Nowhere on Packet8’s site does it say anything about that, but Vonage addresses that issue in its FAQ.

Second, there were times when you couldn’t dial any phone number in particular area codes. We live in 480 and my in-laws live in 602. Occasionally, my wife would call her family and it would just ring and ring. No voice mail pick up at all. She didn’t think anything of it until she tried number after number. A quick call on her cell phone revealed that the problem was indeed with Packet8. How the hell is that any kind of service!

All in all, Packet8 made us long for the days of AT&T CallVantage. Sure, you had to reset the telephone adapter almost on a daily basis and there was the occasional dropped call if you didn’t answer call waiting fast enough, but at least you could *always* make calls. I had high hopes for Packet8 since they seemed more technical and hacker-friendly. But I really think that they don’t deliver on that promise and their web site is a joke compared to AT&T’s.

We’ve had Vonage for a little over a week now and I deeply regret ever having listened to the asshole that said I shouldn’t switch to them in the first place. He definitely wasn’t worth the respect I gave him at the time. It has been a joy: the web site is comprehensive and the development seems ongoing. We’ve had no problems with the service and it connected us to 911 perfectly when we needed that recently.

Slow as Government

March 22, 2006

What?! Windows Vista is delayed again? This time past the invaluable Christmas buying spree? The hell you say! It just can’t be because XP was released in October 2001 and no major operating system developer would go 5½ years between releases, would they?