The fix in both cases? Simply copy the contents of the CD to the local harddisk (a network share works just as well) and then run setup.exe from there. Don’t be fooled by the suggested “solutions” from Microsoft or Installshield like I was.

sigh.

According to the official blog post, the new and improved Delicious brings us speed, usability, and oh so good looks among other features and it was a long time in the making. The Yahoo acquisition was announced on Dec 9, 2005 and the new site finally went live a little over two and a half years later on July 31, 2008. So why did it take them so long?

A key change as a result of the Yahoo! acquisition was their decision to rewrite the whole thing in PHP using the Symfony framework, for no other reason than that it’s the current corporate standard at Yahoo!. Oh, and, coincidentally, Yahoo! Bookmarks was also built on PHP+Symfony.

So now it starts to make a bit of sense: you take a system being actively used by millions of users around the world and you start over from scratch with the goal of building it bigger and better, toss in a couple of hot buzzwords to meet Web 2.0 compliance guidelines, and before you know it 2 years have gone by.

I find it very hard to believe that with all the talent and the thousands of man years combined software development experience over there, that no one understands the pros and cons of rewriting vs refactoring a code base, especially given the enormous success of the service and the relatively trouble-free history as compared to, say, Twitter.

At the same time, I understand it all too well. From where I sit, and having been involved in a similar situation in the past as well as with my current employer, the decision to move to PHP was clearly not based on a cost/benefit analysis of maintaing the current system. In fact, I wonder if they even understood what the real problems, if any, were with the existing system before deciding to not just rewrite it, but write it in another language.

Moving to another language is a pretty drastic step to take and will rarely solve your problem.

Lately I’ve been working on optimizing our stored procedure library which is primarily responsible for generating all sorts of fancy reports for the users. We use lots of nested procedure calls and finding potential targets for optimization is a tricky and time consuming job. Enter Ghetto Profile.

It’s still pretty basic but gets the job done. How’s it work?

1
2
3

ghetto_profile.rb --attach crappy_code.sql
mysql < crappy_code.sql
ghetto_profile.rb --stats -uroot -pmysql

The attach command simply wraps all stored procedure calls with debug statements which can later be analyzed using the stats command.

CALL debug.ON('stored_proc1'); -- added by ghetto_profile
    CALL stored_proc1(...params...);
CALL debug.off('stored_proc1'); -- added by ghetto_profile

And the output of the stats command is something like this:

total time: 3718.3220
 
stored_proc1
         pct:   75.3873
         total: 2803.1430 (46.72)
         avg:   0.0895
         count: 31327
         min:   0.0860000000000001
         max:   0.343
 
stored_proc2
         pct:   22.9479
         total: 853.2760 (14.22)
         avg:   0.0273
         count: 31240
         min:   0.0
         max:   0.371
 
stored_proc3
         pct:   1.1224
         total: 41.7340 (0.70)
         avg:   0.0007
         count: 59020
         min:   0.0
         max:   0.074

Wow, that stored_proc1 is taking up 75% of the time! Hacking ensues…

total time: 1151.3400
 
stored_proc1
         pct:   0.4372
         total: 5.0340 (0.08)
         avg:   0.0001
         count: 43892
         min:   0.0
         max:   0.007

Totally sweet! Ok, enough talk. Get some:

Ghetto Profile 0.7 (requires ruby, BSD licsensed)
debug.tgz (helper package to setup the debug database and procs)

Fortunately, that all appears to be changing. I was glad to learn that the recently announced Live Documents is made entirely by an Indian company based out of Bangalore (though it was founded by Sabeer Bahtia of Hotmail fame). I’d be curious how their product does in the Indian market, as opposed to the rest of the world, for which it seems to be primarily targeted.

In this regard management is also to blame, especially when it comes to dysfunctional schedules, wrong incentives, poor hiring, and demoralizing policies.

Where their earlier releases (S3, Elastic Compute Cloud [EC2], Flexible Payments, Mechanical Turk) commoditized much of the infrastructure required for building scalable applications, SimpleDB (SDB) and the earlier Simple Queue Service (SQS) are bringing cutting edge technologies and design patterns to the masses. First they made it cheap and easy to have a cluster; now they’ve made it cheap and easy to use a cluster! Amazing.

What’s even more startling is just how much Amazon gets it, and just how far off base Salesforce was earlier this year when they announced Force.com, as a “platform as a service”.

Then again, maybe they’re not even competing services at all.

Amazon is clearly providing services targeted towards developers and entrepreneurs with the goal of enabling them to explore new and innovative ideas by lowering the cost of entry. They’re providing the basic building blocks for developers to do exactly what they have done (and spent the last 10 years building) and they’re providing it at a very competitive price.

The Force.com proposition is different: they market to the same target audience but the selling point is not their tools — they offer a run of the mill JAVA-based platform — their selling point is the market that they can deliver. A built-in customer base of salesforce.com [enterprise] users and a marketplace for connecting those users with the applications they want.

But I digress. Amazon’s SimpleDB is an important, if small, step towards moving the web to column-oriented databases like Google’s BigTable, or the relatively unknown, opensource Hadoop project, now largely sponsored by Yahoo!.

What sticks out to me most, however, is the choice of name. It’s called “SimpleDB” and yet it’s neither a “database”, as most would understand it, nor is the concept of a column-oriented database “simple”; it requires an orthogonal way of thinking. What is clear, however, is that the choice of name was a very deliberate move by Amazon to market this technology to the masses. It will take some time for developers to come around and see the light, but when they do, we’re in for another huge advance in dynamic web applications.