Wednesday, November 23, 2005

SETI@home Will be Killed On Dec 15th, Will get a new life at BOINC

The well known Distributed computing experiment SETI@home will be switched off on December 15. Don't get alarmed, it will still be actiove under the BOINC (Berkeley Open Infrastructure for Network Computing) and continue to do the work. So if you are a member, you should have gotten an email by now with instructions as how to rejoin the project under BOINC. I know the inquirer reported as "SETI@home killed off ". They always try to be dramatic, and their acting is quite good.
I am a member of two BOINC projects;
Einstein@home: search for gravitational signals emitted by pulsars
World Community Grid: advance our knowledge of human disease. (Requires 5.2.1 or greater)
So Come join us, Find an alien, no not that type, Extraterrestrial Intelligence type!

Below are some parts of the technical news leading to the switch over;
November 22, 2005 - 21:30 UTC
We began sending out the mass e-mail yesterday warning SETI@home classic users that we are going to close down the old project on December 15th. It was sent to all 200,000 of the active classic users by this morning. Inactive classic users are being e-mailed at this point.

Due to the influx of new BOINC users (and the unfortunate timing of some googlebots and other web spiders) the load on our web server was extremely high for the past 12 hours. To fix this, we finally deployed a second web server to split the load. As DNS updates spread throughout the internet, the load on klaatu (the original single web server) decreases while the load on penguin (the new secondary web server) increases. Both are Sun D220R's (2 x 440MHz Sparc, 2 GB RAM).

Part of the problem was that the web servers were configured to spawn more many clients than actually necessary, which left lingering, unused threads open on the database, which in turn lead to the database running out of connections. Some users saw messages to this effect when the load on the web servers was at its highest.

This looked like a database problem, when in fact we are currently enjoying a 10% performance boost on the database. Last week we moved some memory off the myisam tables (which contain web forum info and not much else) and slated it for the innodb tables (which contain user, host, result, workunit, etc. tables). The myisam tables didn't need the excess memory.

By the way, the master database merge (see below) is currently on for the beginning of next week.

November 15, 2005 - 21:00 UTC
(updated 22:45 UTC - see addendum below)

Today we started the big master database merge. This step is simple in essence: we are combining all the scientific data from SETI@home classic and SETI@home/BOINC into one big database. However, this is the culmination of many months of effort.

What happened during those months? Among other things, we had to migrate all the data off of one server onto another, find and remove redundant data, add new fields to old records and populate them, write and test software to merge databases while keeping all relational constraints intact... Basically a lot of cleanup, a lot of testing, and backing up the entire set of databases between every major step.
October 10, 2005 - 23:30 UTC
Last week we finished the first phase of the science "migration." What this means is that all of the scientific results from SETI@home classic have been migrated over to the BOINC science database. There is a bit of cleanup work to be done, but the next big phase is merging these data with BOINC data. We will soon be able to shut down the classic master science database, which is currently running on the same server as the BOINC scheduler.

No comments: