Ok, so part of why I'm hating the locking behaviour of MySQL is because I'm playing with scheduling a large distributed job for a personal project. I've talked to some folk about Beowulf, and it doesn't seem to offer me much... Does anyone have any recommended reading on how research clusters solve this sort of problem that they would like to share in the comments?
Tags for this post: blog(
)
posted at: 14:12 | path: /diary | permanent link to this entry
Chris Samuel
From my perspective HPC clusters are doing batch computing with a queueing system (Torque, a well maintained open source version of PBS, in our case) and a scheduler that allocates jobs to nodes (Moab for us, but its free sibling Maui scheduler from ClusterResources is very capable).
For filesystems, well we still use NFS (quick, easy, works everywhere, probably not ideal) but others use things like Lustre, GPFS, CXFS, etc, with distributed lock managers, etc.
It all depends on what you are trying to do really - you may be best off approaching Stewart to see if he's got any bright ideas, given he works for MySQL now he should have some.. :-)