Clustering
A few weeks ago I saw a guy (X-Ravin) on Overclock.net who had set up a cluster for folding. I asked him how his set up works, and he was kind enough to PM me and explain it very well:
Hey I know this was a few weeks ago, but I just saw where you were asking about my cluster setup. It’s very simple to setup a basic Beowulf cluster with old machines, as long as they can boot from the network. For folding, there really isn’t a whole lot to it. Notfred has a good guide to diskless folding, just google notfred. As for the physical setup, I have a master node with 2 NICs in it. One NIC connects to my home network which is on a basic 4 port Best Buy router. The other NIC connects to a big 24 port Netgear unmanaged switch. In Windows 7 I simply bridge the 2 NICs together so that the internet is shared. That sums up the hardware, now on to the software. On the master node I installed TFTP and configured it as a DHCP server with an IP range above my home network, but still in the same subnet. I then configured TFTP to run in PXE mode, and set it to send the pxe boot file from notfred’s package. Once this is all configured, I just start booting nodes one by one and watching them come online. Finally, since notfred’s distro has samba, you can use fahmon on any computer on your home network or the private cluster network.
Now this isn’t what I normally use the cluster for because folding isn’t a cluster application. Each node is simply acting as a individual computer. Using PelicanHPC Linux I use what is call the Message Passing Interface aka MPI to write and execute programs that use the cluster as a real cluster. Setting up Pelican is even easier, but using MPI can be a bit tougher unless you are familiar with programming in c,c++ or fortran. Anyways, to get a Pelican cluster up, you can use the same setup as above, but internet is optional. Download the latest PelicanHPC DVD and boot it. After it loads, it will list the command to build the cluster, then it will walk you through booting the nodes etc. That’s it! Now you can write and execute MPI programs with no setup. Quite handy, although setting up MPI in Ubuntu isn’t very difficult. It’s getting all the NFS and hosts stuff down that’s tough.
Hope that gives you a start!
I really wish I had a bunch of old computers so I could toy around with setting up a cluster. Maybe one day I’ll get around to it. X-Ravin, thank you for the very thorough explanation.
X-Ravin’s Blog – He does some really amazing stuff. Half of it is completely over my head, but it’s fun to read and attempt to understand, haha.
No comments yet.