Setting up AWS Cluster to use snow in R

Setting up AWS Cluster

I wanted to setup an AWS cluster to take a shot at a Kaggle contest – DunnHumby Challenge

For this, I found StarCluster to be of great help. It allows you to set-up AWS nodes in a few lines of code and does much more (choosing AMIs and cluster configurations)

Make sure you use the Bioconductor AMI which comes bundled with R and a host of installed packages.

I used the package “snowfall” for parallel processing.

Relevant SO questions I had asked

About indiacrunchin
Any sufficiently advanced technology is indistinguishable from magic -- Arthur C. Clarke

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

%d bloggers like this: