Amazon Web Services: This Is Why Chicks Dig Vlad

Warning to the opposite sex: Reading this blog post may (0.000% chance) affect your sexual desire for me. You’ve been warned.

Have you ever said something to a girl and just had her eyes roll straight back in her head? Let’s face it, with an engineering degree, MCSE, MVP and few other credentials my sexappeal from the opposite sex is very far in the red to say the least…. but one of my motos is “the only way you can make sure you’re not a complete failure is to keep on trying to take things to a new low with each passing day” so here goes what I believe will be the post that absolutely eliminates even the last of my female audience. Libido, watch out!

Earlier this year I was very excited at the invitation to attend the MVP summit. Contrary to the popular belief, there was very little koolade there and the types of conversations that went on would blow any geeks mind. It is what I imagine the life in the Silicon Valley was like during the .com boom and now the Web 2.0 on the rise, 24/7 discussions about technology and curiosity about technology I otherwise would dismiss outright. So one night at the bar we’re talking about AWS (this is obviously outside of my Exchange MVP hive).

I explained to the guy what we do, and he explained to me what they do with this new kids web test app that they are developing. Just like any other regional application they have their peak times during which both the network, the CPU and everything related to it spike through the roof. Traditionally, this is where you’d go into designing high capacity clusters, load balancing hardware and switches, basically bend over and bite the infrastructure pillow cause the datacenter quote you’re about to get is going to hurt some place really personal – your wallet. So we keep on talking, really all angles of it and he tells me about the Amazon Web Services and how that whole thing works (for the sake of the story just assume that the technical limitations didn’t work out for his app); Here is the concept:

“Amazon has this concept of Elastic Clusters, where you basically lease time and space on their clusters. There are two deliverables – the virtual machine and storage/network. You pay for the number of hours a virtual machine runs plus any storage and network you consume. You can configure as many virtual machines as you want and you can also spin up as many virtual machines as you want.

So the concept is pretty simple. I have a 4 hour block during which my demand skyrockets. During the other 20 hours of the day the activity is minimal at best. I can scale up the service on demand by firing up as many virtual machines as I need during peak times and only paying for that 4 hour block, at 10 cents per hour per virtual machine! Network is about $0.18 per gigabyte and storage is around that price as well.” – Some Unmemorable .NET MVP

Yes ladies, thats the sexy stuff right there (are they gone yet?) that makes the virtualization and resource availability a totally scalable variable cost that can grow and shrink on demand. Starting a new web app – your costs are minimal. Your application popularity explodes, WSJ reviews it – you add resources on demand and just pay for the usage. Your application gets a fatal bug, competitor moves in, etc – you scale back down without a huge infrastructure to pay off. Brilliant!

So how does one actually do this?

Ok, math first. 10 cents an hour for a vm running 24/7 is roughly $80 a month. What’s in the box? Each VM is supposedly an X86 equivalent of 1.7 GHz processor, 1.7 GB ram, 160GB local disk and 250Mb/sec local network. There is a video out there but let me walk you through standing up your CentOS image with Windows:

First, sign up for AWS and EC2. Download and install Java runtime environment.

Create a new directory, C:\EC2 and download the X509 certificates and personal certificates from AWS site to this directory. Download the Amazon EC2 API and extract bin and lib directories to the C:\EC2 directory. Now, create the following batch file:

echo off
set EC2_HOME=C:\EC2
set PATH=%PATH%;EC2_HOME%\bin
set EC2_PRIVATE_KEY=C:\EC2\PrivateKey.pem
set EC2_CERT=C:\EC2\509Certificate.pem
set JAVA_HOME=C:\Program Files\Java\jre1.6.0
“%JAVA_HOME\bin\java” -version

Go to the command prompt, execute the batch file and if all went well you should see the JRE version info.  Time to get started, first find the image you want to roll out:

ec2–describe-images -x all

IMAGE   ami-febd5897    khaz_fc4_dev_fuse_alfresco/image.manifest.xml   602961847481    available   public
IMAGE   ami-a38b6eca    amis.winelibrary.com/rails_svn.manifest.xml     609234845463    available   public
IMAGE   ami-40e50029    ezNetllcBucket/ezimage1.manifest.xml    610305670733    available       public
IMAGE   ami-268f6a4f    rightscale-images/CentOS5V1.img.manifest.xml    635201719205    available   public
IMAGE   ami-2c8f6a45    rightscale-images/FC6V2.img.manifest.xml        635201719205    available   public
IMAGE   ami-9a9e7bf3    rightscale-images/CentOS5V1_6.img.manifest.xml  635201719205    available   public
IMAGE   ami-08806561    steveodom_ec2_images/Apr022007.manifest.xml     844412190991    available   public
IMAGE   ami-8db95ce4    icn-public-ec2/debian.3-1.img.manifest.xml      845261616107    available   public
IMAGE   ami-c69471af    icn-public-ec2/fedora7-i386.img.manifest.xml    845261616107    available   public
IMAGE   ami-de9c79b7    bals/ec2/drupal/image.manifest.xml      875251158881    available       public
IMAGE   ami-be8f6ad7    mybucket-rajesh2/my-image-1.fs.manifest.xml     876813118668    available   public
IMAGE   ami-099c7960    strategoit/ec2-images/fedora7-i386.img.manifest.xml     945771494425    available       public
IMAGE   ami-85997cec    mt4/image.manifest.xml  959411691118    available       public
IMAGE   ami-83886dea    ami-jb/image_bundles/loadtester/image.manifest.xml      971351813114    available       public
IMAGE   ami-a69471cf    workspace.globus.org/hello1.manifest.xml        971725951873    available   public
IMAGE   ami-34ba5f5d    neotactics-images/fc4-base-cryptofs.manifest.xml        978951556539    available       public
IMAGE   ami-0b967362    sitemason-pub/slackware-11.0.manifest.xml       986331161080    available   public
IMAGE   ami-20b65349    ec2-public-images/fedora-core4-base.manifest.xml        amazon  available   public
IMAGE   ami-22b6534b    ec2-public-images/fedora-core4-mysql.manifest.xml       amazon  available   public
IMAGE   ami-23b6534a    ec2-public-images/fedora-core4-apache.manifest.xml      amazon  available   public
IMAGE   ami-25b6534c    ec2-public-images/fedora-core4-apache-mysql.manifest.xml        amazon  available       public
IMAGE   ami-26b6534f    ec2-public-images/developer-image.manifest.xml  amazon  available       public
IMAGE   ami-2bb65342    ec2-public-images/getting-started.manifest.xml  amazon  available       public
IMAGE   ami-bd9d78d4    ec2-public-images/demo-paid-AMI.manifest.xml    amazon  available       public  A79EC0DB

These are the public images available for rollout. They are basically shell deployments of Linux distributions that you can model into your own VM with your requirements and demands. So, lets create a certificate for SSH (certificate authentication)

ec2–add-keypair vlad-keypair

This simply creates a certificate for me to login to the virtual machine. Now, let me pick a virtual machine to use and tell it to use my certificate:

ec2–run-instances ami-9a9e7bf3 -k vlad-keypair

All set. This is where the AWS prostitute clock starts ticking, $0.10 per hour folks. This can take a few minutes because right now that instance is firing up (ie, the virtual machine is starting) so time for me to open up some inbound ports. I’m opening up SSH, http/https, ftp and some magical port for vladnet stuff.

ec2–authorize default -p 21

ec2–authorize default -p 22

ec2–authorize default -p 80

ec2–authorize default -p 443

ec2–authorize default -p 3425

Now to check on my virtual machine….

ec2–describe-instances

RESERVATION     r-0b907162      343618739966    default
INSTANCE        i-ec6d8f85      ami-9a9e7bf3                    pending         0

Notice that the vm is still starting up (pending); according to Amazon this can take up to 10 minutes. But when it does launch I will be given my URL to access this site via SSH.

INSTANCE        i-ec6d8f85      ami-9a9e7bf3    ec2-72-44-49-248.z-1.compute-1.amazonaws.com    domU-12-31-36-00-24-E4.z-1.compute-1.internal   running           0

And we’re up, my hostname as indicated above is: ec2-72-44-49-248.z-1.compute-1.amazonaws.com

The cool thing about ec2–run-instances is that I could have passed it a count parameter, firing up a number of these VMs, literally as many as I needed and could pay $0.10 per server per hour to run.

Finally, shut it down:

ec2–terminate-instances i-a0688ac9

RESERVATION     r-0b907162      343618739966    default
INSTANCE        i-ec6d8f85      ami-9a9e7bf3    ec2-72-44-49-248.z-1.compute-1.amazonaws.com    domU-12-31-36-00-24-E4.z-1.compute-1.internal   shutting-down           0

Ok, so I know you’re sitting there thinking. Ok, WTF, who cares. Well, here is a beauty of all this. Let’s imagine you had a web app that for one reason or another had 4 hours a day where the demand was just through the roof. You could either purchase 8 servers from Own Web Now and run your infrastructure on top of that or you could get the Elastic Clusters from Amazon. Here is some math for you.

Eight servers, running for four hours a day, for five days a week for four weeks a month comes out to a total monthly expenditure of $64.00

Thats sixty four dollars a month.

Now of course on top of that you have the bandwidth and storage fees but you’d have those costs everywhere else. This is huge in terms of software availability and the whole SaaS pricing because the cost of hardware and infrastructure invesment is pretty much 0. So is the infrastructure consulting and product evaluation, it makes it possible to have your entire infrastructure designed in an afternoon and be in business by the evening. And all the costs are directly proportional to your popularity which better be exponentially proportional to your revenues. How amazing is that?

So… Ladies? Hello? Anyone awake?

Mind you, compared to the dev work I do on ExhangeDefender and Shockey Monkey, this goes into the “exciting” category. And to think that Katie was concerned having me on the road all the time with all the hot IT women all over me… with me turning up my charm… talking about elastic clusters… scalable infrastructure… SaaS…

Note: No, I am not kidding. No, thats not really me but if you squint.. (well, really, if you close your eyes and run at a wall its pretty close) I am happily married, not taking any responsibility for the sexual attraction that may result from you reading this blog post.

“No baby, thats not a roll of quarters in my pocket. Yes I’m happy to see you but thats a roll of $100’s I’m pulling in since I moved my web app to elastic clusters. Yeeeeeeeaaaaaaaaaaaah.”’