Ragged Clouds

cloudConfusionThis post is about technology platforms and my horrendous confusion with them. If your interests lie elsewhere please stop reading this post right now.

Cloud platforms and related technology have great promise. I have now taken fairly detailed look on these and truth is not so clear cut. For some applications they are great, but for others they are far from ideal. Especially databases and transactionality in volatile cloud infrastucture makes my brain ache way too much. Things will certainly improve in future, but my need is for today.

Some of alphabet soup I have recently studied is listed below. I certainly enjoyed studying all this as my former experience has all been with bare metal hardware or standard virtualization stuff.

Aws, Azure, DbShard, Dynamo, Cloudfront, Ebs, Ec2, ElastiCache, Elb, Federated Sql, Joyent, Heroku, Memcached, NoSql, OpenStack, Rackspace, Rds, Rightscale, S3, Sharding, Swf, Sqs, Xeround.

Promise of horizontal scalability with cloud infrastructure seemed ideal and very attractive to me. Tried to map my plans to this model but soon realized that this project does not map so well. This project requires multi-row datastore transactionality. Potentially very CPU and memory intensive app servers might be required. I could implement these on top of current clouds but this would be a major effort and my time is better spent elsewhere. There are providers attempting to improve the situation, but they are not quite yet there in my opinion.

The current infrastucture plan is fairly simple:

  • Host with Amazon Web Services for now. They are most advanced and provide many possible growth trajectories. Azure looked interesting but their max RAM limits are lower plus they are still somewhat behind functionality-wise.
  • Front end servers will be standard EC2 instances behind load balancer, hosting ASP.NET web site apps.
  • App servers will be bigger EC2 instances behind their own load balancer, or possibly communicating via message queues. If appservers require more RAM than currently available within aws will just externalize these server to custom built high-end hardware hosted elsewhere. This is not perfect solution and changes to this might become necessary, but for now am thinking this way.
  • For now am going 100% with RDBMS model. Debated between MySQL variants and SQL Server a bit, SQL Server won as I have slight preference to that over MySQL. Will likely start with Amazon RDS hosted DB instance and migrate to better system if needed later on. This is also not a perfect solution and changes to this might become necessary, but for now I am thinking this way.

After doing some early prototypes and test runs it is clear that the above model would certainly be sufficient to several million users and likely beyond.

Leave a Reply

Your email address will not be published. Required fields are marked *