I took part earlier this week in the Amazon Web Service update session organised in London by Amazon. The objective of this session was to give an overview of the new features available in AWS... and, believe me, there are quite a lot.



It was actually fairly impressive to see the sheer amount of innovations that are being rolled out on a monthly basis (at least 10 new features/improvements every month): the platform is being actively expanded and this is definitely part of a strategy to spread the competition thin by offering many functionalities to grab market shares (based on the number of participants in the audience, it looks like it is working well so far...). Let's review of few of the key elements presented during the meeting.



Glacier

Glacier is the newest service added to the "storage suite" as a complement to S3 and EBS. The current positioning of these different solutions is the following:



  • EBS (Elastic Block Storage) offers block level storage volumes to be mounted on EC2 instances, with fast data access, high availability but no guaranteed durability
  • S3 offers object storage, with fast access and very durable, typically used in web applications to store videos, css, static HTML pages etc.
  • Glacier is similar to S3 but offers an extremely durable solution (11 9 durability i.e. 99.99999999999%) but with slow retrieval time (3 to 5h through an asynchronous access). The typical use case for Glacier is backup and archiving, knowing that the price point is very interesting when compared to the other storage solutions but also to traditional tape archiving: $0.01 / GB / month. An S3 integration should be rolled-out soon to allow for instance to move objects from S3 to Glacier automatically based on a set of rules.




As it is clear in the description above, the palette of services keeps on expanding to propose more specialist solutions with different trade-off points and associated pricing strategies to make them appealing to a larger set of clients. Designing a complex application leveraging the cloud definitely becomes a more advanced science, the reward being a better cost efficiency.

CloudSearch

Available since April 2012, CloudSearch is a search engine "in the cloud" with near real-time indexing that allows to add a search service to any website built on top of AWS. Its most interesting feature is built-in auto-scaling that will automatically add more search instances if traffic increases (and decrease their number afterwards) or/and expand the storage based on the volume of data that is searchable. This service is still a standalone service i.e. you have to move data "into" CloudSearch" make it searchable (instead of, for instance, tagging existing data in S3 as "search-enabled";).

A progressive price drop is planned for this service (already -20% since the launch), which is a common pattern for AWS services when adoption and scale increase.

Storage improvements

Various storage improvements were also announced to offer more options to architects, with a focus on performance-sensitive applications. It is worth mentioning:



  • Provisionned IOPS: a solution to get consistent, predictable performance for i/o intensive tasks - it is now possible to have consistently up to 2,000 IOPS on an EBS block for instance
  • EBS optimised instances which have dedicated throughput with the EC2 instance to which they are attached, which is typically interesting for database traffic
  • High I/O instances with a compute capability of 35 ECU and 60+ GB of RAM, as well as SSD disks. Netflix was presented as a new user of these instances and they were able to get rid of a full memcached caching layer responsible for maintaining a good throughput... but with only half the price!




Elastic Beanstalk

Beanstalk is Amazon's PaaS one-click platform that includes some interesting auto-scaling features: the objective here is to provide a fully working stack to deploy applications without having to worry about any infrLSE: IBM_blank/emastructure or middleware. New containers are now available for Python and Ruby (on top of the ones already in place for .NET, Java and pHp). RDS extensions are available to allow automatic database schema generation from the object model, which is a sought-after feature of these lightweight web frameworks. As adoption increases, a set of more advanced hooks into the containers were announced to progressively give more control over the PaaS containers.

Reserved instances marketplace

Reserved instances are an existing mechanism available to reduce cost of compute through capacity pre-buying. One of the challenges for clients can be to anticipate correctly capacity requirements at a longer horizon (e.g. 3 years): it is now possible 9 months within a 3-years contract to resell pre-bought capacity to other AWS clients, with total liberty to set the price (knowing that there is a 12% commission fee going to Amazon...). Many specialist have commented already that this is the "future" of Cloud computing: a fully flexible and liquid compute market place where users can trade capacity.

To conclude

Overall, many new exciting features in this update and, when contemplating the richness of the platform, I believe it is fair to consider Amazon as a "supermarket of cloud": no matter what you need to do, there should something in there to help you out, the main challenge being to make sure you are buying the most cost-effective solution for your problem.

Based on a lot of the questions asked by the audience, it was interesting to see that there is now an upgrade challenge to the platform as well: many users have built applications on AWS in the last years and new features are sometimes not always "simply activable": a migration is for instance required when upgrading from an existing EBS storage to an optimised EBS storage. When you have an application live in the Cloud with live client data, leveraging new features can still be challenging and each improvement comes with its specific upgrade strategy. The good news is that it should be relatively easy to test it out on a mirror cloud infrastructure and turn everything off afterwards...
Andre Nedelcoux