It has been about 6 months since I last blogged about work, so I figured an update may be in order, especially since today CohesiveFT announced availability of VPN-Cubed on Flexiant’s cloud offerings.

We’ve been very busy on VPN-Cubed engineering side. Along with features already on the roadmap, we delivered several enhancements that were directly driven or requested by VPN-Cubed users. On the product support side, we continued to expand a range of devices with which VPN-Cubed can do IPsec interop, which now include even ones I personally have never heard about before. We grew our experience and expertise in the area of troubleshooting intra-cloud and cloud-to-datacenter connectivity issues (there are many!). We’ve also worked on a few projects that required non-trivial topologies or interconnects, successfully mapping customer requirements to VPN-Cubed capabilities.

One theme that I have had in my head for some time now, is VPN-Cubed as the networking fabric of the Intercloud. Let me explain.

VPS was a predecessor of modern IaaS clouds. In VPS land, boxes are usually provisioned individually, one by one. Typical setups in VPS consisted of 1, 2 or 3 boxes. Networking 3 independent boxes together is relatively straightforward.

At the beginning of IaaS era, I imagine most setups were also 1 or 2 boxes. But as IaaS is gaining ground, topologies headed to the cloud are getting bigger, more complex and more dependent on access to external resources. Setting up networking consistently is becoming a bigger deal. But it’s not the end.

One of the roles of Intercloud is providing customers with an alternative (competition, in other words) – if one doesn’t like cloud A, she may take entire topology to cloud B. I’d say 99 of 100 public cloud justification documents being submitted to CIOs worldwide today include a statement saying something like this: “If this cloud provider fails to deliver what we need at a level we need it, we will switch to another provider.” This is actually not as easy in practice as it may sound.

Each cloud’s networking has unique aspects, no two are alike. Public IPs, private IPs, dynamic or not, customer assignable or not, eth0 private or public, cloud-provided firewall exists or not, peculiarities of firewall – these are some of the differences (as of today, I have set up boxes in 6 IaaS clouds with admin endpoints facing public Internet – I have seen many network setups). Taking images of N boxes from one cloud and dropping them in another cloud is well understood, recreating one cloud’s networking in another cloud is where the challenge is.

It is here where I think VPN-Cubed shines as a customer-controlled network abstraction – it’s an overlay built on top of service provider’s network, which allows it to be identical no matter what the underlying infrastructure looks like.

Same story plays out when an application is hyper-distributed and runs in multiple clouds or multiple regions of one cloud (where regions are connected via public Internet). And here as well VPN-Cubed provides an abstraction that allows one to treat all compute resources as being on the same network, regardless where they are actually located at the moment.

At the same time, VPN-Cubed can be appealing to topologies that don’t care about Intercloud. Networking and network security are areas that don’t get enough attention from cloud developers today, because developers are used to working within a perimeter. Excessively wide-open security group setups, using public IPs instead of private for communications, disabled local firewalls – these are all time bombs. They don’t affect the app right now (”look, it works!”) but they can be catastrophic over time when they could become an attack vector. For such topologies, VPN-Cubed provides a virtual perimeter that confines authorized communications to a mutually-authenticated tunnel encrypted end-to-end (are you sure you want to continue forcing your incoming web traffic to HTTPS but not encrypting writes and reads from app servers to database? or do you think application-level encryption could be better, faster or easier to maintain than transport-level?)

To get started with VPN-Cubed, visit http://cohesiveft.com/vpncubed. If you have a question about how VPN-Cubed can help in your particular use case, you can ask here.

UPDATE 08/21/2010: Randy Bias in his excellent new post touches on the very similar theme:

Where is the lock-in then? If it’s not the hypervisor, what makes moving from one cloud to another so difficult? Simply put, it’s architectural differences. Every cloud chooses to do storage and networking differently.

Today CohesiveFT team officially launches VPN-Cubed for Amazon EC2, a product that has been in beta for several weeks now. Check out the announcement on Elastic Server blog, which talks about both Pay and Free Editions, or check out the product page.

VPN-Cubed for EC2 is a self-service preconfigured solution that allows you to build overlay networks inside Amazon EC2 cloud, either in a single region (US or EU) or spanning multiple regions. Building a private network across the Atlantic can not be any easier or cheaper than this! All you need to get started is familiarity with EC2 – we packaged the rest into AMIs and wrote detailed step-by-step documentation.

The product has all the benefits of our regular VPN-Cubed offering:

• customer assigned IP addresses in the cloud
• encrypted communications between all hosts
• built-in high availability and failover, no single points of failure (there is no single master server in case you are wondering)
• support for IP multicast inside EC2 cloud (without VPN-Cubed, your multicast-based applications will not work in EC2)

And in addition, we created an easy-to-use web-based admin tool to make configuration and monitoring your private topology in the cloud even simpler.

VPN-Cubed for EC2 is a great way for you to quickly try it out, see how it works and how it can help you take your cloud operations to the next level. And if you need greater flexibility, more complex interconnects, customized discovery, agent-based monitoring, further traffic optimization or want to use VPN-Cubed outside of EC2 – contact us and we can tailor VPN-Cubed to meet your needs.

I personally think the title is somewhat misleading. It would have been more appropriately named “10 Reasons Enterprises Aren’t Ready to Take Their Entire In-House IT Operations to the Cloud.” The difference is huge. Enterprises can totally trust the cloud to perform certain operations. Massive data crunching tasks that need to run occasionally are perfect for this.

There is a single reason why we don’t hear more about enterprises adopting clouds (yet!) – it’s not that easy. First of all, if we have a massive dataset to run through a computation cycle, this dataset first needs to be transferred somewhere where cluster nodes can get to it. In case of Amazon Web Services,S3 is where one could put it. But before we can transfer the data, we need to extract it from the source (database, data warehouse etc). This can be easier said than done.

Once the dataset is ready, a number of cluster nodes need to be started – you need an AMI and a private communication mechanism for your instances. You also will need discovery tools, because EC2 assigns dynamic IP addresses and without discovery, your cluster nodes will not be aware of each other. And these are only high level steps…

So, if you are an enterprise and you would like to show GigaOM that you do trust the cloud, are you on your own to make it all happen? I happen to know the answer. I work for company called CohesiveFT and our Elastic Server platform can help you in several important ways. Firstly, remember that first step of extracting the dataset from your internal system? How would you feel if I told you that you can skip this step – instead you can set up a private virtual network between your Amazon EC2 instances and your corporate datacenter so that cluster nodes can access the data source directly? If you are interested, check out our VcubeV multisourcing technique. It will also help you sort out the problem of dynamically assigned IP addresses (hint: VcubeV virtual IP addresses can be static).

Secondly, you can use Upload Your Package feature to easily embed your home-grown software to be included in every cluster node. You will save quite a lot of time if you use a nice web GUI to assemble your cluster node instead of building and bundling an AMI manually. Patch management will also be easier – consider a couple of clicks to upload a new version of software and rebuild the server, vs. repeating entire bundling process from its very beginning.

And finally, Elastic Server On Demand can launch your servers in EC2  as easily as it can build a vmware image of exactly the same stack for you to test locally.

If you are an enterprise looking for help to get started in the cloud, you now know where to find us.

• Mac. I got a Macbook Pro as my work laptop. Feels great every time I sit down to work, even though I am not a very demanding desktop user – browser, email, IM, rss reader for non-public feeds, terminal, occasional Office document work, occasional PDF to read. And most importantly – VMware Fusion to run my Linux development box.
• Debian. Our paths surprisingly have not crossed until this year. Probably because I lost interest in low level OS maintenance a long time ago and was not actively looking for a new distro to try (my home server is running Red Hat Linux 7.0 with everything important upgraded to later versions – I don’t use RPMs and prefer GNU Stow for package management). Overall, I can’t say I dislike it. apt-get is nice. And deb packaging format imho is superior to rpm (writing standalone rules files using GNU make is easier than writing shell within spec files). Like rpm however, apt-get does not handle edge cases, when I don’t want computer to think it’s smarter than me.
• Ruby. The second attempt to learn ruby was successful. Ruby now joined Python on the list of my favorite languages. They are both equally great for doing all kinds of development, provided your project doesn’t use both at the same time – each has its own patterns and I personally find it somewhat hard to adjust when I try to write in both in parallel. I am also happy that Ruby joined Python on my list, not displaced it (several years back, Python did displace Perl on my list) – so I am kind of bi-lingual now, which mimics my real life situation with real languages.
• Rails. Yes, Ruby on Rails and Django on Python are my favorites now.
• RabbitMQ. This is a big one. For quite some time I have been searching for a highly available messaging bus as a holy grail of building and operating a scalable system. I think I found what I was looking for in RabbitMQ and its implementation of AMQP standard.

Elastic Server On Demand is built with each of these somehow tied to it behind the scenes. Shaping up very good so far…

And if you want to get going with Amazon EC2 but don’t want to waste your time or resources on bundling your physical machines into virtual, ESOD can help and you can be running your instance with *your* code in Amazon EC2 cloud in a matter of minutes.

In short, take a Dam Tour!