Libcloud update - Key pair management methods are now part of the base API

Yesterday, I have merged a Libcloud pull request which promotes SSH key pair management methods to be part of the base Libcloud compute API.

In this post I’m going to talk a bit about the project history and evolution and show how to utilize this new functionality.

History and Background

Libcloud was originally developed in 2009 at Cloudkick to solve a problem of talking to multiple different cloud provider APIs.

Later that year, the project joined Apache Incubator and in May of 2011, the project graduated from the incubator to a top level project.

An example of how Libcloud did not came to be.

First couple of versions were simple and only exposed a small API (~6 methods) for managing cloud / virtual servers.

A list of methods supported in the first few versions of Libcloud. Source: Apache Libcloud @ Open Source Bridge presentation.

Down the road, the cloud evolution pace and competition increased and providers started adding more and more features and new services. On top of that, demand from our users also grew, so it made sense for us to start thinking about increasing the project scope and add support for other cloud services.

As such, version 0.5.0 was born in 2011. This version represented a very important milestone for the project. It was a first release which moved away from only supporting the compute API and added support for managing cloud load balancers and object storage.

Not long afterwards, version 0.6.0 which added support for a brand new DNS API was released.

Since then, we haven’t added support for any other new APIs, but have spent a lot of time improving the existing functionality, adding new features, adding support for new providers and improving the library all around.

If you are curious about what we have been working on lately, you should have a look at the Upgrade Notes and Changelog for Libcloud 0.14.0, a new stable version which should be released some time in the next couple of weeks.

SSH key pair management methods promotion

Functionality for managing key pairs was already available in some drivers as part of the extension methods and arguments for quite some time now. In Libcloud, extension methods expose provider specific functionality and usually differ from one provider to another.

Not long ago, we have spent some time unifying those arguments and method, but there were still some minor differences between different providers which made up for not so pleasant experience for out users.

Because of that, I have decided to again evaluate how much sense it makes for us to promote those methods to be part of the base Libcloud compute API. It’s important to keep in mind that Libcloud acts as a lowest common denominator which means that only functionality which is exposed by majority of providers supported in Libcloud can be part of the base API.

It turned out that most of the providers we support offer key pair management functionality which means those methods are indeed a good candidate to be part of the base API. Because of that, I have I decided to write up a proposal.

Documentation for the new functionality.

After some feedback and tweaks to the proposed interface, I have implemented the proposed changes and updated the existing code. To ease the migration and make it less painful for users which rely on the existing extension methods, I have decided to deprecate those methods and leave them in place until the next major release.

Working with the new API

The example bellow demonstrates how to use new ssh key pair management methods which are now part of the base compute API.

import os
from pprint import pprint

from libcloud.compute.types import Provider
from libcloud.compute.providers import get_driver

cls = get_driver(Provider.EXOSCALE)
driver = cls('api key', 'api secret key')

# Create a new key pair. Most providers will return generated private key in
# the response which can be accessed at key_pair.private_key
key_pair = driver.create_key_pair(name='test-key-pair-1')
pprint(key_pair)

# Import an existing public key from a file. If you have public key as a
# string, you can use import_key_pair_from_string method instead.
key_file_path = os.path.expanduser('~/.ssh/id_rsa_test.pub')
key_pair = driver.import_key_pair_from_file(name='test-key-pair-2',
                                            key_file_path=key_file_path)
pprint(key_pair)

# Retrieve information about previously created key pair
key_pair = driver.get_key_pair(name='test-key-pair-1')
pprint(key_pair)

# Delete a key pair we have previously created
status = driver.delete_key_pair(key_pair=key_pair)
pprint(status)

As you can see, I have used Exoscale provider in my example, but it should work exactly the same with other providers which support this functionality. Currently those providers are Amazon EC2, OpenStack (and other OpenStack based providers such as Rackspace) and CloudStack (and other CloudStack based providers such as Exoscale and Ikoula).

For a full list of providers which support this functionality, please refer to the supported providers / methods page.

Conclusion

I hope the addition of the SSH key pair management methods to the base compute API will make it even easier for our users to work with multiple providers and pave the surface for the promotion of other methods which will make Libcloud more suitable for other more complex / advanced use cases.