Today, we are thrilled to announce that Atlassian is releasing Cloudtoken as Open Source. Cloudtoken is a federated authentication tool for AWS IAM Roles. The new tool allows users to benefit from the security IAM Roles provide, mitigates the one-hour credential expiration and improves the development lifecycle by simulating an EC2 environment available locally. The source is available on Bitbucket and the tool is available as Python packages, installable via pip. Please see the README for more information.
But now for a little history…
The Old Way, The New Way & The Roadblock
At Atlassian we’ve been using AWS for a while now. Like many organizations, early in our journey we provisioned IAM Users for any staff that required access to AWS. This worked initially and allowed us to quickly get our first accounts and services provisioned but, as we increased our AWS usage, we quickly encountered scalability issues with user management. Specifically there was no automated process to add staff to groups or revoke access if staff moved to another team or left the company.
After looking into the issue we decided to implement Federated Authentication and move to using IAM Roles for staff access. Staff were then able to authenticate with our internal Identity Provider, which has knowledge of our internal departmental groupings and is aware when staff members move teams or leave the company. After a staff member is authenticated, they are presented with a list of IAM Roles they have permission to assume. IAM Roles also come with the added security benefit that credentials expire after one hour, which significantly limits the risk posed by leaked credentials.
This solution ticked all the boxes and definitely seemed the way forward. Then we hit a roadblock.
When using IAM Users it was easy for staff to download their AWS credentials and configure them in the awscli tool or any other AWS application they had on their workstations, but unfortunately there is no easy or standard way to obtain credentials when using IAM Roles. After looking at the existing tools available, we decided that none provided the features or flexibility we required. So, we decided to develop a new tool that would fit our needs.
A Better Way: Cloudtoken
The command line utility Cloudtoken is written in Python, it enables the use of ephemeral AWS credentials provided by IAM Roles on your local workstation. It features a pluggable architecture that allows for custom authentication sources such as ADFS, Shibboleth and SimpleSAMLPHP and custom handling of credentials.
Additionally Cloudtoken can be started in daemon mode, which replicates the instance metadata endpoint (http://169.254.169.254) that AWS makes available in the EC2 environment. When run in daemon mode, Cloudtoken will automatically refresh the credentials every 45 minutes to prevent the credentials from expiring. This allows ephemeral credentials to be used for long running tasks, such as API calls made over a period greater than one hour (such as iterating over a large list of objects in an S3 bucket).
Having the instance metadata endpoint available in our local development environment allows us to develop applications with a greater degree of confidence that they will behave consistently across environments.
Here’s an example of Cloudtoken running in daemon mode:
$ cloudtoken -d Launching in daemon mode... Configuring link-local address on lo0...done. Available roles to choose from: 1. arn:aws:iam::123456789012:role/Read-Only-Role 2. arn:aws:iam::123456789012:role/Sysadmin-Role 3. arn:aws:iam::123456789012:role/User-Role Enter number of role you want: 1 Using role arn:aws:iam::123456789012:role/Read-Only-Role Metadata proxy now available on http://169.254.169.254 ------------------------------------------------------
An example of querying the instance metadata endpoint:
$ curl http://169.254.169.254/latest/meta-data ami-id ami-launch-index ami-manifest-path block-device-mapping/ hostname iam/ instance-action instance-id instance-type kernel-id local-hostname local-ipv4 mac metrics/ network/ placement/ profile public-keys/ reservation-id security-groups services/
Awesome, how can I get more information on this and contribute?
The Cloud Engineering team at Atlassian would love for Cloudtoken to become the communities tool of choice for command line authentication with cloud providers. We hope the community will assist us in this by contributing code and plugins via the Bitbucket repository and we can’t wait to see what everyone comes up with. The repository is also where people can raise bugs, feature requests and read the documentation.