This blog post was contributed by Sebastian Hesse, a Software Engineer at K15t, based on his presentation at Developer Day 2019.
As organizations search for flexible and scalable data solutions, the pull for going serverless has never been stronger. However, applying emerging technologies to your company's processes is easier said than done. Luckily, you're not alone in your pursuit of serverless cloud app development.
With first-hand experience in implementing serverless app development as a software engineer at K15t (an Atlassian Platinum Solution Partner and Marketplace Vendor), I'll reveal some of my best practices and discuss the roadblocks we were able to overcome while going serverless.
Before we look into the best practices, we need to set up our development environment. There are two ways you can develop serverless functions: locally or directly in the cloud.
Developing a function locally allows you to execute, test, and debug code on your local machine, while the alternative requires you to upload and execute your code in the cloud. In the instance of local development, you have the choice of using different frameworks, like the Serverless framework, the Serverless Application Model (SAM), or LocalStack. In order to go live with your app, you'll still need to use one of the cloud providers like AWS, Azure, Google, or similar to upload and execute your code. For this article, we're going to focus on AWS Lambda.
One key drawback we saw with the cloud execution is that debugging your code can be more challenging. To make a well-informed decision about your development approach, it is important to consider three things:
- You must have a test environment that closely resembles your production environment. So don't only use your local machine, because then you’ll always simplify something, be it services, access policies, or something else.
- Local code updates are generally faster because you don't need to update or upload your code into the cloud (although this process can be optimized using small tools or a clever approach).
- Debugging your code along the way should always be part of your process. Locally running tests of your business logic will help you easily spot errors in your code, so pay close attention to split infrastructure code and business logic.
Another important point is to keep track of all your Lambda functions. The best way to do this is to describe your infrastructure using one of the frameworks above to help keep track of the Lambda functions deployed in your environment. You may also utilize a CI/CD pipeline, which allows you to automate your deployment as much as possible.
Keep it simple
The first best practice when going serverless is to limit the scope of your functions. For example, if there are different tasks you want to accomplish like receiving a webhook, processing the webhook data, and then sending a notification, you could put them all into one Lambda function.
However, this will reduce your app's scalability. I suggest keeping your functions simple and separate your concerns. Consider focusing on just one task your function will perform and pour your energy into delivering that functionality very well. This will lead to better scalability and reusability in your app. This also reduces the file size of your Lambda function, so you may also be able to reduce the cold-start time.
Every millisecond matters here, because the bigger your artifact, the slower the startup time of your function, so decide carefully on which dependencies you’ll include in your code. This is especially important for Java programmers! Avoid using frameworks like Spring – it just slows down your function.
If you want to use your Lambda functions as a REST API, look for services like API Gateway or AWS App Sync to connect with your Lambda functions. However, keep in mind that due to the cold-start problem, the response times of your Lambda functions might have some spikes.
Key in on communication
If you follow the advice from above to split up your Lambda functions, then communication between your functions becomes very important in order to exchange data and create a flow within your app. Depending on your needs, you can utilize:
- Synchronous communication: Directly call another Lambda function from within a Lambda function
- Asynchronous communication: Upload data to a service and let this service trigger another Lambda function
Both approaches have their drawbacks: synchronous communication results in a tight coupling of your Lambda functions and might be problematic if you need to exchange big chunks of data, whereas asynchronous communication always involves a different party and can get pricey. However, in order to be more flexible and improve the scalability of your app, asynchronous communication is the preferred choice.
Ensure all aspects are scalable
Many services like AWS and Jira have the capacity to automatically scale applications, leading many developers to overlook this step. However, you may find yourself working with numerous services in order to achieve the desired functionality for your serverless application. These services won't all have the same scalability. Protect your code from malfunctioning by setting up a queue, or buffer requests if necessary, to hedge this issue. One service that proved to be very useful for us was Amazon Kinesis, a streaming service capable of handling lots of data, which we use to buffer all incoming webhooks.
Make sure 15 minutes is enough time to execute your app
In Lambda, your functions have 15 minutes to run before they time out. Given that this limit was recently increased from 5 minutes, this may not seem like a big concern. But depending on your app, instances can quickly add up.
You can optimize this a bit with some more parallelization within your Lambda function, but to really circumvent these time limits, consider using other approaches like recursion (calling yourself from within a Lambda function to continue processing) or AWS Step Functions, which manage the execution workflow around the time limit. You may also need to consider the viability of outsourcing this to another service.
Understand data limits
Other important aspects to take into consideration are the limits of a Lambda function. For example, you have up to 3 gigabytes available for your memory allocation, but be aware that memory allocation and CPU power correlate: the more memory you allocate, the more CPU power you have. The same is true for network and I/O throughput which correlate to memory allocation as well.
Furthermore, you only have about 512 megabytes available for temporary local storage. If you're in the middle of a project (or ideally in the planning stage) and feel as though these limitations will be too challenging to overcome, then serverless may not be the way to go for this particular set of code.
Serverless to the rescue
If you're considering (or in the middle of) creating a serverless app, you can utilize these key takeaways from my team at K15t to create a proper scope and avoid potential snags in the dev launch cycle. Let me know what you think about my best practices or reach out to me directly at firstname.lastname@example.org with any questions!
Watch Sebastian’s full presentation from Developer Day 2019 to learn even more.
Sebastian is a software engineer at K15t, where he has taken on a leading role in designing and implementing a high-scale cloud architecture for Backbone Issue Sync for Jira. He’s eager to share his experience with others, and when he’s not developing for K15t’s apps, you can find him speaking at local AWS user group meetups.