How OpenFaaS came to rescue us!

An overview of our experience with serverless computing and speeding up Iconscout.

Being a bootstrapped startup, it’s an important decision to decide which technologies to invest in. Whether it’s cloud VMs, cloud services or even email providers. We were in the same situation when we opted to use OpenFaaS for our upcoming major update of Iconscout.

Previously, we’ve worked with Azure FunctionsAWS LambdaServerless and many more to do small on-demand actions. And now with this update, we wanted to move to serverless architecture for all the tasks. Majority of our cloud usage is for processing of files we’re receiving from our contributors. And also this is the most time & cost consuming activity. We started looking for a service or hosted solution, using which we can deploy our services in seconds without much human interruption. We are a small team of dedicated people who are involved in development, deployment and/or maintenance of Iconscout, thus we’ve to take these constraints in our mind. Plus, cutting down infrastructure cost matters a lot to us.

Use Case

Let’s get into our use case first, and then I’ll show you how we are working with OpenFaaS.

Iconscout is a Design Resources Marketplace, where designers can upload their icons and buyers can choose from our collection of 900,000+ icons for their projects. We are receiving thousands of vector SVG icons from our designer community on weekly basis. Once designer submits his/her icons, we then convert those vector SVGs into 11+ different sizes and formats, including PNG (from 512×512 to 16×16), AI, EPS, PDF, PSD, ICO, ICNS, with and without watermarked versions. Also, we generate different Icon Pack mockups to display them on-site, for SEO, OpenGraph images and many more purposes. Plus, we’ve unique feature to download one click zip files for our premium users.

Image/Icon conversion output on Iconscout
Image/Icon conversion output on Iconscout

Previously, in Iconscout, we used traditional monolithic and microservices. During these conversions, we are dealing with thousands of files, which move to and fro from our storage providers. We were using traditional single-threaded PHP workers to process these tasks.

Use case 1: File Conversations

File Type conversions from SVG
File Type conversions from SVG

Use case 2: Zip download

Creating a zip on request
Creating a zip on request

Problems

  • Sometimes these tasks were so heavy that, they took 20–30 minutes to process.
  • Continuous memory outage, high usage of CPU.
  • Fetching files from storage in single threaded process was time taking due to network latency.
  • Manual scaling which was time-consuming.

Possible Solutions

We started exploring possible solutions to solve our problems. As mentioned above, we wanted some reliable service or self-hosted solution which is easily scalable and low-cost solution.

Azure Functions

We gave first try to Azure Functions as we’re already using them for testing purposes. And below are the points that forced us to look for alternatives.

  • It runs on Windows, thus makes many binaries un-supported. And we’re working on Linux Environment for development, thus we have to make an extra effort to test and deploy those functions.
  • While first-time deployment, it runs `npm install`, which was time taking(~30–40 sec) for each function.
  • It runs on Azure Web Services, adding a base cost whether we use it or not, and that was higher than our alternatives.

AWS Lambda

This was our next chance to go for due to its popularity among the developers. But our journey continued after this too!

  • Have to upload `zip package` to make the function live! (I said, we needed something which doesn’t require much human interruption.)
  • Runs in AWS Linux, thus we need to compile some of our `npm_modules` in that base, we’re heavily dependent on image libraries such as, ImageMagick, Sharp, etc.
  • Auto-deployment is not available via any mode, if you want it you need to use s3, which is again somewhat manual process.

OpenFaaS comes to rescue!

Finally, we found OpenFaaS! In the beginning, it took time to learn about how it works, how to develop and deploy it. Thanks to Alex Ellis, for being there actively with us on every stage.

Alex is the most helpful & active member amongst all the communities that we came across.

We started testing OpenFaaS, for small on the go event like generating zip, as well as for long-running workers of our image conversions. We were already using Docker for our development purpose, thus utilized Docker Swarm for OpenFaaS. And guess what? Those 20–30 minute time taking tasks came down to 5–7 minutes duration. And most important part is, We can deploy it to a new VM in a moment! Yes, we have done it for our testing, staging and production environments.

OpenFaas Portal to check status and invoke functions
OpenFaas Portal to check status and invoke functions

Let me give you an example diagram of how we’ve utilized OpenFaaS for our upcoming update.

Example 1: File Conversations

Functions for converting SVGs to PNG-16 to 512, AI, EPS, watermarked icons
Functions for converting SVGs to PNG-16 to 512, AI, EPS, watermarked icons

Example 2: Generating Zip on the fly

OpenFaaS creates Zip on the fly, and returns the download URL to user
OpenFaaS creates Zip on the fly and returns the download URL to the user

Example 3: Generating Custom sizes on request

User requesting file in 300px x 300px via OpenFaaS
User requesting file in 300px x 300px via OpenFaaS

Key Points

  • A wide range of programming languages is supported, such as NodeJS, PHP, C#, Go, and even native binaries, i.e. Phantomjs, imagemagick.
  • Access to an OpenFaaS portal to try out the functions using GUI.
  • Based on Docker Images, code, push & deploy in seconds.
  • Docker Swarm (native Kubernetes integration is also available) makes it easy to manage the infrastructure of more than one VM easy.
  • Deploy in seconds, auto-scaling, remote deploy
  • Powerful faas-cli to write & play with new functions

In simple words,

“With OpenFaaS it was incredibly easy for us to develop functions and integrate them into our applications. I was able to get started in minutes and I found the portability really important for our use-case.”

It was a very good experience to work with OpenFaaS and Alex Ellis. If you want to get started serverless computing, just give it a shot. You can find getting started guide at OpenFaaS docs and do join OpenFaaS community on Slack. I bet you won’t regret it.

I hope you liked this article and looking forward to comments & suggestions from you guys! Do share with your teammates.😉

 

Share This:


Also published on Medium.