A couple of weeks ago, I had the pleasure of attending ScaleConf here in Wellington for the second time. For the second year running, the conference was sponsored by Catalyst Cloud.
For those unfamiliar with it, ScaleConf is pretty much exactly what it says on the tin: speakers from all over the world, describing to an audience of systems admins, developers and managers, their first-hand experiences in running web applications and websites at cloud scale, using a variety of techniques, technologies, services and metrics.
You may find it interesting to note that cloud scale is completely unquantifiable. The system in question should naturally be running at least one cloud service, a term that comes with its own limitations, and could refer to any range of metrics from requests per second, to data throughput and everything in between. What we might define as large with regard to a given metric for one company, isn’t necessarily so large for another.
This relative metric observation was especially noticeable in the stories coming from New Zealand companies at the conference. Both years, New Zealand has been represented at ScaleConf by some truly innovate companies: Vend, Timely, Xero and of course Catalyst, but what constitutes “scale” for each, is different to overseas companies who have also presented – MySQL, AWS, Atlassian and Hashicorp.
Timely are a New Zealand company that provides SaaS appointment management services for small businesses up and down the country. Timely has a geographically distributed workforce, arguably a metric unto itself with regard to scale, and with accounts in the thousands, pales in comparison to the computing, storage, and traffic capabilities of Microsoft, AWS and even Atlassian. But numbers aren’t everything and Andrew Schofield, one of the company’s co-founders, described Timely’s challenges in meeting the service levels expected by its customers, and warned us to avoid premature scaling. “It’s OK not to, until you need to…[sic].”
Daniel Larson works for Microsoft New Zealand and touched on an issue which became a recurring theme throughout the conference and for me personally days after. That is, as service providers, architects and developers, we cannot afford simply to take what we have known – technology, process, methodologies – and transplant them directly or with only minimal changes into brand new paradigms. In the short term, things may appear as though they are working, but longer term, the true benefits of new paradigms become very apparent. Unfortunately, so do the deficiencies in applying legacy thinking, which stifle true innovation and disruption.
The last speaker I wanted to mention was Pam Rucinque. Pam quickly dispelled the myth that serverless means “without server” (well, of course, but with a name like that one should at least check). Pam and her team at ThoughtWorks worked with a client to develop a mobile application that required back-office data sources, but where the management overhead and costs involved in managing servers, load-balancing, caching and the rest of it, howsoever simplified with browser-based configuration, was too much. It was decided therefore to host the application on the AWS Lambda service, Amazon’s FaaS (Function as a Service) offering. While the concept itself sounds exactly like the process any standard web server request goes through (listen, action, respond) Lambda does this but without your (directly) needing to pay for the “listen” part.
Developers tend to want to shy away from making frequent requests to a web service endpoint, given the overheads in latency and service’s tendencies to rate-limit demand over a threshold. But with Lambda, the opposite would seem to be the case. Make as many lightweight network API calls as you like, and worry less about API service levels, and instead concentrate on what you do best, building the app itself.
Thanks again to Catalyst for giving me the opportunity to attend gigs like ScaleConf. If you haven’t been before, and you like what you’ve heard, then indications from the organisers are positive that there will be a ScaleConf 2018, and I highly recommend you go if you are able.
***
Russell is a Senior Developer in Catalyst’s SilverStripe team, and has been with the company in Wellington since mid-2016. When he’s not busy espousing the benefits of readable code, team development standards and CI, he can be found playing horsie with his kids, and designing a killer blockchain app at home in Kapiti.
