Recent Posts

Http4s Streams and Multipart Form-Data File Uploads

10 minute read

, ,

Streaming is the primary mechanism to reduce memory requirements for processing large datasets. The approach is to view only a small window of data at a time, allowing data to stream through in manageable amounts matching the data window size to the amount of RAM available. A practical example is a file-upload, where multi-GBs file streams can be handled by MBs of server RAM. However, enforcing streaming in software code is prone to errors, and misuse or incompatible method implementations will lead to breaking stream semantics, and ultimately to OOM exceptions. This article focuses on streams within the context of file uploads, using the Http4s library for examples.

Compiling Scala Native in a GitHub Action; Alternatives to GraalVM

11 minute read

, ,

Scala Native is a compiler and JDK written in Scala with the goal of removing Scala’s dependency on the JVM. This isn’t meant to achieve a higher performance such as with JDKs, and it is targeting a specialized use-case not considered to be today’s typical Scala development. Its competitors are Rust and Go, not GraalVM, Java or Kotlin. This article goes through common steps and challenges encountered when compiling Scala Native for linux with a GitHub Action.

Data Transfers and Egress within a GitHub Action

2 minute read

, ,

The free tier of GitHub Packages has limited bandwidth to download private artifacts; which can make it unsuitable for use in a CI/CD pipeline for projects on a budget. In an effort to increase GitHub Packages’ usability, this article develops an alternative approach minimizing the dependency on GitHub Packages as hot storage, but preserving it as a viable cold storage, durable storage solution.

List Lookup Cache with Scala ZIO

5 minute read

, ,

In-memory caches mapping Key => Value are a simple and versatile tool to reduce number of calls to an origin datasource. There are many use-cases requiring multiple cache calls, preferring Seq[Key] => Seq[Value]. Can a standard cache implementation be expanded to efficiently handle this scenario?

The Many Different Forms of Concurrency and Parallelism

10 minute read

, ,

Modern software design requires the understanding of the different layers of concurrency and parallelism that can exist. Abstractions exposed by libraries and frameworks can inadvertently hide layers of parallelism when their focus is the simplification of others; and libraries trying to treat all levels of parallelism equality can be limited to low level concepts for their common interface. In order to optimally design and avoid errors, all levels of concurrency and parallelism need to be understood no matter what framework is chosen.

SBT Parallel Compile Optimizations using Quill Sub-Projects

9 minute read

, ,

Scala is slow to compile. Advanced syntax constructs and a robust type system can increase developer productivity and runtime reliability but also create extra work for the compiler. Macro libraries such as Quill are effectively programs written for the compiler, and can represent an unbounded amount of work depending on what they are trying to accomplish. Are there ways to structure our Scala 3 code to ensure that we can embrace the rich macro ecosystem without excessively long compile times?

ZIO Migration from Akka and Scala Futures

6 minute read

, , ,

Akka / Apache Pekko is a robust and popular Scala framework used to build concurrent production-grade software. One of the concurrency primitives it uses is the standard scala.concurrent.Future class. Before these existed in Scala, there was the Twitter Future offering similar, but expanded functionality including cancellation / interruptibility. Ignoring the functional coding style promoted by ZIO for a second, the concurrency primitive used by ZIO, known as ZIO[R, E, A] can be viewed as a more advanced Future[A].

Akka gRPC with Let’s Encrypt

3 minute read

, ,

The Electric Frontier Foundation (EFF) has recommendations about encrypting the web; there is no reason to be running servers over unencrypted HTTP any longer. It is irresponsible to your users and unnecessary, as such there is a formal mechanism called HTTP Strict Transport Security (HSTS) that enforces HTTPS for all requests at the domain level. Taking it further, modern browsers include a set of domains which can only work over HTTPS, it started with Google TLDs such as .dev and .app, but it is growing https://hstspreload.org/.

Trade Audit Mobile App Infrastructure

3 minute read

, , , , ,

The release of the Trade Audit mobile app is almost here. It is currently in MVP stage, but its infrastructure is a pretty typical cloud based deployment. This article discusses design choices made, evaluating how effective they were.

Lit Custom Components for SVG Generation

4 minute read

, ,

SVG markup is very similar to HTML, and the Lit Web Components library can be used to not only generate HTML custom components, but also manipulate SVG in a similar way using Lit templates. Lit is a small 5kb library that removes the boilerplate code of DOM generation, and is highly interoperable with all web frameworks since it relies on browser native custom elements.

JVM versus Python for AWS Lambda Functions

3 minute read

, ,

The suitability of programming languages across different domains is a contested topic. AWS Lambda Functions are a serverless solution that can be used for a wide range of problems from tiny to large tasks. For lightweight tasks how does the JVM stack up?