Posts

8 On-Device Inference Challenges

One of the trends I find the most exciting these days, is the local inference of LLMs on mobile. That’s for sure related to my time as a mobile- (or, rather iOS-) developer. Yet, the domain brings so many compelling technical challenges that it’s hard not be fascinated about. As I wrote in a previous LinkedIn post, local inference offers many benefits for AI, especially generative AI, primarily in terms of distributed resource consumption, privacy, and offline usage. However, it is clear that we are still in the very early days, with numerous obstacles to overcome before this becomes a fully viable option. That said, with the field of generative AI evolving rapidly, this approach—particularly within a vision where intelligent agents play a central role—could quickly become a key technological element. Today, we will examine the main challenges that developers, researchers, and companies are likely to face when bringing LLM inference to mobile devices. We’ve identified 8 challenges, which we will dive into right away.

Starting 2023 (A New Hope)

I’m particularly glad to remark that 2023 is starting with a newfound energy - and new habits.

Rebase Key Bindings for Sublime Merge

Editing Git history (or Interactive Rebasing) in Sublime Merge is a joy. And can be even better, with some keybindings!

Understanding The Basics of Kotlin Multiplatform Projects with Kotlin 1.3

Kotlin Multiplatform: testing a shared module supporting iOS and Android

Conceiving your app as modular can be greatly beneficial to your codebase’s health. In particular, designing a separation between your view and business logic can decrease the coupling between your layers and improve maintainability as well as testability. Tests are indeed your app’s best friends, as they can, at the same time, verify the code you wrote works as intended and act as a specification for the implemented functionality.