As part of getting into SLM for small AI devices, we are going to look at inference speed on the Raspberry Pi. Mostly six models are of interest:

• tinyllama
• tinydolphin
• phi3
• smallthinker
• granite3.1-moe
• llama3.2:1b

Each model has a small number of parameters, to make sure we get usable speeds for inference.

We will get speed figures on simple inference for each of them.

In a previous post, we installed Arch Linux on the raspberry pi (make sure to have a look at Running Arch on Pi ).

I have some software packages on the Arch User Repositry (AUR), making it easy to deploy on various devices. It used to be a bit tricky to install Arch on the raspberry pi, but now it is possible to get to a running system quite fast, so here are notes on how to get going, and get your Pi running with Arch.

Download the ISO

First of all, we need the ISO for install. The iso/zip I downloaded was from rasparch on sourceforge. Directly burned the file to the SD card, using standard tools Etcher. I inserted the SD card in the Pi, plug the power, and the small beast started booting …

Back to Clojure for a bit with this library to access Ollama via my favourite language. Sheer amount of reliable Java+Clojure open source libraries has just made my life easier in the last few weeks.

Basically, Pyjama has full support for all the Ollama API endpoints, generate, chat, pull etc.. you name it.

Right now it is available as a clojure deps library, you can use it in your project, by adding the git coordinates to the deps.edn file.

The Goal was to have a go program that converts JSON data i have on a project to a CSV file. Looks like that was easy for ChatGPT so did a few extra steps to generate some Golang code, first naively, then streaming through the input json file. What follows is the ChatGPT conversation.

You: turn this json snippet into csv: [ { “Id”: 0, “Date”: “2022-11-01T09:00:00+09:00”, “Project”: null, “Cost”: 5000, “Estimate”: 6000, “Updated”: “2022-11-01T00:00:00Z” }, { “Id”: 0, “Date”: “2022-10-01T09:00:00+09:00”, “Project”: null, “Cost”: 4000, “Estimate”: 5000, “Updated”: “2022-11-01T00:00:00Z” } ]

I hadn’t put any thought into using the github or gitlab CLI for a long time, but both are pretty useful. We are using standard github for projects and also a self installed gitlab. Both CLIs are golang programs.

First we need to authenticate.

⋊> ◦ gh auth login                                                         

? What account do you want to log into? GitHub.com
? You're already logged into github.com. Do you want to re-authenticate? Yes
? What is your preferred protocol for Git operations? SSH
? Upload your SSH public key to your GitHub account? /Users/niko/.ssh/id_rsa_m1.pub
? Title for your SSH key: GitHub CLI
? How would you like to authenticate GitHub CLI? Login with a web browser

! First copy your one-time code: 4024-84CC
Press Enter to open github.com in your browser... 
✓ Authentication complete.
- gh config set -h github.com git_protocol ssh
✓ Configured git protocol

✓ Uploaded the SSH key to your GitHub account: 
✓ Logged in as hellonico

gh extension install redraw/gh-install  

https://pkg.go.dev/github.com/cli/go-gh#section-readme

We recently deployed a Kubernetes cluster based on KubeSpray at a client. Since there is no out of the box cloud storage, one of the first task to clear for the cluster deployment is the data persistence.

We gave sshfs a try, and then thought we could actually also manage the distributed persistence using Goofys. Goofys allows you to simply mount an AWS S3 bucket as a file system. In effect, all the S3 data is synchronized between the different nodes of the cluster.