racket-audio-native-libs/README.md

racket-audio-native-libs

Native library distribution repository for racket-audio.

This repository contains the platform-specific native libraries used by racket-audio. It uses one Git branch per published Racket package.

The main branch contains the build and maintenance files. The other branches contain the actual Racket packages that are registered in the Racket Package Index.

Branch structure

The repository uses the following branches:

main
native-libs
x86_64-win32
x86_64-linux
aarch64-linux
aarch64-macosx

Their purpose is:

main
    Build scripts, documentation and maintenance files.

native-libs
    Meta-package: racket-audio-native-libs.
    Selects the correct platform package.

x86_64-win32
    Native libraries for 64-bit Windows.

x86_64-linux
    Native libraries for 64-bit Intel/AMD Linux.

aarch64-linux
    Native libraries for 64-bit ARM Linux, for example a 64-bit Raspberry Pi.

aarch64-macosx
    Native libraries for Apple Silicon macOS, for example an M1 Mac.

Each platform branch is a separate Racket package.

Package names

The intended Racket package names are:

racket-audio-native-libs
racket-audio-x86_64-win32
racket-audio-x86_64-linux
racket-audio-aarch64-linux
racket-audio-aarch64-macosx

racket-audio itself should depend on:

racket-audio-native-libs

The racket-audio-native-libs package then selects the correct platform package.

This keeps the Racket code in racket-audio independent of the operating system and architecture.

Native package layout

A platform branch contains the native libraries directly in the root of the branch, together with info.rkt.

For example, the x86_64-win32 branch may look like:

info.rkt
portaudio.dll
libsndfile-1.dll
libmpg123-0.dll
...

The exact DLL names depend on the libraries used by racket-audio.

The corresponding info.rkt uses copy-foreign-libs:

#lang info

(define collection 'multi)

(define version "0.1.0")

(define pkg-desc
  "native libraries for \"racket-audio\" on \"x86_64-win32\"")

(define pkg-authors
  '(hans))

(define install-platform
  "win32\\x86_64")

(define copy-foreign-libs
  '("portaudio.dll"
    "libsndfile-1.dll"))

(define deps
  '("base"))

Every native library that must be installed with the package must be listed in copy-foreign-libs.

This also includes DLLs, shared objects or dylibs that are runtime dependencies of the primary audio libraries.

During package setup, Racket copies these files to a location that can be found by ffi-lib.

Meta-package

The native-libs branch contains the package:

racket-audio-native-libs

It contains no native libraries itself.

Its info.rkt selects the correct package based on the current Racket platform.

Conceptually:

racket-audio
    |
    +-- racket-audio-native-libs
            |
            +-- racket-audio-x86_64-win32
            +-- racket-audio-x86_64-linux
            +-- racket-audio-aarch64-linux
            +-- racket-audio-aarch64-macosx

Only the package matching the current platform is installed.

Use from racket-audio

The racket-audio package should declare racket-audio-native-libs as a dependency in its info.rkt.

For example:

(define deps
  '("base"
    "racket-audio-native-libs"))

The FFI code should preferably load libraries by their logical name instead of using machine-specific absolute paths.

For example:

(ffi-lib "portaudio")

rather than:

(ffi-lib "C:\\some\\local\\directory\\portaudio.dll")

The native package is responsible for installing the actual library for the current platform.

Development workflow

Development and package preparation take place on main.

The platform branches should contain only the files required for the published Racket package.

A typical Windows workflow is:

main
    |
    | prepare/test Windows DLL package
    v
x86_64-win32
    |
    | commit + push
    v
Racket Package Index

The same approach can later be used for Linux, Raspberry Pi and macOS.

Adding or updating a Windows package

Start on main:

git switch main
git pull

Prepare or collect the required 64-bit Windows DLL files.

Before publishing, verify that all runtime dependencies are included.

Then switch to the platform branch:

git switch x86_64-win32

Update:

info.rkt
*.dll

Make sure copy-foreign-libs contains every DLL that belongs to the package.

Commit and push:

git add info.rkt *.dll
git commit -m "Update Windows native libraries"
git push

Adding or updating Linux x86_64

Use:

git switch x86_64-linux

The branch will normally contain:

info.rkt
*.so
*.so.*

Include all shared libraries required at runtime.

After testing:

git add .
git commit -m "Update Linux x86_64 native libraries"
git push

Adding or updating Raspberry Pi / Linux ARM64

For a Raspberry Pi running a 64-bit operating system and 64-bit Racket, use:

git switch aarch64-linux

This package is intended for:

aarch64-linux

Build the libraries natively on the Raspberry Pi where practical.

After testing:

git add .
git commit -m "Update Linux ARM64 native libraries"
git push

Adding or updating Apple Silicon macOS

For an Apple Silicon Mac, such as an M1 Mac, use:

git switch aarch64-macosx

The branch normally contains:

info.rkt
*.dylib

Build and test the libraries on the Apple Silicon target machine.

After testing:

git add .
git commit -m "Update Apple Silicon native libraries"
git push

Testing a platform package locally

Before registering or updating a package in the Racket Package Index, install the platform branch locally and verify that racket-audio can load and use the libraries.

For example, from a checkout of the platform branch:

raco pkg install --auto .

After changing an already installed package, use the normal Racket package update/reinstall workflow as appropriate.

The important test is that the existing FFI code can load the libraries without using local absolute paths.

Racket Package Index

Each published branch is registered as a separate Racket package.

The source URL can point directly to the corresponding Git branch.

Conceptually:

racket-audio-native-libs
    <repository-url>#native-libs

racket-audio-x86_64-win32
    <repository-url>#x86_64-win32

racket-audio-x86_64-linux
    <repository-url>#x86_64-linux

racket-audio-aarch64-linux
    <repository-url>#aarch64-linux

racket-audio-aarch64-macosx
    <repository-url>#aarch64-macosx

The Racket Package Index therefore does not need a separate manually uploaded ZIP file. The Git branch itself is the package source.

Initial branch publication

If the branches exist locally but not yet on origin, publish them with:

git push -u origin native-libs
git push -u origin x86_64-win32
git push -u origin x86_64-linux
git push -u origin aarch64-linux
git push -u origin aarch64-macosx

After that, a normal:

git push

is sufficient when working on a branch that has its upstream configured.

Build automation

The first goal is to make the packaging work reliably using already available native libraries.

After that, main can grow into a build orchestrator using Racket itself.

The intended direction is:

Windows machine
    -> build x86_64-win32

Linux x86_64 machine
    -> build x86_64-linux

Raspberry Pi 64-bit
    -> build aarch64-linux

Apple Silicon Mac
    -> build aarch64-macosx

The build orchestration can use racket-makefile, the Racket git module and, for remote targets, SSH or another small remote-execution layer.

The native libraries should preferably be built on the target architecture instead of cross-compiled, unless there is a good reason to do otherwise.

Versioning

The version in each platform package should correspond to the native library set delivered by that branch.

When changing the packaged native libraries, update the package version where appropriate and keep the meta-package dependency versions compatible.

Licenses

Native libraries may have licenses that differ from the license of the Racket wrapper.

Before publishing a new library or version, verify:

- the upstream license;
- whether binary redistribution is permitted;
- whether license or copyright files must be included;
- whether additional runtime libraries have separate license requirements.

Do not assume that the license of racket-audio automatically covers the native binaries.