Tiny music executables on Linux
category: code [glöplog]
Hello. I'm writing a tracker aiming for producing small executable music programs (< 4 KB). I use NASM for the core player.
I managed to find a way to compile tiny programs for 16-bit MS-DOS. But as I work primarily on Linux, I wanted to do the same thing for 32-bit ELFs.
However, I really struggle to find a way to get below 4 KB. Linking default audio library explodes the executable size, but without that it's very hard to produce any sound at all.
So, what are common practices to have sound in tiny Linux productions? I work on 64-bit Ubuntu 20, if that matters.
I'll be grateful for any references.
I managed to find a way to compile tiny programs for 16-bit MS-DOS. But as I work primarily on Linux, I wanted to do the same thing for 32-bit ELFs.
However, I really struggle to find a way to get below 4 KB. Linking default audio library explodes the executable size, but without that it's very hard to produce any sound at all.
So, what are common practices to have sound in tiny Linux productions? I work on 64-bit Ubuntu 20, if that matters.
I'll be grateful for any references.
Maybe you could check how it's done in this ?
Why not C then? I understand you're aiming for insanely low level control, but on any semi-serious OS, it's quite a gamble.
Linking libraries is… very unlikely to work well in a 4K. Your best shot is probably to open /dev/dsp and write stuff in there (possibly after some ioctls to set sample rate); OSS emulation should generally work even on modern systems.
you could look for an algorithm called "import by hash", also you want to write a binary encoder with an optimization called "section reordering". There are linkers that do this for you (crinkler, 1kpaq)
on linux it's a bit annoying tho. I recommend windows. but if you insist, talk to an epoqe member, they have a compressing linker that's competitive
@tomcat: the language they use for this doesn't matter, c, c++, asm, even rust has been done.
I prefer asm/nasm because you have the highest amount of control and it's not in your way with optimizations
on linux it's a bit annoying tho. I recommend windows. but if you insist, talk to an epoqe member, they have a compressing linker that's competitive
@tomcat: the language they use for this doesn't matter, c, c++, asm, even rust has been done.
I prefer asm/nasm because you have the highest amount of control and it's not in your way with optimizations
Hey, I suggest taking a look here: http://www.sizecoding.org/wiki/Linux#Adding_Sound
Thank you all for your replies!
I'll definitely take a look on that.
/dev/dsp seems obsolete nowadays. There are some interfaces though, but I'd go that way only if other options fail. Nevertheless, thanks for the suggestion.
This is nice, I'll do my research on that, thanks!
Eventually, I'll try to cover that OS too.
Quote:
Maybe you could check how it's done in this ?
I'll definitely take a look on that.
Quote:
Linking libraries is… very unlikely to work well in a 4K. Your best shot is probably to open /dev/dsp and write stuff in there (possibly after some ioctls to set sample rate); OSS emulation should generally work even on modern systems.
/dev/dsp seems obsolete nowadays. There are some interfaces though, but I'd go that way only if other options fail. Nevertheless, thanks for the suggestion.
Quote:
you could look for an algorithm called "import by hash", also you want to write a binary encoder with an optimization called "section reordering". There are linkers that do this for you (crinkler, 1kpaq)
This is nice, I'll do my research on that, thanks!
Quote:
I recommend windows
Eventually, I'll try to cover that OS too.
Quote:
/dev/dsp seems obsolete nowadays. There are some interfaces though, but I'd go that way only if other options fail. Nevertheless, thanks for the suggestion.
waveOut() is obsolete on Windows, too, but I don't think you'll see a lot of 4Ks using WASAPI nevertheless. 4K is so squeezed that you don't really care about what's obsolete, only what works.
1. `/dev/dsp` just doesn't exist on modern (last ~20 years) systems.
2. Don't link with libraries at link time. Use `dlopen()` and friends manually. It's way more compact.
3. If you're allowed to use SDL, specifically libsdl1, use `SDL_OpenAudio()`. It's basically the best audio out API out there -- just a single function, and then your callback gets called asking for more audio samples. See this example of 1k intro for Linux from 14 years ago that still works (if 32-bit support is enabled).
4. If SDL is not allowed, use PulseAudio simple API. It's blocking and not callback-based, so requires ~three calls: to create a thread, then inside that thread: init and then write the buffer (either all at once or in a loop). PulseAudio is the modern (last 15 years) audio API standard, it's not going anywhere (pipewire implements PulseAudio API as well).
5. I wouldn't recommend using ALSA directly: too verbose, weird compat issues, etc.
2. Don't link with libraries at link time. Use `dlopen()` and friends manually. It's way more compact.
3. If you're allowed to use SDL, specifically libsdl1, use `SDL_OpenAudio()`. It's basically the best audio out API out there -- just a single function, and then your callback gets called asking for more audio samples. See this example of 1k intro for Linux from 14 years ago that still works (if 32-bit support is enabled).
4. If SDL is not allowed, use PulseAudio simple API. It's blocking and not callback-based, so requires ~three calls: to create a thread, then inside that thread: init and then write the buffer (either all at once or in a loop). PulseAudio is the modern (last 15 years) audio API standard, it's not going anywhere (pipewire implements PulseAudio API as well).
5. I wouldn't recommend using ALSA directly: too verbose, weird compat issues, etc.
Our own compression linker cold is still not public yet, still some issues we want to resolve beforehand.
But there's smol which can be used together with onekpaq to achieve something quite similar, that also supports 32 bit, although I would recommend using 64 bit mode in order to work on an ubuntu or similar distro out of the box.
Theres the linux size coding wiki, where the techniques used by smol for dynamic linking are explained in depth. We're doing the same basically, the biggest difference being that we're not using mem_fd and execve but rather load the program interpreter (or ld runtime part) manually.
Some useful links:
https://gitlab.com/PoroCYon/4klang-linux
https://gitlab.com/PoroCYon/linux-4k-intro-template
https://in4k.github.io/wiki/lsc-wiki-rtld
But there's smol which can be used together with onekpaq to achieve something quite similar, that also supports 32 bit, although I would recommend using 64 bit mode in order to work on an ubuntu or similar distro out of the box.
Theres the linux size coding wiki, where the techniques used by smol for dynamic linking are explained in depth. We're doing the same basically, the biggest difference being that we're not using mem_fd and execve but rather load the program interpreter (or ld runtime part) manually.
Some useful links:
https://gitlab.com/PoroCYon/4klang-linux
https://gitlab.com/PoroCYon/linux-4k-intro-template
https://in4k.github.io/wiki/lsc-wiki-rtld
Oh, and blackles nutrition facts repo is useful for seeing onekpaq + smol in action:
https://github.com/Suricrasia-Online/Nutrition-Facts
https://github.com/Suricrasia-Online/Nutrition-Facts
Quote:
1. `/dev/dsp` just doesn't exist on modern (last ~20 years) systems.
It exists on mine :-) I don't think I installed anything funky, but it outputs data just fine into Pipewire.
This is a 2019 installation, so definitely within last 20 years.
/dev/dsp not available on ubuntu and (according to some stackoverflow comment) hasn't been since version 9
/dev/dsp is an obsolete OSS thing and only exists on modern systems if you run some kind of compatibility layer. These days, every linux system supports ALSA and the aplay command instead (whether directly or via pipewire). So the modern replacement for writing data to /dev/dsp is to just run aplay in a subprocess and pipe data to it. This can be accomplished in around 100 bytes. The Linux page on the sizecoding wiki discusses a few different ways to do it.
Thank you all once again for all materials.
I need to digest that :).
I need to digest that :).
Quote:
/dev/dsp is an obsolete OSS thing and only exists on modern systems if you run some kind of compatibility layer.
I keep being fascinated at that people assume that compatibility layers are somehow illegal to use in a 4K :-) I can understand the argument “those layers are not installed on the compo PC” (would almost certainly depend on the rules, just like the existence of /usr/bin/aplay), but just this repeated “it's obsolete” is very confusing to me. An open() call takes much less than 100 bytes, after all.
(And yes, of course I know the difference between OSS and ALSA and JACK and PulseAudio and PipeWire, and thankfully nobody needs to care about ESD anymore)
Quote:
I keep being fascinated at that people assume that compatibility layers are somehow illegal to use in a 4K :-) I can understand the argument “those layers are not installed on the compo PC” (would almost certainly depend on the rules, just like the existence of /usr/bin/aplay), but just this repeated “it's obsolete” is very confusing to me. An open() call takes much less than 100 bytes, after all.
The main issue with using "obsolete" stuff and compat layers is that it drastically diminishes the accessibility of a prod. It just doesn't look very appealing to do unguided research, install 100s of MiBs of support libraries and tools, and then do involved system-specific configuration step (with unclear prospect of success) to run a single few KiBs intro for a minute or two.
My <=1k Linux intros (which already at the time of their releases depended on very shaky and not even always available for all HW `/dev/fb` and `/dev/dsp` compat layers) aged like fine milk in that regard.
Windows intros also do "illegal" and "obsolete" tricks, but these have been polished by decades of intense community focus.
For Linux it's just a pile of incoherent independently discovered ad-hoc tricks.
That being said, if we were to discuss what's the minimal "Linux small intro standard base", then I'd say that it should be just libsdl1 availability, and that's it. It's the cheapest way to get both GL context and audio output, consistently and on all systems (X11, Wayland, and KMS+GBM directly even; Pulse, Pipe, ALSA, ...). The only concern for me is whether it itself is going to be deprecated and phased away, given that SDL2 and 3 exist now.
I wrote a library a while back to handle audio output for Linux specifically for sizecoding usage. It gives you an interface similar to shadertoy's audio shader thing. It supports portaudio, alsa and aplay piping as backends.
hope this helps out!
hope this helps out!
Quote:
Our own compression linker cold is still not public yet, still some issues we want to resolve beforehand.
But there's smol which can be used together with onekpaq to achieve something quite similar, that also supports 32 bit, although I would recommend using 64 bit mode in order to work on an ubuntu or similar distro out of the box.
Theres the linux size coding wiki, where the techniques used by smol for dynamic linking are explained in depth. We're doing the same basically, the biggest difference being that we're not using mem_fd and execve but rather load the program interpreter (or ld runtime part) manually.
Some useful links:
https://gitlab.com/PoroCYon/4klang-linux
https://gitlab.com/PoroCYon/linux-4k-intro-template
https://in4k.github.io/wiki/lsc-wiki-rtld
FYI what I used for https://www.pouet.net/prod.php?which=82613 (source code is public btw) is exactly the "pipe,fork,dup2,execve" with aplay method described in http://www.sizecoding.org/wiki/Linux#Adding_Sound
using 64-bit ELF binaries is required *if* you're doing dynamic linking, because what's available by default (= what's on the compomachine) are only the 64-bit libraries, no 32-bit ones. however, because the pipe-to-aplay method can be done using raw syscalls, the binary can still be 32-bit
this is exactly how I used 4klang, oidos, V2, etc. (which are all written in 32-bit assembly) on Linux: inside the intro binary (which has to be 64-bit for dynamic linking to OpenGL), embed a *second*, statically-linked 32-bit binary that does the audio, and launch that and do some synchronization bullshit. Of course, if you're not doing any graphics, you don't need to have an "outer" 64-bit binary at all of course.
Open Source is what original "sceners" were about. Sourcers maybe, they didn't call it "Scener". That has a humorous association to a hatted entity aswel. Maybe it should be taken over!
A good
--------------- sample(/multisample)>18dbFLT>env>FX Sends
tracker could be good.
A moden two band master with basssaturation/compression to clip gives easy loudness aswell.
V
A good
--------------- sample(/multisample)>18dbFLT>env>FX Sends
tracker could be good.
A moden two band master with basssaturation/compression to clip gives easy loudness aswell.
V
If you're interested in the byte-wrangling side of tiny ELFs, check out "A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux". It goes down to 42 bytes, but 64 bytes is comfortable. I've used it for BGGP5 (binary golf grand prix) last year.
Sorry, 4*5* bytes (on 32-bit ELF) is the limit in the Teensy ELF tutorial
Quote:
Windows intros also do "illegal" and "obsolete" tricks, but these have been polished by decades of intense community focus.
For Linux it's just a pile of incoherent independently discovered ad-hoc tricks.
Well, with dnload, smol+onekpaq and now cold (while unlord is also working with something), I'd like to think we're arriving at the polish-from-community-focus.
Quote:
Quote:/dev/dsp is an obsolete OSS thing and only exists on modern systems if you run some kind of compatibility layer.
I keep being fascinated at that people assume that compatibility layers are somehow illegal to use in a 4K :-) I can understand the argument “those layers are not installed on the compo PC” (would almost certainly depend on the rules, just like the existence of /usr/bin/aplay), but just this repeated “it's obsolete” is very confusing to me. An open() call takes much less than 100 bytes, after all.
how to run a /dev/dsp and /dev/fb0 demo, a guide:
- modprobe snd_pcm_oss
- notice the module isn't available on your system/distro at all ("but who uses ubuntu?" the compomachine does)
- recompile your kernel, now with the module enabled
- modprobe the vfb driver to have a framebuffer you can watch (using a viewer program) while inside your X11/Wayland desktop
- demo hardcoded to use /dev/fb0, so you can't use vfb anyway
- go back to textmode
- /dev/fb0 has strict permission checks, so now the easiest way to proceed is to run the demo as root
it's not just an annoyance for end-users, I doubt many compo orgas (who are probably more Windows-minded to begin with) want to deal with this bullshit either