surplus/docs/developing.md

338 lines
12 KiB
Markdown
Raw Normal View History

2024-06-18 10:59:55 +00:00
# the developers handbook
!!! abstract
i (mark), heavily use nix to manage my projects, either with
[devbox](https://github.com/jetpack-io/devbox) or flakes
if you are going to develop for surplus or its' sibling projects (except surplus on wheels,
which only needs `shfmt` and `shellcheck`), i would recommend you install Nix using
[Determinate Systems' Nix Installer](https://github.com/DeterminateSystems/nix-installer):
```text
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
```
if i, a very very inexperienced Nix and NixOS user to give a rundown on why to use it:
1. **environments and builds are reproducible**
nix is part package manager, operating system (as NixOS), and functional programming
language. because it's functional, having X as an input will always produce Y as an output,
no matter what. even in the event of environmental, social, economic, or structural collapse
**what does this mean for you?**
when i use `nix develop` and you use `nix develop` to start a development environment, your
version of python will be the same as my version of python. your version of go will be same
as my version of go, etc
if i can build it, and the locked inputs of the nix flake are still available on the internet,
you can build it too
2. **the nix store**, located literally at `/nix/store`
it's where it stores every package you need, separate and isolated from other packages.
lets say you have a tool that needs python (3.8), and another tool that needs python (3.11).
nix store will download and store the binaries for both python installations, instead of
sharing the earliest downloaded python version for both tools
**what does this mean for you?**
whatever project you're working on that can use nix for development environments and builds
will not dirty anything else on your system. any build dependencies of surplus provided with
`nix develop` will **not** mess up software installed for other projects or even the system.
neat, if you ask me tbh
**tl;dr**: things will just werk with nix. but if you see all of this and go, "eh. i can manage
what i install.", then power to you! i list down exactly what prerequisite software needs to be
installed for each project anyways, so have fun! (●'◡'●)
## surplus (and Documentation)
### environment setup
!!! note
all prerequisite software are available in a nix flake. if you want a reproducible environment,
run `nix develop --impure` in the repository root
- for NixOS users, [nix-ld](https://github.com/Mic92/nix-ld) is needed. if you use flakes to
declare your system, follow accordingly. else, use `/etc/nixos/configuration.nix`
for NixOS-on-WSL users, use [nix-ld-rs](https://github.com/nix-community/nix-ld-rs)
once you're done installing `nix-ld` or `nix-ld-rs`,
don't forget to run `sudo nixos-rebuild switch`
prerequisite software:
- [Python](https://www.python.org/downloads/), 3.11 or newer
- [Hatch](https://hatch.pypa.io/latest/): dependency management and build tool
to start a development environment:
```text
hatch shell
```
for docs:
```text
hatch -e docs shell
```
### workflow for python code
TODO
### workflow for markdown documentation
run the documentation server with:
```text
hatch run docs:serve
```
i personally don't use a linter for markdown files, if it looks good on my code editor, then
whatever. if you're going to contribute back, i ask for three things:
- run it through a spell checker or something similar
- line limit of 100
- should be readable as-is on a code editor, **not the markdown preview pane**.
my stance is, if you can afford a fancy preview of the markdown file, use the nice-ned
documentation website. else, read it as a plaintext file
(make it look pretty on the doc site and in plaintext)
---
## surplus on wheels
### environment setup
!!! note
all prerequisite software are available in a nix flake. if you want a reproducible environment,
run `nix develop` in `src/surplus-on-wheels`
prerequisite software:
- [shfmt](https://github.com/patrickvane/shfmt): formatter
- [ShellCheck](https://www.shellcheck.net/): static analyser
### workflow
2024-06-18 17:30:04 +00:00
!!! note
alternatively, run `check.sh` inside `src/surplus-on-wheels`
2024-06-18 10:59:55 +00:00
- formatting s+ow:
- run `shfmt s+ow > s+ow.new`
- mv `s+ow.new` into `s+ow`
sometimes when piping shfmt's output immediately into the same file
results in the file being empty :(
- checking s+ow:
- run `shellcheck s+ow`
if there's no output, that means it passed :)
- if commiting back into the repository, try it out on your Termux system for a day or two,
just to make sure it runs correctly
---
## surplus on wheels: Telegram Bridge
### environment setup
!!! note
all prerequisite software are available in a nix flake. if you want a reproducible environment,
run `nix develop` in `src/spow-telegram-bridge`. it uses
[poetry2nix](https://github.com/nix-community/poetry2nix), so you won't need to run
`poetry shell` afterwards. if you've changed the `pyproject.toml` file,
just exit and re-run `nix develop`
prerequisite software:
- [Python](https://www.python.org/downloads/), 3.11 or newer
- [Poetry](https://python-poetry.org/): dependency management and build tool
to start a development environment:
```text
poetry shell
```
### workflow
after modifying,
2024-06-18 17:30:04 +00:00
1. check the source code:
1. `mypy bridge.py`
2. `ruff format bridge.py`
3. `ruff check bridge.py`
!!! note
alternatively, run `check.sh` inside `src/spow-telegram-bridge`
2. and then [test the binary](#workflow-for-testing-the-binary)
2024-06-18 10:59:55 +00:00
if the bridge behaves nominally, [bump the version](#versioning-surplus-on-wheels-telegram-bridge)
and commit!
---
## surplus on wheels: WhatsApp Bridge
### environment setup
!!! note
all prerequisite software are available in a nix flake. if you want a reproducible environment,
run `nix develop` in `src/spow-whatsapp-bridge`
the flake will pull in the Android SDK and NDK for building on Termux, and as such can only be
ran on `x86_64-linux` and `x86_64-darwin`
prerequisite software:
- [Go](https://go.dev): 1.22 or newer
- [Android NDK](https://developer.android.com/ndk/downloads), if building for Termux
### workflow for modifying bridge code
the bridge's code is just modified [mdtest](https://github.com/tulir/whatsmeow/tree/main/mdtest)
code, and as such, whenever in doubt, do a diff between mdtest and the bridge code
after modifying,
2024-06-18 17:30:04 +00:00
1. check the source code:
1. `go fmt bridge.go`
2. `go vet bridge.go`
3. `golint bridge.go`
!!! note
alternatively, run `check.sh` inside `src/spow-whatsapp-bridge`
2. [build a binary](#workflow-for-building-a-binary)
3. [test the binary](#workflow-for-testing-the-binary)
4. and if all goes well, [bump the version](#versioning-surplus-on-wheels-whatsapp-bridge)
2024-06-18 10:59:55 +00:00
and commit!
### workflow for bumping dependencies
- check with your editor, plugin, or online if there's newer patch/minor (see
[semantic versioning](https://semver.org/)) versions to update to
- change the `go.mod` accordingly
after bumping,
1. [build a binary](#workflow-for-building-a-binary)
2. [test the binary](#workflow-for-testing-the-binary)
3. and if all goes well, [bump the version](#versioning-surplus-on-wheels-whatsapp-bridge)
and commit!
### workflow for building a binary
ensure you already have c compiler on the system (if you're using `nix develop` then yes you do), then run:
```text
CGO_ENABLED=1 go build
```
nix users can alternatively run:
```text
2024-06-18 17:30:04 +00:00
nix build
2024-06-18 10:59:55 +00:00
```
instructions to build a Termux build are located at the
[bridges' documentation page](onwheels/whatsapp-bridge.md#anywhere-else), however nix users can run
the following instead for a reproducible, deterministic and hermetic build command:
```text
nix build .#termux
```
the resulting build will be in `result/spow-whatsapp-bridge`
### workflow for testing the binary
- test it out, making sure that you write dummy test text to `~/.cache/s+ow/message` before running
the binary
1. run `s+ow-whatsapp-bridge login` first
2. run `s+ow-whatsapp-bridge list` if you don't already have a chat ID
to send the test message to
3. run `s+ow-whatsapp-bridge` type or copy and paste in a `wa:`-prefixed chat ID
after it logs in, and verify it sends
if the bridge behaves nominally, [bump the version](#versioning-surplus-on-wheels-whatsapp-bridge)
and commit!
## workflow for versioning and tagging releases
### versioning surplus
format: `YEAR.MAJOR.MINOR[-PRERELEASE]` ([semantic versioning](https://semver.org/))
example: `2024.0.0`, `2024.0.0-beta`
change: update the `__version__` variable in `src/surplus/surplus.py`
### versioning surplus on wheels
i've tried to make surplus on wheels as reliable as it could be given a POSIX compliant shell and
commands you'd find available on virtually every linux system, Termux included
as such, it doesn't really follow a versioning scheme as it doesn't need to. also there's no
automatic updater for it, which would be overkill anyway
### versioning surplus on wheels: Telegram Bridge
format: `REVISION.YYYY.WW[+BUILD]` ([calendar versioning](https://calver.org/))
example: `2.2024.24`, `2.2024.24+1`
change: `version` key in `src/spow-telegram-bridge/pyproject.toml`
`REVISION` here meaning any general revision/change
the Telegram Bridge relies on [Telethon](https://github.com/LonamiWebs/Telethon/), which also
follows [semantic versioning](https://semver.org/). so, as long as major isn't bumped, or
as long as Telegram doesn't become Discord, the MTProto APIs to talk to Telegram should be
stable.
however because Telethon also relies on a bunch of networking libraries, it made some sense to
still do weekly builds to bump dependencies, getting pipx to download the newest compatible
dependencies as compared to dubiously running some sort of script to `pipx inject` dependencies
under normal circumstances, a non-working version of the bridge would and **should not have a
version bump**. but for any reason if an already tagged bridge is faulty and/or erroneous in
normal/expected usage, add a revision number to the end after a period (see example above)
### versioning surplus on wheels: WhatsApp Bridge
format: `REVISION.YYYY.WW[+BUILD]` ([calendar versioning](https://calver.org/))
example: `2.2024.25`, `2.2024.25+1`
change: `version` attribute of `bridge` attribute set in `src/spow-whatsapp-bridge/flake.nix`
`REVISION` here meaning any general revision/change
the WhatsApp Bridge relies on [whatsmeow](https://github.com/tulir/whatsmeow), a rolling release
library due to the volatile, undocumented nature of WhatsApp's multidevice API and also directly
and indirectly relies on a bunch of networking libraries:
``` title="src/spow-whatsapp-bridge/go.mod"
--8<-- "src/spow-whatsapp-bridge/go.mod"
```
as such, it uses a calendar versioning scheme and is built weekly
under normal circumstances, a non-working version of the bridge would and **should not have a
version bump**. but for any reason if an already tagged bridge is faulty and/or erroneous in
normal/expected usage, add a revision number to the end after a period (see example above)
---
## i've made my changes. what now?
if you're contributing back to surplus and/or the sibling projects, firstly, thanks!
see [the contributor's handbook](contributing.md) for what's next