mark/surplus
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity

meta: add surplus on wheels (#43)

Browse source
This commit is contained in:
Mark Joshwel 2023-10-30 01:38:49 +08:00 committed by GitHub
parent b21ddc0eeb
commit c10c013167
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 571 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

198
README.md
View file

@ -5,6 +5,7 @@ surplus is a Python script to convert
to iOS Shortcuts-like shareable text.
- [installation](#installation)
- [on Termux: surplus on wheels](#on-termux-surplus-on-wheels)
- [usage](#usage)
- [command-line usage](#command-line-usage)
- [example api usage](#example-api-usage)
@ -51,6 +52,197 @@ feel free to grab that and embed it into your own program as you see fit.
see [licence](#licence) for licensing information.
### on Termux: surplus on wheels
surplus on wheels (s+ow) is a pure shell script to get your location using
`termux-location`, process it through surplus, and send it to a WhatsApp user/group using a
[modified mdtest demonstration binary from the tulir/whatsmeow project](https://github.com/markjoshwel/whatsmeow-termux/tree/main/mdtest).
> [!IMPORTANT]
> if you just want to use surplus by itself, follow the normal installation guide above.
there are two ways to install and setup s+ow:
- [by itself](#by-itself)
- or [with an hourly cronjob](#with-an-hourly-cronjob)
see [s+ow usage instructions here](#usi).
#### by itself
1. firstly install python and termux-api if you haven't already:
```text
pkg install python termux-api
```
also install the accompanying the Termux:API app from [F-Froid](https://f-droid.org/en/packages/com.termux.api/).
2. install surplus:
```text
pip install https://github.com/markjoshwel/surplus/releases/latest/download/surplus-latest-py3-none-any.whl
```
3. install the modified mdtest binary for aarch64:
```text
wget https://github.com/markjoshwel/whatsmeow-termux/releases/latest/download/mdtest.tar.gz
tar -xvf mdtest.tar.gz
chmod +x mdtest
mkdir -p ~/.local/bin/
mv mdtest ~/.local/bin/
rm mdtest.tar.gz
```
4. install surplus on wheels:
```text
mkdir -p ~/.local/bin/
curl https://raw.githubusercontent.com/markjoshwel/surplus/s+ow/s+ow > ~/.local/bin/s+ow
chmod +x ~/.local/bin/s+ow
```
if `~/.local/bin` is not in your `$PATH`, add the following to your shell's rc file:
```shell
export PATH="$HOME/.local/bin:$PATH"
```
#### with an hourly cronjob
> [!IMPORTANT]
> these instructions rely on following the previous instructions, and assumes that s+ow works.
1. install necessary packages to run cron jobs:
```text
pkg install cronie termux-services
```
2. restart termux and start the cron service:
```text
sv-enable cron
```
3. setup the cron job:
> [!IMPORTANT]
> fill in the `JID_NOMINAL_TARGET` and `JID_ERRORED_TARGET` variables before running s+ow.
> [(see using s+ow)](#using-sow)
run the following command:
```text
crontab -e
```
and add the following text:
```text
59 * * * * (sleep 30; JID_NOMINAL_TARGET="" JID_ERRORED_TARGET="" LOCATION_PRIORITISE_NETWORK=n SPOW_CRON=y ~/.local/bin/s+ow)
```
this will run s+ow every hour, thirty seconds before a new hour. modify the variables
as per your needs. see [using s+ow](#using-sow) for more information.
#### using s+ow
for first-time setup of mdtest, run the following command and pair your WhatsApp account
with mdtest:
```text
~/.local/bin/s+ow mdtest
```
wait for mdtest to sync with WhatsApp. you can safely leave after a minute or after the
console stops moving. whichever comes first.
s+ow uses two environment variables:
1. `JID_NOMINAL_TARGET`
JID of the WhatsApp user/group to send the location to if everything runs correctly.
2. `JID_ERRORED_TARGET`
JID of the WhatsApp user/group to send the stderr/logs to if something goes wrong.
3. `SPOW_CRON`
set as non-empty to declare that s+ow is being run as a cron job.
cron jobs are run thirty seconds in advance to attempt to display surplus output
on time as waiting for a GPS lock may be slow.
4. `LOCATION_PRIORITISE_NETWORK`
set as non-empty to declare that s+ow can just use network location instead of GPS
if GPS is taking too long.
you should only turn this on if punctuality means that much to you, or you’re in a
country with cell towers close by or everywhere, like Singapore.
setting it to `n` will also be treated as empty.
the JIDs can be obtained by sending a message to the user/group, while running
`s+ow mdtest`, and examining the output for your message. JIDs are email address-like
strings.
you can fake your s+ow messages by either:
1. setting a dummy `last` file in s+ow cache
`$HOME/.cache/s+ow/last` is used as the fallback response when a part of s+ow (either
`termux-location` or `surplus` errors out). you can set this file to whatever you want
and just disable location permissions for Termux.
2. setting a `fake` file in s+ow cache
> [!IMPORTANT]
> this is currently unimplemented.
you can also write text to `$HOME/.cache/s+ow/fake` to fake upcoming messages. the file
is delimited by two newlines. as such, arrange the file like so:
```text
The Clementi Mall
3155 Commonwealth Avenue West
Westpeak Terrace
129588
Southwest, Singapore
Westgate
3 Gateway Drive
Jurong East
608532
Southwest, Singapore
...
```
on every run of s+ow, the first group of lines will be consumed, and the file will be
updated with the remaining lines. if the file is empty, it will be deleted.
#### quick install scripts
> [!WARNING]
> these scripts assume you're starting from a fresh base install of Termux.
> if you have already cron jobs, then manually carry out the instructiions in
> [with an hourly cronjob](#with-an-hourly-cronjob).
1. setup s+ow:
```text
curl https://raw.githubusercontent.com/markjoshwel/surplus/s+ow/termux-s+ow-setup | sh
```
2. restart termux
3. setup cron job:
```text
curl https://raw.githubusercontent.com/markjoshwel/surplus/s+ow/termux-s+ow-setup-cron | sh
```
you can then run `crontab -e` to edit the variables as per your needs.
see [using s+ow](#using-sow) for more information.
## usage
### command-line usage
@ -317,7 +509,7 @@ variables
split_query -> ['Temasek', 'Polytechnic']
original_query -> 'Temasek Polytechnic'
```
```text
>>> surplus("77Q4+7X Austin, Texas, USA", surplus.Behaviour())
@ -624,7 +816,7 @@ line breakdown of shareable text output, accompanied by their Nominatim keys:
}
```
- `SHAREABLE_TEXT_LINE_SETTINGS: dict[str, dict[int, tuple[str, bool]]]`
- `SHAREABLE_TEXT_LINE_SETTINGS: dict[str, dict[int, tuple[str, bool]]]`
a dictionary of iso3166-2 country-portion string keys with a dictionary as their values
@ -650,7 +842,7 @@ line breakdown of shareable text output, accompanied by their Nominatim keys:
```
- `SHAREABLE_TEXT_NAMES: dict[str, tuple[str, ...]]`
a dictionary of iso3166-2 country-portion string keys with a tuple of strings as their
values
a tuple of strings containing Nominatim keys used in shareable text line 0-2 and

346
s+ow Executable file
View file

@ -0,0 +1,346 @@
#!/bin/sh
# surplus on wheels (s+ow): a pure shell script to run surplus with mdtest using the termux-api
# shellcheck disable=SC2059
LOCATION_FALLBACK="%d%d%d\nSingapore?"
# shellcheck disable=SC2269
LOCATION_PRIORITISE_NETWORK="$LOCATION_PRIORITISE_NETWORK"
LOCATION_TIMEOUT=${LOCATION_TIMEOUT:-50}
# shellcheck disable=SC2269
JID_NOMINAL_TARGET="$JID_NOMINAL_TARGET"
# shellcheck disable=SC2269
JID_ERRORED_TARGET="$JID_ERRORED_TARGET"
MDTEST_BIN="$HOME/.local/bin/mdtest"
MDTEST_DIR="$HOME/.local/share/mdtest"
SPOW_CACHE_DIR="$HOME/.cache/s+ow"
# per-tool session logs
SPOW_NETLC_OUT="$SPOW_CACHE_DIR/location.net.json"
SPOW_GPSLC_OUT="$SPOW_CACHE_DIR/location.gps.json"
SPOW_LOCTN_OUT="$SPOW_CACHE_DIR/location.json"
SPOW_SPLUS_OUT="$SPOW_CACHE_DIR/surplus.out.log"
SPOW_SPLUS_ERR="$SPOW_CACHE_DIR/surplus.err.log"
# per-session collated logs
SPOW_SESH_OUT="$SPOW_CACHE_DIR/out.log"
SPOW_SESH_ERR="$SPOW_CACHE_DIR/err.log"
# per-week collated logs
SPOW_WEEK_PRE="$SPOW_CACHE_DIR/$(date +%Y)W$(date +"%V")"
SPOW_WEEK_OUT="$SPOW_WEEK_PRE.out.log"
SPOW_WEEK_ERR="$SPOW_WEEK_PRE.err.log"
# last successful surplus output
SPOW_LAST_OUT="$SPOW_CACHE_DIR/last"
# list of fakes
# shellcheck disable=SC2034
SPOW_FAKE_OUT="$SPOW_CACHE_DIR/fake"
# check for network location priority
if [ "$LOCATION_PRIORITISE_NETWORK" = "n" ]; then
LOCATION_PRIORITISE_NETWORK=""
fi
# ensure commands exist
if ! command -v termux-location >/dev/null 2>&1; then
printf "s+ow: error: termux-location is not installed.\ninstall it with 'pkg install termux-api' and with installing the termux:api app from the play store or f-droid.\n"
exit 1
fi
if ! command -v surplus >/dev/null 2>&1; then
printf "s+ow: error: surplus is not installed.\ninstall it with 'pip install https://github.com/markjoshwel/surplus/releases/latest/download/surplus-latest-py3-none-any.whl'\n"
exit 1
fi
if ! command -v ~/.local/bin/mdtest >/dev/null 2>&1; then
printf "s+ow: error: mdtest is not installed.\ninstall it by getting a release or building it from source from https://github.com/markjoshwel/whatsmeow-termux\n"
exit 1
fi
# ensure directories
mkdir -p "$SPOW_CACHE_DIR" "$MDTEST_DIR"
# create new session logs
rm -f "$SPOW_LOCTN_OUT" "$SPOW_SPLUS_OUT" "$SPOW_SPLUS_ERR" \
"$SPOW_SESH_OUT" "$SPOW_SESH_ERR"
touch "$SPOW_NETLC_OUT" "$SPOW_GPSLC_OUT" "$SPOW_LOCTN_OUT" \
"$SPOW_SPLUS_OUT" "$SPOW_SPLUS_ERR" \
"$SPOW_SESH_OUT" "$SPOW_SESH_ERR" \
"$SPOW_WEEK_OUT" "$SPOW_WEEK_ERR"
status=0 # 0 is nominal
# 1 is an termux-location error
# 2 is a surplus error
# helper functions
locate() {
# spawn termux-location processes
(
termux-location -p "network" >"$SPOW_NETLC_OUT"
if [ -s "$SPOW_NETLC_OUT" ]; then
printf "net" | tee -a "$SPOW_SESH_ERR"
else
printf "net?" | tee -a "$SPOW_SESH_ERR"
fi
cat "$SPOW_NETLC_OUT" >> "$SPOW_SESH_OUT"
) &
tl_net_pid="$!"
sleep 1
(
termux-location -p "gps" >"$SPOW_GPSLC_OUT"
if [ -s "$SPOW_GPSLC_OUT" ]; then
printf "gps" | tee -a "$SPOW_SESH_ERR"
else
printf "gps?" | tee -a "$SPOW_SESH_ERR"
fi
cat "$SPOW_GPSLC_OUT" >> "$SPOW_SESH_OUT"
) &
tl_gps_pid="$!"
# wait until timeout or both finished
printf "running termux-location" | tee -a "$SPOW_SESH_ERR"
while [ "$LOCATION_TIMEOUT" -gt 0 ]; do
# get process statuses
kill -0 "$tl_net_pid" >/dev/null 2>&1
tl_net_status="$?"
kill -0 "$tl_gps_pid" >/dev/null 2>&1
tl_gps_status="$?"
# break if both finished
if [ "$tl_net_status" -eq 1 ] && [ "$tl_gps_status" -eq 1 ]; then
break
fi
# exception: if network is proritised: just use that
if [ "$tl_net_status" -eq 1 ] && [ -n "$LOCATION_PRIORITISE_NETWORK" ]; then
# break only if theres an actual response
if [ -s "$SPOW_NETLC_OUT" ]; then
break
fi
# else just keep on waiting for gps to finish
fi
sleep 1
printf "." | tee -a "$SPOW_SESH_ERR"
LOCATION_TIMEOUT=$((LOCATION_TIMEOUT - 1))
done
if [ "$LOCATION_TIMEOUT" -eq 0 ]; then
printf " errored (timeout)\n" | tee -a "$SPOW_SESH_ERR"
else
printf " nominal\n" | tee -a "$SPOW_SESH_ERR"
fi
# check outputs
printf "determining output: " | tee -a "$SPOW_SESH_ERR"
if [ -s "$SPOW_NETLC_OUT" ] && [ -s "$SPOW_GPSLC_OUT" ]; then
printf "both succeeded, "
acc_net="$(grep "\"accuracy\"" <"$SPOW_NETLC_OUT" | awk -F ': ' '{print $2}' | tr -d ',')"
acc_gps="$(grep "\"accuracy\"" <"$SPOW_GPSLC_OUT" | awk -F ': ' '{print $2}' | tr -d ',')"
# compare accuracy
if awk -v n1="$acc_net" -v n2="$acc_gps" 'BEGIN { if (n1 < n2) exit 0; else exit 1; }'; then
printf "choosing network (%s < %s)" "$acc_net" "$acc_gps" | tee -a "$SPOW_SESH_ERR"
cat "$SPOW_NETLC_OUT" >"$SPOW_LOCTN_OUT"
else
printf "choosing gps (%s < %s)" "$acc_gps" "$acc_net" | tee -a "$SPOW_SESH_ERR"
cat "$SPOW_GPSLC_OUT" >"$SPOW_LOCTN_OUT"
fi
cat "$SPOW_GPSLC_OUT" >"$SPOW_LOCTN_OUT"
else
# one or none succeeded
if [ -s "$SPOW_NETLC_OUT" ]; then
if [ -n "$LOCATION_PRIORITISE_NETWORK" ]; then
printf "using network (prioritised)" | tee -a "$SPOW_SESH_ERR"
else
printf "using network" | tee -a "$SPOW_SESH_ERR"
fi
cat "$SPOW_NETLC_OUT" >"$SPOW_LOCTN_OUT"
fi
if [ -s "$SPOW_GPSLC_OUT" ]; then
printf "using gps" | tee -a "$SPOW_SESH_ERR"
cat "$SPOW_GPSLC_OUT" >"$SPOW_LOCTN_OUT"
fi
fi
if [ ! -s "$SPOW_LOCTN_OUT" ]; then
printf "none (error)" | tee -a "$SPOW_SESH_ERR"
fi
printf "\n" | tee -a "$SPOW_SESH_ERR"
}
gensharetext() {
surplus -td "$1" >"$SPOW_SPLUS_OUT" 2>"$SPOW_SPLUS_ERR"
ret="$?"
cat "$SPOW_SPLUS_OUT" >>"$SPOW_SESH_OUT"
cat "$SPOW_SPLUS_ERR" >>"$SPOW_SESH_ERR"
return "$ret"
}
send() {
(cd "$MDTEST_DIR" && "$MDTEST_BIN" send "$1" "$2")
}
notify_start() {
termux-notification \
--priority "min" \
--ongoing \
--id "s+ow" \
--title "surplus on wheels" \
--content "s+ow has started running."
}
notify() {
# $1 is text
# $2 is attempt number (if any)
attempt_text="$1"
if [ $# -eq 2 ]; then
attempt_text="$1 (attempt $2)"
fi
termux-notification \
--priority "min" \
--ongoing \
--id "s+ow" \
--title "surplus on wheels" \
--content "$attempt_text"
}
notify_end() {
# $1 is s+ow status (0, 1, 2)
# $2 is termux-location run number
# $3 is sent type (0, 1, 2)
# $4 is sharetext
termux-notification \
--priority "min" \
--id "s+ow" \
--title "surplus on wheels" \
--content "$(printf 'Run has finished. (%d, %d, %d)\n\n%s' "$1" "$2" "$3" "$4")"
}
# program functions
mdtest() {
(cd "$MDTEST_DIR" && "$MDTEST_BIN")
}
run() {
notify_start
printf "[run! stdout (%s)]\n" "$(date)" >>"$SPOW_SESH_OUT"
printf "[run! stderr (%s)]\n" "$(date)" >>"$SPOW_SESH_ERR"
# termux-location
location=""
for locate_run in 1 2 3; do # run three times in case :p
notify "Running termux-location" "$locate_run"
locate
if [ ! -s "$SPOW_LOCTN_OUT" ]; then
# erroneous: is empty
echo "s+ow: error: failed to get location" >>"$SPOW_SESH_ERR"
status=1
else
# nominal: is not empty
location="$(cat "$SPOW_LOCTN_OUT")"
status=0
break
fi
done
# surplus
printf "running surplus... "
notify "Running surplus -td $location"
if [ "$status" -eq 0 ]; then
if gensharetext "$location"; then
# surplus ran nominally
cp "$SPOW_SPLUS_OUT" "$SPOW_LAST_OUT"
status=0
printf "nominal\n"
else
# something happened :^)
status=2
printf "errored\n"
fi
else
printf "skipped\n"
fi
# if cron: wait until its the new hour
if [ -n "$SPOW_CRON" ]; then
printf "waiting until the new hour...\n"
while [ "$(date +'%M')" -eq 59 ]; do
printf " $(date)\n"
sleep 1
done
printf "done\n"
fi
# mdtest/send message
printf "sending message(s)... "
notify "Sending message(s)"
sent_type=0 # 0 for freshly made sharetext
# 1 for recycling a last location
# 2 for using fallback template
sharetext=""
if [ "$status" -eq 0 ]; then
# s+ow has behaved nominally until now, send as per normal
sharetext="$(cat "$SPOW_SPLUS_OUT")"
printf "\n"
send "$JID_NOMINAL_TARGET" "$sharetext"
else
# something has gone wrong, send an appropriate fallback
sharetext=""
if [ -s "$SPOW_LAST_OUT" ]; then
# use last successful location
sharetext="$(cat "$SPOW_LAST_OUT")"
sent_type=1
printf "using last...\n"
else
# no last location, use fallback
# shellcheck disable=SC2059
sharetext="$(printf "$LOCATION_FALLBACK" "$status" "$locate_run" "$sent_type")"
sent_type=2
printf "using fallback... \n"
fi
send "$JID_NOMINAL_TARGET" "$sharetext"
send "$JID_ERRORED_TARGET" "$(cat "$SPOW_SESH_ERR")"
fi
done_msg="$(printf "done (%d, %d, %d)\n" "$status" "$locate_run" "$sent_type")"
echo "$done_msg"
echo "$done_msg" >>"$SPOW_SESH_ERR"
# cleanup
printf "%s\n\n" "$(cat "$SPOW_SESH_OUT")" >>"$SPOW_WEEK_OUT"
printf "%s\n\n" "$(cat "$SPOW_SESH_ERR")" >>"$SPOW_WEEK_ERR"
notify_end "$status" "$locate_run" "$sent_type" "$sharetext"
}
# script entry
if [ "$1" = "mdtest" ]; then
mdtest
elif [ -z "$1" ]; then
# ensure JID targets are set
if [ -z "$JID_NOMINAL_TARGET" ] || [ -z "$JID_ERRORED_TARGET" ]; then
echo "s+ow: error: JID_NOMINAL_TARGET and JID_ERRORED_TARGET are not set"
exit 1
fi
run
else
echo "usage: $0 [mdtest]
surplus on wheels: a pure shell script to run surplus with mdtest using the termux-api
choices
$0 mdtest
run mdtest for testing or authentication
$0
run surplus on wheels normally"
fi

27
termux-s+ow-setup Normal file
View file

@ -0,0 +1,27 @@
#!/bin/sh
set -e
# get packages
yes | pkg upgrade
yes | pkg install python cronie termux-api termux-services wget
# install surplus
pip install https://github.com/markjoshwel/surplus/releases/latest/download/surplus-latest-py3-none-any.whl
# install whatsmeow
wget https://github.com/markjoshwel/whatsmeow-termux/releases/latest/download/mdtest.tar.gz
tar -xvf mdtest.tar.gz
chmod +x mdtest
mkdir -p ~/.local/bin/
mv mdtest ~/.local/bin/
rm mdtest.tar.gz
# install s+ow
mkdir -p ~/.local/bin/
curl https://raw.githubusercontent.com/markjoshwel/surplus/s+ow/s+ow > ~/.local/bin/s+ow
chmod +x ~/.local/bin/s+ow
# setup path
echo "export PATH=\$PATH:\$HOME/.local/bin/" >> ~/.profile
printf "\ns+ow setup complete\n"

3
termux-s+ow-setup-cron Normal file
View file

@ -0,0 +1,3 @@
#!/bin/sh
sv-enable crond
printf "59 * * * *\t(sleep 30; JID_NOMINAL_TARGET=\"\" JID_ERRORED_TARGET=\"\" LOCATION_PRIORITISE_NETWORK=n SPOW_CRON=y ~/.local/bin/s+ow)\n" | crontab -