mark/surplus
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity

Merge pull request #24 from markjoshwel/type-type-conversion

Browse source 
type to type conversion
This commit is contained in:
Mark Joshwel 2023-09-07 01:48:01 +08:00 committed by GitHub
parent 14193a705e 619f4d71ae
commit 6415802f43
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
6 changed files with 1444 additions and 315 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

457
README.md
View file

@ -22,6 +22,7 @@ to iOS Shortcuts-like shareable text.
- [what counts as "incorrect"](#what-counts-as-incorrect)
- [output technical details](#the-technical-details-of-surpluss-output)
- [api reference](#api-reference)
- [details on the fingerprinted user agent](#details-on-the-fingerprinted-user-agent)
- [licence](#licence)
```text
@ -62,27 +63,29 @@ see [licence](#licence) for licensing information.
### command-line usage
```text
usage: surplus [-h] [-d] [-v] [-c {pluscode,localcode,latlong,string}]
usage: surplus [-h] [-d] [-v] [-c {pluscode,localcode,latlong,sharetext}]
[-u USER_AGENT]
[query ...]
Google Maps Plus Code to iOS Shortcuts-like shareable text
positional arguments:
query full-length Plus Code (6PH58QMF+FX),
shortened Plus Code/'local code' (8QMF+FX Singapore),
latlong (1.3336875, 103.7749375),
string query (e.g., 'Wisma Atria'),
or '-' to read from stdin
query full-length Plus Code (6PH58QMF+FX), shortened
Plus Code/'local code' (8QMF+FX Singapore),
latlong (1.3336875, 103.7749375), string query
(e.g., 'Wisma Atria'), or '-' to read from stdin
options:
-h, --help show this help message and exit
-d, --debug prints lat, long and reverser response dict to
stderr
-v, --version prints version information to stderr and exits
-c {pluscode,localcode,latlong,sharetext},
--convert-to {pluscode,localcode,latlong,sharetext}
-c {pluscode,localcode,latlong,sharetext}, --convert-to {pluscode,localcode,latlong,sharetext}
converts query a specific output type, defaults
to 'sharetext'
-u USER_AGENT, --user-agent USER_AGENT
user agent string to use for geocoding service,
defaults to fingerprinted user agent string
```
### example api usage
@ -193,7 +196,7 @@ and do the following:
before moving on.
2. include the erroneous query.
(_the Plus Code/local code/latlong coord/query string you passed into surplus_)
(_the Plus Code/local code/latlong coordinate/query string you passed into surplus_)
3. include output from the terminal with the
[`--debug` flag](#command-line-usage) passed to the surplus CLI or with
@ -264,12 +267,12 @@ of incorrect outputs.
```text
$ s+ --debug 8QJF+RP Singapore
surplus version 2.1.0, debug mode
surplus version 2.1.0, debug mode (latest@future, Tue 05 Sep 2023 23:38:59 +0800)
debug: parse_query: behaviour.query=['8QJF+RP', 'Singapore']
debug: _match_plus_code: portion_plus_code='8QJF+RP', portion_locality='Singapore'
debug: cli: query=Result(value=LocalCodeQuery(code='8QJF+RP', locality='Singapore'), error=None)
debug: cli: latlong.get()=Latlong(latitude=1.3320625, longitude=103.7743125)
debug: cli: location={'amenity': 'Ngee Ann Polytechnic', 'house_number': '535', 'road': 'Clementi Road', 'suburb': 'Bukit Timah', 'city': 'Singapore', 'county': 'Northwest', 'ISO3166-2-lvl6': 'SG-03', 'postcode': '599489', 'country': 'Singapore', 'country_code': 'sg', 'raw': "{...}", 'latitude': '1.33318835', 'longitude': '103.77461234638255'}
debug: latlong_result.get()=Latlong(latitude=1.3320625, longitude=103.7743125)
debug: location={...}
debug: _generate_text: seen_names=['Ngee Ann Polytechnic', 'Clementi Road']
debug: _generate_text_line: [True] -> True -------- 'Ngee Ann Polytechnic'
debug: _generate_text_line: [True] -> True -------- '535'
@ -295,32 +298,43 @@ Northwest, Singapore
variables
- **variable `behaviour.query`**
- **variables `behaviour.query`, `split_query` and `original_query`**
the original query string or a list of strings from space-splitting the original query
(_`split_query` and `original_query` are only shown if query is a latlong coordinate
or query string_)
`behaviour.query` is the original query string or a list of strings from space-splitting the original query
string passed to [`parse_query()`](#def-parse_query) for parsing
`split_query` is the original query string split by spaces
`original_query` is a single non-split string
```text
$ s+ 77Q4+7X Austin, Texas, USA
--------------------------
$ s+ Temasek Polytechnic
-------------------
query
behaviour.query -> ['77Q4+7X', 'Austin', 'Texas', 'USA']
behaviour.query -> ['Temasek', 'Polytechnic']
split_query -> ['Temasek', 'Polytechnic']
original_query -> 'Temasek Polytechnic'
```
```text
>>> surplus("77Q4+7X Austin, Texas, USA", surplus.Behaviour())
behaviour.query -> '77Q4+7X Austin, Texas, USA'
split_query -> ['77Q4+7X', 'Austin,', 'Texas,', 'USA']
original_query -> '77Q4+7X Austin, Texas, USA'
```
- **variables `portion_plus_code` and `portion_locality`**
(_only shown if the query is a local code, not shown on full-length plus codes,
(_only shown if the query is a local code, not shown on full-length Plus Codes,
latlong coordinates or string queries_)
represents the plus code and locality portions of a
[shortened plus code](https://en.wikipedia.org/wiki/Open_Location_Code#Common_usage_and_shortening)
represents the Plus Code and locality portions of a
[shortened Plus Code](https://en.wikipedia.org/wiki/Open_Location_Code#Common_usage_and_shortening)
(_referred to as a "local code" in the codebase_) respectively
- **variable `query`**
@ -330,23 +344,23 @@ variables
this variable is displayed to show what query type [`parse_query()`](#def-parse_query) has
recognised, and if there were any errors during query parsing
- **expression `latlong.get()=`**
- **expression `latlong_result.get()=`**
(_only shown if the query is a plus code_)
(_only shown if the query is a Plus Code_)
the latitude longitude coordinates derived from the plus code
the latitude longitude coordinates derived from the Plus Code
- **variable `location`**
the response dictionary from the reverser function passed to
[`surplus()`](#def-surplus)
for more information on the reverser function, see [`Behaviour`](#class-behaviour) and
[`default_reverser`](#def-default_reverser)
for more information on the reverser function, see
[`SurplusReverserProtocol`](#surplusreverserprotocol)
- **variable `seen_names`**
a list of unique important names found in certain nominatim keys used in final output
a list of unique important names found in certain Nominatim keys used in final output
lines 0-3
- **`_generate_text_line` seen name checks**
@ -525,7 +539,13 @@ line breakdown of shareable text output, accompanied by their Nominatim keys:
- [types](#types)
- [`Query`](#query)
- [`ResultType`](#resulttype)
- [`SurplusGeocoderProtocol`](#surplusgeocoderprotocol)
- [`SurplusReverserProtocol`](#surplusreverserprotocol)
- [`class Behaviour`](#class-behaviour)
- [`class SurplusDefaultGeocoding`](#class-surplusdefaultgeocoding)
- [`SurplusDefaultGeocoding.update_geocoding_functions()`](#surplusdefaultgeocodingupdate_geocoding_functions)
- [`SurplusDefaultGeocoding.geocoder()`](#surplusdefaultgeocodinggeocoder)
- [`SurplusDefaultGeocoding.reverser()`](#surplusdefaultgeocodingreverser)
- [`class ConversionResultTypeEnum`](#class-conversionresulttypeenum)
- [`class Result`](#class-result)
- [`Result.__bool__()`](#result__bool__)
@ -548,8 +568,8 @@ line breakdown of shareable text output, accompanied by their Nominatim keys:
- [`StringQuery.__str__()`](#stringquery__str__)
- [`def surplus()`](#def-surplus)
- [`def parse_query()`](#def-parse_query)
- [`def default_geocoder()`](#def-default_geocoder)
- [`def default_reverser()`](#def-default_reverser)
- [`def generate_fingerprinted_user_agent`](#def-generate_fingerprinted_user_agent)
- [details on the fingerprinted user agent](#details-on-the-fingerprinted-user-agent)
### constants
@ -558,30 +578,55 @@ line breakdown of shareable text output, accompanied by their Nominatim keys:
a tuple of integers representing the version of surplus, in the format
`[major, minor, patch]`
- `VERSION_SUFFIX: Final[str]`
`BUILD_BRANCH: Final[str]`
`BUILD_COMMIT: Final[str]`
`BUILD_DATETIME: Final[datetime]`
- `VERSION_SUFFIX: typing.Final[str]`
`BUILD_BRANCH: typing.Final[str]`
`BUILD_COMMIT: typing.Final[str]`
`BUILD_DATETIME: typing.Final[datetime]`
string and a [datetime.datetime](https://docs.python.org/3/library/datetime.html) object
containing version and build information, set by [releaser.py](releaser.py)
- `SHAREABLE_TEXT_LINE_0_KEYS: tuple[str, ...]`
`SHAREABLE_TEXT_LINE_1_KEYS: tuple[str, ...]`
`SHAREABLE_TEXT_LINE_2_KEYS: tuple[str, ...]`
`SHAREABLE_TEXT_LINE_3_KEYS: tuple[str, ...]`
`SHAREABLE_TEXT_LINE_4_KEYS: tuple[str, ...]`
`SHAREABLE_TEXT_LINE_5_KEYS: tuple[str, ...]`
`SHAREABLE_TEXT_LINE_6_KEYS: tuple[str, ...]`
- `CONNECTION_MAX_RETRIES: int = 9`
`CONNECTION_WAIT_SECONDS: int = 10`
a tuple of strings containing nominatim keys used in shareable text line 0-6
defines if and how many times to retry a connection, alongside how many seconds to wait
in between tries, for Nominatim
- `SHAREABLE_TEXT_NAMES: tuple[str, ...]`
> [!NOTE]
> this constant only affects the default surplus Nominatim geocoding functions. custom
> functions do not read from this, unless deliberately programmed to do so
a tuple of strings containing nominatim keys used in shareable text line 0-2 and
- `SHAREABLE_TEXT_LINE_0_KEYS: typing.Final[tuple[str, ...]]`
`SHAREABLE_TEXT_LINE_1_KEYS: typing.Final[tuple[str, ...]]`
`SHAREABLE_TEXT_LINE_2_KEYS: typing.Final[tuple[str, ...]]`
`SHAREABLE_TEXT_LINE_3_KEYS: typing.Final[tuple[str, ...]]`
`SHAREABLE_TEXT_LINE_4_KEYS: typing.Final[tuple[str, ...]]`
`SHAREABLE_TEXT_LINE_5_KEYS: typing.Final[tuple[str, ...]]`
`SHAREABLE_TEXT_LINE_6_KEYS: typing.Final[tuple[str, ...]]`
a tuple of strings containing Nominatim keys used in shareable text line 0-6
- `SHAREABLE_TEXT_NAMES: typing.Final[tuple[str, ...]]`
a tuple of strings containing Nominatim keys used in shareable text line 0-2 and
special keys in line 3
- `EMPTY_LATLONG: Latlong`
- `SHAREABLE_TEXT_LOCALITY: dict[str, tuple[str, ...]]`
a dictionary of iso3166-2 country-portion strings with a tuples of strings as their
values
used when generating the locality portions of shortened Plus Codes/local codes
```python
{
"default": (...),
"SG": (...,),
...
}
```
- `EMPTY_LATLONG: typing.Final[Latlong]`
a constant for an empty latlong coordinate, with latitude and longitude set to 0.0
### exception classes
@ -619,6 +664,91 @@ ResultType = TypeVar("ResultType")
[generic type](https://docs.python.org/3/library/typing.html#generics) used by
[`Result`](#class-result)
#### `SurplusGeocoderProtocol`
[typing_extensions.Protocol](https://mypy.readthedocs.io/en/stable/protocols.html#callback-protocols)
class for documentation and static type checking of surplus geocoder functions
- **signature and conforming function signature**
```python
class SurplusGeocoderProtocol(Protocol):
def __call__(self, place: str) -> Latlong:
...
```
functions that conform to this protocol should have the following signature:
```python
def example(place: str) -> Latlong: ...
```
- **information on conforming functions**
function takes in a location name as a string, and returns a [Latlong](#class-latlong).
**function MUST supply a `bounding_box` attribute to the to-be-returned
[Latlong](#class-latlong).** the bounding box is used when surplus shortens Plus Codes.
function can and should be at minimum
[`functools.lru_cache()`-wrapped](https://docs.python.org/3/library/functools.html#functools.lru_cache)
if the geocoding service asks for caching
exceptions are handled by the caller
#### `SurplusReverserProtocol`
[typing_extensions.Protocol](https://mypy.readthedocs.io/en/stable/protocols.html#callback-protocols)
class for documentation and static type checking of surplus reverser functions
- **signature and conforming function signature**
```python
class SurplusReverserProtocol(Protocol):
def __call__(self, latlong: Latlong, level: int = 18) -> dict[str, Any]:
...
```
functions that conform to this protocol should have the following signature:
```python
def example(self, latlong: Latlong, level: int = 18) -> dict[str, Any]: ...
```
- **information on conforming functions**
function takes in a [Latlong](#class-latlong) object and return a dictionary with [`SHAREABLE_TEXT_LINE_*_KEYS`](#constants) keys at the dictionaries' top-level.
keys are used to access address information.
function should also take in an int representing the level of detail for the returned
address, 0-18 (country-level to building), inclusive. should default to 18.
keys for latitude, longitude and an iso3166-2 (or closest equivalent) should also be
included at the dictionaries top level as the keys `latitude`, `longitude` and
`ISO3166-2` (non-case sensitive, or at least something starting with `ISO3166`)
respectively.
```python
{
'ISO3166-2-lvl6': 'SG-03',
'amenity': 'Ngee Ann Polytechnic',
...
'country': 'Singapore',
'latitude': 1.33318835,
'longitude': 103.77461234638255,
'postcode': '599489',
'raw': {...},
}
```
function can and should be at minimum
[`functools.lru_cache()`-wrapped](https://docs.python.org/3/library/functools.html#functools.lru_cache)
if the geocoding service asks for caching
see the [playground notebook](/playground.ipynb) in repository root for detailed
sample output
exceptions are handled by the caller
### `class Behaviour`
[`typing.NamedTuple`](https://docs.python.org/3/library/typing.html#typing.NamedTuple)
@ -630,15 +760,13 @@ attributes
original user-passed query string or a list of strings from splitting user-passed query
string by spaces
- `geocoder: typing.Callable[[str], Latlong] = default_geocoder`
name string to location function, must take in a string and return a
[`Latlong`](#class-latlong), exceptions are handled by the caller
- `geocoder: SurplusGeocoderProtocol = default_geocoding.geocoder`
name string to location function, see
[`SurplusGeocoderProtocol`](#surplusgeocoderprotocol) for more information
- `reverser: Callable[[Latlong], dict[str, Any]] = default_reverser`
[`Latlong`](#class-latlong) object to dictionary function, must take in a string and return a
dict. keys found in SHAREABLE_TEXT_LINE_*_KEYS used to access address details are placed
top-level in the dict, exceptions are handled by the caller.
see the [playground notebook](playground.ipynb) for example output
- `reverser: SurplusReverserProtocol = default_geocoding.reverser`
Latlong object to address information dictionary function, see
[`SurplusReverserProtocol`](#surplusreverserprotocol) for more information
- `stderr: typing.TextIO = sys.stderr`
[TextIO-like object](https://docs.python.org/3/library/io.html#text-i-o)
@ -659,6 +787,87 @@ attributes
- `convert_to_type: ConversionResultTypeEnum = ConversionResultTypeEnum.SHAREABLE_TEXT`
what type to convert the query to
### `class SurplusDefaultGeocoding`
> [!IMPORTANT]
> this has replaced the now deprecated default geocoding functions, `default_geocoder()`
> and `default_reverser()`, in surplus 2.1.0 and later.
see [SurplusGeocoderProtocol](#surplusgeocoderprotocol) and
[SurplusReverserProtocol](#surplusreverserprotocol) for more information how to
implement a compliant custom geocoder functions.
[`dataclasses.dataclass`](https://docs.python.org/3/library/dataclasses.html) providing
the default geocoding functionality for surplus, via
[OpenStreetMap Nominatim](https://nominatim.openstreetmap.org/)
attributes
- `user_agent: str = default_fingerprint`
pass in a custom user agent here, else it will be the default
[fingerprinted user agent](#details-on-the-fingerprinted-user-agent)
example usage
```python
from surplus import surplus, Behaviour, SurplusDefaultGeocoding
geocoding = SurplusDefaultGeocoding("custom user agent")
geocoding.update_geocoding_functions() # not necessary but recommended
behaviour = Behaviour(
...,
geocoder=geocoding.geocoder,
reverser=geocoding.reverser
)
result = surplus("query", behaviour=behaviour)
...
```
methods
- [`def update_geocoding_functions(self) -> None: ...`](#surplusdefaultgeocodingupdate_geocoding_functions)
- [`def geocoder(self, place: str) -> Latlong: ...`](#surplusdefaultgeocodinggeocoder)
- [`def reverser(self, latlong: Latlong, level: int = 18) -> dict[str, Any]: ...`](#surplusdefaultgeocodingreverser)
#### `SurplusDefaultGeocoding.update_geocoding_functions()`
re-initialise the geocoding functions with the current user agent, also generate a new
user agent if not set properly
it is recommended to call this before using surplus as by default the geocoding functions
are uninitialised
- signature
```python
def update_geocoding_functions(self) -> None: ...
```
#### `SurplusDefaultGeocoding.geocoder()`
> [!WARNING]
> this function is primarily given to be passed into a [`Behaviour`](#class-behaviour)
> object, and is not meant to be called directly.
default geocoder for surplus
see [SurplusGeocoderProtocol](#surplusgeocoderprotocol) for more information on surplus
geocoder functions
#### `SurplusDefaultGeocoding.reverser()`
> [!WARNING]
> this function is primarily given to be passed into a [`Behaviour`](#class-behaviour)
> object, and is not meant to be called directly.
default reverser for surplus
see [SurplusReverserProtocol](#surplusreverserprotocol) for more information on surplus
reverser functions
### `class ConversionResultTypeEnum`
[enum.Enum](https://docs.python.org/3/library/enum.html)
@ -772,6 +981,11 @@ attributes
- `latitude: float`
- `longitude: float`
- `bounding_box: tuple[float, float, float, float] | None = None`
a four-tuple representing a bounding box, `(lat1, lat2, lon1, lon2)` or None.
the user does not need to enter this. the attribute is only used when shortening plus
codes, and would be supplied by the geocoding service during shortening.
methods
@ -809,15 +1023,15 @@ methods
- signature
```python
def to_lat_long_coord(self, geocoder: Callable[[str], Latlong]) -> Result[Latlong]:
def to_lat_long_coord(self, geocoder: SurplusGeocoderProtocol) -> Result[Latlong]:
...
```
- arguments
- `geocoder: typing.Callable[[str], Latlong]`
name string to location function, must take in a string and return a
[`Latlong`](#class-latlong), exceptions are handled by the caller
- `geocoder: SurplusGeocoderProtocol`
name string to location function, see
[SurplusGeocoderProtocol](#surplusgeocoderprotocol) for more information
- returns [`Result`](#class-result)[`[Latlong]`](#class-latlong)
@ -861,15 +1075,15 @@ exclusive method that returns a full-length Plus Code as a string
- signature
```python
def to_full_plus_code(self, geocoder: Callable[[str], Latlong]) -> Result[str]:
def to_full_plus_code(self, geocoder: SurplusGeocoderProtocol) -> Result[str]:
...
```
- arguments
- `geocoder: typing.Callable[[str], Latlong]`
name string to location function, must take in a string and return a
[`Latlong`](#class-latlong), exceptions are handled by the caller
- `geocoder: SurplusGeocoderProtocol`
name string to location function, see
[SurplusGeocoderProtocol](#surplusgeocoderprotocol) for more information
- returns [`Result`](#class-result)`[str]`
@ -880,15 +1094,15 @@ method that returns a latitude-longitude coordinate pair
- signature
```python
def to_lat_long_coord(self, geocoder: Callable[[str], Latlong]) -> Result[Latlong]:
def to_lat_long_coord(self, geocoder: SurplusGeocoderProtocol) -> Result[Latlong]:
...
```
- arguments
- `geocoder: typing.Callable[[str], Latlong]`
name string to location function, must take in a string and return a
[`Latlong`](#class-latlong), exceptions are handled by the caller
- `geocoder: SurplusGeocoderProtocol`
name string to location function, see
[SurplusGeocoderProtocol](#surplusgeocoderprotocol) for more information
- returns [`Result`](#class-result)[`[Latlong]`](#class-latlong)
@ -925,15 +1139,15 @@ method that returns a latitude-longitude coordinate pair
- signature
```python
def to_lat_long_coord(self, geocoder: Callable[[str], Latlong]) -> Result[Latlong]:
def to_lat_long_coord(self, geocoder: SurplusGeocoderProtocol) -> Result[Latlong]:
...
```
- arguments
- `geocoder: typing.Callable[[str], Latlong]`
name string to location function, must take in a string and return a
[`Latlong`](#class-latlong), exceptions are handled by the caller
- `geocoder: SurplusGeocoderProtocol`
name string to location function, see
[SurplusGeocoderProtocol](#surplusgeocoderprotocol) for more information
- returns [`Result`](#class-result)[`[Latlong]`](#class-latlong)
@ -970,15 +1184,15 @@ method that returns a latitude-longitude coordinate pair
- signature
```python
def to_lat_long_coord(self, geocoder: Callable[[str], Latlong]) -> Result[Latlong]:
def to_lat_long_coord(self, geocoder: SurplusGeocoderProtocol) -> Result[Latlong]:
...
```
- arguments
- `geocoder: typing.Callable[[str], Latlong]`
name string to location function, must take in a string and return a
[`Latlong`](#class-latlong), exceptions are handled by the caller
- `geocoder: SurplusGeocoderProtocol`
name string to location function, see
[SurplusGeocoderProtocol](#surplusgeocoderprotocol) for more information
- returns [`Result`](#class-result)[`[Latlong]`](#class-latlong)
@ -1031,35 +1245,90 @@ function that parses a query string into a query object
- returns [`Result`](#class-result)[`[Query]`](#query)
### `def default_geocoder()`
### `def generate_fingerprinted_user_agent()`
default geocoder for surplus, uses OpenStreetMap Nominatim
> [!NOTE]
> function is not used by surplus and not directly by the user, but is exposed for
> convenience being [Behaviour](#class-behaviour) objects.
> pass in a custom function to [Behaviour](#class-behaviour) to override the default reverser.
function that attempts to return a unique user agent string.
- signature
```python
def default_geocoder(place: str) -> Latlong:
```
```python
def generate_fingerprinted_user_agent() -> Result[str]:
```
### `def default_reverser()`
- returns [`Result[str]`](#class-result)
default reverser for surplus, uses OpenStreetMap Nominatim
this result will always have a valid value as erroneous results will have a
resulting value of `'surplus/<version> (generic-user)'`
> [!NOTE]
> function is not used by surplus and not directly by the user, but is exposed for
> convenience being [Behaviour](#class-behaviour) objects.
> pass in a custom function to [Behaviour](#class-behaviour) to override the default reverser.
valid results will have a value of `'surplus/<version> (<fingerprin hasht>)'`, where
the fingerprint hash is a 12 character hexadecimal string
- signature
#### details on the fingerprinted user agent
```python
def default_reverser(latlong: Latlong) -> dict[str, Any]:
```
**why do this in the first place?**
if too many people use surplus at the same time,
Nominatim will start to think it's just one person being greedy. so to prevent this,
surplus will try to generate a unique user agent string for each user through
fingerprinting.
at the time of writing, the pre-hashed fingerprint string is as follows:
```python
unique_info: str = f"{version}-{system_info}-{hostname}-{mac_address}"
```
it contains the following, in order, alongside an example:
1. `version` - the surplus version alongside a suffix, if any
```text
2.1.0-local
```
2. `system_info` - generic machine and operating system information
```text
Linux-6.5.0-locietta-WSL2-xanmod1-x86_64-with-glibc2.35
```
3. `hostname` - your computer's hostname
```text
mark
```
4. `mac_address` - your computer's mac address
```text
A9:36:3C:98:79:33
```
after hashing, this string becomes a 12 character hexadecimal string, as shown below:
```text
surplus/2.1.0-local (1fdbfa0b0cfb)
^^^^^^^^^^
this is the hashed result of unique_info
```
if at any time the retrieval of any of these four elements fail, surplus will just give
up and default to `'surplus/<version> (generic-user)'`.
if any of this seems weird to you, that's fine. pass in a custom user agent flag to
surplus with `-u` or `--user-agent` to override the default user agent, or override the
default user agent in your own code by passing in a custom user agent string to
[`Behaviour`](#class-behaviour).
```text
$ surplus --user_agent "a-shiny-custom-and-unique-user-agent" 77Q4+7X Austin, Texas, USA
...
```
```python
>>> from surplus import surplus, Behaviour
>>> surplus(..., Behaviour(user_agent="a-shiny-custom-and-unique-user-agent"))
...
```
## licence
@ -1070,12 +1339,6 @@ python module docstring.
however, direct dependencies of surplus are licensed under different, but still permissive
and open-source licences.
```text
geopy 2.4.0 Python Geocoding Toolbox
└── geographiclib >=1.52,<3
pluscodes 2022.1.3 Compute Plus Codes (Open Location Codes).
```
- [geopy](https://pypi.org/project/geopy/):
Python Geocoding Toolbox

451
playground.ipynb
View file

@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# surplus 2.0.0 playground notebook\n",
"# surplus 2.x.y playground notebook\n",
"\n",
"wrangling with environments for devbox users using codium/vs code:\n",
"\n",
@ -48,7 +48,9 @@
"source": [
"from surplus import PlusCodeQuery, LocalCodeQuery, LatlongQuery, StringQuery\n",
"from surplus import Latlong, Result\n",
"from surplus import default_geocoder, default_reverser"
"from surplus import SurplusDefaultGeocoding\n",
"\n",
"geocoding = SurplusDefaultGeocoding()"
]
},
{
@ -60,7 +62,7 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 3,
"metadata": {},
"outputs": [
{
@ -78,9 +80,9 @@
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mZeroDivisionError\u001b[0m Traceback (most recent call last)",
"\u001b[1;32m/home/m/works/surplus/playground.ipynb Cell 5\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#W4sdnNjb2RlLXJlbW90ZQ%3D%3D?line=7'>8</a>\u001b[0m \u001b[39mprint\u001b[39m(\u001b[39m\"\u001b[39m\u001b[39m{}\u001b[39;00m\u001b[39m\\t\u001b[39;00m\u001b[39m{:<40}\u001b[39;00m\u001b[39m\\t\u001b[39;00m\u001b[39m{}\u001b[39;00m\u001b[39m\"\u001b[39m\u001b[39m.\u001b[39mformat(\u001b[39mbool\u001b[39m(nom_result), \u001b[39mrepr\u001b[39m(nom_result\u001b[39m.\u001b[39merror), nom_result\u001b[39m.\u001b[39mget()))\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#W4sdnNjb2RlLXJlbW90ZQ%3D%3D?line=8'>9</a>\u001b[0m \u001b[39mprint\u001b[39m(\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#W4sdnNjb2RlLXJlbW90ZQ%3D%3D?line=9'>10</a>\u001b[0m \u001b[39m\"\u001b[39m\u001b[39m{}\u001b[39;00m\u001b[39m\\t\u001b[39;00m\u001b[39m{:<40}\u001b[39;00m\u001b[39m\\t\u001b[39;00m\u001b[39m{}\u001b[39;00m\u001b[39m\"\u001b[39m\u001b[39m.\u001b[39mformat(\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#W4sdnNjb2RlLXJlbW90ZQ%3D%3D?line=10'>11</a>\u001b[0m \u001b[39mbool\u001b[39m(exc_result), \u001b[39mrepr\u001b[39m(exc_result\u001b[39m.\u001b[39merror), exc_result\u001b[39m.\u001b[39mcry(string\u001b[39m=\u001b[39m\u001b[39mTrue\u001b[39;00m)\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#W4sdnNjb2RlLXJlbW90ZQ%3D%3D?line=11'>12</a>\u001b[0m )\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#W4sdnNjb2RlLXJlbW90ZQ%3D%3D?line=12'>13</a>\u001b[0m )\n\u001b[0;32m---> <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#W4sdnNjb2RlLXJlbW90ZQ%3D%3D?line=13'>14</a>\u001b[0m \u001b[39mprint\u001b[39m(\u001b[39m\"\u001b[39m\u001b[39m{}\u001b[39;00m\u001b[39m\\t\u001b[39;00m\u001b[39m{:<40}\u001b[39;00m\u001b[39m\\t\u001b[39;00m\u001b[39m{}\u001b[39;00m\u001b[39m\"\u001b[39m\u001b[39m.\u001b[39mformat(\u001b[39mbool\u001b[39m(exc_result), \u001b[39mrepr\u001b[39m(exc_result\u001b[39m.\u001b[39merror), exc_result\u001b[39m.\u001b[39;49mget()))\n",
"File \u001b[0;32m~/works/surplus/surplus.py:247\u001b[0m, in \u001b[0;36mResult.get\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 245\u001b[0m \u001b[39m\"\"\"method that returns self.value if Result is non-erroneous else raises error\"\"\"\u001b[39;00m\n\u001b[1;32m 246\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39misinstance\u001b[39m(\u001b[39mself\u001b[39m\u001b[39m.\u001b[39merror, \u001b[39mBaseException\u001b[39;00m):\n\u001b[0;32m--> 247\u001b[0m \u001b[39mraise\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39merror\n\u001b[1;32m 248\u001b[0m \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mvalue\n",
"\u001b[1;32m/home/m/works/surplus/playground.ipynb Cell 5\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#W4sdnNjb2RlLXJlbW90ZQ%3D%3D?line=0'>1</a>\u001b[0m nom_result \u001b[39m=\u001b[39m Result[\u001b[39mint\u001b[39m](\u001b[39m3\u001b[39m)\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#W4sdnNjb2RlLXJlbW90ZQ%3D%3D?line=2'>3</a>\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[0;32m----> <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#W4sdnNjb2RlLXJlbW90ZQ%3D%3D?line=3'>4</a>\u001b[0m \u001b[39m1\u001b[39;49m \u001b[39m/\u001b[39;49m \u001b[39m0\u001b[39;49m\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#W4sdnNjb2RlLXJlbW90ZQ%3D%3D?line=4'>5</a>\u001b[0m \u001b[39mexcept\u001b[39;00m \u001b[39mException\u001b[39;00m \u001b[39mas\u001b[39;00m exc:\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#W4sdnNjb2RlLXJlbW90ZQ%3D%3D?line=5'>6</a>\u001b[0m exc_result \u001b[39m=\u001b[39m Result[\u001b[39mint\u001b[39m](\u001b[39m-\u001b[39m\u001b[39m1\u001b[39m, error\u001b[39m=\u001b[39mexc)\n",
"\u001b[1;32m/home/m/works/surplus/playground.ipynb Cell 5\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#X26sdnNjb2RlLXJlbW90ZQ%3D%3D?line=7'>8</a>\u001b[0m \u001b[39mprint\u001b[39m(\u001b[39m\"\u001b[39m\u001b[39m{}\u001b[39;00m\u001b[39m\\t\u001b[39;00m\u001b[39m{:<40}\u001b[39;00m\u001b[39m\\t\u001b[39;00m\u001b[39m{}\u001b[39;00m\u001b[39m\"\u001b[39m\u001b[39m.\u001b[39mformat(\u001b[39mbool\u001b[39m(nom_result), \u001b[39mrepr\u001b[39m(nom_result\u001b[39m.\u001b[39merror), nom_result\u001b[39m.\u001b[39mget()))\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#X26sdnNjb2RlLXJlbW90ZQ%3D%3D?line=8'>9</a>\u001b[0m \u001b[39mprint\u001b[39m(\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#X26sdnNjb2RlLXJlbW90ZQ%3D%3D?line=9'>10</a>\u001b[0m \u001b[39m\"\u001b[39m\u001b[39m{}\u001b[39;00m\u001b[39m\\t\u001b[39;00m\u001b[39m{:<40}\u001b[39;00m\u001b[39m\\t\u001b[39;00m\u001b[39m{}\u001b[39;00m\u001b[39m\"\u001b[39m\u001b[39m.\u001b[39mformat(\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#X26sdnNjb2RlLXJlbW90ZQ%3D%3D?line=10'>11</a>\u001b[0m \u001b[39mbool\u001b[39m(exc_result), \u001b[39mrepr\u001b[39m(exc_result\u001b[39m.\u001b[39merror), exc_result\u001b[39m.\u001b[39mcry(string\u001b[39m=\u001b[39m\u001b[39mTrue\u001b[39;00m)\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#X26sdnNjb2RlLXJlbW90ZQ%3D%3D?line=11'>12</a>\u001b[0m )\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#X26sdnNjb2RlLXJlbW90ZQ%3D%3D?line=12'>13</a>\u001b[0m )\n\u001b[0;32m---> <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#X26sdnNjb2RlLXJlbW90ZQ%3D%3D?line=13'>14</a>\u001b[0m \u001b[39mprint\u001b[39m(\u001b[39m\"\u001b[39m\u001b[39m{}\u001b[39;00m\u001b[39m\\t\u001b[39;00m\u001b[39m{:<40}\u001b[39;00m\u001b[39m\\t\u001b[39;00m\u001b[39m{}\u001b[39;00m\u001b[39m\"\u001b[39m\u001b[39m.\u001b[39mformat(\u001b[39mbool\u001b[39m(exc_result), \u001b[39mrepr\u001b[39m(exc_result\u001b[39m.\u001b[39merror), exc_result\u001b[39m.\u001b[39;49mget()))\n",
"File \u001b[0;32m~/works/surplus/surplus/surplus.py:270\u001b[0m, in \u001b[0;36mResult.get\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m 268\u001b[0m \u001b[39m\"\"\"method that returns self.value if Result is non-erroneous else raises error\"\"\"\u001b[39;00m\n\u001b[1;32m 269\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39misinstance\u001b[39m(\u001b[39mself\u001b[39m\u001b[39m.\u001b[39merror, \u001b[39mBaseException\u001b[39;00m):\n\u001b[0;32m--> 270\u001b[0m \u001b[39mraise\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39merror\n\u001b[1;32m 271\u001b[0m \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mvalue\n",
"\u001b[1;32m/home/m/works/surplus/playground.ipynb Cell 5\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#X26sdnNjb2RlLXJlbW90ZQ%3D%3D?line=0'>1</a>\u001b[0m nom_result \u001b[39m=\u001b[39m Result[\u001b[39mint\u001b[39m](\u001b[39m3\u001b[39m)\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#X26sdnNjb2RlLXJlbW90ZQ%3D%3D?line=2'>3</a>\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[0;32m----> <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#X26sdnNjb2RlLXJlbW90ZQ%3D%3D?line=3'>4</a>\u001b[0m \u001b[39m1\u001b[39;49m \u001b[39m/\u001b[39;49m \u001b[39m0\u001b[39;49m\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#X26sdnNjb2RlLXJlbW90ZQ%3D%3D?line=4'>5</a>\u001b[0m \u001b[39mexcept\u001b[39;00m \u001b[39mException\u001b[39;00m \u001b[39mas\u001b[39;00m exc:\n\u001b[1;32m <a href='vscode-notebook-cell://wsl%2Balpine/home/m/works/surplus/playground.ipynb#X26sdnNjb2RlLXJlbW90ZQ%3D%3D?line=5'>6</a>\u001b[0m exc_result \u001b[39m=\u001b[39m Result[\u001b[39mint\u001b[39m](\u001b[39m-\u001b[39m\u001b[39m1\u001b[39m, error\u001b[39m=\u001b[39mexc)\n",
"\u001b[0;31mZeroDivisionError\u001b[0m: division by zero"
]
}
@ -111,27 +113,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Result(value=Latlong(latitude=1.3336875, longitude=103.7746875), error=None)"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"PlusCodeQuery(code=\"6PH58QMF+FV\").to_lat_long_coord(geocoder=default_geocoder)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [
{
@ -146,16 +128,12 @@
}
],
"source": [
"plus_code = LocalCodeQuery(code=\"8QMF+FV\", locality=\"Singapore\").to_full_plus_code(\n",
" geocoder=default_geocoder\n",
")\n",
"\n",
"PlusCodeQuery(code=plus_code.get()).to_lat_long_coord(geocoder=default_geocoder)"
"PlusCodeQuery(code=\"6PH58QMF+FV\").to_lat_long_coord(geocoder=geocoding.geocoder)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [
{
@ -170,20 +148,22 @@
}
],
"source": [
"LocalCodeQuery(code=\"8QMF+FV\", locality=\"Singapore\").to_lat_long_coord(\n",
" geocoder=default_geocoder\n",
")"
"plus_code = LocalCodeQuery(code=\"8QMF+FV\", locality=\"Singapore\").to_full_plus_code(\n",
" geocoder=geocoding.geocoder\n",
")\n",
"\n",
"PlusCodeQuery(code=plus_code.get()).to_lat_long_coord(geocoder=geocoding.geocoder)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Result(value=Latlong(latitude=1.33318835, longitude=103.77461234638255), error=None)"
"Result(value=Latlong(latitude=1.3336875, longitude=103.7746875), error=None)"
]
},
"execution_count": 6,
@ -192,14 +172,14 @@
}
],
"source": [
"LatlongQuery(\n",
" latlong=Latlong(latitude=1.33318835, longitude=103.77461234638255)\n",
").to_lat_long_coord(geocoder=default_geocoder)"
"LocalCodeQuery(code=\"8QMF+FV\", locality=\"Singapore\").to_lat_long_coord(\n",
" geocoder=geocoding.geocoder\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
@ -214,7 +194,29 @@
}
],
"source": [
"StringQuery(query=\"Ngee Ann Polytechnic\").to_lat_long_coord(geocoder=default_geocoder)"
"LatlongQuery(\n",
" latlong=Latlong(latitude=1.33318835, longitude=103.77461234638255)\n",
").to_lat_long_coord(geocoder=geocoding.geocoder)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Result(value=Latlong(latitude=1.33318835, longitude=103.77461234638255), error=None)"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"StringQuery(query=\"Ngee Ann Polytechnic\").to_lat_long_coord(geocoder=geocoding.geocoder)"
]
},
{
@ -282,7 +284,7 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
@ -298,6 +300,7 @@
" 'house_number': '535',\n",
" 'latitude': 1.33318835,\n",
" 'longitude': 103.77461234638255,\n",
" 'neighbourhood': 'Ewart Park',\n",
" 'postcode': '599489',\n",
" 'raw': {'address': {'ISO3166-2-lvl6': 'SG-03',\n",
" 'amenity': 'Ngee Ann Polytechnic',\n",
@ -306,6 +309,7 @@
" 'country_code': 'sg',\n",
" 'county': 'Northwest',\n",
" 'house_number': '535',\n",
" 'neighbourhood': 'Ewart Park',\n",
" 'postcode': '599489',\n",
" 'road': 'Clementi Road',\n",
" 'suburb': 'Bukit Timah'},\n",
@ -315,8 +319,9 @@
" '103.7701481',\n",
" '103.7783945'],\n",
" 'class': 'amenity',\n",
" 'display_name': 'Ngee Ann Polytechnic, 535, Clementi Road, Bukit '\n",
" 'Timah, Singapore, Northwest, 599489, Singapore',\n",
" 'display_name': 'Ngee Ann Polytechnic, 535, Clementi Road, Ewart '\n",
" 'Park, Bukit Timah, Singapore, Northwest, 599489, '\n",
" 'Singapore',\n",
" 'importance': 0.34662169301918117,\n",
" 'lat': '1.33318835',\n",
" 'licence': 'Data © OpenStreetMap contributors, ODbL 1.0. '\n",
@ -325,7 +330,7 @@
" 'name': 'Ngee Ann Polytechnic',\n",
" 'osm_id': 2535118,\n",
" 'osm_type': 'relation',\n",
" 'place_id': 297946059,\n",
" 'place_id': 250910125,\n",
" 'place_rank': 30,\n",
" 'type': 'university'},\n",
" 'road': 'Clementi Road',\n",
@ -337,15 +342,363 @@
"import pprint\n",
"\n",
"latlong = LocalCodeQuery(code=\"8QMF+FV\", locality=\"Singapore\").to_lat_long_coord(\n",
" default_geocoder\n",
" geocoder=geocoding.geocoder\n",
")\n",
"if not latlong:\n",
" latlong.cry()\n",
"\n",
"else:\n",
" location = default_reverser(latlong.get())\n",
" location = geocoding.reverser(latlong.get())\n",
" pprint.pprint(location)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2.1.0: adventures in of shortening global/full Plus Codes"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### testing rate-limited and cached default geocoding functions"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"test_geocoding = SurplusDefaultGeocoding(user_agent=\"surplus/playground\")"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1\n",
"2\n",
"3\n",
"4\n",
"5\n",
"\n",
"1\n",
"2\n",
"3\n",
"4\n",
"5\n",
"\n",
"3.1107698050s\t->\t0.0000886890s\t\t(-3.1106811160002508s)\n"
]
}
],
"source": [
"from timeit import timeit\n",
"\n",
"\n",
"test_stmt = \"\"\"\\\n",
"print(1)\n",
"test_geocoding.geocoder(\"Wisma Atria\") # instant\n",
"print(2)\n",
"test_geocoding.geocoder(\"Temasek Polytechnic\") # after 1 second\n",
"print(3)\n",
"location = test_geocoding.geocoder(\"Ngee Ann Polytechnic\") # after 1 second\n",
"print(4)\n",
"test_geocoding.reverser(f\"{location.latitude}, {location.longitude}\") # instant\n",
"print(5)\n",
"test_geocoding.reverser(f\"{location.latitude}, {location.longitude}\") # instant (cached)\n",
"print()\n",
"\"\"\"\n",
"\n",
"time_cold_call = timeit(test_stmt, globals=globals(), number=1) # expecting 3-4 seconds\n",
"time_2nd_call = timeit(test_stmt, globals=globals(), number=1) # should be instant\n",
"\n",
"print(\n",
" f\"{time_cold_call:.10f}s\\t->\\t{time_2nd_call:.10f}s\\t\\t({time_2nd_call - time_cold_call}s)\"\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### reversing the query latlong and using the address information to form a locality"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"level = 13"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"St Lucia, St Lucia, Queensland, Australia\n",
"Austin, Travis County, Texas, United States\n"
]
}
],
"source": [
"(\n",
" au_response := geocoding.reverser(\n",
" (\n",
" au_target := (\n",
" LocalCodeQuery(\n",
" \"G227+XF\", \"St Lucia, Queensland, Australia\"\n",
" ).to_lat_long_coord(geocoding.geocoder)\n",
" )\n",
" ).get(),\n",
" level=level,\n",
" )\n",
")\n",
"\n",
"au_locality = f\"{au_response['suburb']}, {au_response['city_district']}, {au_response['state']}, {au_response['country']}\"\n",
"print(au_locality)\n",
"\n",
"(\n",
" us_response := geocoding.reverser(\n",
" (\n",
" us_target := (\n",
" LocalCodeQuery(\"77Q4+7X\", \"Austin, Texas, USA\").to_lat_long_coord(\n",
" geocoding.geocoder\n",
" )\n",
" )\n",
" ).get(),\n",
" level=level,\n",
" )\n",
")\n",
"\n",
"us_locality = f\"{us_response['city']}, {us_response['county']}, {us_response['state']}, {us_response['country']}\"\n",
"print(us_locality)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### getting boundary boxes"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'addresstype': 'suburb',\n",
" 'boundingbox': ['-27.5187362', '-27.4787362', '152.9881642', '153.0281642'],\n",
" 'class': 'place',\n",
" 'display_name': 'St Lucia, Brisbane City, Queensland, 4072, Australia',\n",
" 'importance': 0.27501,\n",
" 'lat': '-27.4987362',\n",
" 'licence': 'Data © OpenStreetMap contributors, ODbL 1.0. '\n",
" 'http://osm.org/copyright',\n",
" 'lon': '153.0081642',\n",
" 'name': 'St Lucia',\n",
" 'osm_id': 88800268,\n",
" 'osm_type': 'node',\n",
" 'place_id': 54477898,\n",
" 'place_rank': 19,\n",
" 'type': 'suburb'}\n",
"\n",
"Latlong(latitude=-27.4987362, longitude=153.0081642, bounding_box=[-27.5187362, -27.4787362, 152.9881642, 153.0281642])\n"
]
}
],
"source": [
"from geopy.geocoders import Nominatim\n",
"from pprint import pprint\n",
"\n",
"target_query: Result[Latlong] = au_target\n",
"target_locality: str = au_locality\n",
"\n",
"raw_geocoding = Nominatim(user_agent=\"surplus/playground\")\n",
"latlong = raw_geocoding.geocode(target_locality)\n",
"pprint(latlong.raw)\n",
"print()\n",
"\n",
"# done: now implmented in surplus as surplus.Latlong.bounding_box\n",
"locality_latlong = geocoding.geocoder(target_locality)\n",
"pprint(locality_latlong)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[-27.5187362, -27.4787362, 152.9881642, 153.0281642]\n",
"(True, True, True, True)\n",
"(True, True, True, True)\n"
]
}
],
"source": [
"# based on <https://github.com/google/open-location-code/wiki/Guidance-for-shortening-codes>\n",
"\n",
"target_latlong = target_query.get()\n",
"if locality_latlong.bounding_box is None:\n",
" ... # raise some error\n",
"\n",
"print(locality_latlong.bounding_box)\n",
"check1 = (\n",
" # The center point of the feature is within 0.4 degrees latitude and 0.4 degrees longitude\n",
" (\n",
" (target_latlong.latitude - 0.4)\n",
" <= locality_latlong.latitude\n",
" <= (target_latlong.latitude + 0.4)\n",
" ),\n",
" (\n",
" (target_latlong.longitude - 0.4)\n",
" <= locality_latlong.longitude\n",
" <= (target_latlong.longitude + 0.4)\n",
" ),\n",
" # The bounding box of the feature is less than 0.8 degrees high and wide.\n",
" abs(locality_latlong.bounding_box[0] - locality_latlong.bounding_box[1]) < 0.8,\n",
" abs(locality_latlong.bounding_box[2] - locality_latlong.bounding_box[3]) < 0.8,\n",
")\n",
"\n",
"\n",
"check2 = (\n",
" # The center point of the feature is within 0.4 degrees latitude and 0.4 degrees longitude\n",
" (\n",
" (target_latlong.latitude - 8)\n",
" <= locality_latlong.latitude\n",
" <= (target_latlong.latitude + 8)\n",
" ),\n",
" (\n",
" (target_latlong.longitude - 8)\n",
" <= locality_latlong.longitude\n",
" <= (target_latlong.longitude + 8)\n",
" ),\n",
" # The bounding box of the feature is less than 0.8 degrees high and wide.\n",
" abs(locality_latlong.bounding_box[0] - locality_latlong.bounding_box[1]) < 16,\n",
" abs(locality_latlong.bounding_box[2] - locality_latlong.bounding_box[3]) < 16,\n",
")\n",
"\n",
"print(check1)\n",
"print(check2)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"G227+XF St Lucia, St Lucia, Queensland, Australia\n"
]
}
],
"source": [
"from pluscodes import encode\n",
"\n",
"target_plus_code = encode(\n",
" lat=target_latlong.latitude, lon=target_latlong.longitude, code_length=10\n",
")\n",
"portion_plus_code = \"\"\n",
"\n",
"if check1:\n",
" portion_plus_code = target_plus_code[4:]\n",
" print(portion_plus_code, target_locality)\n",
"\n",
"elif check2:\n",
" portion_plus_code = target_plus_code[2:]\n",
" print(portion_plus_code, target_locality)\n",
"\n",
"else:\n",
" print(\n",
" \"info: could not determine a suitable geographical feature to use as locality for shortening.\"\n",
" )\n",
" print(plus_code)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## machine fingerprinting attempt\n",
"\n",
"because of nominatim's acceptable usage policy \n",
"<https://operations.osmfoundation.org/policies/nominatim/>"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from hashlib import shake_256 as _hashlib_shake_256\n",
"from platform import platform as _platform_platform\n",
"from socket import gethostname as _socket_gethostname\n",
"from uuid import getnode as _uuid_getnode\n",
"from surplus import VERSION, VERSION_SUFFIX\n",
"\n",
"\n",
"def generate_fingerprinted_user_agent() -> Result[str]:\n",
" \"\"\"\n",
" function that attempts to return a unique user agent string.\n",
"\n",
" returns Result[str]\n",
" this result will always have a valid value as erroneous results will have a\n",
" resulting value of 'surplus/<version>/generic-user'\n",
" valid results will have a value of 'surplus/<version>/<fingerprint>', where\n",
" fingerprint is a 12 character hexadecimal string\n",
" \"\"\"\n",
" version: str = \".\".join([str(v) for v in VERSION]) + VERSION_SUFFIX\n",
"\n",
" try:\n",
" system_info: str = _platform_platform()\n",
" hostname: str = _socket_gethostname()\n",
" mac_address: str = \":\".join(\n",
" [\n",
" \"{:02x}\".format((_uuid_getnode() >> elements) & 0xFF)\n",
" for elements in range(0, 2 * 6, 2)\n",
" ][::-1]\n",
" )\n",
" unique_info: str = f\"{version}-{system_info}-{hostname}-{mac_address}\"\n",
"\n",
" print(f\"{version=}\")\n",
" print(f\"{system_info=}\")\n",
" print(f\"{hostname=}\")\n",
" print(f\"{mac_address=}\")\n",
"\n",
" except Exception as exc:\n",
" return Result[str](f\"surplus/{version} (generic-user)\", error=exc)\n",
"\n",
" fingerprint: str = _hashlib_shake_256(unique_info.encode()).hexdigest(5)\n",
"\n",
" return Result[str](f\"surplus/{version} ({fingerprint})\")"
]
}
],
"metadata": {

6
poetry.lock generated
View file

@ -14,14 +14,14 @@ files = [
[[package]]
name = "asttokens"
version = "2.3.0"
version = "2.4.0"
description = "Annotate AST trees with source code positions"
category = "dev"
optional = false
python-versions = "*"
files = [
{file = "asttokens-2.3.0-py2.py3-none-any.whl", hash = "sha256:bef1a51bc256d349e9f94e7e40e44b705ed1162f55294220dd561d24583d9877"},
{file = "asttokens-2.3.0.tar.gz", hash = "sha256:2552a88626aaa7f0f299f871479fc755bd4e7c11e89078965e928fb7bb9a6afe"},
{file = "asttokens-2.4.0-py2.py3-none-any.whl", hash = "sha256:cf8fc9e61a86461aa9fb161a14a0841a03c405fa829ac6b202670b3495d2ce69"},
{file = "asttokens-2.4.0.tar.gz", hash = "sha256:2e0171b991b2c959acc6c49318049236844a5da1d65ba2672c4880c1c894834e"},
]
[package.dependencies]

14
surplus/__init__.py
View file

@ -32,10 +32,14 @@ For more information, please refer to <http://unlicense.org/>
# surplus was and would've been a single-file module, but typing is in the way :(
# https://github.com/python/typing/issues/1333
from .surplus import default_geocoder # deprecated, emulation function
from .surplus import default_reverser # deprecated, emulation function
from .surplus import (
BUILD_BRANCH,
BUILD_COMMIT,
BUILD_DATETIME,
CONNECTION_MAX_RETRIES,
CONNECTION_WAIT_SECONDS,
EMPTY_LATLONG,
SHAREABLE_TEXT_LINE_0_KEYS,
SHAREABLE_TEXT_LINE_1_KEYS,
@ -44,8 +48,8 @@ from .surplus import (
SHAREABLE_TEXT_LINE_4_KEYS,
SHAREABLE_TEXT_LINE_5_KEYS,
SHAREABLE_TEXT_LINE_6_KEYS,
SHAREABLE_TEXT_LOCALITY,
SHAREABLE_TEXT_NAMES,
USER_AGENT,
VERSION,
VERSION_SUFFIX,
Behaviour,
@ -60,13 +64,15 @@ from .surplus import (
PlusCodeNotFoundError,
PlusCodeQuery,
Query,
Result,
ResultType,
StringQuery,
SurplusDefaultGeocoding,
SurplusException,
UnavailableFeatureError,
SurplusGeocoderProtocol,
SurplusReverserProtocol,
cli,
default_geocoder,
default_reverser,
generate_fingerprinted_user_agent,
handle_args,
parse_query,
surplus,

801
surplus/surplus.py
View file

File diff suppressed because it is too large Load diff

30
test.py
View file

@ -100,22 +100,26 @@ tests: list[ContinuityTest] = [
),
ContinuityTest(
query="Ngee Ann Polytechnic, Singapore",
expected=(
"Ngee Ann Polytechnic\n"
"535 Clementi Road\n"
"Bukit Timah\n"
"599489\n"
"Northwest, Singapore"
),
expected=[
(
"Ngee Ann Polytechnic\n"
"535 Clementi Road\n"
"Bukit Timah\n"
"599489\n"
"Northwest, Singapore"
)
],
),
ContinuityTest(
query="1.3521, 103.8198",
expected=(
"MacRitchie Nature Trail\n"
"Central Water Catchment\n"
"574325\n"
"Central, Singapore"
),
expected=[
(
"MacRitchie Nature Trail\n"
"Central Water Catchment\n"
"574325\n"
"Central, Singapore"
)
],
),
]