This patch adds a new webchat config section to specify:
web_ip (the ip address to listen on)
web_port
latitude (latitude to use for the GPS beacon button)
longitude (long to use for the GPS beacon button)
The kiss clients now detect if the incomming packet is a third party
packet and then sends up the subpacket instead of the encapsulated
packet up to the consumer.
This immediately breaks the beacon button.
This patch removes the dep for pyopenssl and cryptography
so that aprsd install on the rpi.
Unfortunately in order for the web page to get the Lat/Lon, the
browser must be connected over SSL. Will have to create a workaround
for this later.
The KISS client sends the path as part of the headers, so we had
to strip out the path from the payload of each message so the path
wouldn't get listed twice.
When running the web admin interface with
'python -m aprsd.wsgi' the Flask app global now uses
the web_ip config entry for listening. Also disabled
debug output.
This patch adjusts the backoff mechanism for aprs client
reconnect to a max backoff sleep of 5 seconds. This prevents
an exponential backoff when connection retrying.
This patch starts the work to replace flask-socketio with
python-socketio so that uwsgi can be used instead of gunicorn.
uwsgi can support websockets.
Have to rework webchat command next
This patch removed the dependency on flask-classful. This required
making all of the flask web routing non class based.
This patch also changes the aprsis class to allow retries for failed
connections when the aprsis servers are full and not responding to
login requests.
You can now fetch and view the stats of a live running aprsd server
if it has enabled the rpc server in the config file's rpc_settings
block.
You just have to match the magic word as specified in the config file to
authorize against the rpc server.
aprsd fetch-stats --ip-address <ip of aprsd> --port <port> --magic-word
<magic word>
This patch replaces the ratelimiter library with rush for rate limiting
as the ratelimiter package doesn't work with python 3.11.
This patch also refactors the flask.pu to admin_web.py and
aprsd.py to main.py
Added the ability to start the rpc server for fetching stats from the
listen command. If the rpc server is enabled in config, the rpc
server will now start.
Added geopy as a dependency for the location plugin.
The us weather service API is now broken upstream.
Reworked the requirements.txt and dev-requirements.txt files
This patch updates the aprsd listen command to add the packet-plugins
argument which allows enabling a single plugin to work against the
packets recieved from the aprsis network.
This patch adds basic ratelimiting to sending out AckPackets
and non AckPackets. This provides a basic way to prevent
aprsd from sending out packets as fast as possible, which isn't
great for a bandwidth limited network.
This patch also adds some keepalive checks to all threads in the
threadslist as well as the network client objects (apris, kiss)
For whatever reason passing in group in python 3.9.x
fails for importlib_metadata.entry_points. This patch
fetches all and filters through them to get the real
oslo.config.opts entry points now. This is to find all
of the config options of aprsd and the plugins
After adding the rpc service for aprsd server and separating the
admin web REST interface, healthcheck no longer worked. The stats
are available via rpc now.
This patch adds the ObjectPacket. This is used by the REPEAT plugins
to send out an object in message packet to let radios tune directly
to the station.
The regex search is now by default case insensitive.
Also update each core plugin to better match the command.
ping plugin can now match on
p
p foo
ping
pIng
Weather plugins can now match on
w
wx
wX
Wx KM6LYW
weather
WeaTher
This patch moves the plugin manager to early in the startup
process so that the plugins get loaded, which also means each
plugin's custom config settings will be in the CONF object.
This allows dumping the entire CONF with all the plugin settings.
This patch introduces rpyc based RPC client/server for
the flask web interface to call into the running aprsd server
command to fetch stats, logs, etc to send to the browser.
This allows running the web interface via gunicorn command
gunicorn -k gevent --reload --threads 10 -w 1 aprsd.flask:app --log-level DEBUG
This patch is the initial conversion of the custom config
and config file yaml format to oslo_config's configuration mechanism.
The resulting config format is now an ini type file.
The default location is ~/.config/aprsd/aprsd.conf
This is a backwards incompatible change. You will have to rebuild
the config file and edit it.
Also any aprsd plugins can now define config options in code and
add an setup.cfg entry_point definition
oslo_config.opts =
foo.conf = foo.conf:list_opts
This patch decouples sending a message from the internals of
the Packet classes. This allows the rest of the code to use
Packet objects as type hints in methods to enforce Packets
in the plugins.
The send method was moved to a single place in the threads.tx.send()
This patch updates both the webchat and listen commands
to be able to use the new queue based packet RX processing.
APRSD used to start a thread for every packet received, now
packets are pushed into a queue for processing by other threads
already running.
This patch cleans up the Packet class attributes used to
keep track of how many times packets have been sent and
the last time they were sent. This is used by the PacketTracker
and the tx threads for transmitting packets
The messaging.py now is nothing but a shell that
contains a link to packets.NULL_MESSAGE to help maintain
some backwards compatibility with plugins.
Packets dataclass has fully replaced messaging objects.
This patch adds the needed code to construct the raw output
string for sending a GPSPacket.
TODO: Need to incorporate speed, course, rng, position ambiguity ?
TODO: Need to add option to 'compress' the output location data.
This patch reworks all the packet processing to use the new
Packets objects. Nuked all of the messaging classes.
backwards incompatible changes
all messaging.py classes are now gone and replaced by
packets.py classes
With more testing of the webchat beaconing, found a problem
with the packet format for the beacon. This patch fixes the
packet format of the beacon.
Also added a timeout when trying to get the GPS location in the browser,
otherwise it could never come back.
the device detector was taking 1 minute on a raspi to parse out the
user-agent string from the browser. user-agents takes 2 seconds,
which still isn't great, but 'doable' for the webchat interface.
This patch refactors the rx threads a bit to reuse some code
responsible for processing acks when packets are received.
This also eliminates a custom thread in the webchat command for
processing received packets now that there is common code in the base
classes.
This patch fixes an issue where aprsd was deciding if it was
supposed to process a packet destined for itself or not. It was
making a case sensitive comparison. This patch makes that comparison
case insensitive for the callsign itself.
This patchset allow getting the GPS coordinates from the browser's
geolocation API (which can be denied by user), then send's the GPS
coordinates to aprsd via socketio and then aprsd sends a beacon.
This allows the APRS network to know the location of the person running
the webchat app via browser so packets can get routed back to it.
This patch reworks the KISS client to get rid of
aioax25 as it was too difficult to work with due to
heavy use of asyncio.
Switched to the kiss3 pypi library.
This patch updates the main aprsd threads class to use
a shared queue to notify all aprsd thread classes they need
to exit. This ensures any closing down of sockets, etc happens from
inside the context of the thread itself, not the MainThread that
calls stop.
This patch changes how aprsd identifies itself when connected to
any client, which is not relying on the login for each client.
There are 3 supported clients currently
aprsis,
tcpkiss
serialkiss.
Each client has their own potential login/callsign to connect
to the remote. This patch tells aprsd to use the new config option
aprsd.callsign as a means to identify itself. It will accept
packets as <aprsd.callsign> and reply as <aprsd.callsign> regardless
of which client object is being used to connect to the remote.
Note: this breaks backwards compatibility. This patch now requires
the new config option
aprsd:
callsign: <callsign>
This patch creates a threads directory and separates out
the contents of threads.py into separate files in the
threads directory to make it easier to find and maintain.
This patch updates the config option checking for
required fields in the config yaml file. Specifically
for the existence of the aprsd: section
and the required fields for the 3 supported client types
apris,
kiss serial,
kiss tcp
This patch changes the base Message class to
ensure that all printing of the message class only
outputs the message in the truncated and bad word filtering
enabled in the log.
The APRS_LOGIN and APRS_PASSWORD arguments now fallback
to the config file if it exists.
First it checks the passed in parameters, then checks the
environement vars, then checks the parsed config to find the
login and password.
This patch also adds unit tests for the send-message command to
check the fallback.
This patch fixes an issue with the processing of packets
and updateing the watchlist. Previously after the
notify plugin processed the packet it would update the watchlist.
This doesn't work when there are more than 1 notify plugins
enabled, only the first notify plugin seeing the packet will
recognize that the callsign is old.
This patch updates the logging facility to ensure that
logging to a file works even when --quiet mode is selected.
Also update the listen and list-plugins command to show
a console.status line while waiting for results to come in.
This patch updates the ouput of the list-plugins command.
This also adds the ability to show the available plugins
to install that are published packages on pypi.org.
This also shows the list of installed packages from pypi.org
The watchlist notify plugin is supposed to send an APRS message
to the configured callsign. This patch makes sure that the
message is sent to the notify_callsign
The python rich library is extensive and has a really nice
log format that is easier to read and has built in formatting
and coloring of the log output.
To enable rich logging add rich_logging: True in the config file.
This patch updates the healthcheck command to not require
the aprsd.yml config file to exist. The healthcheck
calls a running aprsd, collects the stats to determine if it's
healthy.
This removed the requirement of running APRSD for specifying
the aprs.fi key in the config file. The plugins that need the
key have been updated to set enabled = False when the key is missing.
This patch refactors the cli to incorporate
the dev, send-message, listen commands into the main aprsd app.
This also moves the command line completion installer/show into
it's own subgroup.
This patch fixes a problem with the packets object
not being initialized correctly for the send-message command
from the command line.
Also adds the --wait-response option for send-message, which by
default is now False
This patch ensures that the pickle file is opened and closed correctly
as well as trapping for any exceptions that might occur while loading
a pickle file.
This patch initializes all of the MsgTrack, WatchList and SeenList
prior to the plugins loading. Some plugins may kick off messages
being sent immediately. So everything has to be ready to go
prior to the plugins being loaded.
This patch adds the new objectstore Mixin class that enables
classes that store their date in self.data as a serializeable dict,
to be able to be stored to disk at shutdown and loaded at startup.
The SeenList and WatchList are now saved/loaded to/from disk.
This patch adds the always enabled HelpPlugin. This plugin
now will respond to the 'help' or 'h' commands that will
automatically build a help string based on the number of
enabled plugins. It will also respond to
help <plugin> with the plugin specific help
This patch updates the aprsd-dev command's log file format
to use what's defined as the default and/or use the config file
setting like aprsd server does.
The email plugin was still using globals for tracking
the check_email_delay as well as the config. This
patch creates a new singleton thread safe mechanism for
check_email_delay with the EmailInfo class.
This patch completely refactors and simplifies how the clients
are created and used. There is no need now to have a separate
KISSRXThread. Since all the custom work for the KISS client is
encapsulated in the kiss client itself, the same RX thread and
callback mechanism works for both the APRSIS client and KISS Client
objects. There is also no need to determine which transport
(aprsis vs kiss) is being used at runtime by any of the messages
objects. The same API works for both APRSIS and KISS Client objects
Started noticing that aprs-is keepalive messages just stop
getting sent. This causes aprsd to basically disconnect from
the APRS network. Added a check into the KeepAlive thread to
restart the aprs-is connecter if the last time we got a keepalive
from apris is > 5 minutes.
This patch enables the ability for plugins to return:
* string
* list of strings
* message object
* list of strings and message ojects
Each string will be encapsulated in a message object prior being sent.
each message object will be sent directly.
Each list will be iterated over and processed according to the above 2
options.
This allows the admin interface to see which plugins are registered and
enabled. Enabled is a flag that is set in the setup() method of the
plugin. This gives the plugin developer a chance to disable the plugin
if something isn't right at setup time. This allows aprsd to ignore
plugins that are registered but not emabled.
If the logging queue gets full, due to a maxsize being set,
then any further logs will result on lots of errors being dumped
to stderr as the queue is full.
This patch adds a live view of the aprsd logfile in
the admin ui. This uses a new Log QueueHandler and the
threads.logging_queue to push log entries into a queue.
The flask websockets server will push those log entries up
to a connected client browser.
When the admin user users the web ui to send a message
a new client instance is created with login credentials for
that particular message. This patch ensures that send_direct
uses that client.
This patch adjusts the py3-email-validation usage. Since we
upgraded to 1.0.2, the signature has changed. This patch adjusts
the signature usage so it works again.
This patch adds APRS KISS connectivity. I have tested this with
a running Direwolf install via either a serial KISS connection or
the optional new TCPKISS connection, both to Direwolf.
This adds the new required aioax25 python library for the underlying
KISS and AX25 support.
NOTE: For the TCPKISS connection, this patch requires a pull request
patch the aioax25 library to include a TCP Based KISS TNC client to
enable the TCPKISS client So you will need to pull down this PR
https://github.com/sjlongland/aioax25/pull/7
To enable this,
Edit your aprsd.yml file and enable one of the 2 KISS connections.
Only one is supported at a time.
kiss:
serial:
enabled: True
device: /dev/ttyS1
baudrate: 9600
or
kiss:
tcp:
enabled: True
host: "ip address/hostname of direwolf"
port: "direwolf configured kiss port"
This patch alters the Message object classes to be able to
send messages out via the aprslib socket connection to the APRS-IS
network on the internet, or via the direwolf KISS TCP socket,
depending on the origination of the initial message coming in.
If an APRS message comes in via APRS-IS, then replies will go out
APRS-IS. IF an APRS message comes in via direwolf, then replies
will go out via direwolf KISS TCP socket. Both can work at the same
time.
TODO: I need some real APRS message packets to verify that
the new thread is processing packets correctly through the plugins
and able to send the resulting messages back out to direwolf.
Have a hard coded callsign for now in the kissclient consumer call,
just so I can see messages coming in from direwolf. I dont' have an
APRS capable radio at the moment to send messages directly to direwolf.
Might need to write a simple python socket server to send fake APRS
messages to aprsd kiss, just for finishing up development.
Since all outbound messages have a send() method that starts
a separate there, there really is no reason for the transmit queue
thread at all. All it did was get a message from the queue and then
call send on it, which would start another thread. This removes that
intermediate TXThread. When you want to send a message just call
send() on the message object.
This patch updates the select timeouts for threads. This allows
threads to exit quicker when user hits CTRL-C.
Updates the KeepAlive Thread to include total packets.
This patch adds plugin rx/tx processing of packets.
This tracks how many messages a plugin processes (recieves) and
how many packets result in a plugin sending a message out.
This patch also adds a new plugins tab on the admin page.
This patch updates the APRSDPluginBase class to include
standard methods for allowing plugins to create, start, stop
threads that the plugin might need/use. Also update the aprsd-dev
to correctly start the threads and stop them for testing plugin
functionality.
Also added more unit tests and fake objects for unit tests.
This patch refactors how the recieved message processing happens.
We now handle all incoming packets the same. Removed the notification
thread to handle the watchlist packets. This is now done with a
unified plugins architecture that allows different capabilities
via the new plugin structure. All packets sent to us will be
sent through all of the plugins. It's the plugins job to decide what to
do with that packet or ignore it.
Email is no longer a special case for the most part. All email
functions have been migrated to the EmailPlugin, including starting the
EmailThread, which works in the background to check for new emails and
send those to the registered callsign. The EmailPlugin now starts the
EmailThread itself.
All plugins are now build on the new APRSDPluginBase which has a common
set of features. The APRSDPluginBase calls self.setup() upon creation,
which allows all plugins to do whatever they want for initiali startup.
The EmailPlugin uses setup() to start the EmailThread if email is
enabled.
This patch adds the ability for plugins to send multiple messages
back in response to a command/message. The plugin simple needs
to return a list of messages (Strings). Each string in that list
will result in a separate message being sent back to the originator
of the message.
This patch updates the Admin UI to display the APRS icon symbol
associated with a mic-e packet on the watch list tab for all
entries in the watch list.
This patch updates the notification thread to send all packets
through the notification plugins. The plugins themselves need to
do smart filter to not reply to every packet. This allows for
more interesting plugins.
Also fixed an issue with the messages tab in the admin ui, not
showing all of the recieved packets. The messages tab now also
sees all the packets that aprsd recieves.
This patch adds a new optional feature called Watch list.
Aprsd will filter IN all aprs packets from a list of callsigns.
APRSD will keep track of the last time a callsign has been seen.
When the configured timeout value has been reached, the next time
a callsign is seen, APRSD will send the next packet from that callsign
through the new notification plugins list.
The new BaseNotifyPlugin is the default core APRSD notify based plugin.
When it gets a packet it will construct a reply message to be sent
to the configured alert callsign to alert them that the seen callsign
is now on the APRS network.
This basically acts as a notification that your watched callsign list is
available on APRS.
The new configuration options:
aprsd:
watch_list:
# The callsign to send a message to once a watch list callsign
# is now seen on APRS-IS
alert_callsign: NOCALL
# The time in seconds to wait for notification.
# The default is 12 hours.
alert_time_seconds: 43200
# The list of callsigns to watch for
callsigns:
- WB4BOR
- KFART
# Enable/disable this feature
enabled: false
# The list of notify based plugins to load for
# processing a new seen packet from a callsign.
enabled_plugins:
- aprsd.plugins.notify.BaseNotifyPlugin
This patch also adds a new section in the Admin UI for showing the
watch list and the age of the last seen packet for each callsing since
APRSD startup.
This patch fixes a bug in the AckThread. The thread loop
was exiting after the first attempt to send the ack.
Thread loops have to return True, in order to be called again
as this is the mechanism in which aprsd gracefully shuts down all
threads.
This patch updates the admin UI to include 3 tabs
of content.
Charts
messages
config
The charts tab is the existing line charts.
The messages tab shows a list of RX (green) and TX (red) messages
from/to aprsd.
The config tab shows the config loaded at startup time.
This patch adds the dumping out of a flattened config to the log
at startup. This is helpful for seeing what aprsd server is actually
using for config entries at startup and since it's in the log, you can
reference it.
This patch adds a message counter for each plugin. When the regex for
a plugin passes and the message is pass into the plugin for processing,
that message is tracked. This message count is reported by the stats
tracking object now for the web admin ui.
This patch adds usage of update_checker to check to make sure the
version of APRSD being launched is the latest version. Also added a
call to upate_checker as part of the KeepAlive thread. It will
call update_check every hour. If there is no aprsd connectivitity,
the update check will silently fail.
This patch cleans up the layout of the admin web page stats graphs
as well as adds in the email stats. Added the titles to each
graph, so you know what you are looking at.
This patch reworks the stats object dict and includes more data.
Also includes aprsis last update timestamp (from last recieved message).
This is used to help determine if the aprsis server connection is still
alive and well.
This patch adds the aprsd-lnav.json formatting file.
This is useful when you want to tail the logfile with the lnav
log tailing app.
http://lnav.org/
To install the aprsd-lnav.json formatter
1) install lnav
2) lnav -i aprsd-lnav.json
3) lnav -C -- just to test it out
The next time you launch aprsd do it with this
aprsd server --loglevel DEBUG | lnav
This patch also updates the logging output from the flask
web service to 1) disable flask web url logging and 2)
use the same output format as the rest of the app.
Hemna
ranguli