This patch fixes a bug in the AckThread. The thread loop
was exiting after the first attempt to send the ack.
Thread loops have to return True, in order to be called again
as this is the mechanism in which aprsd gracefully shuts down all
threads.
This patch updates the admin UI to include 3 tabs
of content.
Charts
messages
config
The charts tab is the existing line charts.
The messages tab shows a list of RX (green) and TX (red) messages
from/to aprsd.
The config tab shows the config loaded at startup time.
This patch adds the dumping out of a flattened config to the log
at startup. This is helpful for seeing what aprsd server is actually
using for config entries at startup and since it's in the log, you can
reference it.
This patch adds a message counter for each plugin. When the regex for
a plugin passes and the message is pass into the plugin for processing,
that message is tracked. This message count is reported by the stats
tracking object now for the web admin ui.
This patch adds usage of update_checker to check to make sure the
version of APRSD being launched is the latest version. Also added a
call to upate_checker as part of the KeepAlive thread. It will
call update_check every hour. If there is no aprsd connectivitity,
the update check will silently fail.
This patch cleans up the layout of the admin web page stats graphs
as well as adds in the email stats. Added the titles to each
graph, so you know what you are looking at.
This patch reworks the stats object dict and includes more data.
Also includes aprsis last update timestamp (from last recieved message).
This is used to help determine if the aprsis server connection is still
alive and well.
This patch adds the aprsd-lnav.json formatting file.
This is useful when you want to tail the logfile with the lnav
log tailing app.
http://lnav.org/
To install the aprsd-lnav.json formatter
1) install lnav
2) lnav -i aprsd-lnav.json
3) lnav -C -- just to test it out
The next time you launch aprsd do it with this
aprsd server --loglevel DEBUG | lnav
This patch also updates the logging output from the flask
web service to 1) disable flask web url logging and 2)
use the same output format as the rest of the app.
This patch moves the default log format string and date format string
to the config file, so users can format the logs as they see fit.
The default log format also includes the file and line number that
posted the log entry.
The new entries in the config are here:
aprsd:
logformat: "String here"
dateformat: "string here"
This patch adds 2 items. First it adds the new StockPlugin,
which fetches stock quotes from yahoo finance rest API using
the yfinance python module.
2nd, the web interface contains a new url /plugins, which allows
aprsd to reload all of it's plugins from disk. This is useful for
development where the dev is editing an existing plugin and wants to
run the edited plugin without restarting aprsd itself. The /plugins
url requires admin login credentials.
TODO: would be nice to live reload the aprsd.yml config file, so plugin
reloading can start new plugins defined in aprsd.yml between /plugins
being reloaded.
The Email Thread has been unstable due to some IMAP servers
being crap. This patch wraps more of the imap server calls
in try except blocks to try and trap errors.
This patch fixes the CTRL-C signal_handler.
This patch also adds the new Messages WEB UI page
as well as the save url, which are both behind an
http basic auth.
The flask web service now has users in the config file
aprsd:
web:
users:
admin: <password>
When calling LocationPlugin with a callsign outside of the US,
the forecast.weather gov wasn't raising an exception. A valid json
dict was coming back, but it didn't have location data we were
expecting.
This patch adds the healthcheck app that uses the flask stats url
to fetch the internal stats of a running aprsd server. If the server is
up the stats will return and be checked for 'healthy' status.
IF the url fails to return, healthcheck will exit with -1. You can use
this script to restart aprsd if healthcheck exits with -1 status.
There is a check against the email thread. The email thread updates a
deadman's timer every 5 seconds. If that time gets older than 5
minutes, then healthcheck will say that's a failure and exit with -1.
You can call healthcheck and restart aprsd if it fails (exit -1)
This patch adds the stats object to collect statistics of
the running server. This also optionally adds the ability
to run a flask web service on a port to use as a keepalive
healthcheck.
This patch reorganizes the config file layout and options
to make more logical sense as well as make it more readable.
This breaks backwards compatibility.
This patch adds the openweathermap weather plugin.
Also adds a new config option to set the overall
units setting from imperial (default) to metric.
to change it add the following to the ~/.config/aprsd/aprsd.yaml
...
aprsd:
units: metric
This patch adds 2 new time plugins to allow admins to use their
opencagedata APIkey or openweathermap API key to fetch the timezone
from the lat/lon GPS coordinates for the callsign requesting the time.
This will enable fetching the time local to the ham radio's last beacon,
and not time local to the aprsd server instance running. If the
location is not found, then the timezone will default to UTC.
The 2 new plugins are
- aprsd.plugins.time.TimeOpenCageDataPlugin
Fetches timezone from lat/lon using the opencagedata api that can be
found here: https://opencagedata.com/dashboard#api-keys
This requires a new ~/.config/aprsd/aprsd.yml entry to specify the
api key.
opencagedata:
apiKey: <the api key hash here>
- aprsd.plugins.time.TimeOWMPlugin
Fetches the timezone from lat/lon using the openweathermap api
that can be found here: https://home.openweathermap.org/api_keys
This requires a new ~/.config/aprsd/aprsd.yml entry to specify the
api key.
openweathermap:
apiKey: <the api key hash here>
The existing time plugin had a hard coded PDT for pacific timezone,
when it wasn't. This patch adds some real timezone conversion from
utc to the tz of the running aprsd server. This will eventually allow
us to use either the tz of the running aprsd and/or the tz of the
calling callsign if we can just get the tz string from the location
beacon of the caller's callsign.
This patch adds a new CLI app called aprsd-dev. arpsd-dev is
used specifically for developing plugins. It allows you to run a
plugin directly without the need to run aprsd server.
This patch also adds the Weather Metar plugin called WxPlugin.
You can use it to fetch METAR from the nearest station for a callsign
or from a known METAR station id. Call WxPlugin with a message of
'wx' for closest metar station or 'wx KAUN' for metar at KAUN wx station
This patch re-enables some log.debug messages for email, to ensure
we can see emailthread is running correctly. Also adds a timestamp
to the query pending messages, so radios don't think it's a duplicate
message.
This patch adds a new add_config_comments() function in utils.py
that allows you to insert a comment string in a raw_yaml string
that's already been created from the yaml.dump() call.
This patch adds the login failure checking for the
send-message command as well as a new command line option
--no-ack. The new option enables sending the message directly
to aprs-is servers and then exiting immediately. It doesn't wait
for an ack to come back.
This patch adds the new required aprs.fi api key. This key is used
by 2 of the core plugins, locationPlugin and weatherPlugin.
You must set the apiKey in the config, or aprsd won't start.
This patch adds a check for a failed login to ARPS due to
LoginError. This accounts for bad accounts or username/password
failures. aprsd server will exit immediately upon failed login now.
This patch updates the restart_delayed method to accept the count of
messages to restart as well as the most_recent flag that sorts the
messages based on most recent first. If you want the oldest first,
then pass in False
This patch moves all of the plugins out of plugin.py
into their own separate plugins/<plugin>.py file. This
makes it easier to maintain each plugin.
NOTE: You will have to update your ~/.config/aprsd/aprsd.yml
to change the python location path for each plugin enabled.
For example:
OLD:
enabled_plugins:
- aprsd.plugin.EmailPlugin
TO NEW
enabled_plugins:
- aprsd.plugins.email.EmailPlugin
This patch disables the MX record checking for
email address shortcuts. verizon is a shit
smtp host that won't let you check emails as
existing/valid. Email validation still is checked
against RFC based regex for email address as
well as blacklist checking.
TODO(hemna): make this optionally enabled
by config file.
This patch moves all of the plugins out of plugin.py
into their own separate plugins/<plugin>.py file. This
makes it easier to maintain each plugin.
NOTE: You will have to update your ~/.config/aprsd/aprsd.yml
to change the python location path for each plugin enabled.
For example:
OLD:
enabled_plugins:
- aprsd.plugin.EmailPlugin
TO NEW
enabled_plugins:
- aprsd.plugins.email.EmailPlugin
This patch fixes the email.get_email_from_shortcut. It ensures that
if the lookup isn't found in the shortcut list, it simply returns
the original value. This patch also adds a unit test to specifically
test this function to always return the correct value.
This patch also updates the QueryPlugin to allow the configured user
to immediately resend all Delayed messages!
This patch updates the QueryPlugin to allow the configured user
to immediately Flush/delete all messages!
This patch adds the new MsgTrack object replacing the
global ack_dict. the ack_dict was not thread safe.
the new MsgTrack is a singleton object that keeps track of
all outbound TextMessage objects. When a TextMessage.send() is called
it is added to the MsgTrack object, and when an ack is received for that
message, the message is removed from the MsgTrack object.
TODO: Add an automatic mechanism for saving the messages in MsgTrack
so that when CTRL-C is called to exit aprsd server, then the MsgTrack
state is saved to storage. When aprsd server is started up again, add
the option to try and reload state of MsgTrack.
This patch also reworked the email thread into an APRSDThread object
that can exit gracefully with CTRL-C.
NOTE: Don't call sleep() with a long time (greater than 5 seconds), as
it causes a delay in exiting aprsd until the last sleep() finishes.
Since aprsd has so many threads now for processing incoming messages and
outgoing messages, we need to coordinate all thread operations so that
they don't block the exiting of the app.
This patch reworks the threading code for processing
messages. This patch also extends the aprslib IS class
to allow us to stop processing the consumer packets when
someone hits CTRL-C correctly. Alloing the app to exit.
This patch updates the messaging lib to use Message Objects
for each message type (text, ack) that know how to send
themselves with the same interface.
This patch refactored the process_packet method
and adjusted the logic for determining if we got
a message to filter on. We now look at the format
to make a determination. Also isolated the processing
of message packets, ack packets and mic-e packets into
their own functions.
This patch ensures that after we get an ACK, then proessing of the ack
message is complete and we don't try and send the ack through plugin
filtering.
Created send_ack_direct for the send-message command.
Also added logic to the send-message command to ensure we wait for an
ack from the command sent to APRSD and we also wait for a response
message and send an ack to that response before we exit.
This patch fixes a minor issue with the new send-message command
You now should use nargs to send the email command because it includes
a - as the start. click assumed that any -<foo> looks ike an argument.
So call aprsd with
aprsd send-command <callsign> -- -wb sendmap
This patch also adds -h as a help option for aprsd to make it simpler to
type.
This patch adds the VersionPlugin so you can remotely request the
version of aprsd that's running.
This commit adds the new send-message command for sending messages.
This also redoes the logging of sent/rx'd packets to a single method
which is syncrhonized, so we don't get intermixed log messages for
packets.
Also adds email address validation during startup, and
optionally disables the validation via a command line switch. without
email validation for production running aprsd, emails sent can turn up
garbage and cause issues when those emails are received by aprsd
message processing as invalid content.
This branch refactors the majority of main.py out into individual
modules to compartmentalize the code. Migrated all email related
features unti email.py, sending of messages into messaging.py
Also refactored all of the socket code to use aprslib for all APRS-IS
communication as well as message/packet processing.
Moved the email command into it's own Plugin.
Some emails that had an unknown character set. when that happens
we can't decode the body of the message properly, so the default body
string was being used, and was attempting to be decoded. Only byte
strings can be decoded, so the default string is now labeled as a byte
encoding.
This patch reworked the loading of plugins. Python2 is dead.
Previously was using the imp module to easily load the plugins from
a separate directory, which is a bit of a pita. Found a python3 lib
that takes care of the difference between py3.3, 3.4 and 3.5+ module
loading called "thesmuggler"
This patch fixes the Usage string for a call message
that isn't matched by any plugin.
Plugin object now must impleent a 'command_name' attribute
that is the usage string for that plugin.
This patch adds the new APRSD Command Plugin architecture.
All Comand plugins must implement the same object API, which includes
plugin object is subclass of APRSDPluginBase
version attribute
command_regex attribute
command method
When an APRS command is detected, then the regex is run against
the command. If the command_regex matches, then the plugin's
command() method will be called. If the command() method returns
a string, then that string is sent as a reply to the APRS caller.
A new aprs.yml config section is added to support selecting
which plugins to enable.
If you want all plugins enabled, then omit "enabled_plugins" entirely
from the aprs section of the config.
To load custom plugins:
1) create a directory with an __init__.py file
2) Add a plugin.py file that contains your plugin
Look at the exmaples directory for an example plugin.
This patch refactors the socket management
to use select, as well as refactor all of the
commands into a COMMAND_ENVELOPE dictionary.
This patch also adds the Dockerfile and
docker-compose.yml files
This patch includes lots of changes to tox environment for
automatically detecting pep8 failures, which can cause python2 vs
python3 failures after install.
The following tox commands have been added
tox -efmt-check - This checks the python syntax and formatting
tox -efmt - Automatically fixes python syntax formatting that
fmt-check complains about.
tox -etype-check - check on types
tox -elint - flake8 run
This patch also changes where the default config file is located.
The new location is ~/.config/aprsd/aprsd.yml
You can now also specify a custom config file on the command line
with the -c or --config option as well.
Hemna
Craig Lamparter