Commit Graph

599 Commits

Author SHA1 Message Date
Hemna 44696fbc56 Ignore callsign case while processing packets
This patch fixes an issue where aprsd was deciding if it was
supposed to process a packet destined for itself or not.  It was
making a case sensitive comparison.  This patch makes that comparison
case insensitive for the callsign itself.
2022-11-30 13:57:25 -05:00
Hemna 1a1d00242b Send GPS Beacon from webchat interface
This patchset allow getting the GPS coordinates from the browser's
geolocation API (which can be denied by user), then send's the GPS
coordinates to aprsd via socketio and then aprsd sends a beacon.

This allows the APRS network to know the location of the person running
the webchat app via browser so packets can get routed back to it.
2022-11-25 13:25:09 -05:00
Hemna d4bf0f1e3c Remove email validation
The package/library being used for email validation is basically
defunct now.
2022-11-25 09:29:41 -05:00
Hemna 5acddbd466 Fixed failing unit tests
This patch re-adds in the pytz lib for the generic time plugins.
2022-11-23 13:28:38 -05:00
Hemna 528bdb99e7 Removed TimeOpenCageDataPlugin
This patch removes the TimeOpenCageDataPlugin as it's been superceded
by the aprsd-timeopencage-plugin
2022-11-23 13:02:46 -05:00
Hemna fc1ca52593 Dump config with aprsd dev test-plugin
This patch adds the dumping of the config read for the
aprsd dev test-plugin command
2022-11-23 13:02:46 -05:00
Hemna 7d970cbe70 Got webchat working with KISS tcp
This patch reworks the KISS client to get rid of
aioax25 as it was too difficult to work with due to
heavy use of asyncio.

Switched to the kiss3 pypi library.
2022-11-23 13:01:43 -05:00
Hemna d717a22717 Added click auto_envvar_prefix
This allows setting environment variables that are
prefixed with APRSD_
2022-11-23 13:01:06 -05:00
Hemna 9b0c626b59 Update aprsd thread base class to use queue
This patch updates the main aprsd threads class to use
a shared queue to notify all aprsd thread classes they need
to exit.  This ensures any closing down of sockets, etc happens from
inside the context of the thread itself, not the MainThread that
calls stop.
2022-11-23 13:01:06 -05:00
Hemna 967959e7b3 Update packets to use wrapt
This patch updates the aprsd/packets.py to use wrapt for it's method
lock synchornization.
2022-11-23 13:01:06 -05:00
Hemna 2ce50d8861 Try sending raw APRSFrames to aioax25
This seems to work sending out, but still getting
third-party dropped packets as response from the local repeater.
2022-11-23 13:01:06 -05:00
Hemna ad79ed1261 Use new aprsd.callsign as the main callsign
This patch changes how aprsd identifies itself when connected to
any client, which is not relying on the login for each client.
There are 3 supported clients currently
aprsis,
tcpkiss
serialkiss.

Each client has their own potential login/callsign to connect
to the remote.  This patch tells aprsd to use the new config option
aprsd.callsign as a means to identify itself.  It will accept
packets as <aprsd.callsign> and reply as <aprsd.callsign> regardless
of which client object is being used to connect to the remote.

Note: this breaks backwards compatibility.  This patch now requires
the new config option
aprsd:
  callsign: <callsign>
2022-11-23 13:01:01 -05:00
Hemna 5f28788180 Fixed access to threads refactor 2022-11-23 13:00:37 -05:00
Hemna 585d55f10d Added webchat command
This patch adds the new aprsd webchat command which shows
a new webpage that allows you to aprsd chat with multiple
callsigns
2022-11-23 13:00:36 -05:00
Hemna 1ccb2f7695 Moved log.py to logging
Also renamed logging/logging.py to logging/rich.py
2022-11-23 13:00:36 -05:00
Hemna a62843920a Moved trace.py to utils
This patch moves trace.py to the utils directory
2022-11-23 13:00:36 -05:00
Hemna 29b84b453b Fixed pep8 errors 2022-11-23 13:00:36 -05:00
Hemna 347a6d69f7 Refactored threads.py
This patch creates a threads directory and separates out
the contents of threads.py into separate files in the
threads directory to make it easier to find and maintain.
2022-11-23 13:00:36 -05:00
Hemna bed060f1c5 Refactor utils to directory
This patch moves the utils.py to utils/__init__.py
and fuzzyclock.py to utils
and separates the ring_buffer to it's own file in utils
2022-11-23 13:00:36 -05:00
Walter A. Boring IV fdc8bfafc0
Merge pull request #96 from ranguli/fix-pypi-scraping
Fix #92 (PyPI scraping)
2022-11-01 14:01:37 -04:00
ranguli 9ffd320353 Fix PyPI scraping 2022-10-27 12:33:31 -02:30
ranguli cc2918377e Fix typo on exception 2022-10-26 16:46:50 -02:30
Hemna 1c052a63c0 Fixed up config option checking for KISS
This patch updates the config option checking for
required fields in the config yaml file.  Specifically
for the existence of the aprsd: section
and the required fields for the 3 supported client types
apris,
kiss serial,
kiss tcp
2022-02-21 16:04:33 -05:00
Hemna e739441268 Fix logging issue with log messages
This patch changes the base Message class to
ensure that all printing of the message class only
outputs the message in the truncated and bad word filtering
enabled in the log.
2022-02-11 10:03:02 -05:00
Hemna 6257c9ea90 FIX: logging exceptions
This patch fixes the logging of exceptions in the email
plugin.
2022-01-26 14:39:14 -05:00
Hemna e8100d8777 Removed debug code 2022-01-07 15:17:16 -05:00
Hemna 764730c123 Updated list-plugins
This patch updates the README.rst with the new format for
`aprsd list-plugins`.
2021-12-15 10:48:16 -05:00
Hemna 2f6e7e17e8 Added unit tests for dev test-plugin
Also added a check to make sure that the aprs_login
parameter is passed in for use as the fromcallsign.
2021-12-12 16:35:26 -05:00
Hemna a7bbde4a43 Send Message command defaults to config
The APRS_LOGIN and APRS_PASSWORD arguments now fallback
to the config file if it exists.

First it checks the passed in parameters, then checks the
environement vars, then checks the parsed config to find the
login and password.

This patch also adds unit tests for the send-message command to
check the fallback.
2021-12-12 16:13:08 -05:00
Hemna 3b9970c0e7 Fixed an KISS config disabled issue
This patch fixes a small bug when both KISS interfaces are disabled.
2021-12-11 07:46:43 -05:00
Hemna e57a2e2ffc Fixed a bug with multiple notify plugins enabled
This patch fixes an issue with the processing of packets
and updateing the watchlist.  Previously after the
notify plugin processed the packet it would update the watchlist.
This doesn't work when there are more than 1 notify plugins
enabled, only the first notify plugin seeing the packet will
recognize that the callsign is old.
2021-12-10 14:20:57 -05:00
Hemna 592b328956 Unify the logging to file and stdout
This patch updates the logging facility to ensure that
logging to a file works even when --quiet mode is selected.
Also update the listen and list-plugins command to show
a console.status line while waiting for results to come in.
2021-12-10 10:49:09 -05:00
Hemna cd62db95c1 Added new feature to list-plugins command
This patch updates the ouput of the list-plugins command.
This also adds the ability to show the available plugins
to install that are published packages on pypi.org.

This also shows the list of installed packages from pypi.org
2021-12-08 17:16:17 -05:00
Hemna 440c8d54ad Tightened up the packet logging 2021-12-07 15:00:38 -05:00
Hemna 8ea00e9888 Added unit tests for USWeatherPlugin, USMetarPlugin 2021-12-07 13:31:58 -05:00
Hemna 0ad791bdd9 Added NotifyPlugin unit tests and more
This patch restructures the unit tests for plugins.
This also adds unit tests for the NotifyPlugin
2021-12-07 11:25:14 -05:00
Hemna 96cc07d15f Small cleanup on packet logging
This patch reduces some of the leading whitespace
to the message/packet logging to the log file.
2021-12-06 14:35:49 -05:00
Hemna d3dd08714b Reduced the APRSIS connection reset to 2 minutes
The time in which the KeepAlive Thread would reset the APRS-IS
socket connection used to be 5 minutes.   This patch changes
that to 2 minutes.
2021-12-06 14:34:22 -05:00
Hemna 055835cb3c Fixed the NotifyPlugin
The watchlist notify plugin is supposed to send an APRS message
to the configured callsign.  This patch makes sure that the
message is sent to the notify_callsign
2021-12-06 14:11:34 -05:00
Hemna b5b286e75c Fixed some pep8 errors 2021-12-03 09:10:33 -05:00
Hemna 1233137caf Add tracing for dev command
This patch enables tracing output in the log for the dev
test-plugin command
2021-12-03 08:53:08 -05:00
Hemna 1d5f76defc Added python rich library based logging.
The python rich library is extensive and has a really nice
log format that is easier to read and has built in formatting
and coloring of the log output.

To enable rich logging add rich_logging: True in the config file.
2021-12-03 08:05:03 -05:00
Hemna 5e50792e80 fixed the failure during loading for objectstore
This patch fixes a silent failure of loading data from the objectstore
2021-11-13 15:07:28 -05:00
Hemna 6fb610582d Fixed dev command missing initialization
This patch fixes a few issues when running test-plugin command.
It was missing some initialization of the stats and packets classes.
2021-11-13 09:56:19 -05:00
Hemna bda2ef00dd Fix admin logging tab 2021-11-12 12:17:45 -05:00
Hemna 446484e631 Added new list-plugins command
This patch adds the new list-plugins command that shows the
list of built in plugins for APRSD.
2021-11-12 11:36:22 -05:00
Hemna a8a6b1aa07 Don't require check-version command to have a config
This patch removes the need for check-version to have a
config file.
2021-11-12 10:23:27 -05:00
Hemna 8842fb1b44 Healthcheck command doesn't need the aprsd.yml config
This patch updates the healthcheck command to not require
the aprsd.yml config file to exist.   The healthcheck
calls a running aprsd, collects the stats to determine if it's
healthy.
2021-11-10 11:52:51 -05:00
Hemna 152132b0ed Fix test failures 2021-11-10 11:51:21 -05:00
Hemna 7787dc1be4 Removed requirement for aprs.fi key
This removed the requirement of running APRSD for specifying
the aprs.fi key in the config file.  The plugins that need the
key have been updated to set enabled = False when the key is missing.
2021-11-10 11:01:10 -05:00
Hemna 9469410929 Removed stock plugin. 2021-11-09 15:02:54 -05:00
Hemna 88db485eb4 Removed the stock plugin
This patch removed the built in stock plugin from APRSD.
This helps clean up the requirement tree from the yfinance
python module that pulled in a lot of other requirements.

The stock plugin is it's own separate repo and module now.

https://github.com/hemna/aprsd-stock-plugin

https://pypi.org/project/aprsd-stock-plugin/
2021-11-09 14:53:20 -05:00
Hemna cdcb98e438 Cleaned up some verbose output & colorized output
Some commands now have some color if the shell detects it supports it.
2021-11-08 12:18:23 -05:00
Hemna 89727e2b8e Reworked all the common arguments
This patch reworks all the common arguments for the commands
and subcommands

--loglevel
--config_file
--quiet

These are all now processed in 1 place.
2021-11-08 11:52:41 -05:00
Hemna 617973f561 Fixed test-plugin 2021-11-05 16:40:07 -04:00
Hemna 9187b9781a Ensure common params are honored 2021-11-05 16:26:24 -04:00
Hemna 8287c09ce5 pep8 2021-11-05 14:38:23 -04:00
Hemna 82def598f0 Added healthcheck to the cmds
this patch moves the healthcheck to it's own command.
aprsd healthcheck
2021-11-05 14:21:36 -04:00
Hemna 3463c6eb96 Removed the need for FROMCALL in dev test-plugin
We already use the env var for APRS_LOGIN, so that is now
used for the test-plugin command.
Also cleaned up some help text
2021-11-05 14:05:24 -04:00
Hemna 2ead6a97da Pep8 failures 2021-11-05 13:42:27 -04:00
Hemna 7d0006b0a6 Refactor the cli
This patch refactors the cli to incorporate
the dev, send-message, listen commands into the main aprsd app.
This also moves the command line completion installer/show into
it's own subgroup.
2021-11-05 13:36:33 -04:00
Hemna 49f3ea8339 Fixed a problem with send-message command
This patch fixes a problem with the packets object
not being initialized correctly for the send-message command
from the command line.

Also adds the --wait-response option for send-message, which by
default is now False
2021-11-05 10:39:47 -04:00
Hemna cefb581bb8 Be more careful picking data to/from disk
This patch ensures that the pickle file is opened and closed correctly
as well as trapping for any exceptions that might occur while loading
a pickle file.
2021-11-02 08:52:59 -04:00
Hemna 95fecd2394 Ensure plugins are last to be loaded.
This patch initializes all of the MsgTrack, WatchList and SeenList
prior to the plugins loading.  Some plugins may kick off messages
being sent immediately.  So everything has to be ready to go
prior to the plugins being loaded.
2021-10-25 11:22:46 -04:00
Hemna c8c23e6185 Fixed email connecting to smtp server
Fixed an issue with not passing config in the smtp_connect
2021-10-25 11:12:29 -04:00
Hemna e009791b75 Converted MsgTrack to ObjectStoreMixin 2021-10-22 16:07:20 -04:00
Hemna 89701c8a70 Make sure SeenList update has a from in packet
This makes sure that the packet being processed by the seenlist
has a from address.
2021-10-21 08:40:40 -04:00
Hemna 66c5d85b89 Ensure PacketList is initialized 2021-10-20 15:48:35 -04:00
Hemna 8ee8b149f1 Added SIGTERM to signal_handler 2021-10-20 15:37:54 -04:00
Hemna 0d51634ec2 Enable configuring where to save the objectstore data
This patch adds a new config entry aprsd.save_location
which is the directory used to store and load the objectstore
data.
2021-10-20 14:39:12 -04:00
Hemna 135e21cd8d PEP8 cleanup 2021-10-20 14:10:54 -04:00
Hemna 4233827dea Added objectstore Mixin
This patch adds the new objectstore Mixin class that enables
classes that store their date in self.data as a serializeable dict,
to be able to be stored to disk at shutdown and loaded at startup.

The SeenList and WatchList are now saved/loaded to/from disk.
2021-10-20 14:07:22 -04:00
Hemna 9b2212245f Added -num option to aprsd-dev test-plugin
This allows the user to specify how many times in a loop
to call the plugin.  The Default is 1.
2021-10-20 11:46:55 -04:00
Hemna 9150f3b6ff Only call stop_threads if it exists 2021-10-10 14:50:04 -04:00
Hemna 278bb6e882 Added new SeenList
This patch adds the seen list feature.  It tracks the callsign of every
packet that aprsd sees.
2021-10-09 14:29:25 -04:00
Hemna 004795dbf1 Added plugin version to stats reporting
This patch adds version to the plugin stats collected.
2021-10-09 09:31:51 -04:00
Hemna 3b7924b13d Added new HelpPlugin
This patch adds the always enabled HelpPlugin.  This plugin
now will respond to the 'help' or 'h' commands that will
automatically build a help string based on the number of
enabled plugins.  It will also respond to
help <plugin> with the plugin specific help
2021-10-08 12:01:04 -04:00
Hemna 2bf85db21b Updated aprsd-dev to use config for logfile format
This patch updates the aprsd-dev command's log file format
to use what's defined as the default and/or use the config file
setting like aprsd server does.
2021-10-08 08:47:24 -04:00
Hemna 5b17228811 removed usage of config.check_config_option
check_config_option has been superceeded by the config UserDict
object's ability to see if a config option exists.
2021-10-07 10:11:48 -04:00
Hemna 725bb2fe35 Fixed send-message after config/client rework
This patch fixes the send-message from the command line
ability after the complete rework of the client classes.
2021-10-07 10:05:19 -04:00
Hemna f8d87d05bb Fixed issue with flask config
Flask was trying to serialize the UserDict object.  Use the
data (dict) inside of it instead.
2021-10-06 15:17:09 -04:00
Hemna 30671cbdbc Added some server startup info logs
This patch adds some general info logs around starting the
client connection as well as loading the plugins.
2021-10-06 12:55:17 -04:00
Hemna fdc8c0cd66 Increase email delay to +10
This patch updates the increasing of the email check delay to += 10
seconds instead of +1.
2021-10-06 12:12:49 -04:00
Hemna c097c31258 Updated dev to use plugin manager
Also ensure that main creates the client prior to starting the
plugins.
2021-10-06 12:09:52 -04:00
Hemna e3c5c7b408 Fixed notify plugins
The notify base filter() was missing the @hookimpl
2021-10-06 12:08:29 -04:00
Hemna 491644ece6 Added new Config object.
The config object now has builtin dot notation getter with default

config.get("some.path.here", default="Not found")
2021-10-04 15:37:14 -04:00
Hemna a6ed7b894b Fixed email plugin's use of globals
The email plugin was still using globals for tracking
the check_email_delay as well as the config.  This
patch creates a new singleton thread safe mechanism for
check_email_delay with the EmailInfo class.
2021-10-04 11:36:13 -04:00
Hemna 270be947b5 Refactored client classes
This patch completely refactors and simplifies how the clients
are created and used.  There is no need now to have a separate
KISSRXThread.  Since all the custom work for the KISS client is
encapsulated in the kiss client itself, the same RX thread and
callback mechanism works for both the APRSIS client and KISS Client
objects.  There is also no need to determine which transport
(aprsis vs kiss) is being used at runtime by any of the messages
objects.  The same API works for both APRSIS and KISS Client objects
2021-09-17 09:32:30 -04:00
Hemna 23e3876e7b Refactor utils usage
This patch separates out the config from the utils.py
utils.py has grown into a catchall for everything and this
patch is the start of that cleanup.
2021-09-16 17:08:30 -04:00
Hemna 560e152742 Fixed issue of aprs-is missing keepalive
Started noticing that aprs-is keepalive messages just stop
getting sent.  This causes aprsd to basically disconnect from
the APRS network.  Added a check into the KeepAlive thread to
restart the aprs-is connecter if the last time we got a keepalive
from apris is > 5 minutes.
2021-09-13 13:22:06 -04:00
Hemna 69b215d4d8 Fixed packet processing issue with aprsd send-message
This patch adds the missing PacketList initialization
for the send-message command
2021-09-10 15:39:07 -04:00
Hemna 1b9a9935fc Enable plugins to return message object
This patch enables the ability for plugins to return:
* string
* list of strings
* message object
* list of strings and message ojects

Each string will be encapsulated in a message object prior being sent.
each message object will be sent directly.
Each list will be iterated over and processed according to the above 2
options.
2021-09-08 14:45:15 -04:00
Hemna 3faf41b203 Added enabled flag for every plugin object
This allows the admin interface to see which plugins are registered and
enabled.  Enabled is a flag that is set in the setup() method of the
plugin.  This gives the plugin developer a chance to disable the plugin
if something isn't right at setup time.   This allows aprsd to ignore
plugins that are registered but not emabled.
2021-09-08 14:25:12 -04:00
Hemna 7e6dffb34b Ensure plugin threads are valid
This patch makes sure that the plugin threads returned from
create_threads is somewhat valid
2021-09-08 13:44:20 -04:00
Hemna 9eff99dde7 Removed fixed size on logging queue
If the logging queue gets full, due to a maxsize being set,
then any further logs will result on lots of errors being dumped
to stderr as the queue is full.
2021-09-07 13:43:48 -04:00
Hemna d6b3df93f1 Added Logfile tab in Admin ui
This patch adds a live view of the aprsd logfile in
the admin ui.  This uses a new Log QueueHandler and the
threads.logging_queue to push log entries into a queue.
The flask websockets server will push those log entries up
to a connected client browser.
2021-09-07 13:13:36 -04:00
Hemna dfaf3aa3d1 Update dev.py
This patch ensures a valid packet is passed into the plugin prior to
testing.
2021-09-03 16:48:00 -04:00
Hemna 62ce84b315 Allow passing in aprsis_client
When the admin user users the web ui to send a message
a new client instance is created with login credentials for
that particular message.  This patch ensures that send_direct
uses that client.
2021-09-02 11:17:15 -04:00
Hemna 8ada789d4d Fixed a problem with the AVWX plugin not working
the regex for the plugin was not matching correctly
2021-09-02 11:06:25 -04:00
Hemna 558710d348 Remove some noisy trace in email plugin
This removes the trace decorators from the email login
functions.  They have been stable for a while now.
2021-09-02 10:03:43 -04:00
Hemna 1ea6c05dec Fixed issue at startup with notify plugin
Ensure that the aprsis client is configured prior to starting
any plugins.
2021-09-02 09:54:13 -04:00
Hemna 0f6df5fc05 Fixed email validation
This patch adjusts the py3-email-validation usage.  Since we
upgraded to 1.0.2, the signature has changed.  This patch adjusts
the signature usage so it works again.
2021-09-02 09:43:33 -04:00
Hemna 1635feb820 Removed values from forms 2021-09-02 09:12:44 -04:00
Hemna c58031d772 Added send-message to the main admin UI 2021-09-02 08:56:25 -04:00
Hemna c537b54df6 Updated requirements 2021-09-01 17:38:59 -04:00
Hemna 84ce60bc50 Cleaned up some pep8 failures 2021-09-01 17:11:35 -04:00
Hemna c941379a5c Upgraded the send-message POC to use websockets
This patch updates the send message Admi page to use
websockets.  It makes updates to the messages list instant.
2021-09-01 17:10:59 -04:00
Hemna 23cbf32814 New Admin ui send message page working. 2021-09-01 17:10:13 -04:00
Hemna 6d3258e833 Send Message via admin Web interface
This patch adds the ability to send a message from the
admin interface's send-message.html page.
2021-09-01 17:06:56 -04:00
Hemna ca438c9c60 Updated Admin UI to show KISS connections
This updates the top area of the Admin UI to reflect the
connection type (aprs-is vs kiss).
2021-09-01 16:39:50 -04:00
Hemna f4dee4b202 Got TX/RX working with aioax25+direwolf over TCP
This patch gets APRSD fully working with the TCPKISS socket
to direwolf.
2021-09-01 14:48:22 -04:00
Hemna 54c9a6b55a Rebased from master 2021-08-30 13:34:25 -04:00
Hemna b53e2ba7fe Added the ability to use direwolf KISS socket
This patch adds APRS KISS connectivity.  I have tested this with
a running Direwolf install via either a serial KISS connection or
the optional new TCPKISS connection, both to Direwolf.

This adds the new required aioax25 python library for the underlying
KISS and AX25 support.

NOTE: For the TCPKISS connection, this patch requires a pull request
patch the aioax25 library to include a TCP Based KISS TNC client to
enable the TCPKISS client  So you will need to pull down this PR
https://github.com/sjlongland/aioax25/pull/7

To enable this,
  Edit your aprsd.yml file and enable one of the 2 KISS connections.
  Only one is supported at a time.

  kiss:
     serial:
         enabled: True
         device: /dev/ttyS1
         baudrate: 9600

  or

  kiss:
      tcp:
          enabled: True
          host: "ip address/hostname of direwolf"
          port: "direwolf configured kiss port"

This patch alters the Message object classes to be able to
send messages out via the aprslib socket connection to the APRS-IS
network on the internet, or via the direwolf KISS TCP socket,
depending on the origination of the initial message coming in.

If an APRS message comes in via APRS-IS, then replies will go out
APRS-IS.  IF an APRS message comes in via direwolf, then replies
will go out via direwolf KISS TCP socket.   Both can work at the same
time.

TODO:  I need some real APRS message packets to verify that
the new thread is processing packets correctly through the plugins
and able to send the resulting messages back out to direwolf.

Have a hard coded callsign for now in the kissclient consumer call,
just so I can see messages coming in from direwolf.  I dont' have an
APRS capable radio at the moment to send messages directly to direwolf.
Might need to write a simple python socket server to send fake APRS
messages to aprsd kiss, just for finishing up development.
2021-08-30 13:28:39 -04:00
Hemna ec92b07e31 Silence some log noise
Removed an email thread log at the start of the loop.
Also bumped the Keepalivethread time to 60 seconds
2021-08-25 08:25:36 -04:00
Hemna 61967b5fe8 Removed TXThread
Since all outbound messages have a send() method that starts
a separate there, there really is no reason for the transmit queue
thread at all.  All it did was get a message from the queue and then
call send on it, which would start another thread.  This removes that
intermediate TXThread.   When you want to send a message just call
send() on the message object.
2021-08-24 15:22:50 -04:00
Hemna 2e9b42d7af Added days to uptime string formatting
The uptime string formatter was missing days.
2021-08-24 14:08:24 -04:00
Hemna 0f384b0e85 Updated select timeouts
This patch updates the select timeouts for threads.  This allows
threads to exit quicker when user hits CTRL-C.

Updates the KeepAlive Thread to include total packets.
2021-08-24 13:31:33 -04:00
Hemna 8b5f21eece Rebase from master and run gray
This patch is a rebase of master after the introduction
of switching from black to gray code formatting.
2021-08-23 14:08:14 -04:00
Hemna 8e627c98b3 Added tracking plugin processing
This patch adds plugin rx/tx processing of packets.
This tracks how many messages a plugin processes (recieves) and
how many packets result in a plugin sending a message out.

This patch also adds a new plugins tab on the admin page.
2021-08-23 13:45:01 -04:00
Hemna 86777d838c Added threads functions to APRSDPluginBase
This patch updates the APRSDPluginBase class to include
standard methods for allowing plugins to create, start, stop
threads that the plugin might need/use.  Also update the aprsd-dev
to correctly start the threads and stop them for testing plugin
functionality.
Also added more unit tests and fake objects for unit tests.
2021-08-23 13:44:58 -04:00
Hemna 5f4cf89733 Refactor Message processing and MORE
This patch refactors how the recieved message processing happens.
We now handle all incoming packets the same.  Removed the notification
thread to handle the watchlist packets.  This is now done with a
unified plugins architecture that allows different capabilities
via the new plugin structure.  All packets sent to us will be
sent through all of the plugins.  It's the plugins job to decide what to
do with that packet or ignore it.

Email is no longer a special case for the most part.  All email
functions have been migrated to the EmailPlugin, including starting the
EmailThread, which works in the background to check for new emails and
send those to the registered callsign.   The EmailPlugin now starts the
EmailThread itself.

All plugins are now build on the new APRSDPluginBase which has a common
set of features.  The APRSDPluginBase calls self.setup() upon creation,
which allows all plugins to do whatever they want for initiali startup.
The EmailPlugin uses setup() to start the EmailThread if email is
enabled.
2021-08-23 13:43:53 -04:00
Hemna e175f77347 Use Gray instead of Black for code formatting.
The Black code formatter sucks with respect to function
declarations with a lot of params.  Completely unreadable.
2021-08-23 13:32:09 -04:00
Hemna f1f8aed8c4 Fixed LOG.debug issue in weather plugin 2021-08-19 19:07:45 -04:00
Hemna d71b0df314 Fixed aprsd-dev
This patch fixes running the aprsd-dev plugin development tool.
It currently only works for message based plugins.
2021-08-18 20:14:03 -04:00
Hemna 349250685b Enable multiple replies for plugins
This patch adds the ability for plugins to send multiple messages
back in response to a command/message.  The plugin simple needs
to return a list of messages (Strings).  Each string in that list
will result in a separate message being sent back to the originator
of the message.
2021-08-13 12:36:48 -04:00
Hemna 840c8a990e Put in a fix for aprslib parse exceptions
This patch adds a fix for the aprslib consumer function
to ensure that we don't bail when logging a ParseError
2021-08-13 10:31:45 -04:00
Hemna ed4995b6eb Fixed time plugin 2021-07-29 20:17:58 -04:00
Hemna 6740ff80be Updated the charts Added the packets chart
This patch adds the APRS Packets chart to the charts admin ui.
Also moves the raw json as it's own tab
2021-07-22 20:44:20 -04:00
Hemna be8179415a Added showing symbol images to watch list
This patch updates the Admin UI to display the APRS icon symbol
associated with a mic-e packet on the watch list tab for all
entries in the watch list.
2021-07-21 09:21:04 -04:00
Hemna 2fceba10e1 Reworked the notification threads and admin ui.
This patch updates the notification thread to send all packets
through the notification plugins.   The plugins themselves need to
do smart filter to not reply to every packet.  This allows for
more interesting plugins.

Also fixed an issue with the messages tab in the admin ui, not
showing all of the recieved packets.   The messages tab now also
sees all the packets that aprsd recieves.
2021-07-17 14:30:29 -04:00
Hemna 3d38402be2 Fixed small bug with packets get_packet_type
This fixes an issue with trying to decode the packet type.
Also updated some of the log entries.
2021-07-16 12:15:04 -04:00
Hemna 7dc4fb3e77 Move version string output to top of log 2021-07-16 12:11:51 -04:00
Hemna 1a1fcba1c4 Add new watchlist feature
This patch adds a new optional feature called Watch list.
Aprsd will filter IN all aprs packets from a list of callsigns.
APRSD will keep track of the last time a callsign has been seen.
When the configured timeout value has been reached, the next time
a callsign is seen, APRSD will send the next packet from that callsign
through the new notification plugins list.

The new BaseNotifyPlugin is the default core APRSD notify based plugin.
When it gets a packet it will construct a reply message to be sent
to the configured alert callsign to alert them that the seen callsign
is now on the APRS network.

This basically acts as a notification that your watched callsign list is
available on APRS.

The new configuration options:
aprsd:
    watch_list:
        # The callsign to send a message to once a watch list callsign
        # is now seen on APRS-IS
        alert_callsign: NOCALL
        # The time in seconds to wait for notification.
        # The default is 12 hours.
        alert_time_seconds: 43200
        # The list of callsigns to watch for
        callsigns:
          - WB4BOR
          - KFART
        # Enable/disable this feature
        enabled: false
        # The list of notify based plugins to load for
        # processing a new seen packet from a callsign.
        enabled_plugins:
        - aprsd.plugins.notify.BaseNotifyPlugin

This patch also adds a new section in the Admin UI for showing the
watch list and the age of the last seen packet for each callsing since
APRSD startup.
2021-07-16 08:31:38 -04:00
Hemna 562ae52c1e Fixed the Ack thread not resending acks
This patch fixes a bug in the AckThread.  The thread loop
was exiting after the first attempt to send the ack.
Thread loops have to return True, in order to be called again
as this is the mechanism in which aprsd gracefully shuts down all
threads.
2021-07-15 14:11:30 -04:00
Hemna 3c45d8bd0f reworked the admin ui to use semenatic ui more 2021-07-14 15:00:23 -04:00
Hemna 5afc7fb664 Added messages count to admin messages list.
This patch adds a simple count of packets shown in the
messages list on the admin ui.
2021-07-14 10:29:12 -04:00
Hemna de62579852 Add admin UI tabs for charts, messages, config
This patch updates the admin UI to include 3 tabs
of content.
Charts
messages
config

The charts tab is the existing line charts.
The messages tab shows a list of RX (green) and TX (red) messages
from/to aprsd.
The config tab shows the config loaded at startup time.
2021-07-12 12:12:14 -04:00
Hemna 1c66555450 Removed a noisy debug log 2021-07-09 15:22:53 -04:00
Hemna 9a1ab1c0d6 Dump out the config during startup
This patch adds the dumping out of a flattened config to the log
at startup.  This is helpful for seeing what aprsd server is actually
using for config entries at startup and since it's in the log, you can
reference it.
2021-07-05 10:57:22 -04:00
Hemna 3ae5717452 Added message counts for each plugin.
This patch adds a message counter for each plugin.  When the regex for
a plugin passes and the message is pass into the plugin for processing,
that message is tracked.  This message count is reported by the stats
tracking object now for the web admin ui.
2021-06-17 16:37:47 -04:00
Hemna 17302aa76d Added aprsd version checking
This patch adds usage of update_checker to check to make sure the
version of APRSD being launched is the latest version.  Also added a
call to upate_checker as part of the KeepAlive thread.  It will
call update_check every hour.  If there is no aprsd connectivitity,
the update check will silently fail.
2021-05-04 10:06:43 -04:00
Hemna fc1ee19516 Removed debug log for KeepAlive thread
No need to dump out the length of the keepalive string for now.
2021-04-05 14:14:33 -04:00
Hemna 23c219f0d2 Fixed version unit tests 2021-04-05 08:55:46 -04:00
Hemna 7b019d24f0 Updated stats output for KeepAlive thread
Also added the aprsd uptime to the VersionPlugin
2021-04-02 18:54:00 -04:00
Hemna 3ac42edd82 Force all the graphs to 0 minimum
This patch updates all the graphs to have a minimum
Y value of 0.  Doesn't make sense to have negative messages.
2021-04-02 11:57:06 -04:00
Hemna d6806c429c Added email messages graphs
This patch cleans up the layout of the admin web page stats graphs
as well as adds in the email stats.  Added the titles to each
graph, so you know what you are looking at.
2021-04-02 11:47:52 -04:00
Hemna bf8d2c6088 Reworked the stats dict output and healthcheck
This patch reworks the stats object dict and includes more data.
Also includes aprsis last update timestamp (from last recieved message).
This is used to help determine if the aprsis server connection is still
alive and well.
2021-04-01 23:12:25 -04:00
Hemna 123266c9ad Added callsign to the web index page
This patch adds the aprs-is server callsign that aprsd is listening
on for messages.
2021-03-31 11:32:09 -04:00
Hemna 34d2c31d90 Added log config for flask and lnav config file
This patch adds the aprsd-lnav.json formatting file.
This is useful when you want to tail the logfile with the lnav
log tailing app.

http://lnav.org/

To install the aprsd-lnav.json formatter
1) install lnav
2) lnav -i aprsd-lnav.json
3) lnav -C  -- just to test it out

The next time you launch aprsd do it with this
aprsd server --loglevel DEBUG | lnav

This patch also updates the logging output from the flask
web service to 1) disable flask web url logging and 2)
use the same output format as the rest of the app.
2021-03-31 11:07:39 -04:00
Hemna d1a2a14370 Added showing APRS-IS server to stats
This patch updates the client.py to collect which APRS-IS server
that aprsd is connected to and displays that info on the stats web page.
2021-03-30 10:43:31 -04:00
Hemna fb979eda94 Provide an initial datapoint on rendering index
This patch adds a single data point when rendering the
initial stats for the index page.
2021-03-30 10:18:56 -04:00
Hemna 6297ebeb67 Make the index page behind auth
This patch makes the index page ask for login/password in order
to see the stats.
2021-03-30 09:55:14 -04:00
Hemna f10372b320 Added acks with messages graphs 2021-03-26 11:13:32 -04:00
Hemna c7d10f53a3 Updated web stats index to show messages and ram usage
This patch updates the main index page to show both the
graph of tx/rx messages as well as peak/current ram usage.
2021-03-24 16:07:09 -04:00
Hemna f211e5cabb Added aprsd web index page
This patch adds an index page for the flask web server that users
can hit at /
2021-03-24 10:45:03 -04:00
Hemna 0a038dae44 Added log format and dateformat to config file
This patch moves the default log format string and date format string
to the config file, so users can format the logs as they see fit.
The default log format also includes the file and line number that
posted the log entry.

The new entries in the config are here:
aprsd:
  logformat: "String here"
  dateformat: "string here"
2021-02-25 13:32:50 -05:00
Hemna e6cafeb3d2 Added plugin live reload and StockPlugin
This patch adds 2 items.  First it adds the new StockPlugin,
which fetches stock quotes from yahoo finance rest API using
the yfinance python module.

2nd, the web interface contains a new url /plugins, which allows
aprsd to reload all of it's plugins from disk.  This is useful for
development where the dev is editing an existing plugin and wants to
run the edited plugin without restarting aprsd itself.  The /plugins
url requires admin login credentials.

TODO: would be nice to live reload the aprsd.yml config file, so plugin
reloading can start new plugins defined in aprsd.yml between /plugins
being reloaded.
2021-02-18 16:31:52 -05:00
Hemna 9ba44a076c Removed some noisy debug log.
Use the tracing instead to enable the debugging of
email calls
2021-02-10 10:37:39 -05:00
Hemna 131919bdfb Wrap another server call with try except
Dreamhost email is total garbage.  Stop using it.
2021-02-05 15:32:36 -05:00
Hemna a5cc274ff5 Wrap all imap calls with try except blocks
The Email Thread has been unstable due to some IMAP servers
being crap.  This patch wraps more of the imap server calls
in try except blocks to try and trap errors.
2021-02-03 11:00:20 -05:00
Craig Lamparter 47135c6086 EmailThread was exiting because of IMAP timeout, added exceptions for this 2021-02-02 11:13:17 -08:00
Hemna db2b537317 Added memory tracing in keeplive 2021-01-29 11:02:21 -05:00
Hemna 0b44fc08eb Fixed tox pep8 failure for trace 2021-01-29 10:15:20 -05:00
Hemna af48c43eb2 Added tracing facility
You can enable debug tracing iff loglevel == DEBUG AND
config file has aprsd:trace:True
2021-01-29 10:07:49 -05:00
Hemna 94bad95e26 Fixed email login issue.
This patch undoes an overzealous reworking of the
config.  the arps login didn't move.
2021-01-26 13:33:39 -05:00
Craig Lamparter 57d768e010 duplicate email messages from RF would generate usage response 2021-01-26 09:18:43 -08:00
Hemna 030b02551f Enable debug logging for smtp and imap
Add the new config options for
aprsd:
  email:
    imap:
      debug: True

    smtp:
      debug: True
2021-01-25 16:16:08 -05:00
Craig Lamparter cfb172481d more debug around email thread 2021-01-25 12:54:24 -08:00
Craig Lamparter 3ca0eeff56 debug around EmailThread hanging or vanishing 2021-01-25 12:24:20 -08:00
Hemna c1e6792721 Fixed resend email after config rework
This patch fixes 1 missed access to the shortcuts after
the restructuring of the config file
2021-01-25 15:15:53 -05:00
Hemna aa290692ab Added flask messages web UI and basic auth
This patch fixes the CTRL-C signal_handler.
This patch also adds the new Messages WEB UI page
as well as the save url, which are both behind an
http basic auth.

The flask web service now has users in the config file
aprsd:
  web:
    users:
      admin: <password>
2021-01-25 11:24:39 -05:00
Hemna 0d18e54969 Fixed an issue with LocationPlugin
When calling LocationPlugin with a callsign outside of the US,
the forecast.weather gov wasn't raising an exception.  A valid json
dict was coming back, but it didn't have location data we were
expecting.
2021-01-22 16:32:49 -05:00
Hemna 51894bbab8 Cleaned up the KeepAlive output
This patch cleans up the KeepAlive output a bit.
2021-01-22 16:05:48 -05:00
Hemna c870207a96 Added healthcheck app
This patch adds the healthcheck app that uses the flask stats url
to fetch the internal stats of a running aprsd server.  If the server is
up the stats will return and be checked for 'healthy' status.
IF the url fails to return, healthcheck will exit with -1.  You can use
this script to restart aprsd if healthcheck exits with -1 status.

There is a check against the email thread.  The email thread updates a
deadman's timer every 5 seconds.   If that time gets older than 5
minutes, then healthcheck will say that's a failure and exit with -1.

You can call healthcheck and restart aprsd if it fails (exit -1)
2021-01-22 12:51:11 -05:00
Hemna 5c949343ec Added Flask web thread and stats collection
This patch adds the stats object to collect statistics of
the running server.  This also optionally adds the ability
to run a flask web service on a port to use as a keepalive
healthcheck.
2021-01-21 20:58:47 -05:00
Hemna 9630279d14 First hack at flask 2021-01-21 15:11:44 -05:00
Hemna 982f24c5f5 Allow email to be disabled.
The config file defaults to email being off now.  This requires
the user to set the email settings anyway, so the default is off.
2021-01-21 13:50:19 -05:00
Hemna ce5b09233c Reworked the config file and options
This patch reorganizes the config file layout and options
to make more logical sense as well as make it more readable.

This breaks backwards compatibility.
2021-01-21 13:32:19 -05:00
Hemna a35cb04ca7 Updated documentation and config output
This patch reformats the sample-config output for more
informative comments for the 3 external services:
openweathermap
opencagedata
avwx-api
2021-01-21 10:05:49 -05:00
Hemna fefb626c97 Fixed extracting lat/lon
This patch fixes an issue when aprs.fi returns a non error, but
doesn't have any real entries as the response
2021-01-20 19:51:59 -05:00
Hemna 2349024539 Added openweathermap weather plugin
This patch adds the openweathermap weather plugin.
Also adds a new config option to set the overall
units setting from imperial (default) to metric.

to change it add the following to the ~/.config/aprsd/aprsd.yaml

...
aprsd:
  units: metric
2021-01-20 16:12:17 -05:00
Hemna fc3a747aa4 Added new time plugins
This patch adds 2 new time plugins to allow admins to use their
opencagedata APIkey or openweathermap API key to fetch the timezone
from the lat/lon GPS coordinates for the callsign requesting the time.

This will enable fetching the time local to the ham radio's last beacon,
and not time local to the aprsd server instance running.  If the
location is not found, then the timezone will default to UTC.

The 2 new plugins are
- aprsd.plugins.time.TimeOpenCageDataPlugin
   Fetches timezone from lat/lon using the opencagedata api that can be
   found here:  https://opencagedata.com/dashboard#api-keys

   This requires a new ~/.config/aprsd/aprsd.yml entry to specify the
   api key.
   opencagedata:
       apiKey: <the api key hash here>

- aprsd.plugins.time.TimeOWMPlugin
   Fetches the timezone from lat/lon using the openweathermap api
   that can be found here:  https://home.openweathermap.org/api_keys

   This requires a new ~/.config/aprsd/aprsd.yml entry to specify the
   api key.
   openweathermap:
       apiKey: <the api key hash here>
2021-01-20 10:19:49 -05:00
Hemna 9f38fd179e Fixed TimePlugin timezone issue
The existing time plugin had a hard coded PDT for pacific timezone,
when it wasn't.   This patch adds some real timezone conversion from
utc to the tz of the running aprsd server.   This will eventually allow
us to use either the tz of the running aprsd and/or the tz of the
calling callsign if we can just get the tz string from the location
beacon of the caller's callsign.
2021-01-19 11:26:07 -05:00
Craig Lamparter ca05676c98 remove fortune white space 2021-01-17 08:02:45 -08:00
Craig Lamparter 83f42dd7b7 Merge branch 'master' of https://github.com/craigerl/aprsd 2021-01-17 07:57:10 -08:00
Craig Lamparter 7de2820caa change query char from ? to ! 2021-01-17 07:55:59 -08:00
Hemna cc8fd178ce Added aprsd-dev plugin test cli and WxPlugin
This patch adds a new CLI app called aprsd-dev.  arpsd-dev is
used specifically for developing plugins.  It allows you to run a
plugin directly without the need to run aprsd server.

This patch also adds the Weather Metar plugin called WxPlugin.
You can use it to fetch METAR from the nearest station for a callsign
or from a known METAR station id.  Call WxPlugin with a message of
'wx' for closest metar station or 'wx KAUN' for metar at KAUN wx station
2021-01-15 22:30:34 -05:00
Hemna 0e9cfdd847 Fix tox tests. 2021-01-15 12:06:09 -05:00
Craig Lamparter f538fb26ae fix usage statement 2021-01-15 08:51:12 -08:00
Hemna d81bfd6fd5 Enabled some emailthread messages and added timestamp
This patch re-enables some log.debug messages for email, to ensure
we can see emailthread is running correctly.  Also adds a timestamp
to the query pending messages, so radios don't think it's a duplicate
message.
2021-01-15 11:12:43 -05:00
Hemna 7486770bdc Fixed main server client initialization
This fixes the usage of the singleton client class which
houses/creates the aprslib client.  We were getting multiple
logins, now we get one.
2021-01-14 15:44:07 -05:00
Hemna 72fa550250 Removed flask code 2021-01-14 14:36:36 -05:00
Hemna 0aa905ebba Changed default log level to INFO
Also adjusted some of the logging for main, messaging and threads
to be more sane
2021-01-14 14:32:59 -05:00
Craig Lamparter 0b5c9dacf0 fix query command syntax ?, ?3, ?d(elete), ?a(ll) 2021-01-14 11:21:26 -08:00
Hemna 4ca5c29d49 Fixed latitude reporting in locationPlugin
The latitude was always 0, because it was the
altitude.
2021-01-14 14:02:43 -05:00
Craig Lamparter 7e3b95fd01 get rid of some debug noise from tracker and email delay 2021-01-14 10:51:00 -08:00
Hemna 18acd64334 fixed sample-config double print 2021-01-14 13:43:10 -05:00
Craig Lamparter e11a84bf05 make sample config easier to interpret 2021-01-14 10:41:40 -08:00
Hemna 42b2e227e1 Fixed comments 2021-01-14 12:44:58 -05:00
Hemna cdde9c290b Added the ability to add comments to the config file
This patch adds a new add_config_comments() function in utils.py
that allows you to insert a comment string in a raw_yaml string
that's already been created from the yaml.dump() call.
2021-01-14 12:38:30 -05:00
Hemna 54072a2103 Added --raw format for sending messages
aprsd send-message --raw "RAW APRS MESSAGE HERE"
2021-01-12 14:50:49 -05:00
Hemna f022a3e421 Fixed --quiet option
This patch fixes the --quiet option for both send-message
and server commands.   Don't write anything to STDOUT.
2021-01-12 11:26:12 -05:00
Hemna 90c4c6c59d Added send-message login checking and --no-ack
This patch adds the login failure checking for the
send-message command as well as a new command line option
--no-ack.   The new option enables sending the message directly
to aprs-is servers and then exiting immediately.  It doesn't wait
for an ack to come back.
2021-01-12 11:18:17 -05:00
Hemna bdeaf6348a Added new config for aprs.fi API Key
This patch adds the new required aprs.fi api key.  This key is used
by 2 of the core plugins, locationPlugin and weatherPlugin.

You must set the apiKey in the config, or aprsd won't start.
2021-01-12 09:51:36 -05:00
Hemna 3dd23fa2ad Added a fix for failed logins to APRS-IS
This patch adds a check for a failed login to ARPS due to
LoginError.  This accounts for bad accounts or username/password
failures.  aprsd server will exit immediately upon failed login now.
2021-01-12 09:31:04 -05:00
Hemna 7ab26135c2 Fixed fortune plugin failures
On alpine containers the fortune options aren't all available
and we were silently failing.  Updated the fortune plugin to capture
shell failures.
2021-01-11 14:14:16 -05:00
Hemna ac4c3d6562 Extend APRS.IS object to change login string
This patch copies the aprslib.inet IS object's _send_login()
method so we can change the login app identification string.
2021-01-11 13:08:30 -05:00
Craig Lamparter d9141dc2d0 update query plugin to resend last N messages. syntax: ?rN 2021-01-11 09:13:37 -08:00
Hemna 1ce2a56140 Updated MsgTrack restart_delayed
This patch updates the restart_delayed method to accept the count of
messages to restart as well as the most_recent flag that sorts the
messages based on most recent first.  If you want the oldest first,
then pass in False
2021-01-11 11:03:41 -05:00
Hemna a385d171bd refactor Plugin objects to plugins directory
This patch moves all of the plugins out of plugin.py
into their own separate plugins/<plugin>.py file.  This
makes it easier to maintain each plugin.

NOTE:  You will have to update your ~/.config/aprsd/aprsd.yml

to change the python location path for each plugin enabled.

For example:
OLD:
  enabled_plugins:
   - aprsd.plugin.EmailPlugin

TO NEW
  enabled_plugins:
   - aprsd.plugins.email.EmailPlugin
2021-01-11 08:28:08 -05:00
Craig Lamparter 0aa7fe7a14 change query character syntax, don't reply that we're resending stuff 2021-01-10 14:44:40 -08:00
Hemna e6dee3a5b0 Disable MX record validation
This patch disables the MX record checking for
email address shortcuts.  verizon is a shit
smtp host that won't let you check emails as
existing/valid.  Email validation still is checked
against RFC based regex for email address as
well as blacklist checking.

TODO(hemna): make this optionally enabled
by config file.
2021-01-10 13:22:23 -05:00
Craig Lamparter e7f2ebf17e switch command characters for query plugin 2021-01-09 15:50:04 -08:00
Hemna 1763e94f93 Fix broken test 2021-01-09 18:44:36 -05:00
Craig Lamparter 45dd3f0205 undo git disaster 2021-01-09 15:39:37 -08:00
Craig Lamparter a33462327a swap Query command characters a bit 2021-01-09 14:02:16 -08:00
Hemna d5a34b4d11 refactor Plugin objects to plugins directory
This patch moves all of the plugins out of plugin.py
into their own separate plugins/<plugin>.py file.  This
makes it easier to maintain each plugin.

NOTE:  You will have to update your ~/.config/aprsd/aprsd.yml

to change the python location path for each plugin enabled.

For example:
OLD:
  enabled_plugins:
   - aprsd.plugin.EmailPlugin

TO NEW
  enabled_plugins:
   - aprsd.plugins.email.EmailPlugin
2021-01-09 10:00:37 -05:00
Hemna 231c15b1af Lots of fixes 2021-01-08 20:58:18 -05:00
Hemna 4c0150dd97 Added more pre-commit hook tests
also added pre-commit job for tox.
2021-01-08 20:57:58 -05:00
Hemna 9f4cc27a11 Fixed email shortcut lookup
This patch fixes the email.get_email_from_shortcut.  It ensures that
if the lookup isn't found in the shortcut list, it simply returns
the original value.  This patch also adds a unit test to specifically
test this function to always return the correct value.
2021-01-08 19:35:16 -05:00
Hemna dbc891f738 Cleaned out old ack_dict
This patch removes remnants of ack_dict from the code.
The new mechanism is the MsgTrack object queue.
2021-01-08 15:11:27 -05:00
Craig Lamparter 1697395e84 add null reply for send_email 2021-01-08 11:02:57 -08:00
Hemna e83ee31310 Fixed failures caused by last commit
This patch fixes test failures caused by last commit.
2021-01-08 10:57:34 -05:00
Craig Lamparter 99c5867162 don't tell radio emails were sent, ack is enuf 2021-01-07 16:22:45 -08:00
Hemna 75f610d971 Added pre-commit hooks
This patch adds pre-commit hook support to ensure
code passes basic checks prior to allowing a commit.
2021-01-06 17:54:50 -05:00
Hemna af0d4491c3 Added QueryPlugin resend all delayed msgs or Flush
This patch also updates the QueryPlugin to allow the configured user
to immediately resend all Delayed messages!

This patch updates the QueryPlugin to allow the configured user
to immediately Flush/delete all messages!
2020-12-30 09:12:38 -05:00
Hemna 28f3daf6d0 Added QueryPlugin
Query Plugin looks for ^/?.* and currently only responds with
the list of pending messages in the MsgTrack queue.
2020-12-30 08:15:49 -05:00
Hemna 2659a0b3b9 Added support to save/load MsgTrack on exit/start
This patch added saving of the MsgTrack list of messages at aprsd exit.
The will be loaded at startup unless you pass in the --flush option.
2020-12-30 07:41:34 -05:00
Hemna 2e90c0bdbb Creation of MsgTrack object and other stuff
This patch adds the new MsgTrack object replacing the
global ack_dict.  the ack_dict was not thread safe.
the new MsgTrack is a singleton object that keeps track of
all outbound TextMessage objects.  When a TextMessage.send() is called
it is added to the MsgTrack object, and when an ack is received for that
message, the message is removed from the MsgTrack object.

TODO:  Add an automatic mechanism for saving the messages in MsgTrack
so that when CTRL-C is called to exit aprsd server, then the MsgTrack
state is saved to storage.   When aprsd server is started up again, add
the option to try and reload state of MsgTrack.

This patch also reworked the email thread into an APRSDThread object
that can exit gracefully with CTRL-C.

NOTE: Don't call sleep() with a long time (greater than 5 seconds), as
it causes a delay in exiting aprsd until the last sleep() finishes.
Since aprsd has so many threads now for processing incoming messages and
outgoing messages, we need to coordinate all thread operations so that
they don't block the exiting of the app.
2020-12-29 10:40:38 -05:00
Hemna f65707cb8c reworked threading
This patch reworks the threading code for processing
messages.   This patch also extends the aprslib IS class
to allow us to stop processing the consumer packets when
someone hits CTRL-C correctly.  Alloing the app to exit.
2020-12-24 13:40:59 -05:00
Hemna 9768003c2a Reworked messaging lib
This patch updates the messaging lib to use Message Objects
for each message type (text, ack) that know how to send
themselves with the same interface.
2020-12-23 13:12:04 -05:00
Hemna 1d898ea20f Refactored the main process_packet method
This patch refactored the process_packet method
and adjusted the logic for determining if we got
a message to filter on.  We now look at the format
to make a determination.  Also isolated the processing
of message packets, ack packets and mic-e packets into
their own functions.
2020-12-21 11:59:58 -05:00
Hemna b2609548f2 Added fix for an unknown packet type
If we get a packet that doesn't have a message in it, nor an ack,
we don't know what to do with it, so log it and then stop processing it.
2020-12-20 20:28:56 -05:00
Hemna 18b269a39d Ensure fortune is installed
This patch uses python3's shutil to find the path
to fortune app, which can be very different depending on
the host OS that aprsd is running on.
2020-12-20 19:04:33 -05:00
Hemna d3ee1b78bc Fixed issue when RX ack
This patch ensures that after we get an ACK, then proessing of the ack
message is complete and we don't try and send the ack through plugin
filtering.

Created send_ack_direct for the send-message command.

Also added logic to the send-message command to ensure we wait for an
ack from the command sent to APRSD and we also wait for a response
message and send an ack to that response before we exit.
2020-12-20 16:35:23 -05:00
Hemna 3261710bf8 Fixed send-message with email command and others
This patch fixes a minor issue with the new send-message command
You now should use nargs to send the email command because it includes
a - as the start.  click assumed that any -<foo>  looks ike an argument.
So call aprsd with

aprsd send-command <callsign> -- -wb sendmap

This patch also adds -h as a help option for aprsd to make it simpler to
type.

This patch adds the VersionPlugin so you can remotely request the
version of aprsd that's running.
2020-12-20 12:21:30 -05:00
Hemna fa51f8fdf2 Big patch
This commit adds the new send-message command for sending messages.
This also redoes the logging of sent/rx'd packets to a single method
which is syncrhonized, so we don't get intermixed log messages for
packets.

Also adds email address validation during startup, and
optionally disables the validation via a command line switch.  without
email validation for production running aprsd, emails sent can turn up
garbage and cause issues when those emails are received by aprsd
message processing as invalid content.
2020-12-19 16:35:53 -05:00
Hemna 08c73a17d1 Major refactor
This branch refactors the majority of main.py out into individual
modules to compartmentalize the code.  Migrated all email related
features unti email.py, sending of messages into messaging.py

Also refactored all of the socket code to use aprslib for all APRS-IS
communication as well as message/packet processing.

Moved the email command into it's own Plugin.
2020-12-18 16:21:35 -05:00
Hemna 96bae795a5 Fix unknown characterset emails
Some emails that had an unknown character set.  when that happens
we can't decode the body of the message properly, so the default body
string was being used, and was attempting to be decoded.  Only byte
strings can be decoded, so the default string is now labeled as a byte
encoding.
2020-12-15 17:59:17 -05:00
Hemna 8a6273867c Updated loggin timestamp to include []
Added timestamp inside of [] for easier parsing
2020-12-15 11:10:46 -05:00
Hemna 63079a817d Don't use the dirname for the plugin path search 2020-12-15 09:30:10 -05:00
Hemna 1385f5ee27 Reworked Plugin loading
This patch reworked the loading of plugins.  Python2 is dead.
Previously was using the imp module to easily load the plugins from
a separate directory, which is a bit of a pita.  Found a python3 lib
that takes care of the difference between py3.3, 3.4 and 3.5+ module
loading called "thesmuggler"
2020-12-15 07:50:10 -05:00
Hemna eba093b62a Fixed an issue with weather plugin
Weather plugin had an invalid call to rstrip()
2020-12-14 14:18:12 -05:00
Hemna 2873e35f14 Fixed the usage string after plugins introduced
This patch fixes the Usage string for a call message
that isn't matched by any plugin.

Plugin object now must impleent a 'command_name' attribute
that is the usage string for that plugin.
2020-12-13 21:51:39 -05:00
Hemna d09a66006b Created plugin.py for Command Plugins
This patch adds the new APRSD Command Plugin architecture.
All Comand plugins must implement the same object API, which includes
plugin object is subclass of APRSDPluginBase
version attribute
command_regex attribute
command method

When an APRS command is detected, then the regex is run against
the command.  If the command_regex matches, then the plugin's
command() method will be called.   If the command() method returns
a string, then that string is sent as a reply to the APRS caller.

A new aprs.yml config section is added to support selecting
which plugins to enable.

If you want all plugins enabled, then omit "enabled_plugins" entirely
from the aprs section of the config.

To load custom plugins:
1) create a directory with an __init__.py file
2) Add a plugin.py file that contains your plugin

Look at the exmaples directory for an example plugin.
2020-12-13 20:57:30 -05:00
Hemna 00d99bc2c4 Refactor networking and commands
This patch refactors the socket management
to use select, as well as refactor all of the
commands into a COMMAND_ENVELOPE dictionary.

This patch also adds the Dockerfile and
docker-compose.yml files
2020-12-11 19:29:18 -05:00
Craig Lamparter 43509ea9e6 get rid of some debug statements 2020-12-11 10:23:14 -08:00
Craig Lamparter bb1a2ee61a yet another unicode problem, in resend_email fixed 2020-12-11 09:49:26 -08:00
Craig Lamparter d3bb44feec reset default email check delay to 60, fix a few comments 2020-12-11 06:53:38 -08:00
Hemna 53b8f21535 Update tox environment to fix formatting python errors
This patch includes lots of changes to tox environment for
automatically detecting pep8 failures, which can cause python2 vs
python3 failures after install.

The following tox commands have been added
tox -efmt-check  - This checks the python syntax and formatting
tox -efmt        - Automatically fixes python syntax formatting that
                   fmt-check complains about.
tox -etype-check - check on types
tox -elint       - flake8 run

This patch also changes where the default config file is located.
The new location is ~/.config/aprsd/aprsd.yml

You can now also specify a custom config file on the command line
with the -c or --config option as well.
2020-12-11 08:47:56 -05:00
Craig Lamparter 2bebd83449 fixed fortune. yet another unicode issue, tested in py3 and py2 2020-12-10 15:54:38 -08:00
Craig Lamparter c5517a7cb5 lose some logging statements 2020-12-10 15:29:29 -08:00
Craig Lamparter 86bf42802c completely off urllib now, tested locate/weather in py2 and py3 2020-12-10 15:15:19 -08:00
Craig Lamparter b38f561b84 add urllib import back until i replace all calls with requests 2020-12-10 12:19:40 -08:00
Craig Lamparter ad0f96d1aa cleaned up weather code after switch to requests ... from urllib. works on py2 and py3 2020-12-10 10:21:31 -08:00
Craig Lamparter 00432cf5bb switch from urlib to requests for weather, tested in py3 and py2. still need to update locate, and all other http calls 2020-12-10 10:11:35 -08:00
Craig Lamparter 4b09fad876 imap tags are unicode in py3. .decode tags 2020-12-10 09:10:59 -08:00
Hemna 50fb090557 Initial conversion to click 2020-12-09 09:47:37 -05:00
craigerl b553987350 Reconnect on socket timeout 2020-12-09 09:40:23 -05:00
Craig Lamparter 735cb3a557 clean up code around closed_socket and reconnect 2020-12-09 09:34:34 -05:00
Hemna 51832ea75e Fixed all pep8 errors and some py3 errors
This introduced the six lib which can translate common
py2 vs py3 incompatibilities.
https://six.readthedocs.io/
2020-12-09 09:31:36 -05:00
craigerl b552f9611f fix check_email_thread to do proper threading, take delay as arg 2020-12-09 09:27:44 -05:00
Craig Lamparter f69f9a2027 found another .decode that didn't include errors='ignore' 2020-12-08 17:07:23 -08:00
Craig Lamparter 90fcb9973b some failed attempts at getting the first txt or html from a multipart message, currently sends the last 2020-12-08 15:31:56 -08:00
Craig Lamparter 3c16149e71 fix parse_email unicode probs by using body.decode(errors='ignore').. again 2020-12-08 15:01:07 -08:00
Craig Lamparter 44228c63b3 fix parse_email unicode probs by using body.decode(errors='ignore') 2020-12-08 14:59:28 -08:00
Craig Lamparter 371a7a08ac clean up code around closed_socket and reconnect 2020-12-08 09:27:24 -08:00
Craig Lamparter e6995c3605 socket timeout 5 minutes 2020-12-07 10:08:21 -08:00
Craig Lamparter 4b67df2907 Detect closed socket, reconnect, with a bit more grace 2020-12-06 12:01:32 -08:00
Craig Lamparter b8f37fac6e can detect closed socket and reconnect now 2020-12-06 11:54:11 -08:00
craigerl 3d7e3a4e54 more debugging messages trying to find rare tight loop in main 2020-12-06 08:59:33 -08:00
craigerl a4faa7614c main loop went into tight loop, more debug prints 2020-12-05 19:23:17 -08:00
craigerl b597228719 main loop went into tight loop, added debug print before every continue 2020-12-05 18:59:58 -08:00
craigerl 3ae7a056b6 George Carlin profanity filter 2020-12-05 12:50:08 -08:00
craigerl cf73add2db added decaying email check timer which resets with activity 2020-12-05 11:30:31 -08:00
Walter A. Boring IV a3813eaf3a
Merge branch 'master' into py3 2020-12-04 16:48:28 -05:00
Hemna 4084ddfe31 Fixed all pep8 errors and some py3 errors
This introduced the six lib which can translate common
py2 vs py3 incompatibilities.
https://six.readthedocs.io/
2020-12-04 16:46:34 -05:00
Hemna 06a63f541e Fixed all pep8 errors and some py3 errors
This introduced the six lib which can translate common
py2 vs py3 incompatibilities.
https://six.readthedocs.io/
2020-12-04 16:43:30 -05:00
craigerl ecd0e8f95e Reconnect on socket timeout 2020-12-04 16:43:12 -05:00
craigerl 5e96706f7a socket reconnect on timeout testing 2020-12-04 16:42:45 -05:00
craigerl 32e146d105 socket timeout of 300 instead of 60 2020-12-04 13:37:29 -08:00
craigerl 766acbb777 Reconnect on socket timeout 2020-12-04 09:43:11 -08:00
craigerl 712d66c389 socket reconnect on timeout testing 2020-12-04 07:21:37 -08:00
Hemna 0c40689743 Fixed all pep8 errors and some py3 errors
This introduced the six lib which can translate common
py2 vs py3 incompatibilities.
https://six.readthedocs.io/
2020-12-04 09:07:32 -05:00
craigerl e171e16854 fix check_email_thread to do proper threading, take delay as arg 2020-12-03 14:48:27 -08:00
craigerl abc63791f1 fix bugs after beautification and yaml config additions. Convert to sockets. case insensitive commands 2020-12-02 12:25:39 -08:00
craigerl 78add91f3c fix INBOX 2020-12-02 09:42:23 -08:00
Walter A. Boring IV d1a50c6559 Added tox support
This patch adds support for tox.  Tox is used to run various
python compliance tests.  This enables pep8 tests, as well as python2
and python3 compatibility as well as coverage and documentation
building.
2019-05-15 20:12:59 +00:00
Walter A. Boring IV da153b64f6 Fixed SMTP settings
This patch reads the SMTP settings from the config.yml now.
Also added a logfile entry to the aprs: section of the config.yml
so the logfile can be placed anywhere.
2018-11-29 14:19:52 -05:00
Walter A. Boring IV bd47a2bdea Created fake_aprs.py
Moved the config parsing to utils, so it's shared with
main.py (aprs) and fake_aprs.py
2018-11-29 13:20:53 -05:00
Walter A. Boring IV 7abc47098b select inbox if gmail server 2018-11-29 10:01:30 -05:00
Walter A. Boring IV a9eb1714c2 removed ASS 2018-11-29 09:33:24 -05:00
Walter A. Boring IV 89d038e93d Added a try block around imap login 2018-11-29 09:30:51 -05:00
Walter A. Boring IV 81d01c3bc6 Added port and fixed telnet user
This patch adds the port to the aprs: section of the config.yml
as well as fixes a possible issue with the user telnet auth command
where the user is a string and the port is an int.  python can't
concatonate a string with an int.
2018-11-29 09:20:15 -05:00
Walter A. Boring IV ce7a30aa78 Require ~/.aprsd/config.yml
This patch completes the migration to using a config.yml file.
~/.aprsd/config.yml is now required and all options for callsign,
imap, aprs user, passwords are in the config.  If there is no existing
~/.aprsd/config.yml file, then the app will output a sample config
and exit.

This patch also adds a global logging facility that allows logging all
commands to aprsd.log as well as stdout.  You can disable logging to
stdout by adding --quiet on the command line.  You can specify the log
level with --loglevel INFO.  By default the log level is DEBUG.

This patch also updates some formatting issues and small refactoring
to ensure that the logging facility and config is read prior to starting
any network connections and/or services.
2018-11-29 08:22:41 -05:00
Walter A. Boring IV 8d1d2933e8 added test to ensure shortcuts in config.yml
This patch adds a simple test to ensure that 'shortcuts' is part of
the config.yml
2018-11-21 15:25:31 -08:00
Walter A. Boring IV e3964c5426 added exit if missing config file 2018-11-21 15:23:06 -08:00
Walter A. Boring IV 5717504f11 Added reading of a config file
This patch adds support to read a ~/.aprsd/config.yml file.
If one doesn't exist, it puts out an example yaml string to stdout
that can be copied into a file and edited.

Since this patch adds a new external requirement (pyyaml) you need
to re-install the app for dev with
pip install -e .
2018-11-21 15:20:11 -08:00
Walter A. Boring IV ecd797d91e First stab at migrating this to a pytpi repo
This patch does some refactoring of the code and the directory
structure to conform to the needs of a pypi project.
The python code now lives in the aprsd directory so it acts like a real
python package that can be installed/included/used.
The aprsd.py is now aprds/main.py

This patch also adds support for using pbr, which enables a consistent
bin install that you can then call as 'aprsd' from the command line.

To use this as a developer you should create a virtualenv
virtualenv .venv
source .venv/bin/activate
pip install -e .

now you can edit the aprds/main.py and then test it by immediately
running aprsd from the command line.

The -e option for pip allows you to install the package as an editable
package in the .venv, so you can hack on it and not need to re-install
every time you make a change.
2018-11-21 13:56:29 -08:00