Commit Graph

335 Commits

Author SHA1 Message Date
Hemna 1ccb2f7695 Moved log.py to logging
Also renamed logging/logging.py to logging/rich.py
2022-11-23 13:00:36 -05:00
Hemna a62843920a Moved trace.py to utils
This patch moves trace.py to the utils directory
2022-11-23 13:00:36 -05:00
Hemna 29b84b453b Fixed pep8 errors 2022-11-23 13:00:36 -05:00
Hemna 347a6d69f7 Refactored threads.py
This patch creates a threads directory and separates out
the contents of threads.py into separate files in the
threads directory to make it easier to find and maintain.
2022-11-23 13:00:36 -05:00
Hemna bed060f1c5 Refactor utils to directory
This patch moves the utils.py to utils/__init__.py
and fuzzyclock.py to utils
and separates the ring_buffer to it's own file in utils
2022-11-23 13:00:36 -05:00
Walter A. Boring IV fdc8bfafc0
Merge pull request #96 from ranguli/fix-pypi-scraping
Fix #92 (PyPI scraping)
2022-11-01 14:01:37 -04:00
ranguli 9ffd320353 Fix PyPI scraping 2022-10-27 12:33:31 -02:30
ranguli cc2918377e Fix typo on exception 2022-10-26 16:46:50 -02:30
Hemna 1c052a63c0 Fixed up config option checking for KISS
This patch updates the config option checking for
required fields in the config yaml file.  Specifically
for the existence of the aprsd: section
and the required fields for the 3 supported client types
apris,
kiss serial,
kiss tcp
2022-02-21 16:04:33 -05:00
Hemna e739441268 Fix logging issue with log messages
This patch changes the base Message class to
ensure that all printing of the message class only
outputs the message in the truncated and bad word filtering
enabled in the log.
2022-02-11 10:03:02 -05:00
Hemna 6257c9ea90 FIX: logging exceptions
This patch fixes the logging of exceptions in the email
plugin.
2022-01-26 14:39:14 -05:00
Hemna e8100d8777 Removed debug code 2022-01-07 15:17:16 -05:00
Hemna 764730c123 Updated list-plugins
This patch updates the README.rst with the new format for
`aprsd list-plugins`.
2021-12-15 10:48:16 -05:00
Hemna 2f6e7e17e8 Added unit tests for dev test-plugin
Also added a check to make sure that the aprs_login
parameter is passed in for use as the fromcallsign.
2021-12-12 16:35:26 -05:00
Hemna a7bbde4a43 Send Message command defaults to config
The APRS_LOGIN and APRS_PASSWORD arguments now fallback
to the config file if it exists.

First it checks the passed in parameters, then checks the
environement vars, then checks the parsed config to find the
login and password.

This patch also adds unit tests for the send-message command to
check the fallback.
2021-12-12 16:13:08 -05:00
Hemna 3b9970c0e7 Fixed an KISS config disabled issue
This patch fixes a small bug when both KISS interfaces are disabled.
2021-12-11 07:46:43 -05:00
Hemna e57a2e2ffc Fixed a bug with multiple notify plugins enabled
This patch fixes an issue with the processing of packets
and updateing the watchlist.  Previously after the
notify plugin processed the packet it would update the watchlist.
This doesn't work when there are more than 1 notify plugins
enabled, only the first notify plugin seeing the packet will
recognize that the callsign is old.
2021-12-10 14:20:57 -05:00
Hemna 592b328956 Unify the logging to file and stdout
This patch updates the logging facility to ensure that
logging to a file works even when --quiet mode is selected.
Also update the listen and list-plugins command to show
a console.status line while waiting for results to come in.
2021-12-10 10:49:09 -05:00
Hemna cd62db95c1 Added new feature to list-plugins command
This patch updates the ouput of the list-plugins command.
This also adds the ability to show the available plugins
to install that are published packages on pypi.org.

This also shows the list of installed packages from pypi.org
2021-12-08 17:16:17 -05:00
Hemna 440c8d54ad Tightened up the packet logging 2021-12-07 15:00:38 -05:00
Hemna 8ea00e9888 Added unit tests for USWeatherPlugin, USMetarPlugin 2021-12-07 13:31:58 -05:00
Hemna 0ad791bdd9 Added NotifyPlugin unit tests and more
This patch restructures the unit tests for plugins.
This also adds unit tests for the NotifyPlugin
2021-12-07 11:25:14 -05:00
Hemna 96cc07d15f Small cleanup on packet logging
This patch reduces some of the leading whitespace
to the message/packet logging to the log file.
2021-12-06 14:35:49 -05:00
Hemna d3dd08714b Reduced the APRSIS connection reset to 2 minutes
The time in which the KeepAlive Thread would reset the APRS-IS
socket connection used to be 5 minutes.   This patch changes
that to 2 minutes.
2021-12-06 14:34:22 -05:00
Hemna 055835cb3c Fixed the NotifyPlugin
The watchlist notify plugin is supposed to send an APRS message
to the configured callsign.  This patch makes sure that the
message is sent to the notify_callsign
2021-12-06 14:11:34 -05:00
Hemna b5b286e75c Fixed some pep8 errors 2021-12-03 09:10:33 -05:00
Hemna 1233137caf Add tracing for dev command
This patch enables tracing output in the log for the dev
test-plugin command
2021-12-03 08:53:08 -05:00
Hemna 1d5f76defc Added python rich library based logging.
The python rich library is extensive and has a really nice
log format that is easier to read and has built in formatting
and coloring of the log output.

To enable rich logging add rich_logging: True in the config file.
2021-12-03 08:05:03 -05:00
Hemna 5e50792e80 fixed the failure during loading for objectstore
This patch fixes a silent failure of loading data from the objectstore
2021-11-13 15:07:28 -05:00
Hemna 6fb610582d Fixed dev command missing initialization
This patch fixes a few issues when running test-plugin command.
It was missing some initialization of the stats and packets classes.
2021-11-13 09:56:19 -05:00
Hemna bda2ef00dd Fix admin logging tab 2021-11-12 12:17:45 -05:00
Hemna 446484e631 Added new list-plugins command
This patch adds the new list-plugins command that shows the
list of built in plugins for APRSD.
2021-11-12 11:36:22 -05:00
Hemna a8a6b1aa07 Don't require check-version command to have a config
This patch removes the need for check-version to have a
config file.
2021-11-12 10:23:27 -05:00
Hemna 8842fb1b44 Healthcheck command doesn't need the aprsd.yml config
This patch updates the healthcheck command to not require
the aprsd.yml config file to exist.   The healthcheck
calls a running aprsd, collects the stats to determine if it's
healthy.
2021-11-10 11:52:51 -05:00
Hemna 152132b0ed Fix test failures 2021-11-10 11:51:21 -05:00
Hemna 7787dc1be4 Removed requirement for aprs.fi key
This removed the requirement of running APRSD for specifying
the aprs.fi key in the config file.  The plugins that need the
key have been updated to set enabled = False when the key is missing.
2021-11-10 11:01:10 -05:00
Hemna 9469410929 Removed stock plugin. 2021-11-09 15:02:54 -05:00
Hemna 88db485eb4 Removed the stock plugin
This patch removed the built in stock plugin from APRSD.
This helps clean up the requirement tree from the yfinance
python module that pulled in a lot of other requirements.

The stock plugin is it's own separate repo and module now.

https://github.com/hemna/aprsd-stock-plugin

https://pypi.org/project/aprsd-stock-plugin/
2021-11-09 14:53:20 -05:00
Hemna cdcb98e438 Cleaned up some verbose output & colorized output
Some commands now have some color if the shell detects it supports it.
2021-11-08 12:18:23 -05:00
Hemna 89727e2b8e Reworked all the common arguments
This patch reworks all the common arguments for the commands
and subcommands

--loglevel
--config_file
--quiet

These are all now processed in 1 place.
2021-11-08 11:52:41 -05:00
Hemna 617973f561 Fixed test-plugin 2021-11-05 16:40:07 -04:00
Hemna 9187b9781a Ensure common params are honored 2021-11-05 16:26:24 -04:00
Hemna 8287c09ce5 pep8 2021-11-05 14:38:23 -04:00
Hemna 82def598f0 Added healthcheck to the cmds
this patch moves the healthcheck to it's own command.
aprsd healthcheck
2021-11-05 14:21:36 -04:00
Hemna 3463c6eb96 Removed the need for FROMCALL in dev test-plugin
We already use the env var for APRS_LOGIN, so that is now
used for the test-plugin command.
Also cleaned up some help text
2021-11-05 14:05:24 -04:00
Hemna 2ead6a97da Pep8 failures 2021-11-05 13:42:27 -04:00
Hemna 7d0006b0a6 Refactor the cli
This patch refactors the cli to incorporate
the dev, send-message, listen commands into the main aprsd app.
This also moves the command line completion installer/show into
it's own subgroup.
2021-11-05 13:36:33 -04:00
Hemna 49f3ea8339 Fixed a problem with send-message command
This patch fixes a problem with the packets object
not being initialized correctly for the send-message command
from the command line.

Also adds the --wait-response option for send-message, which by
default is now False
2021-11-05 10:39:47 -04:00
Hemna cefb581bb8 Be more careful picking data to/from disk
This patch ensures that the pickle file is opened and closed correctly
as well as trapping for any exceptions that might occur while loading
a pickle file.
2021-11-02 08:52:59 -04:00
Hemna 95fecd2394 Ensure plugins are last to be loaded.
This patch initializes all of the MsgTrack, WatchList and SeenList
prior to the plugins loading.  Some plugins may kick off messages
being sent immediately.  So everything has to be ready to go
prior to the plugins being loaded.
2021-10-25 11:22:46 -04:00
Hemna c8c23e6185 Fixed email connecting to smtp server
Fixed an issue with not passing config in the smtp_connect
2021-10-25 11:12:29 -04:00
Hemna e009791b75 Converted MsgTrack to ObjectStoreMixin 2021-10-22 16:07:20 -04:00
Hemna 89701c8a70 Make sure SeenList update has a from in packet
This makes sure that the packet being processed by the seenlist
has a from address.
2021-10-21 08:40:40 -04:00
Hemna 66c5d85b89 Ensure PacketList is initialized 2021-10-20 15:48:35 -04:00
Hemna 8ee8b149f1 Added SIGTERM to signal_handler 2021-10-20 15:37:54 -04:00
Hemna 0d51634ec2 Enable configuring where to save the objectstore data
This patch adds a new config entry aprsd.save_location
which is the directory used to store and load the objectstore
data.
2021-10-20 14:39:12 -04:00
Hemna 135e21cd8d PEP8 cleanup 2021-10-20 14:10:54 -04:00
Hemna 4233827dea Added objectstore Mixin
This patch adds the new objectstore Mixin class that enables
classes that store their date in self.data as a serializeable dict,
to be able to be stored to disk at shutdown and loaded at startup.

The SeenList and WatchList are now saved/loaded to/from disk.
2021-10-20 14:07:22 -04:00
Hemna 9b2212245f Added -num option to aprsd-dev test-plugin
This allows the user to specify how many times in a loop
to call the plugin.  The Default is 1.
2021-10-20 11:46:55 -04:00
Hemna 9150f3b6ff Only call stop_threads if it exists 2021-10-10 14:50:04 -04:00
Hemna 278bb6e882 Added new SeenList
This patch adds the seen list feature.  It tracks the callsign of every
packet that aprsd sees.
2021-10-09 14:29:25 -04:00
Hemna 004795dbf1 Added plugin version to stats reporting
This patch adds version to the plugin stats collected.
2021-10-09 09:31:51 -04:00
Hemna 3b7924b13d Added new HelpPlugin
This patch adds the always enabled HelpPlugin.  This plugin
now will respond to the 'help' or 'h' commands that will
automatically build a help string based on the number of
enabled plugins.  It will also respond to
help <plugin> with the plugin specific help
2021-10-08 12:01:04 -04:00
Hemna 2bf85db21b Updated aprsd-dev to use config for logfile format
This patch updates the aprsd-dev command's log file format
to use what's defined as the default and/or use the config file
setting like aprsd server does.
2021-10-08 08:47:24 -04:00
Hemna 5b17228811 removed usage of config.check_config_option
check_config_option has been superceeded by the config UserDict
object's ability to see if a config option exists.
2021-10-07 10:11:48 -04:00
Hemna 725bb2fe35 Fixed send-message after config/client rework
This patch fixes the send-message from the command line
ability after the complete rework of the client classes.
2021-10-07 10:05:19 -04:00
Hemna f8d87d05bb Fixed issue with flask config
Flask was trying to serialize the UserDict object.  Use the
data (dict) inside of it instead.
2021-10-06 15:17:09 -04:00
Hemna 30671cbdbc Added some server startup info logs
This patch adds some general info logs around starting the
client connection as well as loading the plugins.
2021-10-06 12:55:17 -04:00
Hemna fdc8c0cd66 Increase email delay to +10
This patch updates the increasing of the email check delay to += 10
seconds instead of +1.
2021-10-06 12:12:49 -04:00
Hemna c097c31258 Updated dev to use plugin manager
Also ensure that main creates the client prior to starting the
plugins.
2021-10-06 12:09:52 -04:00
Hemna e3c5c7b408 Fixed notify plugins
The notify base filter() was missing the @hookimpl
2021-10-06 12:08:29 -04:00
Hemna 491644ece6 Added new Config object.
The config object now has builtin dot notation getter with default

config.get("some.path.here", default="Not found")
2021-10-04 15:37:14 -04:00
Hemna a6ed7b894b Fixed email plugin's use of globals
The email plugin was still using globals for tracking
the check_email_delay as well as the config.  This
patch creates a new singleton thread safe mechanism for
check_email_delay with the EmailInfo class.
2021-10-04 11:36:13 -04:00
Hemna 270be947b5 Refactored client classes
This patch completely refactors and simplifies how the clients
are created and used.  There is no need now to have a separate
KISSRXThread.  Since all the custom work for the KISS client is
encapsulated in the kiss client itself, the same RX thread and
callback mechanism works for both the APRSIS client and KISS Client
objects.  There is also no need to determine which transport
(aprsis vs kiss) is being used at runtime by any of the messages
objects.  The same API works for both APRSIS and KISS Client objects
2021-09-17 09:32:30 -04:00
Hemna 23e3876e7b Refactor utils usage
This patch separates out the config from the utils.py
utils.py has grown into a catchall for everything and this
patch is the start of that cleanup.
2021-09-16 17:08:30 -04:00
Hemna 560e152742 Fixed issue of aprs-is missing keepalive
Started noticing that aprs-is keepalive messages just stop
getting sent.  This causes aprsd to basically disconnect from
the APRS network.  Added a check into the KeepAlive thread to
restart the aprs-is connecter if the last time we got a keepalive
from apris is > 5 minutes.
2021-09-13 13:22:06 -04:00
Hemna 69b215d4d8 Fixed packet processing issue with aprsd send-message
This patch adds the missing PacketList initialization
for the send-message command
2021-09-10 15:39:07 -04:00
Hemna 1b9a9935fc Enable plugins to return message object
This patch enables the ability for plugins to return:
* string
* list of strings
* message object
* list of strings and message ojects

Each string will be encapsulated in a message object prior being sent.
each message object will be sent directly.
Each list will be iterated over and processed according to the above 2
options.
2021-09-08 14:45:15 -04:00
Hemna 3faf41b203 Added enabled flag for every plugin object
This allows the admin interface to see which plugins are registered and
enabled.  Enabled is a flag that is set in the setup() method of the
plugin.  This gives the plugin developer a chance to disable the plugin
if something isn't right at setup time.   This allows aprsd to ignore
plugins that are registered but not emabled.
2021-09-08 14:25:12 -04:00
Hemna 7e6dffb34b Ensure plugin threads are valid
This patch makes sure that the plugin threads returned from
create_threads is somewhat valid
2021-09-08 13:44:20 -04:00
Hemna 9eff99dde7 Removed fixed size on logging queue
If the logging queue gets full, due to a maxsize being set,
then any further logs will result on lots of errors being dumped
to stderr as the queue is full.
2021-09-07 13:43:48 -04:00
Hemna d6b3df93f1 Added Logfile tab in Admin ui
This patch adds a live view of the aprsd logfile in
the admin ui.  This uses a new Log QueueHandler and the
threads.logging_queue to push log entries into a queue.
The flask websockets server will push those log entries up
to a connected client browser.
2021-09-07 13:13:36 -04:00
Hemna dfaf3aa3d1 Update dev.py
This patch ensures a valid packet is passed into the plugin prior to
testing.
2021-09-03 16:48:00 -04:00
Hemna 62ce84b315 Allow passing in aprsis_client
When the admin user users the web ui to send a message
a new client instance is created with login credentials for
that particular message.  This patch ensures that send_direct
uses that client.
2021-09-02 11:17:15 -04:00
Hemna 8ada789d4d Fixed a problem with the AVWX plugin not working
the regex for the plugin was not matching correctly
2021-09-02 11:06:25 -04:00
Hemna 558710d348 Remove some noisy trace in email plugin
This removes the trace decorators from the email login
functions.  They have been stable for a while now.
2021-09-02 10:03:43 -04:00
Hemna 1ea6c05dec Fixed issue at startup with notify plugin
Ensure that the aprsis client is configured prior to starting
any plugins.
2021-09-02 09:54:13 -04:00
Hemna 0f6df5fc05 Fixed email validation
This patch adjusts the py3-email-validation usage.  Since we
upgraded to 1.0.2, the signature has changed.  This patch adjusts
the signature usage so it works again.
2021-09-02 09:43:33 -04:00
Hemna 1635feb820 Removed values from forms 2021-09-02 09:12:44 -04:00
Hemna c58031d772 Added send-message to the main admin UI 2021-09-02 08:56:25 -04:00
Hemna c537b54df6 Updated requirements 2021-09-01 17:38:59 -04:00
Hemna 84ce60bc50 Cleaned up some pep8 failures 2021-09-01 17:11:35 -04:00
Hemna c941379a5c Upgraded the send-message POC to use websockets
This patch updates the send message Admi page to use
websockets.  It makes updates to the messages list instant.
2021-09-01 17:10:59 -04:00
Hemna 23cbf32814 New Admin ui send message page working. 2021-09-01 17:10:13 -04:00
Hemna 6d3258e833 Send Message via admin Web interface
This patch adds the ability to send a message from the
admin interface's send-message.html page.
2021-09-01 17:06:56 -04:00
Hemna ca438c9c60 Updated Admin UI to show KISS connections
This updates the top area of the Admin UI to reflect the
connection type (aprs-is vs kiss).
2021-09-01 16:39:50 -04:00
Hemna f4dee4b202 Got TX/RX working with aioax25+direwolf over TCP
This patch gets APRSD fully working with the TCPKISS socket
to direwolf.
2021-09-01 14:48:22 -04:00
Hemna 54c9a6b55a Rebased from master 2021-08-30 13:34:25 -04:00
Hemna b53e2ba7fe Added the ability to use direwolf KISS socket
This patch adds APRS KISS connectivity.  I have tested this with
a running Direwolf install via either a serial KISS connection or
the optional new TCPKISS connection, both to Direwolf.

This adds the new required aioax25 python library for the underlying
KISS and AX25 support.

NOTE: For the TCPKISS connection, this patch requires a pull request
patch the aioax25 library to include a TCP Based KISS TNC client to
enable the TCPKISS client  So you will need to pull down this PR
https://github.com/sjlongland/aioax25/pull/7

To enable this,
  Edit your aprsd.yml file and enable one of the 2 KISS connections.
  Only one is supported at a time.

  kiss:
     serial:
         enabled: True
         device: /dev/ttyS1
         baudrate: 9600

  or

  kiss:
      tcp:
          enabled: True
          host: "ip address/hostname of direwolf"
          port: "direwolf configured kiss port"

This patch alters the Message object classes to be able to
send messages out via the aprslib socket connection to the APRS-IS
network on the internet, or via the direwolf KISS TCP socket,
depending on the origination of the initial message coming in.

If an APRS message comes in via APRS-IS, then replies will go out
APRS-IS.  IF an APRS message comes in via direwolf, then replies
will go out via direwolf KISS TCP socket.   Both can work at the same
time.

TODO:  I need some real APRS message packets to verify that
the new thread is processing packets correctly through the plugins
and able to send the resulting messages back out to direwolf.

Have a hard coded callsign for now in the kissclient consumer call,
just so I can see messages coming in from direwolf.  I dont' have an
APRS capable radio at the moment to send messages directly to direwolf.
Might need to write a simple python socket server to send fake APRS
messages to aprsd kiss, just for finishing up development.
2021-08-30 13:28:39 -04:00
Hemna ec92b07e31 Silence some log noise
Removed an email thread log at the start of the loop.
Also bumped the Keepalivethread time to 60 seconds
2021-08-25 08:25:36 -04:00