sdrangel/dsd/README.md

# Digital Speech Decoder 1.7.0-dev
Copyright (C) 2010 DSD Author
GPG Key ID: 0x3F1D7FD0 (74EF 430D F7F2 0A48 FCE6  F630 FAA2 635D 3F1D 7FD0)

    Permission to use, copy, modify, and/or distribute this software for any
    purpose with or without fee is hereby granted, provided that the above
    copyright notice and this permission notice appear in all copies.

    THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH
    REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
    AND FITNESS.  IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT,
    INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
    LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
    OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
    PERFORMANCE OF THIS SOFTWARE.

DSD is able to decode several digital voice formats from discriminator
tap audio and synthesize  the decoded speech.  Speech
synthesis requires mbelib, which is a separate package.


### Supported formats

#### P25 Phase 1

Widely deployed radio standard used in public safety and amateur radio.

Support includes decoding and synthesis of speech,
display of all link control info, and the ability to save
and replay .imb data files

#### ProVoice

EDACS Digital voice format used by public safety and amateur radio.

Support includes decoding and synthesis of speech and
the ability to save and replay .imb data files.

Note: not enabled by default, use `-fp` to enable.

#### X2-TDMA

Two slot TDMA system currently being deployed by several
public safety organizations.   Based on the DMR
standard with extensions for P25 style signaling.

Support includes decoding and synthesis of speech,
display of all link control info, and the ability to save
and replay .amb data files

#### DMR/MOTOTRBO

"Digital Mobile Radio" Eurpoean two slot TDMA standard.
MOTOTRBO is a popular implementation of this standard.

Support includes decoding and synthesis of speech and
the ability to save and replay .amb data files.

#### NXDN

Digital radio standard used by NEXEDGE and IDAS brands.
Supports both 9600 baud (12.5 kHz) and
4800 baud (6.25 kHz) digital voice.

Support includes decoding and synthesis of speech and
the ability to save and replay .amb data files.

#### D-STAR

Amateur radio digital voice standard

This is an earlier version of the AMBE codec than the one
used by most of the protocols. Support for this was added by
various developers.

### Unsupported formats in version 1.6 considered for future development:

#### P25 Phase 2

This is not yet a published standard.  Full support is
expected once the standard is published and there are
systems operating to test against.  Phase 2 will use
a vocoder supported by mbelib.

#### OpenSKY

It is possible that the four slot version uses a vocoder
supported by mbelib.  The two slot version does not.

### Supported demodulation optimizations in version 1.6:

#### C4FM

Continuous envelope 2 or 4 level FSK with relatively
sharp transitions between symbols.  Used by most P25
systems.

Optimizations include calibrating decision points only
during sync, 4/10 sample window per symbol, and symbol
edge timing calibration.

#### GFSK

Continuous envelope 2 or 4 level FSK with a narrower
Gaussian/"raised cosine" filter that affects transitions
between symbols.  Used by DMR/MOTOTRBO, NXDN and many
others.  Noisy C4FM signals may be detected as GFSK

but this is ok, the optimization changes will help with
noisy signals.

Optimizations are similar to C4FM except symbol transitions
are only kept out of the middle 4 samples and only the
middle two samples are used.

#### QPSK

Quadrature Phase Shift Keying (and variants) used in
some P25 systems and all known X2-TDMA systems.  May be
advertised under the marketing term "LSM"

Optimizations include continuous decision point
calibration, using middle two samples, and using the
symbol midpoint "spike" for symbol timing.

## Installation

DSD should easily compile on any Linux or *BSD system with gcc.
There are some debugging/development options in `config.h` that
normal users will want to leave disabled as they can severely
impact performance.

### Requirements
* cmake
* mbelib
* sndfile

### Example building instructions on Ubuntu:

````
sudo apt-get update
sudo apt-get install git make cmake # Update packages
git clone <URL of git repository>   # Something like: git@github.com:USERNAME/dsd.git
cd dsd                              # Move into source folder
mkdir build                         # Create build directory
cd build                            # Move to build directory
cmake ..                            # Create Makefile for current system
make                                # Compiles DSD
sudo make install                   # Installs DSD to the system
````

## Operation

There are two main operating modes, "Live scanner" and "Play files"

    Usage: dsd [options]            Live scanner mode

Live Scanner mode takes 48KHz/16 bit mono audio samples from a
sound card input and decodes speech in real time.  Options are provided
for controling information display and saving mbe data files.

The synthesized speech can be  output to a soundcard and/or a
.wav file.

    Usage:  dsd [options] -r <files> Read/Play saved mbe data from file(s)


Play files mode reads mbe data from files specified on the command
line (including wildcards) and synthesizes speech from those files.
The synthesized speech can be  output to a soundcard and/or a
.wav file. The `-r` command line options is used to activate Play files
mode.

### Display modes

There are two main display modes in Live scanner mode.  "Errorbars"
and "Datascope".

Errorbars mode output for P25 Phase 1 looks like this:

````
Sync:  -P25p1   mod: C4FM inlvl: 39% nac:  5C2 src:        0 tg: 32464  TDULC
Sync:  -P25p1   mod: C4FM inlvl: 39% nac:  5C2 src:        0 tg: 32464  TDULC
Sync:  -P25p1   mod: C4FM inlvl: 39% nac:  5C2 src:        0 tg: 32464  TDULC
Sync:  -P25p1   mod: C4FM inlvl: 39% nac:  5C2 src:        0 tg: 32464  TDULC
Sync:  -P25p1   mod: C4FM inlvl: 38% nac:  5C2 src:        0 tg: 32464  TDU
Sync:  -P25p1   mod: C4FM inlvl: 38% nac:  5C2 src:        0 tg: 32464  HDU
Sync:  -P25p1   mod: C4FM inlvl: 42% nac:  5C2 src:        0 tg: 32464  LDU1  e:
Sync: (-P25p1)  mod: C4FM inlvl: 39% nac:  5C2 src:    52610 tg: 32464 (LDU2) e:
Sync:  -P25p1   mod: C4FM inlvl: 38% nac:  5C2 src:    52610 tg: 32464  LDU1  e:
Sync:  -P25p1   mod: C4FM inlvl: 39% nac:  5C2 src:    52610 tg: 32464  LDU2  e:
Sync:  -P25p1   mod: C4FM inlvl: 39% nac:  5C2 src:    52610 tg: 32464  LDU1  e:
Sync:  -P25p1   mod: C4FM inlvl: 39% nac:  5C2 src:    52610 tg: 32464  LDU2  e:
Sync:  -P25p1   mod: C4FM inlvl: 39% nac:  5C2 src:    52610 tg: 32464  LDU1  e:
````

* "Sync" indicates the frame type detected and whether the polarity is
positive or negative.  DSD automatically detects and handles either
polarity except for DMR/MOTOTRBO/X2-TDMA which unfortunatley use both
sync polarities.

* Most combinations of transmitter, receiver and soundcard show netagive
(-) polarity for X2-TDMA signals and (+) polarity for DMR/MOTOTRBO so
those are the defaults.

    * You may need to use the `-x` option to select non-inverted polarity if
    you are not getting usable X2-TDMA/MOTOTRBO/DMR speech.  As they use both
    normal and inverted sync it is not possible to detect polariy
    automatically.

* "mod" indicates the current demodulation optimizations.

* "inlvl" indicates the audio input level.  QPSK signals tend to appear
much "wider" than C4FM from a discriminator tap so it is important
to set your input gain using a QPSK signal if you plan to montir them.
It is not necessary nor desirable to get to 100%, in fact your sound
card may max out below 100%.  It is best to use the Datascope mode for
setting input gain (see below).  Typical values with good results are
40% for C4FM and 66% for QPSK.

* "nac" is the P25 Phase 1 Network Access Code.  This is a 12 bit field
in each P25 Phase 1 header.  It should not be confused with the 16
bit System ID used in non-P25 trunking control channels.

* "src" is the radio id of the trasmitting subscriber unit.

* "tg" is the talkgroup derived from link control information.

* "HDU/LDU1/LDU2/TDU/TDULC" are P25 Phase 1 frame types, referred to as
frame subtype within DSD.

* "e:" is the beginning of the errorbars display.  Each "=" indicates a
detected error within the voice data.  "R" and "M" indicat that a voice
frame was repeated or muted due to excessive errors.

* Values in parentheses () indicate an assumption (soft decision) was
made based on the previous frame.

Errorbars mode output for X2-TDMA looks like this:
````
Sync:  -X2-TDMA  mod: QPSK inlvl: 59% src:    17211 tg:   197 [SLOT0]  slot1   VOICE e:
Sync:  -X2-TDMA  mod: QPSK inlvl: 47% src:    17211 tg:   197 [SLOT0]  slot1   VOICE e:
Sync:  -X2-TDMA  mod: QPSK inlvl: 43% src:    17211 tg:   197 [SLOT0]  slot1   VOICE e:
Sync: (-X2-TDMA) mod: QPSK inlvl: 28% src:    17211 tg:   197 [SLOT0]  slot1   VOICE e:
````

DMR/MOTOTRBO display is similar except it does not yet show source
and talkgroup information.

As of version 1.2 DSD shows which specific TDMA slots are active (with
capital SLOT letters) and which slot is currently being monitored (with
square brackets [].  Noisy/degraded signals will affect the accuracy
of this display.

The frame subtypes (Voice/LC etc) are shown based on the DMR standard
types.

Datascope mode output looks like this:

````
Demod mode:     C4FM                Nac:                      8C3
Frame Type:     P25 Phase 1         Talkgroup:              16528
Frame Subtype:  LDU1                Source:                     0
TDMA activity:   slot0   slot1      Voice errors:
+----------------------------------------------------------------+
|                   #    ^      !|      ^   #                    |
|                    *           |          *                    |
|                    *           |          *                    |
|                    *           |  *       *                    |
|                    *      *    |  *       *                    |
|                    *      *    |  **      *                    |
|                    *      **   |  **      *                    |
|                   **      **   |  **      *                    |
|                   **      **   |  **      *                    |
|                   **      **   |  **      *                    |
+----------------------------------------------------------------+
                          C4FM Example
````
````
Demod mode:     C4FM                Nac:                      126
Frame Type:     P25 Phase 1         Talkgroup:              25283
Frame Subtype:  LDU2                Source:                     0
TDMA activity:   slot0   slot1      Voice errors:
+----------------------------------------------------------------+
|            #      ^            !            ^       #          |
|                          *     |                               |
|                          *     |                               |
|                          **    |                               |
|                          **    |                    *          |
|            *             **    |    *               *          |
|            **            **    |    *               *          |
|            ***           **    |    **              *          |
|            ***           **    |    ***             *          |
|            ***          ****   |   ****           * *          |
+----------------------------------------------------------------+
                          QPSK Example
````

At the top is various information about the signal, similar to the
information provided in Errorbars mode.  The large box is similar to
a spectrum analyzer viewing the channel bandwidth.

The horizontal axis is the input audio level, minimum on the left and
maximum on the right.   The vertical axis is the number of samples
seend at each audio level.

The "*" symbols represent the number of audio
samples that were at each level during the aggregation period.
(default = 36 symbols) The `-S` options controls the aggregation period
as well as the QPSK tracking symbol buffer, so changing that will affect
QPSK performance as well as the Datascope display.

As you can see from the figures above, clean C4FM signals tend to have
four very sharply defined audio levels.  The datascope pattern also
tends to be faily stable with minor shifts left and right as the
receiver tries to frequency track any DC offset.

QPSK signals on the other hand tend to appear much broader (and artifact
of how they are distored by FM PLL discriminators).  They also tend
to vary wildly in width and centering.  This is especially true when
monitoring simulcast systems. Muliple QPSK signals interfere much more
dramatically with an FM discriminator than C4FM signals.

    For this reason it is important to isolate your receiver to one
    transmitter tower, _especially_ for QPSK signals.

    The "#" symbols indicate the detected min/max values that are used
    to calibrate the symbol decision points. These are indicated by
    "!" for the center decision point and "^" for the mid decision points.

### Display Options

There are several options to control the type and quantity of
information displayed in Errorbars mode:

````
-e             Show Frame Info and errorbars (default)
-pe            Show P25 encryption sync bits
-pl            Show P25 link control bits
-ps            Show P25 status bits and low speed data
-pt            Show P25 talkgroup info
-q             Don't show Frame Info/errorbars
-s             Datascope (disables other display options)
-t             Show symbol timing during sync
-v <num>       Frame information Verbosity
-z <num>       Frame rate for datascope
````

Most of these options are self explanitory.  Symbol timing is a noisy
option that allows you to view the quality of the frame sync samples
and accuracy of the symbol timing adjustments.

Symbol Timing display looks like this:
````
Symbol Timing:
----------
----------
----------
----------
----------
-+++++++++ 1
+---------- 0
----------
++++++++++ 0
++++++++++
---------- 0
----------
++++++++++ 0
++++++++++
++++++++++
++++++++++
---------- 0
++++++++++ 0
---------- 0
++++++++++ 0
++++++++++
++++++++++
++++++++++
++++++++++
C4FM example
````
````
Symbol Timing:
+---------
----------
----------
----------
-----X---- 5
--+++O++++- 4
----------
----X----- 4
++++O++--- 4
--++O++++- 4
----X----- 4
----------
++++O+++-- 4
-+++O+++-- 4
--++O+++-- 4
--++O+++-- 4
----------
++++O++++- 4
----------
++++O+++-- 4
-+++O++++- 4
-+++O+++++ 4
-+++O++--- 4
--++O+++-- 4
QPSK example
````

Symbol timing is only displayed for symbols during the frame sync
period.  Each horizontal line represents the 10 audio samples for each
symbol.  "-" indicates an audio sample below the center reference level
and "+" represents a sample above center.  "X" indicates a low spike
below a reference threshold  (reference minimum for C4FM and 80%
of reference minimum for QPSK).  "O" represents a high spike above
the high reference threshold.  The numbers to the right indicate which
sample position the targeted transition occurred (+/- for C4FM or
spike high/low for QPSK).  The number of audio samples for the next
symbol are adjusted to get this value closer to the target (0 for
C4FM and 4 for QPSK).  This shows how DSD maintains accurate symbol
timing.  Symbol timing adjustments are only made during sync, which
is the only time reliable transitions can be observed.

In both examples above the symbol timing was off by one sample at
the beginning of the frame sync period and was adjusted.  Generally
if you see any spike values "X/O" in C4FM mode, or lots of them in
QPSK mode it indicates noise on the input signal.

### Input/Output Options

````
-i <device/file> Audio input device/file (default is /dev/audio)
-o <device>      Audio output device (default is /dev/audio)
-d <dir>         Create mbe data files, use this directory
-r <files>       Read/Play saved mbe data from file(s)
-g <num>         Audio output gain (default = 0 = auto)
-n               Do not send synthesized speech to audio output device
-w <file>        Output synthesized speech to a .wav file
````

The audio in device can be a sound card OR a .wav file if the file
is in the exact format 48k/16bits/mono/pcm.  Audio in should be an
unfilterd discriminator tap signal.

The audio out device should be a sound card (use the `-w` options to
output to a .wav file).

If the audio in device is the same as the audio out device, the
synthesized speech has to be upsampled to the 48k sample rate required
for input.  A fast upsample function is provided but still leaves some
artifacts.

    The best sound and minimum cpu usage is achieved with separate sound
    cards for input and output

If you specify different input/output devices DSD will use 8k as the
output sample rate and the lack of resampling results in much better
audio as well as lowe cpu consumption.

If you are using onboard "AC97" sound device you may find that DSD uses
much more cpu than expected, in some cases more than is available.
This is because many AC97 sound devices are designed to rely on CPU
processing power instead of hardware.  You may also find that 8k sample
rate output is upsampled in the driver using a very basic algorithim
resulting in severe distortion.  The solution is to use a real hardware
sound device (pci card, usb device etc).

As of version 1.2 DSD now automatically levels the output audio. This
greately improves readability and eliminates the painful effects of
noise bursts.  You can specify a fixed audio output gain with the -g
option.

### Scanner control options:
````
-B <num>       Serial port baud rate (default=115200)
-C <device>    Serial port for scanner control (default=/dev/ttyUSB0)
-R <num>      Resume scan after <num> TDULC frames or any PDU or TSDU
````

On some P25 systems Packet Data Units (PDU) are sent on the same
frequencies used for voice traffic.  If done constantly this can
be a severe hinderance to scanning the system in conventional
mode.  The -R option enables sending a "resume scan" command to
a scanner connected to a serial port.  Use `-B` and `-C` to set the baud
rate and serial port device if necessary.

### Decoder options
````
-fa           Auto-detect frame type (default)
-f1           Decode only P25 Phase 1
-fd           Decode only D-STAR*
-fi           Decode only NXDN48* (6.25 kHz) / IDAS*
-fn           Decode only NXDN96 (12.5 kHz)
-fp           Decode only ProVoice*
-fr           Decode only DMR/MOTOTRBO
-fx           Decode only X2-TDMA
-l            Disable Filters (not recommended)
-ma           Auto-select modulation optimizations (default)
-mc           Use only C4FM modulation optimizations
-mg           Use only GFSK modulation optimizations
-mq           Use only QPSK modulation optimizations
-pu           Unmute Encrypted P25
-u <num>      Unvoiced speech quality (default=3)
-xx           Expect non-inverted X2-TDMA signal
-xr           Expect inverted DMR/MOTOTRBO signal
````
\* denotes frame types that cannot be auto-detected.

ProVoice and NXDN48 not auto-detected as use different symbol
rates (9600 and 2400) than most formats (4800).

MBE speech synthesis is broken down into two main types of sounds,
"Voiced" and "Unvoiced".  Voiced speech bands are synthesized with
a single sine wave centered in the frequency band with the appropriate
phase and amplitude.

Unvoiced speech is supposed to be generated with a noise source, 256
point DFT a number of band filters, followed by a 256 point inverse DFT.
For computational simplicity mbelib uses a different method.  For each
unvoiced speech band, a number of sine waves are generated, each with a
different random initial phase.  The number of waves used per band is
controlled by the `-u` option.  A setting of 4 would approximate the
performance of the 256 point DFT method as the maximum number of voice
bands is 56, and very low frequencies are not synthesized.  Values less
than 3 have a noticable lack of unvoiced speech and/or artifacts.  The
defualt of 3 provides good speech quality with reasonable cpu use.
Increasing the quality above the default rapidly consumes more CPU for
increasingly diminishing returns.


#### Advanced decoder options
````
-A <num>      QPSK modulation auto detection threshold (default=26)
-S <num>      Symbol buffer size for QPSK decision point tracking
            (default=36)
-M <num>      Min/Max buffer size for QPSK decision point tracking
            (default=15)
````

### Encryption

Decryption of speech is **NOT** supported, even if you lawfully posess the
encryption keys.  Decryption support will not be added in the future as
the authors wish to steer as far away from the legal issues associated
with encryption as possible.


    We realize that there are many legitemate and lawful uses of decryption
    software including system/interoperability testing and lawful monitoring.
    This software is distributed under a liberal BSD license so there is
    nothing to stop others from supplying patches, forking this project or
    incorporating it into a commercial product and adding decryption support.

    There is support for displaying the encryption sync bits transmitted in
    the clear on P25 Phase 1 systems.  These bits do not allow for the
    decryption of signals without the secret encryption keys.  The
    encryption sync bits are useful for determining whether a signal is
    encrypted vs merely noisy or degraded.  As the encryption sync bits
    typically include long strings of zeros when a transmission is not
    encrypted they can also be used to visually estimate bit error rates.