The existing LTC code for padding meassages for PSS signatures
contained a small error. In particular, the PSS-passing algorithms is
supposed to be given (bitlength of key - 1) as an argument. The LTC
code passes (bitlength of key), and subtracts 1 in the middle of the
PSS-padding. This subtraction unfortunately comes too late: a
calculation using that argument has already been made. Fortunately,
this bug only appeared if the bit-length of the key was 1 mod 8, and
so is unlikely to show up in practice. Still, this patch fixes the
problem.
Conflicts:
src/pk/pkcs1/pkcs_1_pss_decode.c
Because many of the hash-functions implemented by LTC use the length
of the input when padding the input out to a block-length, LTC keeps
track of the input length in a 64-bit integer. However, it did not
previously test for overflow of this value. Since many of the
hash-functions implemented by LTC are defined for inputs of length
2^128 bits or more, this means that LTC was incorrectly implementing
these hash functions for extremely long inputs. Also, this might have
been a minor security problem: A clever attacker might have been able
to take a message with a known hash and find another message (longer
by 2^64 bits) that would be hashed to the same value by LTC.
Fortunately, LTC uses a pre-processor macro to make the actual code
for hashing, and so this problem could be fixed by adding an
overflow-check to that macro.
There would have been a call to mp_clear_multi() of all the key parameters
that are not yet allocated, in the case where the calculations of p, q,
tmp1 or tmp2 created an error.
This also includes a proposed improvement from the OLPC project to free
elements in the reverse order as they were allocated.
There could have been a 'goto error', which misses the free of rnd and
rndi even if they were initialized.
This could happen in cases where a private key operation was done and
afterwards one of the operations like reading back or conversion, would
have failed (which is likely not to happen)
This also includes a proposed improvement from the OLPC project to free
elements in the reverse order as they were allocated.
