This patch adds support for the specific corner case of performing HMAC
on an empty string (i.e. payload length is zero). This solves the last
failing cryptomgr extratests for HMAC.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch fixes an issue with hash and HMAC operations that perform
"large" intermediate updates (i.e. combined size > 2 hash blocks) by
actually making use of the hardware's hash continue capabilities.
The original implementation would cache these updates in a buffer that
was 2 hash blocks in size and fail if all update calls combined would
overflow that buffer. Which caused the cryptomgr extra tests to fail.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The driver was loading the initial digest for hash operations into
the hardware explicitly, but this is not needed as the hardware can
handle that by itself, which is more efficient and avoids any context
record coherence issues.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch backs out some changes done with commit 082ec2d484 -
"add support for HMAC updates" as that update just works around the
issue for the basic tests by providing twice the amount of buffering,
but this does not solve the case of much larger data blocks such as
those performed by the extra tests.
This is in preparation of an actual solution in the next patch(es),
which does not actually require any extra buffering at all.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Replace sg_nents_for_len by sg_nents when DMA mapping/unmapping buffers
and when looping over the SG entries. This fix cases where the SG
entries aren't used fully, which would in such cases led to using fewer
SG entries than needed (and thus the engine wouldn't have access to the
full input data and the result would be wrong).
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds support for HMAC updates in the Inside Secure SafeXcel
crypto engine driver. Updates were supported for hash algorithms, but
were never enabled for HMAC ones. This fixes boot time test issues.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch fixes the queued len computation, which could theoretically
be wrong if req->len[1] - req->processed[1] > 1. Be future-proof here,
and fix it.
Fixes: b460edb623 ("crypto: inside-secure - sha512 support")
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
A request is zeroed in safexcel_ahash_exit_inv(). This request total
size is EIP197_AHASH_REQ_SIZE while the memset zeroing it uses
sizeof(struct ahash_request), which happens to be less than
EIP197_AHASH_REQ_SIZE. This patch fixes it.
Fixes: f6beaea304 ("crypto: inside-secure - authenc(hmac(sha256), cbc(aes)) support")
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch unify the way the cache related data is zeroed when the cache
buffer is DMA unmapped. It should not change the driver behaviour, but
improves the code safety and readability.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When sending an ahash request, the code checks for the extra variable
not to be 0. This check is useless as the extra variable can't be 0 at
this point (it is checked on the line just before).
This patch does not modify the driver behaviour in any way.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This cosmetic patch moves a comment before the condition it is related
to. The patch does not change the driver behaviour in any way.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use the appropriate SPDX license identifiers and drop the license text.
This patch is only cosmetic.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Some crypto API users allocating a tfm with crypto_alloc_$FOO() are also
specifying the type flags for $FOO, e.g. crypto_alloc_shash() with
CRYPTO_ALG_TYPE_SHASH. But, that's redundant since the crypto API will
override any specified type flag/mask with the correct ones.
So, remove the unneeded flags.
This patch shouldn't change any actual behavior.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch main goal is to improve driver performance by moving the
crypto request from a list to a RDR ring shadow.
This is possible since there is one producer and one consume for this
RDR request shadow and one ring descriptor is left unused.
Doing this change eliminates the use of spinlock when accessing the
descriptor ring and the need to dynamicaly allocate memory per crypto
request.
The crypto request is placed in the first RDR shadow descriptor only
if there are enough descriptors, when the result handler is invoked,
it fetches the first result descriptor from RDR shadow.
Signed-off-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds support for the hmac(md5) algorithm in the Inside Secure
SafeXcel cryptographic engine driver.
Signed-off-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the MD5 algorithm support to the Inside Secure SafeXcel
cryptographic engine driver.
Signed-off-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds support for the eip197d engine to the Inside Secure
SafeXcel cryptographic driver. This new engine is similar to the eip197b
and reuse most of its code.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add a flags field in the private structure, and a first flag for engines
needing context invalidation (currently only the eip197b). The
invalidation is needed when the engine includes a TRC cache, which will
also be true for the upcoming addition of the eip197d engine.
Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
EIP engines do not support the same set of algorithms. So far the
supported engines in the Inside Secure SafeXcel driver support the same
set of algorithms, but that won't be true for all engines. This patch
adds an 'engines' field in the algorithm definitions so that they only
are registered when using a compatible cryptographic engine.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
At first we used two compatibles in the SafeXcel driver, named after the
engine revision: eip97 and eip197. However this family of engines has
more precise versions and in fact we're supporting the eip97ies and
eip197b. More versions will be supported in the future, such as the
eip197d, and we'll need to differentiate them.
This patch fixes the compatibles used in the driver, to now use precise
ones. The two historical compatibles are kept for backward
compatibility.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the hmac(sha384) algorithm support to the Inside Secure
SafeXcel driver.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the sha384 algorithm support to the Inside Secure
SafeXcel driver.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the hmac(sha512) algorithm support to the Inside Secure
SafeXcel driver.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the sha512 algorithm support to the Inside Secure
SafeXcel driver.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
A counter is given to the engine when finishing hash computation. It
currently uses the blocksize while it counts the number of 64 bytes
blocks given to the engine. This works well for all algorithms so far,
as SHA1, SHA224 and SHA256 all have a blocksize of 64 bytes, but others
algorithms such as SHA512 wouldn't work.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch reworks the way invalidation request handlers handle the
result descriptor errors, to use the common error handling function.
This improves the drivers in terms of readability and maintainability.
Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds support for the first AEAD algorithm in the Inside
Secure SafeXcel driver, authenc(hmac(sha256),cbc(aes)). As this is the
first AEAD algorithm added to this driver, common AEAD functions are
added as well.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch improves the error reporting from the Inside Secure driver to
the upper layers and crypto consumers. All errors reported by the engine
aren't fatal, and some may be genuine.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch removes the use of VLAs to allocate requests on the stack, by
removing both SKCIPHER_REQUEST_ON_STACK and AHASH_REQUEST_ON_STACK. As
we still need to allocate requests on the stack to ease the creation of
invalidation requests a new, non-VLA, definition is used:
EIP197_REQUEST_ON_STACK.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the hmac(sha224) support to the Inside Secure
cryptographic engine driver.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds the hmac(sha256) support to the Inside Secure
cryptographic engine driver.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patches uses the state size of the algorithms instead of their
digest size to copy the ipad and opad in the context. This doesn't fix
anything as the state and digest size are the same for many algorithms,
and for all the hmac currently supported by this driver. However
hmac(sha224) use the sha224 hash function which has a different digest
and state size. This commit prepares the addition of such algorithms.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patches update the way the digest is copied from the state buffer
to the result buffer, so that the copy only happen after the state
buffer was DMA unmapped, as otherwise the buffer would be owned by the
device.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch improves the send error path as it wasn't handling all error
cases. A new label is added, and some of the goto are updated to point
to the right labels, so that the code is more robust to errors.
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patches moves the digest information from the transformation
context to the request context. This fixes cases where HMAC init
functions were called and override the digest value for a short period
of time, as the HMAC init functions call the SHA init one which reset
the value. This lead to a small percentage of HMAC being incorrectly
computed under heavy load.
Fixes: 1b44c5a60c ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
[Ofer here did all the work, from seeing the issue to understanding the
root cause. I only made the patch.]
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add the missing unlock before return from function
safexcel_ahash_send_req() in the error handling case.
Fixes: cff9a17545 ("crypto: inside-secure - move cache result dma mapping to request")
Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com>
Acked-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch updates the safexcel_hmac_init_pad() function to also wait
for completion when the digest return code is -EBUSY, as it would mean
the request is in the backlog to be processed later.
Fixes: 1b44c5a60c ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In heavy traffic the DMA mapping is overwritten by multiple requests as
the DMA address is stored in a global context. This patch moves this
information to the per-hash request context so that it can't be
overwritten.
Fixes: 1b44c5a60c ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In heavy traffic the DMA mapping is overwritten by multiple requests as
the DMA address is stored in a global context. This patch moves this
information to the per-hash request context so that it can't be
overwritten.
Fixes: 1b44c5a60c ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Signed-off-by: Ofer Heifetz <oferh@marvell.com>
[Antoine: rebased the patch, small fixes, commit message.]
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds a label to unmap the result buffer in the hash send
function error path.
Fixes: 1b44c5a60c ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When exiting a transformation, the cra_exit() helper is called in each
driver providing one. The Inside Secure SafeXcel driver has one, which
is responsible of freeing some areas and of sending one invalidation
request to the crypto engine, to invalidate the context that was used
during the transformation.
We could see in some setups (when lots of transformations were being
used with a short lifetime, and hence lots of cra_exit() calls) NULL
pointer dereferences and other weird issues. All these issues were
coming from accessing the tfm context.
The issue is the invalidation request completion is checked using a
wait_for_completion_interruptible() call in both the cipher and hash
cra_exit() helpers. In some cases this was interrupted while the
invalidation request wasn't processed yet. And then cra_exit() returned,
and its caller was freeing the tfm instance. Only then the request was
being handled by the SafeXcel driver, which lead to the said issues.
This patch fixes this by using wait_for_completion() calls in these
specific cases.
Fixes: 1b44c5a60c ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch fixes the cache length computation as cache_len could end up
being a negative value. The check between the queued size and the
block size is updated to reflect the caching mechanism which can cache
up to a full block size (included!).
Fixes: 809778e02c ("crypto: inside-secure - fix hash when length is a multiple of a block")
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch fixes the extra cache computation when the queued data is a
multiple of a block size. This fixes the hash support in some cases.
Fixes: 809778e02c ("crypto: inside-secure - fix hash when length is a multiple of a block")
Signed-off-by: Antoine Tenart <antoine.tenart@bootlin.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch fixes the hash support in the SafeXcel driver when the update
size is a multiple of a block size, and when a final call is made just
after with a size of 0. In such cases the driver should cache the last
block from the update to avoid handling 0 length data on the final call
(that's a hardware limitation).
Cc: stable@vger.kernel.org
Fixes: 1b44c5a60c ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch adds a parameter in the SafeXcel ahash request structure to
keep track of the number of SG entries mapped. This allows not to call
dma_unmap_sg() when dma_map_sg() wasn't called in the first place. This
also removes a warning when the debugging of the DMA-API is enabled in
the kernel configuration: "DMA-API: device driver tries to free DMA
memory it has not allocated".
Cc: stable@vger.kernel.org
Fixes: 1b44c5a60c ("crypto: inside-secure - add SafeXcel EIP197 crypto engine driver")
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The Inside Secure SafeXcel driver was firstly designed to support the
EIP197 cryptographic engine which is an evolution (with much more
feature, better performances) of the EIP97 cryptographic engine. This
patch convert the Inside Secure SafeXcel driver to support both engines
(EIP97 + EIP197).
The main differences are the register offsets and the context
invalidation process which is EIP197 specific. This patch adds an
indirection on the register offsets and adds checks not to send any
invalidation request when driving the EIP97. A new compatible is added
as well to bind the driver from device trees.
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This patch moves the request dequeueing into a workqueue to improve the
coalescing of interrupts when sending requests to the engine; as the
engine is capable of having one single interrupt for n requests sent.
Using a workqueue allows to send more request at once.
Suggested-by: Ofer Heifetz <oferh@marvell.com>
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The SafeXcel context isn't used in the cache invalidation function. This
cosmetic patch removes it (as well as from the function prototype in the
header file and when the function is called).
Signed-off-by: Ofer Heifetz <oferh@marvell.com>
[Antoine: commit message]
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When initializing the IVs crypto_ahash_update() is called, which at some
point will call crypto_enqueue_request(). This function can return
-EBUSY when no resource is available and the request is queued. Since
this is a valid case, -EBUSY shouldn't be treated as an error.
Signed-off-by: Ofer Heifetz <oferh@marvell.com>
[Antoine: commit message]
Signed-off-by: Antoine Tenart <antoine.tenart@free-electrons.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
