Commit Graph

3984 Commits

Author SHA1 Message Date
Sven Auhagen
c672041590 crypto: inside-secure - irq balance
Balance the irqs of the inside secure driver over all
available cpus.
Currently all interrupts are handled by the first CPU.

From my testing with IPSec AES-GCM 256
on my MCbin with 4 Cores I get a 50% speed increase:

Before the patch: 99.73 Kpps
With the patch: 151.25 Kpps

Signed-off-by: Sven Auhagen <sven.auhagen@voleatech.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-31 18:08:59 +10:00
Herbert Xu
ac855b3c10 crypto: omap-aes - Fix sparse and compiler warnings
This patch fixes all the sparse and W=1 compiler warnings in the
driver.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-23 17:34:19 +10:00
Tom Rix
c06c76602e crypto: qat - fix double free in qat_uclo_create_batch_init_list
clang static analysis flags this error

qat_uclo.c:297:3: warning: Attempt to free released memory
  [unix.Malloc]
                kfree(*init_tab_base);
                ^~~~~~~~~~~~~~~~~~~~~

When input *init_tab_base is null, the function allocates memory for
the head of the list.  When there is problem allocating other list
elements the list is unwound and freed.  Then a check is made if the
list head was allocated and is also freed.

Keeping track of the what may need to be freed is the variable 'tail_old'.
The unwinding/freeing block is

	while (tail_old) {
		mem_init = tail_old->next;
		kfree(tail_old);
		tail_old = mem_init;
	}

The problem is that the first element of tail_old is also what was
allocated for the list head

		init_header = kzalloc(sizeof(*init_header), GFP_KERNEL);
		...
		*init_tab_base = init_header;
		flag = 1;
	}
	tail_old = init_header;

So *init_tab_base/init_header are freed twice.

There is another problem.
When the input *init_tab_base is non null the tail_old is calculated by
traveling down the list to first non null entry.

	tail_old = init_header;
	while (tail_old->next)
		tail_old = tail_old->next;

When the unwinding free happens, the last entry of the input list will
be freed.

So the freeing needs a general changed.
If locally allocated the first element of tail_old is freed, else it
is skipped.  As a bit of cleanup, reset *init_tab_base if it came in
as null.

Fixes: b4b7e67c91 ("crypto: qat - Intel(R) QAT ucode part of fw loader")
Cc: <stable@vger.kernel.org>
Signed-off-by: Tom Rix <trix@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-23 17:34:17 +10:00
Tero Kristo
fd92028e4c crypto: sa2ul - add device links to child devices
The child devices for sa2ul (like the RNG) have hard dependency towards
the parent, they can't function without the parent enabled. Add device
link for this purpose so that the dependencies are taken care of properly.

Signed-off-by: Tero Kristo <t-kristo@ti.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-23 17:34:17 +10:00
Keerthy
d2c8ac187f crypto: sa2ul - Add AEAD algorithm support
Add support for sa2ul hardware AEAD for hmac(sha256),cbc(aes) and
hmac(sha1),cbc(aes) algorithms.

Signed-off-by: Keerthy <j-keerthy@ti.com>
[t-kristo@ti.com: number of bug fixes, major refactoring and cleanup of
 code]
Signed-off-by: Tero Kristo <t-kristo@ti.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-23 17:34:17 +10:00
Keerthy
2dc53d0047 crypto: sa2ul - add sha1/sha256/sha512 support
Add support for sha1/sha256/sha512 sa2ul based hardware authentication.
With the hash update mechanism, we always use software fallback
mechanism for now, as there is no way to fetch the partial hash state
from the HW accelerator. HW accelerator is only used when digest is
called for a data chunk of known size.

Signed-off-by: Keerthy <j-keerthy@ti.com>
[t-kristo@ti.com: various bug fixes, major cleanups and refactoring of code]
Signed-off-by: Tero Kristo <t-kristo@ti.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-23 17:34:17 +10:00
Keerthy
7694b6ca64 crypto: sa2ul - Add crypto driver
Adds a basic crypto driver and currently supports AES/3DES
in cbc mode for both encryption and decryption.

Signed-off-by: Keerthy <j-keerthy@ti.com>
[t-kristo@ti.com: major re-work to fix various bugs in the driver and to
 cleanup the code]
Signed-off-by: Tero Kristo <t-kristo@ti.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-23 17:34:16 +10:00
Christophe JAILLET
1b3eeb8761 crypto: chelsio - Fix some pr_xxx messages
At the top this file, we have:
   #define pr_fmt(fmt) "chcr:" fmt

So there is no need to repeat "chcr : " in some error message when the
pr_xxx macro is used.
This would lead to log "chcr:chcr : blabla"

Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:14 +10:00
Christophe JAILLET
d110cf0ac1 crypto: chelsio - Avoid some code duplication
The error handling path of 'chcr_authenc_setkey()' is the same as this
error handling code.

So just 'goto out' as done everywhere in the function to simplify the code.

Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:14 +10:00
Hui Tang
10f33d391e crypto: hisilicon/hpre - disable FLR triggered by hardware
for Hi1620 hardware, we should disable these hardware flr:
1. BME_FLR - bit 7,
2. PM_FLR - bit 11,
3. SRIOV_FLR - bit 12,
Or HPRE may goto D3 state, when we bind and unbind HPRE quickly,
as it does FLR triggered by BME/PM/SRIOV.

Fixes: c8b4b477079d("crypto: hisilicon - add HiSilicon HPRE accelerator")
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Meng Yu <yumeng18@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:12 +10:00
Meng Yu
dadbe4c117 crypto: hisilicon/hpre - update debugfs interface parameters
Update debugfs interface parameters, and adjust the
processing logic inside the corresponding function.

Fixes: 848974151618("crypto: hisilicon - Add debugfs for HPRE")
Signed-off-by: Meng Yu <yumeng18@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:12 +10:00
Meng Yu
a0c34e8dd8 crypto: hisilicon/hpre - Add a switch in sriov_configure
If CONFIG_PCI_IOV is not enabled, we can not use "sriov_configure".

Fixes: 5ec302a364bf("crypto: hisilicon - add SRIOV support for HPRE")
Signed-off-by: Meng Yu <yumeng18@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Reviewed-by: Shukun Tan <tanshukun1@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:11 +10:00
Meng Yu
a14f6609b0 crypto: hisilicon/hpre - Modify the Macro definition and format
1. Bit 1 to bit 5 are NFE, not CE.
2. Macro 'HPRE_VF_NUM' is defined in 'qm.h', so delete it here.
3. Delete multiple blank lines.
4. Adjust format alignment.

Signed-off-by: Meng Yu <yumeng18@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Reviewed-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:11 +10:00
Hui Tang
6bc937b0b9 crypto: hisilicon/hpre - HPRE_OVERTIME_THRHLD can be written by debugfs
Registers in "hpre_dfx_files" can only be cleaned to zero but
HPRE_OVERTIME_THRHLD, which can be written as any number.

Fixes: 64a6301ebee7("crypto: hisilicon/hpre - add debugfs for ...")
Signed-off-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Meng Yu <yumeng18@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Reviewed-by: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:11 +10:00
Meng Yu
d7ba2c09e1 crypto: hisilicon/hpre - Init the value of current_q of debugfs
Initialize current queue number as HPRE_PF_DEF_Q_NUM, or it is zero
and we can't set its value by "current_q_write".

Signed-off-by: Meng Yu <yumeng18@huawei.com>
Reviewed-by: Zaibo Xu <xuzaibo@huawei.com>
Reviewed-by: Hui Tang <tanghui20@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:10 +10:00
Mikulas Patocka
b8aa7dc5c7 crypto: drivers - set the flag CRYPTO_ALG_ALLOCATES_MEMORY
Set the flag CRYPTO_ALG_ALLOCATES_MEMORY in the crypto drivers that
allocate memory.

drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c: sun8i_ce_cipher
drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c: sun8i_ss_cipher
drivers/crypto/amlogic/amlogic-gxl-core.c: meson_cipher
drivers/crypto/axis/artpec6_crypto.c: artpec6_crypto_common_init
drivers/crypto/bcm/cipher.c: spu_skcipher_rx_sg_create
drivers/crypto/caam/caamalg.c: aead_edesc_alloc
drivers/crypto/caam/caamalg_qi.c: aead_edesc_alloc
drivers/crypto/caam/caamalg_qi2.c: aead_edesc_alloc
drivers/crypto/caam/caamhash.c: hash_digest_key
drivers/crypto/cavium/cpt/cptvf_algs.c: process_request
drivers/crypto/cavium/nitrox/nitrox_aead.c: nitrox_process_se_request
drivers/crypto/cavium/nitrox/nitrox_skcipher.c: nitrox_process_se_request
drivers/crypto/ccp/ccp-crypto-aes-cmac.c: ccp_do_cmac_update
drivers/crypto/ccp/ccp-crypto-aes-galois.c: ccp_crypto_enqueue_request
drivers/crypto/ccp/ccp-crypto-aes-xts.c: ccp_crypto_enqueue_request
drivers/crypto/ccp/ccp-crypto-aes.c: ccp_crypto_enqueue_request
drivers/crypto/ccp/ccp-crypto-des3.c: ccp_crypto_enqueue_request
drivers/crypto/ccp/ccp-crypto-sha.c: ccp_crypto_enqueue_request
drivers/crypto/chelsio/chcr_algo.c: create_cipher_wr
drivers/crypto/hisilicon/sec/sec_algs.c: sec_alloc_and_fill_hw_sgl
drivers/crypto/hisilicon/sec2/sec_crypto.c: sec_alloc_req_id
drivers/crypto/inside-secure/safexcel_cipher.c: safexcel_queue_req
drivers/crypto/inside-secure/safexcel_hash.c: safexcel_ahash_enqueue
drivers/crypto/ixp4xx_crypto.c: ablk_perform
drivers/crypto/marvell/cesa/cipher.c: mv_cesa_skcipher_dma_req_init
drivers/crypto/marvell/cesa/hash.c: mv_cesa_ahash_dma_req_init
drivers/crypto/marvell/octeontx/otx_cptvf_algs.c: create_ctx_hdr
drivers/crypto/n2_core.c: n2_compute_chunks
drivers/crypto/picoxcell_crypto.c: spacc_sg_to_ddt
drivers/crypto/qat/qat_common/qat_algs.c: qat_alg_skcipher_encrypt
drivers/crypto/qce/skcipher.c: qce_skcipher_async_req_handle
drivers/crypto/talitos.c : talitos_edesc_alloc
drivers/crypto/virtio/virtio_crypto_algs.c: __virtio_crypto_skcipher_do_req
drivers/crypto/xilinx/zynqmp-aes-gcm.c: zynqmp_aes_aead_cipher

Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
[EB: avoid overly-long lines]
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:10 +10:00
Herbert Xu
2c2e18369f crypto: ccp - Silence strncpy warning
This patch kills an strncpy by using strscpy instead.  The name
would be silently truncated if it is too long.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Acked-by: John Allen <john.allen@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:06 +10:00
Ard Biesheuvel
f441ba2ad3 crypto: mediatek - use AES library for GCM key derivation
The Mediatek accelerator driver calls into a dynamically allocated
skcipher of the ctr(aes) variety to perform GCM key derivation, which
involves AES encryption of a single block consisting of NUL bytes.

There is no point in using the skcipher API for this, so use the AES
library interface instead.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:04 +10:00
Ard Biesheuvel
56ca499f1b crypto: sahara - permit asynchronous skcipher as fallback
Even though the sahara driver implements asynchronous versions of
ecb(aes) and cbc(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.

Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Horia Geantă <horia.geanta@nxp.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:04 +10:00
Ard Biesheuvel
90e2f78271 crypto: qce - permit asynchronous skcipher as fallback
Even though the qce driver implements asynchronous versions of ecb(aes),
cbc(aes)and xts(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.

Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.

While at it, remove the pointless memset() from qce_skcipher_init(), and
remove the call to it qce_skcipher_init_fallback().

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:03 +10:00
Ard Biesheuvel
dc6e71c9d9 crypto: picoxcell - permit asynchronous skcipher as fallback
Even though the picoxcell driver implements asynchronous versions of
ecb(aes) and cbc(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.

Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Jamie Iles <jamie@jamieiles.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:03 +10:00
Ard Biesheuvel
c9598d4e13 crypto: mxs-dcp - permit asynchronous skcipher as fallback
Even though the mxs-dcp driver implements asynchronous versions of
ecb(aes) and cbc(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.

Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Horia Geantă <horia.geanta@nxp.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:03 +10:00
Ard Biesheuvel
d8c6d1886c crypto: chelsio - permit asynchronous skcipher as fallback
Even though the chelsio driver implements asynchronous versions of
cbc(aes) and xts(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.

Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:03 +10:00
Ard Biesheuvel
413b61ce0b crypto: ccp - permit asynchronous skcipher as fallback
Even though the ccp driver implements an asynchronous version of xts(aes),
the fallback it allocates is required to be synchronous. Given that SIMD
based software implementations are usually asynchronous as well, even
though they rarely complete asynchronously (this typically only happens
in cases where the request was made from softirq context, while SIMD was
already in use in the task context that it interrupted), these
implementations are disregarded, and either the generic C version or
another table based version implemented in assembler is selected instead.

Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Acked-by: John Allen <john.allen@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:03 +10:00
Ard Biesheuvel
44b5917537 crypto: sun8i-ss - permit asynchronous skcipher as fallback
Even though the sun8i-ss driver implements asynchronous versions of
ecb(aes) and cbc(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.

Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Acked-by: Corentin Labbe <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:02 +10:00
Ard Biesheuvel
31abd3eb3d crypto: sun8i-ce - permit asynchronous skcipher as fallback
Even though the sun8i-ce driver implements asynchronous versions of
ecb(aes) and cbc(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.

Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Acked-by: Corentin Labbe <clabbe.montjoie@gmail.com>
Tested-by: Corentin Labbe <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:02 +10:00
Ard Biesheuvel
89fb00f245 crypto: sun4i - permit asynchronous skcipher as fallback
Even though the sun4i driver implements asynchronous versions of ecb(aes)
and cbc(aes), the fallbacks it allocates are required to be synchronous.
Given that SIMD based software implementations are usually asynchronous
as well, even though they rarely complete asynchronously (this typically
only happens in cases where the request was made from softirq context,
while SIMD was already in use in the task context that it interrupted),
these implementations are disregarded, and either the generic C version
or another table based version implemented in assembler is selected
instead.

Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Tested-by: Corentin Labbe <clabbe.montjoie@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:02 +10:00
Ard Biesheuvel
6a99d7a2d7 crypto: omap-aes - permit asynchronous skcipher as fallback
Even though the omap-aes driver implements asynchronous versions of
ecb(aes), cbc(aes) and ctr(aes), the fallbacks it allocates are required
to be synchronous. Given that SIMD based software implementations are
usually asynchronous as well, even though they rarely complete
asynchronously (this typically only happens in cases where the request was
made from softirq context, while SIMD was already in use in the task
context that it interrupted), these implementations are disregarded, and
either the generic C version or another table based version implemented in
assembler is selected instead.

Since falling back to synchronous AES is not only a performance issue, but
potentially a security issue as well (due to the fact that table based AES
is not time invariant), let's fix this, by allocating an ordinary skcipher
as the fallback, and invoke it with the completion routine that was given
to the outer request.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:02 +10:00
Ard Biesheuvel
1d63e4557f crypto: amlogic-gxl - permit async skcipher as fallback
Even though the amlogic-gxl driver implements asynchronous versions of
ecb(aes) and cbc(aes), the fallbacks it allocates are required to be
synchronous. Given that SIMD based software implementations are usually
asynchronous as well, even though they rarely complete asynchronously
(this typically only happens in cases where the request was made from
softirq context, while SIMD was already in use in the task context that
it interrupted), these implementations are disregarded, and either the
generic C version or another table based version implemented in assembler
is selected instead.

Since falling back to synchronous AES is not only a performance issue,
but potentially a security issue as well (due to the fact that table
based AES is not time invariant), let's fix this, by allocating an
ordinary skcipher as the fallback, and invoke it with the completion
routine that was given to the outer request.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Tested-by: Corentin Labbe <clabbe@baylibre.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:02 +10:00
Ard Biesheuvel
3f368b886e crypto: amlogic-gxl - default to build as module
The AmLogic GXL crypto accelerator driver is built into the kernel if
ARCH_MESON is set. However, given the single image policy of arm64, its
defconfig enables all platforms by default, and so ARCH_MESON is usually
enabled.

This means that the AmLogic driver causes the arm64 defconfig build to
pull in a huge chunk of the crypto stack as a builtin as well, which is
undesirable, so let's make the amlogic GXL driver default to 'm' instead.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Tested-by: Corentin Labbe <clabbe@baylibre.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:01 +10:00
Longfang Liu
38c3b74edf crypto: hisilicon/sec2 - fix some coding styles
Modify some log output interfaces and
update author information

Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:01 +10:00
Longfang Liu
6b534f7aaf crypto: hisilicon/sec2 - update debugfs interface parameters
Update debugfs interface parameters, and adjust the
processing logic inside the corresponding function

Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:01 +10:00
Longfang Liu
d0228aeb4d crypto: hisilicon/sec2 - update SEC initialization and reset
Updates the initialization and reset of SEC driver's
register operation.

Signed-off-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:01 +10:00
Kai Ye
9597efc3ae crypto:hisilicon/sec2 - update busy processing logic
As before, if a SEC queue is at the 'fake busy' status,
the request with a 'fake busy' flag will be sent into hardware
and the sending function returns busy. After the request is
finished, SEC driver's call back will identify the 'fake busy' flag,
and notifies the user that hardware is not busy now by calling
user's call back function.

Now, a request sent into busy hardware will be cached in the
SEC queue's backlog, return '-EBUSY' to user.
After the request being finished, the cached requests will
be processed in the call back function. to notify the
corresponding user that SEC queue can process more requests.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Reviewed-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:00 +10:00
Kai Ye
7dc95d0ea3 crypto: hisilicon/sec2 - clear SEC debug regs
SEC debug registers aren't cleared even if its driver is removed,
so add a clearing operation in driver removing.

Signed-off-by: Kai Ye <yekai13@huawei.com>
Reviewed-by: Longfang Liu <liulongfang@huawei.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:00 +10:00
Herbert Xu
eeedb61837 crypto: caam - Remove broken arc4 support
The arc4 algorithm requires storing state in the request context
in order to allow more than one encrypt/decrypt operation.  As this
driver does not seem to do that, it means that using it for more
than one operation is broken.

Fixes: eaed71a44a ("crypto: caam - add ecb(*) support")
Link: https://lore.kernel.org/linux-crypto/CAMj1kXGvMe_A_iQ43Pmygg9xaAM-RLy=_M=v+eg--8xNmv9P+w@mail.gmail.com
Link: https://lore.kernel.org/linux-crypto/20200702101947.682-1-ardb@kernel.org
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Acked-by: Horia Geantă <horia.geanta@nxp.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-16 21:49:00 +10:00
Barry Song
813ec3f1fe crypto: hisilicon/zip - permit users to specify NUMA node
If users don't specify NUMA node, the driver will use the ZIP module near
the CPU allocating acomp. Otherwise, it uses the ZIP module according to
the requirement of users.

Cc: Zhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: Barry Song <song.bao.hua@hisilicon.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 18:25:23 +10:00
Herbert Xu
d9dd5ef3d3 crypto: ccp - Fix sparse warnings
This patch fixes a number of endianness marking issues in the ccp
driver.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Acked-by: John Allen <john.allen@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 18:25:22 +10:00
Rikard Falkeborn
ad6a0664c2 crypto: virtio - constify features[] and id_table[]
features[] and id_table[] are not modified and can be made const to
allow the compiler to put them in read-only memory.

Before:
   text    data     bss     dec     hex filename
  11534    2056     160   13750    35b6 drivers/crypto/virtio/virtio_crypto_core.o

After:
   text    data     bss     dec     hex filename
  11630    1992     128   13750    35b6 drivers/crypto/virtio/virtio_crypto_core.o

Signed-off-by: Rikard Falkeborn <rikard.falkeborn@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 18:25:22 +10:00
Giovanni Cabiddu
a85211f36f crypto: qat - fallback for xts with 192 bit keys
Forward requests to another provider if the key length for AES-XTS is
192 bits as this is not supported by the QAT accelerators.

This fixes the following issue reported with the option
CONFIG_CRYPTO_MANAGER_EXTRA_TESTS:

  alg: skcipher: qat_aes_xts setkey failed on test vector "random: len=3204 klen=48"; expected_error=0, actual_error=-22, flags=0x1

Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 18:25:16 +10:00
Giovanni Cabiddu
5fb8b70d20 crypto: qat - remove unused field in skcipher ctx
Remove tfm field in qat_alg_skcipher_ctx structure.
This is not used.

Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 18:25:16 +10:00
Giovanni Cabiddu
b185a68710 crypto: qat - validate xts key
Validate AES-XTS key using the function xts_verify_key() to prevent
malformed keys.

Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 18:25:15 +10:00
Giovanni Cabiddu
528f776df6 crypto: qat - allow xts requests not multiple of block
Allow AES-XTS requests that are not multiple of the block size.
If a request is smaller than the block size, return -EINVAL.

This fixes the following issue reported by the crypto testmgr self-test:

  alg: skcipher: qat_aes_xts encryption failed on test vector "random: len=116 klen=64"; expected_error=0, actual_error=-22, cfg="random: inplace may_sleep use_finup src_divs=[<reimport>45.85%@+4077, <flush>54.15%@alignmask+18]"

Fixes: 96ee111a65 ("crypto: qat - return error for block...")
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 18:25:14 +10:00
Herbert Xu
95a62311ed Merge branch 'ux500'
The change on the ux500 branch is needed by the regulator API.
2020-07-09 16:02:06 +10:00
Lee Jones
3cfa435c69 crypto: ux500/hash - Add namespacing to hash_init()
A recent change to the Regulator consumer API (which this driver
utilises) add prototypes for the some suspend functions.  These
functions require including header file include/linux/suspend.h.

The following tree of includes affecting this driver will be
present:

   In file included from include/linux/elevator.h:6,
                    from include/linux/blkdev.h:288,
                    from include/linux/blk-cgroup.h:23,
                    from include/linux/writeback.h:14,
                    from include/linux/memcontrol.h:22,
                    from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from include/linux/regulator/consumer.h:35,
                    from drivers/crypto/ux500/hash/hash_core.c:28:

include/linux/elevator.h pulls in include/linux/hashtable.h which
contains its own version of hash_init().  This confuses the build
system and results in the following error (amongst others):

 drivers/crypto/ux500/hash/hash_core.c:1362:19: error: passing argument 1 of '__hash_init' from incompatible pointer type [-Werror=incompatible-pointer-types]
 1362 |  return hash_init(req);

Fix this by namespacing the local hash_init() such that the
source of confusion is removed.

Cc: Linus Walleij <linus.walleij@linaro.org>
Cc: David S. Miller <davem@davemloft.net>
Cc: linux-crypto@vger.kernel.org
Signed-off-by: Lee Jones <lee.jones@linaro.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-09 15:49:28 +10:00
Herbert Xu
c414943119 crypto: cpt - Fix sparse warnings
This patch fixes all the sparse warnings in the octeontx driver.
Some of these are just trivial type changes.

However, some of the changes are non-trivial on little-endian hosts.
Obviously the driver appears to be broken on either LE or BE as it
was doing different things.  I've taken the BE behaviour as the
correct one.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-03 14:18:35 +10:00
Fenghua Yu
f532ed2a9b crypto: hisilicon/qm - Change type of pasid to u32
PASID is defined as "int" although it's a 20-bit value and shouldn't be
negative int. To be consistent with PASID type in iommu, define PASID
as "u32".

Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Fenghua Yu <fenghua.yu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-03 14:18:35 +10:00
John Allen
8a302808c6 crypto: ccp - Fix use of merged scatterlists
Running the crypto manager self tests with
CONFIG_CRYPTO_MANAGER_EXTRA_TESTS may result in several types of errors
when using the ccp-crypto driver:

alg: skcipher: cbc-des3-ccp encryption failed on test vector 0; expected_error=0, actual_error=-5 ...

alg: skcipher: ctr-aes-ccp decryption overran dst buffer on test vector 0 ...

alg: ahash: sha224-ccp test failed (wrong result) on test vector ...

These errors are the result of improper processing of scatterlists mapped
for DMA.

Given a scatterlist in which entries are merged as part of mapping the
scatterlist for DMA, the DMA length of a merged entry will reflect the
combined length of the entries that were merged. The subsequent
scatterlist entry will contain DMA information for the scatterlist entry
after the last merged entry, but the non-DMA information will be that of
the first merged entry.

The ccp driver does not take this scatterlist merging into account. To
address this, add a second scatterlist pointer to track the current
position in the DMA mapped representation of the scatterlist. Both the DMA
representation and the original representation of the scatterlist must be
tracked as while most of the driver can use just the DMA representation,
scatterlist_map_and_copy() must use the original representation and
expects the scatterlist pointer to be accurate to the original
representation.

In order to properly walk the original scatterlist, the scatterlist must
be walked until the combined lengths of the entries seen is equal to the
DMA length of the current entry being processed in the DMA mapped
representation.

Fixes: 63b945091a ("crypto: ccp - CCP device driver and interface support")
Signed-off-by: John Allen <john.allen@amd.com>
Cc: stable@vger.kernel.org
Acked-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-03 14:18:34 +10:00
Sivaprakash Murugesan
df12ef60c8 crypto: qce/sha - Do not modify scatterlist passed along with request
Crypto test driver's test_ahash_speed calls crypto_ahash_update and
crypto_ahash_final APIs repeatedly for all the available test vector
buffer lengths.

if we mark the end for scatterlist based on the current vector size then
the subsequent vectors might fail if the later buffer lengths are higher.

To avoid this, in qce do not mark the end of scatterlist in update API,
the qce_ahash_async_req_handle API already takes care of this copying
right amount of buffer from the request scatter list.

Signed-off-by: Sivaprakash Murugesan <sivaprak@codeaurora.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-03 14:18:34 +10:00
Sivaprakash Murugesan
a668ee56ff crypto: qce - re-initialize context on import
crypto testmgr deliberately corrupts the request context while passing
vectors to the import. This is to make sure that drivers do not rely on
request but they take all the necessary input from io vec passed to it.

qce casts the request context from request parameter, since it is corrupted
the sub squent hash request fails and qce hangs.

To avoid this re-initialize request context on import. The qce import
API alreasy takes care of taking the input vectors from passed io vec.

Signed-off-by: Sivaprakash Murugesan <sivaprak@codeaurora.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2020-07-03 14:18:34 +10:00