linux/arch/x86/crypto
Tianjia Zhang 930ab34d90 crypto: x86/sm3 - add AVX assembly implementation
This patch adds AVX assembly accelerated implementation of SM3 secure
hash algorithm. From the benchmark data, compared to pure software
implementation sm3-generic, the performance increase is up to 38%.

The main algorithm implementation based on SM3 AES/BMI2 accelerated
work by libgcrypt at:
https://gnupg.org/software/libgcrypt/index.html

Benchmark on Intel i5-6200U 2.30GHz, performance data of two
implementations, pure software sm3-generic and sm3-avx acceleration.
The data comes from the 326 mode and 422 mode of tcrypt. The abscissas
are different lengths of per update. The data is tabulated and the
unit is Mb/s:

update-size |     16      64     256    1024    2048    4096    8192
------------+-------------------------------------------------------
sm3-generic | 105.97  129.60  182.12  189.62  188.06  193.66  194.88
sm3-avx     | 119.87  163.05  244.44  260.92  257.60  264.87  265.88

Signed-off-by: Tianjia Zhang <tianjia.zhang@linux.alibaba.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2022-01-28 16:51:11 +11:00
..
.gitignore .gitignore: add SPDX License Identifier 2020-03-25 11:50:48 +01:00
aegis128-aesni-asm.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
aegis128-aesni-glue.c crypto: remove CRYPTO_TFM_RES_BAD_KEY_LEN 2020-01-09 11:30:53 +08:00
aes_ctrby8_avx-x86_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
aesni-intel_asm.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
aesni-intel_avx-x86_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
aesni-intel_glue.c crypto: x86/aesni - don't require alignment of data 2021-12-31 18:10:55 +11:00
blake2s-core.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
blake2s-glue.c lib/crypto: blake2s: include as built-in 2022-01-07 00:25:25 +01:00
blake2s-shash.c lib/crypto: blake2s: include as built-in 2022-01-07 00:25:25 +01:00
blowfish_glue.c crypto: x86/blowfish - drop CTR mode implementation 2021-01-14 17:10:29 +11:00
blowfish-x86_64-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
camellia_aesni_avx2_glue.c crypto: x86 - use local headers for x86 specific shared declarations 2021-01-14 17:10:30 +11:00
camellia_aesni_avx_glue.c crypto: x86 - use local headers for x86 specific shared declarations 2021-01-14 17:10:30 +11:00
camellia_glue.c crypto: x86 - use local headers for x86 specific shared declarations 2021-01-14 17:10:30 +11:00
camellia-aesni-avx2-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
camellia-aesni-avx-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
camellia-x86_64-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
camellia.h crypto: x86 - use local headers for x86 specific shared declarations 2021-01-14 17:10:30 +11:00
cast5_avx_glue.c crypto: x86/cast5 - drop dependency on glue helper 2021-01-14 17:10:29 +11:00
cast5-avx-x86_64-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
cast6_avx_glue.c crypto: x86/cast6 - drop dependency on glue helper 2021-01-14 17:10:29 +11:00
cast6-avx-x86_64-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
chacha_glue.c crypto: algapi - Remove skbuff.h inclusion 2020-08-20 14:04:28 +10:00
chacha-avx2-x86_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
chacha-avx512vl-x86_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
chacha-ssse3-x86_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
crc32-pclmul_asm.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
crc32-pclmul_glue.c x86: Fix various typos in comments, take #2 2021-03-21 23:50:28 +01:00
crc32c-intel_glue.c crypto: x86/crc32c-intel - Use CRC32 mnemonic 2020-08-21 14:45:28 +10:00
crc32c-pcl-intel-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
crct10dif-pcl-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
crct10dif-pclmul_glue.c crypto: Convert to new CPU match macros 2020-03-24 21:36:06 +01:00
curve25519-x86_64.c crypto: x86/curve25519 - use in/out register constraints more precisely 2021-12-24 14:18:22 +11:00
des3_ede_glue.c crypto: x86/des3 - remove redundant assignment of variable nbytes 2021-12-17 16:59:46 +11:00
des3_ede-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
ecb_cbc_helpers.h crypto: x86 - add some helper macros for ECB and CBC modes 2021-01-14 17:10:29 +11:00
ghash-clmulni-intel_asm.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
ghash-clmulni-intel_glue.c crypto: Convert to new CPU match macros 2020-03-24 21:36:06 +01:00
glue_helper-asm-avx2.S crypto: x86/glue-helper - drop CTR helper routines 2021-01-14 17:10:28 +11:00
glue_helper-asm-avx.S crypto: x86/glue-helper - drop CTR helper routines 2021-01-14 17:10:28 +11:00
Makefile crypto: x86/sm3 - add AVX assembly implementation 2022-01-28 16:51:11 +11:00
nh-avx2-x86_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
nh-sse2-x86_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
nhpoly1305-avx2-glue.c crypto: algapi - Remove skbuff.h inclusion 2020-08-20 14:04:28 +10:00
nhpoly1305-sse2-glue.c crypto: algapi - Remove skbuff.h inclusion 2020-08-20 14:04:28 +10:00
poly1305_glue.c crypto: poly1305 - fix poly1305_core_setkey() declaration 2021-04-02 18:28:12 +11:00
poly1305-x86_64-cryptogams.pl crypto: x86/poly1305 - Use TEST %reg,%reg instead of CMP $0,%reg 2020-12-04 18:13:16 +11:00
serpent_avx2_glue.c crypto: x86 - use local headers for x86 specific shared declarations 2021-01-14 17:10:30 +11:00
serpent_avx_glue.c crypto: x86 - use local headers for x86 specific shared declarations 2021-01-14 17:10:30 +11:00
serpent_sse2_glue.c crypto: x86 - use local headers for x86 specific shared declarations 2021-01-14 17:10:30 +11:00
serpent-avx2-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
serpent-avx-x86_64-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
serpent-avx.h crypto: x86 - use local headers for x86 specific shared declarations 2021-01-14 17:10:30 +11:00
serpent-sse2-i586-asm_32.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
serpent-sse2-x86_64-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
serpent-sse2.h crypto: x86 - use local headers for x86 specific shared declarations 2021-01-14 17:10:30 +11:00
sha1_avx2_x86_64_asm.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
sha1_ni_asm.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
sha1_ssse3_asm.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
sha1_ssse3_glue.c crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
sha256_ni_asm.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
sha256_ssse3_glue.c crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
sha256-avx2-asm.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
sha256-avx-asm.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
sha256-ssse3-asm.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
sha512_ssse3_glue.c crypto: sha - split sha.h into sha1.h and sha2.h 2020-11-20 14:45:33 +11:00
sha512-avx2-asm.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
sha512-avx-asm.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
sha512-ssse3-asm.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
sm3_avx_glue.c crypto: x86/sm3 - add AVX assembly implementation 2022-01-28 16:51:11 +11:00
sm3-avx-asm_64.S crypto: x86/sm3 - add AVX assembly implementation 2022-01-28 16:51:11 +11:00
sm4_aesni_avx2_glue.c crypto: x86/sm4 - add AES-NI/AVX2/x86_64 implementation 2021-08-27 16:30:18 +08:00
sm4_aesni_avx_glue.c crypto: x86/sm4 - export reusable AESNI/AVX functions 2021-08-27 16:30:18 +08:00
sm4-aesni-avx2-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
sm4-aesni-avx-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
sm4-avx.h crypto: x86/sm4 - export reusable AESNI/AVX functions 2021-08-27 16:30:18 +08:00
twofish_avx_glue.c crypto: x86 - use local headers for x86 specific shared declarations 2021-01-14 17:10:30 +11:00
twofish_glue_3way.c x86: Fix various typos in comments 2021-03-18 15:31:53 +01:00
twofish_glue.c crypto: prefix module autoloading with "crypto-" 2014-11-24 22:43:57 +08:00
twofish-avx-x86_64-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
twofish-i586-asm_32.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
twofish-x86_64-asm_64-3way.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
twofish-x86_64-asm_64.S x86: Prepare asm files for straight-line-speculation 2021-12-08 12:25:37 +01:00
twofish.h crypto: x86 - use local headers for x86 specific shared declarations 2021-01-14 17:10:30 +11:00