Minki/linux
1
0
Fork 1
mirror of https://github.com/torvalds/linux.git synced 2024-12-15 07:33:56 +00:00
Code Issues Packages Projects Releases Wiki Activity
541a496644
linux/arch/s390/include/asm/vx-insn.h

679 lines
13 KiB
C
Raw Normal View History

License cleanup: add SPDX GPL-2.0 license identifier to files with no license Many source files in the tree are missing licensing information, which makes it harder for compliance tools to determine the correct license. By default all files without license information are under the default license of the kernel, which is GPL version 2. Update the files which contain no license information with the 'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally binding shorthand, which can be used instead of the full boiler plate text. This patch is based on work done by Thomas Gleixner and Kate Stewart and Philippe Ombredanne. How this work was done: Patches were generated and checked against linux-4.14-rc6 for a subset of the use cases: - file had no licensing information it it. - file was a */uapi/* one with no licensing information in it, - file was a */uapi/* one with existing licensing information, Further patches will be generated in subsequent months to fix up cases where non-standard license headers were used, and references to license had to be inferred by heuristics based on keywords. The analysis to determine which SPDX License Identifier to be applied to a file was done in a spreadsheet of side by side results from of the output of two independent scanners (ScanCode & Windriver) producing SPDX tag:value files created by Philippe Ombredanne. Philippe prepared the base worksheet, and did an initial spot review of a few 1000 files. The 4.13 kernel was the starting point of the analysis with 60,537 files assessed. Kate Stewart did a file by file comparison of the scanner results in the spreadsheet to determine which SPDX license identifier(s) to be applied to the file. She confirmed any determination that was not immediately clear with lawyers working with the Linux Foundation. Criteria used to select files for SPDX license identifier tagging was: - Files considered eligible had to be source code files. - Make and config files were included as candidates if they contained >5 lines of source - File already had some variant of a license header in it (even if <5 lines). All documentation files were explicitly excluded. The following heuristics were used to determine which SPDX license identifiers to apply. - when both scanners couldn't find any license traces, file was considered to have no license information in it, and the top level COPYING file license applied. For non */uapi/* files that summary was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 11139 and resulted in the first patch in this series. If that file was a */uapi/* path one, it was "GPL-2.0 WITH Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 WITH Linux-syscall-note 930 and resulted in the second patch in this series. - if a file had some form of licensing information in it, and was one of the */uapi/* ones, it was denoted with the Linux-syscall-note if any GPL family license was found in the file or had no licensing in it (per prior point). Results summary: SPDX license identifier # files ---------------------------------------------------|------ GPL-2.0 WITH Linux-syscall-note 270 GPL-2.0+ WITH Linux-syscall-note 169 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17 LGPL-2.1+ WITH Linux-syscall-note 15 GPL-1.0+ WITH Linux-syscall-note 14 ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5 LGPL-2.0+ WITH Linux-syscall-note 4 LGPL-2.1 WITH Linux-syscall-note 3 ((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3 ((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1 and that resulted in the third patch in this series. - when the two scanners agreed on the detected license(s), that became the concluded license(s). - when there was disagreement between the two scanners (one detected a license but the other didn't, or they both detected different licenses) a manual inspection of the file occurred. - In most cases a manual inspection of the information in the file resulted in a clear resolution of the license that should apply (and which scanner probably needed to revisit its heuristics). - When it was not immediately clear, the license identifier was confirmed with lawyers working with the Linux Foundation. - If there was any question as to the appropriate license identifier, the file was flagged for further research and to be revisited later in time. In total, over 70 hours of logged manual review was done on the spreadsheet to determine the SPDX license identifiers to apply to the source files by Kate, Philippe, Thomas and, in some cases, confirmation by lawyers working with the Linux Foundation. Kate also obtained a third independent scan of the 4.13 code base from FOSSology, and compared selected files where the other two scanners disagreed against that SPDX file, to see if there was new insights. The Windriver scanner is based on an older version of FOSSology in part, so they are related. Thomas did random spot checks in about 500 files from the spreadsheets for the uapi headers and agreed with SPDX license identifier in the files he inspected. For the non-uapi files Thomas did random spot checks in about 15000 files. In initial set of patches against 4.14-rc6, 3 files were found to have copy/paste license identifier errors, and have been fixed to reflect the correct identifier. Additionally Philippe spent 10 hours this week doing a detailed manual inspection and review of the 12,461 patched files from the initial patch version early this week with: - a full scancode scan run, collecting the matched texts, detected license ids and scores - reviewing anything where there was a license detected (about 500+ files) to ensure that the applied SPDX license was correct - reviewing anything where there was no detection but the patch license was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied SPDX license was correct This produced a worksheet with 20 files needing minor correction. This worksheet was then exported into 3 different .csv files for the different types of files to be modified. These .csv files were then reviewed by Greg. Thomas wrote a script to parse the csv files and add the proper SPDX tag to the file, in the format that the file expected. This script was further refined by Greg based on the output to detect more types of files automatically and to distinguish between header and source .c files (which need different comment types.) Finally Greg ran the script using the .csv files to generate the patches. Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 14:07:57 +00:00
/* SPDX-License-Identifier: GPL-2.0 */
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
/*
* Support for Vector Instructions
*
* Assembler macros to generate .byte/.word code for particular
* vector instructions that are supported by recent binutils (>= 2.26) only.
*
* Copyright IBM Corp. 2015
* Author(s): Hendrik Brueckner <brueckner@linux.vnet.ibm.com>
*/
#ifndef __ASM_S390_VX_INSN_H
#define __ASM_S390_VX_INSN_H
#ifdef __ASSEMBLY__
/* Macros to generate vector instruction byte code */
/* GR_NUM - Retrieve general-purpose register number
*
* @opd: Operand to store register number
* @r64: String designation register in the format "%rN"
*/
.macro GR_NUM opd gr
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
\opd = 255
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.ifc \gr,%r0
\opd = 0
.endif
.ifc \gr,%r1
\opd = 1
.endif
.ifc \gr,%r2
\opd = 2
.endif
.ifc \gr,%r3
\opd = 3
.endif
.ifc \gr,%r4
\opd = 4
.endif
.ifc \gr,%r5
\opd = 5
.endif
.ifc \gr,%r6
\opd = 6
.endif
.ifc \gr,%r7
\opd = 7
.endif
.ifc \gr,%r8
\opd = 8
.endif
.ifc \gr,%r9
\opd = 9
.endif
.ifc \gr,%r10
\opd = 10
.endif
.ifc \gr,%r11
\opd = 11
.endif
.ifc \gr,%r12
\opd = 12
.endif
.ifc \gr,%r13
\opd = 13
.endif
.ifc \gr,%r14
\opd = 14
.endif
.ifc \gr,%r15
\opd = 15
.endif
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.if \opd == 255
\opd = \gr
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.endif
.endm
/* VX_NUM - Retrieve vector register number
*
* @opd: Operand to store register number
* @vxr: String designation register in the format "%vN"
*
* The vector register number is used for as input number to the
* instruction and, as well as, to compute the RXB field of the
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
* instruction.
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
*/
.macro VX_NUM opd vxr
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
\opd = 255
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.ifc \vxr,%v0
\opd = 0
.endif
.ifc \vxr,%v1
\opd = 1
.endif
.ifc \vxr,%v2
\opd = 2
.endif
.ifc \vxr,%v3
\opd = 3
.endif
.ifc \vxr,%v4
\opd = 4
.endif
.ifc \vxr,%v5
\opd = 5
.endif
.ifc \vxr,%v6
\opd = 6
.endif
.ifc \vxr,%v7
\opd = 7
.endif
.ifc \vxr,%v8
\opd = 8
.endif
.ifc \vxr,%v9
\opd = 9
.endif
.ifc \vxr,%v10
\opd = 10
.endif
.ifc \vxr,%v11
\opd = 11
.endif
.ifc \vxr,%v12
\opd = 12
.endif
.ifc \vxr,%v13
\opd = 13
.endif
.ifc \vxr,%v14
\opd = 14
.endif
.ifc \vxr,%v15
\opd = 15
.endif
.ifc \vxr,%v16
\opd = 16
.endif
.ifc \vxr,%v17
\opd = 17
.endif
.ifc \vxr,%v18
\opd = 18
.endif
.ifc \vxr,%v19
\opd = 19
.endif
.ifc \vxr,%v20
\opd = 20
.endif
.ifc \vxr,%v21
\opd = 21
.endif
.ifc \vxr,%v22
\opd = 22
.endif
.ifc \vxr,%v23
\opd = 23
.endif
.ifc \vxr,%v24
\opd = 24
.endif
.ifc \vxr,%v25
\opd = 25
.endif
.ifc \vxr,%v26
\opd = 26
.endif
.ifc \vxr,%v27
\opd = 27
.endif
.ifc \vxr,%v28
\opd = 28
.endif
.ifc \vxr,%v29
\opd = 29
.endif
.ifc \vxr,%v30
\opd = 30
.endif
.ifc \vxr,%v31
\opd = 31
.endif
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.if \opd == 255
\opd = \vxr
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.endif
.endm
/* RXB - Compute most significant bit used vector registers
*
* @rxb: Operand to store computed RXB value
* @v1: First vector register designated operand
* @v2: Second vector register designated operand
* @v3: Third vector register designated operand
* @v4: Fourth vector register designated operand
*/
.macro RXB rxb v1 v2=0 v3=0 v4=0
\rxb = 0
.if \v1 & 0x10
\rxb = \rxb | 0x08
.endif
.if \v2 & 0x10
\rxb = \rxb | 0x04
.endif
.if \v3 & 0x10
\rxb = \rxb | 0x02
.endif
.if \v4 & 0x10
\rxb = \rxb | 0x01
.endif
.endm
/* MRXB - Generate Element Size Control and RXB value
*
* @m: Element size control
* @v1: First vector register designated operand (for RXB)
* @v2: Second vector register designated operand (for RXB)
* @v3: Third vector register designated operand (for RXB)
* @v4: Fourth vector register designated operand (for RXB)
*/
.macro MRXB m v1 v2=0 v3=0 v4=0
rxb = 0
RXB rxb, \v1, \v2, \v3, \v4
.byte (\m << 4) | rxb
.endm
/* MRXBOPC - Generate Element Size Control, RXB, and final Opcode fields
*
* @m: Element size control
* @opc: Opcode
* @v1: First vector register designated operand (for RXB)
* @v2: Second vector register designated operand (for RXB)
* @v3: Third vector register designated operand (for RXB)
* @v4: Fourth vector register designated operand (for RXB)
*/
.macro MRXBOPC m opc v1 v2=0 v3=0 v4=0
MRXB \m, \v1, \v2, \v3, \v4
.byte \opc
.endm
/* Vector support instructions */
/* VECTOR GENERATE BYTE MASK */
.macro VGBM vr imm2
VX_NUM v1, \vr
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.word (0xE700 | ((v1&15) << 4))
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.word \imm2
MRXBOPC 0, 0x44, v1
.endm
.macro VZERO vxr
VGBM \vxr, 0
.endm
.macro VONE vxr
VGBM \vxr, 0xFFFF
.endm
/* VECTOR LOAD VR ELEMENT FROM GR */
.macro VLVG v, gr, disp, m
VX_NUM v1, \v
GR_NUM b2, "%r0"
GR_NUM r3, \gr
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.word 0xE700 | ((v1&15) << 4) | r3
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.word (b2 << 12) | (\disp)
MRXBOPC \m, 0x22, v1
.endm
.macro VLVGB v, gr, index, base
VLVG \v, \gr, \index, \base, 0
.endm
.macro VLVGH v, gr, index
VLVG \v, \gr, \index, 1
.endm
.macro VLVGF v, gr, index
VLVG \v, \gr, \index, 2
.endm
.macro VLVGG v, gr, index
VLVG \v, \gr, \index, 3
.endm
RAID/s390: add SIMD implementation for raid6 gen/xor Using vector registers is slightly faster: raid6: vx128x8 gen() 19705 MB/s raid6: vx128x8 xor() 11886 MB/s raid6: using algorithm vx128x8 gen() 19705 MB/s raid6: .... xor() 11886 MB/s, rmw enabled vs the software algorithms: raid6: int64x1 gen() 3018 MB/s raid6: int64x1 xor() 1429 MB/s raid6: int64x2 gen() 4661 MB/s raid6: int64x2 xor() 3143 MB/s raid6: int64x4 gen() 5392 MB/s raid6: int64x4 xor() 3509 MB/s raid6: int64x8 gen() 4441 MB/s raid6: int64x8 xor() 3207 MB/s raid6: using algorithm int64x4 gen() 5392 MB/s raid6: .... xor() 3509 MB/s, rmw enabled Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-23 11:30:24 +00:00
/* VECTOR LOAD REGISTER */
.macro VLR v1, v2
VX_NUM v1, \v1
VX_NUM v2, \v2
.word 0xE700 | ((v1&15) << 4) | (v2&15)
.word 0
MRXBOPC 0, 0x56, v1, v2
.endm
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
/* VECTOR LOAD */
.macro VL v, disp, index="%r0", base
VX_NUM v1, \v
GR_NUM x2, \index
GR_NUM b2, \base
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.word 0xE700 | ((v1&15) << 4) | x2
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.word (b2 << 12) | (\disp)
MRXBOPC 0, 0x06, v1
.endm
/* VECTOR LOAD ELEMENT */
.macro VLEx vr1, disp, index="%r0", base, m3, opc
VX_NUM v1, \vr1
GR_NUM x2, \index
GR_NUM b2, \base
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.word 0xE700 | ((v1&15) << 4) | x2
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.word (b2 << 12) | (\disp)
MRXBOPC \m3, \opc, v1
.endm
.macro VLEB vr1, disp, index="%r0", base, m3
VLEx \vr1, \disp, \index, \base, \m3, 0x00
.endm
.macro VLEH vr1, disp, index="%r0", base, m3
VLEx \vr1, \disp, \index, \base, \m3, 0x01
.endm
.macro VLEF vr1, disp, index="%r0", base, m3
VLEx \vr1, \disp, \index, \base, \m3, 0x03
.endm
.macro VLEG vr1, disp, index="%r0", base, m3
VLEx \vr1, \disp, \index, \base, \m3, 0x02
.endm
/* VECTOR LOAD ELEMENT IMMEDIATE */
.macro VLEIx vr1, imm2, m3, opc
VX_NUM v1, \vr1
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.word 0xE700 | ((v1&15) << 4)
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.word \imm2
MRXBOPC \m3, \opc, v1
.endm
.macro VLEIB vr1, imm2, index
VLEIx \vr1, \imm2, \index, 0x40
.endm
.macro VLEIH vr1, imm2, index
VLEIx \vr1, \imm2, \index, 0x41
.endm
.macro VLEIF vr1, imm2, index
VLEIx \vr1, \imm2, \index, 0x43
.endm
.macro VLEIG vr1, imm2, index
VLEIx \vr1, \imm2, \index, 0x42
.endm
/* VECTOR LOAD GR FROM VR ELEMENT */
.macro VLGV gr, vr, disp, base="%r0", m
GR_NUM r1, \gr
GR_NUM b2, \base
VX_NUM v3, \vr
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.word 0xE700 | (r1 << 4) | (v3&15)
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.word (b2 << 12) | (\disp)
MRXBOPC \m, 0x21, v3
.endm
.macro VLGVB gr, vr, disp, base="%r0"
VLGV \gr, \vr, \disp, \base, 0
.endm
.macro VLGVH gr, vr, disp, base="%r0"
VLGV \gr, \vr, \disp, \base, 1
.endm
.macro VLGVF gr, vr, disp, base="%r0"
VLGV \gr, \vr, \disp, \base, 2
.endm
.macro VLGVG gr, vr, disp, base="%r0"
VLGV \gr, \vr, \disp, \base, 3
.endm
/* VECTOR LOAD MULTIPLE */
s390: add alignment hints to vector load and store The z14 introduced alignment hints to increase the performance of vector loads and stores. The kernel uses an implicit alignmenet of 8 bytes for the vector registers, set the alignment hint to 3. Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2019-02-06 17:06:03 +00:00
.macro VLM vfrom, vto, disp, base, hint=3
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
VX_NUM v1, \vfrom
VX_NUM v3, \vto
s390/vx: remove comments from macros which break LLVM's IAS LLVM's integrated assembler does not like comments within macros: <instantiation>:3:19: error: too many positional arguments GR_NUM b2, 1 /* Base register */ ^ Remove them, since they are obvious anyway. Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-05-03 08:48:41 +00:00
GR_NUM b2, \base
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.word 0xE700 | ((v1&15) << 4) | (v3&15)
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.word (b2 << 12) | (\disp)
s390: add alignment hints to vector load and store The z14 introduced alignment hints to increase the performance of vector loads and stores. The kernel uses an implicit alignmenet of 8 bytes for the vector registers, set the alignment hint to 3. Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2019-02-06 17:06:03 +00:00
MRXBOPC \hint, 0x36, v1, v3
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.endm
s390/crypto: add SIMD implementation for ChaCha20 Add an implementation of the ChaCha20 stream cipher (see e.g. RFC 7539) that makes use of z13's vector instruction set extension. The original implementation is by Andy Polyakov which is adapted for kernel use. Four to six blocks are processed in parallel resulting in a performance gain for inputs >= 256 bytes. chacha20-generic 1 operation in 622 cycles (256 bytes) 1 operation in 2346 cycles (1024 bytes) chacha20-s390 1 operation in 218 cycles (256 bytes) 1 operation in 647 cycles (1024 bytes) Cc: Andy Polyakov <appro@openssl.org> Reviewed-by: Harald Freudenberger <freude@de.ibm.com> Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2021-11-04 14:58:51 +00:00
/* VECTOR STORE */
.macro VST vr1, disp, index="%r0", base
VX_NUM v1, \vr1
GR_NUM x2, \index
s390/vx: remove comments from macros which break LLVM's IAS LLVM's integrated assembler does not like comments within macros: <instantiation>:3:19: error: too many positional arguments GR_NUM b2, 1 /* Base register */ ^ Remove them, since they are obvious anyway. Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-05-03 08:48:41 +00:00
GR_NUM b2, \base
s390/crypto: add SIMD implementation for ChaCha20 Add an implementation of the ChaCha20 stream cipher (see e.g. RFC 7539) that makes use of z13's vector instruction set extension. The original implementation is by Andy Polyakov which is adapted for kernel use. Four to six blocks are processed in parallel resulting in a performance gain for inputs >= 256 bytes. chacha20-generic 1 operation in 622 cycles (256 bytes) 1 operation in 2346 cycles (1024 bytes) chacha20-s390 1 operation in 218 cycles (256 bytes) 1 operation in 647 cycles (1024 bytes) Cc: Andy Polyakov <appro@openssl.org> Reviewed-by: Harald Freudenberger <freude@de.ibm.com> Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2021-11-04 14:58:51 +00:00
.word 0xE700 | ((v1&15) << 4) | (x2&15)
.word (b2 << 12) | (\disp)
MRXBOPC 0, 0x0E, v1
.endm
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
/* VECTOR STORE MULTIPLE */
s390: add alignment hints to vector load and store The z14 introduced alignment hints to increase the performance of vector loads and stores. The kernel uses an implicit alignmenet of 8 bytes for the vector registers, set the alignment hint to 3. Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2019-02-06 17:06:03 +00:00
.macro VSTM vfrom, vto, disp, base, hint=3
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
VX_NUM v1, \vfrom
VX_NUM v3, \vto
s390/vx: remove comments from macros which break LLVM's IAS LLVM's integrated assembler does not like comments within macros: <instantiation>:3:19: error: too many positional arguments GR_NUM b2, 1 /* Base register */ ^ Remove them, since they are obvious anyway. Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2022-05-03 08:48:41 +00:00
GR_NUM b2, \base
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.word 0xE700 | ((v1&15) << 4) | (v3&15)
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.word (b2 << 12) | (\disp)
s390: add alignment hints to vector load and store The z14 introduced alignment hints to increase the performance of vector loads and stores. The kernel uses an implicit alignmenet of 8 bytes for the vector registers, set the alignment hint to 3. Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2019-02-06 17:06:03 +00:00
MRXBOPC \hint, 0x3E, v1, v3
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.endm
/* VECTOR PERMUTE */
.macro VPERM vr1, vr2, vr3, vr4
VX_NUM v1, \vr1
VX_NUM v2, \vr2
VX_NUM v3, \vr3
VX_NUM v4, \vr4
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.word 0xE700 | ((v1&15) << 4) | (v2&15)
.word ((v3&15) << 12)
MRXBOPC (v4&15), 0x8C, v1, v2, v3, v4
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.endm
/* VECTOR UNPACK LOGICAL LOW */
.macro VUPLL vr1, vr2, m3
VX_NUM v1, \vr1
VX_NUM v2, \vr2
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.word 0xE700 | ((v1&15) << 4) | (v2&15)
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.word 0x0000
MRXBOPC \m3, 0xD4, v1, v2
.endm
.macro VUPLLB vr1, vr2
VUPLL \vr1, \vr2, 0
.endm
.macro VUPLLH vr1, vr2
VUPLL \vr1, \vr2, 1
.endm
.macro VUPLLF vr1, vr2
VUPLL \vr1, \vr2, 2
.endm
s390/crypto: add SIMD implementation for ChaCha20 Add an implementation of the ChaCha20 stream cipher (see e.g. RFC 7539) that makes use of z13's vector instruction set extension. The original implementation is by Andy Polyakov which is adapted for kernel use. Four to six blocks are processed in parallel resulting in a performance gain for inputs >= 256 bytes. chacha20-generic 1 operation in 622 cycles (256 bytes) 1 operation in 2346 cycles (1024 bytes) chacha20-s390 1 operation in 218 cycles (256 bytes) 1 operation in 647 cycles (1024 bytes) Cc: Andy Polyakov <appro@openssl.org> Reviewed-by: Harald Freudenberger <freude@de.ibm.com> Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2021-11-04 14:58:51 +00:00
/* VECTOR PERMUTE DOUBLEWORD IMMEDIATE */
.macro VPDI vr1, vr2, vr3, m4
VX_NUM v1, \vr1
VX_NUM v2, \vr2
VX_NUM v3, \vr3
.word 0xE700 | ((v1&15) << 4) | (v2&15)
.word ((v3&15) << 12)
MRXBOPC \m4, 0x84, v1, v2, v3
.endm
/* VECTOR REPLICATE */
.macro VREP vr1, vr3, imm2, m4
VX_NUM v1, \vr1
VX_NUM v3, \vr3
.word 0xE700 | ((v1&15) << 4) | (v3&15)
.word \imm2
MRXBOPC \m4, 0x4D, v1, v3
.endm
.macro VREPB vr1, vr3, imm2
VREP \vr1, \vr3, \imm2, 0
.endm
.macro VREPH vr1, vr3, imm2
VREP \vr1, \vr3, \imm2, 1
.endm
.macro VREPF vr1, vr3, imm2
VREP \vr1, \vr3, \imm2, 2
.endm
.macro VREPG vr1, vr3, imm2
VREP \vr1, \vr3, \imm2, 3
.endm
/* VECTOR MERGE HIGH */
.macro VMRH vr1, vr2, vr3, m4
VX_NUM v1, \vr1
VX_NUM v2, \vr2
VX_NUM v3, \vr3
.word 0xE700 | ((v1&15) << 4) | (v2&15)
.word ((v3&15) << 12)
MRXBOPC \m4, 0x61, v1, v2, v3
.endm
.macro VMRHB vr1, vr2, vr3
VMRH \vr1, \vr2, \vr3, 0
.endm
.macro VMRHH vr1, vr2, vr3
VMRH \vr1, \vr2, \vr3, 1
.endm
.macro VMRHF vr1, vr2, vr3
VMRH \vr1, \vr2, \vr3, 2
.endm
.macro VMRHG vr1, vr2, vr3
VMRH \vr1, \vr2, \vr3, 3
.endm
/* VECTOR MERGE LOW */
.macro VMRL vr1, vr2, vr3, m4
VX_NUM v1, \vr1
VX_NUM v2, \vr2
VX_NUM v3, \vr3
.word 0xE700 | ((v1&15) << 4) | (v2&15)
.word ((v3&15) << 12)
MRXBOPC \m4, 0x60, v1, v2, v3
.endm
.macro VMRLB vr1, vr2, vr3
VMRL \vr1, \vr2, \vr3, 0
.endm
.macro VMRLH vr1, vr2, vr3
VMRL \vr1, \vr2, \vr3, 1
.endm
.macro VMRLF vr1, vr2, vr3
VMRL \vr1, \vr2, \vr3, 2
.endm
.macro VMRLG vr1, vr2, vr3
VMRL \vr1, \vr2, \vr3, 3
.endm
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
/* Vector integer instructions */
RAID/s390: add SIMD implementation for raid6 gen/xor Using vector registers is slightly faster: raid6: vx128x8 gen() 19705 MB/s raid6: vx128x8 xor() 11886 MB/s raid6: using algorithm vx128x8 gen() 19705 MB/s raid6: .... xor() 11886 MB/s, rmw enabled vs the software algorithms: raid6: int64x1 gen() 3018 MB/s raid6: int64x1 xor() 1429 MB/s raid6: int64x2 gen() 4661 MB/s raid6: int64x2 xor() 3143 MB/s raid6: int64x4 gen() 5392 MB/s raid6: int64x4 xor() 3509 MB/s raid6: int64x8 gen() 4441 MB/s raid6: int64x8 xor() 3207 MB/s raid6: using algorithm int64x4 gen() 5392 MB/s raid6: .... xor() 3509 MB/s, rmw enabled Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-23 11:30:24 +00:00
/* VECTOR AND */
.macro VN vr1, vr2, vr3
VX_NUM v1, \vr1
VX_NUM v2, \vr2
VX_NUM v3, \vr3
.word 0xE700 | ((v1&15) << 4) | (v2&15)
.word ((v3&15) << 12)
MRXBOPC 0, 0x68, v1, v2, v3
.endm
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
/* VECTOR EXCLUSIVE OR */
.macro VX vr1, vr2, vr3
VX_NUM v1, \vr1
VX_NUM v2, \vr2
VX_NUM v3, \vr3
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.word 0xE700 | ((v1&15) << 4) | (v2&15)
.word ((v3&15) << 12)
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
MRXBOPC 0, 0x6D, v1, v2, v3
.endm
/* VECTOR GALOIS FIELD MULTIPLY SUM */
.macro VGFM vr1, vr2, vr3, m4
VX_NUM v1, \vr1
VX_NUM v2, \vr2
VX_NUM v3, \vr3
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.word 0xE700 | ((v1&15) << 4) | (v2&15)
.word ((v3&15) << 12)
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
MRXBOPC \m4, 0xB4, v1, v2, v3
.endm
.macro VGFMB vr1, vr2, vr3
VGFM \vr1, \vr2, \vr3, 0
.endm
.macro VGFMH vr1, vr2, vr3
VGFM \vr1, \vr2, \vr3, 1
.endm
.macro VGFMF vr1, vr2, vr3
VGFM \vr1, \vr2, \vr3, 2
.endm
.macro VGFMG vr1, vr2, vr3
VGFM \vr1, \vr2, \vr3, 3
.endm
/* VECTOR GALOIS FIELD MULTIPLY SUM AND ACCUMULATE */
.macro VGFMA vr1, vr2, vr3, vr4, m5
VX_NUM v1, \vr1
VX_NUM v2, \vr2
VX_NUM v3, \vr3
VX_NUM v4, \vr4
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.word 0xE700 | ((v1&15) << 4) | (v2&15)
.word ((v3&15) << 12) | (\m5 << 8)
MRXBOPC (v4&15), 0xBC, v1, v2, v3, v4
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
.endm
.macro VGFMAB vr1, vr2, vr3, vr4
VGFMA \vr1, \vr2, \vr3, \vr4, 0
.endm
.macro VGFMAH vr1, vr2, vr3, vr4
VGFMA \vr1, \vr2, \vr3, \vr4, 1
.endm
.macro VGFMAF vr1, vr2, vr3, vr4
VGFMA \vr1, \vr2, \vr3, \vr4, 2
.endm
.macro VGFMAG vr1, vr2, vr3, vr4
VGFMA \vr1, \vr2, \vr3, \vr4, 3
.endm
/* VECTOR SHIFT RIGHT LOGICAL BY BYTE */
.macro VSRLB vr1, vr2, vr3
VX_NUM v1, \vr1
VX_NUM v2, \vr2
VX_NUM v3, \vr3
s390/vx: allow to include vx-insn.h with .include To make the vx-insn.h more versatile avoid cpp preprocessor macros and allow to use plain numbers for vector and general purpose register operands. With that you can emit an .include from a C file into the assembler text and then use the vx-insn macros in inline assemblies. For example: asm (".include \"asm/vx-insn.h\""); static inline void xor_vec(int x, int y, int z) { asm volatile("VX %0,%1,%2" : : "i" (x), "i" (y), "i" (z)); } Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-25 08:40:19 +00:00
.word 0xE700 | ((v1&15) << 4) | (v2&15)
.word ((v3&15) << 12)
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
MRXBOPC 0, 0x7D, v1, v2, v3
.endm
RAID/s390: add SIMD implementation for raid6 gen/xor Using vector registers is slightly faster: raid6: vx128x8 gen() 19705 MB/s raid6: vx128x8 xor() 11886 MB/s raid6: using algorithm vx128x8 gen() 19705 MB/s raid6: .... xor() 11886 MB/s, rmw enabled vs the software algorithms: raid6: int64x1 gen() 3018 MB/s raid6: int64x1 xor() 1429 MB/s raid6: int64x2 gen() 4661 MB/s raid6: int64x2 xor() 3143 MB/s raid6: int64x4 gen() 5392 MB/s raid6: int64x4 xor() 3509 MB/s raid6: int64x8 gen() 4441 MB/s raid6: int64x8 xor() 3207 MB/s raid6: using algorithm int64x4 gen() 5392 MB/s raid6: .... xor() 3509 MB/s, rmw enabled Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2016-08-23 11:30:24 +00:00
/* VECTOR REPLICATE IMMEDIATE */
.macro VREPI vr1, imm2, m3
VX_NUM v1, \vr1
.word 0xE700 | ((v1&15) << 4)
.word \imm2
MRXBOPC \m3, 0x45, v1
.endm
.macro VREPIB vr1, imm2
VREPI \vr1, \imm2, 0
.endm
.macro VREPIH vr1, imm2
VREPI \vr1, \imm2, 1
.endm
.macro VREPIF vr1, imm2
VREPI \vr1, \imm2, 2
.endm
.macro VREPIG vr1, imm2
VREP \vr1, \imm2, 3
.endm
/* VECTOR ADD */
.macro VA vr1, vr2, vr3, m4
VX_NUM v1, \vr1
VX_NUM v2, \vr2
VX_NUM v3, \vr3
.word 0xE700 | ((v1&15) << 4) | (v2&15)
.word ((v3&15) << 12)
MRXBOPC \m4, 0xF3, v1, v2, v3
.endm
.macro VAB vr1, vr2, vr3
VA \vr1, \vr2, \vr3, 0
.endm
.macro VAH vr1, vr2, vr3
VA \vr1, \vr2, \vr3, 1
.endm
.macro VAF vr1, vr2, vr3
VA \vr1, \vr2, \vr3, 2
.endm
.macro VAG vr1, vr2, vr3
VA \vr1, \vr2, \vr3, 3
.endm
.macro VAQ vr1, vr2, vr3
VA \vr1, \vr2, \vr3, 4
.endm
/* VECTOR ELEMENT SHIFT RIGHT ARITHMETIC */
.macro VESRAV vr1, vr2, vr3, m4
VX_NUM v1, \vr1
VX_NUM v2, \vr2
VX_NUM v3, \vr3
.word 0xE700 | ((v1&15) << 4) | (v2&15)
.word ((v3&15) << 12)
MRXBOPC \m4, 0x7A, v1, v2, v3
.endm
.macro VESRAVB vr1, vr2, vr3
VESRAV \vr1, \vr2, \vr3, 0
.endm
.macro VESRAVH vr1, vr2, vr3
VESRAV \vr1, \vr2, \vr3, 1
.endm
.macro VESRAVF vr1, vr2, vr3
VESRAV \vr1, \vr2, \vr3, 2
.endm
.macro VESRAVG vr1, vr2, vr3
VESRAV \vr1, \vr2, \vr3, 3
.endm
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
s390/crypto: add SIMD implementation for ChaCha20 Add an implementation of the ChaCha20 stream cipher (see e.g. RFC 7539) that makes use of z13's vector instruction set extension. The original implementation is by Andy Polyakov which is adapted for kernel use. Four to six blocks are processed in parallel resulting in a performance gain for inputs >= 256 bytes. chacha20-generic 1 operation in 622 cycles (256 bytes) 1 operation in 2346 cycles (1024 bytes) chacha20-s390 1 operation in 218 cycles (256 bytes) 1 operation in 647 cycles (1024 bytes) Cc: Andy Polyakov <appro@openssl.org> Reviewed-by: Harald Freudenberger <freude@de.ibm.com> Signed-off-by: Patrick Steuer <patrick.steuer@de.ibm.com> Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
2021-11-04 14:58:51 +00:00
/* VECTOR ELEMENT ROTATE LEFT LOGICAL */
.macro VERLL vr1, vr3, disp, base="%r0", m4
VX_NUM v1, \vr1
VX_NUM v3, \vr3
GR_NUM b2, \base
.word 0xE700 | ((v1&15) << 4) | (v3&15)
.word (b2 << 12) | (\disp)
MRXBOPC \m4, 0x33, v1, v3
.endm
.macro VERLLB vr1, vr3, disp, base="%r0"
VERLL \vr1, \vr3, \disp, \base, 0
.endm
.macro VERLLH vr1, vr3, disp, base="%r0"
VERLL \vr1, \vr3, \disp, \base, 1
.endm
.macro VERLLF vr1, vr3, disp, base="%r0"
VERLL \vr1, \vr3, \disp, \base, 2
.endm
.macro VERLLG vr1, vr3, disp, base="%r0"
VERLL \vr1, \vr3, \disp, \base, 3
.endm
/* VECTOR SHIFT LEFT DOUBLE BY BYTE */
.macro VSLDB vr1, vr2, vr3, imm4
VX_NUM v1, \vr1
VX_NUM v2, \vr2
VX_NUM v3, \vr3
.word 0xE700 | ((v1&15) << 4) | (v2&15)
.word ((v3&15) << 12) | (\imm4)
MRXBOPC 0, 0x77, v1, v2, v3
.endm
s390/vx: add vector instruction support for older binutils versions Older binutils versions do not include support for the vector instruction formats. Add assembler macros for vector instruction mnemonics to easily encode and generate vector instructions. Signed-off-by: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
2015-03-30 15:32:52 +00:00
#endif /* __ASSEMBLY__ */
#endif /* __ASM_S390_VX_INSN_H */
Reference in New Issue Copy Permalink