Commit Graph

24 Commits

Author SHA1 Message Date
Andy Lutomirski
5a83d60c07 x86/fpu: Remove irq_ts_save() and irq_ts_restore()
Now that lazy FPU is gone, we don't use CR0.TS (except possibly in
KVM guest mode).  Remove irq_ts_save(), irq_ts_restore(), and all of
their callers.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: kvm list <kvm@vger.kernel.org>
Link: http://lkml.kernel.org/r/70b9b9e7ba70659bedcb08aba63d0f9214f338f2.1477951965.git.luto@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2016-11-01 07:47:54 +01:00
Borislav Petkov
362f924b64 x86/cpufeature: Remove unused and seldomly used cpu_has_xx macros
Those are stupid and code should use static_cpu_has_safe() or
boot_cpu_has() instead. Kill the least used and unused ones.

The remaining ones need more careful inspection before a conversion can
happen. On the TODO.

Signed-off-by: Borislav Petkov <bp@suse.de>
Link: http://lkml.kernel.org/r/1449481182-27541-4-git-send-email-bp@alien8.de
Cc: David Sterba <dsterba@suse.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Matt Mackall <mpm@selenic.com>
Cc: Chris Mason <clm@fb.com>
Cc: Josef Bacik <jbacik@fb.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
2015-12-19 11:49:55 +01:00
Ingo Molnar
df6b35f409 x86/fpu: Rename i387.h to fpu/api.h
We already have fpu/types.h, move i387.h to fpu/api.h.

The file name has become a misnomer anyway: it offers generic FPU APIs,
but is not limited to i387 functionality.

Reviewed-by: Borislav Petkov <bp@alien8.de>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-05-19 15:47:30 +02:00
Kees Cook
5d26a105b5 crypto: prefix module autoloading with "crypto-"
This prefixes all crypto module loading with "crypto-" so we never run
the risk of exposing module auto-loading to userspace via a crypto API,
as demonstrated by Mathias Krause:

https://lkml.org/lkml/2013/3/4/70

Signed-off-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2014-11-24 22:43:57 +08:00
Marek Vasut
1f4fe5a3fa crypto: padlock - tfm->__crt_alg->cra_name directly
Signed-off-by: Marek Vasut <marex@denx.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2014-05-22 21:03:16 +08:00
Andi Kleen
3bd391f056 crypto: Add support for x86 cpuid auto loading for x86 crypto drivers
Add support for auto-loading of crypto drivers based on cpuid features.
This enables auto-loading of the VIA and Intel specific drivers
for AES, hashing and CRCs.

Requires the earlier infrastructure patch to add x86 modinfo.
I kept it all in a single patch for now.

I dropped the printks when the driver cpuid doesn't match (imho
drivers never should print anything in such a case)

One drawback is that udev doesn't know if the drivers are used or not,
so they will be unconditionally loaded at boot up. That's better
than not loading them at all, like it often happens.

Cc: Dave Jones <davej@redhat.com>
Cc: Kay Sievers <kay.sievers@vrfy.org>
Cc: Jen Axboe <axboe@kernel.dk>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Huang Ying <ying.huang@intel.com>
Signed-off-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Thomas Renninger <trenn@suse.de>
Acked-by: H. Peter Anvin <hpa@zytor.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2012-01-26 16:48:10 -08:00
Brilly Wu
0475add3c2 crypto: padlock - Add SHA-1/256 module for VIA Nano
Add new SHA-1/256 module that never needs any fallback and just calls the
PadLock hardware instruction supported from VIA Nano processors to implement
the "update" and "final" function.

They are respectively named "sha1_alg_nano" and "sha256_alg_nano", and will
be used on any VIA Nano processor or the later ones. On VIA C7 CPU, the
"sha1_alg" and "sha256_alg" modules will still be used as before.

Signed-off-by: Brilly Wu <brillywu@viatech.com.cn>
Signed-off-by: Kary Jin <karyjin@viatech.com.cn>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2011-03-27 10:45:18 +08:00
Herbert Xu
2149308873 crypto: padlock - Move padlock.h into include/crypto
This patch moves padlock.h from drivers/crypto into include/crypto
so that it may be used by the via-rng driver.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2011-01-07 14:52:00 +11:00
Herbert Xu
a8d7ac2797 crypto: padlock-sha - Add import/export support
As the padlock driver for SHA uses a software fallback to perform
partial hashing, it must implement custom import/export functions.
Otherwise hmac which depends on import/export for prehashing will
not work with padlock-sha.

Reported-by: Wolfgang Walter <wolfgang.walter@stwm.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2010-02-02 06:50:25 +11:00
Herbert Xu
4c6ab3ee4c crypto: padlock-sha - Fix stack alignment
The PadLock hardware requires the output buffer for SHA to be
128-bit aligned.  We currentply place the buffer on the stack,
and ask gcc to align it to 128 bits.  That doesn't work on i386
because the kernel stack is only aligned to 32 bits.  This patch
changes the code to align the buffer by hand so that the hardware
doesn't fault on unaligned buffers.

Reported-by: Séguier Régis <rguier@e-teleport.net>
Tested-by: Séguier Régis <rguier@e-teleport.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-09-21 23:21:53 -07:00
Herbert Xu
e9b25f16cd crypto: padlock - Fix hashing of partial blocks
When we encounter partial blocks in finup, we'll invoke the xsha
instruction with a bogus count that is not a multiple of the block
size.  This patch fixes it.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-16 10:33:27 +08:00
Herbert Xu
faae890883 crypto: padlock - Fix compile error on i386
The previous change to allow hashing from states other than the
initial broke compilation on i386 because the inline assembly
tried to squeeze a u64 into a 32-bit register.  As we've already
checked for 32-bit overflows we can simply truncate it to u32,
or unsigned long so that we don't truncate at all on x86-64.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-15 18:37:48 +08:00
Herbert Xu
bbbee4679a crypto: padlock - Switch sha to shash
This patch converts the padlock-sha implementation to shash.
In doing so the existing mechanism of storing the data until
final is no longer viable as we do not have a way of allocating
data in crypto_shash_init and then reliably freeing it.

This is just as well because a better way of handling the problem
is to hash everything but the last chunk using normal sha code
and then provide the intermediate result to the padlock device.

This is good enough because the primary application of padlock-sha
is IPsec and there the data is laid out in the form of an hmac
header followed by the rest of the packet.  In essence we can
provide all the data to the padlock as the hmac header only needs
to be hashed once.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-14 12:58:02 +08:00
Herbert Xu
7d02460826 crypto: padlock - Use shash fallback for sha
This patch changes padlock sha fallback to shash instead of hash.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-07-11 18:23:35 +08:00
Herbert Xu
a760a6656e crypto: api - Fix module load deadlock with fallback algorithms
With the mandatory algorithm testing at registration, we have
now created a deadlock with algorithms requiring fallbacks.
This can happen if the module containing the algorithm requiring
fallback is loaded first, without the fallback module being loaded
first.  The system will then try to test the new algorithm, find
that it needs to load a fallback, and then try to load that.

As both algorithms share the same module alias, it can attempt
to load the original algorithm again and block indefinitely.

As algorithms requiring fallbacks are a special case, we can fix
this by giving them a different module alias than the rest.  Then
it's just a matter of using the right aliases according to what
algorithms we're trying to find.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2009-02-26 14:06:31 +08:00
Suresh Siddha
e49140120c crypto: padlock - fix VIA PadLock instruction usage with irq_ts_save/restore()
Wolfgang Walter reported this oops on his via C3 using padlock for
AES-encryption:

##################################################################

BUG: unable to handle kernel NULL pointer dereference at 000001f0
IP: [<c01028c5>] __switch_to+0x30/0x117
*pde = 00000000
Oops: 0002 [#1] PREEMPT
Modules linked in:

Pid: 2071, comm: sleep Not tainted (2.6.26 #11)
EIP: 0060:[<c01028c5>] EFLAGS: 00010002 CPU: 0
EIP is at __switch_to+0x30/0x117
EAX: 00000000 EBX: c0493300 ECX: dc48dd00 EDX: c0493300
ESI: dc48dd00 EDI: c0493530 EBP: c04cff8c ESP: c04cff7c
 DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
Process sleep (pid: 2071, ti=c04ce000 task=dc48dd00 task.ti=d2fe6000)
Stack: dc48df30 c0493300 00000000 00000000 d2fe7f44 c03b5b43 c04cffc8 00000046
       c0131856 0000005a dc472d3c c0493300 c0493470 d983ae00 00002696 00000000
       c0239f54 00000000 c04c4000 c04cffd8 c01025fe c04f3740 00049800 c04cffe0
Call Trace:
 [<c03b5b43>] ? schedule+0x285/0x2ff
 [<c0131856>] ? pm_qos_requirement+0x3c/0x53
 [<c0239f54>] ? acpi_processor_idle+0x0/0x434
 [<c01025fe>] ? cpu_idle+0x73/0x7f
 [<c03a4dcd>] ? rest_init+0x61/0x63
 =======================

Wolfgang also found out that adding kernel_fpu_begin() and kernel_fpu_end()
around the padlock instructions fix the oops.

Suresh wrote:

These padlock instructions though don't use/touch SSE registers, but it behaves
similar to other SSE instructions. For example, it might cause DNA faults
when cr0.ts is set. While this is a spurious DNA trap, it might cause
oops with the recent fpu code changes.

This is the code sequence  that is probably causing this problem:

a) new app is getting exec'd and it is somewhere in between
   start_thread() and flush_old_exec() in the load_xyz_binary()

b) At pont "a", task's fpu state (like TS_USEDFPU, used_math() etc) is
   cleared.

c) Now we get an interrupt/softirq which starts using these encrypt/decrypt
   routines in the network stack. This generates a math fault (as
   cr0.ts is '1') which sets TS_USEDFPU and restores the math that is
   in the task's xstate.

d) Return to exec code path, which does start_thread() which does
   free_thread_xstate() and sets xstate pointer to NULL while
   the TS_USEDFPU is still set.

e) At the next context switch from the new exec'd task to another task,
   we have a scenarios where TS_USEDFPU is set but xstate pointer is null.
   This can cause an oops during unlazy_fpu() in __switch_to()

Now:

1) This should happen with or with out pre-emption. Viro also encountered
   similar problem with out CONFIG_PREEMPT.

2) kernel_fpu_begin() and kernel_fpu_end() will fix this problem, because
   kernel_fpu_begin() will manually do a clts() and won't run in to the
   situation of setting TS_USEDFPU in step "c" above.

3) This was working before the fpu changes, because its a spurious
   math fault  which doesn't corrupt any fpu/sse registers and the task's
   math state was always in an allocated state.

With out the recent lazy fpu allocation changes, while we don't see oops,
there is a possible race still present in older kernels(for example,
while kernel is using kernel_fpu_begin() in some optimized clear/copy
page and an interrupt/softirq happens which uses these padlock
instructions generating DNA fault).

This is the failing scenario that existed even before the lazy fpu allocation
changes:

0. CPU's TS flag is set

1. kernel using FPU in some optimized copy  routine and while doing
kernel_fpu_begin() takes an interrupt just before doing clts()

2. Takes an interrupt and ipsec uses padlock instruction. And we
take a DNA fault as TS flag is still set.

3. We handle the DNA fault and set TS_USEDFPU and clear cr0.ts

4. We complete the padlock routine

5. Go back to step-1, which resumes clts() in kernel_fpu_begin(), finishes
the optimized copy routine and does kernel_fpu_end(). At this point,
we have cr0.ts again set to '1' but the task's TS_USEFPU is stilll
set and not cleared.

6. Now kernel resumes its user operation. And at the next context
switch, kernel sees it has do a FP save as TS_USEDFPU is still set
and then will do a unlazy_fpu() in __switch_to(). unlazy_fpu()
will take a DNA fault, as cr0.ts is '1' and now, because we are
in __switch_to(), math_state_restore() will get confused and will
restore the next task's FP state and will save it in prev tasks's FP state.
Remember, in __switch_to() we are already on the stack of the next task
but take a DNA fault for the prev task.

This causes the fpu leakage.

Fix the padlock instruction usage by calling them inside the
context of new routines irq_ts_save/restore(), which clear/restore cr0.ts
manually in the interrupt context. This will not generate spurious DNA
in the  context of the interrupt which will fix the oops encountered and
the possible FPU leakage issue.

Reported-and-bisected-by: Wolfgang Walter <wolfgang.walter@stwm.de>
Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-08-13 22:02:26 +10:00
Jeremy Katz
b43e726b32 crypto: padlock - Make module loading quieter when hardware isn't available
When loading aes or sha256 via the module aliases, the padlock modules
also try to get loaded.  Make the error message for them not being
present only be a NOTICE rather than an ERROR so that use of 'quiet'
will suppress the messages

Signed-off-by: Jeremy Katz <katzj@redhat.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2008-07-10 20:35:16 +08:00
Herbert Xu
68e3f5dd4d [CRYPTO] users: Fix up scatterlist conversion errors
This patch fixes the errors made in the users of the crypto layer during
the sg_init_table conversion.  It also adds a few conversions that were
missing altogether.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-27 00:52:07 -07:00
Jan Glauber
5265eeb2b0 [CRYPTO] sha: Add header file for SHA definitions
There are currently several SHA implementations that all define their own
initialization vectors and size values. Since this values are idential
move them to a header file under include/crypto.

Signed-off-by: Jan Glauber <jang@de.ibm.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-10-10 16:55:50 -07:00
Sebastian Siewior
ad5d27899f [CRYPTO] sha: Load the SHA[1|256] module by an alias
Loading the crypto algorithm by the alias instead of by module directly
has the advantage that all possible implementations of this algorithm
are loaded automatically and the crypto API can choose the best one
depending on its priority.

Additionally it ensures that the generic implementation as well as the
HW driver (if available) is loaded in case the HW driver needs the
generic version as fallback in corner cases.

Also remove the probe for sha1 in padlock's init code.

Quote from Herbert:
  The probe is actually pointless since we can always probe when
  the algorithm is actually used which does not lead to dead-locks
  like this.

Signed-off-by: Sebastian Siewior <sebastian@breakpoint.cc>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2007-10-10 16:55:50 -07:00
Herbert Xu
6010439f47 [CRYPTO] padlock: Convert padlock-sha to use crypto_hash
This patch converts padlock-sha to use crypto_hash for its fallback.
It also changes the fallback selection to use selection by type instead
of name.  This is done through the new CRYPTO_ALG_NEED_FALLBACK bit,
which is set if and only if an algorithm needs a fallback of the same
type.

Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2006-09-21 11:46:22 +10:00
Michal Ludvig
58ec415289 [CRYPTO] padlock-sha: TFMs don't need to be static
TFMs are local variables. No need to declare them 
static. After all one is enough.

Signed-off-by: Michal Ludvig <michal@logix.cz>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2006-09-21 11:41:00 +10:00
Adrian Bunk
cb17530b0a [CRYPTO] padlock-sha: Make 2 functions static
This patch makes two needlessly global functions static.

Signed-off-by: Adrian Bunk <bunk@stusta.de>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2006-09-21 11:41:00 +10:00
Michal Ludvig
6c83327515 [CRYPTO] padlock: Driver for SHA1 / SHA256 algorithms
Support for SHA1 / SHA256 algorithms in VIA C7 processors.

Signed-off-by: Michal Ludvig <michal@logix.cz>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2006-09-21 11:40:59 +10:00