[RFC] padlock aes, unification of setkey()

From Sebastian Siewior [off-list ref] # This line is ignored.

Subject: [RFC] padlock aes, unification of setkey()

Hello Herbert,

I sit on those two since November. Back then Michal dropped me an email
and told me that he will test it and get back to me. This didn't happen
so far.
The binary format of the key was the same, the last time I checked, so
the second patch could really work :)

One thing I'm concerned about is the stack utilization. The initial
version had a structure with 256 bytes on the stack. Mine has a bigger
structure with 484 bytes. I'm not sure if it is better to dynamically
allocate it, move it to the private key structure or pad the generic
aes structure in order to enforce the required alignment.

Sebastian

The key expansion routine could be get little more generic, become
a kernel doc entry and then get exported.

Signed-off-by: Sebastian Siewior <redacted>
---
 crypto/aes_generic.c |   56 +++++++++++++++++++++++++++++++++++++++++--------
 include/crypto/aes.h |    8 +++++-
 2 files changed, 53 insertions(+), 11 deletions(-)

diff --git a/crypto/aes_generic.c b/crypto/aes_generic.c
index f33a99c..9322531 100644
--- a/crypto/aes_generic.c
+++ b/crypto/aes_generic.c

@@ -229,18 +229,29 @@ static void __init gen_tabs(void)
 	ctx->key_enc[8 * i + 15] = t;			\
 } while (0)
 
-int crypto_aes_set_key(struct crypto_tfm *tfm, const u8 *in_key,
+/**
+ * crypto_aes_expand_key - Expands the AES key as described in FIPS-197
+ * @ctx:	The location where the computed key will be stored.
+ * @in_key:	The supplied key.
+ * @key_len:	The length of the supplied key.
+ *
+ * Returns 0 on success. The function fails only if an invalid key size (or
+ * pointer) is supplied.
+ * The expanded key size is 240 bytes (max of 14 rounds with a unique 16 bytes
+ * key schedule plus a 16 bytes key which is used before the first round).
+ * The decryption key is prepared for the "Equivalent Inverse Cipher" as
+ * described in FIPS-197. The first slot (16 bytes) of each key (enc or dec) is
+ * for the initial combination, the second slot for the first round and so on.
+ */
+int crypto_aes_expand_key(struct crypto_aes_ctx *ctx, const u8 *in_key,
 		unsigned int key_len)
 {
-	struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm);
 	const __le32 *key = (const __le32 *)in_key;
-	u32 *flags = &tfm->crt_flags;
 	u32 i, t, u, v, w, j;
 
-	if (key_len % 8) {
-		*flags |= CRYPTO_TFM_RES_BAD_KEY_LEN;
+	if (key_len != AES_KEYSIZE_128 && key_len != AES_KEYSIZE_192 &&
+			key_len != AES_KEYSIZE_256)
 		return -EINVAL;
-	}
 
 	ctx->key_length = key_len;
 

@@ -250,20 +261,20 @@ int crypto_aes_set_key(struct crypto_tfm *tfm, const u8 *in_key,
 	ctx->key_dec[key_len + 27] = ctx->key_enc[3] = le32_to_cpu(key[3]);
 
 	switch (key_len) {
-	case 16:
+	case AES_KEYSIZE_128:
 		t = ctx->key_enc[3];
 		for (i = 0; i < 10; ++i)
 			loop4(i);
 		break;
 
-	case 24:
+	case AES_KEYSIZE_192:
 		ctx->key_enc[4] = le32_to_cpu(key[4]);
 		t = ctx->key_enc[5] = le32_to_cpu(key[5]);
 		for (i = 0; i < 8; ++i)
 			loop6(i);
 		break;
 
-	case 32:
+	case AES_KEYSIZE_256:
 		ctx->key_enc[4] = le32_to_cpu(key[4]);
 		ctx->key_enc[5] = le32_to_cpu(key[5]);
 		ctx->key_enc[6] = le32_to_cpu(key[6]);

@@ -284,6 +295,33 @@ int crypto_aes_set_key(struct crypto_tfm *tfm, const u8 *in_key,
 	}
 	return 0;
 }
+EXPORT_SYMBOL_GPL(crypto_aes_expand_key);
+
+/**
+ * crypto_aes_set_key - Set the AES key.
+ * @tfm:	The %crypto_tfm that is used in the context.
+ * @in_key:	The input key.
+ * @key_len:	The size of the key.
+ *
+ * Returns 0 on success, on failure the %CRYPTO_TFM_RES_BAD_KEY_LEN flag in tfm
+ * is set. The function uses crypto_aes_expand_key() to expand the key.
+ * &crypto_aes_ctx _must_ be the private data embedded in @tfm which is
+ * retrieved with crypto_tfm_ctx().
+ */
+int crypto_aes_set_key(struct crypto_tfm *tfm, const u8 *in_key,
+		unsigned int key_len)
+{
+	struct crypto_aes_ctx *ctx = crypto_tfm_ctx(tfm);
+	u32 *flags = &tfm->crt_flags;
+	int ret;
+
+	ret = crypto_aes_expand_key(ctx, in_key, key_len);
+	if (!ret)
+		return 0;
+
+	*flags |= CRYPTO_TFM_RES_BAD_KEY_LEN;
+	return -EINVAL;
+}
 EXPORT_SYMBOL_GPL(crypto_aes_set_key);
 
 /* encrypt a block of text */

diff --git a/include/crypto/aes.h b/include/crypto/aes.h
index d480b76..40008d6 100644
--- a/include/crypto/aes.h
+++ b/include/crypto/aes.h

@@ -14,11 +14,13 @@
 #define AES_KEYSIZE_192		24
 #define AES_KEYSIZE_256		32
 #define AES_BLOCK_SIZE		16
+#define AES_MAX_KEYLENGTH	(15 * 16)
+#define AES_MAX_KEYLENGTH_U32	(AES_MAX_KEYLENGTH / sizeof(u32))
 
 struct crypto_aes_ctx {
 	u32 key_length;
-	u32 key_enc[60];
-	u32 key_dec[60];
+	u32 key_enc[AES_MAX_KEYLENGTH_U32];
+	u32 key_dec[AES_MAX_KEYLENGTH_U32];
 };
 
 extern u32 crypto_ft_tab[4][256];

@@ -28,4 +30,6 @@ extern u32 crypto_il_tab[4][256];
 
 int crypto_aes_set_key(struct crypto_tfm *tfm, const u8 *in_key,
 		unsigned int key_len);
+int crypto_aes_expand_key(struct crypto_aes_ctx *ctx, const u8 *in_key,
+		unsigned int key_len);
 #endif

-- 
1.5.3.7

Padlock AES' setkey routine is the same as exported by the generic
implementation. So we could use it.

Cc: Michal Ludvig <redacted>
Signed-off-by: Sebastian Siewior <redacted>
---
 drivers/crypto/Kconfig       |    1 +
 drivers/crypto/padlock-aes.c |  320 +++---------------------------------------
 2 files changed, 20 insertions(+), 301 deletions(-)

diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig
index 6b658d8..5647146 100644
--- a/drivers/crypto/Kconfig
+++ b/drivers/crypto/Kconfig

@@ -27,6 +27,7 @@ config CRYPTO_DEV_PADLOCK_AES
 	tristate "PadLock driver for AES algorithm"
 	depends on CRYPTO_DEV_PADLOCK
 	select CRYPTO_BLKCIPHER
+	select CRYPTO_AES
 	help
 	  Use VIA PadLock for AES algorithm.
 

diff --git a/drivers/crypto/padlock-aes.c b/drivers/crypto/padlock-aes.c
index 08fc240..36ec298 100644
--- a/drivers/crypto/padlock-aes.c
+++ b/drivers/crypto/padlock-aes.c

@@ -5,42 +5,6 @@
  *
  * Copyright (c) 2004  Michal Ludvig <michal@logix.cz>
  *
- * Key expansion routine taken from crypto/aes_generic.c
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * ---------------------------------------------------------------------------
- * Copyright (c) 2002, Dr Brian Gladman <brg@gladman.me.uk>, Worcester, UK.
- * All rights reserved.
- *
- * LICENSE TERMS
- *
- * The free distribution and use of this software in both source and binary
- * form is allowed (with or without changes) provided that:
- *
- *   1. distributions of this source code include the above copyright
- *      notice, this list of conditions and the following disclaimer;
- *
- *   2. distributions in binary form include the above copyright
- *      notice, this list of conditions and the following disclaimer
- *      in the documentation and/or other associated materials;
- *
- *   3. the copyright holder's name is not used to endorse products
- *      built using this software without specific written permission.
- *
- * ALTERNATIVELY, provided that this notice is retained in full, this product
- * may be distributed under the terms of the GNU General Public License (GPL),
- * in which case the provisions of the GPL apply INSTEAD OF those given above.
- *
- * DISCLAIMER
- *
- * This software is provided 'as is' with no explicit or implied warranties
- * in respect of its properties, including, but not limited to, correctness
- * and/or fitness for purpose.
- * ---------------------------------------------------------------------------
  */
 
 #include <crypto/algapi.h>

@@ -54,9 +18,6 @@
 #include <asm/byteorder.h>
 #include "padlock.h"
 
-#define AES_EXTENDED_KEY_SIZE	64	/* in uint32_t units */
-#define AES_EXTENDED_KEY_SIZE_B	(AES_EXTENDED_KEY_SIZE * sizeof(uint32_t))
-
 /* Control word. */
 struct cword {
 	unsigned int __attribute__ ((__packed__))

@@ -70,218 +31,23 @@ struct cword {
 
 /* Whenever making any changes to the following
  * structure *make sure* you keep E, d_data
- * and cword aligned on 16 Bytes boundaries!!! */
+ * and cword aligned on 16 Bytes boundaries and
+ * the Hardware can access 16 * 16 bytes of E and d_data
+ * (only the first 15 * 16 bytes matter but the HW reads
+ * more).
+ */
 struct aes_ctx {
+	u32 E[AES_MAX_KEYLENGTH_U32]
+		__attribute__ ((__aligned__(PADLOCK_ALIGNMENT)));
+	u32 d_data[AES_MAX_KEYLENGTH_U32]
+		__attribute__ ((__aligned__(PADLOCK_ALIGNMENT)));
 	struct {
 		struct cword encrypt;
 		struct cword decrypt;
 	} cword;
 	u32 *D;
-	int key_length;
-	u32 E[AES_EXTENDED_KEY_SIZE]
-		__attribute__ ((__aligned__(PADLOCK_ALIGNMENT)));
-	u32 d_data[AES_EXTENDED_KEY_SIZE]
-		__attribute__ ((__aligned__(PADLOCK_ALIGNMENT)));
 };
 
-/* ====== Key management routines ====== */
-
-static inline uint32_t
-generic_rotr32 (const uint32_t x, const unsigned bits)
-{
-	const unsigned n = bits % 32;
-	return (x >> n) | (x << (32 - n));
-}
-
-static inline uint32_t
-generic_rotl32 (const uint32_t x, const unsigned bits)
-{
-	const unsigned n = bits % 32;
-	return (x << n) | (x >> (32 - n));
-}
-
-#define rotl generic_rotl32
-#define rotr generic_rotr32
-
-/*
- * #define byte(x, nr) ((unsigned char)((x) >> (nr*8))) 
- */
-static inline uint8_t
-byte(const uint32_t x, const unsigned n)
-{
-	return x >> (n << 3);
-}
-
-#define E_KEY ctx->E
-#define D_KEY ctx->D
-
-static uint8_t pow_tab[256];
-static uint8_t log_tab[256];
-static uint8_t sbx_tab[256];
-static uint8_t isb_tab[256];
-static uint32_t rco_tab[10];
-static uint32_t ft_tab[4][256];
-static uint32_t it_tab[4][256];
-
-static uint32_t fl_tab[4][256];
-static uint32_t il_tab[4][256];
-
-static inline uint8_t
-f_mult (uint8_t a, uint8_t b)
-{
-	uint8_t aa = log_tab[a], cc = aa + log_tab[b];
-
-	return pow_tab[cc + (cc < aa ? 1 : 0)];
-}
-
-#define ff_mult(a,b)    (a && b ? f_mult(a, b) : 0)
-
-#define f_rn(bo, bi, n, k)					\
-    bo[n] =  ft_tab[0][byte(bi[n],0)] ^				\
-             ft_tab[1][byte(bi[(n + 1) & 3],1)] ^		\
-             ft_tab[2][byte(bi[(n + 2) & 3],2)] ^		\
-             ft_tab[3][byte(bi[(n + 3) & 3],3)] ^ *(k + n)
-
-#define i_rn(bo, bi, n, k)					\
-    bo[n] =  it_tab[0][byte(bi[n],0)] ^				\
-             it_tab[1][byte(bi[(n + 3) & 3],1)] ^		\
-             it_tab[2][byte(bi[(n + 2) & 3],2)] ^		\
-             it_tab[3][byte(bi[(n + 1) & 3],3)] ^ *(k + n)
-
-#define ls_box(x)				\
-    ( fl_tab[0][byte(x, 0)] ^			\
-      fl_tab[1][byte(x, 1)] ^			\
-      fl_tab[2][byte(x, 2)] ^			\
-      fl_tab[3][byte(x, 3)] )
-
-#define f_rl(bo, bi, n, k)					\
-    bo[n] =  fl_tab[0][byte(bi[n],0)] ^				\
-             fl_tab[1][byte(bi[(n + 1) & 3],1)] ^		\
-             fl_tab[2][byte(bi[(n + 2) & 3],2)] ^		\
-             fl_tab[3][byte(bi[(n + 3) & 3],3)] ^ *(k + n)
-
-#define i_rl(bo, bi, n, k)					\
-    bo[n] =  il_tab[0][byte(bi[n],0)] ^				\
-             il_tab[1][byte(bi[(n + 3) & 3],1)] ^		\
-             il_tab[2][byte(bi[(n + 2) & 3],2)] ^		\
-             il_tab[3][byte(bi[(n + 1) & 3],3)] ^ *(k + n)
-
-static void
-gen_tabs (void)
-{
-	uint32_t i, t;
-	uint8_t p, q;
-
-	/* log and power tables for GF(2**8) finite field with
-	   0x011b as modular polynomial - the simplest prmitive
-	   root is 0x03, used here to generate the tables */
-
-	for (i = 0, p = 1; i < 256; ++i) {
-		pow_tab[i] = (uint8_t) p;
-		log_tab[p] = (uint8_t) i;
-
-		p ^= (p << 1) ^ (p & 0x80 ? 0x01b : 0);
-	}
-
-	log_tab[1] = 0;
-
-	for (i = 0, p = 1; i < 10; ++i) {
-		rco_tab[i] = p;
-
-		p = (p << 1) ^ (p & 0x80 ? 0x01b : 0);
-	}
-
-	for (i = 0; i < 256; ++i) {
-		p = (i ? pow_tab[255 - log_tab[i]] : 0);
-		q = ((p >> 7) | (p << 1)) ^ ((p >> 6) | (p << 2));
-		p ^= 0x63 ^ q ^ ((q >> 6) | (q << 2));
-		sbx_tab[i] = p;
-		isb_tab[p] = (uint8_t) i;
-	}
-
-	for (i = 0; i < 256; ++i) {
-		p = sbx_tab[i];
-
-		t = p;
-		fl_tab[0][i] = t;
-		fl_tab[1][i] = rotl (t, 8);
-		fl_tab[2][i] = rotl (t, 16);
-		fl_tab[3][i] = rotl (t, 24);
-
-		t = ((uint32_t) ff_mult (2, p)) |
-		    ((uint32_t) p << 8) |
-		    ((uint32_t) p << 16) | ((uint32_t) ff_mult (3, p) << 24);
-
-		ft_tab[0][i] = t;
-		ft_tab[1][i] = rotl (t, 8);
-		ft_tab[2][i] = rotl (t, 16);
-		ft_tab[3][i] = rotl (t, 24);
-
-		p = isb_tab[i];
-
-		t = p;
-		il_tab[0][i] = t;
-		il_tab[1][i] = rotl (t, 8);
-		il_tab[2][i] = rotl (t, 16);
-		il_tab[3][i] = rotl (t, 24);
-
-		t = ((uint32_t) ff_mult (14, p)) |
-		    ((uint32_t) ff_mult (9, p) << 8) |
-		    ((uint32_t) ff_mult (13, p) << 16) |
-		    ((uint32_t) ff_mult (11, p) << 24);
-
-		it_tab[0][i] = t;
-		it_tab[1][i] = rotl (t, 8);
-		it_tab[2][i] = rotl (t, 16);
-		it_tab[3][i] = rotl (t, 24);
-	}
-}
-
-#define star_x(x) (((x) & 0x7f7f7f7f) << 1) ^ ((((x) & 0x80808080) >> 7) * 0x1b)
-
-#define imix_col(y,x)       \
-    u   = star_x(x);        \
-    v   = star_x(u);        \
-    w   = star_x(v);        \
-    t   = w ^ (x);          \
-   (y)  = u ^ v ^ w;        \
-   (y) ^= rotr(u ^ t,  8) ^ \
-          rotr(v ^ t, 16) ^ \
-          rotr(t,24)
-
-/* initialise the key schedule from the user supplied key */
-
-#define loop4(i)                                    \
-{   t = rotr(t,  8); t = ls_box(t) ^ rco_tab[i];    \
-    t ^= E_KEY[4 * i];     E_KEY[4 * i + 4] = t;    \
-    t ^= E_KEY[4 * i + 1]; E_KEY[4 * i + 5] = t;    \
-    t ^= E_KEY[4 * i + 2]; E_KEY[4 * i + 6] = t;    \
-    t ^= E_KEY[4 * i + 3]; E_KEY[4 * i + 7] = t;    \
-}
-
-#define loop6(i)                                    \
-{   t = rotr(t,  8); t = ls_box(t) ^ rco_tab[i];    \
-    t ^= E_KEY[6 * i];     E_KEY[6 * i + 6] = t;    \
-    t ^= E_KEY[6 * i + 1]; E_KEY[6 * i + 7] = t;    \
-    t ^= E_KEY[6 * i + 2]; E_KEY[6 * i + 8] = t;    \
-    t ^= E_KEY[6 * i + 3]; E_KEY[6 * i + 9] = t;    \
-    t ^= E_KEY[6 * i + 4]; E_KEY[6 * i + 10] = t;   \
-    t ^= E_KEY[6 * i + 5]; E_KEY[6 * i + 11] = t;   \
-}
-
-#define loop8(i)                                    \
-{   t = rotr(t,  8); ; t = ls_box(t) ^ rco_tab[i];  \
-    t ^= E_KEY[8 * i];     E_KEY[8 * i + 8] = t;    \
-    t ^= E_KEY[8 * i + 1]; E_KEY[8 * i + 9] = t;    \
-    t ^= E_KEY[8 * i + 2]; E_KEY[8 * i + 10] = t;   \
-    t ^= E_KEY[8 * i + 3]; E_KEY[8 * i + 11] = t;   \
-    t  = E_KEY[8 * i + 4] ^ ls_box(t);    \
-    E_KEY[8 * i + 12] = t;                \
-    t ^= E_KEY[8 * i + 5]; E_KEY[8 * i + 13] = t;   \
-    t ^= E_KEY[8 * i + 6]; E_KEY[8 * i + 14] = t;   \
-    t ^= E_KEY[8 * i + 7]; E_KEY[8 * i + 15] = t;   \
-}
-
 /* Tells whether the ACE is capable to generate
    the extended key for a given key_len. */
 static inline int

@@ -321,17 +87,13 @@ static int aes_set_key(struct crypto_tfm *tfm, const u8 *in_key,
 	struct aes_ctx *ctx = aes_ctx(tfm);
 	const __le32 *key = (const __le32 *)in_key;
 	u32 *flags = &tfm->crt_flags;
-	uint32_t i, t, u, v, w;
-	uint32_t P[AES_EXTENDED_KEY_SIZE];
-	uint32_t rounds;
+	struct crypto_aes_ctx gen_aes;
 
 	if (key_len % 8) {
 		*flags |= CRYPTO_TFM_RES_BAD_KEY_LEN;
 		return -EINVAL;
 	}
 
-	ctx->key_length = key_len;
-
 	/*
 	 * If the hardware is capable of generating the extended key
 	 * itself we must supply the plain key for both encryption

@@ -339,10 +101,10 @@ static int aes_set_key(struct crypto_tfm *tfm, const u8 *in_key,
 	 */
 	ctx->D = ctx->E;
 
-	E_KEY[0] = le32_to_cpu(key[0]);
-	E_KEY[1] = le32_to_cpu(key[1]);
-	E_KEY[2] = le32_to_cpu(key[2]);
-	E_KEY[3] = le32_to_cpu(key[3]);
+	ctx->E[0] = le32_to_cpu(key[0]);
+	ctx->E[1] = le32_to_cpu(key[1]);
+	ctx->E[2] = le32_to_cpu(key[2]);
+	ctx->E[3] = le32_to_cpu(key[3]);
 
 	/* Prepare control words. */
 	memset(&ctx->cword, 0, sizeof(ctx->cword));

@@ -361,56 +123,13 @@ static int aes_set_key(struct crypto_tfm *tfm, const u8 *in_key,
 	ctx->cword.encrypt.keygen = 1;
 	ctx->cword.decrypt.keygen = 1;
 
-	switch (key_len) {
-	case 16:
-		t = E_KEY[3];
-		for (i = 0; i < 10; ++i)
-			loop4 (i);
-		break;
-
-	case 24:
-		E_KEY[4] = le32_to_cpu(key[4]);
-		t = E_KEY[5] = le32_to_cpu(key[5]);
-		for (i = 0; i < 8; ++i)
-			loop6 (i);
-		break;
-
-	case 32:
-		E_KEY[4] = le32_to_cpu(key[4]);
-		E_KEY[5] = le32_to_cpu(key[5]);
-		E_KEY[6] = le32_to_cpu(key[6]);
-		t = E_KEY[7] = le32_to_cpu(key[7]);
-		for (i = 0; i < 7; ++i)
-			loop8 (i);
-		break;
-	}
-
-	D_KEY[0] = E_KEY[0];
-	D_KEY[1] = E_KEY[1];
-	D_KEY[2] = E_KEY[2];
-	D_KEY[3] = E_KEY[3];
-
-	for (i = 4; i < key_len + 24; ++i) {
-		imix_col (D_KEY[i], E_KEY[i]);
-	}
-
-	/* PadLock needs a different format of the decryption key. */
-	rounds = 10 + (key_len - 16) / 4;
-
-	for (i = 0; i < rounds; i++) {
-		P[((i + 1) * 4) + 0] = D_KEY[((rounds - i - 1) * 4) + 0];
-		P[((i + 1) * 4) + 1] = D_KEY[((rounds - i - 1) * 4) + 1];
-		P[((i + 1) * 4) + 2] = D_KEY[((rounds - i - 1) * 4) + 2];
-		P[((i + 1) * 4) + 3] = D_KEY[((rounds - i - 1) * 4) + 3];
+	if (crypto_aes_expand_key(&gen_aes, in_key, key_len)) {
+		*flags |= CRYPTO_TFM_RES_BAD_KEY_LEN;
+		return -EINVAL;
 	}
 
-	P[0] = E_KEY[(rounds * 4) + 0];
-	P[1] = E_KEY[(rounds * 4) + 1];
-	P[2] = E_KEY[(rounds * 4) + 2];
-	P[3] = E_KEY[(rounds * 4) + 3];
-
-	memcpy(D_KEY, P, AES_EXTENDED_KEY_SIZE_B);
-
+	memcpy(ctx->E, gen_aes.key_enc, AES_MAX_KEYLENGTH);
+	memcpy(ctx->D, gen_aes.key_dec, AES_MAX_KEYLENGTH);
 	return 0;
 }
 

@@ -677,7 +396,6 @@ static int __init padlock_init(void)
 		return -ENODEV;
 	}
 
-	gen_tabs();
 	if ((ret = crypto_register_alg(&aes_alg)))
 		goto aes_err;
 

-- 
1.5.3.7

Hello,

I'm got my Via Epia SN Board a few days ago and could test everything related to the
padlock engine, I'm especially interested in the aes-{lrw,xts} combo, this doesn't work at
the moment (last tested with 2.6.25-rc1).

Cheers,
Stefan

Sebastian Siewior schrieb:

From Sebastian Siewior [off-list ref] # This line is ignored.
Subject: [RFC] padlock aes, unification of setkey()

Hello Herbert,

I sit on those two since November. Back then Michal dropped me an email
and told me that he will test it and get back to me. This didn't happen
so far.
The binary format of the key was the same, the last time I checked, so
the second patch could really work :)

One thing I'm concerned about is the stack utilization. The initial
version had a structure with 256 bytes on the stack. Mine has a bigger
structure with 484 bytes. I'm not sure if it is better to dynamically
allocate it, move it to the private key structure or pad the generic
aes structure in order to enforce the required alignment.

Sebastian


-
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

* Stefan Hellermann | 2008-02-24 12:54:20 [+0100]:

Hello,

Hello,

I'm got my Via Epia SN Board a few days ago and could test everything related to the
padlock engine, I'm especially interested in the aes-{lrw,xts} combo, this doesn't work at

Cool,

the moment (last tested with 2.6.25-rc1).

Could you be a little more specific on "it doesn't work"?
Do you pass the tcrypt test at least?
Does it* work without the HW acceleration?

Cheers,
Stefan

*: it means a dm-crypt encrypted partition I guess.

Sebastian

okay, I pulled current linus git and build it with attached config. Then I tried the
following:

# rmmod padlock-aes
# cryptsetup -c aes-xts-plain -s 256 luksFormat /dev/sda1
.... works
# modprobe padlock-aes
padlock: Using VIA PadLock ACE for AES algorithm.
# cryptsetup -c aes-xts-plain -s 256 luksFormat /dev/sda1

general protection fault: 0000 [#1]
Modules linked in: padlock_aes xts gf128mul cifs [last unloaded: padlock_aes]

Pid: 988, comm: kcryptd Not tainted (2.6.25-rc2-via #121)
EIP: 0060:[<f881d801>] EFLAGS: 00010206 CPU: 0
EIP is at aes_encrypt+0x31/0x60 [padlock_aes]
EAX: f7468af0 EBX: f7616860 ECX: 00000001 EDX: f7616830
ESI: f7468500 EDI: f762de88 EBP: f762de88 ESP: f762de64
 DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
Process kcryptd (pid: 988, ti=f762c000 task=f746eff0 task.ti=f762c000)
Stack: f75e3770 fffb6000 fffb7e00 00000200 f88280a3 f75e3770 f762debc f762df04
       00000010 00000000 00000000 00000000 00000000 f7616400 f881d7d0 f75e3770
       f75c0600 f7468500 c048ab44 f8828272 f881d7d0 f881d7d0 c2b4bf20 fffb7e00
Call Trace:
 [<f88280a3>] crypt+0x83/0x110 [xts]
 [<f881d7d0>] aes_encrypt+0x0/0x60 [padlock_aes]
 [<f8828272>] encrypt+0x42/0x50 [xts]
 [<f881d7d0>] aes_encrypt+0x0/0x60 [padlock_aes]
 [<f881d7d0>] aes_encrypt+0x0/0x60 [padlock_aes]
 [<c021b05b>] async_encrypt+0x3b/0x50
 [<c02fcce9>] crypt_convert+0x1b9/0x270
 [<c02fcf4d>] kcryptd_crypt+0x1ad/0x220
 [<c02fcda0>] kcryptd_crypt+0x0/0x220
 [<c012ae6b>] run_workqueue+0xab/0x140
 [<c012b5e0>] worker_thread+0x0/0x90
 [<c012b639>] worker_thread+0x59/0x90
 [<c012e100>] autoremove_wake_function+0x0/0x40
 [<c012b5e0>] worker_thread+0x0/0x90
 [<c012dd22>] kthread+0x42/0x70
 [<c012dce0>] kthread+0x0/0x70
 [<c010437b>] kernel_thread_helper+0x7/0x1c
 =======================
Code: 0c 89 d7 8d 50 3f 89 74 24 08 83 e2 f0 89 ce 89 5c 24 04 9c 9d 89 c8 35 f0 0f 00 00
a9 ff 0f 00 00 8d 5a 30 74 19 b9 01 00 00 00 <f3> 0f a7 c8 8b 5c 24 04 8b 74 24 08 8b 7c
24 0c 83 c4 10 c3 89
EIP: [<f881d801>] aes_encrypt+0x31/0x60 [padlock_aes] SS:ESP 0068:f762de64
---[ end trace 526de21aa54fb137 ]---
note: kcryptd[988] exited with preempt_count 2
------------[ cut here ]------------
kernel BUG at arch/x86/mm/highmem_32.c:87!
invalid opcode: 0000 [#2]
Modules linked in: padlock_aes xts gf128mul cifs [last unloaded: padlock_aes]

Pid: 990, comm: udevd Tainted: G      D  (2.6.25-rc2-via #121)
EIP: 0060:[<c01177bb>] EFLAGS: 00010286 CPU: 0
EIP is at kmap_atomic_prot+0xbb/0xd0
EAX: da5f9163 EBX: c2b4bae0 ECX: 00000163 EDX: 00000003
ESI: fffff000 EDI: 0000000c EBP: 00000003 ESP: f76a1dec
 DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
Process udevd (pid: 990, ti=f76a0000 task=f7608000 task.ti=f76a0000)
Stack: f76a1e58 f7530c80 00000080 f76a1e7c 00000080 c0140dfb 00000000 c2b4bae0
       f76a1e78 00000080 00000000 c2b4bae0 00000000 f799a3f8 c014305f 00001000
       c0173a73 f799a360 00000000 00000000 f75e83c4 f76a1ef4 f76a1eac f75e8380
Call Trace:
 [<c0140dfb>] file_read_actor+0xcb/0x100
 [<c014305f>] generic_file_aio_read+0x2df/0x590
 [<c0173a73>] dput+0x13/0xb0
 [<c0177d8b>] mntput_no_expire+0x1b/0x70
 [<c0163ce5>] do_sync_read+0xd5/0x120
 [<c012e100>] autoremove_wake_function+0x0/0x40
 [<c0162240>] __dentry_open+0x160/0x1b0
 [<c0214b8c>] security_file_permission+0xc/0x10
 [<c0163d8e>] rw_verify_area+0x5e/0xd0
 [<c0163c10>] do_sync_read+0x0/0x120
 [<c016459d>] vfs_read+0x9d/0x140
 [<c0152653>] insert_vm_struct+0x53/0xa0
 [<c01679ed>] kernel_read+0x3d/0x60
 [<c0167abf>] prepare_binprm+0xaf/0xe0
 [<c0168fe5>] do_execve+0x115/0x1b0
 [<c0101c3f>] sys_execve+0x2f/0x60
 [<c0103780>] sysenter_past_esp+0x6d/0xa5
 =======================
Code: 2c b0 54 c0 8b 14 24 29 f8 e8 02 d4 ff ff 90 e8 3c d5 ff ff 83 e8 01 74 17 8d 45 45
83 c4 04 c1 e0 0c 29 c6 89 f0 5b 5e 5f 5d c3 <0f> 0b eb fe 90 8d b6 00 00 00 00 8d b6 00
00 00 00 eb db 66 90
EIP: [<c01177bb>] kmap_atomic_prot+0xbb/0xd0 SS:ESP 0068:f76a1dec
---[ end trace 526de21aa54fb137 ]---
note: udevd[990] exited with preempt_count 1
BUG: scheduling while atomic: udevd/990/0x10000001
Pid: 990, comm: udevd Tainted: G      D  2.6.25-rc2-via #121
 [<c03abacb>] schedule+0x1eb/0x250
 [<c011a073>] __cond_resched+0x13/0x30
 [<c03abbf7>] _cond_resched+0x27/0x30
 [<c011e916>] put_files_struct+0x96/0xb0
 [<c011fa2e>] do_exit+0x11e/0x6b0
 [<c011d84b>] printk+0x1b/0x20
 [<c0104cd2>] die+0x152/0x160
 [<c0105070>] do_invalid_op+0x0/0x90
 [<c01050f1>] do_invalid_op+0x81/0x90
 [<c01177bb>] kmap_atomic_prot+0xbb/0xd0
 [<c016c8b5>] __link_path_walk+0x915/0xbf0
 [<c037b258>] rpcauth_lookup_credcache+0x68/0x1b0
 [<c037b066>] rpcauth_lookupcred+0x56/0xa0
 [<c037b258>] rpcauth_lookup_credcache+0x68/0x1b0
 [<c03acdf2>] error_code+0x6a/0x70
 [<c01177bb>] kmap_atomic_prot+0xbb/0xd0
 [<c0140dfb>] file_read_actor+0xcb/0x100
 [<c014305f>] generic_file_aio_read+0x2df/0x590
 [<c0173a73>] dput+0x13/0xb0
 [<c0177d8b>] mntput_no_expire+0x1b/0x70
 [<c0163ce5>] do_sync_read+0xd5/0x120
 [<c012e100>] autoremove_wake_function+0x0/0x40
 [<c0162240>] __dentry_open+0x160/0x1b0
 [<c0214b8c>] security_file_permission+0xc/0x10
 [<c0163d8e>] rw_verify_area+0x5e/0xd0
 [<c0163c10>] do_sync_read+0x0/0x120
 [<c016459d>] vfs_read+0x9d/0x140
 [<c0152653>] insert_vm_struct+0x53/0xa0
 [<c01679ed>] kernel_read+0x3d/0x60
 [<c0167abf>] prepare_binprm+0xaf/0xe0
 [<c0168fe5>] do_execve+0x115/0x1b0
 [<c0101c3f>] sys_execve+0x2f/0x60
 [<c0103780>] sysenter_past_esp+0x6d/0xa5
 =======================
------------[ cut here ]------------
kernel BUG at arch/x86/mm/highmem_32.c:87!
invalid opcode: 0000 [#3]
Modules linked in: padlock_aes xts gf128mul cifs [last unloaded: padlock_aes]

Pid: 172, comm: udevd Tainted: G      D  (2.6.25-rc2-via #121)
EIP: 0060:[<c01177bb>] EFLAGS: 00010286 CPU: 0
EIP is at kmap_atomic_prot+0xbb/0xd0
EAX: da5f9163 EBX: c2b4fb80 ECX: 00000163 EDX: 00000003
ESI: fffff000 EDI: 0000000c EBP: 00000003 ESP: f7433ee8
 DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
Process udevd (pid: 172, ti=f7432000 task=f746eaa0 task.ti=f7432000)
Stack: f7681200 00000000 f7fb51b4 00000000 c2b4fb80 c014dbac f7433f1c bf93c578
       f74782c0 f7fb5180 c2b4c900 00000000 000004f0 f7fb51b4 da7dc065 f747e4f0
       c014f2b8 f747e4f0 f7405bf8 f7fb51c0 da7dc065 da7dc065 c0130c62 00000001
Call Trace:
 [<c014dbac>] do_wp_page+0x9c/0x4d0
 [<c014f2b8>] handle_mm_fault+0x508/0x690
 [<c0130c62>] enqueue_hrtimer+0x72/0x100
 [<c0115d5c>] do_page_fault+0x11c/0x6f0
 [<c0119b23>] hrtick_set+0x83/0x100
 [<c0115c40>] do_page_fault+0x0/0x6f0
 [<c03acdf2>] error_code+0x6a/0x70
 =======================
Code: 2c b0 54 c0 8b 14 24 29 f8 e8 02 d4 ff ff 90 e8 3c d5 ff ff 83 e8 01 74 17 8d 45 45
83 c4 04 c1 e0 0c 29 c6 89 f0 5b 5e 5f 5d c3 <0f> 0b eb fe 90 8d b6 00 00 00 00 8d b6 00
00 00 00 eb db 66 90
EIP: [<c01177bb>] kmap_atomic_prot+0xbb/0xd0 SS:ESP 0068:f7433ee8
---[ end trace 526de21aa54fb137 ]---
note: udevd[172] exited with preempt_count 1
BUG: scheduling while atomic: udevd/172/0x10000001
Pid: 172, comm: udevd Tainted: G      D  2.6.25-rc2-via #121
 [<c03abacb>] schedule+0x1eb/0x250
 [<c011a073>] __cond_resched+0x13/0x30
 [<c03abbf7>] _cond_resched+0x27/0x30
 [<c011e916>] put_files_struct+0x96/0xb0
 [<c011fa2e>] do_exit+0x11e/0x6b0
 [<c011d84b>] printk+0x1b/0x20
 [<c0104cd2>] die+0x152/0x160
 [<c0105070>] do_invalid_op+0x0/0x90
 [<c01050f1>] do_invalid_op+0x81/0x90
 [<c01177bb>] kmap_atomic_prot+0xbb/0xd0
 [<c03acdf2>] error_code+0x6a/0x70
 [<c01177bb>] kmap_atomic_prot+0xbb/0xd0
 [<c014dbac>] do_wp_page+0x9c/0x4d0
 [<c014f2b8>] handle_mm_fault+0x508/0x690
 [<c0130c62>] enqueue_hrtimer+0x72/0x100
 [<c0115d5c>] do_page_fault+0x11c/0x6f0
 [<c0119b23>] hrtick_set+0x83/0x100
 [<c0115c40>] do_page_fault+0x0/0x6f0
 [<c03acdf2>] error_code+0x6a/0x70
 =======================
BUG: scheduling while atomic: udevd/990/0x10000001
Pid: 990, comm: udevd Tainted: G      D  2.6.25-rc2-via #121
 [<c03abacb>] schedule+0x1eb/0x250
 [<c011a073>] __cond_resched+0x13/0x30
 [<c03abbf7>] _cond_resched+0x27/0x30
 [<c011e916>] put_files_struct+0x96/0xb0
 [<c011fa2e>] do_exit+0x11e/0x6b0
 [<c011d84b>] printk+0x1b/0x20
 [<c0104cd2>] die+0x152/0x160
 [<c0105070>] do_invalid_op+0x0/0x90
 [<c01050f1>] do_invalid_op+0x81/0x90
 [<c01177bb>] kmap_atomic_prot+0xbb/0xd0
 [<c016c8b5>] __link_path_walk+0x915/0xbf0
 [<c037b258>] rpcauth_lookup_credcache+0x68/0x1b0
 [<c037b066>] rpcauth_lookupcred+0x56/0xa0
 [<c037b258>] rpcauth_lookup_credcache+0x68/0x1b0
 [<c03acdf2>] error_code+0x6a/0x70
 [<c01177bb>] kmap_atomic_prot+0xbb/0xd0
 [<c0140dfb>] file_read_actor+0xcb/0x100
 [<c014305f>] generic_file_aio_read+0x2df/0x590
 [<c0173a73>] dput+0x13/0xb0
 [<c0177d8b>] mntput_no_expire+0x1b/0x70
 [<c0163ce5>] do_sync_read+0xd5/0x120
 [<c012e100>] autoremove_wake_function+0x0/0x40
 [<c0162240>] __dentry_open+0x160/0x1b0
 [<c0214b8c>] security_file_permission+0xc/0x10
 [<c0163d8e>] rw_verify_area+0x5e/0xd0
 [<c0163c10>] do_sync_read+0x0/0x120
 [<c016459d>] vfs_read+0x9d/0x140
 [<c0152653>] insert_vm_struct+0x53/0xa0
 [<c01679ed>] kernel_read+0x3d/0x60
 [<c0167abf>] prepare_binprm+0xaf/0xe0
 [<c0168fe5>] do_execve+0x115/0x1b0
 [<c0101c3f>] sys_execve+0x2f/0x60
 [<c0103780>] sysenter_past_esp+0x6d/0xa5
 =======================
------------[ cut here ]------------
kernel BUG at arch/x86/mm/highmem_32.c:87!
invalid opcode: 0000 [#4]
Modules linked in: padlock_aes xts gf128mul cifs [last unloaded: padlock_aes]

Pid: 989, comm: udevd Tainted: G      D  (2.6.25-rc2-via #121)
EIP: 0060:[<c01177bb>] EFLAGS: 00010286 CPU: 0
EIP is at kmap_atomic_prot+0xbb/0xd0
EAX: da5f9163 EBX: c2b6b560 ECX: 00000163 EDX: 00000003
ESI: fffff000 EDI: 0000000c EBP: 00000003 ESP: f762fe84
 DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
Process udevd (pid: 989, ti=f762e000 task=f7608550 task.ti=f762e000)
Stack: f7817ad0 c2b6b560 c2b6b560 00000000 00000000 c01461e7 00000002 00000044
       c017429b 00000000 c04755e4 c04755c8 00000002 00000000 001280d2 c0475b3c
       00000002 00000001 00000000 00000000 c04755c8 f7608550 001280d2 00000000
Call Trace:
 [<c01461e7>] get_page_from_freelist+0x257/0x460
 [<c017429b>] d_lookup+0x1b/0x40
 [<c0146449>] __alloc_pages+0x59/0x350
 [<c012e7c7>] posix_cpu_timers_exit_group+0x57/0x70
 [<c014f04b>] handle_mm_fault+0x29b/0x690
 [<c0115d5c>] do_page_fault+0x11c/0x6f0
 [<c011f801>] sys_wait4+0x81/0xb0
 [<c0115c40>] do_page_fault+0x0/0x6f0
 [<c03acdf2>] error_code+0x6a/0x70
 =======================
Code: 2c b0 54 c0 8b 14 24 29 f8 e8 02 d4 ff ff 90 e8 3c d5 ff ff 83 e8 01 74 17 8d 45 45
83 c4 04 c1 e0 0c 29 c6 89 f0 5b 5e 5f 5d c3 <0f> 0b eb fe 90 8d b6 00 00 00 00 8d b6 00
00 00 00 eb db 66 90
EIP: [<c01177bb>] kmap_atomic_prot+0xbb/0xd0 SS:ESP 0068:f762fe84
---[ end trace 526de21aa54fb137 ]---
note: udevd[989] exited with preempt_count 1
BUG: scheduling while atomic: udevd/989/0x10000001
Pid: 989, comm: udevd Tainted: G      D  2.6.25-rc2-via #121
 [<c03abacb>] schedule+0x1eb/0x250
 [<c011a073>] __cond_resched+0x13/0x30
 [<c03abbf7>] _cond_resched+0x27/0x30
 [<c011e916>] put_files_struct+0x96/0xb0
 [<c011fa2e>] do_exit+0x11e/0x6b0
 [<c011d84b>] printk+0x1b/0x20
 [<c0104cd2>] die+0x152/0x160
 [<c0105070>] do_invalid_op+0x0/0x90
 [<c01050f1>] do_invalid_op+0x81/0x90
 [<c01177bb>] kmap_atomic_prot+0xbb/0xd0
 [<c023284e>] number+0x31e/0x330
 [<c014620c>] get_page_from_freelist+0x27c/0x460
 [<c03acdf2>] error_code+0x6a/0x70
 [<c01177bb>] kmap_atomic_prot+0xbb/0xd0
 [<c01461e7>] get_page_from_freelist+0x257/0x460
 [<c017429b>] d_lookup+0x1b/0x40
 [<c0146449>] __alloc_pages+0x59/0x350
 [<c012e7c7>] posix_cpu_timers_exit_group+0x57/0x70
 [<c014f04b>] handle_mm_fault+0x29b/0x690
 [<c0115d5c>] do_page_fault+0x11c/0x6f0
 [<c011f801>] sys_wait4+0x81/0xb0
 [<c0115c40>] do_page_fault+0x0/0x6f0
 [<c03acdf2>] error_code+0x6a/0x70
 =======================






here the cryptsetup tool seems to be frozen, after I tried to start "top" in another
console I got this:





------------[ cut here ]------------
kernel BUG at arch/x86/mm/highmem_32.c:87!
invalid opcode: 0000 [#5]
Modules linked in: padlock_aes xts gf128mul cifs [last unloaded: padlock_aes]

Pid: 949, comm: bash Tainted: G      D  (2.6.25-rc2-via #121)
EIP: 0060:[<c01177bb>] EFLAGS: 00010286 CPU: 0
EIP is at kmap_atomic_prot+0xbb/0xd0
EAX: da5f9163 EBX: c2b4fbc0 ECX: 00000163 EDX: 00000003
ESI: fffff000 EDI: 0000000c EBP: 00000003 ESP: f75ffe84
 DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
Process bash (pid: 949, ti=f75fe000 task=f74deaa0 task.ti=f75fe000)
Stack: f759f000 c2b4fbc0 c2b4fbc0 00000000 00000000 c01461e7 00000002 00000044
       f7480000 00000000 c04755e4 c04755c8 00000002 00000000 001280d2 c0475b3c
       00000002 00000001 00000000 00000000 c04755c8 f74deaa0 001280d2 00000000
Call Trace:
 [<c01461e7>] get_page_from_freelist+0x257/0x460
 [<c0146449>] __alloc_pages+0x59/0x350
 [<c014f04b>] handle_mm_fault+0x29b/0x690
 [<c0115d5c>] do_page_fault+0x11c/0x6f0
 [<c0125c5e>] sys_rt_sigaction+0x6e/0xa0
 [<c0115c40>] do_page_fault+0x0/0x6f0
 [<c03acdf2>] error_code+0x6a/0x70
 =======================
Code: 2c b0 54 c0 8b 14 24 29 f8 e8 02 d4 ff ff 90 e8 3c d5 ff ff 83 e8 01 74 17 8d 45 45
83 c4 04 c1 e0 0c 29 c6 89 f0 5b 5e 5f 5d c3 <0f> 0b eb fe 90 8d b6 00 00 00 00 8d b6 00
00 00 00 eb db 66 90
EIP: [<c01177bb>] kmap_atomic_prot+0xbb/0xd0 SS:ESP 0068:f75ffe84
---[ end trace 526de21aa54fb137 ]---
note: bash[949] exited with preempt_count 1
BUG: scheduling while atomic: bash/949/0x10000001
Pid: 949, comm: bash Tainted: G      D  2.6.25-rc2-via #121
 [<c03abacb>] schedule+0x1eb/0x250
 [<c011a073>] __cond_resched+0x13/0x30
 [<c03abbf7>] _cond_resched+0x27/0x30
 [<c011e916>] put_files_struct+0x96/0xb0
 [<c011fa2e>] do_exit+0x11e/0x6b0
 [<c011d84b>] printk+0x1b/0x20
 [<c0104cd2>] die+0x152/0x160
 [<c0105070>] do_invalid_op+0x0/0x90
 [<c01050f1>] do_invalid_op+0x81/0x90
 [<c01177bb>] kmap_atomic_prot+0xbb/0xd0
 [<c026dfa6>] tty_wait_until_sent+0x26/0xd0
 [<c02753f2>] scrup+0xd2/0xe0
 [<c0131d47>] atomic_notifier_call_chain+0x17/0x20
 [<c03acdf2>] error_code+0x6a/0x70
 [<c01177bb>] kmap_atomic_prot+0xbb/0xd0
 [<c01461e7>] get_page_from_freelist+0x257/0x460
 [<c0146449>] __alloc_pages+0x59/0x350
 [<c014f04b>] handle_mm_fault+0x29b/0x690
 [<c0115d5c>] do_page_fault+0x11c/0x6f0
 [<c0125c5e>] sys_rt_sigaction+0x6e/0xa0
 [<c0115c40>] do_page_fault+0x0/0x6f0
 [<c03acdf2>] error_code+0x6a/0x70
 =======================
------------[ cut here ]------------
kernel BUG at arch/x86/mm/highmem_32.c:87!
invalid opcode: 0000 [#6]
Modules linked in: padlock_aes xts gf128mul cifs [last unloaded: padlock_aes]

Pid: 901, comm: login Tainted: G      D  (2.6.25-rc2-via #121)
EIP: 0060:[<c01177bb>] EFLAGS: 00010286 CPU: 0
EIP is at kmap_atomic_prot+0xbb/0xd0
EAX: da5f9163 EBX: c2b595e0 ECX: 00000163 EDX: 00000003
ESI: fffff000 EDI: 0000000c EBP: 00000003 ESP: f7437d34
 DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
Process login (pid: 901, ti=f7436000 task=f75ae000 task.ti=f7436000)
Stack: c01413f0 f7437dc8 c2b595e0 00000180 00000680 c0140add fffffff4 00000180
       005c5680 00000000 00000980 c01420d4 00000180 00000000 00000180 00000000
       f7437ddc f7437dd8 00000000 f7437ee0 f75e8680 f78ad888 c03bece0 f78ad7f0
Call Trace:
 [<c01413f0>] __grab_cache_page+0x20/0xc0
 [<c0140add>] iov_iter_copy_from_user_atomic+0x2d/0x80
 [<c01420d4>] generic_file_buffered_write+0x134/0x620
 [<c0102789>] __switch_to+0x89/0x140
 [<c014283d>] __generic_file_aio_write_nolock+0x27d/0x4d0
 [<c0374152>] rpc_release_client+0x32/0x60
 [<c0142af2>] generic_file_aio_write+0x62/0xd0
 [<c01c739e>] nfs_update_inode+0xfe/0x740
 [<c01c64d5>] nfs_file_write+0xa5/0x170
 [<c01c7e21>] __nfs_revalidate_inode+0xc1/0x270
 [<c0163bc5>] do_sync_write+0xd5/0x120
 [<c012e100>] autoremove_wake_function+0x0/0x40
 [<c0173350>] fcntl_setlk+0x40/0x230
 [<c0214b8c>] security_file_permission+0xc/0x10
 [<c0163d8e>] rw_verify_area+0x5e/0xd0
 [<c0163af0>] do_sync_write+0x0/0x120
 [<c016445f>] vfs_write+0x9f/0x140
 [<c01637dc>] vfs_llseek+0x3c/0x50
 [<c0164a21>] sys_write+0x41/0x70
 [<c0103780>] sysenter_past_esp+0x6d/0xa5
 =======================
Code: 2c b0 54 c0 8b 14 24 29 f8 e8 02 d4 ff ff 90 e8 3c d5 ff ff 83 e8 01 74 17 8d 45 45
83 c4 04 c1 e0 0c 29 c6 89 f0 5b 5e 5f 5d c3 <0f> 0b eb fe 90 8d b6 00 00 00 00 8d b6 00
00 00 00 eb db 66 90
EIP: [<c01177bb>] kmap_atomic_prot+0xbb/0xd0 SS:ESP 0068:f7437d34
---[ end trace 526de21aa54fb137 ]---
note: login[901] exited with preempt_count 2
BUG: scheduling while atomic: login/901/0x10000002
Pid: 901, comm: login Tainted: G      D  2.6.25-rc2-via #121
 [<c03abacb>] schedule+0x1eb/0x250
 [<c011a073>] __cond_resched+0x13/0x30
 [<c03abbf7>] _cond_resched+0x27/0x30
 [<c011e916>] put_files_struct+0x96/0xb0
 [<c011fa2e>] do_exit+0x11e/0x6b0
 [<c011d84b>] printk+0x1b/0x20
 [<c0104cd2>] die+0x152/0x160
 [<c0105070>] do_invalid_op+0x0/0x90
 [<c01050f1>] do_invalid_op+0x81/0x90
 [<c01177bb>] kmap_atomic_prot+0xbb/0xd0
 [<c012e100>] autoremove_wake_function+0x0/0x40
 [<c037abcc>] rpcauth_unwrap_resp+0x6c/0xa0
 [<c0373922>] call_decode+0x192/0x7b0
 [<c03acdf2>] error_code+0x6a/0x70
 [<c01177bb>] kmap_atomic_prot+0xbb/0xd0
 [<c01413f0>] __grab_cache_page+0x20/0xc0
 [<c0140add>] iov_iter_copy_from_user_atomic+0x2d/0x80
 [<c01420d4>] generic_file_buffered_write+0x134/0x620
 [<c0102789>] __switch_to+0x89/0x140
 [<c014283d>] __generic_file_aio_write_nolock+0x27d/0x4d0
 [<c0374152>] rpc_release_client+0x32/0x60
 [<c0142af2>] generic_file_aio_write+0x62/0xd0
 [<c01c739e>] nfs_update_inode+0xfe/0x740
 [<c01c64d5>] nfs_file_write+0xa5/0x170
 [<c01c7e21>] __nfs_revalidate_inode+0xc1/0x270
 [<c0163bc5>] do_sync_write+0xd5/0x120
 [<c012e100>] autoremove_wake_function+0x0/0x40
 [<c0173350>] fcntl_setlk+0x40/0x230
 [<c0214b8c>] security_file_permission+0xc/0x10
 [<c0163d8e>] rw_verify_area+0x5e/0xd0
 [<c0163af0>] do_sync_write+0x0/0x120
 [<c016445f>] vfs_write+0x9f/0x140
 [<c01637dc>] vfs_llseek+0x3c/0x50
 [<c0164a21>] sys_write+0x41/0x70
 [<c0103780>] sysenter_past_esp+0x6d/0xa5
 =======================
BUG: scheduling while atomic: login/901/0x00000002
Pid: 901, comm: login Tainted: G      D  2.6.25-rc2-via #121
 [<c03abacb>] schedule+0x1eb/0x250
 [<c0379d44>] rpc_wait_bit_killable+0x14/0x30
 [<c03abe62>] __wait_on_bit+0x42/0x70
 [<c0379d30>] rpc_wait_bit_killable+0x0/0x30
 [<c0379d30>] rpc_wait_bit_killable+0x0/0x30
 [<c03abeea>] out_of_line_wait_on_bit+0x5a/0x70
 [<c012e140>] wake_bit_function+0x0/0x60
 [<c037a162>] __rpc_execute+0xa2/0x240
 [<c014887a>] pagevec_lookup_tag+0x2a/0x40
 [<c0373fca>] rpc_run_task+0x2a/0x60
 [<c037409c>] rpc_call_sync+0x3c/0x60
 [<c01d1fa4>] nfs3_rpc_wrapper+0x34/0x60
 [<c01d2318>] nfs3_proc_getattr+0x48/0x80
 [<c01c7e98>] __nfs_revalidate_inode+0x138/0x270
 [<c01d0000>] nfs_writeback_done+0x190/0x1f0
 [<c01d0790>] __nfs_write_mapping+0x30/0x60
 [<c01d0814>] nfs_write_mapping+0x54/0x70
 [<c01c674a>] nfs_file_flush+0x7a/0xa0
 [<c0161fbe>] filp_close+0x2e/0x80
 [<c011e911>] put_files_struct+0x91/0xb0
 [<c011fa2e>] do_exit+0x11e/0x6b0
 [<c011d84b>] printk+0x1b/0x20
 [<c0104cd2>] die+0x152/0x160
 [<c0105070>] do_invalid_op+0x0/0x90
 [<c01050f1>] do_invalid_op+0x81/0x90
 [<c01177bb>] kmap_atomic_prot+0xbb/0xd0
 [<c012e100>] autoremove_wake_function+0x0/0x40
 [<c037abcc>] rpcauth_unwrap_resp+0x6c/0xa0
 [<c0373922>] call_decode+0x192/0x7b0
 [<c03acdf2>] error_code+0x6a/0x70
 [<c01177bb>] kmap_atomic_prot+0xbb/0xd0
 [<c01413f0>] __grab_cache_page+0x20/0xc0
 [<c0140add>] iov_iter_copy_from_user_atomic+0x2d/0x80
 [<c01420d4>] generic_file_buffered_write+0x134/0x620
 [<c0102789>] __switch_to+0x89/0x140
 [<c014283d>] __generic_file_aio_write_nolock+0x27d/0x4d0
 [<c0374152>] rpc_release_client+0x32/0x60
 [<c0142af2>] generic_file_aio_write+0x62/0xd0
 [<c01c739e>] nfs_update_inode+0xfe/0x740
 [<c01c64d5>] nfs_file_write+0xa5/0x170
 [<c01c7e21>] __nfs_revalidate_inode+0xc1/0x270
 [<c0163bc5>] do_sync_write+0xd5/0x120
 [<c012e100>] autoremove_wake_function+0x0/0x40
 [<c0173350>] fcntl_setlk+0x40/0x230
 [<c0214b8c>] security_file_permission+0xc/0x10
 [<c0163d8e>] rw_verify_area+0x5e/0xd0
 [<c0163af0>] do_sync_write+0x0/0x120
 [<c016445f>] vfs_write+0x9f/0x140
 [<c01637dc>] vfs_llseek+0x3c/0x50
 [<c0164a21>] sys_write+0x41/0x70
 [<c0103780>] sysenter_past_esp+0x6d/0xa5
 =======================
------------[ cut here ]------------
kernel BUG at arch/x86/mm/highmem_32.c:87!
invalid opcode: 0000 [#7]
Modules linked in: padlock_aes xts gf128mul cifs [last unloaded: padlock_aes]

Pid: 1, comm: init Tainted: G      D  (2.6.25-rc2-via #121)
EIP: 0060:[<c01177bb>] EFLAGS: 00010286 CPU: 0
EIP is at kmap_atomic_prot+0xbb/0xd0
EAX: da5f9163 EBX: c2b79960 ECX: 00000163 EDX: 00000003
ESI: fffff000 EDI: 0000000c EBP: 00000003 ESP: f7c69f00
 DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
Process init (pid: 1, ti=f7c68000 task=f7c5e000 task.ti=f7c68000)
Stack: c047169c b7f801c0 00000180 f7c69f58 00000180 c0140dfb 00000000 c2b79960
       00000000 00000180 00000000 c2b79960 00000d80 00000000 c015cd0c 00000d80
       00000000 f75e8800 00000000 00000000 f7406b14 0000a7c0 00000000 00000180
Call Trace:
 [<c0140dfb>] file_read_actor+0xcb/0x100
 [<c015cd0c>] shmem_file_read+0x10c/0x2a0
 [<c015cc00>] shmem_file_read+0x0/0x2a0
 [<c016459d>] vfs_read+0x9d/0x140
 [<c01649b1>] sys_read+0x41/0x70
 [<c0103780>] sysenter_past_esp+0x6d/0xa5
 =======================
Code: 2c b0 54 c0 8b 14 24 29 f8 e8 02 d4 ff ff 90 e8 3c d5 ff ff 83 e8 01 74 17 8d 45 45
83 c4 04 c1 e0 0c 29 c6 89 f0 5b 5e 5f 5d c3 <0f> 0b eb fe 90 8d b6 00 00 00 00 8d b6 00
00 00 00 eb db 66 90
EIP: [<c01177bb>] kmap_atomic_prot+0xbb/0xd0 SS:ESP 0068:f7c69f00
---[ end trace 526de21aa54fb137 ]---
note: init[1] exited with preempt_count 1
Kernel panic - not syncing: Attempted to kill init!


Sebastian Siewior schrieb:

* Stefan Hellermann | 2008-02-24 12:54:20 [+0100]:

quoted

Hello,

Hello,

quoted

I'm got my Via Epia SN Board a few days ago and could test everything related to the
padlock engine, I'm especially interested in the aes-{lrw,xts} combo, this doesn't work at

Cool,

quoted

the moment (last tested with 2.6.25-rc1).

Could you be a little more specific on "it doesn't work"?
Do you pass the tcrypt test at least?
Does it* work without the HW acceleration?

quoted

Cheers,
Stefan

*: it means a dm-crypt encrypted partition I guess.

Sebastian
-
To unsubscribe from this list: send the line "unsubscribe linux-crypto" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

* Stefan Hellermann | 2008-02-24 21:07:03 [+0100]:

general protection fault: 0000 [#1]
Modules linked in: padlock_aes xts gf128mul cifs [last unloaded: padlock_aes]

Pid: 988, comm: kcryptd Not tainted (2.6.25-rc2-via #121)
EIP: 0060:[<f881d801>] EFLAGS: 00010206 CPU: 0
EIP is at aes_encrypt+0x31/0x60 [padlock_aes]
EAX: f7468af0 EBX: f7616860 ECX: 00000001 EDX: f7616830
ESI: f7468500 EDI: f762de88 EBP: f762de88 ESP: f762de64
DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
Process kcryptd (pid: 988, ti=f762c000 task=f746eff0 task.ti=f762c000)
Stack: f75e3770 fffb6000 fffb7e00 00000200 f88280a3 f75e3770 f762debc f762df04
      00000010 00000000 00000000 00000000 00000000 f7616400 f881d7d0 f75e3770
      f75c0600 f7468500 c048ab44 f8828272 f881d7d0 f881d7d0 c2b4bf20 fffb7e00
Call Trace:
[<f88280a3>] crypt+0x83/0x110 [xts]
[<f881d7d0>] aes_encrypt+0x0/0x60 [padlock_aes]
[<f8828272>] encrypt+0x42/0x50 [xts]
[<f881d7d0>] aes_encrypt+0x0/0x60 [padlock_aes]
[<f881d7d0>] aes_encrypt+0x0/0x60 [padlock_aes]
[<c021b05b>] async_encrypt+0x3b/0x50
[<c02fcce9>] crypt_convert+0x1b9/0x270
[<c02fcf4d>] kcryptd_crypt+0x1ad/0x220
[<c02fcda0>] kcryptd_crypt+0x0/0x220
[<c012ae6b>] run_workqueue+0xab/0x140
[<c012b5e0>] worker_thread+0x0/0x90
[<c012b639>] worker_thread+0x59/0x90
[<c012e100>] autoremove_wake_function+0x0/0x40
[<c012b5e0>] worker_thread+0x0/0x90
[<c012dd22>] kthread+0x42/0x70
[<c012dce0>] kthread+0x0/0x70
[<c010437b>] kernel_thread_helper+0x7/0x1c
=======================
Code: 0c 89 d7 8d 50 3f 89 74 24 08 83 e2 f0 89 ce 89 5c 24 04 9c 9d 89 c8 35 f0 0f 00 00
a9 ff 0f 00 00 8d 5a 30 74 19 b9 01 00 00 00 <f3> 0f a7 c8 8b 5c 24 04 8b 74 24 08 8b 7c
24 0c 83 c4 10 c3 89
EIP: [<f881d801>] aes_encrypt+0x31/0x60 [padlock_aes] SS:ESP 0068:f762de64
---[ end trace 526de21aa54fb137 ]---

This is exactly the xcrypt instruction. I can reproduce what I thing is
the same bug on my geode board. For some reason the stack gets
overwritten. I will dig later a little more.

Sebastian

The XTS blockmode uses a copy of the IV which is saved on the stack
and may or may not be properly aligned. If it is not, it will break
hardware cipher like the geode or padlock.
This patch moves the copy of IV to the private structre which has the
same aligment as the underlying cipher.

Signed-off-by: Sebastian Siewior <redacted>
---
Stefan, please try the following patch, it should fix your xts problem.

 crypto/xts.c |   32 +++++++++++++++++---------------
 1 files changed, 17 insertions(+), 15 deletions(-)

diff --git a/crypto/xts.c b/crypto/xts.c
index 8eb08bf..4457022 100644
--- a/crypto/xts.c
+++ b/crypto/xts.c

@@ -24,7 +24,17 @@
 #include <crypto/b128ops.h>
 #include <crypto/gf128mul.h>
 
+struct sinfo {
+	be128 t;
+	struct crypto_tfm *tfm;
+	void (*fn)(struct crypto_tfm *, u8 *, const u8 *);
+};
+
 struct priv {
+	/* s.t being the first member in this struct enforces proper alignment
+	 * required by the underlying cipher without explicit knowing the it.
+	 */
+	struct sinfo s;
 	struct crypto_cipher *child;
 	struct crypto_cipher *tweak;
 };

@@ -76,12 +86,6 @@ static int setkey(struct crypto_tfm *parent, const u8 *key,
 	return 0;
 }
 
-struct sinfo {
-	be128 t;
-	struct crypto_tfm *tfm;
-	void (*fn)(struct crypto_tfm *, u8 *, const u8 *);
-};
-
 static inline void xts_round(struct sinfo *s, void *dst, const void *src)
 {
 	be128_xor(dst, &s->t, src);		/* PP <- T xor P */

@@ -97,13 +101,12 @@ static int crypt(struct blkcipher_desc *d,
 	int err;
 	unsigned int avail;
 	const int bs = crypto_cipher_blocksize(ctx->child);
-	struct sinfo s = {
-		.tfm = crypto_cipher_tfm(ctx->child),
-		.fn = fn
-	};
-	be128 *iv;
 	u8 *wsrc;
 	u8 *wdst;
+	struct sinfo *s = &ctx->s;
+
+	s->tfm = crypto_cipher_tfm(ctx->child);
+	s->fn = fn;
 
 	err = blkcipher_walk_virt(d, w);
 	if (!w->nbytes)

@@ -115,17 +118,16 @@ static int crypt(struct blkcipher_desc *d,
 	wdst = w->dst.virt.addr;
 
 	/* calculate first value of T */
-	iv = (be128 *)w->iv;
-	tw(crypto_cipher_tfm(ctx->tweak), (void *)&s.t, w->iv);
+	tw(crypto_cipher_tfm(ctx->tweak), (void *)&s->t, w->iv);
 
 	goto first;
 
 	for (;;) {
 		do {
-			gf128mul_x_ble(&s.t, &s.t);
+			gf128mul_x_ble(&s->t, &s->t);
 
 first:
-			xts_round(&s, wdst, wsrc);
+			xts_round(s, wdst, wsrc);
 
 			wsrc += bs;
 			wdst += bs;

-- 
1.5.3.4

Sebastian Siewior schrieb:

The XTS blockmode uses a copy of the IV which is saved on the stack
and may or may not be properly aligned. If it is not, it will break
hardware cipher like the geode or padlock.
This patch moves the copy of IV to the private structre which has the
same aligment as the underlying cipher.

Signed-off-by: Sebastian Siewior <redacted>

It works now! Thanks!
But I get much lower speed than with aes-cbc-essiv:sha256.
With xts I get 57MB/s while reading the cryptodev with dd, and >90% sys in top, 0% wait
With cbc-essiv I get about 75MB/s while reading it with dd, 60% sys int top, 30% wait
without cryptodev I get 75MB/s while reading the raw lvm-volume with dd, 40% sys, 50% wait
I do a blockdev --flushbufs beetween each read.


Tested-by: Stefan Hellermann <redacted>

* Stefan Hellermann | 2008-03-02 13:04:37 [+0100]:

But I get much lower speed than with aes-cbc-essiv:sha256.

Yes, I expected this :)
The aes-cbc operation is supported directly in HW. So the
driver just says here is the key, source, destination, length and now do
it. So the HW fetches the key once and is going to process the whole
request (lets say 4 KiB) in one go.

The XTS blockmode on the other hand encrypts encrypts only 16 bytes in
one go and performs some GF operations in between. This is
repeated until we encrypt the whole request. So for a 4 KiB we need here
257 calls to the HW instead of one (the one extra is to encrypt the IV).
For every encryption call we have to reset the HW key. According to the
spec fetching the key from memory takes more time than the whole
encryption process as it (in case of a 16 byte block). This might still
be faster than the pure software solution.
Anyway, XTS is way more complex than CBC and part of it is done in
software what we can't change.

With xts I get 57MB/s while reading the cryptodev with dd, and >90% sys in top, 0% wait
With cbc-essiv I get about 75MB/s while reading it with dd, 60% sys int top, 30% wait
without cryptodev I get 75MB/s while reading the raw lvm-volume with dd, 40% sys, 50% wait
I do a blockdev --flushbufs beetween each read.

According to this numbers I would say in CBC mode the HD is breaking in
XTS the CPU is.
I could try to tune it a little but don't expect much.
Could you please compare xts with and without padlock?

Sebastian

Sebastian Siewior schrieb:

* Stefan Hellermann | 2008-03-02 13:04:37 [+0100]:

quoted

But I get much lower speed than with aes-cbc-essiv:sha256.

Yes, I expected this :)
The aes-cbc operation is supported directly in HW. So the
driver just says here is the key, source, destination, length and now do
it. So the HW fetches the key once and is going to process the whole
request (lets say 4 KiB) in one go.

The XTS blockmode on the other hand encrypts encrypts only 16 bytes in
one go and performs some GF operations in between. This is
repeated until we encrypt the whole request. So for a 4 KiB we need here
257 calls to the HW instead of one (the one extra is to encrypt the IV).
For every encryption call we have to reset the HW key. According to the
spec fetching the key from memory takes more time than the whole
encryption process as it (in case of a 16 byte block). This might still
be faster than the pure software solution.
Anyway, XTS is way more complex than CBC and part of it is done in
software what we can't change.

Ah, good to know! Could this information be placed in the Kconfig-help for padlock_aes?

quoted

With xts I get 57MB/s while reading the cryptodev with dd, and >90% sys in top, 0% wait
With cbc-essiv I get about 75MB/s while reading it with dd, 60% sys int top, 30% wait
without cryptodev I get 75MB/s while reading the raw lvm-volume with dd, 40% sys, 50% wait
I do a blockdev --flushbufs beetween each read.

According to this numbers I would say in CBC mode the HD is breaking in
XTS the CPU is.
I could try to tune it a little but don't expect much.
Could you please compare xts with and without padlock?

Yes, xts with padlock is almost 3 times faster.
20-21MB/s read in dd without padlock_aes, >90% CPU sys, 0% wait
57-58MB/s read in dd with padlock_aes, >90% CPU sys, 0% wait

I tried lrw-benbi/lrw-plain this time, but it doesn't work, with or without padlock_aes.
dmesg logs:
device-mapper: table: 252:6: crypt: Error allocating crypto tfm
device-mapper: ioctl: error adding target to table
device-mapper: ioctl: device doesn't appear to be in the dev hash table

... but I will use cbc-essiv, if I ever need better encryption I can take xts-plain. (no
need for lrw-benbi)

Thanks
Stefan

I tried lrw-benbi/lrw-plain this time, but it doesn't work, with or without padlock_aes.
dmesg logs:
device-mapper: table: 252:6: crypt: Error allocating crypto tfm
device-mapper: ioctl: error adding target to table
device-mapper: ioctl: device doesn't appear to be in the dev hash table

forget this ... with CONFIG_CRYPTO_LRW unset it can't work. But with your new patch it
works even with padlock_aes.


Thanks
Stefan

Sebastian Siewior schrieb:

Padlock AES' setkey routine is the same as exported by the generic
implementation. So we could use it.

I tested this and "[RFC] generic_aes: export generic setkey" on a padlock-enabled Via
board, and did the following test:

Create, open, write to, read from and close a linux dm-crypt device with aes-cbc-essiv,
aes-lrw-benbi and aes-xts-plain.

Then I took a huge encrypted disk-image (encrypted without this patches), opened it with
cryptsetup-luks, booted the OS from the disc over iscsi, started a filesystem-check. The
check completed successful.

So I think this and the other patch are save.


Tested-by: Stefan Hellermann <redacted>