From patchwork Fri Sep 23 02:40:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhangfei Gao X-Patchwork-Id: 608577 Delivered-To: patch@linaro.org Received: by 2002:a17:522:c983:b0:460:3032:e3c4 with SMTP id kr3csp37632pvb; Thu, 22 Sep 2022 19:41:01 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5rD3bNgonYJ3H6bgnOg+i1zcaaFgeG6scpbROL8oPc071Aq5MhLMeCH0OP0082POj1xA/S X-Received: by 2002:a5d:5a0d:0:b0:228:da3f:738 with SMTP id bq13-20020a5d5a0d000000b00228da3f0738mr3723441wrb.526.1663900861596; Thu, 22 Sep 2022 19:41:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1663900861; cv=none; d=google.com; s=arc-20160816; b=PZn6IUq/97qVrtdTx9oMqdIKpP5MGymXiKTNE+LZESdfKzKXGxXlRFeiT7tXJASQBH mddIIvvZvcNKiQusI73nPuvd4GwL5MQj4o7jLwGaqXi8xZl1m/8V65z3eotHpQr6966S CL6NKdCTSozmPbP6kaEb3rGcoKHcFb8dSI0czx5YNnte4F0hUaIAygyslHo2UHUz3N9u pPdPcsxvC0BIrXGtfe1TUvl/oP2S6qFD3F5fZ/aGf4nQT5W4PExXJ5XA6GnC+VATL1Bo AYbs9zWrvyV3tgyAqmiDFQI7Nay6R8OAtf1vjKJHn6o5Uk1m7UwCAWNS4TQ4tgAt0fsx H0Kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=47GX/bSxDO5Dj/HGxysTpVv0eGr5z+i20Q/9QaKJEcE=; b=okkaVD7RRGhL18CYDcCUHtMB97Pu2rQNJt7iamcurVapl3s0Qan91/Dv2zW3wY9VdD HxgZ35G2GpRzZAt+8QuMgUOS9bynZMQRQy3aS0KnBdXd+gypRnLdz23QhBMPXAZJPFz6 4/paFxaV81tUGBzQP2ejFqANBwnHZR7wp8Hzfuz+bfcLD8/CLx4KsLaMGhQALmP+y+vr wD8TZH/C87kHdMPO6dMdsa+zdYx3pktS48eo0/7bsX8LQ/qWZuOxLQhgrXT2yvDB4h80 D+NLdC2TQ8SCw5v4JtOotO3ti/XxDOB9KWF7gOwnkuMO5qdfqvx5M9vonfMyE4+DfHPj 4DAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=A9kKS2Hq; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from mails.dpdk.org (mails.dpdk.org. [217.70.189.124]) by mx.google.com with ESMTP id h14-20020a05600016ce00b002216dd6a282si5743624wrf.70.2022.09.22.19.41.01; Thu, 22 Sep 2022 19:41:01 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) client-ip=217.70.189.124; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=A9kKS2Hq; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 217.70.189.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 604F042B6E; Fri, 23 Sep 2022 04:41:00 +0200 (CEST) Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by mails.dpdk.org (Postfix) with ESMTP id 908B142836 for ; Fri, 23 Sep 2022 04:40:59 +0200 (CEST) Received: by mail-pl1-f174.google.com with SMTP id t3so10554531ply.2 for ; Thu, 22 Sep 2022 19:40:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date; bh=47GX/bSxDO5Dj/HGxysTpVv0eGr5z+i20Q/9QaKJEcE=; b=A9kKS2HqfIDmaRO515rRGWd/gOldU0SxiQoBPFAxEuJ/uVvSFYaGYA1ugvgNKGtyYX Bkr6C6Onm8vvVcUWKlFKxFGjHKxIl64zEeJ/jTXlXVaL8651wB+JhPGxrKt4zlpYPUHO tNoqeo3Pq5PwFf5nveN8t3Hb6WLlSqXwh9Y6hYdSKPiLoHd+4E//AuyvAWZ97A1kl7fu WrzIPVxuJxgLvqGY5/VQFolnU40zZuc96vbU48GBF/z/1Eij8lWYCmzf56JEHkU1bRFu neh1YxTxraQqo68T1b2TWHDPqBzlHt7JJpEvf2VYMnZsuHUVdifQRorCyXq0ZaZ5bBZi xnJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date; bh=47GX/bSxDO5Dj/HGxysTpVv0eGr5z+i20Q/9QaKJEcE=; b=gsvjbXoZ2HpDCvq49JDCn7GbPQZy5IfwYGhIIJJqt3TfxxMUmzJkqYIevqb3qVisQG Yy2xshr3jHTyNP1IIxJ4kBp8bbwWjj8/fHgLP9ehuckSsLmFGJoTldBqYu2sDsE3xHi1 6rZKGhXDklQ3RqsLkQ/jALuiFSKdPTKnUCdB7ZXvk3J5jlwBZr/k3XrZSB96/RFP3RPh oMVNUtKe4iH5wPX3MJJfWZVxYNAgcp0wkrwq++64Se7ot3hsdNiYJMmfKJaPJkiQsgFR PxHbTT1IKhWybkGIf0hijsaOHwYAv8+zx7H8iOPAHcuwo0sGU8F/nSmZlshAoAYU/FTx C7PA== X-Gm-Message-State: ACrzQf2kur8IaJeOcSf0sJk+tyWKSRbmYvUCt/OGRL1lnWJhCo5RZaey xXEjOAv+YaB1RKLrw5+R3FbskReCf0X9GLP4 X-Received: by 2002:a17:90b:1bc3:b0:203:84d:59b1 with SMTP id oa3-20020a17090b1bc300b00203084d59b1mr18889570pjb.37.1663900858915; Thu, 22 Sep 2022 19:40:58 -0700 (PDT) Received: from localhost.localdomain ([199.101.192.56]) by smtp.gmail.com with ESMTPSA id t5-20020a625f05000000b0053e8940c4absm5075011pfb.209.2022.09.22.19.40.50 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Sep 2022 19:40:58 -0700 (PDT) From: Zhangfei Gao To: Akhil Goyal , Declan Doherty , Fan Zhang , Ashish Gupta , Ray Kinsella Cc: dev@dpdk.org, acc@openeuler.org, Zhangfei Gao Subject: [PATCH v2 3/5] crypto/uadk: support cipher algorithms Date: Fri, 23 Sep 2022 10:40:21 +0800 Message-Id: <20220923024023.15849-4-zhangfei.gao@linaro.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220923024023.15849-1-zhangfei.gao@linaro.org> References: <20220923024023.15849-1-zhangfei.gao@linaro.org> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Cipher algorithms: * ``RTE_CRYPTO_CIPHER_AES_ECB`` * ``RTE_CRYPTO_CIPHER_AES_CBC`` * ``RTE_CRYPTO_CIPHER_AES_XTS`` * ``RTE_CRYPTO_CIPHER_DES_CBC`` Signed-off-by: Zhangfei Gao --- drivers/crypto/uadk/uadk_crypto_pmd.c | 254 +++++++++++++++++++++++++- 1 file changed, 252 insertions(+), 2 deletions(-) diff --git a/drivers/crypto/uadk/uadk_crypto_pmd.c b/drivers/crypto/uadk/uadk_crypto_pmd.c index aad42524d6..592d747a5d 100644 --- a/drivers/crypto/uadk/uadk_crypto_pmd.c +++ b/drivers/crypto/uadk/uadk_crypto_pmd.c @@ -30,11 +30,26 @@ struct uadk_qp { } __rte_cache_aligned; enum uadk_chain_order { + UADK_CHAIN_ONLY_CIPHER, UADK_CHAIN_NOT_SUPPORTED }; struct uadk_crypto_session { + handle_t handle_cipher; enum uadk_chain_order chain_order; + + /* IV parameters */ + struct { + uint16_t length; + uint16_t offset; + } iv; + + /* Cipher Parameters */ + struct { + enum rte_crypto_cipher_operation direction; + /* cipher operation direction */ + struct wd_cipher_req req; + } cipher; } __rte_cache_aligned; enum uadk_crypto_version { @@ -43,6 +58,7 @@ enum uadk_crypto_version { }; struct uadk_crypto_priv { + bool env_cipher_init; enum uadk_crypto_version version; } __rte_cache_aligned; @@ -56,6 +72,86 @@ RTE_LOG_REGISTER_DEFAULT(uadk_crypto_logtype, INFO); ## __VA_ARGS__) static const struct rte_cryptodev_capabilities uadk_crypto_v2_capabilities[] = { + { /* AES ECB */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_AES_ECB, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 8 + }, + .iv_size = { + .min = 0, + .max = 0, + .increment = 0 + } + }, } + }, } + }, + { /* AES CBC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_AES_CBC, + .block_size = 16, + .key_size = { + .min = 16, + .max = 32, + .increment = 8 + }, + .iv_size = { + .min = 16, + .max = 16, + .increment = 0 + } + }, } + }, } + }, + { /* AES XTS */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_AES_XTS, + .block_size = 1, + .key_size = { + .min = 32, + .max = 64, + .increment = 32 + }, + .iv_size = { + .min = 0, + .max = 0, + .increment = 0 + } + }, } + }, } + }, + { /* DES CBC */ + .op = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + {.sym = { + .xform_type = RTE_CRYPTO_SYM_XFORM_CIPHER, + {.cipher = { + .algo = RTE_CRYPTO_CIPHER_DES_CBC, + .block_size = 8, + .key_size = { + .min = 8, + .max = 8, + .increment = 0 + }, + .iv_size = { + .min = 8, + .max = 8, + .increment = 0 + } + }, } + }, } + }, /* End of symmetric capabilities */ RTE_CRYPTODEV_END_OF_CAPABILITIES_LIST() }; @@ -83,8 +179,15 @@ uadk_crypto_pmd_stop(struct rte_cryptodev *dev __rte_unused) /* Close device */ static int -uadk_crypto_pmd_close(struct rte_cryptodev *dev __rte_unused) +uadk_crypto_pmd_close(struct rte_cryptodev *dev) { + struct uadk_crypto_priv *priv = dev->data->dev_private; + + if (priv->env_cipher_init) { + wd_cipher_env_uninit(); + priv->env_cipher_init = false; + } + return 0; } @@ -238,19 +341,107 @@ uadk_crypto_sym_session_get_size(struct rte_cryptodev *dev __rte_unused) } static enum uadk_chain_order -uadk_get_chain_order(const struct rte_crypto_sym_xform *xform __rte_unused) +uadk_get_chain_order(const struct rte_crypto_sym_xform *xform) { enum uadk_chain_order res = UADK_CHAIN_NOT_SUPPORTED; + if (xform != NULL) { + if (xform->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + if (xform->next == NULL) + res = UADK_CHAIN_ONLY_CIPHER; + } + } + return res; } +static int +uadk_set_session_cipher_parameters(struct rte_cryptodev *dev, + struct uadk_crypto_session *sess, + struct rte_crypto_sym_xform *xform) +{ + struct uadk_crypto_priv *priv = dev->data->dev_private; + struct rte_crypto_cipher_xform *cipher = &xform->cipher; + struct wd_cipher_sess_setup setup = {0}; + struct sched_params params = {0}; + int ret; + + if (!priv->env_cipher_init) { + ret = wd_cipher_env_init(NULL); + if (ret < 0) + return -EINVAL; + priv->env_cipher_init = true; + } + + sess->cipher.direction = cipher->op; + sess->iv.offset = cipher->iv.offset; + sess->iv.length = cipher->iv.length; + + switch (cipher->algo) { + /* Cover supported cipher algorithms */ + case RTE_CRYPTO_CIPHER_AES_CTR: + setup.alg = WD_CIPHER_AES; + setup.mode = WD_CIPHER_CTR; + sess->cipher.req.out_bytes = 64; + break; + case RTE_CRYPTO_CIPHER_AES_ECB: + setup.alg = WD_CIPHER_AES; + setup.mode = WD_CIPHER_ECB; + sess->cipher.req.out_bytes = 16; + break; + case RTE_CRYPTO_CIPHER_AES_CBC: + setup.alg = WD_CIPHER_AES; + setup.mode = WD_CIPHER_CBC; + if (cipher->key.length == 16) + sess->cipher.req.out_bytes = 16; + else + sess->cipher.req.out_bytes = 64; + break; + case RTE_CRYPTO_CIPHER_AES_XTS: + setup.alg = WD_CIPHER_AES; + setup.mode = WD_CIPHER_XTS; + if (cipher->key.length == 16) + sess->cipher.req.out_bytes = 32; + else + sess->cipher.req.out_bytes = 512; + break; + default: + ret = -ENOTSUP; + goto env_uninit; + } + + params.numa_id = -1; /* choose nearby numa node */ + setup.sched_param = ¶ms; + sess->handle_cipher = wd_cipher_alloc_sess(&setup); + if (!sess->handle_cipher) { + UADK_LOG(ERR, "uadk failed to alloc session!\n"); + ret = -EINVAL; + goto env_uninit; + } + + ret = wd_cipher_set_key(sess->handle_cipher, cipher->key.data, cipher->key.length); + if (ret) { + wd_cipher_free_sess(sess->handle_cipher); + UADK_LOG(ERR, "uadk failed to set key!\n"); + ret = -EINVAL; + goto env_uninit; + } + + return 0; + +env_uninit: + wd_cipher_env_uninit(); + priv->env_cipher_init = false; + return ret; +} + static int uadk_crypto_sym_session_configure(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, struct rte_cryptodev_sym_session *session, struct rte_mempool *mp) { + struct rte_crypto_sym_xform *cipher_xform = NULL; struct uadk_crypto_session *sess; int ret; @@ -263,11 +454,23 @@ uadk_crypto_sym_session_configure(struct rte_cryptodev *dev, sess->chain_order = uadk_get_chain_order(xform); switch (sess->chain_order) { + case UADK_CHAIN_ONLY_CIPHER: + cipher_xform = xform; + break; default: ret = -ENOTSUP; goto err; } + if (cipher_xform) { + ret = uadk_set_session_cipher_parameters(dev, sess, cipher_xform); + if (ret != 0) { + UADK_LOG(ERR, + "Invalid/unsupported cipher parameters"); + goto err; + } + } + set_sym_session_private_data(session, dev->driver_id, sess); return 0; @@ -288,6 +491,11 @@ uadk_crypto_sym_session_clear(struct rte_cryptodev *dev, return; } + if (priv_sess->handle_cipher) { + wd_cipher_free_sess(priv_sess->handle_cipher); + priv_sess->handle_cipher = 0; + } + set_sym_session_private_data(sess, dev->driver_id, NULL); rte_mempool_put(rte_mempool_from_obj(priv_sess), priv_sess); } @@ -307,12 +515,49 @@ static struct rte_cryptodev_ops uadk_crypto_pmd_ops = { .sym_session_clear = uadk_crypto_sym_session_clear, }; +static void +uadk_process_cipher_op(struct rte_crypto_op *op, + struct uadk_crypto_session *sess, + struct rte_mbuf *msrc, struct rte_mbuf *mdst) +{ + uint32_t off = op->sym->cipher.data.offset; + int ret; + + if (!sess) { + op->status = RTE_COMP_OP_STATUS_INVALID_ARGS; + return; + } + + sess->cipher.req.src = rte_pktmbuf_mtod_offset(msrc, uint8_t *, off); + sess->cipher.req.in_bytes = op->sym->cipher.data.length; + sess->cipher.req.dst = rte_pktmbuf_mtod_offset(mdst, uint8_t *, off); + sess->cipher.req.out_buf_bytes = sess->cipher.req.in_bytes; + sess->cipher.req.iv_bytes = sess->iv.length; + sess->cipher.req.iv = rte_crypto_op_ctod_offset(op, uint8_t *, + sess->iv.offset); + if (sess->cipher.direction == RTE_CRYPTO_CIPHER_OP_ENCRYPT) + sess->cipher.req.op_type = WD_CIPHER_ENCRYPTION; + else + sess->cipher.req.op_type = WD_CIPHER_DECRYPTION; + + do { + ret = wd_do_cipher_sync(sess->handle_cipher, &sess->cipher.req); + } while (ret == -WD_EBUSY); + + if (sess->cipher.req.out_buf_bytes > sess->cipher.req.in_bytes) + op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED; + + if (ret) + op->status = RTE_COMP_OP_STATUS_ERROR; +} + static uint16_t uadk_crypto_enqueue_burst(void *queue_pair, struct rte_crypto_op **ops, uint16_t nb_ops) { struct uadk_qp *qp = queue_pair; struct uadk_crypto_session *sess = NULL; + struct rte_mbuf *msrc, *mdst; struct rte_crypto_op *op; uint16_t enqd = 0; int i, ret; @@ -320,6 +565,8 @@ uadk_crypto_enqueue_burst(void *queue_pair, struct rte_crypto_op **ops, for (i = 0; i < nb_ops; i++) { op = ops[i]; op->status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED; + msrc = op->sym->m_src; + mdst = op->sym->m_dst ? op->sym->m_dst : op->sym->m_src; if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { if (likely(op->sym->session != NULL)) @@ -330,6 +577,9 @@ uadk_crypto_enqueue_burst(void *queue_pair, struct rte_crypto_op **ops, } switch (sess->chain_order) { + case UADK_CHAIN_ONLY_CIPHER: + uadk_process_cipher_op(op, sess, msrc, mdst); + break; default: op->status = RTE_CRYPTO_OP_STATUS_ERROR; break;