Skip to content

Commit 450c3eb

Browse files
sallyseoksallyseok
authored
[PyTorch][Inference][Graviton] PT2.3 Inference Currency for Graviton ec2/sm (#4016)
* initial commit * add allowlists, pin numpy<2 * allowlist torch
1 parent a0c2c77 commit 450c3eb

7 files changed

+515
-1
lines changed

‎pytorch/inference/buildspec-graviton-2-3.yml

+67
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
account_id: &ACCOUNT_ID <set-$ACCOUNT_ID-in-environment>
2+
prod_account_id: &PROD_ACCOUNT_ID 763104351884
3+
region: &REGION <set-$REGION-in-environment>
4+
framework: &FRAMEWORK pytorch
5+
version: &VERSION 2.3.0
6+
short_version: &SHORT_VERSION "2.3"
7+
arch_type: graviton
8+
# autopatch_build: "True"
9+
10+
repository_info:
11+
inference_repository: &INFERENCE_REPOSITORY
12+
image_type: &INFERENCE_IMAGE_TYPE inference
13+
root: !join [ *FRAMEWORK, "/", *INFERENCE_IMAGE_TYPE ]
14+
repository_name: &REPOSITORY_NAME !join [pr, "-", *FRAMEWORK, "-", *INFERENCE_IMAGE_TYPE, "-", graviton]
15+
repository: &REPOSITORY !join [ *ACCOUNT_ID, .dkr.ecr., *REGION, .amazonaws.com/, *REPOSITORY_NAME ]
16+
release_repository_name: &RELEASE_REPOSITORY_NAME !join [ *FRAMEWORK, "-", *INFERENCE_IMAGE_TYPE, "-", graviton]
17+
release_repository: &RELEASE_REPOSITORY !join [ *PROD_ACCOUNT_ID, .dkr.ecr., *REGION, .amazonaws.com/, *RELEASE_REPOSITORY_NAME ]
18+
19+
context:
20+
inference_context: &INFERENCE_CONTEXT
21+
torchserve-ec2-entrypoint:
22+
source: docker/build_artifacts/torchserve-ec2-entrypoint.py
23+
target: torchserve-ec2-entrypoint.py
24+
torchserve-entrypoint:
25+
source: docker/build_artifacts/torchserve-entrypoint.py
26+
target: torchserve-entrypoint.py
27+
deep_learning_container:
28+
source: ../../src/deep_learning_container.py
29+
target: deep_learning_container.py
30+
config:
31+
source: docker/build_artifacts/config.properties
32+
target: config.properties
33+
34+
images:
35+
BuildEC2PytorchGravitonCPUInferencePy3DockerImage:
36+
<<: *INFERENCE_REPOSITORY
37+
build: &PYTORCH_CPU_INFERENCE_PY3 false
38+
image_size_baseline: 10000
39+
device_type: &DEVICE_TYPE cpu
40+
os_version: &OS_VERSION ubuntu20.04
41+
torch_serve_version: &TORCHSERVE_VERSION 0.11.0
42+
python_version: &DOCKER_PYTHON_VERSION py3
43+
tag_python_version: &TAG_PYTHON_VERSION py311
44+
tag: !join [ *VERSION, "-", *DEVICE_TYPE, "-", *TAG_PYTHON_VERSION, "-", *OS_VERSION , "-ec2"]
45+
# latest_release_tag: !join [ *VERSION, "-", *DEVICE_TYPE, "-", *TAG_PYTHON_VERSION, "-", *OS_VERSION, "-ec2" ]
46+
docker_file: !join [ docker/, *SHORT_VERSION, /, *DOCKER_PYTHON_VERSION, /Dockerfile.graviton., *DEVICE_TYPE ]
47+
target: ec2
48+
# build_tag_override: "beta:2.3.0-cpu-py311-ubuntu20.04-ec2"
49+
context:
50+
<<: *INFERENCE_CONTEXT
51+
BuildSageMakerPytorchGravitonCPUInferencePy3DockerImage:
52+
<<: *INFERENCE_REPOSITORY
53+
build: &PYTORCH_CPU_INFERENCE_PY3 false
54+
image_size_baseline: 10000
55+
device_type: &DEVICE_TYPE cpu
56+
os_version: &OS_VERSION ubuntu20.04
57+
torch_serve_version: &TORCHSERVE_VERSION 0.11.0
58+
tool_kit_version: &SM_TOOLKIT_VERSION 2.0.24
59+
python_version: &DOCKER_PYTHON_VERSION py3
60+
tag_python_version: &TAG_PYTHON_VERSION py311
61+
tag: !join [ *VERSION, "-", *DEVICE_TYPE, "-", *TAG_PYTHON_VERSION, "-", *OS_VERSION , "-sagemaker"]
62+
# latest_release_tag: !join [ *VERSION, "-", *DEVICE_TYPE, "-", *TAG_PYTHON_VERSION, "-", *OS_VERSION, "-sagemaker" ]
63+
docker_file: !join [ docker/, *SHORT_VERSION, /, *DOCKER_PYTHON_VERSION, /Dockerfile.graviton., *DEVICE_TYPE ]
64+
target: sagemaker
65+
# build_tag_override: "beta:2.3.0-cpu-py311-ubuntu20.04-sagemaker"
66+
context:
67+
<<: *INFERENCE_CONTEXT

‎pytorch/inference/buildspec-graviton.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
buildspec_pointer: buildspec-graviton-2-2.yml
1+
buildspec_pointer: buildspec-graviton-2-3.yml

‎pytorch/inference/docker/2.3/py3/Dockerfile.ec2.graviton.cpu.os_scan_allowlist.json

+91
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
{
2+
"linux": [
3+
{
4+
"description": " In the Linux kernel, the following vulnerability has been resolved: bpf: Fix hashtab overflow check on 32-bit arches The hashtab code relies on roundup_pow_of_two() to compute the number of hash buckets, and contains an overflow check by checking if the resulting value is 0. However, on 32-bit arches, the roundup code itself can overflow by doing a 32-bit left-shift of an unsigned long value, which is undefined behaviour, so it is not guaranteed to truncate neatly. This was triggered by syzbot on the DEVMAP_HASH type, which contains the same check, copied from the hashtab code. So apply the same fix to hashtab, by moving the overflow check to before the roundup.",
5+
"vulnerability_id": "CVE-2024-26884",
6+
"name": "CVE-2024-26884",
7+
"package_name": "linux",
8+
"package_details": {
9+
"file_path": null,
10+
"name": "linux",
11+
"package_manager": "OS",
12+
"version": "5.4.0",
13+
"release": "187.207"
14+
},
15+
"remediation": {
16+
"recommendation": {
17+
"text": "None Provided"
18+
}
19+
},
20+
"cvss_v3_score": 7.8,
21+
"cvss_v30_score": 0,
22+
"cvss_v31_score": 7.8,
23+
"cvss_v2_score": 0,
24+
"cvss_v3_severity": "HIGH",
25+
"source_url": "https://people.canonical.com/~ubuntu-security/cve/2024/CVE-2024-26884.html",
26+
"source": "UBUNTU_CVE",
27+
"severity": "HIGH",
28+
"status": "ACTIVE",
29+
"title": "CVE-2024-26884 - linux",
30+
"reason_to_ignore": "N/A"
31+
},
32+
{
33+
"description": " In the Linux kernel, the following vulnerability has been resolved: net: ip_tunnel: make sure to pull inner header in ip_tunnel_rcv() Apply the same fix than ones found in : 8d975c15c0cd (\"ip6_tunnel: make sure to pull inner header in __ip6_tnl_rcv()\") 1ca1ba465e55 (\"geneve: make sure to pull inner header in geneve_rx()\") We have to save skb->network_header in a temporary variable in order to be able to recompute the network_header pointer after a pskb_inet_may_pull() call. pskb_inet_may_pull() makes sure the needed headers are in skb->head. syzbot reported: BUG: KMSAN: uninit-value in __INET_ECN_decapsulate include/net/inet_ecn.h:253 [inline] BUG: KMSAN: uninit-value in INET_ECN_decapsulate include/net/inet_ecn.h:275 [inline] BUG: KMSAN: uninit-value in IP_ECN_decapsulate include/net/inet_ecn.h:302 [inline] BUG: KMSAN: uninit-value in ip_tunnel_rcv+0xed9/0x2ed0 net/ipv4/ip_tunnel.c:409 __INET_ECN_decapsulate include/net/inet_ecn.h:253 [inline] INET_ECN_decapsulate include/net/inet_ecn.h:275 [inline] IP_ECN_",
34+
"vulnerability_id": "CVE-2024-26882",
35+
"name": "CVE-2024-26882",
36+
"package_name": "linux",
37+
"package_details": {
38+
"file_path": null,
39+
"name": "linux",
40+
"package_manager": "OS",
41+
"version": "5.4.0",
42+
"release": "187.207"
43+
},
44+
"remediation": {
45+
"recommendation": {
46+
"text": "None Provided"
47+
}
48+
},
49+
"cvss_v3_score": 7.8,
50+
"cvss_v30_score": 0,
51+
"cvss_v31_score": 7.8,
52+
"cvss_v2_score": 0,
53+
"cvss_v3_severity": "HIGH",
54+
"source_url": "https://people.canonical.com/~ubuntu-security/cve/2024/CVE-2024-26882.html",
55+
"source": "UBUNTU_CVE",
56+
"severity": "HIGH",
57+
"status": "ACTIVE",
58+
"title": "CVE-2024-26882 - linux",
59+
"reason_to_ignore": "N/A"
60+
},
61+
{
62+
"description": " In the Linux kernel, the following vulnerability has been resolved: aoe: fix the potential use-after-free problem in aoecmd_cfg_pkts This patch is against CVE-2023-6270. The description of cve is: A flaw was found in the ATA over Ethernet (AoE) driver in the Linux kernel. The aoecmd_cfg_pkts() function improperly updates the refcnt on `struct net_device`, and a use-after-free can be triggered by racing between the free on the struct and the access through the `skbtxq` global queue. This could lead to a denial of service condition or potential code execution. In aoecmd_cfg_pkts(), it always calls dev_put(ifp) when skb initial code is finished. But the net_device ifp will still be used in later tx()->dev_queue_xmit() in kthread. Which means that the dev_put(ifp) should NOT be called in the success path of skb initial code in aoecmd_cfg_pkts(). Otherwise tx() may run into use-after-free because the net_device is freed. This patch removed the dev_put(ifp) in the success path in aoecmd_cfg_pkts(), and added dev_p",
63+
"vulnerability_id": "CVE-2024-26898",
64+
"name": "CVE-2024-26898",
65+
"package_name": "linux",
66+
"package_details": {
67+
"file_path": null,
68+
"name": "linux",
69+
"package_manager": "OS",
70+
"version": "5.4.0",
71+
"release": "187.207"
72+
},
73+
"remediation": {
74+
"recommendation": {
75+
"text": "None Provided"
76+
}
77+
},
78+
"cvss_v3_score": 7.8,
79+
"cvss_v30_score": 0,
80+
"cvss_v31_score": 7.8,
81+
"cvss_v2_score": 0,
82+
"cvss_v3_severity": "HIGH",
83+
"source_url": "https://people.canonical.com/~ubuntu-security/cve/2024/CVE-2024-26898.html",
84+
"source": "UBUNTU_CVE",
85+
"severity": "HIGH",
86+
"status": "ACTIVE",
87+
"title": "CVE-2024-26898 - linux",
88+
"reason_to_ignore": "N/A"
89+
}
90+
]
91+
}

‎pytorch/inference/docker/2.3/py3/Dockerfile.ec2.graviton.cpu.py_scan_allowlist.json

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"65213": "From OpenSSL: Due to the low severity of this issue we are not issuing new releases of OpenSSL at this time. The fix will be included in the next releases when they become available.",
3+
"70612": "This Jinja2 CVE is disputed and has no fix.",
4+
"71670": "torch CVE for versions <2.2.2 and there is no fix"
5+
}

0 commit comments

Comments
 (0)