You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/faqs.md
+6-10Lines changed: 6 additions & 10 deletions
Original file line number
Diff line number
Diff line change
@@ -42,11 +42,11 @@ Neural Magic does not support training of deep learning models at this time. We
42
42
43
43
**Do you have version compatibility on TensorFlow?**
44
44
45
-
Our inference engine supports all versions of TensorFlow 1.x.; support for TensorFlow 2.0 is coming soon. We have specific performance improvements for SSD models currently for TensorFlow 1.12.
45
+
Our inference engine supports all versions of TensorFlow <= 2.0; support for the Keras API is through TensorFlow 2.0.
46
46
47
47
**Do you run on AMD hardware?**
48
48
49
-
The DeepSparse Engine is validated to work on x86 Intel (Haswell generation and later) and AMD CPUs running Linux. It is highly recommended to run on a CPU with AVX-512 instructions available for optimal algorithms to be enabled. Specific support details for some algorithms over different microarchitectures [is available](https://docs.neuralmagic.com/deepsparse/source/hardware.html).
49
+
The DeepSparse Engine is validated to work on x86 Intel (Haswell generation and later) and AMD CPUs running Linux. It is highly recommended to run on a CPU with AVX-512 instructions available for optimal algorithms to be enabled. Specific support details for some algorithms over different microarchitectures [is available](https://docs.neuralmagic.com/deepsparse/source/hardware.html).
50
50
51
51
We are open to opportunities to expand our support footprint for different CPU-based processor architectures, based on market adoption and deep learning use cases.
52
52
@@ -56,11 +56,11 @@ We currently do not support ARM and it’s on the Neural Magic roadmap; however,
56
56
57
57
**To what use cases is the Deep Sparse Platform best suited?**
58
58
59
-
We focus on the models and use cases related to computer vision due to cost sensitivity and both real time and throughput constraints. The belief now is GPUs are required for deployment.
59
+
We focus on the models and use cases related to computer vision and NLP due to cost sensitivity and both real time and throughput constraints. The belief now is GPUs are required for deployment.
60
60
61
61
**What types of models does Neural Magic support?**
62
62
63
-
Today, we offer support for CNN-based computer vision models, specifically classification and object detection model types. We are continuously adding models to [our supported model list and SparseZoo](https://docs.neuralmagic.com/sparsezoo). Additionally, we are investigating model architectures beyond computer vision such as NLP models like BERT.
63
+
Today, we offer support for CNN-based computer vision models, specifically classification and object detection model types. We are continuously adding models to [our supported model list and SparseZoo](https://docs.neuralmagic.com/sparsezoo). Additionally, we are investigating model architectures beyond computer vision. As of June 2021, NLP models like BERT are now available.
64
64
65
65
**Is dynamic shape supported?**
66
66
@@ -74,10 +74,6 @@ ___
74
74
75
75
## Benchmarking FAQs
76
76
77
-
**What is the average estimated savings for users??**
78
-
79
-
This will vary but, in some cases, we are seeing 3x-10x savings. Typically, we offer 5-6x more price performance than hardware accelerators.
80
-
81
77
**Do you have benchmarks to compare and contrast?**
82
78
83
79
Yes. Check out our [benchmark demo video](https://neuralmagic.com/blog/neural-magic-demo/) or [contact us](https://neuralmagic.com/contact/) to discuss your particular performance requirements. If you’d rather observe performance for yourself, [head over to the Neural Magic GitHub repo](https://github.com/neuralmagic) to check out our tools and generate your own benchmarks in your environment.
@@ -120,7 +116,7 @@ For example, Neural Magic has been successful in removing 90% of ResNet-50 weigh
120
116
121
117
**When does sparsification actually happen?**
122
118
123
-
In a scenario in which you want to sparsify and then run your own model in the DeepSparse Engine, you would first sparsify your model to achieve the desired level of performance and accuracy using Neural Magic’s Sparsify and SparseML tooling.
119
+
In a scenario in which you want to sparsify and then run your own model in the DeepSparse Engine, you would first sparsify your model to achieve the desired level of performance and accuracy using Neural Magic’s [Sparsify](https://docs.neuralmagic.com/sparseml/ and [SparseML](https://docs.neuralmagic.com/sparseml/) tooling.
124
120
125
121
**What does the sparsification process look like?**
126
122
@@ -136,7 +132,7 @@ For transfer learning, our tooling allows you to save the sparse architecture le
136
132
137
133
**Do you support INT8 and INT16 (quantized) operations?**
138
134
139
-
Currently, the DeepSparse Engine runs at FP32 and has some support for INT8. With the release of the Intel Cascade Lake generation chips and later, Intel CPUs now include VNNI instructions and support both INT8 and INT16 operations. On machines with VNNI support, the engine has INT8 support for the ONNX operators QLinearConv, QuantizeLinear, DequantizeLinear, and QLinearMatMul with constant weights. The DeepSparse Engine also supports 8-bit QLinearAdd, an ONNX Runtime custom operator.
135
+
Currently, the DeepSparse Engine runs at FP32 and has some support for INT8. With the release of the Intel Cascade Lake generation chips and later, Intel CPUs now include VNNI instructions and support both INT8 and INT16 operations. On machines with VNNI support, the engine has INT8 support for the ONNX operators QLinearConv, QuantizeLinear, DequantizeLinear, and QLinearMatMul with constant weights. The DeepSparse Engine also supports 8-bit QLinearAdd, an ONNX Runtime custom operator.
140
136
141
137
**Do you support FP16 (half precision) operations?**
Copy file name to clipboardExpand all lines: docs/source/getstarted.md
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -62,7 +62,7 @@ Our Sparsify and SparseML tools allow us to easily reach industry leading levels
62
62
<tr>
63
63
<td><strong>Use Cases (Domains)</strong>
64
64
</td>
65
-
<td>Image Classification, Object Detection
65
+
<td>Image Classification, Object Detection, NLP
66
66
</td>
67
67
</tr>
68
68
<tr>
@@ -75,7 +75,7 @@ Our Sparsify and SparseML tools allow us to easily reach industry leading levels
75
75
76
76
Today, we offer support for convolutional neural network-based computer vision models, specifically classification and object detection model types such as [the models in SparseZoo](https://docs.neuralmagic.com/sparsezoo/source/models.html).
77
77
78
-
We are continuously exploring models to add to our supported [model list](https://docs.neuralmagic.com/sparsezoo/source/models.html) and SparseZoo including model architectures beyond computer vision. Popular NLP models such as BERT are on the Neural Magic roadmap; [subscribe for updates](http://neuralmagic.com/subscribe).
78
+
We are continuously exploring models to add to our supported [model list](https://docs.neuralmagic.com/sparsezoo/source/models.html) and SparseZoo including model architectures beyond computer vision and NLP; [Subscribe for updates](http://neuralmagic.com/subscribe).
0 commit comments