Skip to content

Commit 405c468

Browse files
authored
Update faqs.md (#66)
* Update faqs.md * Update getstarted.md Inclusion of NLP support Removing a question that could use a better response until GTM alignment is made.
1 parent 40cbca9 commit 405c468

File tree

2 files changed

+8
-12
lines changed

2 files changed

+8
-12
lines changed

docs/source/faqs.md

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -42,11 +42,11 @@ Neural Magic does not support training of deep learning models at this time. We
4242

4343
**Do you have version compatibility on TensorFlow?**
4444

45-
Our inference engine supports all versions of TensorFlow 1.x.; support for TensorFlow 2.0 is coming soon. We have specific performance improvements for SSD models currently for TensorFlow 1.12.
45+
Our inference engine supports all versions of TensorFlow <= 2.0; support for the Keras API is through TensorFlow 2.0.
4646

4747
**Do you run on AMD hardware?**
4848

49-
The DeepSparse Engine is validated to work on x86 Intel (Haswell generation and later) and AMD CPUs running Linux. It is highly recommended to run on a CPU with AVX-512 instructions available for optimal algorithms to be enabled. Specific support details for some algorithms over different microarchitectures [is available](https://docs.neuralmagic.com/deepsparse/source/hardware.html).
49+
The DeepSparse Engine is validated to work on x86 Intel (Haswell generation and later) and AMD CPUs running Linux. It is highly recommended to run on a CPU with AVX-512 instructions available for optimal algorithms to be enabled. Specific support details for some algorithms over different microarchitectures [is available](https://docs.neuralmagic.com/deepsparse/source/hardware.html).
5050

5151
We are open to opportunities to expand our support footprint for different CPU-based processor architectures, based on market adoption and deep learning use cases.
5252

@@ -56,11 +56,11 @@ We currently do not support ARM and it’s on the Neural Magic roadmap; however,
5656

5757
**To what use cases is the Deep Sparse Platform best suited?**
5858

59-
We focus on the models and use cases related to computer vision due to cost sensitivity and both real time and throughput constraints. The belief now is GPUs are required for deployment.
59+
We focus on the models and use cases related to computer vision and NLP due to cost sensitivity and both real time and throughput constraints. The belief now is GPUs are required for deployment.
6060

6161
**What types of models does Neural Magic support?**
6262

63-
Today, we offer support for CNN-based computer vision models, specifically classification and object detection model types. We are continuously adding models to [our supported model list and SparseZoo](https://docs.neuralmagic.com/sparsezoo). Additionally, we are investigating model architectures beyond computer vision such as NLP models like BERT.
63+
Today, we offer support for CNN-based computer vision models, specifically classification and object detection model types. We are continuously adding models to [our supported model list and SparseZoo](https://docs.neuralmagic.com/sparsezoo). Additionally, we are investigating model architectures beyond computer vision. As of June 2021, NLP models like BERT are now available.
6464

6565
**Is dynamic shape supported?**
6666

@@ -74,10 +74,6 @@ ___
7474

7575
## Benchmarking FAQs
7676

77-
**What is the average estimated savings for users??**
78-
79-
This will vary but, in some cases, we are seeing 3x-10x savings. Typically, we offer 5-6x more price performance than hardware accelerators.
80-
8177
**Do you have benchmarks to compare and contrast?**
8278

8379
Yes. Check out our [benchmark demo video](https://neuralmagic.com/blog/neural-magic-demo/) or [contact us](https://neuralmagic.com/contact/) to discuss your particular performance requirements. If you’d rather observe performance for yourself, [head over to the Neural Magic GitHub repo](https://github.com/neuralmagic) to check out our tools and generate your own benchmarks in your environment.
@@ -120,7 +116,7 @@ For example, Neural Magic has been successful in removing 90% of ResNet-50 weigh
120116

121117
**When does sparsification actually happen?**
122118

123-
In a scenario in which you want to sparsify and then run your own model in the DeepSparse Engine, you would first sparsify your model to achieve the desired level of performance and accuracy using Neural Magic’s Sparsify and SparseML tooling.
119+
In a scenario in which you want to sparsify and then run your own model in the DeepSparse Engine, you would first sparsify your model to achieve the desired level of performance and accuracy using Neural Magic’s [Sparsify](https://docs.neuralmagic.com/sparseml/ and [SparseML](https://docs.neuralmagic.com/sparseml/) tooling.
124120

125121
**What does the sparsification process look like?**
126122

@@ -136,7 +132,7 @@ For transfer learning, our tooling allows you to save the sparse architecture le
136132

137133
**Do you support INT8 and INT16 (quantized) operations?**
138134

139-
Currently, the DeepSparse Engine runs at FP32 and has some support for INT8. With the release of the Intel Cascade Lake generation chips and later, Intel CPUs now include VNNI instructions and support both INT8 and INT16 operations. On machines with VNNI support, the engine has INT8 support for the ONNX operators QLinearConv, QuantizeLinear, DequantizeLinear, and QLinearMatMul with constant weights. The DeepSparse Engine also supports 8-bit QLinearAdd, an ONNX Runtime custom operator.
135+
Currently, the DeepSparse Engine runs at FP32 and has some support for INT8. With the release of the Intel Cascade Lake generation chips and later, Intel CPUs now include VNNI instructions and support both INT8 and INT16 operations. On machines with VNNI support, the engine has INT8 support for the ONNX operators QLinearConv, QuantizeLinear, DequantizeLinear, and QLinearMatMul with constant weights. The DeepSparse Engine also supports 8-bit QLinearAdd, an ONNX Runtime custom operator.
140136

141137
**Do you support FP16 (half precision) operations?**
142138

docs/source/getstarted.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ Our Sparsify and SparseML tools allow us to easily reach industry leading levels
6262
<tr>
6363
<td><strong>Use Cases (Domains)</strong>
6464
</td>
65-
<td>Image Classification, Object Detection
65+
<td>Image Classification, Object Detection, NLP
6666
</td>
6767
</tr>
6868
<tr>
@@ -75,7 +75,7 @@ Our Sparsify and SparseML tools allow us to easily reach industry leading levels
7575

7676
Today, we offer support for convolutional neural network-based computer vision models, specifically classification and object detection model types such as [the models in SparseZoo](https://docs.neuralmagic.com/sparsezoo/source/models.html).
7777

78-
We are continuously exploring models to add to our supported [model list](https://docs.neuralmagic.com/sparsezoo/source/models.html) and SparseZoo including model architectures beyond computer vision. Popular NLP models such as BERT are on the Neural Magic roadmap; [subscribe for updates](http://neuralmagic.com/subscribe).
78+
We are continuously exploring models to add to our supported [model list](https://docs.neuralmagic.com/sparsezoo/source/models.html) and SparseZoo including model architectures beyond computer vision and NLP; [Subscribe for updates](http://neuralmagic.com/subscribe).
7979

8080
### Notes
8181

0 commit comments

Comments
 (0)