+
+
+## Installation
+### Release Binary Install
```bash
-pip install -r requirements.txt
+pip install intel-extension-for-transformers
```
-#### Install NLP Toolkit
+### Install From Source
+#### Install Intel® Extension for Transformers
```bash
-git clone https://github.com/intel-innersource/frameworks.ai.nlp-toolkit.intel-nlp-toolkit.git nlp_toolkit
-cd nlp_toolkit
+git clone https://github.com/intel/intel-extension-for-transformers.git intel_extension_for_transformers
+cd intel_extension_for_transformers
+# Install Dependency
+pip install -r requirements.txt
git submodule update --init --recursive
+# Install intel_extension_for_transformers
python setup.py install
```
+## Getting Started
### Quantization
```python
-from nlp_toolkit import QuantizationConfig, metric, objectives
-from nlp_toolkit.optimization.trainer import NLPTrainer
+from intel_extension_for_transformers import QuantizationConfig, metric, objectives
+from intel_extension_for_transformers.optimization.trainer import NLPTrainer
# Replace transformers.Trainer with NLPTrainer
# trainer = transformers.Trainer(...)
@@ -50,8 +63,8 @@ Please refer to [quantization document](docs/quantization.md) for more details.
### Pruning
```python
-from nlp_toolkit import PrunerConfig, PruningConfig
-from nlp_toolkit.optimization.trainer import NLPTrainer
+from intel_extension_for_transformers import PrunerConfig, PruningConfig
+from intel_extension_for_transformers.optimization.trainer import NLPTrainer
# Replace transformers.Trainer with NLPTrainer
# trainer = transformers.Trainer(...)
@@ -66,8 +79,8 @@ Please refer to [pruning document](docs/pruning.md) for more details.
### Distillation
```python
-from nlp_toolkit import DistillationConfig, Criterion
-from nlp_toolkit.optimization.trainer import NLPTrainer
+from intel_extension_for_transformers import DistillationConfig, Criterion
+from intel_extension_for_transformers.optimization.trainer import NLPTrainer
# Replace transformers.Trainer with NLPTrainer
# trainer = transformers.Trainer(...)
@@ -84,7 +97,7 @@ Please refer to [distillation document](docs/distillation.md) for more details.
Data augmentation provides the facilities to generate synthesized NLP dataset for further model optimization. The data augmentation supports text generation on popular fine-tuned models like GPT, GPT2, and other text synthesis approaches from [nlpaug](https://github.com/makcedward/nlpaug).
```python
-from nlp_toolkit.preprocessing.data_augmentation import DataAugmentation
+from intel_extension_for_transformers.preprocessing.data_augmentation import DataAugmentation
aug = DataAugmentation(augmenter_type="TextGenerationAug")
aug.input_dataset = "original_dataset.csv" # example: https://huggingface.co/datasets/glue/viewer/sst2/train
aug.column_names = "sentence"
@@ -97,10 +110,10 @@ raw_datasets = load_dataset("csv", data_files=aug.output_path, delimiter="\t", s
Please refer to [data augmentation document](docs/data_augmentation.md) for more details.
### Neural Engine
-Neural Engine is one of reference deployments that NLP toolkit provides. Neural Engine aims to demonstrate the optimal performance of extremely compressed NLP models by exploring the optimization opportunities from both HW and SW.
+Neural Engine is one of reference deployments that Intel Extension for Transformers provides. Neural Engine aims to demonstrate the optimal performance of extremely compressed NLP models by exploring the optimization opportunities from both HW and SW.
```python
-from nlp_toolkit.backends.neural_engine.compile import compile
+from intel_extension_for_transformers.backends.neural_engine.compile import compile
# /path/to/your/model is a TensorFlow pb model or ONNX model
model = compile('/path/to/your/model')
inputs = ... # [input_ids, segment_ids, input_mask]
@@ -112,8 +125,8 @@ Please refer to [Neural Engine](examples/deployment/) for more details.
### Quantized Length Adaptive Transformer
Quantized Length Adaptive Transformer leverages sequence-length reduction and low-bit representation techniques to further enhance model inference performance, enabling adaptive sequence-length sizes to accommodate different computational budget requirements with an optimal accuracy efficiency tradeoff.
```python
-from nlp_toolkit import QuantizationConfig, DynamicLengthConfig, metric, objectives
-from nlp_toolkit.optimization.trainer import NLPTrainer
+from intel_extension_for_transformers import QuantizationConfig, DynamicLengthConfig, metric, objectives
+from intel_extension_for_transformers.optimization.trainer import NLPTrainer
# Replace transformers.Trainer with NLPTrainer
# trainer = transformers.Trainer(...)
@@ -131,4 +144,77 @@ trainer.set_dynamic_config(dynamic_config=dynamic_length_config)
model = trainer.quantize(quant_config=q_config)
```
-Please refer to paper [QuaLA-MiniLM](https://arxiv.org/pdf/2210.17114.pdf) and [code](examples/optimization/pytorch/huggingface/question-answering/dynamic) for details
\ No newline at end of file
+Please refer to paper [QuaLA-MiniLM](https://arxiv.org/pdf/2210.17114.pdf) and [code](examples/optimization/pytorch/huggingface/question-answering/dynamic) for details
+
+
+### Transformers-accelerated Neural Engine
+Transformers-accelerated Neural Engine is one of reference deployments that Intel® Extension for Transformers provides. Neural Engine aims to demonstrate the optimal performance of extremely compressed NLP models by exploring the optimization opportunities from both HW and SW.
+
+```python
+from intel_extension_for_transformers.backends.neural_engine.compile import compile
+# /path/to/your/model is a TensorFlow pb model or ONNX model
+model = compile('/path/to/your/model')
+inputs = ... # [input_ids, segment_ids, input_mask]
+model.inference(inputs)
+```
+
+Please refer to [example](examples/deployment/neural_engine/sparse/distilbert_base_uncased/) in [Transformers-accelerated Neural Engine](examples/deployment/) and paper [Fast Distilbert on CPUs](https://arxiv.org/abs/2211.07715) for more details.
+
+### Transformers-accelerated Libraries
+Transformers-accelerated Libraries is a high-performance operator computing library implemented by assembly. Transformers-accelerated Libraries contains a JIT domain, a kernel domain, and a scheduling proxy framework.
+
+```C++
+#include "interface.hpp"
+ ...
+ operator_desc op_desc(ker_kind, ker_prop, eng_kind, ts_descs, op_attrs);
+ sparse_matmul_desc spmm_desc(op_desc);
+ sparse_matmul spmm_kern(spmm_desc);
+ std::vector| Framework | +TensorFlow | +Intel TensorFlow | +PyTorch | +IPEX | +
|---|---|---|---|---|
| Version | +2.10.0 + 2.9.1 + | 2.10.0 + 2.9.1 + | 1.13.0+cpu + 1.12.0+cpu + | 1.13.0 + 1.12.0 |
+
+ EULA: Apache 2.0
+ Third Party Programs: https://github.com/intel/intel_extension_for_transformers/blob/1.0a/third_party_programs.txt +
+ Intel® Extension for Transformers. + ' diff --git a/docs/AutoDistillation design.md b/docs/AutoDistillation design.md index 664cd1c9541..ec529056d0b 100644 --- a/docs/AutoDistillation design.md +++ b/docs/AutoDistillation design.md @@ -11,7 +11,7 @@ In Model Exploration, a search engine will search for a better compressed model Flash Distillation is the stage for training the searched model to discover its potential.
-In Evaluation stage, trained model will be evaluated to measure its performances (e.g. the prediction accuracy, the hardware performance etc.) inorder to select the best model architecture. +In Evaluation stage, the trained model will be evaluated to measure its performances (e.g. the prediction accuracy, the hardware performance etc.) in order to select the best model architecture.
For implementing AutoDistillation, a framework class ***'AutoDistillation'*** is designed for excuting the total pipeline, and a criterion class ***'IntermediateLayersKnowledgeDistillationLoss'*** is designed for handling Flash Distillation with existing Distillation class. @@ -34,19 +34,19 @@ Within each iteration, ***model_arch_proposition*** method will propose a promis **1. search_space** (e.g. {'hidden_size':[64, 128], 'layer_num':[4, 8]})
-**2. model_builder** (function for building model instance based on the specific sample point in the search space, ***need provided by user***) +**2. model_builder** (the function for building model instance based on the specific sample point in the search space, ***need provided by user***)
-**3. advisor** (search algorithm instance e.g. Bayesian Optimization, Random Search) +**3. advisor** (the search algorithm instance e.g. Bayesian Optimization, Random Search)
-**4. train_func** (train function to train the model) +**4. train_func** (the train function to train the model)
-**5. eval_func** (evaluation function to evaluate the model) +**5. eval_func** (the evaluation function to evaluate the model)
-**6. config** (configuration, ***need provided by user***) +**6. config** (the configuration, ***need provided by user***)
**7. search_result** (store results of the search process) @@ -62,7 +62,7 @@ Within each iteration, ***model_arch_proposition*** method will propose a promis **2. search_loop** (begin search iterations)
-**3. train_evaluate** (process of one search iteration to train and evaluate the model proposed by search algorithm) +**3. train_evaluate** (the process of one search iteration to train and evaluate the model proposed by search algorithm) ## **3. Criterion Class Design** ***IntermediateLayersKnowledgeDistillationLoss*** is designed for calculating the knowledge distillation loss of the intermediate layer features. @@ -71,7 +71,7 @@ Within each iteration, ***model_arch_proposition*** method will propose a promis To deal the issue of dimension mismatch between the intermediate layer features of the teacher model and the student model, feature_matchers is provided for matching the features dimension.
-For example, shape of a feature from the teacher model is (8, 512), shape of a corresponding feature from the student model is (8, 128), then feature_matcher will be a linear transformation layer whose weight has a shape of (128, 512). +For example, the shape of a feature from the teacher model is (8, 512), and the shape of a corresponding feature from the student model is (8, 128), then the feature_matcher will be a linear transformation layer whose weight has a shape of (128, 512). ### **Class IntermediateLayersKnowledgeDistillationLoss** @@ -111,7 +111,7 @@ For example, shape of a feature from the teacher model is (8, 512), shape of a c **2. init_feature_matcher** (initialize the feature_matcher instance)
-**3. teacher_model_forward** (run forward for teacher_model) +**3. teacher_model_forward** (run forward for the teacher_model)
**4. loss_cal** (calculate loss) @@ -252,7 +252,7 @@ trainer.autodistillation_config = { best_model_arch = trainer.autodistillation(teacher_model, model_builder, train_func=train_func, eval_func=eval_func) ``` -### **flash distillation config example** +### **Flash distillation config example** ```yaml model: @@ -275,7 +275,7 @@ distillation: add_origin_loss: False ``` -### **regular distillation config example** +### **Regular distillation config example** ```yaml model: diff --git a/docs/benchmark.md b/docs/benchmark.md index 8bb12e9bb0b..158fe2a7308 100644 --- a/docs/benchmark.md +++ b/docs/benchmark.md @@ -1,6 +1,6 @@ # Benchmark The Benchmark is inherited from transformers [Benchmark] -(https://github.com/huggingface/transformers/blob/main/docs/source/en/benchmarks.mdx). Right now, The classes `PyTorchBenchmark` and `ExecutorBenchmark` can help users to check performance benefit and model size benefit of FP32 and INT8 model on PyTorch and Executor backends. +(https://github.com/huggingface/transformers/blob/main/docs/source/en/benchmarks.mdx). Right now, The classes `PyTorchBenchmark` and `ExecutorBenchmark` can help users to check the performance benefit and the model size benefit of FP32 and INT8 model on PyTorch and Executor backends. The benchmark classes [PyTorchBenchmark] and [ExecutorBenchmark] expect an object of type [PyTorchBenchmarkArguments] and [ExecutorBenchmarkArguments]. @@ -8,10 +8,10 @@ The benchmark classes [PyTorchBenchmark] and [ExecutorBenchmark] expect an objec ## PyTorchBenchmark -The PyTorchBenchmark is only for inference when the input model is an INT8 model. +The PyTorchBenchmark is only used for inference when the input model is an INT8 model. ```py -from nlp_toolkit.optimization.benchmark import PyTorchBenchmark, PyTorchBenchmarkArguments +from intel_extension_for_transformers.optimization.benchmark import PyTorchBenchmark, PyTorchBenchmarkArguments MODEL_ID_FP32 = "distilbert-base-uncased-finetuned-sst-2-english" MODEL_ID_INT8 = "Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static" @@ -55,10 +55,10 @@ Intel/distilbert-base-uncased- 1 128 0.017 ## ExecutorBenchmark -The ExecutorBenchmark is only for inference. The ONNX model is generated by our `export_to_onnx` API. For more details, please [go to](export.md). The MODEL_NAME is the pytorch model name you used for exporting the ONNX model. +The ExecutorBenchmark is only used for inference. The ONNX model is generated by our `export_to_onnx` API. For more details, please [go to](export.md). The MODEL_NAME is the pytorch model name you used for exporting the ONNX model. ```py -from nlp_toolkit.optimization.benchmark import ExecutorBenchmark, ExecutorBenchmarkArguments +from intel_extension_for_transformers.optimization.benchmark import ExecutorBenchmark, ExecutorBenchmarkArguments from transformers import AutoConfig MODEL_ID_FP32 = 'fp32.onnx' diff --git a/docs/data_augmentation.md b/docs/data_augmentation.md index 99d25df14e9..2504ee9627a 100644 --- a/docs/data_augmentation.md +++ b/docs/data_augmentation.md @@ -1,5 +1,5 @@ # Data Augmentation: The Tool for Augmenting NLP Datasets -Data Augmentation is a tool to helps you with augmenting nlp datasets for your machine learning projects. The tool integrated [nlpaug](https://github.com/makcedward/nlpaug) and other methods from Intel Lab. +Data Augmentation is a tool to helps you with augmenting nlp datasets for your machine learning projects. This tool integrates [nlpaug](https://github.com/makcedward/nlpaug) and other methods from Intel Lab. ## Getting Started! ### Installation @@ -7,16 +7,16 @@ Data Augmentation is a tool to helps you with augmenting nlp datasets for your m pip install nlpaug pip install transformers>=4.12.0 -#### Install Nlp-toolkit -git clone https://github.com/intel-innersource/frameworks.ai.nlp-toolkit.intel-nlp-toolkit.git nlp_toolkit -cd nlp_toolkit +#### Install Intel_Extension_for_Transformers +git clone https://github.com/intel/intel-extension-for-transformers.git intel_extension_for_transformers +cd intel_extension_for_transformers git submodule update --init --recursive python setup.py install ### Data Augmentation #### Script(Please refer to [example](tests/test_data_augmentation.py)) ```python - from nlp_toolkit.preprocessing.data_augmentation import DataAugmentation + from intel_extension_for_transformers.preprocessing.data_augmentation import DataAugmentation aug = DataAugmentation(augmenter_type="TextGenerationAug") aug.input_dataset = "dev.csv" aug.output_path = os.path.join(self.result_path, "test1.cvs") @@ -49,7 +49,7 @@ python setup.py install |"ContextualWordEmbsForSentenceAug"|refer to ["ContextualWordEmbsForSentenceAug"](https://github.com/makcedward/nlpaug/blob/40794970124c26ce2e587e567738247bf20ebcad/nlpaug/augmenter/sentence/context_word_embs_sentence.py#L77) | | #### Text Generation Augmenter -The text generation augment contains the recipe to run data augmentation algorithm based on conditional text generation using auto-regressive transformer model (like GPT, GPT-2, Transformer-XL, XLNet, CTRL) in order to automatically generate labeled data. +The text generation augment contains the recipe to run data augmentation algorithm based on the conditional text generation using auto-regressive transformer model (like GPT, GPT-2, Transformer-XL, XLNet, CTRL) in order to automatically generate labeled data. Our approach follows algorithms described by [Not Enough Data? Deep Learning to the Rescue!](https://arxiv.org/abs/1911.03118) and [Natural Language Generation for Effective Knowledge Distillation](https://www.aclweb.org/anthology/D19-6122.pdf). - First, we fine-tune an auto-regressive model on the training set. Each sample contains both the label and the sentence. @@ -58,7 +58,7 @@ Our approach follows algorithms described by [Not Enough Data? Deep Learning to example: ```python from datasets import load_dataset - from nlp_toolkit.preprocessing.utils import EOS + from intel_extension_for_transformers.preprocessing.utils import EOS for split in {'train', 'validation'}: dataset = load_dataset('glue', 'sst2', split=split) with open('SST-2/' + split + '.txt', 'w') as fw: @@ -66,9 +66,9 @@ Our approach follows algorithms described by [Not Enough Data? Deep Learning to fw.write(str(d['label']) + '\t' + d['sentence'] + EOS + '\n') ``` - - Fine-tuning Causal Language Model + - Fine-tune Causal Language Model - You can use the script [run_clm.py](https://github.com/huggingface/transformers/tree/v4.6.1/examples/pytorch/language-modeling/run_clm.py) from transformers examples for fine-tuning GPT2 (gpt2-medium) on SST-2. The loss is that of causal language modeling. + You can use the script [run_clm.py](https://github.com/huggingface/transformers/tree/v4.6.1/examples/pytorch/language-modeling/run_clm.py) from transformers examples for fine-tuning GPT2 (gpt2-medium) on SST-2 task. The loss is that of causal language modeling. ```shell DATASET=SST-2 @@ -90,7 +90,7 @@ Our approach follows algorithms described by [Not Enough Data? Deep Learning to - Second, we generate labeled data. Given class labels sampled from the training set, we use the fine-tuned language model to predict sentences with below script: ```python - from nlp_toolkit.preprocessing.data_augmentation import DataAugmentation + from intel_extension_for_transformers.preprocessing.data_augmentation import DataAugmentation aug = DataAugmentation(augmenter_type="TextGenerationAug") aug.input_dataset = "/your/original/training_set.csv" aug.output_path = os.path.join(self.result_path, "/your/augmented/dataset.cvs") @@ -104,8 +104,8 @@ This data augmentation algorithm can be used in several scenarios, like model di augmenter_arguments: |parameter |Type|Description |default value | |:---------|:---|:---------------------------------------------------|:-------------| -|"model_name_or_path"|String|Language modeling model to generate data, refer to [line](nlp_toolkit/preprocessing/data_augmentation.py#L181)|NA| -|"stop_token"|String|Stop token used in input data file |[EOS](nlp_toolkit/preprocessing/utils.py#L7)| +|"model_name_or_path"|String|Language modeling model to generate data, refer to [line](intel_extension_for_transformers/preprocessing/data_augmentation.py#L181)|NA| +|"stop_token"|String|Stop token used in input data file |[EOS](intel_extension_for_transformers/preprocessing/utils.py#L7)| |"num_return_sentences"|Integer|Total samples to generate, -1 means the number of the input samples |-1| |"temperature"|float|parameter for CLM model |1.0| |"k"|float|top K |0.0| diff --git a/docs/distillation.md b/docs/distillation.md index 203104fcfd1..6f5c52b25b9 100644 --- a/docs/distillation.md +++ b/docs/distillation.md @@ -1,6 +1,6 @@ # Distillation ## Introduction -Knowledge distillation is one of popular approaches of network compression, which transfers knowledge from a large model to a smaller one without loss of validity. As smaller models are less expensive to evaluate, they can be deployed on less powerful hardware (such as a mobile device). Graph shown below is the workflow of the distillation, the teacher model will take the same input that feed into the student model to produce the output that contains knowledge of the teacher model to instruct the student model. +Knowledge distillation is one of popular approaches of the network compression, which transfers knowledge from a large model to a smaller one without loss of validity. As smaller models are less expensive to be evaluated, they can be deployed on less powerful hardwares (such as mobile devices). The graph shown below is the workflow of the distillation, the teacher model will take the same input that feed into the student model to produce the output that contains knowledge of the teacher model to instruct the student model.
@@ -8,8 +8,8 @@ Knowledge distillation is one of popular approaches of network compression, whic ## usage ### script: ```python -from nlp_toolkit import metric, objectives, DistillationConfig, Criterion -from nlp_toolkit.optimization.trainer import NLPTrainer +from intel_extension_for_transformers import metric, objectives, DistillationConfig, Criterion +from intel_extension_for_transformers.optimization.trainer import NLPTrainer # Replace transformers.Trainer with NLPTrainer # trainer = transformers.Trainer(......) trainer = NLPTrainer(......) @@ -23,7 +23,7 @@ model = trainer.distill( Please refer to [example](../examples/optimize/pytorch/huggingface/text-classification/distillation/run_glue.py) for the details. ### Create an instance of Metric -The Metric define which metric will used to measure the performance of tuned models. +The Metric defines which metric will be used to measure the performance of tuned models. - example: ```python metric = metrics.Metric(name="eval_accuracy") @@ -66,7 +66,7 @@ The DistillationConfig contains all the information related to the model distill ### Distill with Trainer - Distill with Trainer - NLPTrainer inherits from transformers.Trainer, so you can create trainer like you do in transformers examples. Then you can distill model with trainer.distill function. + NLPTrainer inherits from transformers.Trainer, so you can create a trainer as in examples of Transformers. Then you can distill model with trainer.distill function. ```python model = trainer.distill( distillation_config=d_conf, teacher_model=teacher_model diff --git a/docs/examples.md b/docs/examples.md index d3e123d42d7..7095149e1d2 100644 --- a/docs/examples.md +++ b/docs/examples.md @@ -1,6 +1,6 @@ Examples === -NLP Toolkit is a powerful toolkit with multiple model optimization techniques for Natural Language Processing Models, including quantization, pruning, distillation, auto distillation and orchestrate. Meanwhile NLP Toolkit provides Neural Engine, an optimized backend for NLP models to demonstrate the deployment. +Intel Extension for Transformers is a powerful toolkit with multiple model optimization techniques for Natural Language Processing Models, including quantization, pruning, distillation, auto distillation and orchestrate. Meanwhile Intel Extension for Transformers provides Neural Engine, an optimized backend for NLP models to demonstrate the deployment. ## Quantization ### Stock PyTorch Examples @@ -12,8 +12,6 @@ NLP Toolkit is a powerful toolkit with multiple model optimization techniques fo
| Model | +Task | +Dataset | +QuantizationAwareTraining | +No Trainer quantization | +
|---|---|---|---|---|
| textattack/bert-base-uncased-MRPC | +text-classification | +MRPC | +✔ | ++ |
| echarlaix/bert-base-uncased-sst2-acc91.1-d37-hybrid | +text-classification | +SST-2 | ++ | ✔ | +
| Dataset | PostTrainingDynamic | PostTrainingStatic | -QuantizationAwareTraining | -No Trainer quantization | @@ -156,19 +153,15 @@ NLP Toolkit is a powerful toolkit with multiple model optimization techniques fodistilbert-base-uncased-distilled-squad | question-answering | SQuAD | -WIP :star: | +✔ | -WIP :star: | -WIP :star: |
|---|---|---|---|---|---|---|---|
| bert-large-uncased-whole-word-maskinuned-squad | question-answering | SQuAD | -WIP :star: | +✔ | -WIP :star: | -WIP :star: |
| Model Name | +Datatype | +Optimization Method | +Modelsize (MB) | +InferenceResult | +|||
|---|---|---|---|---|---|---|---|
| Accuracy(F1) | +Latency(ms) | +GFLOPS** | +Speedup(comparedwith BERT Base) | +||||
| BERT Base | +fp32 | +None | +415.47 | +88.58 | +56.56 | +35.3 | +1x | +
| LA-MiniLM | +fp32 | +Drop and restore base MiniLMv2 | +115.04 | +89.28 | +16.99 | +4.76 | +3.33x | +
| LA-MiniLM(269, 253, 252, 202, 104, 34)* | +fp32 | +Evolution search (best config) | +115.04 | +87.76 | +11.44 | +2.49 | +4.94x | +
| QuaLA-MiniLM | +int8 | +Quantization base LA-MiniLM | +84.85 | +88.85 | +7.84 | +4.76 | +7.21x | +
| QuaLA-MiniLM(315,251,242,159,142,33)* | +int8 | +Evolution search (best config) | +84.86 | +87.68 | +6.41 | +2.55 | +8.82x | +
Intel TensorFlow
| Model | +Task | +Dataset | +Datatype | +|||
|---|---|---|---|---|---|---|
| INT8 | +BF16 | +|||||
| Intel/distilbert-base-uncased-squadv1.1-sparse-80-1x4-block-pruneofa | +question-answering | +SQuAD | +✔ | WIP :star: | ||
| Intel/bert-mini-sst2-distilled-sparse-90-1X4-block | text-classification | SST-2 | -N/A | -N/A | ✔ | WIP :star: |
(PostTrainingDynamic) | / | / | ✔ | +| INT8 PyTorch Model
(PostTrainingStatic) | / | / | ✔ | +| INT8 PyTorch Model
(QuantizationAwareTraining) | / | / | ✔ | -### API usage + +## Examples + +### Export to FP32 ONNX Model + +If `export_to_onnx` is called before quantization, we will fetch the FP32 model and export it into a ONNX model. ```py trainer.export_to_onnx( @@ -18,11 +41,10 @@ trainer.export_to_onnx( [verbose=True,] ) ``` ----- -## Export BF16 model +### Export to BF16 ONNX Model -If the flag: `enable_bf16` is True, you will get an ONNX model with BFloat16 weights for ['MatMul', 'Gemm'] node type. This FP32 + BF16 ONNX model can be accelerated by our [executor](../nlp_toolkit/backends/neural_engine/) backend. +If the flag: `enable_bf16` is True, you will get an ONNX model with BFloat16 weights for ['MatMul', 'Gemm'] node type. This FP32 + BF16 ONNX model can be accelerated by our [executor](../intel_extension_for_transformers/backends/neural_engine/) backend. ### API usage @@ -35,14 +57,11 @@ trainer.export_to_onnx( [verbose=True,] ) ``` ----- -## Export INT8 model +### Export to INT8 ONNX Model If `export_to_onnx` is called after quantization, we will fetch the FP32 PyTorch model, convert it into ONNX model and do onnxruntime quantization based on pytorch quantization configuration. -### API usage - ```py trainer.export_to_onnx( save_path=None, @@ -58,4 +77,3 @@ Our executor backend provides highly optimized performance for INT8 `MatMul` nod ```py trainer.enable_executor = True ``` - diff --git a/docs/imgs/arch.png b/docs/imgs/arch.png new file mode 100644 index 00000000000..20b45a044d5 Binary files /dev/null and b/docs/imgs/arch.png differ diff --git a/docs/metrics.md b/docs/metrics.md index 3eb0223082b..e611957de8d 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -1,16 +1,16 @@ # Metric -The Metric define which metric will used to measure the performance of tuned models. +The Metric defines which metric will be used to measure the performance of tuned models. - arguments: |Argument |Type |Description |Default value | |:----------|:----------|:-----------------------------------------------|:----------------| - |name |string |Metric name which evaluate function returns, like:"eval_f1", "eval_accuracy"...| | + |name |string |Metric name which evaluates function returns, like:"eval_f1", "eval_accuracy"...| | |greater_is_better|bool |Used to describe the usage of the metric, like: greater is better for f1, this parameter is only used for quantization.|True| - |is_relative|bool |Used in conjunction with "criterion", if "criterion" is 0.01, and "is_relative" is True, it means that we want to get an optimized model which metric drop <1% relative, if "is_relative" is False, means metric drop <1% absolute, this parameter is only used for quantization.|True | - |criterion |float |Used in conjunction with "is_relative". if "criterion" is 0.01, and "is_relative" is True, it means that we want to get an optimized model which metric drop <1% relative, if "criterion" is 0.02, means metric drop <2% relative, this parameter is only used for quantization.|0.01 | - |weight_ratio|float |Used when there are multiple metrics, for example: you want to focus on both F1 and accuracy, then you will create f1 instance and accuracy instance, and indicate their weight proportion. if weight_ratio of f1 is 0.3, and weight ratio of accuracy is 0.7, then the final metric to tune is f1*0.3 + accuracy*0.7, this parameter is only used for quantization.|None | + |is_relative|bool |Used in conjunction with "criterion". If "criterion" is 0.01, and "is_relative" is True, it means that we want to get an optimized model which metric drop <1% relative, if "is_relative" is False, means metric drop <1% absolute, this parameter is only used for quantization.|True | + |criterion |float |Used in conjunction with "is_relative". If "criterion" is 0.01, and "is_relative" is True, it means that we want to get an optimized model which metric drop <1% relative, if "criterion" is 0.02, means metric drop <2% relative, this parameter is only used for quantization.|0.01 | + |weight_ratio|float |Used when there are multiple metrics, for example: you want to focus on both f1 and accuracy, then you will create f1 instance and accuracy instance, and indicate their weight proportion. If weight_ratio of f1 is 0.3, and weight ratio of accuracy is 0.7, then the final metric to tune is f1*0.3 + accuracy*0.7, this parameter is only used for quantization.|None | - example: ```python - from nlp_toolkit import metric + from intel_extension_for_transformers import metric metric.Metric(name="eval_f1", greater_is_better=True, is_relative=True, criterion=0.01, weight_ratio=None) ``` \ No newline at end of file diff --git a/docs/objectives.md b/docs/objectives.md index 96d0c448d79..15208c3861c 100644 --- a/docs/objectives.md +++ b/docs/objectives.md @@ -1,19 +1,34 @@ # Objective + +1. [Introduction](#introduction) + +1. [Supported Objectives Matrix](#supported-objectives-matrix) + +1. [Examples](#examples) + +## Introduction In terms of evaluating the status of a specific model during tuning, we should have general objectives to measure the status of different models. -NLP Toolkit supports to optimized low-precision recipes for deep learning models to achieve optimal product objectives like inference performance and memory usage with expected accuracy criteria and now supports Objectives which is supported in [INC](https://github.com/intel-innersource/frameworks.ai.lpot.intel-lpot/blob/master/docs/objective.md#built-in-objective-support-list). +Intel Extension for Transformers supports optimized low-precision recipes for deep learning models to achieve optimal product objectives like inference performance and memory usage with expected accuracy criteria. + +## Supported Objectives Matrix: +|Argument |Type |Description |Default value | +|:----------|:----------|:-----------------------------------------------|:----------------| +|name |string |a objective name in [Intel Neural Compressor](https://github.com/intel/neural-compressor/blob/master/docs/objective.md#built-in-objective-support-list). Like "performance", "modelsize",......and so on| / | +|greater_is_better|bool |used to describe the usage of the objective, like: greater is better for performance, but lower is better for modelsize| True | +|weight_ratio|float |used when there are multiple objective.
for example: different weight proportion on performance and modelsize.| None | + +## Examples: + +There are two built-in objective instances: performance, modelsize. Users can also build their own objective as below: -- arguments: - |Argument |Type |Description |Default value | - |:----------|:----------|:-----------------------------------------------|:----------------| - |name |string |the Objective name in [INC](https://github.com/intel-innersource/frameworks.ai.lpot.intel-lpot/blob/master/docs/objective.md#built-in-objective-support-list). Like "performance", "modelsize",......and so on| | - |greater_is_better|bool |Used to describe the usage of the objective, like: greater is better for performance, but lower is better for modelsize|True| - |weight_ratio|float |Used when there are multiple objective, for example: you want to focus on both performance and modelsize, then you will create performance objective instance and modelsize objective instance, and indicate their weight proportion|None | +```python +from intel_extension_for_transformers.objectives import performance, modelsize +``` -- example: - ```python - from nlp_toolkit import objectives - objectives.Objective(name="performance", greater_is_better=True, weight_ratio=None) - ``` +or -- Built-in Objective instance: performance, modelsize. \ No newline at end of file +```python +from intel_extension_for_transformers import objectives +performance = objectives.Objective(name="performance", greater_is_better=True, weight_ratio=None) +``` diff --git a/docs/pipeline.md b/docs/pipeline.md index 0b8e62573dc..7d7c6af5612 100644 --- a/docs/pipeline.md +++ b/docs/pipeline.md @@ -3,18 +3,18 @@ The pipeline is inherited from transformers [pipeline](https://github.com/huggingface/transformers/blob/main/docs/source/en/pipeline_tutorial.mdx), and two more features are appended. * Use a [`pipeline`] for int8 model inference. -* Use a [`pipeline`] for inference on our [executor](../nlp_toolkit/backends/neural_engine/) backend. +* Use a [`pipeline`] for inference on our [executor](../intel_extension_for_transformers/backends/neural_engine/) backend. -Executor is a inference tool for accelerated deployment in NLP-toolkit. +Executor is a inference tool for accelerated deployment in Intel_Extension_for_Transformers. ## Pipeline usage ---- ### **INT8 model** -1. Initializer a pipeline instance with model name and specific task. +1. Initialize a pipeline instance with model name and specific task. ```py - from nlp_toolkit.optimization.pipeline import pipeline + from intel_extension_for_transformers.optimization.pipeline import pipeline text_classifier = pipeline( task="text-classification", model="Intel/distilbert-base-uncased-finetuned-sst-2-english-int8-static", @@ -34,9 +34,9 @@ Executor is a inference tool for accelerated deployment in NLP-toolkit. For executor, we only accept ONNX model now for pipeline. Users can get onnx model from PyTorch model with our existing [API](export.md). Right now, pipeline for executor only supports text-classcification task. -1. Initializer a pipeline instance with an ONNX model, model config, model tokenizer and specific backend. The MODEL_NAME is the pytorch model name you used for exporting the ONNX model. +1. Initialize a pipeline instance with an ONNX model, model config, model tokenizer and specific backend. The MODEL_NAME is the pytorch model name you used for exporting the ONNX model. ```py - from nlp_toolkit.optimization.pipeline import pipeline + from intel_extension_for_transformers.optimization.pipeline import pipeline from transformers import AutoConfig, AutoTokenizer config = AutoConfig.from_pretrained(MODEL_NAME) diff --git a/docs/profiling.md b/docs/profiling.md index faabe4a4e71..09ea0e3fad6 100644 --- a/docs/profiling.md +++ b/docs/profiling.md @@ -1,7 +1,7 @@ # Profiling ## Introduction In terms of improving the performance of the model ,we should evaluate the performance of each operator(op) during inference. -NLP Toolkit supports tracing the profiling of operator latency. +Intel Extension for Transformers supports tracing the profiling of operator latency. ## Usage ### Example run python @@ -11,7 +11,7 @@ ENGINE_PROFILING=1 python run_executor.py --input_model=./model_and_tokenizer/i or run C++ ```shell export ENGINE_PROFILING=1 -
@@ -14,11 +14,11 @@ Quantization is a widely-used model compression technique that can reduce model
-## quantization usage
-### script:
+## Quantization Usage
+### Script:
```python
-from nlp_toolkit import metric, objectives, QuantizationConfig
-from nlp_toolkit.optimization.trainer import NLPTrainer
+from intel_extension_for_transformers import metric, objectives, QuantizationConfig
+from intel_extension_for_transformers.optimization.trainer import NLPTrainer
# Replace transformers.Trainer with NLPTrainer
# trainer = transformers.Trainer(......)
trainer = NLPTrainer(......)
@@ -33,10 +33,10 @@ q_config = QuantizationConfig(
)
model = trainer.quantize(quant_config=q_config)
```
-Please refer to [quantization example](../examples/optimize/pytorch/huggingface/text-classification/quantization/inc/run_glue.py) for the details
+Please refer to [quantization example](../examples/optimize/pytorch/huggingface/text-classification/quantization/inc/run_glue.py) for the details.
### Create an instance of Metric
-The Metric define which metric will used to measure the performance of tuned models.
+The Metric defines which metric will be used to measure the performance of tuned models.
- example:
```python
metric = metrics.Metric(name="eval_f1", greater_is_better=True, is_relative=True, criterion=0.01, weight_ratio=None)
@@ -55,17 +55,17 @@ In terms of evaluating the status of a specific model during tuning, we should h
Please refer to [objective document](objectives.md) for the details.
### Create an instance of QuantizationConfig
-The QuantizationConfig contains all the information related to the model quantization behavior. If you created Metric and Objective instance(default Objective is "performance"), then you can create an instance of QuantizationConfig.
+The QuantizationConfig contains all the information related to the model quantization behavior. If you have created Metric and Objective instance(default Objective is "performance"), then you can create an instance of QuantizationConfig.
- arguments:
|Argument |Type |Description |Default value |
|:----------|:----------|:-----------------------------------------------|:----------------|
- |framework |string |which framework you used |"pytorch" |
+ |framework |string |Which framework you used |"pytorch" |
|approach |string |Which quantization approach you used |"PostTrainingStatic"|
- |timeout |integer |Tuning timeout(seconds), 0 means early stop. combine with max_trials field to decide when to exit|0 |
+ |timeout |integer |Tuning timeout(seconds), 0 means early stop; combine with max_trials field to decide when to exit|0 |
|max_trials |integer |Max tune times |100 |
|metrics |list of Metric|Used to evaluate accuracy of tuning model, no need for NoTrainerOptimizer|None |
- |objectives |list of Objective|objective with accuracy constraint guaranteed|performance|
+ |objectives |list of Objective|Objective with accuracy constraint guaranteed|performance|
- example:
```python
diff --git a/docs/tutorials/pytorch/language-modeling/benchmark.py b/docs/tutorials/pytorch/language-modeling/benchmark.py
index dd52218029a..f4d237f14fe 100644
--- a/docs/tutorials/pytorch/language-modeling/benchmark.py
+++ b/docs/tutorials/pytorch/language-modeling/benchmark.py
@@ -2,8 +2,8 @@
import os
from datasets import load_dataset, load_metric
from itertools import chain
-from nlp_toolkit import metrics, OptimizedModel
-from nlp_toolkit.optimization.trainer import NLPTrainer
+from intel_extension_for_transformers import metrics, OptimizedModel
+from intel_extension_for_transformers.optimization.trainer import NLPTrainer
from argparse import ArgumentParser
from transformers import (
MODEL_FOR_MASKED_LM_MAPPING,
diff --git a/docs/tutorials/pytorch/language-modeling/bert-base-uncased.ipynb b/docs/tutorials/pytorch/language-modeling/bert-base-uncased.ipynb
index 13942895065..87d57b9059b 100644
--- a/docs/tutorials/pytorch/language-modeling/bert-base-uncased.ipynb
+++ b/docs/tutorials/pytorch/language-modeling/bert-base-uncased.ipynb
@@ -43,7 +43,7 @@
"id": "c1816be1",
"metadata": {},
"source": [
- "* Follow [installation](https://github.com/intel-innersource/frameworks.ai.nlp-toolkit.intel-nlp-toolkit#installation) to install **nlp-toolkit**. "
+ "* Follow [installation](https://github.com/intel/intel-extension-for-transformers#installation) to install **intel-extension-for-transformers**. "
]
},
{
@@ -80,8 +80,8 @@
"from dataclasses import dataclass, field\n",
"from datasets import load_dataset, load_metric\n",
"from itertools import chain\n",
- "from nlp_toolkit import metrics, OptimizedModel, QuantizationConfig\n",
- "from nlp_toolkit.optimization.trainer import NLPTrainer\n",
+ "from intel_extension_for_transformers import metrics, OptimizedModel, QuantizationConfig\n",
+ "from intel_extension_for_transformers.optimization.trainer import NLPTrainer\n",
"from transformers import (\n",
" CONFIG_MAPPING,\n",
" MODEL_FOR_MASKED_LM_MAPPING,\n",
diff --git a/docs/tutorials/pytorch/multiple-choice/benchmark.py b/docs/tutorials/pytorch/multiple-choice/benchmark.py
index cb2df70bde5..6fde6834d0f 100644
--- a/docs/tutorials/pytorch/multiple-choice/benchmark.py
+++ b/docs/tutorials/pytorch/multiple-choice/benchmark.py
@@ -3,8 +3,8 @@
import numpy as np
from datasets import load_dataset, load_metric
from itertools import chain
-from nlp_toolkit import metrics, OptimizedModel
-from nlp_toolkit.optimization.trainer import NLPTrainer
+from intel_extension_for_transformers import metrics, OptimizedModel
+from intel_extension_for_transformers.optimization.trainer import NLPTrainer
from argparse import ArgumentParser
from transformers import (
MODEL_FOR_MASKED_LM_MAPPING,
diff --git a/docs/tutorials/pytorch/multiple-choice/bert-base-uncased_SWAG.ipynb b/docs/tutorials/pytorch/multiple-choice/bert-base-uncased_SWAG.ipynb
index 629c33dbfff..b36a053fec3 100644
--- a/docs/tutorials/pytorch/multiple-choice/bert-base-uncased_SWAG.ipynb
+++ b/docs/tutorials/pytorch/multiple-choice/bert-base-uncased_SWAG.ipynb
@@ -57,7 +57,7 @@
}
},
"source": [
- "* Follow [installation](https://github.com/intel-innersource/frameworks.ai.nlp-toolkit.intel-nlp-toolkit#installation) to install **nlp-toolkit**. "
+ "* Follow [installation](https://github.com/intel/intel-extension-for-transformers#installation) to install **intel-extension-for-transformers**. "
]
},
{
@@ -108,8 +108,8 @@
"from dataclasses import dataclass, field\n",
"from datasets import load_dataset\n",
"from itertools import chain\n",
- "from nlp_toolkit import metrics, OptimizedModel, QuantizationConfig\n",
- "from nlp_toolkit.optimization.trainer import NLPTrainer\n",
+ "from intel_extension_for_transformers import metrics, OptimizedModel, QuantizationConfig\n",
+ "from intel_extension_for_transformers.optimization.trainer import NLPTrainer\n",
"from transformers import (\n",
" AutoConfig,\n",
" AutoModelForMultipleChoice,\n",
diff --git a/docs/tutorials/pytorch/question-answering/Dynamic_MiniLM_SQuAD.ipynb b/docs/tutorials/pytorch/question-answering/Dynamic_MiniLM_SQuAD.ipynb
index f6abd2c2e74..b0830c5243e 100644
--- a/docs/tutorials/pytorch/question-answering/Dynamic_MiniLM_SQuAD.ipynb
+++ b/docs/tutorials/pytorch/question-answering/Dynamic_MiniLM_SQuAD.ipynb
@@ -37,7 +37,7 @@
"id": "c1816be1",
"metadata": {},
"source": [
- "* Follow [installation](https://github.com/intel-innersource/frameworks.ai.nlp-toolkit.intel-nlp-toolkit#installation) to install **nlp-toolkit**. "
+ "* Follow [installation](https://github.com/intel/intel-extension-for-transformers#installation) to install **intel-extension-for-transformers**. "
]
},
{
@@ -86,12 +86,12 @@
"from dataclasses import dataclass, field\n",
"from datasets import load_dataset, load_metric\n",
"from itertools import chain\n",
- "from nlp_toolkit import metrics, OptimizedModel, QuantizationConfig, DynamicLengthConfig\n",
- "from nlp_toolkit.optimization.trainer import NLPTrainer\n",
+ "from intel_extension_for_transformers import metrics, OptimizedModel, QuantizationConfig, DynamicLengthConfig\n",
+ "from intel_extension_for_transformers.optimization.trainer import NLPTrainer\n",
"from transformers.trainer_utils import get_last_checkpoint\n",
"from transformers.utils.versions import require_version\n",
"from typing import Optional\n",
- "from nlp_toolkit.optimization.utils.models.modeling_roberta_dynamic import RobertaForQuestionAnswering\n",
+ "from intel_extension_for_transformers.optimization.utils.models.modeling_roberta_dynamic import RobertaForQuestionAnswering\n",
"# to use modeling roberta with LAT:\n",
"transformers.models.roberta.modeling_roberta.RobertaForQuestionAnswering = RobertaForQuestionAnswering\n",
"import collections\n",
@@ -948,21 +948,21 @@
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m
-
+|[Bert-large (SQUAD)](https://github.com/intel/intel-extension-for-transformers/tree/develop/examples/deployment/neural_engine/squad/bert_large)|[Bert-mini (SST2)](https://github.com/intel/intel-extension-for-transformers/tree/develop/examples/deployment/neural_engine/sst2/bert_mini) [MiniLM (SST2)](https://github.com/intel/intel-extension-for-transformers/tree/develop/examples/deployment/neural_engine/sst2/minilm_l6_h384_uncased) [Distilbert (SST2)](https://github.com/intel/intel-extension-for-transformers/tree/develop/examples/deployment/neural_engine/sst2/distilbert_base_uncased) [Distilbert (Emotion)](https://github.com/intel/intel-extension-for-transformers/tree/develop/examples/deployment/neural_engine/emotion/distilbert_base_uncased) [Bert-base (MRPC)](https://github.com/intel/intel-extension-for-transformers/tree/develop/examples/deployment/neural_engine/mrpc/bert_base) [Bert-mini (MRPC)](https://github.com/intel/intel-extension-for-transformers/tree/develop/examples/deployment/neural_engine/mrpc/bert_mini)[Distilbert (MRPC)](https://github.com/intel/intel-extension-for-transformers/tree/develop/examples/deployment/neural_engine/mrpc/distilbert_base_uncased) [Roberta-base (MRPC)](https://github.com/intel/intel-extension-for-transformers/tree/develop/examples/deployment/neural_engine/mrpc/roberta_base)|
#### Installation
Linux is supported only.
@@ -37,15 +31,15 @@ pip install neural-compressor
##### 2. Build neural engine
```
-cd diff --git a/examples/deployment/neural_engine/common.py b/examples/deployment/neural_engine/common.py index 364663c994e..38614651fea 100644 --- a/examples/deployment/neural_engine/common.py +++ b/examples/deployment/neural_engine/common.py @@ -19,8 +19,8 @@ import logging import sys import numpy as np -from nlp_toolkit.backends.neural_engine.compile import compile -from nlp_toolkit.backends.neural_engine.compile.graph import Graph +from intel_extension_for_transformers.backends.neural_engine.compile import compile +from intel_extension_for_transformers.backends.neural_engine.compile.graph import Graph from tqdm import tqdm import time diff --git a/examples/deployment/neural_engine/emotion/distilbert_base_uncased/README.md b/examples/deployment/neural_engine/emotion/distilbert_base_uncased/README.md index 0539b8f006e..7ea2114d2e5 100644 --- a/examples/deployment/neural_engine/emotion/distilbert_base_uncased/README.md +++ b/examples/deployment/neural_engine/emotion/distilbert_base_uncased/README.md @@ -20,20 +20,20 @@ cmake --version conda install cmake sudo apt install autoconf ``` -Install NLP Toolkit from source code +Install Intel Extension for Transformers from source code ```shell -cd
Model Name |
+ Datatype | +Optimization Method |
+ Modelsize (MB) |
+ InferenceResult |
+ |||
|---|---|---|---|---|---|---|---|
Accuracy(F1) |
+ Latency(ms) |
+ GFLOPS** |
+ Speedup (comparedwith BERT Base) |
+ ||||
BERT Base |
+ fp32 | +None |
+ 415.47 |
+ 88.58 |
+ 56.56 |
+ 35.3 |
+ 1x |
+
TinyBERT |
+ fp32 | +Distillation |
+ 253.20 |
+ 88.39 |
+ 32.40 |
+ 17.7 |
+ 1.75x |
+
QuaTinyBERT |
+ int8 | +Distillation + quantization |
+ 132.06 |
+ 87.67 |
+ 15.58 |
+ 17.7 |
+ 3.63x |
+
MiniLMv2 |
+ fp32 | +Distillation |
+ 115.04 |
+ 88.70 |
+ 18.23 |
+ 4.76 |
+ 3.10x |
+
QuaMiniLMv2 |
+ int8 | +Distillation + quantization |
+ 84.85 |
+ 88.54 |
+ 9.14 |
+ 4.76 |
+ 6.18x |
+
LA-MiniLM |
+ fp32 | +Drop and restore base MiniLMv2 |
+ 115.04 |
+ 89.28 |
+ 16.99 |
+ 4.76 |
+ 3.33x |
+
LA-MiniLM(269, 253, 252, 202, 104, 34)* |
+ fp32 | +Evolution search (best config) |
+ 115.04 |
+ 87.76 |
+ 11.44 |
+ 2.49 |
+ 4.94x |
+
QuaLA-MiniLM |
+ int8 | +Quantization base LA-MiniLM |
+ 84.85 |
+ 88.85 |
+ 7.84 |
+ 4.76 |
+ 7.21x |
+
QuaLA-MiniLM(315,251,242,159,142,33)* |
+ int8 | +Evolution search (best config) |
+ 84.86 |
+ 87.68 |
+ 6.41 |
+ 2.55 |
+ 8.82x |
+
| Manufacturer | +Intel Corporation | +
| Product Name | +S2600WFD | +
| BIOS Version | +1SE5C620.86B.02.01.0008.031920191559 | +
| OS | +CentOS Linux release 8.4.2105 | +
| Kernel | +4.18.0-305.3.1.el8.x86_64 | +
| Microcode | +0x5003006 | +
| IRQ Balance | +Eabled | +
| CPU Model | +Intel(R) Xeon Platinum 8280 CPU @ 2.70GHz | +
| Base Frequency | +2.7GHz | +
| Maximum Frequency | +4.0GHz | +
| All-core Maximum Frequency | +3.3GHz | +
| CPU(s) | +112 | +
| Thread(s) per Core | +2 | +
| Core(s) per Socket | +28 | +
| Socket(s) | +2 | +
| NUMA Node(s) | +2 | +
| Turbo | +Enabled | +
| FrequencyGoverner | +Performance | +
| Model | +Sparsity | +Sequencelength | +MAX Throughput (samples/sec) | +Latency (ms) | +BatchSize | +Instance | +Cores/Instance | +Requirement | +
|---|---|---|---|---|---|---|---|---|
| BERT Large | +90% | +16 | +2652 | +36.513 | +8 | +12 | +2 | +BatchSize <= 8 && Latency <= 50ms | +
| 85% | +2181 | +43.628 | +8 | +12 | +2 | +|||
| 80% | +1719 | +41.91 | +6 | +12 | +2 | +|||
| 75% | +1487 | +48.255 | +6 | +12 | +2 | +|||
| 70% | +1220 | +39.143 | +4 | +12 | +2 | +|||
| 90% | +32 | +1304 | +46.066 | +5 | +12 | +2 | +||
| 85% | +1102 | +43.9 | +4 | +12 | +2 | +|||
| 80% | +870 | +41.487 | +3 | +12 | +2 | +|||
| 75% | +750 | +47.82 | +3 | +12 | +2 | +|||
| 70% | +690 | +46.188 | +4 | +8 | +3 | +|||
| 90% | +48 | +910 | +39.921 | +3 | +12 | +2 | +||
| 85% | +729 | +49.71 | +3 | +12 | +2 | +|||
| 80% | +619 | +38.821 | +4 | +6 | +4 | +|||
| 75% | +499 | +47.933 | +2 | +12 | +2 | +|||
| 70% | +429 | +41.936 | +3 | +6 | +4 | +|||
| 90% | +64 | +680 | +35.318 | +2 | +12 | +2 | +||
| 85% | +557 | +43.262 | +2 | +12 | +2 | +|||
| 80% | +467 | +38.589 | +3 | +6 | +4 | +|||
| 75% | +405 | +44.415 | +3 | +6 | +4 | +|||
| 70% | +339 | +35.473 | +2 | +4 | +6 | +|||
| 90% | +80 | +513 | +47.119 | +2 | +12 | +2 | +||
| 85% | +389 | +40.945 | +2 | +8 | +3 | +|||
| 80% | +360 | +49.936 | +3 | +6 | +4 | +|||
| 75% | +274 | +43.843 | +2 | +6 | +4 | +|||
| 70% | +259 | +46.242 | +3 | +6 | +4 | +|||
| 90% | +96 | +442 | +36.241 | +2 | +8 | +3 | +||
| 85% | +364 | +44.016 | +2 | +8 | +3 | +|||
| 80% | +303 | +40.13 | +2 | +6 | +4 | +|||
| 75% | +266 | +45.05 | +2 | +6 | +4 | +|||
| 70% | +221 | +35.973 | +2 | +6 | +4 | +|||
| 90% | +112 | +349 | +45.938 | +2 | +8 | +3 | +||
| 85% | +275 | +43.614 | +2 | +6 | +4 | +|||
| 80% | +217 | +37.039 | +2 | +4 | +6 | +|||
| 75% | +188 | +42.301 | +2 | +4 | +6 | +|||
| 70% | +166 | +48.033 | +2 | +6 | +4 | +|||
| 90% | +128 | +320 | +49.769 | +2 | +8 | +3 | +||
| 85% | +263 | +45.724 | +2 | +6 | +4 | +|||
| 80% | +207 | +38.819 | +2 | +4 | +6 | +|||
| 75% | +181 | +44.021 | +2 | +4 | +6 | +|||
| 70% | +160 | +49.979 | +2 | +6 | +4 | +|||
| 90% | +384 | +74 | +41.235 | +1 | +3 | +8 | +||
| 85% | +63 | +48.091 | +2 | +2 | +12 | +|||
| 80% | +51 | +38.914 | +1 | +2 | +12 | +|||
| 75% | +46 | +43.698 | +1 | +2 | +12 | +|||
| 70% | +42 | +47.905 | +1 | +2 | +12 | +|||
| BERT Base | +90% | +16 | +8972 | +16.076 | +6 | +24 | +1 | +BatchSize <= 8 && Latency <= 20ms | +
| 85% | +7192 | +19.906 | +6 | +24 | +1 | +|||
| 80% | +5482 | +17.509 | +4 | +24 | +1 | +|||
| 75% | +4808 | +19.952 | +4 | +24 | +1 | +|||
| 70% | +3678 | +19.409 | +3 | +24 | +1 | +|||
| 90% | +32 | +4750 | +19.975 | +3 | +24 | +1 | +||
| 85% | +3647 | +19.867 | +3 | +24 | +1 | +|||
| 80% | +2921 | +16.587 | +4 | +12 | +2 | +|||
| 75% | +2576 | +18.768 | +4 | +12 | +2 | +|||
| 70% | +2131 | +16.913 | +3 | +12 | +2 | +|||
| 90% | +48 | +2804 | +17.096 | +4 | +12 | +2 | +||
| 85% | +2263 | +16.037 | +3 | +12 | +2 | +|||
| 80% | +1938 | +18.756 | +3 | +12 | +2 | +|||
| 75% | +1581 | +15.119 | +2 | +12 | +2 | +|||
| 70% | +1409 | +17.015 | +2 | +12 | +2 | +|||
| 90% | +64 | +2116 | +17.113 | +3 | +12 | +2 | +||
| 85% | +1777 | +19.972 | +3 | +12 | +2 | +|||
| 80% | +1474 | +16.392 | +2 | +12 | +2 | +|||
| 75% | +1278 | +18.74 | +4 | +6 | +4 | +|||
| 70% | +1137 | +15.752 | +3 | +6 | +4 | +|||
| 90% | +80 | +1594 | +15.124 | +4 | +6 | +4 | +||
| 85% | +1347 | +17.9 | +4 | +6 | +4 | +|||
| 80% | +1126 | +16 | +3 | +6 | +4 | +|||
| 75% | +993 | +18.258 | +3 | +6 | +4 | +|||
| 70% | +890 | +19.877 | +3 | +6 | +4 | +|||
| 90% | +96 | +1319 | +18.28 | +4 | +6 | +4 | +||
| 85% | +1086 | +16.63 | +3 | +6 | +4 | +|||
| 80% | +931 | +19.545 | +3 | +6 | +4 | +|||
| 75% | +833 | +14.332 | +2 | +6 | +4 | +|||
| 70% | +747 | +16.141 | +2 | +6 | +4 | +|||
| 90% | +112 | +1106 | +16.31 | +3 | +6 | +4 | +||
| 85% | +924 | +19.446 | +4 | +6 | +4 | +|||
| 80% | +719 | +16.698 | +4 | +4 | +6 | +|||
| 75% | +633 | +19.124 | +2 | +6 | +4 | +|||
| 70% | +501 | +15.816 | +2 | +4 | +6 | +|||
| 90% | +128 | +961 | +18.549 | +3 | +6 | +4 | +||
| 85% | +807 | +14.868 | +2 | +6 | +4 | +|||
| 80% | +701 | +17.117 | +4 | +4 | +6 | +|||
| 75% | +613 | +19.618 | +2 | +6 | +4 | +|||
| 70% | +515 | +15.484 | +2 | +4 | +6 | +|||
| BERT Mini | +90% | +16 | +75384 | +0.989 | +3 | +24 | +1 | +BatchSize <= 8 && Latency <= 1ms | +
| 85% | +55628 | +0.917 | +2 | +24 | +1 | +|||
| 80% | +49120 | +0.957 | +4 | +12 | +2 | +|||
| 75% | +41598 | +0.85 | +3 | +12 | +2 | +|||
| 70% | +39218 | +0.913 | +3 | +12 | +2 | +|||
| 90% | +32 | +31211 | +0.788 | +1 | +24 | +1 | +||
| 85% | +28399 | +0.848 | +1 | +24 | +1 | +|||
| 80% | +24910 | +0.966 | +4 | +6 | +4 | +|||
| 75% | +20505 | +0.88 | +3 | +6 | +4 | +|||
| 70% | +18575 | +0.865 | +2 | +8 | +3 | +|||
| 90% | +48 | +25485 | +0.987 | +1 | +24 | +1 | +||
| 85% | +17468 | +0.924 | +2 | +8 | +3 | +|||
| 80% | +16771 | +0.971 | +2 | +8 | +3 | +|||
| 75% | +16004 | +0.996 | +2 | +8 | +3 | +|||
| 70% | +13381 | +0.894 | +2 | +6 | +4 | +|||
| 90% | +64 | +13990 | +0.863 | +1 | +12 | +2 | +||
| 85% | +13154 | +0.917 | +2 | +6 | +4 | +|||
| 80% | +12535 | +0.973 | +2 | +6 | +4 | +|||
| 75% | +12102 | +0.998 | +2 | +6 | +4 | +|||
| 70% | +8426 | +0.961 | +2 | +4 | +6 | +|||
| 90% | +80 | +8736 | +0.917 | +1 | +8 | +3 | +||
| 85% | +8282 | +0.967 | +2 | +6 | +4 | +|||
| 80% | +6659 | +0.898 | +2 | +3 | +8 | +|||
| 75% | +6477 | +0.933 | +2 | +3 | +8 | +|||
| 70% | +6290 | +0.974 | +2 | +3 | +8 | +|||
| 90% | +96 | +8598 | +0.931 | +1 | +8 | +3 | +||
| 85% | +6449 | +0.935 | +2 | +3 | +8 | +|||
| 80% | +6263 | +0.965 | +2 | +3 | +8 | +|||
| 75% | +6080 | +0.988 | +2 | +3 | +8 | +|||
| 70% | +3684 | +0.817 | +1 | +3 | +8 | +|||
| 90% | +112 | +6246 | +0.978 | +1 | +6 | +4 | +||
| 85% | +6374 | +0.947 | +1 | +6 | +4 | +|||
| 80% | +6026 | +0.998 | +1 | +6 | +4 | +|||
| 75% | +3300 | +0.926 | +1 | +3 | +8 | +|||
| 70% | +3226 | +0.935 | +1 | +3 | +8 | +|||
| 90% | +128 | +6221 | +0.958 | +1 | +6 | +4 | +||
| 85% | +6322 | +0.96 | +1 | +6 | +4 | +|||
| 80% | +6081 | +0.985 | +1 | +6 | +4 | +|||
| 75% | +3368 | +0.894 | +1 | +3 | +8 | +|||
| 70% | +3264 | +0.924 | +1 | +3 | +8 | +|||
| DistillBERT | +90% | +16 | +15460 | +6.296 | +8 | +12 | +2 | +BatchSize <= 8 && Latency <= 10ms | +
| 85% | +16 | +13129 | +7.363 | +8 | +12 | +2 | +||
| 80% | +16 | +11323 | +8.578 | +8 | +12 | +2 | +||
| 75% | +16 | +10072 | +9.599 | +8 | +12 | +2 | +||
| 70% | +16 | +8689 | +7.44 | +8 | +8 | +3 | +||
| 90% | +32 | +7901 | +8.205 | +8 | +8 | +3 | +||
| 85% | +32 | +6737 | +9.562 | +8 | +8 | +3 | +||
| 80% | +32 | +5440 | +8.904 | +8 | +6 | +4 | +||
| 75% | +32 | +4920 | +9.786 | +8 | +6 | +4 | +||
| 70% | +32 | +4460 | +8.053 | +6 | +6 | +4 | +||
| 90% | +48 | +5280 | +9.179 | +6 | +8 | +3 | +||
| 85% | +48 | +4339 | +9.244 | +5 | +8 | +3 | +||
| 80% | +48 | +3634 | +9.928 | +5 | +6 | +4 | +||
| 75% | +48 | +3255 | +9.211 | +5 | +6 | +4 | +||
| 70% | +48 | +2928 | +8.197 | +4 | +6 | +4 | +||
| 90% | +64 | +3681 | +9.527 | +6 | +6 | +4 | +||
| 85% | +64 | +2965 | +8.106 | +6 | +4 | +6 | +||
| 80% | +64 | +2741 | +8.862 | +3 | +8 | +3 | +||
| 75% | +64 | +2351 | +8.527 | +5 | +4 | +6 | +||
| 70% | +64 | +2150 | +9.282 | +5 | +4 | +6 | +||
| 90% | +80 | +2918 | +8.316 | +4 | +6 | +4 | +||
| 85% | +80 | +2505 | +9.602 | +4 | +6 | +4 | +||
| 80% | +80 | +2010 | +7.995 | +4 | +4 | +6 | +||
| 75% | +80 | +1872 | +8.609 | +4 | +4 | +6 | +||
| 70% | +80 | +1706 | +9.442 | +4 | +4 | +6 | +||
| 90% | +96 | +2409 | +9.983 | +4 | +6 | +4 | +||
| 85% | +96 | +1993 | +8.013 | +4 | +4 | +6 | +||
| 80% | +96 | +1671 | +9.627 | +4 | +4 | +6 | +||
| 75% | +96 | +1484 | +8.131 | +4 | +3 | +8 | +||
| 70% | +96 | +1342 | +8.967 | +4 | +3 | +8 | +||
| 90% | +112 | +1910 | +8.456 | +4 | +4 | +6 | +||
| 85% | +112 | +1659 | +9.687 | +4 | +4 | +6 | +||
| 80% | +112 | +1293 | +9.245 | +4 | +3 | +8 | +||
| 75% | +112 | +1230 | +9.798 | +4 | +3 | +8 | +||
| 70% | +112 | +1091 | +8.261 | +3 | +3 | +8 | +||
| 90% | +128 | +1627 | +9.908 | +4 | +4 | +6 | +||
| 85% | +128 | +1397 | +8.684 | +3 | +4 | +6 | +||
| 80% | +128 | +1130 | +7.957 | +3 | +3 | +8 | +||
| 75% | +128 | +1081 | +8.353 | +3 | +3 | +8 | +||
| 70% | +128 | +982 | +9.187 | +3 | +3 | +8 | +||
| Manufacturer | +Amazon EC2 | +
| Product Name | +c6i.12xlarge | +
| BIOS Version | +1 | +
| OS | +Ubuntu 20.04.3 LTS | +
| Kernel | +5.15.0-1021-aws | +
| Microcode | +0xd000331 | +
| IRQ Balance | +Disabled | +
| CPU Model | +Intel(R) XeonPlatinum 8375C CPU @ 2.90GHz | +
| Base Frequency | +2.9GHz | +
| Maximum Frequency | +3.9GHz | +
| All-core Maximum Frequency | +3.5GHz | +
| CPU(s) | +48 | +
| Thread(s) per Core | +N/A | +
| Core(s) per Socket | +24 | +
| Socket(s) | +1 | +
| NUMA Node(s) | +1 | +
| Turbo | +Enabled | +
| FrequencyGoverner | +Default | +
| Max C-State | +9 | +
In SparseLib,we will provide a new class named injector for the op-fusion.In the perspective of the developers who want to apply the op-fusion optimization,they can make injector as a member of their jit_kernel class and initalize it in the kernel class's construct function,when they want to apply postop,just need to call **injector->vector_compute** and tell injector what registers has been used by **injector->escape_regs**.Besides,upper level user also should call **injector->prepare_table** to prepare the LUT which postop need in the end of thier xbyak kernel.
injector supports 8 operators currently,there are exp,tanh,gelu,relu,linear,quantize(fp32->u8/s8),dequantize(u8/s8->fp32) and look-up result from LUT(as experimental API now).Injector also supports a postop-chain for apply multiple postops sequentially. +op-fusion is a very widely used optimization approach in Deep-Learning.Consider we have two ops,Conv and Relu,in traditional way,we apply Conv op firstly,then store the value to the memory,after that we load the value and apply Relu.Obviously there have a useless load&store operations,we can fuse the Conv&Relu to remove the useless I/O,this is the key idea about op-fusion.
In Transformers-accelerated Libraries,we will provide a new class named injector for the op-fusion.In the perspective of the developers who want to apply the op-fusion optimization,they can make injector as a member of their jit_kernel class and initalize it in the kernel class's construct function,when they want to apply postop,just need to call **injector->vector_compute** and tell injector what registers has been used by **injector->escape_regs**.Besides,upper level user also should call **injector->prepare_table** to prepare the LUT which postop need in the end of thier xbyak kernel.
injector supports 8 operators currently,there are exp,tanh,gelu,relu,linear,quantize(fp32->u8/s8),dequantize(u8/s8->fp32) and look-up result from LUT(as experimental API now).Injector also supports a postop-chain for apply multiple postops sequentially. # Proposal -## SparseLib developer's perspective +## Transformers-accelerated Libraries developer's perspective ### Framework changes @@ -163,7 +163,7 @@ void jit_eltwiseop_t::generate() { ``` **NOTE:The postops will be apply **`in-place`** and storing work is upper op's task.** -## SparseLib user's perspective +## Transformers-accelerated Libraries user's perspective This is the guide about how to set op-fusion in UT in user's perspective. #### step0.Prepare the postop_attr diff --git a/nlp_toolkit/backends/neural_engine/SparseLib/docs/profiling.md b/intel_extension_for_transformers/backends/neural_engine/kernels/docs/profiling.md similarity index 91% rename from nlp_toolkit/backends/neural_engine/SparseLib/docs/profiling.md rename to intel_extension_for_transformers/backends/neural_engine/kernels/docs/profiling.md index b0c5de9687a..b2bb529a5eb 100644 --- a/nlp_toolkit/backends/neural_engine/SparseLib/docs/profiling.md +++ b/intel_extension_for_transformers/backends/neural_engine/kernels/docs/profiling.md @@ -1,10 +1,16 @@ ### Performance and Profiling -* We support a brief verbose logging for kernel execution + +## Verbose +We support a brief verbose logger for kernel execution ```shell SPARSE_LIB_VERBOSE=1 ./{executable} +sparselib_verbose,info,cpu,runtime:CPU,nthr:224 # general info +sparselib_verbose,exec,cpu,sparse_matmul,shape_256_256_128,14.4658 # first kernel +sparselib_verbose,exec,cpu,sparse_matmul,shape_256_256_128,2.56982 # second kernel ``` -* For advanced users we also support vtune profling for kernels execution through [ITT Tasks](https://www.intel.com/content/www/us/en/develop/documentation/vtune-help/top/api-support/instrumentation-and-tracing-technology-apis/basic-usage-and-configuration/viewing-itt-api-task-data.html), to enable it you can follow the instructions: +## VTune +For advanced users we also support vtune profling for kernels execution through [ITT Tasks](https://www.intel.com/content/www/us/en/develop/documentation/vtune-help/top/api-support/instrumentation-and-tracing-technology-apis/basic-usage-and-configuration/viewing-itt-api-task-data.html), to enable it you can follow the instructions: ```shell mkdir build @@ -17,7 +23,11 @@ make -j SPARSE_LIB_VTUNE=1 ./{executable} ``` -* There is another way to verify the code generation itself via [SDE](https://www.intel.com/content/www/us/en/developer/articles/tool/software-development-emulator.html). SDE is also simulators for future intel CPU hardware so you can verify the implementation using feature ISAs without real hardware support: +We would recommend SSH methods in VTune to analyse details based on GUI. + + +## SDE +There is another way to verify the code generation itself via [SDE](https://www.intel.com/content/www/us/en/developer/articles/tool/software-development-emulator.html). SDE is also simulators for future intel CPU hardware so you can verify the implementation using feature ISAs **without** real hardware support: ```shell SPARSE_LIB_DUMP=1 ./{executable} path/to/sde/xed64 -64 -ir *.bin >> *.txt diff --git a/intel_extension_for_transformers/backends/neural_engine/kernels/docs/validated_data.md b/intel_extension_for_transformers/backends/neural_engine/kernels/docs/validated_data.md new file mode 100644 index 00000000000..3acc8d2a26c --- /dev/null +++ b/intel_extension_for_transformers/backends/neural_engine/kernels/docs/validated_data.md @@ -0,0 +1,8039 @@ +Performance results test on 07/10/2022 with Intel(R) Xeon Platinum 8375C processor on AWS c6i.12xlarge instance. +Performance varies by use, configuration and other factors. See platform configuration for configuration details. For more complete information about performance and benchmark results, visit www.intel.com/benchmarks + +## Single Thread +
| M | +K | +N | +Sparsity | +Latency (ms) | +Gflops | +
|---|---|---|---|---|---|
| 256 | +256 | +16 | +0.7 | +0.003486 | +601.578 | +
| 256 | +256 | +16 | +0.75 | +0.003107 | +675.028 | +
| 256 | +256 | +16 | +0.8 | +0.002614 | +802.259 | +
| 256 | +256 | +16 | +0.85 | +0.002166 | +968.068 | +
| 256 | +256 | +16 | +0.9 | +0.001647 | +1273.64 | +
| 256 | +1024 | +16 | +0.7 | +0.011226 | +747.284 | +
| 256 | +1024 | +16 | +0.75 | +0.009475 | +885.377 | +
| 256 | +1024 | +16 | +0.8 | +0.007641 | +1097.78 | +
| 256 | +1024 | +16 | +0.85 | +0.005845 | +1435.13 | +
| 256 | +1024 | +16 | +0.9 | +0.004071 | +2060.33 | +
| 1024 | +256 | +16 | +0.7 | +0.012745 | +658.198 | +
| 1024 | +256 | +16 | +0.75 | +0.011113 | +754.835 | +
| 1024 | +256 | +16 | +0.8 | +0.009402 | +892.256 | +
| 1024 | +256 | +16 | +0.85 | +0.007602 | +1103.45 | +
| 1024 | +256 | +16 | +0.9 | +0.005494 | +1527.01 | +
| 1024 | +1024 | +16 | +0.7 | +0.044202 | +759.116 | +
| 1024 | +1024 | +16 | +0.75 | +0.036423 | +921.251 | +
| 1024 | +1024 | +16 | +0.8 | +0.029245 | +1147.34 | +
| 1024 | +1024 | +16 | +0.85 | +0.022347 | +1501.5 | +
| 1024 | +1024 | +16 | +0.9 | +0.014906 | +2251.11 | +
| 1024 | +4096 | +16 | +0.7 | +0.216338 | +620.408 | +
| 1024 | +4096 | +16 | +0.75 | +0.179593 | +747.345 | +
| 1024 | +4096 | +16 | +0.8 | +0.135054 | +993.806 | +
| 1024 | +4096 | +16 | +0.85 | +0.094698 | +1417.32 | +
| 1024 | +4096 | +16 | +0.9 | +0.06097 | +2201.37 | +
| 4096 | +1024 | +16 | +0.7 | +0.190508 | +704.526 | +
| 4096 | +1024 | +16 | +0.75 | +0.159566 | +841.14 | +
| 4096 | +1024 | +16 | +0.8 | +0.130863 | +1025.64 | +
| 4096 | +1024 | +16 | +0.85 | +0.103927 | +1291.46 | +
| 4096 | +1024 | +16 | +0.9 | +0.073996 | +1813.85 | +
| 768 | +768 | +16 | +0.7 | +0.02504 | +753.784 | +
| 768 | +768 | +16 | +0.75 | +0.021073 | +895.682 | +
| 768 | +768 | +16 | +0.8 | +0.017099 | +1103.84 | +
| 768 | +768 | +16 | +0.85 | +0.013078 | +1443.24 | +
| 768 | +768 | +16 | +0.9 | +0.009233 | +2044.13 | +
| 768 | +3072 | +16 | +0.7 | +0.112882 | +668.818 | +
| 768 | +3072 | +16 | +0.75 | +0.087939 | +858.525 | +
| 768 | +3072 | +16 | +0.8 | +0.068835 | +1096.79 | +
| 768 | +3072 | +16 | +0.85 | +0.048734 | +1549.17 | +
| 768 | +3072 | +16 | +0.9 | +0.031756 | +2377.43 | +
| 3072 | +756 | +16 | +0.7 | +0.109308 | +679.894 | +
| 3072 | +756 | +16 | +0.75 | +0.092247 | +805.638 | +
| 3072 | +756 | +16 | +0.8 | +0.075911 | +979.016 | +
| 3072 | +756 | +16 | +0.85 | +0.055895 | +1329.61 | +
| 3072 | +756 | +16 | +0.9 | +0.035875 | +2071.57 | +
| 256 | +256 | +32 | +0.7 | +0.00565 | +742.297 | +
| 256 | +256 | +32 | +0.75 | +0.005036 | +832.858 | +
| 256 | +256 | +32 | +0.8 | +0.004475 | +937.182 | +
| 256 | +256 | +32 | +0.85 | +0.003685 | +1138.27 | +
| 256 | +256 | +32 | +0.9 | +0.00271 | +1547.98 | +
| 256 | +1024 | +32 | +0.7 | +0.018348 | +914.386 | +
| 256 | +1024 | +32 | +0.75 | +0.015412 | +1088.57 | +
| 256 | +1024 | +32 | +0.8 | +0.01257 | +1334.74 | +
| 256 | +1024 | +32 | +0.85 | +0.009599 | +1747.72 | +
| 256 | +1024 | +32 | +0.9 | +0.006776 | +2475.86 | +
| 1024 | +256 | +32 | +0.7 | +0.020632 | +813.162 | +
| 1024 | +256 | +32 | +0.75 | +0.018427 | +910.483 | +
| 1024 | +256 | +32 | +0.8 | +0.01647 | +1018.66 | +
| 1024 | +256 | +32 | +0.85 | +0.01358 | +1235.43 | +
| 1024 | +256 | +32 | +0.9 | +0.010552 | +1589.96 | +
| 1024 | +1024 | +32 | +0.7 | +0.072121 | +930.506 | +
| 1024 | +1024 | +32 | +0.75 | +0.059613 | +1125.74 | +
| 1024 | +1024 | +32 | +0.8 | +0.048378 | +1387.18 | +
| 1024 | +1024 | +32 | +0.85 | +0.036675 | +1829.83 | +
| 1024 | +1024 | +32 | +0.9 | +0.024857 | +2699.78 | +
| 1024 | +4096 | +32 | +0.7 | +0.376539 | +712.902 | +
| 1024 | +4096 | +32 | +0.75 | +0.31929 | +840.727 | +
| 1024 | +4096 | +32 | +0.8 | +0.261265 | +1027.45 | +
| 1024 | +4096 | +32 | +0.85 | +0.193579 | +1386.7 | +
| 1024 | +4096 | +32 | +0.9 | +0.118429 | +2266.64 | +
| 4096 | +1024 | +32 | +0.7 | +0.309195 | +868.177 | +
| 4096 | +1024 | +32 | +0.75 | +0.256619 | +1046.05 | +
| 4096 | +1024 | +32 | +0.8 | +0.213454 | +1257.58 | +
| 4096 | +1024 | +32 | +0.85 | +0.165192 | +1624.99 | +
| 4096 | +1024 | +32 | +0.9 | +0.118402 | +2267.16 | +
| 768 | +768 | +32 | +0.7 | +0.040293 | +936.855 | +
| 768 | +768 | +32 | +0.75 | +0.034117 | +1106.45 | +
| 768 | +768 | +32 | +0.8 | +0.027528 | +1371.27 | +
| 768 | +768 | +32 | +0.85 | +0.02113 | +1786.51 | +
| 768 | +768 | +32 | +0.9 | +0.015069 | +2505.01 | +
| 768 | +3072 | +32 | +0.7 | +0.218765 | +690.215 | +
| 768 | +3072 | +32 | +0.75 | +0.180921 | +834.592 | +
| 768 | +3072 | +32 | +0.8 | +0.142778 | +1057.55 | +
| 768 | +3072 | +32 | +0.85 | +0.097692 | +1545.62 | +
| 768 | +3072 | +32 | +0.9 | +0.062003 | +2435.27 | +
| 3072 | +756 | +32 | +0.7 | +0.174101 | +853.73 | +
| 3072 | +756 | +32 | +0.75 | +0.148369 | +1001.79 | +
| 3072 | +756 | +32 | +0.8 | +0.123554 | +1203 | +
| 3072 | +756 | +32 | +0.85 | +0.092366 | +1609.21 | +
| 3072 | +756 | +32 | +0.9 | +0.059128 | +2513.78 | +
| 256 | +256 | +48 | +0.7 | +0.007447 | +844.786 | +
| 256 | +256 | +48 | +0.75 | +0.006572 | +957.332 | +
| 256 | +256 | +48 | +0.8 | +0.005702 | +1103.43 | +
| 256 | +256 | +48 | +0.85 | +0.004831 | +1302.3 | +
| 256 | +256 | +48 | +0.9 | +0.003674 | +1712.4 | +
| 256 | +1024 | +48 | +0.7 | +0.024706 | +1018.6 | +
| 256 | +1024 | +48 | +0.75 | +0.020746 | +1213.03 | +
| 256 | +1024 | +48 | +0.8 | +0.016921 | +1487.22 | +
| 256 | +1024 | +48 | +0.85 | +0.012994 | +1936.79 | +
| 256 | +1024 | +48 | +0.9 | +0.009038 | +2784.57 | +
| 1024 | +256 | +48 | +0.7 | +0.027588 | +912.197 | +
| 1024 | +256 | +48 | +0.75 | +0.024042 | +1046.77 | +
| 1024 | +256 | +48 | +0.8 | +0.020788 | +1210.62 | +
| 1024 | +256 | +48 | +0.85 | +0.017787 | +1414.82 | +
| 1024 | +256 | +48 | +0.9 | +0.014013 | +1795.91 | +
| 1024 | +1024 | +48 | +0.7 | +0.099457 | +1012.13 | +
| 1024 | +1024 | +48 | +0.75 | +0.081973 | +1228.01 | +
| 1024 | +1024 | +48 | +0.8 | +0.066057 | +1523.88 | +
| 1024 | +1024 | +48 | +0.85 | +0.049708 | +2025.1 | +
| 1024 | +1024 | +48 | +0.9 | +0.034191 | +2944.13 | +
| 1024 | +4096 | +48 | +0.7 | +0.500698 | +804.184 | +
| 1024 | +4096 | +48 | +0.75 | +0.420619 | +957.287 | +
| 1024 | +4096 | +48 | +0.8 | +0.337529 | +1192.94 | +
| 1024 | +4096 | +48 | +0.85 | +0.251221 | +1602.79 | +
| 1024 | +4096 | +48 | +0.9 | +0.158668 | +2537.72 | +
| 4096 | +1024 | +48 | +0.7 | +0.412912 | +975.155 | +
| 4096 | +1024 | +48 | +0.75 | +0.346369 | +1162.5 | +
| 4096 | +1024 | +48 | +0.8 | +0.2831 | +1422.3 | +
| 4096 | +1024 | +48 | +0.85 | +0.220585 | +1825.39 | +
| 4096 | +1024 | +48 | +0.9 | +0.156168 | +2578.33 | +
| 768 | +768 | +48 | +0.7 | +0.05384 | +1051.7 | +
| 768 | +768 | +48 | +0.75 | +0.045507 | +1244.28 | +
| 768 | +768 | +48 | +0.8 | +0.036978 | +1531.26 | +
| 768 | +768 | +48 | +0.85 | +0.028483 | +1987.96 | +
| 768 | +768 | +48 | +0.9 | +0.020046 | +2824.7 | +
| 768 | +3072 | +48 | +0.7 | +0.283445 | +799.069 | +
| 768 | +3072 | +48 | +0.75 | +0.236101 | +959.302 | +
| 768 | +3072 | +48 | +0.8 | +0.188745 | +1199.99 | +
| 768 | +3072 | +48 | +0.85 | +0.135532 | +1671.14 | +
| 768 | +3072 | +48 | +0.9 | +0.084554 | +2678.66 | +
| 3072 | +756 | +48 | +0.7 | +0.227976 | +977.971 | +
| 3072 | +756 | +48 | +0.75 | +0.19463 | +1145.53 | +
| 3072 | +756 | +48 | +0.8 | +0.161251 | +1382.65 | +
| 3072 | +756 | +48 | +0.85 | +0.123585 | +1804.05 | +
| 3072 | +756 | +48 | +0.9 | +0.084032 | +2653.19 | +
| 256 | +256 | +64 | +0.7 | +0.009405 | +891.898 | +
| 256 | +256 | +64 | +0.75 | +0.00822 | +1020.53 | +
| 256 | +256 | +64 | +0.8 | +0.007013 | +1196.08 | +
| 256 | +256 | +64 | +0.85 | +0.005799 | +1446.56 | +
| 256 | +256 | +64 | +0.9 | +0.004647 | +1805.15 | +
| 256 | +1024 | +64 | +0.7 | +0.033634 | +997.629 | +
| 256 | +1024 | +64 | +0.75 | +0.027609 | +1215.36 | +
| 256 | +1024 | +64 | +0.8 | +0.022443 | +1495.09 | +
| 256 | +1024 | +64 | +0.85 | +0.017081 | +1964.47 | +
| 256 | +1024 | +64 | +0.9 | +0.011754 | +2854.84 | +
| 1024 | +256 | +64 | +0.7 | +0.03586 | +935.698 | +
| 1024 | +256 | +64 | +0.75 | +0.031008 | +1082.11 | +
| 1024 | +256 | +64 | +0.8 | +0.026049 | +1288.15 | +
| 1024 | +256 | +64 | +0.85 | +0.021516 | +1559.49 | +
| 1024 | +256 | +64 | +0.9 | +0.017011 | +1972.57 | +
| 1024 | +1024 | +64 | +0.7 | +0.133295 | +1006.92 | +
| 1024 | +1024 | +64 | +0.75 | +0.111186 | +1207.14 | +
| 1024 | +1024 | +64 | +0.8 | +0.088111 | +1523.27 | +
| 1024 | +1024 | +64 | +0.85 | +0.066552 | +2016.74 | +
| 1024 | +1024 | +64 | +0.9 | +0.045442 | +2953.63 | +
| 1024 | +4096 | +64 | +0.7 | +0.611718 | +877.645 | +
| 1024 | +4096 | +64 | +0.75 | +0.533329 | +1006.64 | +
| 1024 | +4096 | +64 | +0.8 | +0.430288 | +1247.7 | +
| 1024 | +4096 | +64 | +0.85 | +0.307141 | +1747.96 | +
| 1024 | +4096 | +64 | +0.9 | +0.198987 | +2698.02 | +
| 4096 | +1024 | +64 | +0.7 | +0.556249 | +965.163 | +
| 4096 | +1024 | +64 | +0.75 | +0.462737 | +1160.21 | +
| 4096 | +1024 | +64 | +0.8 | +0.37537 | +1430.24 | +
| 4096 | +1024 | +64 | +0.85 | +0.28321 | +1895.66 | +
| 4096 | +1024 | +64 | +0.9 | +0.203901 | +2633 | +
| 768 | +768 | +64 | +0.7 | +0.071636 | +1053.91 | +
| 768 | +768 | +64 | +0.75 | +0.060352 | +1250.96 | +
| 768 | +768 | +64 | +0.8 | +0.04901 | +1540.46 | +
| 768 | +768 | +64 | +0.85 | +0.037568 | +2009.61 | +
| 768 | +768 | +64 | +0.9 | +0.026748 | +2822.51 | +
| 768 | +3072 | +64 | +0.7 | +0.3572 | +845.437 | +
| 768 | +3072 | +64 | +0.75 | +0.299561 | +1008.11 | +
| 768 | +3072 | +64 | +0.8 | +0.228752 | +1320.16 | +
| 768 | +3072 | +64 | +0.85 | +0.165648 | +1823.08 | +
| 768 | +3072 | +64 | +0.9 | +0.105656 | +2858.23 | +
| 3072 | +756 | +64 | +0.7 | +0.298907 | +994.529 | +
| 3072 | +756 | +64 | +0.75 | +0.255598 | +1163.04 | +
| 3072 | +756 | +64 | +0.8 | +0.20824 | +1427.55 | +
| 3072 | +756 | +64 | +0.85 | +0.159911 | +1858.98 | +
| 3072 | +756 | +64 | +0.9 | +0.111211 | +2673.04 | +
| 256 | +256 | +80 | +0.7 | +0.0153 | +685.358 | +
| 256 | +256 | +80 | +0.75 | +0.013145 | +797.724 | +
| 256 | +256 | +80 | +0.8 | +0.010967 | +956.132 | +
| 256 | +256 | +80 | +0.85 | +0.009211 | +1138.39 | +
| 256 | +256 | +80 | +0.9 | +0.007554 | +1388.1 | +
| 256 | +1024 | +80 | +0.7 | +0.055308 | +758.351 | +
| 256 | +1024 | +80 | +0.75 | +0.045848 | +914.826 | +
| 256 | +1024 | +80 | +0.8 | +0.036726 | +1142.06 | +
| 256 | +1024 | +80 | +0.85 | +0.027535 | +1523.26 | +
| 256 | +1024 | +80 | +0.9 | +0.018464 | +2271.61 | +
| 1024 | +256 | +80 | +0.7 | +0.059788 | +701.527 | +
| 1024 | +256 | +80 | +0.75 | +0.051022 | +822.062 | +
| 1024 | +256 | +80 | +0.8 | +0.042164 | +994.75 | +
| 1024 | +256 | +80 | +0.85 | +0.033344 | +1257.91 | +
| 1024 | +256 | +80 | +0.9 | +0.029639 | +1415.15 | +
| 1024 | +1024 | +80 | +0.7 | +0.22241 | +754.337 | +
| 1024 | +1024 | +80 | +0.75 | +0.18324 | +915.587 | +
| 1024 | +1024 | +80 | +0.8 | +0.145322 | +1154.49 | +
| 1024 | +1024 | +80 | +0.85 | +0.1083 | +1549.15 | +
| 1024 | +1024 | +80 | +0.9 | +0.072884 | +2301.9 | +
| 1024 | +4096 | +80 | +0.7 | +1.11793 | +600.298 | +
| 1024 | +4096 | +80 | +0.75 | +0.929398 | +722.068 | +
| 1024 | +4096 | +80 | +0.8 | +0.720963 | +930.823 | +
| 1024 | +4096 | +80 | +0.85 | +0.510039 | +1315.76 | +
| 1024 | +4096 | +80 | +0.9 | +0.336519 | +1994.21 | +
| 4096 | +1024 | +80 | +0.7 | +0.909941 | +737.508 | +
| 4096 | +1024 | +80 | +0.75 | +0.753487 | +890.644 | +
| 4096 | +1024 | +80 | +0.8 | +0.606477 | +1106.54 | +
| 4096 | +1024 | +80 | +0.85 | +0.459915 | +1459.16 | +
| 4096 | +1024 | +80 | +0.9 | +0.303455 | +2211.49 | +
| 768 | +768 | +80 | +0.7 | +0.123429 | +764.586 | +
| 768 | +768 | +80 | +0.75 | +0.103035 | +915.923 | +
| 768 | +768 | +80 | +0.8 | +0.082503 | +1143.86 | +
| 768 | +768 | +80 | +0.85 | +0.062558 | +1508.55 | +
| 768 | +768 | +80 | +0.9 | +0.042694 | +2210.42 | +
| 768 | +3072 | +80 | +0.7 | +0.60813 | +620.734 | +
| 768 | +3072 | +80 | +0.75 | +0.483053 | +781.462 | +
| 768 | +3072 | +80 | +0.8 | +0.357348 | +1056.36 | +
| 768 | +3072 | +80 | +0.85 | +0.256715 | +1470.45 | +
| 768 | +3072 | +80 | +0.9 | +0.169119 | +2232.09 | +
| 3072 | +756 | +80 | +0.7 | +0.503341 | +738.245 | +
| 3072 | +756 | +80 | +0.75 | +0.42134 | +881.923 | +
| 3072 | +756 | +80 | +0.8 | +0.342843 | +1083.85 | +
| 3072 | +756 | +80 | +0.85 | +0.255199 | +1456.08 | +
| 3072 | +756 | +80 | +0.9 | +0.174091 | +2134.45 | +
| 256 | +256 | +96 | +0.7 | +0.014267 | +881.979 | +
| 256 | +256 | +96 | +0.75 | +0.012397 | +1015.03 | +
| 256 | +256 | +96 | +0.8 | +0.010587 | +1188.58 | +
| 256 | +256 | +96 | +0.85 | +0.008608 | +1461.71 | +
| 256 | +256 | +96 | +0.9 | +0.006554 | +1919.99 | +
| 256 | +1024 | +96 | +0.7 | +0.053254 | +945.119 | +
| 256 | +1024 | +96 | +0.75 | +0.044755 | +1124.61 | +
| 256 | +1024 | +96 | +0.8 | +0.034839 | +1444.7 | +
| 256 | +1024 | +96 | +0.85 | +0.026689 | +1885.89 | +
| 256 | +1024 | +96 | +0.9 | +0.017743 | +2836.69 | +
| 1024 | +256 | +96 | +0.7 | +0.054911 | +916.608 | +
| 1024 | +256 | +96 | +0.75 | +0.047555 | +1058.39 | +
| 1024 | +256 | +96 | +0.8 | +0.040166 | +1253.1 | +
| 1024 | +256 | +96 | +0.85 | +0.032695 | +1539.42 | +
| 1024 | +256 | +96 | +0.9 | +0.025032 | +2010.7 | +
| 1024 | +1024 | +96 | +0.7 | +0.220947 | +911.2 | +
| 1024 | +1024 | +96 | +0.75 | +0.178324 | +1128.99 | +
| 1024 | +1024 | +96 | +0.8 | +0.137361 | +1465.68 | +
| 1024 | +1024 | +96 | +0.85 | +0.103126 | +1952.25 | +
| 1024 | +1024 | +96 | +0.9 | +0.070143 | +2870.23 | +
| 1024 | +4096 | +96 | +0.7 | +1.06648 | +755.11 | +
| 1024 | +4096 | +96 | +0.75 | +0.893959 | +900.831 | +
| 1024 | +4096 | +96 | +0.8 | +0.718893 | +1120.2 | +
| 1024 | +4096 | +96 | +0.85 | +0.537405 | +1498.51 | +
| 1024 | +4096 | +96 | +0.9 | +0.280686 | +2869.06 | +
| 4096 | +1024 | +96 | +0.7 | +0.902588 | +892.219 | +
| 4096 | +1024 | +96 | +0.75 | +0.730678 | +1102.14 | +
| 4096 | +1024 | +96 | +0.8 | +0.582871 | +1381.62 | +
| 4096 | +1024 | +96 | +0.85 | +0.444133 | +1813.21 | +
| 4096 | +1024 | +96 | +0.9 | +0.298489 | +2697.94 | +
| 768 | +768 | +96 | +0.7 | +0.1162 | +974.581 | +
| 768 | +768 | +96 | +0.75 | +0.095595 | +1184.64 | +
| 768 | +768 | +96 | +0.8 | +0.078109 | +1449.85 | +
| 768 | +768 | +96 | +0.85 | +0.059619 | +1899.51 | +
| 768 | +768 | +96 | +0.9 | +0.041405 | +2735.1 | +
| 768 | +3072 | +96 | +0.7 | +0.596568 | +759.318 | +
| 768 | +3072 | +96 | +0.75 | +0.497127 | +911.205 | +
| 768 | +3072 | +96 | +0.8 | +0.396634 | +1142.07 | +
| 768 | +3072 | +96 | +0.85 | +0.286973 | +1578.49 | +
| 768 | +3072 | +96 | +0.9 | +0.167473 | +2704.83 | +
| 3072 | +756 | +96 | +0.7 | +0.472466 | +943.787 | +
| 3072 | +756 | +96 | +0.75 | +0.391055 | +1140.27 | +
| 3072 | +756 | +96 | +0.8 | +0.319865 | +1394.05 | +
| 3072 | +756 | +96 | +0.85 | +0.247491 | +1801.71 | +
| 3072 | +756 | +96 | +0.9 | +0.169874 | +2624.93 | +
| 256 | +256 | +112 | +0.7 | +0.021379 | +686.67 | +
| 256 | +256 | +112 | +0.75 | +0.018228 | +805.358 | +
| 256 | +256 | +112 | +0.8 | +0.015207 | +965.36 | +
| 256 | +256 | +112 | +0.85 | +0.012053 | +1218.01 | +
| 256 | +256 | +112 | +0.9 | +0.010396 | +1412.07 | +
| 256 | +1024 | +112 | +0.7 | +0.077159 | +761.028 | +
| 256 | +1024 | +112 | +0.75 | +0.06424 | +914.074 | +
| 256 | +1024 | +112 | +0.8 | +0.051467 | +1140.93 | +
| 256 | +1024 | +112 | +0.85 | +0.038789 | +1513.85 | +
| 256 | +1024 | +112 | +0.9 | +0.025687 | +2285.98 | +
| 1024 | +256 | +112 | +0.7 | +0.083622 | +702.211 | +
| 1024 | +256 | +112 | +0.75 | +0.071292 | +823.664 | +
| 1024 | +256 | +112 | +0.8 | +0.058915 | +996.697 | +
| 1024 | +256 | +112 | +0.85 | +0.048407 | +1213.06 | +
| 1024 | +256 | +112 | +0.9 | +0.041133 | +1427.57 | +
| 1024 | +1024 | +112 | +0.7 | +0.311691 | +753.569 | +
| 1024 | +1024 | +112 | +0.75 | +0.256635 | +915.232 | +
| 1024 | +1024 | +112 | +0.8 | +0.204973 | +1145.91 | +
| 1024 | +1024 | +112 | +0.85 | +0.153086 | +1534.31 | +
| 1024 | +1024 | +112 | +0.9 | +0.101206 | +2320.83 | +
| 1024 | +4096 | +112 | +0.7 | +1.59625 | +588.583 | +
| 1024 | +4096 | +112 | +0.75 | +1.33246 | +705.106 | +
| 1024 | +4096 | +112 | +0.8 | +1.03849 | +904.703 | +
| 1024 | +4096 | +112 | +0.85 | +0.738939 | +1271.45 | +
| 1024 | +4096 | +112 | +0.9 | +0.480308 | +1956.09 | +
| 4096 | +1024 | +112 | +0.7 | +1.26056 | +745.324 | +
| 4096 | +1024 | +112 | +0.75 | +1.04874 | +895.862 | +
| 4096 | +1024 | +112 | +0.8 | +0.842936 | +1114.59 | +
| 4096 | +1024 | +112 | +0.85 | +0.635554 | +1478.28 | +
| 4096 | +1024 | +112 | +0.9 | +0.42145 | +2229.27 | +
| 768 | +768 | +112 | +0.7 | +0.174215 | +758.375 | +
| 768 | +768 | +112 | +0.75 | +0.145836 | +905.956 | +
| 768 | +768 | +112 | +0.8 | +0.116701 | +1132.13 | +
| 768 | +768 | +112 | +0.85 | +0.087673 | +1506.98 | +
| 768 | +768 | +112 | +0.9 | +0.0595 | +2220.51 | +
| 768 | +3072 | +112 | +0.7 | +0.883258 | +598.333 | +
| 768 | +3072 | +112 | +0.75 | +0.711146 | +743.142 | +
| 768 | +3072 | +112 | +0.8 | +0.543634 | +972.129 | +
| 768 | +3072 | +112 | +0.85 | +0.402406 | +1313.3 | +
| 768 | +3072 | +112 | +0.9 | +0.266945 | +1979.74 | +
| 3072 | +756 | +112 | +0.7 | +0.70404 | +738.913 | +
| 3072 | +756 | +112 | +0.75 | +0.592043 | +878.695 | +
| 3072 | +756 | +112 | +0.8 | +0.475444 | +1094.19 | +
| 3072 | +756 | +112 | +0.85 | +0.358932 | +1449.37 | +
| 3072 | +756 | +112 | +0.9 | +0.240739 | +2160.95 | +
| 256 | +256 | +128 | +0.7 | +0.018438 | +909.907 | +
| 256 | +256 | +128 | +0.75 | +0.015871 | +1057.07 | +
| 256 | +256 | +128 | +0.8 | +0.013504 | +1242.39 | +
| 256 | +256 | +128 | +0.85 | +0.010993 | +1526.22 | +
| 256 | +256 | +128 | +0.9 | +0.00826 | +2031.1 | +
| 256 | +1024 | +128 | +0.7 | +0.061272 | +1095.26 | +
| 256 | +1024 | +128 | +0.75 | +0.060962 | +1100.83 | +
| 256 | +1024 | +128 | +0.8 | +0.046729 | +1436.13 | +
| 256 | +1024 | +128 | +0.85 | +0.034036 | +1971.69 | +
| 256 | +1024 | +128 | +0.9 | +0.023128 | +2901.6 | +
| 1024 | +256 | +128 | +0.7 | +0.071725 | +935.645 | +
| 1024 | +256 | +128 | +0.75 | +0.061692 | +1087.8 | +
| 1024 | +256 | +128 | +0.8 | +0.05216 | +1286.61 | +
| 1024 | +256 | +128 | +0.85 | +0.042327 | +1585.5 | +
| 1024 | +256 | +128 | +0.9 | +0.032111 | +2089.91 | +
| 1024 | +1024 | +128 | +0.7 | +0.286461 | +937.076 | +
| 1024 | +1024 | +128 | +0.75 | +0.242512 | +1106.89 | +
| 1024 | +1024 | +128 | +0.8 | +0.181783 | +1476.68 | +
| 1024 | +1024 | +128 | +0.85 | +0.139021 | +1930.9 | +
| 1024 | +1024 | +128 | +0.9 | +0.087012 | +3085.04 | +
| 1024 | +4096 | +128 | +0.7 | +1.36322 | +787.652 | +
| 1024 | +4096 | +128 | +0.75 | +1.08991 | +985.163 | +
| 1024 | +4096 | +128 | +0.8 | +0.880185 | +1219.91 | +
| 1024 | +4096 | +128 | +0.85 | +0.694533 | +1545.99 | +
| 1024 | +4096 | +128 | +0.9 | +0.366328 | +2931.09 | +
| 4096 | +1024 | +128 | +0.7 | +1.15748 | +927.652 | +
| 4096 | +1024 | +128 | +0.75 | +0.937824 | +1144.93 | +
| 4096 | +1024 | +128 | +0.8 | +0.745357 | +1440.57 | +
| 4096 | +1024 | +128 | +0.85 | +0.558061 | +1924.06 | +
| 4096 | +1024 | +128 | +0.9 | +0.391387 | +2743.43 | +
| 768 | +768 | +128 | +0.7 | +0.157349 | +959.616 | +
| 768 | +768 | +128 | +0.75 | +0.129806 | +1163.23 | +
| 768 | +768 | +128 | +0.8 | +0.103083 | +1464.78 | +
| 768 | +768 | +128 | +0.85 | +0.078442 | +1924.93 | +
| 768 | +768 | +128 | +0.9 | +0.053639 | +2815.02 | +
| 768 | +3072 | +128 | +0.7 | +0.767569 | +786.874 | +
| 768 | +3072 | +128 | +0.75 | +0.641585 | +941.387 | +
| 768 | +3072 | +128 | +0.8 | +0.488558 | +1236.25 | +
| 768 | +3072 | +128 | +0.85 | +0.355651 | +1698.24 | +
| 768 | +3072 | +128 | +0.9 | +0.230945 | +2615.25 | +
| 3072 | +756 | +128 | +0.7 | +0.611059 | +972.97 | +
| 3072 | +756 | +128 | +0.75 | +0.514108 | +1156.45 | +
| 3072 | +756 | +128 | +0.8 | +0.416571 | +1427.23 | +
| 3072 | +756 | +128 | +0.85 | +0.318094 | +1869.08 | +
| 3072 | +756 | +128 | +0.9 | +0.225023 | +2642.14 | +
| 256 | +256 | +256 | +0.7 | +0.037509 | +894.564 | +
| 256 | +256 | +256 | +0.75 | +0.03176 | +1056.49 | +
| 256 | +256 | +256 | +0.8 | +0.026189 | +1281.25 | +
| 256 | +256 | +256 | +0.85 | +0.021115 | +1589.12 | +
| 256 | +256 | +256 | +0.9 | +0.015895 | +2110.99 | +
| 256 | +1024 | +256 | +0.7 | +0.141333 | +949.658 | +
| 256 | +1024 | +256 | +0.75 | +0.133647 | +1004.27 | +
| 256 | +1024 | +256 | +0.8 | +0.1 | +1342.18 | +
| 256 | +1024 | +256 | +0.85 | +0.071033 | +1889.52 | +
| 256 | +1024 | +256 | +0.9 | +0.042992 | +3121.95 | +
| 1024 | +256 | +256 | +0.7 | +0.147474 | +910.11 | +
| 1024 | +256 | +256 | +0.75 | +0.125344 | +1070.79 | +
| 1024 | +256 | +256 | +0.8 | +0.104294 | +1286.92 | +
| 1024 | +256 | +256 | +0.85 | +0.084977 | +1579.47 | +
| 1024 | +256 | +256 | +0.9 | +0.062473 | +2148.41 | +
| 1024 | +1024 | +256 | +0.7 | +0.701639 | +765.167 | +
| 1024 | +1024 | +256 | +0.75 | +0.534615 | +1004.22 | +
| 1024 | +1024 | +256 | +0.8 | +0.359791 | +1492.17 | +
| 1024 | +1024 | +256 | +0.85 | +0.2612 | +2055.4 | +
| 1024 | +1024 | +256 | +0.9 | +0.183193 | +2930.62 | +
| 1024 | +4096 | +256 | +0.7 | +2.92067 | +735.271 | +
| 1024 | +4096 | +256 | +0.75 | +2.4402 | +880.046 | +
| 1024 | +4096 | +256 | +0.8 | +1.69186 | +1269.3 | +
| 1024 | +4096 | +256 | +0.85 | +1.17436 | +1828.64 | +
| 1024 | +4096 | +256 | +0.9 | +0.880179 | +2439.83 | +
| 4096 | +1024 | +256 | +0.7 | +2.62123 | +819.264 | +
| 4096 | +1024 | +256 | +0.75 | +2.14905 | +999.271 | +
| 4096 | +1024 | +256 | +0.8 | +1.64253 | +1307.43 | +
| 4096 | +1024 | +256 | +0.85 | +1.1413 | +1881.61 | +
| 4096 | +1024 | +256 | +0.9 | +0.771439 | +2783.74 | +
| 768 | +768 | +256 | +0.7 | +0.369429 | +817.451 | +
| 768 | +768 | +256 | +0.75 | +0.288236 | +1047.72 | +
| 768 | +768 | +256 | +0.8 | +0.209526 | +1441.3 | +
| 768 | +768 | +256 | +0.85 | +0.148115 | +2038.89 | +
| 768 | +768 | +256 | +0.9 | +0.106089 | +2846.58 | +
| 768 | +3072 | +256 | +0.7 | +1.53204 | +788.465 | +
| 768 | +3072 | +256 | +0.75 | +1.28593 | +939.368 | +
| 768 | +3072 | +256 | +0.8 | +1.0852 | +1113.12 | +
| 768 | +3072 | +256 | +0.85 | +0.685382 | +1762.46 | +
| 768 | +3072 | +256 | +0.9 | +0.408357 | +2958.09 | +
| 3072 | +756 | +256 | +0.7 | +1.4521 | +818.87 | +
| 3072 | +756 | +256 | +0.75 | +1.13466 | +1047.97 | +
| 3072 | +756 | +256 | +0.8 | +0.853916 | +1392.51 | +
| 3072 | +756 | +256 | +0.85 | +0.642672 | +1850.22 | +
| 3072 | +756 | +256 | +0.9 | +0.446442 | +2663.47 | +
| 256 | +256 | +384 | +0.7 | +0.057419 | +876.569 | +
| 256 | +256 | +384 | +0.75 | +0.046081 | +1092.24 | +
| 256 | +256 | +384 | +0.8 | +0.038936 | +1292.69 | +
| 256 | +256 | +384 | +0.85 | +0.031279 | +1609.12 | +
| 256 | +256 | +384 | +0.9 | +0.02345 | +2146.36 | +
| 256 | +1024 | +384 | +0.7 | +0.204966 | +982.242 | +
| 256 | +1024 | +384 | +0.75 | +0.17364 | +1159.45 | +
| 256 | +1024 | +384 | +0.8 | +0.141097 | +1426.86 | +
| 256 | +1024 | +384 | +0.85 | +0.105192 | +1913.89 | +
| 256 | +1024 | +384 | +0.9 | +0.067777 | +2970.44 | +
| 1024 | +256 | +384 | +0.7 | +0.21269 | +946.573 | +
| 1024 | +256 | +384 | +0.75 | +0.191564 | +1050.96 | +
| 1024 | +256 | +384 | +0.8 | +0.156368 | +1287.52 | +
| 1024 | +256 | +384 | +0.85 | +0.1286 | +1565.52 | +
| 1024 | +256 | +384 | +0.9 | +0.093357 | +2156.53 | +
| 1024 | +1024 | +384 | +0.7 | +0.84983 | +947.609 | +
| 1024 | +1024 | +384 | +0.75 | +0.789777 | +1019.66 | +
| 1024 | +1024 | +384 | +0.8 | +0.598168 | +1346.29 | +
| 1024 | +1024 | +384 | +0.85 | +0.433492 | +1857.72 | +
| 1024 | +1024 | +384 | +0.9 | +0.269504 | +2988.11 | +
| 1024 | +4096 | +384 | +0.7 | +3.74845 | +859.348 | +
| 1024 | +4096 | +384 | +0.75 | +3.96945 | +811.505 | +
| 1024 | +4096 | +384 | +0.8 | +2.77082 | +1162.55 | +
| 1024 | +4096 | +384 | +0.85 | +1.76407 | +1826.02 | +
| 1024 | +4096 | +384 | +0.9 | +1.09719 | +2935.88 | +
| 4096 | +1024 | +384 | +0.7 | +3.73162 | +863.224 | +
| 4096 | +1024 | +384 | +0.75 | +3.00241 | +1072.88 | +
| 4096 | +1024 | +384 | +0.8 | +2.39024 | +1347.66 | +
| 4096 | +1024 | +384 | +0.85 | +1.77317 | +1816.65 | +
| 4096 | +1024 | +384 | +0.9 | +1.19536 | +2694.79 | +
| 768 | +768 | +384 | +0.7 | +0.517278 | +875.708 | +
| 768 | +768 | +384 | +0.75 | +0.382208 | +1185.18 | +
| 768 | +768 | +384 | +0.8 | +0.317837 | +1425.21 | +
| 768 | +768 | +384 | +0.85 | +0.233231 | +1942.22 | +
| 768 | +768 | +384 | +0.9 | +0.152925 | +2962.14 | +
| 768 | +3072 | +384 | +0.7 | +2.44309 | +741.658 | +
| 768 | +3072 | +384 | +0.75 | +1.62409 | +1115.66 | +
| 768 | +3072 | +384 | +0.8 | +1.64963 | +1098.39 | +
| 768 | +3072 | +384 | +0.85 | +1.03923 | +1743.55 | +
| 768 | +3072 | +384 | +0.9 | +0.625089 | +2898.69 | +
| 3072 | +756 | +384 | +0.7 | +2.07006 | +861.632 | +
| 3072 | +756 | +384 | +0.75 | +1.64247 | +1085.94 | +
| 3072 | +756 | +384 | +0.8 | +1.33386 | +1337.2 | +
| 3072 | +756 | +384 | +0.85 | +0.994935 | +1792.71 | +
| 3072 | +756 | +384 | +0.9 | +0.68383 | +2608.29 | +
| 256 | +256 | +512 | +0.7 | +0.071358 | +940.456 | +
| 256 | +256 | +512 | +0.75 | +0.061246 | +1095.73 | +
| 256 | +256 | +512 | +0.8 | +0.051679 | +1298.58 | +
| 256 | +256 | +512 | +0.85 | +0.041483 | +1617.73 | +
| 256 | +256 | +512 | +0.9 | +0.030989 | +2165.6 | +
| 256 | +1024 | +512 | +0.7 | +0.282241 | +951.085 | +
| 256 | +1024 | +512 | +0.75 | +0.238855 | +1123.84 | +
| 256 | +1024 | +512 | +0.8 | +0.193713 | +1385.74 | +
| 256 | +1024 | +512 | +0.85 | +0.146781 | +1828.81 | +
| 256 | +1024 | +512 | +0.9 | +0.093896 | +2858.86 | +
| 1024 | +256 | +512 | +0.7 | +0.286141 | +938.122 | +
| 1024 | +256 | +512 | +0.75 | +0.257138 | +1043.94 | +
| 1024 | +256 | +512 | +0.8 | +0.214208 | +1253.16 | +
| 1024 | +256 | +512 | +0.85 | +0.167575 | +1601.88 | +
| 1024 | +256 | +512 | +0.9 | +0.124072 | +2163.54 | +
| 1024 | +1024 | +512 | +0.7 | +1.21431 | +884.237 | +
| 1024 | +1024 | +512 | +0.75 | +0.955251 | +1124.04 | +
| 1024 | +1024 | +512 | +0.8 | +0.775294 | +1384.95 | +
| 1024 | +1024 | +512 | +0.85 | +0.63282 | +1696.76 | +
| 1024 | +1024 | +512 | +0.9 | +0.37333 | +2876.12 | +
| 1024 | +4096 | +512 | +0.7 | +4.87335 | +881.317 | +
| 1024 | +4096 | +512 | +0.75 | +3.65059 | +1176.51 | +
| 1024 | +4096 | +512 | +0.8 | +2.94733 | +1457.24 | +
| 1024 | +4096 | +512 | +0.85 | +2.62241 | +1637.79 | +
| 1024 | +4096 | +512 | +0.9 | +1.47752 | +2906.88 | +
| 4096 | +1024 | +512 | +0.7 | +5.75786 | +745.931 | +
| 4096 | +1024 | +512 | +0.75 | +4.83024 | +889.184 | +
| 4096 | +1024 | +512 | +0.8 | +3.87429 | +1108.58 | +
| 4096 | +1024 | +512 | +0.85 | +2.87685 | +1492.94 | +
| 4096 | +1024 | +512 | +0.9 | +1.75328 | +2449.68 | +
| 768 | +768 | +512 | +0.7 | +0.718776 | +840.289 | +
| 768 | +768 | +512 | +0.75 | +0.561317 | +1076.01 | +
| 768 | +768 | +512 | +0.8 | +0.429166 | +1407.33 | +
| 768 | +768 | +512 | +0.85 | +0.317204 | +1904.07 | +
| 768 | +768 | +512 | +0.9 | +0.208555 | +2896.02 | +
| 768 | +3072 | +512 | +0.7 | +2.94057 | +821.581 | +
| 768 | +3072 | +512 | +0.75 | +2.30458 | +1048.31 | +
| 768 | +3072 | +512 | +0.8 | +1.6702 | +1446.49 | +
| 768 | +3072 | +512 | +0.85 | +1.25799 | +1920.47 | +
| 768 | +3072 | +512 | +0.9 | +0.84121 | +2871.96 | +
| 3072 | +756 | +512 | +0.7 | +3.34423 | +711.127 | +
| 3072 | +756 | +512 | +0.75 | +2.77444 | +857.17 | +
| 3072 | +756 | +512 | +0.8 | +2.10049 | +1132.2 | +
| 3072 | +756 | +512 | +0.85 | +1.52509 | +1559.37 | +
| 3072 | +756 | +512 | +0.9 | +0.926038 | +2568.11 | +
| M | +K | +N | +Sparsity | +Latency (ms) | +Gflops | +
|---|---|---|---|---|---|
| 256 | +256 | +16 | +0.7 | +0.00203 | +1032.94 | +
| 256 | +256 | +16 | +0.75 | +0.001981 | +1058.84 | +
| 256 | +256 | +16 | +0.8 | +0.001794 | +1169.1 | +
| 256 | +256 | +16 | +0.85 | +0.001637 | +1280.99 | +
| 256 | +256 | +16 | +0.9 | +0.00171 | +1226.24 | +
| 256 | +1024 | +16 | +0.7 | +0.003958 | +2119.5 | +
| 256 | +1024 | +16 | +0.75 | +0.003405 | +2463.96 | +
| 256 | +1024 | +16 | +0.8 | +0.003086 | +2718.43 | +
| 256 | +1024 | +16 | +0.85 | +0.002927 | +2866.1 | +
| 256 | +1024 | +16 | +0.9 | +0.002179 | +3849.95 | +
| 1024 | +256 | +16 | +0.7 | +0.004371 | +1918.96 | +
| 1024 | +256 | +16 | +0.75 | +0.004182 | +2005.83 | +
| 1024 | +256 | +16 | +0.8 | +0.003448 | +2433 | +
| 1024 | +256 | +16 | +0.85 | +0.00291 | +2882.63 | +
| 1024 | +256 | +16 | +0.9 | +0.002937 | +2856.46 | +
| 1024 | +1024 | +16 | +0.7 | +0.012095 | +2774.32 | +
| 1024 | +1024 | +16 | +0.75 | +0.010338 | +3245.84 | +
| 1024 | +1024 | +16 | +0.8 | +0.008508 | +3943.97 | +
| 1024 | +1024 | +16 | +0.85 | +0.006595 | +5087.77 | +
| 1024 | +1024 | +16 | +0.9 | +0.004847 | +6923.16 | +
| 1024 | +4096 | +16 | +0.7 | +0.052451 | +2558.92 | +
| 1024 | +4096 | +16 | +0.75 | +0.042874 | +3130.54 | +
| 1024 | +4096 | +16 | +0.8 | +0.032364 | +4147.11 | +
| 1024 | +4096 | +16 | +0.85 | +0.02312 | +5805.3 | +
| 1024 | +4096 | +16 | +0.9 | +0.01522 | +8818.59 | +
| 4096 | +1024 | +16 | +0.7 | +0.045123 | +2974.5 | +
| 4096 | +1024 | +16 | +0.75 | +0.037382 | +3590.4 | +
| 4096 | +1024 | +16 | +0.8 | +0.030275 | +4433.35 | +
| 4096 | +1024 | +16 | +0.85 | +0.023182 | +5789.74 | +
| 4096 | +1024 | +16 | +0.9 | +0.01617 | +8300.6 | +
| 768 | +768 | +16 | +0.7 | +0.007314 | +2580.48 | +
| 768 | +768 | +16 | +0.75 | +0.006332 | +2980.89 | +
| 768 | +768 | +16 | +0.8 | +0.005359 | +3522.29 | +
| 768 | +768 | +16 | +0.85 | +0.004493 | +4200.52 | +
| 768 | +768 | +16 | +0.9 | +0.003418 | +5521.87 | +
| 768 | +3072 | +16 | +0.7 | +0.027003 | +2795.91 | +
| 768 | +3072 | +16 | +0.75 | +0.02197 | +3436.39 | +
| 768 | +3072 | +16 | +0.8 | +0.017748 | +4253.93 | +
| 768 | +3072 | +16 | +0.85 | +0.013134 | +5748.48 | +
| 768 | +3072 | +16 | +0.9 | +0.008974 | +8412.5 | +
| 3072 | +756 | +16 | +0.7 | +0.025719 | +2889.63 | +
| 3072 | +756 | +16 | +0.75 | +0.021897 | +3393.99 | +
| 3072 | +756 | +16 | +0.8 | +0.018093 | +4107.48 | +
| 3072 | +756 | +16 | +0.85 | +0.013742 | +5408.06 | +
| 3072 | +756 | +16 | +0.9 | +0.009924 | +7488.55 | +
| 256 | +256 | +32 | +0.7 | +0.002891 | +1450.74 | +
| 256 | +256 | +32 | +0.75 | +0.002458 | +1706.24 | +
| 256 | +256 | +32 | +0.8 | +0.002523 | +1662.22 | +
| 256 | +256 | +32 | +0.85 | +0.002266 | +1851.32 | +
| 256 | +256 | +32 | +0.9 | +0.001803 | +2326.09 | +
| 256 | +1024 | +32 | +0.7 | +0.005789 | +2898.1 | +
| 256 | +1024 | +32 | +0.75 | +0.005004 | +3353.02 | +
| 256 | +1024 | +32 | +0.8 | +0.00424 | +3956.8 | +
| 256 | +1024 | +32 | +0.85 | +0.003505 | +4786.85 | +
| 256 | +1024 | +32 | +0.9 | +0.002972 | +5644.46 | +
| 1024 | +256 | +32 | +0.7 | +0.006933 | +2420.03 | +
| 1024 | +256 | +32 | +0.75 | +0.005931 | +2828.91 | +
| 1024 | +256 | +32 | +0.8 | +0.005267 | +3185.37 | +
| 1024 | +256 | +32 | +0.85 | +0.004801 | +3494.54 | +
| 1024 | +256 | +32 | +0.9 | +0.003851 | +4356.24 | +
| 1024 | +1024 | +32 | +0.7 | +0.019352 | +3467.87 | +
| 1024 | +1024 | +32 | +0.75 | +0.016625 | +4036.54 | +
| 1024 | +1024 | +32 | +0.8 | +0.013287 | +5050.74 | +
| 1024 | +1024 | +32 | +0.85 | +0.010629 | +6313.59 | +
| 1024 | +1024 | +32 | +0.9 | +0.007591 | +8841.1 | +
| 1024 | +4096 | +32 | +0.7 | +0.097726 | +2746.83 | +
| 1024 | +4096 | +32 | +0.75 | +0.081327 | +3300.69 | +
| 1024 | +4096 | +32 | +0.8 | +0.065632 | +4090 | +
| 1024 | +4096 | +32 | +0.85 | +0.047646 | +5633.92 | +
| 1024 | +4096 | +32 | +0.9 | +0.029536 | +9088.36 | +
| 4096 | +1024 | +32 | +0.7 | +0.073619 | +3646.28 | +
| 4096 | +1024 | +32 | +0.75 | +0.060379 | +4445.85 | +
| 4096 | +1024 | +32 | +0.8 | +0.048994 | +5478.97 | +
| 4096 | +1024 | +32 | +0.85 | +0.036992 | +7256.58 | +
| 4096 | +1024 | +32 | +0.9 | +0.02588 | +10372.2 | +
| 768 | +768 | +32 | +0.7 | +0.011659 | +3237.75 | +
| 768 | +768 | +32 | +0.75 | +0.009791 | +3855.4 | +
| 768 | +768 | +32 | +0.8 | +0.008118 | +4650.12 | +
| 768 | +768 | +32 | +0.85 | +0.006679 | +5652.1 | +
| 768 | +768 | +32 | +0.9 | +0.004939 | +7643.6 | +
| 768 | +3072 | +32 | +0.7 | +0.053777 | +2807.81 | +
| 768 | +3072 | +32 | +0.75 | +0.045914 | +3288.68 | +
| 768 | +3072 | +32 | +0.8 | +0.03588 | +4208.33 | +
| 768 | +3072 | +32 | +0.85 | +0.02529 | +5970.53 | +
| 768 | +3072 | +32 | +0.9 | +0.016742 | +9018.97 | +
| 3072 | +756 | +32 | +0.7 | +0.040705 | +3651.53 | +
| 3072 | +756 | +32 | +0.75 | +0.034291 | +4334.57 | +
| 3072 | +756 | +32 | +0.8 | +0.028568 | +5202.87 | +
| 3072 | +756 | +32 | +0.85 | +0.02187 | +6796.25 | +
| 3072 | +756 | +32 | +0.9 | +0.016186 | +9183 | +
| 256 | +256 | +48 | +0.7 | +0.003155 | +1994.06 | +
| 256 | +256 | +48 | +0.75 | +0.002952 | +2131.62 | +
| 256 | +256 | +48 | +0.8 | +0.002655 | +2369.77 | +
| 256 | +256 | +48 | +0.85 | +0.002436 | +2582.9 | +
| 256 | +256 | +48 | +0.9 | +0.002236 | +2813.73 | +
| 256 | +1024 | +48 | +0.7 | +0.00745 | +3377.93 | +
| 256 | +1024 | +48 | +0.75 | +0.006565 | +3833.08 | +
| 256 | +1024 | +48 | +0.8 | +0.005426 | +4638.05 | +
| 256 | +1024 | +48 | +0.85 | +0.004256 | +5912.38 | +
| 256 | +1024 | +48 | +0.9 | +0.003466 | +7261.4 | +
| 1024 | +256 | +48 | +0.7 | +0.008359 | +3010.51 | +
| 1024 | +256 | +48 | +0.75 | +0.00742 | +3391.82 | +
| 1024 | +256 | +48 | +0.8 | +0.006535 | +3850.81 | +
| 1024 | +256 | +48 | +0.85 | +0.005559 | +4526.9 | +
| 1024 | +256 | +48 | +0.9 | +0.004881 | +5156.31 | +
| 1024 | +1024 | +48 | +0.7 | +0.025726 | +3912.87 | +
| 1024 | +1024 | +48 | +0.75 | +0.021747 | +4628.88 | +
| 1024 | +1024 | +48 | +0.8 | +0.017651 | +5703.16 | +
| 1024 | +1024 | +48 | +0.85 | +0.013712 | +7341.14 | +
| 1024 | +1024 | +48 | +0.9 | +0.009801 | +10270.4 | +
| 1024 | +4096 | +48 | +0.7 | +0.124749 | +3227.71 | +
| 1024 | +4096 | +48 | +0.75 | +0.103385 | +3894.71 | +
| 1024 | +4096 | +48 | +0.8 | +0.083219 | +4838.46 | +
| 1024 | +4096 | +48 | +0.85 | +0.062601 | +6432.03 | +
| 1024 | +4096 | +48 | +0.9 | +0.039799 | +10117.1 | +
| 4096 | +1024 | +48 | +0.7 | +0.101071 | +3983.87 | +
| 4096 | +1024 | +48 | +0.75 | +0.082567 | +4876.71 | +
| 4096 | +1024 | +48 | +0.8 | +0.066852 | +6023.1 | +
| 4096 | +1024 | +48 | +0.85 | +0.048773 | +8255.7 | +
| 4096 | +1024 | +48 | +0.9 | +0.035419 | +11368.2 | +
| 768 | +768 | +48 | +0.7 | +0.014772 | +3833.19 | +
| 768 | +768 | +48 | +0.75 | +0.01278 | +4430.65 | +
| 768 | +768 | +48 | +0.8 | +0.010532 | +5376.08 | +
| 768 | +768 | +48 | +0.85 | +0.008698 | +6509.93 | +
| 768 | +768 | +48 | +0.9 | +0.006436 | +8798 | +
| 768 | +3072 | +48 | +0.7 | +0.069887 | +3240.85 | +
| 768 | +3072 | +48 | +0.75 | +0.058858 | +3848.1 | +
| 768 | +3072 | +48 | +0.8 | +0.047341 | +4784.26 | +
| 768 | +3072 | +48 | +0.85 | +0.034614 | +6543.34 | +
| 768 | +3072 | +48 | +0.9 | +0.022295 | +10159 | +
| 3072 | +756 | +48 | +0.7 | +0.053815 | +4142.99 | +
| 3072 | +756 | +48 | +0.75 | +0.045458 | +4904.58 | +
| 3072 | +756 | +48 | +0.8 | +0.037358 | +5968.05 | +
| 3072 | +756 | +48 | +0.85 | +0.02914 | +7651.21 | +
| 3072 | +756 | +48 | +0.9 | +0.02089 | +10672.7 | +
| 256 | +256 | +64 | +0.7 | +0.003936 | +2131.31 | +
| 256 | +256 | +64 | +0.75 | +0.003322 | +2525.07 | +
| 256 | +256 | +64 | +0.8 | +0.003173 | +2643.78 | +
| 256 | +256 | +64 | +0.85 | +0.002691 | +3117.13 | +
| 256 | +256 | +64 | +0.9 | +0.002358 | +3557.32 | +
| 256 | +1024 | +64 | +0.7 | +0.009382 | +3576.47 | +
| 256 | +1024 | +64 | +0.75 | +0.008289 | +4047.86 | +
| 256 | +1024 | +64 | +0.8 | +0.006724 | +4990.61 | +
| 256 | +1024 | +64 | +0.85 | +0.005148 | +6518.36 | +
| 256 | +1024 | +64 | +0.9 | +0.004061 | +8262.2 | +
| 1024 | +256 | +64 | +0.7 | +0.010712 | +3132.55 | +
| 1024 | +256 | +64 | +0.75 | +0.009203 | +3646.22 | +
| 1024 | +256 | +64 | +0.8 | +0.007943 | +4224.63 | +
| 1024 | +256 | +64 | +0.85 | +0.006981 | +4806.38 | +
| 1024 | +256 | +64 | +0.9 | +0.00561 | +5980.91 | +
| 1024 | +1024 | +64 | +0.7 | +0.034567 | +3882.83 | +
| 1024 | +1024 | +64 | +0.75 | +0.028972 | +4632.68 | +
| 1024 | +1024 | +64 | +0.8 | +0.023258 | +5770.71 | +
| 1024 | +1024 | +64 | +0.85 | +0.017977 | +7466.26 | +
| 1024 | +1024 | +64 | +0.9 | +0.012814 | +10474 | +
| 1024 | +4096 | +64 | +0.7 | +0.158162 | +3394.43 | +
| 1024 | +4096 | +64 | +0.75 | +0.132552 | +4050.27 | +
| 1024 | +4096 | +64 | +0.8 | +0.101844 | +5271.51 | +
| 1024 | +4096 | +64 | +0.85 | +0.080936 | +6633.28 | +
| 1024 | +4096 | +64 | +0.9 | +0.05301 | +10127.8 | +
| 4096 | +1024 | +64 | +0.7 | +0.135839 | +3952.25 | +
| 4096 | +1024 | +64 | +0.75 | +0.11116 | +4829.72 | +
| 4096 | +1024 | +64 | +0.8 | +0.089951 | +5968.46 | +
| 4096 | +1024 | +64 | +0.85 | +0.063302 | +8481.15 | +
| 4096 | +1024 | +64 | +0.9 | +0.046746 | +11484.8 | +
| 768 | +768 | +64 | +0.7 | +0.019263 | +3919.34 | +
| 768 | +768 | +64 | +0.75 | +0.016459 | +4586.97 | +
| 768 | +768 | +64 | +0.8 | +0.013535 | +5578.04 | +
| 768 | +768 | +64 | +0.85 | +0.010771 | +7009.2 | +
| 768 | +768 | +64 | +0.9 | +0.007611 | +9919.29 | +
| 768 | +3072 | +64 | +0.7 | +0.085488 | +3532.56 | +
| 768 | +3072 | +64 | +0.75 | +0.074727 | +4041.24 | +
| 768 | +3072 | +64 | +0.8 | +0.060476 | +4993.56 | +
| 768 | +3072 | +64 | +0.85 | +0.041796 | +7225.26 | +
| 768 | +3072 | +64 | +0.9 | +0.027547 | +10962.8 | +
| 3072 | +756 | +64 | +0.7 | +0.071382 | +4164.51 | +
| 3072 | +756 | +64 | +0.75 | +0.060077 | +4948.16 | +
| 3072 | +756 | +64 | +0.8 | +0.049658 | +5986.36 | +
| 3072 | +756 | +64 | +0.85 | +0.038139 | +7794.53 | +
| 3072 | +756 | +64 | +0.9 | +0.027025 | +10999.9 | +
| 256 | +256 | +80 | +0.7 | +0.005025 | +2086.83 | +
| 256 | +256 | +80 | +0.75 | +0.004519 | +2320.6 | +
| 256 | +256 | +80 | +0.8 | +0.003968 | +2642.46 | +
| 256 | +256 | +80 | +0.85 | +0.00339 | +3093.49 | +
| 256 | +256 | +80 | +0.9 | +0.003153 | +3325.19 | +
| 256 | +1024 | +80 | +0.7 | +0.014795 | +2834.92 | +
| 256 | +1024 | +80 | +0.75 | +0.012623 | +3322.87 | +
| 256 | +1024 | +80 | +0.8 | +0.010269 | +4084.43 | +
| 256 | +1024 | +80 | +0.85 | +0.007859 | +5337.09 | +
| 256 | +1024 | +80 | +0.9 | +0.005703 | +7354.65 | +
| 1024 | +256 | +80 | +0.7 | +0.016169 | +2594.02 | +
| 1024 | +256 | +80 | +0.75 | +0.014448 | +2902.97 | +
| 1024 | +256 | +80 | +0.8 | +0.011705 | +3583.33 | +
| 1024 | +256 | +80 | +0.85 | +0.010149 | +4132.84 | +
| 1024 | +256 | +80 | +0.9 | +0.008793 | +4769.87 | +
| 1024 | +1024 | +80 | +0.7 | +0.05603 | +2994.34 | +
| 1024 | +1024 | +80 | +0.75 | +0.0468 | +3584.89 | +
| 1024 | +1024 | +80 | +0.8 | +0.03744 | +4481.13 | +
| 1024 | +1024 | +80 | +0.85 | +0.028166 | +5956.55 | +
| 1024 | +1024 | +80 | +0.9 | +0.019259 | +8711.21 | +
| 1024 | +4096 | +80 | +0.7 | +0.277815 | +2415.59 | +
| 1024 | +4096 | +80 | +0.75 | +0.229837 | +2919.85 | +
| 1024 | +4096 | +80 | +0.8 | +0.1787 | +3755.4 | +
| 1024 | +4096 | +80 | +0.85 | +0.126545 | +5303.18 | +
| 1024 | +4096 | +80 | +0.9 | +0.083532 | +8033.9 | +
| 4096 | +1024 | +80 | +0.7 | +0.224925 | +2983.61 | +
| 4096 | +1024 | +80 | +0.75 | +0.184256 | +3642.16 | +
| 4096 | +1024 | +80 | +0.8 | +0.146452 | +4582.3 | +
| 4096 | +1024 | +80 | +0.85 | +0.108899 | +6162.5 | +
| 4096 | +1024 | +80 | +0.9 | +0.074066 | +9060.75 | +
| 768 | +768 | +80 | +0.7 | +0.031946 | +2954.15 | +
| 768 | +768 | +80 | +0.75 | +0.026814 | +3519.55 | +
| 768 | +768 | +80 | +0.8 | +0.021706 | +4347.69 | +
| 768 | +768 | +80 | +0.85 | +0.016667 | +5662.34 | +
| 768 | +768 | +80 | +0.9 | +0.011647 | +8102.93 | +
| 768 | +3072 | +80 | +0.7 | +0.150549 | +2507.4 | +
| 768 | +3072 | +80 | +0.75 | +0.119951 | +3147.02 | +
| 768 | +3072 | +80 | +0.8 | +0.089309 | +4226.74 | +
| 768 | +3072 | +80 | +0.85 | +0.063723 | +5923.92 | +
| 768 | +3072 | +80 | +0.9 | +0.042784 | +8823.11 | +
| 3072 | +756 | +80 | +0.7 | +0.122421 | +3035.33 | +
| 3072 | +756 | +80 | +0.75 | +0.102244 | +3634.35 | +
| 3072 | +756 | +80 | +0.8 | +0.082201 | +4520.51 | +
| 3072 | +756 | +80 | +0.85 | +0.062248 | +5969.46 | +
| 3072 | +756 | +80 | +0.9 | +0.042944 | +8652.96 | +
| 256 | +256 | +96 | +0.7 | +0.004811 | +2615.59 | +
| 256 | +256 | +96 | +0.75 | +0.004438 | +2835.29 | +
| 256 | +256 | +96 | +0.8 | +0.003899 | +3227.52 | +
| 256 | +256 | +96 | +0.85 | +0.003438 | +3659.72 | +
| 256 | +256 | +96 | +0.9 | +0.002978 | +4225.52 | +
| 256 | +1024 | +96 | +0.7 | +0.014576 | +3453.11 | +
| 256 | +1024 | +96 | +0.75 | +0.012078 | +4167.21 | +
| 256 | +1024 | +96 | +0.8 | +0.008963 | +5615.31 | +
| 256 | +1024 | +96 | +0.85 | +0.007276 | +6917.06 | +
| 256 | +1024 | +96 | +0.9 | +0.005485 | +9176.61 | +
| 1024 | +256 | +96 | +0.7 | +0.015062 | +3341.54 | +
| 1024 | +256 | +96 | +0.75 | +0.013243 | +3800.69 | +
| 1024 | +256 | +96 | +0.8 | +0.011443 | +4398.48 | +
| 1024 | +256 | +96 | +0.85 | +0.009288 | +5418.82 | +
| 1024 | +256 | +96 | +0.9 | +0.007446 | +6759.98 | +
| 1024 | +1024 | +96 | +0.7 | +0.054398 | +3701.01 | +
| 1024 | +1024 | +96 | +0.75 | +0.044569 | +4517.16 | +
| 1024 | +1024 | +96 | +0.8 | +0.036333 | +5541.17 | +
| 1024 | +1024 | +96 | +0.85 | +0.027049 | +7442.97 | +
| 1024 | +1024 | +96 | +0.9 | +0.018742 | +10742.3 | +
| 1024 | +4096 | +96 | +0.7 | +0.267367 | +3011.99 | +
| 1024 | +4096 | +96 | +0.75 | +0.223615 | +3601.31 | +
| 1024 | +4096 | +96 | +0.8 | +0.1797 | +4481.38 | +
| 1024 | +4096 | +96 | +0.85 | +0.134716 | +5977.82 | +
| 1024 | +4096 | +96 | +0.9 | +0.069275 | +11624.7 | +
| 4096 | +1024 | +96 | +0.7 | +0.222661 | +3616.73 | +
| 4096 | +1024 | +96 | +0.75 | +0.179525 | +4485.75 | +
| 4096 | +1024 | +96 | +0.8 | +0.139298 | +5781.18 | +
| 4096 | +1024 | +96 | +0.85 | +0.095879 | +8399.22 | +
| 4096 | +1024 | +96 | +0.9 | +0.176292 | +4568.02 | +
| 768 | +768 | +96 | +0.7 | +0.070028 | +1617.15 | +
| 768 | +768 | +96 | +0.75 | +0.025834 | +4383.59 | +
| 768 | +768 | +96 | +0.8 | +0.02075 | +5457.53 | +
| 768 | +768 | +96 | +0.85 | +0.016128 | +7021.85 | +
| 768 | +768 | +96 | +0.9 | +0.011601 | +9761.93 | +
| 768 | +3072 | +96 | +0.7 | +0.148512 | +3050.15 | +
| 768 | +3072 | +96 | +0.75 | +0.124238 | +3646.1 | +
| 768 | +3072 | +96 | +0.8 | +0.09953 | +4551.22 | +
| 768 | +3072 | +96 | +0.85 | +0.071432 | +6341.45 | +
| 768 | +3072 | +96 | +0.9 | +0.044154 | +10259.2 | +
| 3072 | +756 | +96 | +0.7 | +0.113174 | +3940.02 | +
| 3072 | +756 | +96 | +0.75 | +0.096166 | +4636.84 | +
| 3072 | +756 | +96 | +0.8 | +0.076584 | +5822.45 | +
| 3072 | +756 | +96 | +0.85 | +0.059291 | +7520.62 | +
| 3072 | +756 | +96 | +0.9 | +0.041661 | +10703.1 | +
| 256 | +256 | +112 | +0.7 | +0.006557 | +2239 | +
| 256 | +256 | +112 | +0.75 | +0.005848 | +2510.44 | +
| 256 | +256 | +112 | +0.8 | +0.005021 | +2923.66 | +
| 256 | +256 | +112 | +0.85 | +0.004663 | +3148.41 | +
| 256 | +256 | +112 | +0.9 | +0.003933 | +3732.77 | +
| 256 | +1024 | +112 | +0.7 | +0.020392 | +2879.53 | +
| 256 | +1024 | +112 | +0.75 | +0.017102 | +3433.58 | +
| 256 | +1024 | +112 | +0.8 | +0.013703 | +4285.32 | +
| 256 | +1024 | +112 | +0.85 | +0.010953 | +5361.24 | +
| 256 | +1024 | +112 | +0.9 | +0.007496 | +7833.17 | +
| 1024 | +256 | +112 | +0.7 | +0.022132 | +2653.19 | +
| 1024 | +256 | +112 | +0.75 | +0.019376 | +3030.55 | +
| 1024 | +256 | +112 | +0.8 | +0.015877 | +3698.38 | +
| 1024 | +256 | +112 | +0.85 | +0.013437 | +4370.15 | +
| 1024 | +256 | +112 | +0.9 | +0.011637 | +5046.22 | +
| 1024 | +1024 | +112 | +0.7 | +0.078075 | +3008.4 | +
| 1024 | +1024 | +112 | +0.75 | +0.065155 | +3604.95 | +
| 1024 | +1024 | +112 | +0.8 | +0.052195 | +4500.11 | +
| 1024 | +1024 | +112 | +0.85 | +0.03941 | +5959.92 | +
| 1024 | +1024 | +112 | +0.9 | +0.026271 | +8940.83 | +
| 1024 | +4096 | +112 | +0.7 | +0.398509 | +2357.6 | +
| 1024 | +4096 | +112 | +0.75 | +0.332153 | +2828.58 | +
| 1024 | +4096 | +112 | +0.8 | +0.259051 | +3626.8 | +
| 1024 | +4096 | +112 | +0.85 | +0.184412 | +5094.69 | +
| 1024 | +4096 | +112 | +0.9 | +0.119678 | +7850.42 | +
| 4096 | +1024 | +112 | +0.7 | +0.313656 | +2995.4 | +
| 4096 | +1024 | +112 | +0.75 | +0.257989 | +3641.72 | +
| 4096 | +1024 | +112 | +0.8 | +0.206049 | +4559.71 | +
| 4096 | +1024 | +112 | +0.85 | +0.152848 | +6146.79 | +
| 4096 | +1024 | +112 | +0.9 | +0.103125 | +9110.51 | +
| 768 | +768 | +112 | +0.7 | +0.044486 | +2969.93 | +
| 768 | +768 | +112 | +0.75 | +0.037334 | +3538.88 | +
| 768 | +768 | +112 | +0.8 | +0.030229 | +4370.61 | +
| 768 | +768 | +112 | +0.85 | +0.023198 | +5695.36 | +
| 768 | +768 | +112 | +0.9 | +0.01587 | +8325.1 | +
| 768 | +3072 | +112 | +0.7 | +0.219777 | +2404.63 | +
| 768 | +3072 | +112 | +0.75 | +0.17764 | +2975.03 | +
| 768 | +3072 | +112 | +0.8 | +0.135864 | +3889.79 | +
| 768 | +3072 | +112 | +0.85 | +0.100424 | +5262.52 | +
| 768 | +3072 | +112 | +0.9 | +0.067755 | +7799.93 | +
| 3072 | +756 | +112 | +0.7 | +0.172356 | +3018.32 | +
| 3072 | +756 | +112 | +0.75 | +0.144231 | +3606.89 | +
| 3072 | +756 | +112 | +0.8 | +0.116019 | +4483.96 | +
| 3072 | +756 | +112 | +0.85 | +0.087388 | +5953.04 | +
| 3072 | +756 | +112 | +0.9 | +0.0596 | +8728.54 | +
| 256 | +256 | +128 | +0.7 | +0.006097 | +2751.74 | +
| 256 | +256 | +128 | +0.75 | +0.005176 | +3241.12 | +
| 256 | +256 | +128 | +0.8 | +0.00449 | +3736.94 | +
| 256 | +256 | +128 | +0.85 | +0.003949 | +4248.73 | +
| 256 | +256 | +128 | +0.9 | +0.003175 | +5284.78 | +
| 256 | +1024 | +128 | +0.7 | +0.020483 | +3276.3 | +
| 256 | +1024 | +128 | +0.75 | +0.013769 | +4873.87 | +
| 256 | +1024 | +128 | +0.8 | +0.011419 | +5876.74 | +
| 256 | +1024 | +128 | +0.85 | +0.008933 | +7512.43 | +
| 256 | +1024 | +128 | +0.9 | +0.006533 | +10272.1 | +
| 1024 | +256 | +128 | +0.7 | +0.019241 | +3487.82 | +
| 1024 | +256 | +128 | +0.75 | +0.016835 | +3986.32 | +
| 1024 | +256 | +128 | +0.8 | +0.014234 | +4714.59 | +
| 1024 | +256 | +128 | +0.85 | +0.011946 | +5617.9 | +
| 1024 | +256 | +128 | +0.9 | +0.009106 | +7369.36 | +
| 1024 | +1024 | +128 | +0.7 | +0.07811 | +3436.63 | +
| 1024 | +1024 | +128 | +0.75 | +0.061481 | +4366.18 | +
| 1024 | +1024 | +128 | +0.8 | +0.046201 | +5810.17 | +
| 1024 | +1024 | +128 | +0.85 | +0.035702 | +7518.72 | +
| 1024 | +1024 | +128 | +0.9 | +0.02221 | +12086.1 | +
| 1024 | +4096 | +128 | +0.7 | +0.326917 | +3284.45 | +
| 1024 | +4096 | +128 | +0.75 | +0.273363 | +3927.89 | +
| 1024 | +4096 | +128 | +0.8 | +0.220994 | +4858.7 | +
| 1024 | +4096 | +128 | +0.85 | +0.166153 | +6462.38 | +
| 1024 | +4096 | +128 | +0.9 | +0.08544 | +12567.2 | +
| 4096 | +1024 | +128 | +0.7 | +0.311348 | +3448.68 | +
| 4096 | +1024 | +128 | +0.75 | +0.243012 | +4418.47 | +
| 4096 | +1024 | +128 | +0.8 | +0.187933 | +5713.44 | +
| 4096 | +1024 | +128 | +0.85 | +0.126819 | +8466.75 | +
| 4096 | +1024 | +128 | +0.9 | +0.085569 | +12548.3 | +
| 768 | +768 | +128 | +0.7 | +0.040478 | +3730.31 | +
| 768 | +768 | +128 | +0.75 | +0.03297 | +4579.78 | +
| 768 | +768 | +128 | +0.8 | +0.026527 | +5692.09 | +
| 768 | +768 | +128 | +0.85 | +0.020782 | +7265.85 | +
| 768 | +768 | +128 | +0.9 | +0.013904 | +10860 | +
| 768 | +3072 | +128 | +0.7 | +0.183003 | +3300.39 | +
| 768 | +3072 | +128 | +0.75 | +0.161443 | +3741.12 | +
| 768 | +3072 | +128 | +0.8 | +0.129767 | +4654.33 | +
| 768 | +3072 | +128 | +0.85 | +0.089335 | +6760.88 | +
| 768 | +3072 | +128 | +0.9 | +0.061068 | +9890.22 | +
| 3072 | +756 | +128 | +0.7 | +0.15525 | +3829.58 | +
| 3072 | +756 | +128 | +0.75 | +0.125772 | +4727.15 | +
| 3072 | +756 | +128 | +0.8 | +0.101434 | +5861.35 | +
| 3072 | +756 | +128 | +0.85 | +0.076508 | +7770.96 | +
| 3072 | +756 | +128 | +0.9 | +0.053732 | +11064.9 | +
| 256 | +256 | +256 | +0.7 | +0.010421 | +3219.91 | +
| 256 | +256 | +256 | +0.75 | +0.009082 | +3694.63 | +
| 256 | +256 | +256 | +0.8 | +0.007813 | +4294.82 | +
| 256 | +256 | +256 | +0.85 | +0.006511 | +5153.44 | +
| 256 | +256 | +256 | +0.9 | +0.005172 | +6487.84 | +
| 256 | +1024 | +256 | +0.7 | +0.032138 | +4176.33 | +
| 256 | +1024 | +256 | +0.75 | +0.02699 | +4972.79 | +
| 256 | +1024 | +256 | +0.8 | +0.022001 | +6100.42 | +
| 256 | +1024 | +256 | +0.85 | +0.016901 | +7941.41 | +
| 256 | +1024 | +256 | +0.9 | +0.012004 | +11180.9 | +
| 1024 | +256 | +256 | +0.7 | +0.03738 | +3590.61 | +
| 1024 | +256 | +256 | +0.75 | +0.03149 | +4262.3 | +
| 1024 | +256 | +256 | +0.8 | +0.026673 | +5031.96 | +
| 1024 | +256 | +256 | +0.85 | +0.021782 | +6161.81 | +
| 1024 | +256 | +256 | +0.9 | +0.016822 | +7978.77 | +
| 1024 | +1024 | +256 | +0.7 | +0.176031 | +3049.86 | +
| 1024 | +1024 | +256 | +0.75 | +0.134442 | +3993.33 | +
| 1024 | +1024 | +256 | +0.8 | +0.084378 | +6362.72 | +
| 1024 | +1024 | +256 | +0.85 | +0.064058 | +8381.05 | +
| 1024 | +1024 | +256 | +0.9 | +0.043787 | +12261 | +
| 1024 | +4096 | +256 | +0.7 | +0.697876 | +3077.17 | +
| 1024 | +4096 | +256 | +0.75 | +0.615338 | +3489.92 | +
| 1024 | +4096 | +256 | +0.8 | +0.350147 | +6133.09 | +
| 1024 | +4096 | +256 | +0.85 | +0.267289 | +8034.33 | +
| 1024 | +4096 | +256 | +0.9 | +0.179695 | +11950.7 | +
| 4096 | +1024 | +256 | +0.7 | +0.703207 | +3053.84 | +
| 4096 | +1024 | +256 | +0.75 | +0.561805 | +3822.47 | +
| 4096 | +1024 | +256 | +0.8 | +0.423052 | +5076.17 | +
| 4096 | +1024 | +256 | +0.85 | +0.254835 | +8426.97 | +
| 4096 | +1024 | +256 | +0.9 | +0.171434 | +12526.6 | +
| 768 | +768 | +256 | +0.7 | +0.093348 | +3235.09 | +
| 768 | +768 | +256 | +0.75 | +0.072563 | +4161.75 | +
| 768 | +768 | +256 | +0.8 | +0.05392 | +5600.73 | +
| 768 | +768 | +256 | +0.85 | +0.037612 | +8029.1 | +
| 768 | +768 | +256 | +0.9 | +0.026314 | +11476.5 | +
| 768 | +3072 | +256 | +0.7 | +0.407388 | +2965.13 | +
| 768 | +3072 | +256 | +0.75 | +0.340939 | +3543.04 | +
| 768 | +3072 | +256 | +0.8 | +0.261028 | +4627.69 | +
| 768 | +3072 | +256 | +0.85 | +0.151649 | +7965.49 | +
| 768 | +3072 | +256 | +0.9 | +0.103379 | +11684.7 | +
| 3072 | +756 | +256 | +0.7 | +0.358769 | +3314.35 | +
| 3072 | +756 | +256 | +0.75 | +0.283046 | +4201.04 | +
| 3072 | +756 | +256 | +0.8 | +0.212152 | +5604.88 | +
| 3072 | +756 | +256 | +0.85 | +0.158502 | +7502 | +
| 3072 | +756 | +256 | +0.9 | +0.107514 | +11059.8 | +
| 256 | +256 | +384 | +0.7 | +0.014833 | +3393.25 | +
| 256 | +256 | +384 | +0.75 | +0.012912 | +3898.17 | +
| 256 | +256 | +384 | +0.8 | +0.011054 | +4553.38 | +
| 256 | +256 | +384 | +0.85 | +0.009253 | +5439.63 | +
| 256 | +256 | +384 | +0.9 | +0.00717 | +7019.88 | +
| 256 | +1024 | +384 | +0.7 | +0.052661 | +3823.07 | +
| 256 | +1024 | +384 | +0.75 | +0.044728 | +4501.11 | +
| 256 | +1024 | +384 | +0.8 | +0.036583 | +5503.22 | +
| 256 | +1024 | +384 | +0.85 | +0.027487 | +7324.31 | +
| 256 | +1024 | +384 | +0.9 | +0.01839 | +10947.9 | +
| 1024 | +256 | +384 | +0.7 | +0.055665 | +3616.74 | +
| 1024 | +256 | +384 | +0.75 | +0.046588 | +4321.41 | +
| 1024 | +256 | +384 | +0.8 | +0.039411 | +5108.38 | +
| 1024 | +256 | +384 | +0.85 | +0.031875 | +6316.23 | +
| 1024 | +256 | +384 | +0.9 | +0.024706 | +8148.88 | +
| 1024 | +1024 | +384 | +0.7 | +0.263316 | +3058.33 | +
| 1024 | +1024 | +384 | +0.75 | +0.198303 | +4061 | +
| 1024 | +1024 | +384 | +0.8 | +0.141966 | +5672.54 | +
| 1024 | +1024 | +384 | +0.85 | +0.105849 | +7608.09 | +
| 1024 | +1024 | +384 | +0.9 | +0.068787 | +11707.3 | +
| 1024 | +4096 | +384 | +0.7 | +0.793822 | +4057.87 | +
| 1024 | +4096 | +384 | +0.75 | +0.996979 | +3230.99 | +
| 1024 | +4096 | +384 | +0.8 | +0.54288 | +5933.58 | +
| 1024 | +4096 | +384 | +0.85 | +0.412174 | +7815.21 | +
| 1024 | +4096 | +384 | +0.9 | +0.274441 | +11737.4 | +
| 4096 | +1024 | +384 | +0.7 | +1.03474 | +3113.09 | +
| 4096 | +1024 | +384 | +0.75 | +0.832701 | +3868.41 | +
| 4096 | +1024 | +384 | +0.8 | +0.564787 | +5703.43 | +
| 4096 | +1024 | +384 | +0.85 | +0.421781 | +7637.2 | +
| 4096 | +1024 | +384 | +0.9 | +0.271587 | +11860.8 | +
| 768 | +768 | +384 | +0.7 | +0.130834 | +3462.28 | +
| 768 | +768 | +384 | +0.75 | +0.094331 | +4802.11 | +
| 768 | +768 | +384 | +0.8 | +0.076012 | +5959.36 | +
| 768 | +768 | +384 | +0.85 | +0.058047 | +7803.72 | +
| 768 | +768 | +384 | +0.9 | +0.039534 | +11458 | +
| 768 | +3072 | +384 | +0.7 | +0.635729 | +2850.17 | +
| 768 | +3072 | +384 | +0.75 | +0.379899 | +4769.52 | +
| 768 | +3072 | +384 | +0.8 | +0.307908 | +5884.68 | +
| 768 | +3072 | +384 | +0.85 | +0.233057 | +7774.65 | +
| 768 | +3072 | +384 | +0.9 | +0.157518 | +11503.1 | +
| 3072 | +756 | +384 | +0.7 | +0.518276 | +3441.47 | +
| 3072 | +756 | +384 | +0.75 | +0.422517 | +4221.44 | +
| 3072 | +756 | +384 | +0.8 | +0.323476 | +5513.94 | +
| 3072 | +756 | +384 | +0.85 | +0.247251 | +7213.84 | +
| 3072 | +756 | +384 | +0.9 | +0.163369 | +10917.8 | +
| 256 | +256 | +512 | +0.7 | +0.019231 | +3489.57 | +
| 256 | +256 | +512 | +0.75 | +0.016815 | +3991.09 | +
| 256 | +256 | +512 | +0.8 | +0.014327 | +4683.96 | +
| 256 | +256 | +512 | +0.85 | +0.011932 | +5624.27 | +
| 256 | +256 | +512 | +0.9 | +0.00915 | +7334.5 | +
| 256 | +1024 | +512 | +0.7 | +0.072314 | +3712.06 | +
| 256 | +1024 | +512 | +0.75 | +0.06138 | +4373.31 | +
| 256 | +1024 | +512 | +0.8 | +0.050021 | +5366.46 | +
| 256 | +1024 | +512 | +0.85 | +0.037695 | +7121.28 | +
| 256 | +1024 | +512 | +0.9 | +0.02494 | +10763.4 | +
| 1024 | +256 | +512 | +0.7 | +0.079663 | +3369.65 | +
| 1024 | +256 | +512 | +0.75 | +0.06167 | +4352.8 | +
| 1024 | +256 | +512 | +0.8 | +0.052133 | +5149.09 | +
| 1024 | +256 | +512 | +0.85 | +0.042188 | +6362.85 | +
| 1024 | +256 | +512 | +0.9 | +0.032574 | +8240.75 | +
| 1024 | +1024 | +512 | +0.7 | +0.380582 | +2821.32 | +
| 1024 | +1024 | +512 | +0.75 | +0.24038 | +4466.86 | +
| 1024 | +1024 | +512 | +0.8 | +0.195271 | +5498.73 | +
| 1024 | +1024 | +512 | +0.85 | +0.146935 | +7307.58 | +
| 1024 | +1024 | +512 | +0.9 | +0.094832 | +11322.6 | +
| 1024 | +4096 | +512 | +0.7 | +1.08794 | +3947.8 | +
| 1024 | +4096 | +512 | +0.75 | +0.913626 | +4701.01 | +
| 1024 | +4096 | +512 | +0.8 | +0.737943 | +5820.19 | +
| 1024 | +4096 | +512 | +0.85 | +0.558473 | +7690.55 | +
| 1024 | +4096 | +512 | +0.9 | +0.369466 | +11624.8 | +
| 4096 | +1024 | +512 | +0.7 | +1.51009 | +2844.18 | +
| 4096 | +1024 | +512 | +0.75 | +1.26974 | +3382.55 | +
| 4096 | +1024 | +512 | +0.8 | +0.778244 | +5518.79 | +
| 4096 | +1024 | +512 | +0.85 | +0.589803 | +7282.03 | +
| 4096 | +1024 | +512 | +0.9 | +0.375284 | +11444.6 | +
| 768 | +768 | +512 | +0.7 | +0.160705 | +3758.31 | +
| 768 | +768 | +512 | +0.75 | +0.135495 | +4457.57 | +
| 768 | +768 | +512 | +0.8 | +0.108718 | +5555.46 | +
| 768 | +768 | +512 | +0.85 | +0.08112 | +7445.53 | +
| 768 | +768 | +512 | +0.9 | +0.053351 | +11320.8 | +
| 768 | +3072 | +512 | +0.7 | +0.614109 | +3934.03 | +
| 768 | +3072 | +512 | +0.75 | +0.518614 | +4658.42 | +
| 768 | +3072 | +512 | +0.8 | +0.418491 | +5772.93 | +
| 768 | +3072 | +512 | +0.85 | +0.31592 | +7647.25 | +
| 768 | +3072 | +512 | +0.9 | +0.211893 | +11401.6 | +
| 3072 | +756 | +512 | +0.7 | +0.836616 | +2842.61 | +
| 3072 | +756 | +512 | +0.75 | +0.699579 | +3399.43 | +
| 3072 | +756 | +512 | +0.8 | +0.544194 | +4370.08 | +
| 3072 | +756 | +512 | +0.85 | +0.379699 | +6263.3 | +
| 3072 | +756 | +512 | +0.9 | +0.228043 | +10428.6 | +
| Manufacturer | +Amazon EC2 | +
| Product Name | +c6i.12xlarge | +
| BIOS Version | +1 | +
| OS | +Ubuntu 20.04.3 LTS | +
| Kernel | +5.15.0-1021-aws | +
| Microcode | +0xd000331 | +
| IRQ Balance | +Disabled | +
| CPU Model | +Intel(R) Xeon Platinum 8375C CPU @ 2.90GHz | +
| Base Frequency | +2.9GHz | +
| Maximum Frequency | +3.9GHz | +
| All-core Maximum Frequency | +3.5GHz | +
| CPU(s) | +48 | +
| Thread(s) per Core | +N/A | +
| Core(s) per Socket | +24 | +
| Socket(s) | +1 | +
| NUMA Node(s) | +1 | +
| Turbo | +Enabled | +
| FrequencyGoverner | +Default | +
| Max C-State | +9 | +