BERT-Large: Prune Once for DistilBERT Inference Performance
By A Mystery Man Writer
Description
Compress BERT-Large with pruning & quantization to create a version that maintains accuracy while beating baseline DistilBERT performance & compression metrics.
![BERT-Large: Prune Once for DistilBERT Inference Performance](https://neurips.cc/media/PosterPDFs/NeurIPS%202023/71044.png?t=1701386253.186801)
NeurIPS 2023
Dipankar Das posted on LinkedIn
![BERT-Large: Prune Once for DistilBERT Inference Performance](https://miro.medium.com/v2/resize:fit:1400/1*9kE4oa0M3io2hd58LiwYgw.png)
Distillation of BERT-Like Models: The Theory
![BERT-Large: Prune Once for DistilBERT Inference Performance](https://developer-blogs.nvidia.com/wp-content/uploads/2020/08/SigOpt-Part-1-Featured.png)
Efficient BERT: Finding Your Optimal Model with Multimetric Bayesian Optimization, Part 1
Excluding Nodes Bug In · Issue #966 · Xilinx/Vitis-AI ·, 57% OFF
![BERT-Large: Prune Once for DistilBERT Inference Performance](https://miro.medium.com/v2/resize:fit:1400/1*WJXZZhJhj6Ff7IRLwFOEaA.png)
Distillation of BERT-Like Models: The Theory
![BERT-Large: Prune Once for DistilBERT Inference Performance](https://miro.medium.com/v2/resize:fit:1400/1*MRpbO6N-V3Z-4xtPxLyi6g.png)
Large Language Models: DistilBERT — Smaller, Faster, Cheaper and Lighter, by Vyacheslav Efimov
![BERT-Large: Prune Once for DistilBERT Inference Performance](https://miro.medium.com/v2/resize:fit:5280/1*rkuJ54qmMVYWuWiIWoGUqA.jpeg)
Poor Man's BERT - Exploring layer pruning
![BERT-Large: Prune Once for DistilBERT Inference Performance](https://d3i71xaburhd42.cloudfront.net/9202a718ce05395b6e17d5301e3a2e8b1021f31b/4-Figure1-1.png)
PDF] Prune Once for All: Sparse Pre-Trained Language Models
from
per adult (price varies by group size)