JUNE 24, 2024

<aside> 🔖 In this blog post, we will deep dive into AlexNet and discuss the key concepts, architectures, and contributions of this seminal work in the field of deep learning for computer vision.

</aside>

Paper Information

• Title: ImageNet Classification with Deep Convolutional Neural Networks
• Authors: Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton
• Published in: Advances in Neural Information Processing Systems 25 (NIPS 2012)
• Link to Paper: [link]

Abstract

The paper presents AlexNet, a deep convolutional neural network designed to classify images in the ImageNet LSVRC-2010 contest. The model significantly outperformed the state-of-the-art at the time, reducing the top-5 error rate from 26.2% to 15.3%. Key innovations include the use of ReLU activation functions, dropout for regularization, and GPU implementation for training acceleration.

Introduction

AlexNet is a groundbreaking architecture in deep learning and computer vision, introducing many standard practices still in use today. Proof of its impact can be seen in the number of citations for this paper, totaling an impressive 156,882 citations to date. AlexNet is a deep learning architecture designed for large-scale image classification. Prior to this work, shallow models could not achieve comparable performance on complex datasets like ImageNet, which includes millions of labeled images across thousands of categories.

Key Contributions

• Depth and Complexity: AlexNet demonstrated that deeper networks with many layers could achieve significantly better performance on complex image classification tasks
• ReLU Activation: Substitute tanh activation function with Rectified Linear Unit, showing lower training time.
• Dropout: Utilized dropout as a regularization technique to prevent overfitting.
• GPU Implementation: Leveraged the power of GPUs to accelerate the training process, making it feasible to train large networks on large datasets.