Anshuman Awasthi

Introduced the Transformer architecture, which is now foundational in NLP models. Key idea is multi-head self-attention allowing models to weigh input relevance dynamically. Revolutionary for parallel processing and long-range dependencies.

Bidirectional encoder representations from transformers. Showed the power of pre-training with masked language modeling and next sentence prediction. Changed the landscape of NLP with transfer learning.

Combines pre-trained parametric and non-parametric memory for language generation. Essential for my current RAG research - implementing variants for domain-specific question answering.

GPT-3 paper demonstrating emergent abilities of large language models. Showed that scaling up models leads to few-shot learning capabilities without task-specific fine-tuning.

Introduced residual connections to solve the vanishing gradient problem in deep networks. ResNet architecture became fundamental for computer vision tasks and influenced many subsequent architectures.

Shows how step-by-step reasoning can be elicited from LLMs through prompting. Currently exploring applications for my research in reasoning-based question answering systems.

Meta's approach to efficient large language models. Demonstrated that smaller, well-trained models can outperform larger ones. Important for resource-constrained applications.

Dense retrieval approach using BERT-based bi-encoders. Key component for my RAG implementations - experimenting with different dense retrieval strategies for domain adaptation.