How We Got Here - AI Timeline from 2015 to 2024
What led to the revolutionary LLMs and ChatGPT that changed everything? What were the roles of Google, OpenAI and other companies in the breakthrough?
As we marvel at the incredible things that LLMs can do today, one might start to wonder: How did we get here?
Did OpenAI just stumble upon a revolutionary idea and single-handedly changed the trajectory of AI advancement? Or was it a slow and prolonged process involving multiple companies, that eventually led to the ChatGPT moment?
In this post on The Ground Truth, let’s rewind to 2015, and see how major milestones AI and Reinforcement Learning (RL) led us to here.
2015: Foundations of Modern AI
Deep Q-network (DQN) - Google DeepMind's published the famous DQN paper on Nature. This groundbreaking work demonstrating how deep neural networks combined with Q-learning could master Atari games using only pixel inputs, marking a significant advancement in deep reinforcement learning.
AlphaGo - Google DeepMind created the first computer program to defeat a professional human Go player, combining Monte Carlo tree search (MCST) with deep neural networks trained by supervised and reinforcement learning.
OpenAI Founded - Sam Altman, Elon Musk, Ilya Sutskever, Greg Brockman and others established OpenAI as a research company with the goal of ensuring artificial general intelligence (AGI) “benefits all of humanity”.
2016: AlphaGo’s Historic Triumph
Historic Go Match Victory - DeepMind's AlphaGo defeated 18-time world champion Lee Sedol 4-1, a landmark achievement that came a decade earlier than many experts predicted, demonstrating AI's capability for creative strategic thinking.
2017: Transformers and Next-Gen Reinforcement Learning
"Attention Is All You Need" - Google researchers published the famous paper introducing the Transformer architecture, which relied entirely on attention mechanisms rather than recurrence or convolution, becoming the foundation for future language models.
AlphaGo Zero - DeepMind introduced AlphaGo Zero, a version of AlphaGo that mastered Go without human data, learning solely through self-play reinforcement learning. AlphaGo Zero achieved superhuman performance, winning 100–0 against the previously published, champion-defeating AlphaGo.
Proximal Policy Optimization (PPO) - OpenAI introduced PPO, a reinforcement learning algorithm designed to be more stable, easier to implement, and more sample-efficient than previous policy gradient methods. PPO has since become the default RL algorithm at OpenAI and other companies.
2018: The Dawn of Large Language Models
BERT - Google AI released this bidirectional language representation model that revolutionized Natural Language Processing (NLP) tasks such as classification and question answering. BERT enabled better understanding of context through considering both left and right context simultaneously. BERT is widely considered the precursor to LLMs.
GPT-1 - OpenAI released the first Generative Pre-trained Transformer, demonstrating the effectiveness of unsupervised pre-training followed by supervised fine-tuning.
2019: Scaling Up Language Models
GPT-2 - OpenAI released GPT-2, a 1.5 billion parameter model with significantly improved text generation capabilities, initially delaying full release due to potential misuse concerns.
XLNet - Google AI and CMU researchers introduced XLNet, a generalized autoregressive pre-training method that overcame limitations of BERT through permutation language modelling.
T5 - Google researchers introduced the Text-to-Text Transfer Transformer, reframing all NLP tasks into a unified text-to-text format.
2020: GPT-3 and Human Feedback Alignment
GPT-3 - OpenAI released GPT-3. This 175 billion parameter language model demonstrating remarkable few-shot learning capabilities across diverse tasks, setting a new standard for language models.
RLHF Research - OpenAI published Learning to Summarize from Human Feedback, demonstrating how reinforcement learning could align language models with human preferences.
2021: Multimodal and Domain-Specific AI
DALL-E - OpenAI introduced DALL-E, a multimodal AI system capable of generating images from text descriptions, demonstrating language models' potential to understand and generate visual content.
Codex - OpenAI released this GPT model fine-tuned on code, powering GitHub Copilot and marking a significant step in AI-assisted programming.
2022: AI's Public Breakthrough - ChatGPT
Gato - DeepMind introduced this "generalist agent" capable of performing hundreds of different tasks across different modalities, demonstrating the potential for single models to handle diverse tasks.
ChatGPT - OpenAI released ChatGPT based on GPT-3.5 and trained with RLHF, becoming the fastest-growing consumer application in history and bringing AI into mainstream consciousness.
Stable Diffusion - Stability AI released an open-source text-to-image model called Stable Diffusion, allowing for wider experimentation and accelerating innovation in generative AI.
2023: The Multimodal and Open-Source Growth
Claude - Anthropic introduced Claude, an AI assistant trained using Constitutional AI, a method developed to create helpful, harmless, and honest AI systems.
GPT-4 - OpenAI released GPT-4, a multimodal large language model capable of accepting image and text inputs, approaching human-level performance on various professional and academic benchmarks.
Llama - Meta AI released Llama, foundation language models ranging from 7B to 65B parameters, spurring innovation in the open-source AI community.
Gemini - Google introduced Gemini, a multimodal AI model designed to work across text, images, audio, video, and code, released in three sizes (Ultra, Pro, and Nano).
2024: Better, Faster and More Capable Models
Claude 3 Family - Anthropic released the Claude 3 model family (Haiku, Sonnet, and Opus), with Claude 3 Opus demonstrating performance competitive with or exceeding GPT-4 on many benchmarks.
GPT-4o - OpenAI released GPT-4o, an "omni" multimodal model capable of processing text, audio, and vision inputs in real-time with reduced latency and more natural voice interactions.
Llama 3 - Meta released Llama 3, open-source large language model in 8B and 70B parameter versions, demonstrating significant improvements and competitive performance with proprietary models.
Claude 3.5 Sonnet - Anthropic released Claude 3.5 Sonnet, an upgraded model featuring improved reasoning, reduced hallucinations, and enhanced capabilities across various tasks including coding and mathematics.
2025 - What’s Next?
As you can see, we have come a long way from 2015. Many important innovations across companies like Google and OpenAI led us to the explosive growth of AI in the past few years.
I am incredibly excited about what’s coming next in 2025.
The history is unfolding in front of us. We are all witnesses.