In short
A Large Language Model (LLM) is a neural network based on transformer architectures, pre-trained on massive corpora using self-supervised learning, and fine-tuned for generation, reasoning, and tool use. Since the introduction of the Transformer in 2017, LLM development has progressed through five distinct architectural phases: foundational pretraining (2018–2020), dense scaling (2021), compute-optimal alignment (2022), the open-weight revolution (2023), and native test-time reasoning with sparse mixture-of-experts (2024 onward).
#Executive summary and architectural eras
- Foundational pre-training (2018–2020): Shift from recurrent architectures (LSTMs) to self-attention. Emergence of masked language modeling (BERT) and autoregressive sequence prediction (GPT-1, GPT-2, GPT-3).
- Dense Scaling & Compute Frontiers (2021): Massive scale-ups (Megatron-Turing 530B, Gopher 280B) and initial explorations of sparse Mixture of Experts (GLaM).
- Compute-Optimal Scaling & RLHF (2022): The Chinchilla paper recalibrates token-to-parameter ratios (70B parameters, about 4x the training data of the 280B Gopher at similar compute); Reinforcement Learning from Human Feedback (RLHF) makes ChatGPT a consumer breakout.
- The Open-Weight Ecosystem (2023): Meta's Llama series, Mistral's sliding-window architectures, and low-rank adaptation democratize fine-tuning on commodity silicon.
- Native Reasoning, MLA & Test-Time Compute (2024–2026): Transition to multi-million token context windows, inference-time chain-of-thought search (OpenAI o1), and high-efficiency Multi-head Latent Attention (DeepSeek-V3/R1).
Scope: the tables stop at January 2025. Later releases (for example the Claude 5.x, GPT-6 and Gemini 4 families) are covered in our 2026 model guide. On 2026-11-03 we opened the arXiv abstract page, developer blog or Hugging Face model card linked in Sources for every row with a paper or vendor link, and checked the date, developer, parameter count and, where the page states them, context length and licence. Where a page gave a different or narrower figure, the row was corrected or softened (for example HyperCLOVA, GPT-NeoX, Cerebras-GPT and DeepSeek-LLM now give the paper date, because we did not open a page stating an earlier release date). Where a developer page does not state a figure, the cell says so. Four limits apply. (1) OpenAI's own pages return an access error to our fetch tool. For GPT-1 (11 Jun 2018), GPT-2 (14 Feb 2019, with the 1.5B size and the staged release) and ChatGPT (30 Nov 2022, RLHF) we read Wayback Machine captures of the OpenAI pages (captured 8 May 2024, 8 May 2024 and 2 May 2024), which show the publication date and text. For GPT-4o (13 May 2024) and o1 (12 Sep 2024) the Wayback copies could not be read; the Wayback index lists captures dated 13 May 2024 and 12 Sep 2024, which shows the pages existed by those days, and the day-level dates otherwise rest on excerpts seen in search results. (2) The GPT-1 parameter count comes from the Hugging Face model card (0.1B); the often-quoted 117M is not stated on any page we opened. (3) The Access Type column and one-line impact descriptions were checked only where a linked page states them; for Claude 2.0 (Jul 2023), Megatron-Turing NLG's October 2021 announcement, Ernie, GLaM and similar entries only the paper date was confirmed. (4) Jurassic, Yi and Nemotron are not in these tables. Exact day-level dates are given only where a page we opened states them. "Undisclosed" means the developer has not published the figure.
#Phase 1: Foundational Transformers & Architecture Emergence (2018–2020)
| Model | Release Date | Organization | Parameters | Key Architectural Feature / Impact | Access Type |
|---|---|---|---|---|---|
| GPT-1 | Jun 2018 | OpenAI | About 0.1B (Hugging Face card lists 0.1B; the commonly cited 117M is not stated there) | Established generative pre-training + discriminative fine-tuning | Open weights |
| BERT | Oct 2018 | BERT-Large about 336M (Hugging Face card; often rounded to 340M) | Bidirectional masked language modeling (MLM) and Next Sentence Prediction | Open source | |
| GPT-2 | Feb 2019 | OpenAI | 1.5B | Demonstrated zero-shot task transfer via parameter scaling; 1.5B model first withheld, released in stages | Open weights (later release) |
| XLNet | Jun 2019 | Google / CMU | Similar to BERT-Large (~340M) | Permutation language modeling overcoming BERT pretrain-finetune mismatch | Open source |
| RoBERTa | Jul 2019 | Meta / UW | 355M | Hyperparameter and dataset optimization over BERT baseline | Open source |
| T5 | Oct 2019 | Up to 11B | Unified Text-to-Text Transfer Transformer framework | Open source | |
| GPT-3 | May 2020 | OpenAI | 175B | Established emergent in-context few-shot learning at scale | Proprietary API |
#Phase 2: Dense Scaling, Conversational Seeds & Compute Boundaries (2021)
| Model | Release Date | Organization | Parameters | Key Architectural Feature / Impact | Access Type |
|---|---|---|---|---|---|
| GPT-Neo | Mar 2021 | EleutherAI | 2.7B | Community-driven, open-source GPT-3 replication | Open weights |
| HyperCLOVA | 2021 (paper Sep 2021) | Naver | 82B | Non-English foundational model optimized for the Korean language | Commercial API |
| GPT-J | mid-2021 | EleutherAI | 6B | Dense open alternative implementing Rotary Position Embeddings (RoPE) | Open weights (Apache 2.0) |
| Megatron-Turing NLG | Paper Jan 2022 | Microsoft / NVIDIA | 530B | Giant transformer training utilizing DeepSpeed & Megatron pipeline parallelism | Research |
| Ernie 3.0 Titan | Dec 2021 | Baidu | Up to 260B | Knowledge-enhanced multi-task pre-training model | API / Research |
| GLaM | Dec 2021 | 1.2T (MoE) | Sparsely activated Mixture-of-Experts; reduced carbon and training compute | Research | |
| Gopher | Dec 2021 | DeepMind | 280B | Extensive analysis of capabilities and failure modes across model sizes | Research |
#Phase 3: Compute-Optimal Scaling Laws & Dialogue Alignment (2022)
| Model | Release Date | Organization | Parameters | Key Architectural Feature / Impact | Access Type |
|---|---|---|---|---|---|
| LaMDA | Jan 2022 (paper) | Up to 137B | Transformer specialized for human-like multi-turn dialogue | Not stated in the paper abstract | |
| GPT-NeoX | Paper Apr 2022 | EleutherAI | 20B | Open autoregressive model; the paper describes a 20-billion-parameter model | Open weights (Apache 2.0 per its Hugging Face card) |
| Chinchilla | Mar 2022 | DeepMind | 70B | Established data-to-parameter scaling laws (4x more tokens vs. parameter size) | Research |
| PaLM | Apr 2022 | 540B | Pathways architecture scaled across 6,144 TPU v4 chips; strong results on multi-step reasoning tasks | Research | |
| OPT | May 2022 | Meta AI | 125M–175B | Suite of decoder-only models; OPT-175B comparable to GPT-3 with about one-seventh the carbon footprint (per the paper) | Shared with researchers |
| YaLM 100B | 2022 | Yandex | 100B | Pre-trained bilingual (Russian/English) foundational model | Open weights |
| Minerva | Jun 2022 | Not stated in abstract | Specialized for solving complex STEM, calculus, and scientific challenges | Research | |
| BLOOM | Jul 2022 (Hugging Face card estimate: 11 Jul 2022; paper Nov 2022) | BigScience / HF | 176B | Collaborative open-science multilingual model: 46 natural and 13 programming languages | Open weights (BigScience RAIL licence 1.0) |
| Galactica | Nov 2022 | Meta AI | 120B | Scientific information processor trained on academic literature and papers | Research |
| ChatGPT (GPT-3.5) | Nov 2022 | OpenAI | Undisclosed | Mainstream consumer breakout via Reinforcement Learning from Human Feedback | Web / API |
#Phase 4: The Open-Weight Revolution & Instruction Tuning (2023)
| Model | Release Date | Organization | Parameters | Key Architectural Feature / Impact | Access Type |
|---|---|---|---|---|---|
| Llama | Feb 2023 | Meta AI | 7B–65B | High performance on modest parameter budgets; kickstarted open research | Research weights |
| Alpaca | Mar 2023 | Stanford | 7B | Fine-tuned from LLaMA 7B on 52K instruction demonstrations; research use only, weights not released at launch | Open research (code and data) |
| GPT-4 | Mar 2023 | OpenAI | Undisclosed | Multimodal model; technical report withholds architecture and size | Proprietary API |
| Cerebras-GPT | Apr 2023 (paper) | Cerebras | 111M–13B | Models trained following Chinchilla scaling rules on a Cerebras Wafer-Scale Cluster | Not stated in the paper abstract |
| Falcon | 2023 (paper Nov 2023) | TII (UAE) | 7B / 40B / 180B | Falcon-180B trained on over 3.5T tokens; Hugging Face card lists a 2048-token sequence length | Open weights (Falcon-180B under the TII licence, gated) |
| BloombergGPT | Mar 2023 (paper) | Bloomberg L.P. | 50B | Domain-specific LLM; 363B-token dataset built from Bloomberg's data sources plus 345B general-purpose tokens | Not stated in the paper abstract |
| Vicuna | Mar 2023 | LMSYS | 13B | Chat model tuned on about 70K user-shared ShareGPT conversations; non-commercial use | Open weights (non-commercial) |
| PaLM 2 | May 2023 | Not disclosed in report | Heavily multilingual, optimized for reasoning, coding, and translation | API | |
| Phi-1 / Phi-2 | Jun–Dec 2023 | Microsoft | 1.3B / 2.7B | Proven efficiency using curated synthetic "Textbooks Are All You Need" | Open weights |
| Llama 2 | Jul 2023 | Meta AI | 7B, 13B, 70B | Open-weight release with a licence that permits commercial use under Meta's terms | Open weights |
| Claude 2 / 2.1 | Jul–Nov 2023 | Anthropic | Undisclosed | Claude 2.1 (21 Nov 2023) offered a 200K-token window and, per Anthropic, about half the false statements of Claude 2.0 | Proprietary API |
| Mistral 7B | Sep 2023 | Mistral AI | 7.3B | Sliding Window Attention (SWA) and Grouped-Query Attention (GQA); announced 27 Sep 2023 | Apache 2.0 |
| DeepSeek-LLM | Paper Jan 2024 | DeepSeek | 7B / 67B | Base and chat models; the paper reports a 2-trillion-token pre-training dataset | Open weights |
| Grok-1 | Mar 2024 (open release) | xAI | 314B (MoE) | Mixture-of-experts with about 25% of weights active per token; custom JAX and Rust training stack; Apache 2.0 | Open weights |
| Gemini 1.0 | Dec 2023 | Google DeepMind | Nano, Pro, Ultra | Built natively multimodal across text, audio, image, and video modalities | Proprietary API |
| Mixtral 8x7B | Dec 2023 (paper Jan 2024) | Mistral AI | About 47B total (13B active) | Sparse MoE, 2 of 8 experts per token; 32k-token context (Mistral announcement, 11 Dec 2023); per its paper matches or outperforms Llama 2 70B and GPT-3.5 on the benchmarks evaluated | Apache 2.0 |
#Phase 5: Massive Context, SLMs & Test-Time Reasoning (2024–2026)
| Model | Release Date | Organization | Parameters | Key Architectural Feature / Impact | Access Type |
|---|---|---|---|---|---|
| OLMo | Feb 2024 (paper 1 Feb 2024) | AI2 | 1B and 7B (Hugging Face cards) | Released with open training data and training and evaluation code | Fully open (Apache 2.0 per the 7B card) |
| Gemma / Gemma 2 | 21 Feb 2024 / mid-2024 (paper 31 Jul 2024) | Google DeepMind | 2B and 7B (Gemma); 2B, 9B and 27B (Gemma 2) | Lightweight open models based on Gemini research and technology | Open weights |
| Gemini 1.5 | Feb 2024 (paper Mar 2024) | Google DeepMind | Flash / Pro | Long-context multimodal model; announced 15 Feb 2024 with a 1M-token window in limited preview (128K standard); the paper reports near-perfect retrieval in tests up to at least 10M tokens | Proprietary API |
| Claude 3 / 3.5 | Mar–Jun 2024 | Anthropic | Haiku, Sonnet, Opus (sizes undisclosed) | Claude 3 family of three tiers announced 4 Mar 2024; Claude 3.5 Sonnet published 21 Jun 2024 | Proprietary API |
| DBRX | Mar 2024 | Databricks | 132B total (36B active) | Fine-grained MoE: 16 experts, 4 chosen per input; 27 Mar 2024; up to 32K-token context | Open weights (licence not named on the announcement page) |
| Command R+ | Apr 2024 | Cohere | Not stated in announcement | Launched first on Microsoft Azure; aimed at enterprise retrieval and tool use | API |
| Llama 3 & 3.1 | 8B and 70B on 18 Apr 2024; paper 31 Jul 2024 | Meta AI | 8B, 70B, 405B | Llama 3 was trained on 8,192-token sequences; the Llama 3.1 paper describes a dense 405B Transformer with a context window up to 128K tokens | Open weights (Meta's licence terms) |
| Phi-3 / Phi-4 | Apr–Dec 2024 | Microsoft | 3.8B–14B | Small language models (phi-3-mini 3.8B, paper 22 Apr 2024; Phi-4 14B, 12 Dec 2024, 16K context) | Open weights (Phi-4 under MIT per its Hugging Face card) |
| GPT-4o / mini | May–Jul 2024 | OpenAI | Undisclosed | End-to-end omnimodal engine handling unified text, vision, and natural audio | Proprietary API |
| Qwen 2.5 | Sep 2024 (report Dec 2024) | Alibaba Cloud | 0.5B to 72B (open models) | Released 19 Sep 2024; pretraining data expanded from 7T to 18T tokens; up to 128K-token input | Open weights (Apache 2.0 except the 3B and 72B variants, per the Qwen blog) |
| OpenAI o1 | Sep 2024 | OpenAI | Undisclosed | Reasoning model that spends more time thinking before it responds | Proprietary API |
| DeepSeek-V3 / R1 | Dec 2024–Jan 2025 | DeepSeek | 671B total (37B active) | V3 (paper 27 Dec 2024; 14.8T tokens; 128K context) uses Multi-head Latent Attention (MLA) and DeepSeekMoE; R1 (paper 22 Jan 2025) trained with reinforcement learning for reasoning | Open weights (R1 under MIT; V3 code under MIT, V3 weights under DeepSeek's model licence, which permits commercial use) |
| Gemini 2.0 Flash | Dec 2024 | Google DeepMind | Undisclosed | Fast Gemini 2.0 model with multimodal input and tool use | Proprietary API |
#Architectural evolution: what changed over seven years
#1. From Dense Scaling to Sparse Mixture-of-Experts (MoE)
Early foundation models (GPT-3, PaLM, OPT) are dense: every parameter is used for every token. Contemporary open models (Mixtral, DBRX, DeepSeek-V3) route each token through a router so only a subset of parameters is active (for example 37B of 671B in DeepSeek-V3), which lowers compute per token compared with a dense model of the same total size.
#2. Attention Mechanisms: From Quadratic MHA to FlashAttention and MLA
Standard Multi-Head Attention (MHA) compute scales quadratically with sequence length , which limited early context windows (GPT-3 used 2,048 tokens). Two advances reduced the cost:
- FlashAttention (Dao et al., 2022): An exact attention algorithm that reduces reads and writes between GPU high-bandwidth memory and on-chip SRAM (arXiv:2205.14135).
- Multi-Head Latent Attention (MLA, DeepSeek-V2, 2024): Compresses the key-value (KV) cache into a low-rank latent representation to cut inference memory; DeepSeek-V3 adopts it (arXiv:2412.19437).
#3. From Pretraining Scaling to Test-Time Compute Scaling
Historically, model capability was tied to compute spent during pretraining (Kaplan et al., 2020; Hoffmann et al., 2022). With OpenAI o1 and DeepSeek-R1, labs also scaled compute at inference: the models spend extra time generating reasoning before they answer. DeepSeek-R1's paper describes eliciting this reasoning with reinforcement learning.
#Frequently asked questions
#What was the first large language model?
There is no single agreed answer, because "large" is relative and neural language models predate transformers. GPT-1 (OpenAI, June 2018, about 0.1B parameters per its Hugging Face card) and BERT (Google, October 2018, BERT-Large at about 336M parameters per its Hugging Face card) are the usual starting points for the transformer era. This list begins there for that reason.
#What is the difference between open-weight and open-source models?
Open-weight releases publish the trained parameters so you can download and run the model, but often not the training data or full training pipeline. Fully open projects such as AI2's OLMo release open training data and training and evaluation code alongside the model. Licences differ, so read the terms before commercial use.
#Why did parameter counts stop growing so fast after 2022?
The Chinchilla paper (Hoffmann et al., 2022) showed that a 70B model trained on about four times more data than the 280B Gopher could do better at similar compute, so labs shifted toward more training tokens per parameter. Sparse mixture-of-experts designs also let total size grow while active parameters stay lower.
#What is a mixture-of-experts model?
A mixture-of-experts (MoE) model contains many expert sub-networks and a router that picks a few for each token. Mixtral activates 2 of 8 experts per token (about 13B of 47B parameters), and DeepSeek-V3 activates 37B of 671B. Compute per token is lower than for a dense model of the same total size.
#Which models count as reasoning models?
OpenAI's o1 (September 2024) and DeepSeek-R1 (January 2025) are the two covered here. Both spend additional computation generating reasoning before answering. DeepSeek-R1's paper, published in Nature in 2025, describes training this behaviour with reinforcement learning, and its weights are openly available.
#Why are some parameter counts marked undisclosed?
Several developers, including OpenAI for GPT-4 (whose report says it contains no details on architecture, including model size) and Google for PaLM 2, do not state model sizes in their technical reports. Figures circulating elsewhere are estimates or leaks, so this list shows "Undisclosed" rather than repeat them.
#Sources
- Vaswani, A., et al. (2017). Attention Is All You Need. arXiv:1706.03762 (NeurIPS 2017).
- Radford, A., et al. (2018). Improving Language Understanding by Generative Pre-Training (GPT-1). OpenAI (blog post, 11 Jun 2018; date read from a Wayback Machine capture of 8 May 2024, because openai.com blocks our fetch tool); openai-community/openai-gpt model card (0.1B parameters, MIT licence; the commonly quoted 117M is not stated there).
- Radford, A., et al. (2019). Better language models and their implications (GPT-2, 14 Feb 2019; "1.5 billion parameters" and the staged release; read from a Wayback Machine capture of 8 May 2024); GPT-2 XL model card ("1.5B parameter version").
- Yang, Z., et al. (2019). XLNet. arXiv:1906.08237 (19 Jun 2019; XLNet-Large has the same hyperparameters as BERT-Large, "a similar model size").
- Liu, Y., et al. (2019). RoBERTa. arXiv:1907.11692 (26 Jul 2019; "L=24, H=1024, A=16, 355M parameters").
- Raffel, C., et al. (2019). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (T5). arXiv:1910.10683 (23 Oct 2019; models up to 11 billion parameters).
- EleutherAI. (2021). GPT-Neo 2.7B (MIT licence; citation dated March 2021) and GPT-J-6B (6.05B parameters, Apache 2.0, rotary position embeddings; citation dated May 2021, so we say mid-2021).
- Yandex. (2022). YaLM 100B (100B parameters, Apache 2.0, English and Russian; the page gives no release date).
- Devlin, J., et al. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805.
- Brown, T., et al. (2020). Language Models are Few-Shot Learners (GPT-3). arXiv:2005.14165.
- Kaplan, J., et al. (2020). Scaling Laws for Neural Language Models. arXiv:2001.08361.
- Du, N., et al. (2021). GLaM: Efficient Scaling of Language Models with Mixture-of-Experts. arXiv:2112.06905.
- Rae, J. W., et al. (2021). Scaling Language Models: Methods, Analysis & Insights from Training Gopher. arXiv:2112.11446.
- Smith, S., et al. (2022). Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B. arXiv:2201.11990.
- Kim, B., et al. (2021). What Changes Can Large-scale Language Models Bring? (HyperCLOVA). arXiv:2109.04650.
- Wang, S., et al. (2021). ERNIE 3.0 Titan. arXiv:2112.12731.
- Thoppilan, R., et al. (2022). LaMDA: Language Models for Dialog Applications. arXiv:2201.08239.
- Hoffmann, J., et al. (2022). Training Compute-Optimal Large Language Models (Chinchilla). arXiv:2203.15556.
- Chowdhery, A., et al. (2022). PaLM: Scaling Language Modeling with Pathways. arXiv:2204.02311.
- Black, S., et al. (2022). GPT-NeoX-20B. arXiv:2204.06745.
- Zhang, S., et al. (2022). OPT: Open Pre-trained Transformer Language Models. arXiv:2205.01068.
- Lewkowycz, A., et al. (2022). Solving Quantitative Reasoning Problems with Language Models (Minerva). arXiv:2206.14858.
- Dao, T., et al. (2022). FlashAttention. arXiv:2205.14135.
- Taylor, R., et al. (2022). Galactica: A Large Language Model for Science. arXiv:2211.09085 (16 Nov 2022; sizes 125M to 120B).
- OpenAI. (2022). Introducing ChatGPT (30 Nov 2022; "trained this model using Reinforcement Learning from Human Feedback"; read from a Wayback Machine capture of 2 May 2024).
- BigScience Workshop. (2022). BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. arXiv:2211.05100.
- Stanford CRFM. (2023). Alpaca: A strong, replicable instruction-following model (13 Mar 2023; LLaMA 7B base; weights not released at launch).
- LMSYS. (2023). Vicuna: An open-source chatbot (30 Mar 2023; 13B; about 70K ShareGPT conversations).
- Touvron, H., et al. (2023). Llama: Open and Efficient Foundation Language Models. arXiv:2302.13971.
- OpenAI. (2023). GPT-4 Technical Report. arXiv:2303.08774.
- Wu, S., et al. (2023). BloombergGPT: A Large Language Model for Finance. arXiv:2303.17564.
- Dey, N., et al. (2023). Cerebras-GPT. arXiv:2304.03208.
- Anil, R., et al. (2023). PaLM 2 Technical Report. arXiv:2305.10403.
- Gunasekar, S., et al. (2023). Textbooks Are All You Need (phi-1). arXiv:2306.11644; Microsoft Research. (2023). Phi-2: The surprising power of small language models.
- Touvron, H., et al. (2023). Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv:2307.09288.
- Jiang, A. Q., et al. (2023). Mistral 7B. arXiv:2310.06825.
- Anthropic. (2023). Introducing Claude 2.1 (21 Nov 2023).
- BigScience. bigscience/bloom model card (RAIL licence 1.0; 176B; release date estimate 11 Jul 2022); TII. tiiuae/falcon-180B (TII licence, 2048-token sequence length); Google. google-bert/bert-large-uncased (about 336M, Apache 2.0); EleutherAI. gpt-neox-20b (Apache 2.0); AI2. OLMo-7B (Apache 2.0).
- Meta. (2024). Introducing Meta Llama 3 (18 Apr 2024; 8B and 70B; trained on 8,192-token sequences); Meta. (2023). Llama 2 (18 Jul 2023; free for research and commercial use).
- Mistral AI. (2023). Announcing Mistral 7B (27 Sep 2023; 7.3B; Apache 2.0) and Mixtral of experts (11 Dec 2023; 32k context; Apache 2.0).
- Qwen Team. (2024). Qwen2.5 blog (19 Sep 2024; Apache 2.0 except the 3B and 72B variants); Qwen/Qwen2.5-72B (72.7B parameters, 131,072-token input context, "qwen" licence); DeepSeek. DeepSeek-V3 (671B total, 37B active, 128K context; code MIT, model licence separate) and DeepSeek-R1 (MIT); Microsoft. phi-4 (14B, MIT, 16K context).
- Google. (2024). Our next-generation model: Gemini 1.5 (15 Feb 2024).
- Almazrouei, E., et al. (2023). The Falcon Series of Open Language Models. arXiv:2311.16867.
- Google. (2023). Introducing Gemini (6 Dec 2023; Ultra, Pro, Nano; built to be multimodal); Gemini Team. (2023). arXiv:2312.11805.
- DeepSeek-AI. (2024). DeepSeek LLM: Scaling Open-Source Language Models with Longtermism. arXiv:2401.02954.
- Jiang, A. Q., et al. (2024). Mixtral of Experts. arXiv:2401.04088.
- Groeneveld, D., et al. (2024). OLMo: Accelerating the Science of Language Models. arXiv:2402.00838.
- Gemini Team. (2024). Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv:2403.05530.
- xAI. (2024). Open Release of Grok-1 (17 March 2024).
- Databricks. (2024). Introducing DBRX (27 March 2024).
- Cohere. (2024). Command R+ on Microsoft Azure (4 April 2024).
- Abdin, M., et al. (2024). Phi-3 Technical Report. arXiv:2404.14219.
- Google. (2024). Gemma: Introducing new state-of-the-art open models (21 Feb 2024; 2B and 7B); Gemma Team. (2024). arXiv:2403.08295.
- Anthropic. (2024). Introducing the Claude 3 family (4 Mar 2024; no sizes given) and Claude 3.5 Sonnet (21 Jun 2024).
- OpenAI. (2024). Hello GPT-4o (13 May 2024; the page and its Wayback copy could not be read, the Wayback index lists a capture dated 13 May 2024, and the day rests on search-result excerpts); GPT-4o System Card (arXiv 25 Oct 2024; does not state the release date). OpenAI. (2024). Learning to reason with LLMs (12 Sep 2024; same situation, Wayback index lists a capture dated 12 Sep 2024); OpenAI o1 System Card (arXiv 21 Dec 2024; does not state the release date).
- Google. (2024). Introducing Gemini 2.0 (11 Dec 2024; no size given).
- Gemma Team. (2024). Gemma 2. arXiv:2408.00118.
- Llama Team. (2024). The Llama 3 Herd of Models. arXiv:2407.21783.
- Qwen Team. (2024). Qwen2.5 Technical Report. arXiv:2412.15115.
- Abdin, M., et al. (2024). Phi-4 Technical Report. arXiv:2412.08905.
- DeepSeek-AI. (2024). DeepSeek-V3 Technical Report. arXiv:2412.19437.
- DeepSeek-AI. (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948 (Nature, vol. 645, 2025).
