APERTUS V1 TECHNICAL REPORT

Democratizing Open and Compliant LLMs for Global Language Environments

Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Cite as: arXiv:2509.14233v2

https://doi.org/10.48550/arXiv.2509.14233

Core Team

Alejandro Hernandez-Cano, Alexander Hagele, Allen Hao Huang, Angelika Romanou, Antoni-Joan Solergibert, Barna Pasztor, Bettina Messmer, Dhia Garbaya, Eduard Frank Durech, Ido Hakimi, Juan García Giraldo, Mete Ismayilzada, Negar Foroutan, Skander Moalla, Tiancheng Chen, Vinko Sabolcec, Yixuan Xu

Advisors

Alexander Ilic, Ana Klimovic, Andreas Krause, Caglar Gulcehre, David Rosenthal, Elliott Ash, Florian Tramer, Joost VandeVondele, Livio Veraldi, Martin Rajman, Thomas Schulthess, Torsten Hoefler

Abstract

We present Apertus, a fully open suite of large language models (LLMs) designed to address two systemic shortcomings in today’s open model ecosystem: data compliance and multilingual representation.

Unlike many prior models that release weights without reproducible data pipelines or regard for content-owner rights, Apertus models are pretrained exclusively on openly available data, retroactively respecting robots.txt exclusions and filtering for non-permissive, toxic, and personally identifiable content.

To mitigate risks of memorization, we adopt the Goldfish objective during pretraining, strongly suppressing verbatim recall of data while retaining downstream task performance.

The Apertus models expand multilingual coverage, training on 15T tokens from over 1,800 languages, with approximately 40% of pretraining data allocated to non-English content.

Released at 8B and 70B scales, Apertus approaches state-of-the-art results among fully open models on multilingual benchmarks.

1. Introduction

The open ecosystem for large language models (LLMs) has flourished rapidly. However, many models overlook:

  1. Data compliance
  2. Multilingual representation

Apertus addresses these limitations by:

  • Pretraining solely on openly available data
  • Respecting robots.txt exclusions retroactively
  • Filtering toxic and PII content
  • Applying the Goldfish objective to limit memorization

It trains on 1,811 languages, with approximately 40% non-English data, and includes 149 languages in post-training.

Models Released

  • swiss-ai/Apertus-8B-2509
  • swiss-ai/Apertus-70B-2509
  • swiss-ai/Apertus-8B-Instruct-2509
  • swiss-ai/Apertus-70B-Instruct-2509

Safety Advisory

While powerful, Apertus models:

  • May hallucinate
  • May generate unsafe or toxic outputs
  • Are text-only (no multimodal support)

Deployment requires additional testing and alignment for specific use cases.