Overleaf docs
Plans and pricingTemplatesUser docsGo to Overleaf
On-premises
On-premises
  • Welcome
    • Server Pro vs. Community Edition
  • Release notes
    • Release notes 5.x.x
      • Doc version recovery
    • Release notes 4.x.x
    • Release notes 3.x.x
      • Full project history migration
    • Release notes 2.x.x
    • Release notes 1.x.x
    • Release notes 0.x.x
  • Getting started
    • Before you start
    • Requirements
      • Skills needed
      • Hardware requirements
      • Software requirements
    • Microservices
    • Server Pro infrastructure
    • What is the Overleaf Toolkit?
  • Installation
    • Introduction
    • Using the Toolkit
      • 1: Download the Toolkit
      • 2: Familiarize yourself with the Toolkit
      • 3: Initialize the configuration
      • 4. Choose Community Edition or Server Pro
      • 5. Personalizing your instance
      • 6. Post-installation tasks
    • Air-gapped/offline deployments
    • Upgrading TeX Live
  • Configuration
    • Overleaf Toolkit
      • Files and locations
      • Toolkit settings
      • Environment variables
      • Server Pro-only configuration
        • LDAP
        • SAML 2.0
        • Sandboxed Compiles
        • Git integration
        • Templates
        • Adding LaTeX user help
      • Logging
      • TLS proxy
      • Branding
      • Localization
      • Email delivery
      • Redis
      • S3
  • Maintenance
    • docker-compose.yml to Toolkit migration
    • Upgrading your deployment
    • Data and backups
      • Exporting projects
    • Extending TeX Live
    • Horizontal scaling
    • S3 migration
    • Updating MongoDB
  • User and project management
    • User management
      • Username migration
    • Understanding license usage
    • Project management
  • Support
    • Project limits
    • Troubleshooting
    • Getting help
    • Support guides
      • Using templates as an individual
    • Overleaf user docs
Powered by GitBook
LogoLogo

Discover Overleaf

  • Home
  • Features

Solutions

  • Plans and pricing
  • For universities
  • For business
  • For government

Resources

  • Templates
  • User docs and LaTeX learning
  • Blog

© Overleaf

On this page
  • When to consider using S3 for data storage
  • S3 compatible object storage options
  • Latency considerations when picking a S3 compatible object storage
  • S3 setup
  • Access Policies
  • Overview of variables
  • MINIO setup

Was this helpful?

Export as PDF
  1. Configuration
  2. Overleaf Toolkit

S3

PreviousRedisNextdocker-compose.yml to Toolkit migration

Last updated 3 months ago

Was this helpful?

This document covers the setup of S3 in Server CE and Server Pro. A can be found on migrating existing data to S3 compatible storage.

When to consider using S3 for data storage

For instances with fewer than 1000 seats we recommend using local disk storage with regular .

For larger instances with more than 1000 seats that reach limits of their local storage (size or throughput), we recommend using a S3 compatible object storage back end over other network based storage solutions like NFS.

S3 compatible object storage options

Here are the most popular options for S3 compatible object storage:

  • , managed, we suggest picking AWS S3 when running Overleaf CE/Server Pro on AWS

  • , self-hosted

  • , self-hosted

  • Other hosting providers also have some kind of managed S3 compatible object storage, you may want to use these instead of running your own when already running Overleaf CE/Server Pro at such a provider.

Latency considerations when picking a S3 compatible object storage

The latency between the Server CE/Server Pro instance and your S3 compatible object storage is a big contributor to the time it takes to complete the migration. The latency also impacts the file-upload performance in Server CE/Server Pro and slow file-downloads can have a big impact on PDF compile times as well. We suggest minimizing the geo-graphical distance between your Server CE/Server Pro instance and the S3 compatible object storage. In a managed environment, this would mean provisioning a bucket in the same region, and for an on-premise solution, running the two on the same campus.

S3 setup

We need four "buckets" and two restricted user accounts.

Buckets should not be publicly accessible

Bucket

Usage

Service

Previously in /var/lib/overleaf/data

overleaf-user-files

project user files

filestore

user_files

overleaf-template-files

template files

filestore

template_files

overleaf-project-blobs

project history blobs

history

history/overleaf-project-blobs

overleaf-chunks

history chunks

history

history/overleaf-chunks

You may want/need to pick a different name, be sure to use the custom buckets in all the commands.

The following will use placeholders for actual credentials:

Environment variable
Description

OVERLEAF_FILESTORE_S3_ACCESS_KEY_ID

The access key/username of the restricted user of the filestore service.

OVERLEAF_FILESTORE_S3_SECRET_ACCESS_KEY

The secret key/password of the restricted user of the filestore service.

OVERLEAF_HISTORY_S3_ACCESS_KEY_ID

The access key/username of the restricted user of the history service.

OVERLEAF_HISTORY_S3_SECRET_ACCESS_KEY

The secret key/password of the restricted user of the history service.

Server CE and Server Pro only needs a small set of permissions on each bucket:

  • create object

  • get object

  • delete object

  • list bucket

Access Policies

Here is how a policy for the filestore user could look like:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::overleaf-user-files"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::overleaf-user-files/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::overleaf-template-files"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::overleaf-template-files/*"
    }
  ]
}

Here is how a policy for the history user could look like:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::overleaf-project-blobs"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::overleaf-project-blobs/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::overleaf-chunks"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::overleaf-chunks/*"
    }
  ]
}

Overview of variables

When using AWS S3

# Enable S3 backend for filestore
OVERLEAF_FILESTORE_BACKEND=s3

# Bucket name for project files
OVERLEAF_FILESTORE_USER_FILES_BUCKET_NAME=overleaf-user-files

# Bucket name for template files
OVERLEAF_FILESTORE_TEMPLATE_FILES_BUCKET_NAME=overleaf-template-files

# Key for filestore user
OVERLEAF_FILESTORE_S3_ACCESS_KEY_ID=...

# Secret for filestore user
OVERLEAF_FILESTORE_S3_SECRET_ACCESS_KEY=...

# Bucket region you picked when creating the buckets.
OVERLEAF_FILESTORE_S3_REGION=""

# Enable S3 backend for history
OVERLEAF_HISTORY_BACKEND=s3

# Bucket name for project history blobs
OVERLEAF_HISTORY_PROJECT_BLOBS_BUCKET=overleaf-project-blobs

# Bucket name for history chunks
OVERLEAF_HISTORY_CHUNKS_BUCKET=overleaf-chunks

# Key for history user
OVERLEAF_HISTORY_S3_ACCESS_KEY_ID=...

# Secret for history user
OVERLEAF_HISTORY_S3_SECRET_ACCESS_KEY=...

# Bucket region you picked when creating the buckets.
OVERLEAF_HISTORY_S3_REGION=""

When using a self-hosted option

# Enable S3 backend for filestore
OVERLEAF_FILESTORE_BACKEND=s3

# Bucket name for project files
OVERLEAF_FILESTORE_USER_FILES_BUCKET_NAME=overleaf-user-files

# Bucket name for template files
OVERLEAF_FILESTORE_TEMPLATE_FILES_BUCKET_NAME=overleaf-template-files

# Key for filestore user
OVERLEAF_FILESTORE_S3_ACCESS_KEY_ID=...

# Secret for filestore user
OVERLEAF_FILESTORE_S3_SECRET_ACCESS_KEY=...

# S3 provider endpoint
OVERLEAF_FILESTORE_S3_ENDPOINT=http://10.10.10.10:9000

# Path style addressing of buckets. Most likely you need to set this to "true".
OVERLEAF_FILESTORE_S3_PATH_STYLE="true"

# Bucket region. Most likely you do not need to configure this.
OVERLEAF_FILESTORE_S3_REGION=""

# Enable S3 backend for history
OVERLEAF_HISTORY_BACKEND=s3

# Bucket name for project history blobs
OVERLEAF_HISTORY_PROJECT_BLOBS_BUCKET=overleaf-project-blobs

# Bucket name for history chunks
OVERLEAF_HISTORY_CHUNKS_BUCKET=overleaf-chunks

# Key for history user
OVERLEAF_HISTORY_S3_ACCESS_KEY_ID=...

# Secret for history user
OVERLEAF_HISTORY_S3_SECRET_ACCESS_KEY=...

# S3 provider endpoint
OVERLEAF_HISTORY_S3_ENDPOINT=http://10.10.10.10:9000

# Path style addressing of buckets. Most likely you need to set this to "true".
OVERLEAF_HISTORY_S3_PATH_STYLE="true"

# Bucket region. Most likely you do not need to configure this.
OVERLEAF_HISTORY_S3_REGION=""

MINIO setup

MINIO_ROOT_USER and MINIO_ROOT_PASSWORD are the root credentials of the MINIO instance.

mc alias set s3 http://10.10.10.10:9000 MINIO_ROOT_USER MINIO_ROOT_PASSWORD

# Put the contents of the policies from the previous section in the
# respective json file policy-filestore.json and policy-history.json.
# Reminder: Replace the bucket names and credentials accordingly.

# filestore buckets, user and policy
mc mb --ignore-existing s3/overleaf-user-files
mc mb --ignore-existing s3/overleaf-template-files
mc admin user add s3 \
  OVERLEAF_FILESTORE_S3_ACCESS_KEY_ID \
  OVERLEAF_FILESTORE_S3_SECRET_ACCESS_KEY
mc admin policy create s3 overleaf-filestore policy-filestore.json
mc admin policy attach s3 overleaf-filestore \
  --user=OVERLEAF_FILESTORE_S3_ACCESS_KEY_ID

# history buckets, user and policy
mc mb --ignore-existing s3/overleaf-project-blobs
mc mb --ignore-existing s3/overleaf-chunks
mc admin user add s3 \
  OVERLEAF_HISTORY_S3_ACCESS_KEY_ID \
  OVERLEAF_HISTORY_S3_SECRET_ACCESS_KEY
mc admin policy create s3 overleaf-history policy-history.json
mc admin policy attach s3 overleaf-history \
  --user=OVERLEAF_HISTORY_S3_ACCESS_KEY_ID

Please follow the for obtaining a copy of mc.

official documentation
separate guide
AWS S3
MINIO
Ceph
consistent backups