Tutorial:Improving Server I/O utilising memory

It has been suggested that this page be merged with Tutorial:Ramdisk enabled server into Tutorial:Improving Server I/O utilising memory.

^[discuss^]

If this merge affects many pages or may potentially be controversial, do not merge the page until a consensus is reached.
Reason: Both cover the same area of information, and attempt to solve the same problem.

This tutorial is intended to guide you through the process of improving server I/O operations by utilising the system memory. This should only be done if disk I/O is of issue to the server. The benefits and downsides of this shall also be covered.

Prerequisites

This article will primarily cover how to set this up on a Linux system. It is assumed you are already familiar with such systems.
Examples assume usage of Java Edition, and may or may not be applicable to other versions.
You already have a Minecraft server set up. Consider reading Tutorial:Setting up a Java Edition server if you haven't.
When using Cron, it is already installed.

Introduction

In computer systems, there are different methods of data storage. Here is primarily focussed upon the short-term and long-term storage of data. Where often long-term storage is something like a hard-disk drive or solid-state drive, and short-term storage is something like computer memory. In almost all cases memory read/writes is magnitudes faster compared to disk read/writes.

With a Minecraft server, everything is initially happening in memory. Eventually, with internal scheduling rules, certain data is committed to disk. This ensures that the session is able to be restored upon restarts of the server, or whole system.

In certain cases, particularly with slower disks such as hard disk drives, this can cause issues which leads to the Minecraft server being overwhelmed. This is commonly observed as Can't keep up! Is the server overloaded? log messages, tick lag, or crashes. It is recommended to verify whether it is indeed disk I/O causing the issue before proceeding. Since the issue may lay elsewhere. This can be achieved through observing CPU performance, if Minecraft has enough memory, et cetera. Collecting metrics can potentially aid with this process. Alternatively, it is possible to temporarily disable autosaving utilising commands/save.

Depending on the severity of the bottleneck, turning off sync-chunk-writes in server.properties may be all you need. Since this can help to reduce I/O delays, and potentially prevent crashes.

Utilising memory to reduce disk I/O strain

There are multiple approaches to utilising the system memory to reduce the strain on disk I/O operations. One will be covered here, another is covered over at Tutorial:Ramdisk enabled server which utilises a ramdisk to have server storage live entirely in RAM.The method inside this article will utilise Minecraft's internal processes to reduce I/O strain. This is done through turning off automatic saves, and scheduling our own periodic saves.

These can be combined, or done separately, whatever is most applicable in your specific use case.

Exposing stdin

Stdin is recommended to be exposed, for custom save scheduling it not recommended to skip this step. However it may not possible to do this in your case, or you'd prefer not to do this. For that, the same can be achieved using solely datapacks. Though, this will be less reliable due to it being affected by lag, and might raise concerns about security. More exact instructions on how to do this with a datapack will be described later in the article.

Using systemd

For this method, it is recommended to set up Minecraft as a systemd unit. Below a minimal file has been provided which will be located at /etc/systemd/system/minecraft.service.

#/etc/systemd/system/minecraft.service
[Unit]
Description=My Minecraft server
Requires=minecraft.socket
After=network.target

# It is highly recommended to set these to a low-privilege user and group,
# otherwise Minecraft is executed as root, which serves a severe security risk.
[Service]
User=minecraft
Group=minecraft

# Sample for when using a startup script
#ExecStart=/path/to/run.sh
ExecStart=java -jar /path/to/server.jar --nogui # Your minecraft start command

Sockets=minecraft.socket
StandardInput=socket

# Read the systemd.unit(5) manpage for further information.

Then a socket file should be created, it is recommended to use the same file name as the service file. Thus, in this example it shall be located at /etc/systemd/system/minecraft.socket.

#/etc/systemd/system/minecraft.socket
[Socket]
Description=STDIN for my Minecraft server
Service=minecraft.service

# Will create a named pipe at /run/minecraft/stdin read/writable
# for the owner and group.
ListenFIFO=%t/minecraft/stdin
RuntimeDirectory=minecraft
SocketMode=0660
RemoveOnStop=yes

# Set to a user who will be allowed to send commands to the server.
SocketUser=minecraft
SocketGroup=minecraft

Using a wrapper script

Alternatively, a startup script can be used. Where you will have to create and edit a file, ensuring to add executing permission.

#!/usr/bin/env sh
set -e # Exit the shell script if an error occurs

# Create the named pipe, if it doesn't exist already
[ -p /path/to/named/pipe ] ||
	mkfifo /path/to/named/pipe

# Start the server
tail -f /path/to/named/pipe |
	java -jar server.jar --nogui

Note that this will make the "normal" standard input inaccessible in an interactive session. However, this shouldn't be a problem.

Custom write scheduling

For this method, autosaving must first be disabled using /save-off each time the server launched. Afterwards, you can choose a delay for each time Minecraft synchronises to disk. It's recommended to set this equal to the value in max-tick-time in server.properties. Note that this value is in miliseconds.

Using Systemd

First, if using systemd for stdin under each respective block.

[Unit]
# ...
Requires=minecraft-save.timer

[Install]
# ...
Also=minecraft-save.service

This ensures the timer is started when the service has been started. Otherwise, you will have to use your own logic, and perhaps consider using cron. (see below)

After that has been done, we can create the file specified in the new Requires field:

#/etc/systemd/system/minecraft-save.timer
[Unit]
Description=My Minecraft server autosave
PartOf=minecraft.service
After=minecraft.service
Requires=minecraft-save.service
Requires=minecraft-disable-autosave.service

[Timer]
Unit=minecraft-save.service
# Set these to your chosen value, the default for
# max-tick-time is 60000, so 60s is used.
# Change both values.
OnActiveSec=60
OnUnitActiveSec=60

[Install]
WantedBy=timers.target
Also=minecraft-disable-autosave.service

Then we need to create minecraft-disable-autosave.service and minecraft-save.service, and ensure they contain something resembling the following:

# /etc/systemd/system/minecraft-save.service
[Unit]
Description=Saves the Minecraft server to disk

[Service]
Type=oneshot
ExecStart=echo save-all
StandardOutput=file:%t/minecraft/stdin

# /etc/systemd/system/minecraft-disable-autosave.service
[Unit]
Description=Disable auto-save for the Minecraft server

[Service]
Type=oneshot
ExecStart=echo save-off
StandardOutput=file:%t/minecraft/stdin

Using a script + Cron

Firstly, you should add the following line to your startup script:

# ...
# Writes to the named pipe as soon as it is being read from. (Ran in background)
echo "save-off" >/path/to/stdin &
# ...

Afterwards, on the user that is executing the Minecraft server, and thus has read/write permission, you should perform crontab -e in an interactive shell. In case you haven't got your preferred editor set in your EDITOR or VISUAL Environment Variable, prefix the command with EDITOR=nano , including the space. This will use GNU Nano as your editor.

Once opened, specify the decided upon delay and the command similar to below:

# <min> <hour> <day-of-month> <day-of-week> <command>
  */1   *      *              *             echo "save-all" >/path/to/stdin

Note that the */n wildcard specify the 'n'-th interval for that unit of time.

Using a datapack (not recommended)

It is not recommended to implement this using a datapack due to issues with lag spikes. It is required to set function-permission-level=4 in server.properties, which can be seen as insecure.

Within the initialisation function the following must be placed:

# Disable auto-saving, so our custom scheduling is preferred.
save-off

# Schedule our saving function to our chosen delay
schedule function namespace:my/function 60s

After that, we shall create the specified function, which will have to contain something like the following:

# Mark all changes as needing to be written, and write them to disk.
save-all

# Schedule ourselves again, with the same delay
schedule function namespace:my/function 60s