Kamil Paral
2018-10-05 17:31:07 UTC
(cross-posting to devel and desktop lists, ideally reply to both)
Hello,
this is a request for feedback regarding adjusting process limits to make
Steam/Wine work better on Fedora.
*A quick background:*
In August Valve announced [1] Proton [2], their own fork of Wine, to get
included in their Steam Play functionality, and allowing Linux players to
run Windows-only games transparently from Steam without any complex
configuration, just with a press of a button. Not everything works of
course, but the success rate is more than decent [3]. As far as I can tell,
the reception by Linux gamers has been *very* enthusiastic.
*The technical details:*
In order to boost performance, Valve included DXVK [4] which converts
DirectX calls to Vulkan (instead of OpenGL in vanilla Wine), and esync [5],
an existing wine patchset that performs process synchronization in a more
efficient manner than vanilla Wine, using file descriptors. You can read
the linked readme for full explanation.
The esync patchset is the main subject of this email. It uses a lot of file
descriptors and the default kernel limits are not sufficient for many
games. Valve notes that in their requirements document (under "FD LIMIT
REQUIREMENTS") [6] and it is further described in the esync readme [5]. The
documents also say that Debian and its derivatives like Ubuntu have already
raised the limit on open file descriptors, so those distributions work out
of the box with esync. Fedora is one of those that doesn't. I wonder if we
can consider changing that.
*Debian and Ubuntu:*
I've installed both Debian (Sid) and Ubuntu (18.10) to verify this, and can
confirm it. The default soft limit stays the same (1024), but the hard
limit is increased from 4096 to 1048576 (2^20). However, this only applies
to the systemd's system instance (systemd --system, PID 1), and not to
systemd user instances (systemd --user). Most of the apps you start in your
session are children of gnome-shell, which doesn't run under the systemd
user instance, so the higher limits apply for them as well (including
Steam). However some apps (probably started via dbus, I'm not really sure,
but importantly this includes also gnome-terminal) are started as children
of the systemd user instance and therefore have the original low limits
applied. I don't know whether this is intentional or just an omission. I
tried really hard to find the place where Debian/Ubuntu patches the
upstream limits, so that I could read some justification/explanation of the
change, but I wasn't able to find it (I searched the available patches for
kernel, systemd and pam).
*Configuring the limits:*
You can display the soft+hard limits of your current terminal using ulimit:
$ ulimit -Sn
1024
$ ulimit -Hn
4096
However, note that gnome-terminal runs under the systemd user session, so
at least in Debian/Ubuntu this will not see higher limits (i.e. neither
will Steam started from the terminal).
You can also use prlimit to see the limits of any running process. You can
use this to check limits of the systemd system instance, systemd user
instance, running Steam, etc.
$ sudo prlimit --nofile --pid 1
RESOURCE DESCRIPTION SOFT HARD UNITS
NOFILE max number of open files 1048576 1048576 files
You can modify the limits on the fly like this:
$ sudo prlimit --nofile=1024:1048576 --pid PID
You can increase the default limits by editing /etc/systemd/system.conf
(and /etc/systemd/user.conf, if you want to edit systemd user session as
well) and setting:
DefaultLimitNOFILE=1024:1048576
Alternatively, you can drop a file like this:
[Manager]
DefaultLimitNOFILE=1024:1048576
into /etc/systemd/system.conf.d/ (and /etc/systemd/user.conf.d/).
*Default limits in Fedora:*
From a technical point of view I'm not able to judge whether raising the
fileno limits by default is a trivial change or something with important
security implications. That's why I'm writing this email to hopefully get
replies from more knowledgeable people. The fact that Debian raised the
limits gives me hope we could do the same in Fedora (perhaps just in
Workstation, if the change in not welcome in the whole distribution). If
somebody can find the justification of Debian devs, that would be great.
I'd very much like to see Fedora (Workstation) being a good choice for
Linux gamers (we already packaged gamemode), and this might be an important
step to make sure Steam games don't work worse than on Ubuntu (but
hopefully even better, due to our more recent drivers).
Thanks for your feedback.
[1]
https://steamcommunity.com/games/221410/announcements/detail/1696055855739350561
[2] https://github.com/ValveSoftware/Proton
[3] https://spcr.netlify.com
[4] https://github.com/doitsujin/dxvk
[5] https://github.com/zfigura/wine/blob/esync/README.esync
[6]
https://github.com/ValveSoftware/Proton/blob/proton_3.7/PREREQS.md#fd-limit-requirements
Hello,
this is a request for feedback regarding adjusting process limits to make
Steam/Wine work better on Fedora.
*A quick background:*
In August Valve announced [1] Proton [2], their own fork of Wine, to get
included in their Steam Play functionality, and allowing Linux players to
run Windows-only games transparently from Steam without any complex
configuration, just with a press of a button. Not everything works of
course, but the success rate is more than decent [3]. As far as I can tell,
the reception by Linux gamers has been *very* enthusiastic.
*The technical details:*
In order to boost performance, Valve included DXVK [4] which converts
DirectX calls to Vulkan (instead of OpenGL in vanilla Wine), and esync [5],
an existing wine patchset that performs process synchronization in a more
efficient manner than vanilla Wine, using file descriptors. You can read
the linked readme for full explanation.
The esync patchset is the main subject of this email. It uses a lot of file
descriptors and the default kernel limits are not sufficient for many
games. Valve notes that in their requirements document (under "FD LIMIT
REQUIREMENTS") [6] and it is further described in the esync readme [5]. The
documents also say that Debian and its derivatives like Ubuntu have already
raised the limit on open file descriptors, so those distributions work out
of the box with esync. Fedora is one of those that doesn't. I wonder if we
can consider changing that.
*Debian and Ubuntu:*
I've installed both Debian (Sid) and Ubuntu (18.10) to verify this, and can
confirm it. The default soft limit stays the same (1024), but the hard
limit is increased from 4096 to 1048576 (2^20). However, this only applies
to the systemd's system instance (systemd --system, PID 1), and not to
systemd user instances (systemd --user). Most of the apps you start in your
session are children of gnome-shell, which doesn't run under the systemd
user instance, so the higher limits apply for them as well (including
Steam). However some apps (probably started via dbus, I'm not really sure,
but importantly this includes also gnome-terminal) are started as children
of the systemd user instance and therefore have the original low limits
applied. I don't know whether this is intentional or just an omission. I
tried really hard to find the place where Debian/Ubuntu patches the
upstream limits, so that I could read some justification/explanation of the
change, but I wasn't able to find it (I searched the available patches for
kernel, systemd and pam).
*Configuring the limits:*
You can display the soft+hard limits of your current terminal using ulimit:
$ ulimit -Sn
1024
$ ulimit -Hn
4096
However, note that gnome-terminal runs under the systemd user session, so
at least in Debian/Ubuntu this will not see higher limits (i.e. neither
will Steam started from the terminal).
You can also use prlimit to see the limits of any running process. You can
use this to check limits of the systemd system instance, systemd user
instance, running Steam, etc.
$ sudo prlimit --nofile --pid 1
RESOURCE DESCRIPTION SOFT HARD UNITS
NOFILE max number of open files 1048576 1048576 files
You can modify the limits on the fly like this:
$ sudo prlimit --nofile=1024:1048576 --pid PID
You can increase the default limits by editing /etc/systemd/system.conf
(and /etc/systemd/user.conf, if you want to edit systemd user session as
well) and setting:
DefaultLimitNOFILE=1024:1048576
Alternatively, you can drop a file like this:
[Manager]
DefaultLimitNOFILE=1024:1048576
into /etc/systemd/system.conf.d/ (and /etc/systemd/user.conf.d/).
*Default limits in Fedora:*
From a technical point of view I'm not able to judge whether raising the
fileno limits by default is a trivial change or something with important
security implications. That's why I'm writing this email to hopefully get
replies from more knowledgeable people. The fact that Debian raised the
limits gives me hope we could do the same in Fedora (perhaps just in
Workstation, if the change in not welcome in the whole distribution). If
somebody can find the justification of Debian devs, that would be great.
I'd very much like to see Fedora (Workstation) being a good choice for
Linux gamers (we already packaged gamemode), and this might be an important
step to make sure Steam games don't work worse than on Ubuntu (but
hopefully even better, due to our more recent drivers).
Thanks for your feedback.
[1]
https://steamcommunity.com/games/221410/announcements/detail/1696055855739350561
[2] https://github.com/ValveSoftware/Proton
[3] https://spcr.netlify.com
[4] https://github.com/doitsujin/dxvk
[5] https://github.com/zfigura/wine/blob/esync/README.esync
[6]
https://github.com/ValveSoftware/Proton/blob/proton_3.7/PREREQS.md#fd-limit-requirements