Date: Mon, 6 Dec 2021 16:16:06 +0100 From: Matthias Gerstner <mgerstner@...e.de> To: oss-security@...ts.openwall.com Subject: tmate-ssh-server: Local Privilege Escalation Issues and DoS issues (CVE-2021-44512, CVE-2021-44513) Hello, this report is about code review results of the tmate terminal sharing software . Some local privilege escalation and remote denial-of-service attack vectors have been identified. Skip to section 6) for the concrete findings. The following sections give a broader overview of the tmate security design. : https://tmate.io 1) What is tmate? ================= Tmate allows a terminal session to be easily shared over the network using the SSH protocol. Other people can attach themselves to the shared terminal session either with full access or with read-only access (just viewing what happens on the terminal). Tmate consists of two forks of the tmux terminal multiplexer's  code base. One fork is for the tmate client side  (the party that is sharing its terminal over the network) and one fork is for the tmate server side  (the central party that relays SSH connections). Both forks originate from the year 2016 and no sync seems to have happened since then. The upstream author states that he doesn't backport fixes any more due to lack of time. The tmate client actively connects to the tmate server side using an outgoing SSH connection. This SSH session will be kept alive and a tmux terminal session is established with some additional instructions shown on the screen on how to attach to the tmux session via the network. When other people connect to the tmate server using a regular SSH client and a secret token as username they will be relayed to the tmate client's tmux session and can fully access or view it (read-only mode). : https://github.com/tmux/tmux : https://github.com/tmate-io/tmate.git : https://github.com/tmate-io/tmate-ssh-server.git 2) Review Scope =============== This review was performed for tmate client version 2.4.0. On the server side the current Git development status of tmate-ssh-server as of commit befd49f4 was used. This is because the last tmate-ssh-server release (version 2.3.0) is already over two years old, and a larger number of unreleased changes is found on the project's master branch by now. Most of the statements and findings in this report should also be true for the 2.3.0 server version release, however. This review was focused on the general security architecture of the tmate protocol and the network faced interfaces. 3) General Cryptographic Security ================================= The tried and tested openssh security model is changed a lot in tmate. In regular SSH the two parties establish a secure connection (secure against eavesdropping), then verify each other. Verification means that the client verifies the fingerprint of the public SSH server host key to make sure it is the correct party it is talking to and no man in the middle is around. Then the server requires authentication of the client as the requested login user e.g. via tunneled cleartext password, public key or PAM module authentication mechanisms etc. In tmate we have three parties: - A: the party that is sharing its terminal using the tmate client. - B: the central tmate server that manages multiple tmate sessions and is typically publicly accessible in some form. - C: the party that is connecting to a shared terminal using a regular SSH client. The typical communication flow goes like this: 1. The tmate client (A) connects to to the tmate server (B). It verifies the server's fingerprint. Server hostname, port and fingerprint are configured via $HOME/.tmate.conf ("tmate-server-host", "tmate-server-port", "tmate-server-rsa-fingerprint"). 2. The tmate server (B) performs no verification at all, because any tmate client is allowed to create a new session on it. It generates a random token consisting of 25 alphanumeric characters. This token uniquely identifies the new session and is communicated back to the tmate client (A). 3. The tmate client (A) displays the random token and the valid SSH URLs to attach to the shared terminal session. It is a command line like follows for full or read-only access respectively: ``` ssh T6PAFr59tsrfEWCaUZg8APCAe@...e-team-mate-server.org ssh ro-4Eu99VBssTnw9Q3LuYQUzcLEy@...e-team-mate-server.org ``` 4. The secret token / command line now needs to be communicated to people that want to attach to the terminal. This step is critical, because it depends on the person that wants to share the terminal and how it shares the token with others. 5. The user (C) wanting to attach to the shared session now needs to connect to the tmate server (B) using the correct secret token to get access to the shared terminal. (C) will verify the fingerprint of (B), but (B) will not authenticate (C) beyond the knowledge of the secret token. The tmate server (B) will now relay data between (A) and (C). So in contrast to the classical SSH setup the two endpoints (A) and (C) are never verifying each other in any way, except via the secret token of 25 characters. So the host fingerprint verification and the user authentication is all condensed into this secret token that needs to be forwarded from (C) to (A) by some means. Any unintended party that gets hold of the secret token can: - gain full or read-only access to the session shared by (A) - replace the session shared by (A) by a malicious session and thus trick (C) (instead of a random token also an explicit fixed token can be used that overwrites existing sessions) Furthermore the tmate-server (B) is a third party in this setup that needs to be fully trusted by both (A) and (C). If (B) is compromised then all security is gone. To summarize, the security model used in tmate makes it easy for people to share their terminals over SSH, but this simplicity may come with a false sense of security, because the mechanisms used in the background are rather complex and the important step 4 depends fully on how the person sharing its terminal is treating the secret token. 4) Default Setup of tmate ========================= When looking at a default installation of the tmate client (A) then it is very simple to share a terminal via the default upstream server "ssh.tmate.io". This upstream server name and its fingerprint are hard coded in the client in source file "options-table.c". This means trust for the default upstream server party (B) is builtin, a decision that should be left up to the user in my opinion. Thus it is enough to just type "tmate" in a shell to immediately share terminal access with the upstream server and possibly give full control to it, should it be compromised in some form. I discussed this with the upstream developer and he agreed to make this an opt-in but he is still considering options to maintain backward-compatibility for existing users. 5) Code Quality and Design ========================== Getting an overview of the critical code is not all that easy. The SSH logic (based on libssh) feels crammed into the tmux code base. The dividing line between original tmux and changed tmate code was difficult to find for me. In some spots `#ifdef TMATE` sections are found, some additional source files are sprinkled in the source directory. Especially on the tmate-ssh-server side the tmux internals are basically openly connected to the Internet and this raises the question whether the tmux developers considered untrusted input in these areas very much. The code paths accessible to unauthenticated users (and for the tmate-ssh server party (B) all connections are unauthenticated) are pretty broad and hard to follow during code review. The network protocol logic which is based on libmsgpackc binary data items does not enforce maximum string lengths which means that a variety of DoS attacks are possible against party B. The one (probably crucial) security measure taken by the tmate server source code to protect itself is the following: - each new session is forked from the master process - each forked session is placed into a "jail" that consists of some separate namespaces and a chroot jail that runs as user "nobody" by default It is time consuming to make sure that there are no major security issues in this area reachable over the network. Fuzzing might be an approach to check this more quickly, I did not go further in this direction, though. If code execution could be achieved in a forked session process on party B) then "only" a way out of the jail needs to be found to reach the same security scope as all the other tmate sessions running on the same host. 6) Individual Findings ====================== These are all issues on the tmate-ssh-server code base running on party B). a) Local security issues in /tmp/tmate (CVE-2021-44512, CVE-2021-44513) ----------------------------------------------------------------------- The tmate-ssh-server maintains a world-writable directory in /tmp/tmate/sessions into which UNIX domain sockets named after the secret tokens are placed: ``` $ ls -ld /tmp/tmate drwx-----x 4 tmate users 80 Sep 1 11:04 /tmp/tmate $ ls -ld /tmp/tmate/sessions drwx----wx 2 tmate users 120 Sep 1 11:33 /tmp/tmate/sessions/ $ ls -l /tmp/tmate/sessions srw-rw---- 1 user users 0 Sep 1 11:05 aVMAvcCWupR3DTK7JF2NfxLeS lrwxrwxrwx 1 user users 25 Sep 1 11:33 ro-4Eu99VBssTnw9Q3LuYQUzcLEy \ -> T6PAFr59tsrfEWCaUZg8APCAe lrwxrwxrwx 1 user users 25 Sep 1 11:05 ro-XjB7FchbDAjmMmNnXf2wry3bV \ -> aVMAvcCWupR3DTK7JF2NfxLeS srw-rw---- 1 user users 0 Sep 1 11:33 T6PAFr59tsrfEWCaUZg8APCAe ``` So this is how different sessions are maintained. If a file is a symlink then it is considered a read-only session, otherwise read-write. This setup is subject to a race condition (CVE-2021-44513): - the creation of these directories is unsafe using `mkdir()` and `chmod()` system calls in `main()` found in source file "tmate-main.c". The code potentially reuses existing directories that belong to other users. Only the following `chmod()`s would fail if the owner does not match. This is a race condition, however, that could be won by using symlinks. In this case a local attacker could gain full control over this directory structure and thus create additional malicious sessions or get control of existing, legit sessions. This setup uses too broad permissions in /tmp/tmate and /tmp/tmate/sessions (CVE-2021-44512): - since /tmp/tmate/sessions is world-writable a local malicious user can create arbitrary new files in there. For example a UNIX domain socket that reaches an attacker controlled local process instead of an actual tmate server session. - although /tmp/tmate/sessions only allows 'wx' for other users this still allows to execute a `readlink()` system call on existing symbolic links in the directory. Thus a local compromised user that has knowledge of the secret token for a read-only session can find out the token for the read-write session and get full access. Both problems are addressed by upstream commit 1c020d1f . I recommended to the upstream author to perform the ownership check first and only then perform the `chmod()` calls, but this change did not happen yet. : https://github.com/tmate-io/tmate-ssh-server/commit/1c020d1f5ca462f5b150b46a027aaa1bbe3c9596 b) Denial-of-Service Attack Vectors ----------------------------------- - there is no limit to the number of sessions that can be created on the tmate ssh server. A simple `ssh someone@...tmate-server -s tmate -p $tmate_port` is enough to establish a new session, get a new session token in /tmp/tmate/sessions and a forked session process. This can be done a lot of times in parallel to consume resources on the host. - by sending overlong `HEADER` cilent identification, or overly big window dimensions via the `SYNC_LAYOUT` message memory exhaustion can be triggered on the host. This will usually "only" kill the forked session process but could also negatively impact availibility of the service to other sessions. Practically all string parameters in the network protocol have no size limit, I only outlined two prominent cases. - I managed to cause a NULL pointer dereference by sending a bad `SYNC_LAYOUT` message, but triggering it is subject to a race condition for some reason. ### Reproducers Attached is a simple Python script that allows to trigger some of the DoS issues: ``` # this will not lead to an immediate OOM but shows the principle of sending # overly long headers $ tmate_dos.py --send-long-header # this should lead to an immediate OOM by specifying overly large pane # parameters $ tmate_dos.py --allocate-gigantic-pane # this often leads to a NULL pointer dereference (not fully reliable for # some reason) $ tmate_dos.py --send-bad-pane-id ``` ### Fixes Some of these aspects have been addressed by upstream commits , , . The upstream author expressed that things like a limit on the number of sessions should be covered by using cgroup limits or similar operating system features. No such setup is available upstream yet though. : https://github.com/tmate-io/tmate-ssh-server/commit/36f073b4ccf05da2fd51cc10a2debb443c592c50 : https://github.com/tmate-io/tmate-ssh-server/commit/b41672b634af4ec8797449e78e4b731e24e26e16 : https://github.com/tmate-io/tmate-ssh-server/commit/1f314123df2bb29cb07427ed8663a81c8d9034fd c) Miscellaneous Bits --------------------- - the tmate-server only seems to support two SSH host key types "rsa" and "ed25519" and thus restricts the available cryptographic algorithms (for example no ECDSA). - the handling of random data via `random_stream_init()` and `random_stream_get()` is a bit peculiar. It caches up to 256 bytes of random data in the process that could in theory be shared with child processes and thus multiple processes could use the same random data. Luckily the only client code currently *always* calls `random_stream_init()` and thus effectively no cached random data is ever used. - file descriptors are not opened with `O_CLOEXEC` flag by default but are only later explicitly closed via `close_fds_except()`. Since there is no `execve()` taking place this is necessary, but having `O_CLOEXEC` to avoid accidental future inheritance to unrelated programs would be sensible, too. 7) Timeline =========== 2021-09-03: I reported the findings to the upstream author offering a maximum of 90 days coordinated disclosure. 2021-10-17: The upstream author published the fixes that are also referenced in this report. I tried to establish a clear publication date for the issues but did not get a clear answer. 2021-11-29: I remembered the upstream author about the maximum 90 days non-disclosure time approaching. I failed to get any concrete information on release plans or finalization of some aspects of the fixes. 2021-11-30: I requested CVEs from Mitre for the most pressing issues found in the report. Cheers Matthias -- Matthias Gerstner <matthias.gerstner@...e.de> Security Engineer https://www.suse.com/security Phone: +49 911 740 53 290 GPG Key ID: 0x14C405C971923553 SUSE Software Solutions Germany GmbH HRB 36809, AG Nürnberg Geschäftsführer: Ivo Totev View attachment "tmate_dos.py" of type "text/x-python" (2151 bytes) Download attachment "signature.asc" of type "application/pgp-signature" (834 bytes)
Powered by blists - more mailing lists
Please check out the Open Source Software Security Wiki, which is counterpart to this mailing list.
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.