MiniTest::Unit::TestCase

Public Class Methods

bench_exp(min, max, base = 10) click to toggle source

Returns a set of ranges stepped exponentially from min to max by powers of base. Eg:

bench_exp(2, 16, 2) # => [2, 4, 8, 16]

 
               # File minitest/benchmark.rb, line 28
def self.bench_exp min, max, base = 10
  min = (Math.log10(min) / Math.log10(base)).to_i
  max = (Math.log10(max) / Math.log10(base)).to_i

  (min..max).map { |m| base ** m }.to_a
end

bench_linear(min, max, step = 10) click to toggle source

Returns a set of ranges stepped linearly from min to max by step. Eg:

bench_linear(20, 40, 10) # => [20, 30, 40]

 
               # File minitest/benchmark.rb, line 41
def self.bench_linear min, max, step = 10
  (min..max).step(step).to_a
rescue LocalJumpError # 1.8.6
  r = []; (min..max).step(step) { |n| r << n }; r
end

bench_range() click to toggle source

Specifies the ranges used for benchmarking for that class. Defaults to exponential growth from 1 to 10k by powers of 10. Override if you need different ranges for your benchmarks.

See also: ::bench_exp and ::bench_linear.

 
               # File minitest/benchmark.rb, line 69
def self.bench_range
  bench_exp 1, 10_000
end

benchmark_suites() click to toggle source

Returns all test suites that have benchmark methods.

 
               # File minitest/benchmark.rb, line 58
def self.benchmark_suites
  TestCase.test_suites.reject { |s| s.benchmark_methods.empty? }
end

Public Instance Methods

assert_performance(validation, &work) click to toggle source

Runs the given work, gathering the times of each run. Range and times are then passed to a given validation proc. Outputs the benchmark name and times in tab-separated format, making it easy to paste into a spreadsheet for graphing or further analysis.

Ranges are specified by ::bench_range.

Eg:

def bench_algorithm
  validation = proc { |x, y| ... }
  assert_performance validation do |x|
    @obj.algorithm
  end
end

 
               # File minitest/benchmark.rb, line 91
def assert_performance validation, &work
  range = self.class.bench_range

  io.print "#{__name__}"

  times = []

  range.each do |x|
    GC.start
    t0 = Time.now
    instance_exec(x, &work)
    t = Time.now - t0

    io.print "\t%9.6f" % t
    times << t
  end
  io.puts

  validation[range, times]
end

assert_performance_constant(threshold = 0.99, &work) click to toggle source

Runs the given work and asserts that the times gathered fit to match a constant rate (eg, linear slope == 0) within a given threshold. Note: because we’re testing for a slope of 0, R^2 is not a good determining factor for the fit, so the threshold is applied against the slope itself. As such, you probably want to tighten it from the default.

See www.graphpad.com/curvefit/goodness_of_fit.htm for more details.

Fit is calculated by fit_linear.

Ranges are specified by ::bench_range.

Eg:

def bench_algorithm
  assert_performance_constant 0.9999 do |x|
    @obj.algorithm
  end
end

 
               # File minitest/benchmark.rb, line 135
def assert_performance_constant threshold = 0.99, &work
  validation = proc do |range, times|
    a, b, rr = fit_linear range, times
    assert_in_delta 0, b, 1 - threshold
    [a, b, rr]
  end

  assert_performance validation, &work
end

assert_performance_exponential(threshold = 0.99, &work) click to toggle source

Runs the given work and asserts that the times gathered fit to match a exponential curve within a given error threshold.

Fit is calculated by fit_exponential.

Ranges are specified by ::bench_range.

Eg:

def bench_algorithm
  assert_performance_exponential 0.9999 do |x|
    @obj.algorithm
  end
end

 
               # File minitest/benchmark.rb, line 161
def assert_performance_exponential threshold = 0.99, &work
  assert_performance validation_for_fit(:exponential, threshold), &work
end

assert_performance_linear(threshold = 0.99, &work) click to toggle source

Runs the given work and asserts that the times gathered fit to match a straight line within a given error threshold.

Fit is calculated by fit_linear.

Ranges are specified by ::bench_range.

Eg:

def bench_algorithm
  assert_performance_linear 0.9999 do |x|
    @obj.algorithm
  end
end

 
               # File minitest/benchmark.rb, line 181
def assert_performance_linear threshold = 0.99, &work
  assert_performance validation_for_fit(:linear, threshold), &work
end

assert_performance_power(threshold = 0.99, &work) click to toggle source

Runs the given work and asserts that the times gathered curve fit to match a power curve within a given error threshold.

Fit is calculated by fit_power.

Ranges are specified by ::bench_range.

Eg:

def bench_algorithm
  assert_performance_power 0.9999 do |x|
    @obj.algorithm
  end
end

 
               # File minitest/benchmark.rb, line 201
def assert_performance_power threshold = 0.99, &work
  assert_performance validation_for_fit(:power, threshold), &work
end

fit_error(xys) click to toggle source

Takes an array of x/y pairs and calculates the general R^2 value.

See: en.wikipedia.org/wiki/Coefficient_of_determination

 
               # File minitest/benchmark.rb, line 210
def fit_error xys
  y_bar  = sigma(xys) { |x, y| y } / xys.size.to_f
  ss_tot = sigma(xys) { |x, y| (y    - y_bar) ** 2 }
  ss_err = sigma(xys) { |x, y| (yield(x) - y) ** 2 }

  1 - (ss_err / ss_tot)
end

fit_exponential(xs, ys) click to toggle source

To fit a functional form: y = ae^(bx).

Takes x and y values and returns [a, b, r^2].

See: mathworld.wolfram.com/LeastSquaresFittingExponential.html

 
               # File minitest/benchmark.rb, line 225
def fit_exponential xs, ys
  n     = xs.size
  xys   = xs.zip(ys)
  sxlny = sigma(xys) { |x,y| x * Math.log(y) }
  slny  = sigma(xys) { |x,y| Math.log(y)     }
  sx2   = sigma(xys) { |x,y| x * x           }
  sx    = sigma xs

  c = n * sx2 - sx ** 2
  a = (slny * sx2 - sx * sxlny) / c
  b = ( n * sxlny - sx * slny ) / c

  return Math.exp(a), b, fit_error(xys) { |x| Math.exp(a + b * x) }
end

fit_linear(xs, ys) click to toggle source

Fits the functional form: a + bx.

Takes x and y values and returns [a, b, r^2].

See: mathworld.wolfram.com/LeastSquaresFitting.html

 
               # File minitest/benchmark.rb, line 247
def fit_linear xs, ys
  n   = xs.size
  xys = xs.zip(ys)
  sx  = sigma xs
  sy  = sigma ys
  sx2 = sigma(xs)  { |x|   x ** 2 }
  sxy = sigma(xys) { |x,y| x * y  }

  c = n * sx2 - sx**2
  a = (sy * sx2 - sx * sxy) / c
  b = ( n * sxy - sx * sy ) / c

  return a, b, fit_error(xys) { |x| a + b * x }
end

fit_power(xs, ys) click to toggle source

To fit a functional form: y = ax^b.

Takes x and y values and returns [a, b, r^2].

See: mathworld.wolfram.com/LeastSquaresFittingPowerLaw.html

 
               # File minitest/benchmark.rb, line 269
def fit_power xs, ys
  n       = xs.size
  xys     = xs.zip(ys)
  slnxlny = sigma(xys) { |x, y| Math.log(x) * Math.log(y) }
  slnx    = sigma(xs)  { |x   | Math.log(x)               }
  slny    = sigma(ys)  { |   y| Math.log(y)               }
  slnx2   = sigma(xs)  { |x   | Math.log(x) ** 2          }

  b = (n * slnxlny - slnx * slny) / (n * slnx2 - slnx ** 2);
  a = (slny - b * slnx) / n

  return Math.exp(a), b, fit_error(xys) { |x| (Math.exp(a) * (x ** b)) }
end

sigma(enum, &block) click to toggle source

Enumerates over enum mapping block if given, returning the sum of the result. Eg:

sigma([1, 2, 3])                # => 1 + 2 + 3 => 7
sigma([1, 2, 3]) { |n| n ** 2 } # => 1 + 4 + 9 => 14

 
               # File minitest/benchmark.rb, line 290
def sigma enum, &block
  enum = enum.map(&block) if block
  enum.inject { |sum, n| sum + n }
end

validation_for_fit(msg, threshold) click to toggle source

Returns a proc that calls the specified fit method and asserts that the error is within a tolerable threshold.

 
               # File minitest/benchmark.rb, line 299
def validation_for_fit msg, threshold
  proc do |range, times|
    a, b, rr = send "fit_#{msg}", range, times
    assert_operator rr, :>=, threshold
    [a, b, rr]
  end
end

Commenting is here to help enhance the documentation. For example, code samples, or clarification of the documentation.

If you have questions about Ruby or the documentation, please post to one of the Ruby mailing lists. You will get better, faster, help that way.

If you wish to post a correction of the docs, please do so, but also file bug report so that it can be corrected for the next release. Thank you.

If you want to help improve the Ruby documentation, please visit Documenting-ruby.org.

Home Classes Methods

In Files

Parent

Methods

Class/Module Index

MiniTest::Unit::TestCase

Public Class Methods

Public Instance Methods