TheLazy.Dev
Portfolio GitHub

Blog

ruby |javascript |kinx |benchmark

Mandelbrot benchmark

krayg

Kray-G

Posted on August 19, 2020

Mandelbrot benchmark

Mandelbrot benchmark

  • Target ... C, PHP, HHVM, Ruby, Python, PyPy, and our Kinx

Introduction

I heard PHP8 would support JIT.

Oh, it sounds great! That is also a good situation for our Kinx to show the ability of the performance by native keyword in Kinx.

Posted this article means that the result was very good. It is rather over my expectation!

Note that please see the article here if you do not know Kinx. I will be very happy if you are interested in it.

Before doing it

Benchmark

Look at here（https://gist.github.com/dstogov/12323ad13d3240aee8f1）, and you will find some benchmarks. So I made some stuffs based on those.

Note that the environment is quite different. That's why I benchmarked it all. And also I choosen the near version because I can't setup it with exactly same versions, and compared with original one.

But as an original comment said, only PHP did cheat. In fact, it is unnecessary to output the result and see an I/O overhead, so I removed it.

How to measure the time

It is being used a timer which has been prepared by the language. And, I found the compile time with source code analysis is not included in the measurement. Although it can't be helped about C.

The reason why I have found it is that I felt something strange about HHVM. When it is HHVM, the result time on display is very fast but my actual feeling is different. The result is faster than PHP, but the real time of HHVM is slower than PHP as an actual elapsed time.

Output

I confirmed that all codes were displaying the following output. Our Kinx was also working as we expected.

                                       *
                                       *
                                       *
                                       *
                                       *
                                      ***
                                     *****
                                     *****
                                      ***
                                       *
                                   *********
                                 *************
                                ***************
                             *********************
                             *********************
                              *******************
                              *******************
                              *******************
                              *******************
                            ***********************
                              *******************
                              *******************
                             *********************
                              *******************
                              *******************
                               *****************
                                ***************
                                 *************
                                   *********
                                       *
                                ***************
                            ***********************
                         * ************************* *
                         *****************************
                      * ******************************* *
                       *********************************
                      ***********************************
                    ***************************************
               *** ***************************************** ***
               *************************************************
                ***********************************************
                 *********************************************
                 *********************************************
                ***********************************************
                ***********************************************
              ***************************************************
               *************************************************
               *************************************************
              ***************************************************
              ***************************************************
         *    ***************************************************    *
       *****  ***************************************************  *****
       ****** *************************************************** ******
      ******* *************************************************** *******
    ***********************************************************************
    ********* *************************************************** *********
       ****** *************************************************** ******
       *****  ***************************************************  *****
              ***************************************************
              ***************************************************
              ***************************************************
              ***************************************************
               *************************************************
               *************************************************
              ***************************************************
                ***********************************************
                ***********************************************
                  *******************************************
                   *****************************************
                 *********************************************
                **** ****************** ****************** ****
                 ***  ****************   ****************  ***
                  *    **************     **************    *
                         ***********       ***********
                         **  *****           *****  **
                          *   *                 *   *
Enter fullscreen mode Exit fullscreen mode

Benchmark it

It is time to benchmark. First of all, let me introduce source codes.

C

Here is a version of gcc.

$ gcc --version
gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Enter fullscreen mode Exit fullscreen mode

C code is like this.

#include <stdio.h>
#include <sys/time.h>

#define BAILOUT 16
#define MAX_ITERATIONS 1000

int mandelbrot(double x, double y)
{
        double cr = y - 0.5;
        double ci = x;
        double zi = 0.0;
        double zr = 0.0;
        int i = 0;

        while(1) {
                i ++;
                double temp = zr * zi;
                double zr2 = zr * zr;
                double zi2 = zi * zi;
                zr = zr2 - zi2 + cr;
                zi = temp + temp + ci;
                if (zi2 + zr2 > BAILOUT)
                        return i;
                if (i > MAX_ITERATIONS)
                        return 0;
        }

}

int main (int argc, const char * argv[]) {
        struct timeval aTv;
        gettimeofday(&aTv, NULL);
        long init_time = aTv.tv_sec;
        long init_usec = aTv.tv_usec;

        int x,y;
        for (y = -39; y < 39; y++) {
                //printf("\n");
                for (x = -39; x < 39; x++) {
                        volatile int i = mandelbrot(x/40.0, y/40.0);
                        //if (i==0)
                        //      printf("*");
                        //else
                        //      printf(" ");
                }
        }
        //printf ("\n");

        gettimeofday(&aTv,NULL);
        double query_time = (aTv.tv_sec - init_time) + (double)(aTv.tv_usec - init_usec)/1000000.0;
        printf ("C Elapsed %0.3f\n", query_time);
    return 0;
}
Enter fullscreen mode Exit fullscreen mode

PHP/HHVM

It is the version of PHP.

$ php --version
PHP 7.2.24-0ubuntu0.18.04.6 (cli) (built: May 26 2020 13:09:11) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.2.24-0ubuntu0.18.04.6, Copyright (c) 1999-2018, by Zend Technologies
Enter fullscreen mode Exit fullscreen mode

It is the version of HHVM.

$ hhvm --version
HipHop VM 3.21.0 (rel)
Compiler: 3.21.0+dfsg-2ubuntu2
Repo schema: ebd0a4633a34187463466c1d3bd327c131251849
Enter fullscreen mode Exit fullscreen mode

There is no differences of the source code between PHP and HHVM.

<?php
define("BAILOUT",16);
define("MAX_ITERATIONS",1000);

class Mandelbrot
{
    function Mandelbrot()
    {
        $d1 = microtime(1);
        for ($y = -39; $y < 39; $y++) {
            for ($x = -39; $x < 39; $x++) {
                $this->iterate($x/40.0,$y/40.0);
            }
        }
        $d2 = microtime(1);
        $diff = $d2 - $d1;
        printf("PHP Elapsed %0.3f\n", $diff);
    }

    function iterate($x,$y)
    {
        $cr = $y-0.5;
        $ci = $x;
        $zr = 0.0;
        $zi = 0.0;
        $i = 0;
        while (true) {
            $i++;
            $temp = $zr * $zi;
            $zr2 = $zr * $zr;
            $zi2 = $zi * $zi;
            $zr = $zr2 - $zi2 + $cr;
            $zi = $temp + $temp + $ci;
            if ($zi2 + $zr2 > BAILOUT)
                return $i;
            if ($i > MAX_ITERATIONS)
                return 0;
        }

    }


}

$m = new Mandelbrot();
?>
Enter fullscreen mode Exit fullscreen mode

Ruby

It is the version of Ruby.

$ ruby --version
ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]
Enter fullscreen mode Exit fullscreen mode

Here is the Ruby's source code.

BAILOUT = 16
MAX_ITERATIONS = 1000

class Mandelbrot

        def initialize
                #puts "Rendering"
                for y in -39...39 do
                        #puts
                        for x in -39...39 do
                                i = iterate(x/40.0,y/40.0)
                                #if (i == 0)
                                #       print "*"
                                #else
                                #       print " "
                                #end
                        end
                end
        end

        def iterate(x,y)
                cr = y-0.5
                ci = x
                zi = 0.0
                zr = 0.0
                i = 0

                while(1)
                        i += 1
                        temp = zr * zi
                        zr2 = zr * zr
                        zi2 = zi * zi
                        zr = zr2 - zi2 + cr
                        zi = temp + temp + ci
                        return i if (zi2 + zr2 > BAILOUT)
                        return 0 if (i > MAX_ITERATIONS)
                end

        end

end

time = Time.now
Mandelbrot.new
#puts
puts "Ruby Elapsed %f" % (Time.now - time)
Enter fullscreen mode Exit fullscreen mode

Python/PyPy

It is the version of Python.

$ python --version
Python 2.7.15+
Enter fullscreen mode Exit fullscreen mode

PyPy's version.

$ pypy --version
Python 2.7.13 (5.10.0+dfsg-3build2, Feb 06 2018, 18:37:50)
[PyPy 5.10.0 with GCC 7.3.0]
Enter fullscreen mode Exit fullscreen mode

Here is the Python's source code. The source code for PyPy is the same.

import sys, time
stdout = sys.stdout

BAILOUT = 16
MAX_ITERATIONS = 1000

class Iterator:
  def __init__(self):
    #print 'Rendering...'
    for y in range(-39, 39):
      #stdout.write('\n')
      for x in range(-39, 39):
        i = self.mandelbrot(x/40.0, y/40.0)

        #if i == 0:
          #stdout.write('*')
        #else:
          #stdout.write(' ')

  def mandelbrot(self, x, y):
    cr = y - 0.5
    ci = x
    zi = 0.0
    zr = 0.0
    i = 0

    while True:
      i += 1
      temp = zr * zi
      zr2 = zr * zr
      zi2 = zi * zi
      zr = zr2 - zi2 + cr
      zi = temp + temp + ci

      if zi2 + zr2 > BAILOUT:
        return i
      if i > MAX_ITERATIONS:
        return 0

t = time.time()
Iterator()
print 'Python Elapsed %.02f' % (time.time() - t)
Enter fullscreen mode Exit fullscreen mode

Kinx/Kinx(native)

Here is the version of Kinx.

$ kinx -v
kinx version 0.9.2
Enter fullscreen mode Exit fullscreen mode

It is the source code of normal Kinx.

const BAILOUT = 16;
const MAX_ITERATIONS = 1000;

function mandelbrot(x, y) {
    var cr = y - 0.5;
    var ci = x;
    var zi = 0.0;
    var zr = 0.0;
    var i = 0;

    while (true) {
        i++;
        var temp = zr * zi;
        var zr2 = zr * zr;
        var zi2 = zi * zi;
        zr = zr2 - zi2 + cr;
        zi = temp + temp + ci;
        if (zi2 + zr2 > BAILOUT)
            return i;
        if (i > MAX_ITERATIONS)
            return 0;
    }
}


var tmr = new SystemTimer();
var x,y;
for (y = -39; y < 39; y++) {
    #System.print("\n");
    for (x = -39; x < 39; x++) {
        var i = mandelbrot(x/40.0, y/40.0);
        #if (i==0)
        #    System.print("*");
        #else
        #    System.print(" ");
    }
}
#System.print("\n");
System.print("Kinx Elapsed %0.3f\n" % tmr.elapsed());
Enter fullscreen mode Exit fullscreen mode

Here is the source code of Kinx with native. The type is not necessary when it can be assumed from the result of expectation, so that was okay only with adding :dbl to the argument.

const BAILOUT = 16;
const MAX_ITERATIONS = 1000;

native mandelbrot(x:dbl, y:dbl) {
    var cr = y - 0.5;
    var ci = x;
    var zi = 0.0;
    var zr = 0.0;
    var i = 0;

    while (true) {
        i++;
        var temp = zr * zi;
        var zr2 = zr * zr;
        var zi2 = zi * zi;
        zr = zr2 - zi2 + cr;
        zi = temp + temp + ci;
        if (zi2 + zr2 > BAILOUT)
            return i;
        if (i > MAX_ITERATIONS)
            return 0;
    }
}


var tmr = new SystemTimer();
var x,y;
for (y = -39; y < 39; y++) {
    #System.print("\n");
    for (x = -39; x < 39; x++) {
        var i = mandelbrot(x/40.0, y/40.0);
        #if (i==0)
        #    System.print("*");
        #else
        #    System.print(" ");
    }
}
#System.print("\n");
System.print("Kinx(native) Elapsed %0.3f\n" % tmr.elapsed());
Enter fullscreen mode Exit fullscreen mode

Result

Here is the result. It is an average of 10 times. The order is the faster one is upside. 'real' is the result of time command.

language version time(sec) time(real)
C 7.4.0 0.018 0.046
PyPy 5.10.0 0.020 0.122
Kinx(native) 0.9.2 0.048 0.107
HHVM 3.21.0 0.068 0.552
PHP 7.2.24 0.182 0.241
Ruby 2.5.1 0.365 0.492
Kinx 0.9.2 0.393 0.457
Python 2.7.15 0.564 0.601

Good! Kinx(native) is faster than HHVM. And I am happy because Kinx normal is almost same as Ruby VM which I feel very fast.

By the way, PyPy is too fast, but the time of real is almost same. I guess the difference is optimization.

The result shows HHVM is slower than PHP in the real of time command. It was because the compilation time would be long. It can not be helped because it is a spec of language. The same penalty of compilation is shown also in Kinx native, although a little bit.

Okay, let's compare with the result in the orginal article. In this benchmark, the differences between environments seem to be a big factor. Only for HHVM, it is strange... but I don't know why. In other cases, it is around 2x faster on my environment.

language version time(sec) Original result Original version
C 7.4.0 0.018 0.022 4.9.2
PyPy 5.10.0 0.020
Kinx(native) 0.9.2 0.048
HHVM 3.21.0 0.068 0.030 3.5.0
PHP 7.2.24 0.182 0.281 7
Ruby 2.5.1 0.365 0.684 2.1.5
Kinx 0.9.2 0.393
Python 2.7.15 0.564 1.128 2.7.8

Conclusion

Benchmarking is very fun, when it is a good result. I couldn't touch native these days, but it is one of characteristic of Kinx, so I want to grow it.

See you next time.

By the way, here is the script to measure it. I used Process implemented recently. I used a result shown at average.

using Process;

var count = 10;
var command = [$$[1], $$[2]];
var r = [];
var re = /[0-9]+\.[0-9]+/;

for (var i = 0; i < count; ++i) {
    var result = "";
    var [r1, w1] = new Pipe();
    var p1 = new Process(command, { out: w1 }).run();
    w1.close();
    while (p1.isAlive() || r1.peek() > 0) {
        var buf = r1.read();
        if (buf.length() < 0) {
            System.println("Error...");
            return 1;
        } else if (buf.length() > 0) {
            result += buf;
        } else {
            // System.println("no input...");
        }
    }

    re.reset(result);
    if (re.find()) {
        r.push(Double.parseDouble(re.group[0].string));
    }
}
var total = r.reduce(&(r, e) => r + e);
System.println("total  : %8.3f" % total);
System.println("count  : %8d" % r.length());
System.println("average: %8.3f" % (total / r.length()));
Enter fullscreen mode Exit fullscreen mode

Thank you!

💖 💪 🙅 🚩
krayg
Kray-G

Posted on August 19, 2020

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Mandelbrot benchmark
ruby Mandelbrot benchmark

August 19, 2020