NumExpr with Intel MKL
======================

Numexpr has support for Intel's VML (included in Intel's MKL) in order to 
accelerate the evaluation of transcendental functions on Intel CPUs.  Here it 
is a small example on the kind of improvement you may get by using it.

A first benchmark
-----------------

Firstly, we are going to exercise how MKL performs when computing a couple of 
simple expressions.  One is a pure algebraic one: :code:`2*y + 4*x` and the other 
contains transcendental functions: :code:`sin(x)**2 + cos(y)**2`.

For this, we are going to use this worksheet_.  I (Francesc Alted) ran this 
benchmark on a Intel Xeon E3-1245 v5 @ 3.50GHz. Here are the results when 
not using MKL::

    NumPy version: 1.11.1
    Time for an algebraic expression:     0.168 s / 6.641 GB/s
    Time for a transcendental expression: 1.945 s / 0.575 GB/s
    Numexpr version: 2.6.1. Using MKL: False
    Time for an algebraic expression:     0.058 s / 19.116 GB/s
    Time for a transcendental expression: 0.283 s / 3.950 GB/s
 

And now, using MKL::

    NumPy version: 1.11.1
    Time for an algebraic expression:     0.169 s / 6.606 GB/s
    Time for a transcendental expression: 1.943 s / 0.575 GB/s
    Numexpr version: 2.6.1. Using MKL: True
    Time for an algebraic expression:     0.058 s / 19.153 GB/s
    Time for a transcendental expression: 0.075 s / 14.975 GB/s


As you can see, numexpr using MKL can be up to 3.8x faster for the case of the 
transcendental expression.  Also, you can notice that the pure algebraic 
expression is not accelerated at all.  This is completely expected, as the 
MKL is offering accelerations for CPU bounded functions (sin, cos, tan, exp, 
log, sinh...) and not pure multiplications or adds.

Finally, note how numexpr+MKL can be up to 26x faster than using a pure NumPy 
solution.  And this was using a processor with just four physical cores; you 
should expect more speedup as you throw more cores at that.

.. _worksheet: https://github.com/pydata/numexpr/blob/master/bench/vml_timing2.py

More benchmarks (older)
-----------------------

Numexpr & VML can both use several threads for doing computations.  Let's see 
how performance improves by using 1 or 2 threads on a 2-core Intel CPU (Core2 
E8400 @ 3.00GHz).

Using 1 thread
^^^^^^^^^^^^^^

Here we have some benchmarks on the improvement of speed that Intel's VML can 
achieve.  First, look at times by some easy expression containing sine and 
cosine operations *without* using VML::

    In [17]: ne.use_vml
    Out[17]: False
    
    In [18]: x = np.linspace(-1, 1, 1e6)
    
    In [19]: timeit np.sin(x)**2+np.cos(x)**2
    10 loops, best of 3: 43.1 ms per loop
    
    In [20]: ne.set_num_threads(1)
    Out[20]: 2
    
    In [21]: timeit ne.evaluate('sin(x)**2+cos(x)**2')
    10 loops, best of 3: 29.5 ms per loop


and now using VML::

    In [37]: ne.use_vml
    Out[37]: True
    
    In [38]: x = np.linspace(-1, 1, 1e6)
    
    In [39]: timeit np.sin(x)**2+np.cos(x)**2
    10 loops, best of 3: 42.8 ms per loop
    
    In [40]: ne.set_num_threads(1)
    Out[40]: 2
    
    In [41]: timeit ne.evaluate('sin(x)**2+cos(x)**2')
    100 loops, best of 3: 19.8 ms per loop

Hey, VML can accelerate computations by a 50% using a single CPU.  That's great!

Using 2 threads
^^^^^^^^^^^^^^^

First, look at the time of the non-VML numexpr when using 2 threads:: 

    In [22]: ne.set_num_threads(2)
    Out[22]: 1
    
    In [23]: timeit ne.evaluate('sin(x)**2+cos(x)**2')
    100 loops, best of 3: 15.3 ms per loop

OK.  We've got an almost perfect 2x improvement in speed with regard to the 1 
thread case.  Let's see about the VML-powered numexpr version::

    In [43]: ne.set_num_threads(2)
    Out[43]: 1
    
    In [44]: timeit ne.evaluate('sin(x)**2+cos(x)**2')
    100 loops, best of 3: 12.2 ms per loop

Ok, that's about 1.6x improvement over the 1 thread VML computation, and 
still a 25% of improvement over the non-VML version.  Good, native numexpr 
multithreading code really looks very efficient!

Numexpr native threading code vs VML's one
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

You may already know that both numexpr and Intel's VML do have support for 
multithreaded computations, but you might be curious about which one is more 
efficient, so here it goes a hint.  First, using the VML multithreaded 
implementation::

    In [49]: ne.set_vml_num_threads(2)
    
    In [50]: ne.set_num_threads(1)
    Out[50]: 1

    In [51]: ne.set_vml_num_threads(2)

    In [52]: timeit ne.evaluate('sin(x)**2+cos(x)**2')
    100 loops, best of 3: 16.8 ms per loop

and now, using the native numexpr threading code::

    In [53]: ne.set_num_threads(2)
    Out[53]: 1

    In [54]: ne.set_vml_num_threads(1)

    In [55]: timeit ne.evaluate('sin(x)**2+cos(x)**2')
    100 loops, best of 3: 12 ms per loop


This means that numexpr's native multithreaded code is about 40% faster than 
VML's for this case.  So, in general, you should use the former with numexpr 
(and this is the default actually).

Mixing numexpr's and VML multithreading capabilities
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Finally, you might be tempted to use both multithreading codes at the same 
time, but you will be deceived about the improvement in performance::

    In [57]: ne.set_vml_num_threads(2)

    In [58]: timeit ne.evaluate('sin(x)**2+cos(x)**2')
    100 loops, best of 3: 17.7 ms per loop

Your code actually performs much worse.  That's normal too because you are 
trying to run 4 threads on a 2-core CPU.  For CPUs with many cores, you may 
want to try with different threading configurations, but as a rule of thumb, 
numexpr's one will generally win.