We are looking into using the Exafusion feature on an Exadata X5 full rack and will start to test in non-production first.
This is well documented in MOS Oracle support note: EXAFUSION - FAQ (Doc ID 2037900.1)
Exafusion
is Direct-to-Wire protocol allows database processes to read and send
Oracle Real Applications Cluster (Oracle RAC) messages directly over the
Infiniband network bypassing the overhead of entering the OS kernel,
and running the normal networking software stack. This improves the
response time and scalability of the Oracle RAC environment on Oracle
Exadata Database Machine. Data is transferred directly from user space
to the Infiniband network, leading to reduced CPU utilization and better
scale-out performance. Exafusion is especially useful for OLTP
applications because per message overhead is particularly apparent in
small OLTP messages.
Exafusion helps small messages by bypassing OS network layer overhead and is ideal for this purpose.
Requirements to use Exafusion -
Oracle Exadata Storage Server Software release 12.1.2.1.1 or later
Oracle Database software Release 12.1.0.2.0 BP6 or later
Mellanox ConnectX-2 and ConnectX-3 Host Channel Adapters (HCAs) are required.
Oracle Enterprise Kernel 2 Quarterly Update 5 (UEK2QU5) kernels (2.6.39-400.2nn) or later are required.
Enabling Exafusion -
Exafusion is disabled by default.
To enable Exafusion, set the EXAFUSION_ENABLED initialization parameter to 1.
To disable Exafusion, set the EXAFUSION_ENABLED initialization parameter to 0.
This parameter cannot be set dynamically. It must be set before instance startup.
All of the instances in an Oracle RAC cluster must enable this
parameter, or all of the instances in an Oracle RAC cluster must disable
the parameter.
