Galera Parameters

As of version 0.8, Galera Cluster accepts parameters as semicolon-separated key value pair lists, such as key1 = value1; key2 = value2. In this way, you can configure an arbitrary number of Galera Cluster parameters in one call. A key consists of parameter group and parameter name: <group>.<name>, where <group> corresponds roughly to some Galera module.

All wsrep_provider_options settings need to be specified on a single line. In case of multiple instances of wsrep_provider_options, only the last one is used.

../_images/support.jpg

Below is a list of all of the Galera parameters. Each is also a link to further down the page where you may find more information. There are a few things to know about this table:

  • For numeric values related to memory size, Galera Cluster accepts the numeric modifiers, K, M, G, and T to represent 210, 220, 230 and 240, respectively.
  • Galera Cluster accepts the following boolean values: 0, 1, YES, NO, TRUE, FALSE, ON, OFF.
  • Time periods must be expressed in the ISO8601 format. See some of the examples below.
  • The parameters that are noted as for debugging only are strictly for use in troubleshooting problems. You should not implement these in production environments.

Parameter

Default

Dynamic
Debug
Only
Initial
Version
Version
Deprecated
base_dir          
base_host detected network address     1.0  
base_port 4567     1.0  
cert.log_conflicts NO
Yes
  2.0  
cert.optimistic_pa YES
Yes
  3.25  
debug NO
Yes
  2.0  
evs.auto_evict 0
No
  3.8  
evs.causal_keepalive_period 0
No
  1.0  
evs.consensus_timeout PT30S
No
Yes 1.0, 2.0  
evs.debug_log_mask 0x1
Yes
  1.0  
evs.delayed_keep_period PT30S
No
  3.8  
evs.delay_margin PT1S
No
  3.8  
evs.evict  
No
  3.8  
evs.inactive_check_period PT1S
No
  1.0  
evs.inactive_timeout PT15S
No
  1.0  
evs.info_log_mask 0
No
  1.0  
evs.install_timeout PT7.5S
Yes
  1.0  
evs.join_retrans_period PT1S
Yes
  1.0  
evs.keepalive_period PT1S
No
  1.0  
evs.max_install_timeouts 1
No
  1.0  
evs.send_window 4
Yes
  1.0  
evs.stats_report_period PT1M
No
  1.0  
evs.suspect_timeout PT5S
No
  1.0  
evs.use_aggregate TRUE
No
  1.0  
evs.user_send_window 2
Yes
  1.0  
evs.view_forget_timeout PT5M
No
  1.0  
evs.version 0
No
Yes 1.0  
gcache.dir working directory
No
  1.0  
gcache.name galera.cache
No
  1.0  
gcache.keep_pages_size 0
No
  1.0  
gcache.mem_size 0
No
     
gcache.page_size 128M
No
  1.0  
gcache.recover no
No
  3.19  
gcache.size 128M
No
  1.0  
gcomm.thread_prio  
No
  3.0  
gcs.fc_debug 0
No
  1.0  
gcs.fc_factor 1.0
No
  1.0  
gcs.fc_limit 16
Yes
  1.0  
gcs.fc_master_slave NO
No
  1.0 4.10
gcs.fc_single_primary NO
No
  4.10  
gcs.max_packet_size 32616
No
  1.0  
gcs.max_throttle 0.25
No
  1.0  
gcs.recv_q_hard_limit LLONG_MAX
No
  1.0  
gcs.recv_q_soft_limit 0.25
No
  1.0  
gcs.sync_donor NO
No
  1.0  
gcs.vote_policy 0
No
  1.0  
gmcast.listen_addr tcp://0.0.0.0:4567
No
  1.0  
gmcast.mcast_addr  
No
  1.0  
gmcast.mcast_ttl 1
No
  1.0  
gmcast.peer_timeout PT3S
No
  1.0  
gmcast.segment 0
No
  3.0  
gmcast.time_wait PT5S
No
  1.0  
gmcast.version n/a
No
Yes 1.0  
ist.recv_addr  
No
  1.0  
ist.recv_bind  
No
  3.0  
pc.announce_timeout PT3S
No
  2.0  
pc.bootstrap n/a
No
  2.0  
pc.checksum FALSE
No
  1.0  
pc.ignore_sb FALSE
Yes
  1.0  
pc.ignore_quorum FALSE
Yes
  1.0  
pc.linger PT2S
No
  1.0  
pc.npvo FALSE
No
  1.0  
pc.recovery TRUE
No
  3.0  
pc.version n/a
No
Yes 1.0  
pc.wait_prim TRUE
No
  1.0  
pc.wait_prim_timeout PT30S
No
  2.0  
pc.weight 1
Yes
  2.4  
protonet.backend asio
No
  1.0 4.14
protonet.version n/a
No
Yes 1.0 4.14
repl.causal_read_timeout PT30S
No
  1.0  
repl.commit_order 3
No
  1.0  
repl.key_format FLAT8
No
  3.0  
repl.max_ws_size 2147483647
No
  3.0  
repl.proto_max 5
No
  2.0  
socket.recv_buf_size auto
Yes
  3.17  
socket.send_buf_size auto
Yes
  3.29  
socket.ssl 0
No
     
socket.ssl_ca  
No
  1.0  
socket.ssl_cert  
No
  1.0  
socket.checksum 1 (vs. 2); 2 (vs. 3)
No
  2.0  
socket.dynamic FALSE
No
  4.8  
socket.ssl_cipher  
No
  1.0  
socket.ssl_compression YES
No
  1.0 4.14
socket.ssl_key  
No
  1.0  
socket.ssl_password_file  
No
  1.0  
socket.ssl_reload  
No
  4.8  

base_dir

Specifies the data directory.

base_host

Global variable for internal use.

Default Value detected network address
Dynamic  
Initial Version ???

Warning

Since this is for internal use only, don’t manually set the base_host variable.

base_port

Global variable for internal use.

Default Value 4567
Dynamic  
Initial Version ???

Warning

Since this is for internal use only, don’t manually set the base_port variable.

cert.log_conflicts

Log details of certification failures.

Default Value NO
Dynamic Yes
Initial Version 2.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="cert.log_conflicts=NO"

cert.optimistic_pa

Controls parallel applying of slave actions. When enabled allows full range of parallelization as determined by certification algorithm. When disabled limits parallel applying window to not exceed that seen on master. In other words, the action starts applying no sooner than all actions it has seen on the master are committed.

Default Value YES
Dynamic Yes
Initial Version 3.25
wsrep_provider_options="cert.optimistic_pa=NO"

debug

Enable debugging.

Default Value NO
Dynamic Yes
Initial Version 2.0
wsrep_provider_options="debug=NO"

evs.auto_evict

Defines how many entries the node allows for given a delayed node before it triggers the Auto Eviction protocol.

Default Value 0
Dynamic No
Initial Version 3.8

Each cluster node monitors the group communication response times from all other nodes. When the cluster registers delayed response from a given node, it adds an entry for that node to its delayed list. If the majority of the cluster nodes show the node as delayed, the node is permanently evicted from the cluster.

This parameter determines how many entries a given node can receive before it triggers Auto Eviction.

When this parameter is set to 0, it disables the Auto Eviction protocol for this node. Even when you disable Auto Eviction, though; the node continues to monitor response times from the cluster.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.auto_evict=5"

For more information on the Auto Eviction process, see Auto-Eviction.

evs.causal_keepalive_period

For developer use only. Defaults to evs.keepalive_period.

Default Value  
Dynamic No
Initial Version 1.0

evs.consensus_timeout

Timeout on reaching the consensus about cluster membership.

Default Value PT30S
Dynamic No
Initial Version 1.0
Deprecated 2.0

This variable is mostly used for troubleshooting purposes and should not be implemented in a production environment.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.consensus_timeout=PT30S"

Note

This feature has been deprecated. It is succeeded by evs.install_timeout.

evs.debug_log_mask

Control EVS debug logging, only effective when wsrep_debug is in use.

Default Value 0x1
Dynamic Yes
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.debug_log_mask=0x1"

evs.delayed_keep_period

Defines how long this node requires a delayed node to remain responsive before it removes an entry from the delayed list.

Default Value PT30S
Dynamic No
Initial Version 3.8

Each cluster node monitors the group communication response times from all other nodes. When the cluster registered delayed responses from a given node, it adds an entry for that node to its delayed list. Nodes that remain on the delayed list can trigger Auto Eviction, which removes them permanently from the cluster.

This parameter determines how long a node on the delayed list must remain responsive before it removes one entry. The number of entries on the delayed list and how long it takes before the node removes all entries depends on how long the delayed node was unresponsive.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.delayed_keep_period=PT45S"

For more information on the delayed list and the Auto Eviction process, see Auto-Eviction.

evs.delay_margin

Defines how long the node allows response times to deviate before adding an entry to the delayed list.

Default Value PT1S
Dynamic No
Initial Version 3.8

Each cluster node monitors group communication response times from all other nodes. When the cluster registers a delayed response from a given node, it adds an entry for that node to its delayed list. Delayed nodes can trigger Auto Eviction, which removes them permanently from the cluster.

This parameter determines how long a delay can run before the node adds an entry to the delayed list. You must set this parameter to a value higher than the round-trip delay time (RTT) between the nodes.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.delay_margin=PT5S"

For more information on the delayed list and the Auto Eviction process, see Auto-Eviction.

evs.evict

If set to the gcomm UUID of some node, that node will be evicted from the cluster. Setting this parameter to an empty string causes the eviction list to be cleared on the node where it is set.

Default Value  
Dynamic No
Initial Version 3.8

For more information on the eviction and Auto Eviction process, see Auto-Eviction.

evs.inactive_check_period

Defines how often you want the node to check for peer inactivity.

Default Value PT1S
Dynamic No
Initial Version 1.0

Each cluster node monitors group communication response times from all other nodes. When the cluster registers a delayed response from a given node, it adds an entry for that node to its delayed list, which can lead to the delayed node’s eviction from the cluster.

This parameter determines how often you want the node to check for delays in the group communication responses from other cluster nodes.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.inactive_check_period=PT1S"

evs.inactive_timeout

Defines a hard limit on node inactivity.

Default Value PT15S
Dynamic No
Initial Version 1.0

Hard limit on the inactivity period, after which the node is pronounced dead.

Each cluster node monitors group communication response times from all other nodes. When the cluster registers a delayed response from a given node, it add an entry for that node to its delayed list, which can lead to the delayed node’s eviction from the cluster.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.inactive_timeout=PT15S"

This parameter sets a hard limit for node inactivity. If a delayed node remains unresponsive for longer than this period, the node pronounces the delayed node as dead.

evs.info_log_mask

Defines additional logging options for the EVS Protocol.

Default Value 0
Dynamic No
Initial Version 1.0

The EVS Protocol monitors group communication response times and controls the node eviction and auto eviction processes. This parameter allows you to enable additional logging options, through a bitmask value.

  • 0x1 Provides extra view change info.
  • 0x2 Provides extra state change info
  • 0x4 Provides statistics
  • 0x8 Provides profiling (only in builds with profiling enabled)

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.info_log_mask=0x4"

evs.install_timeout

Defines the timeout for install message acknowledgments.

Default Value PT7.5S
Dynamic Yes
Initial Version 1.0

Each cluster node monitors group communication response times from all other nodes, checking whether they are responsive or delayed. This parameter determines how long you want the node to wait on install message acknowledgments.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.install_timeout=PT7.5S"

Note

This parameter replaces evs.consensus_timeout.

evs.join_retrans_period

Defines how often the node retransmits EVS join messages when forming cluster membership.

Default Value PT1S
Dynamic Yes
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.join_retrans_period=PT1S"

evs.keepalive_period

Defines how often the node emits keepalive signals.

Default Value PT1S
Dynamic No
Initial Version 1.0

Each cluster node monitors group communication response times from all other nodes. When there is no traffic going out for the cluster to monitor, nodes emit keepalive signals so that other nodes have something to measure. This parameter determines how often the node emits a keepalive signal, absent any other traffic.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.keepalive_period=PT1S"

evs.max_install_timeouts

Defines the number of membership install rounds to try before giving up.

Default Value 1
Dynamic No
Initial Version 1.0

This parameter determines the maximum number of times that the node tries for a membership install acknowledgment, before it stops trying. The total number of rounds it tries is this value plus 2.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.max_install_timeouts=1"

evs.send_window

Defines the maximum number of packets at a time in replication.

Default Value 4
Dynamic Yes
Initial Version 1.0

This parameter determines the maximum number of packets the node uses at a time in replication. For clusters implemented over WAN, you can set this value considerably higher, (for example, 512), than for clusters implemented over LAN.

You must use a value that is greater than evs.user_send_window. The recommended value is double evs.user_send_window.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.send_window=4"

evs.stats_report_period

Control period of EVS statistics reporting. The node is pronounced dead.

Default Value PT1M
Dynamic No
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.stats_report_period=PT1M"

evs.suspect_timeout

Defines the inactivity period after which a node is suspected as dead.

Default Value PT5S
Dynamic No
Initial Version 1.0

Each node in the cluster monitors group communications from all other nodes in the cluster. This parameter determines the period of inactivity before the node suspects another of being dead. If all nodes agree on that, the cluster drops the inactive node.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.suspect_timeout=PT5S"

evs.use_aggregate

Defines whether the node aggregates small packets into one when possible.

Default Value TRUE
Dynamic No
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.use_aggregate=TRUE"

evs.user_send_window

Defines the maximum number of data packets at a time in replication.

Default Value 2
Dynamic Yes
Initial Version 1.0

This parameter determines the maximum number of data packets the node uses at a time in replication. For clusters implemented over WAN, you can set this to a value considerably higher than cluster implementations over LAN, (for example, 512).

You must use a value that is smaller than evs.send_window. The recommended value is half evs.send_window.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.user_send_window=2"

For more information, see evs.send_window.

evs.view_forget_timeout

Defines how long the node saves past views from the view history.

Default Value PT5M
Dynamic No
Initial Version 1.0

Each node maintains a history of past views. This parameter determines how long you want the node to save past views before dropping them from the table.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.view_forget_timeout=PT5M"

evs.version

Defines the EVS Protocol version.

Default Value 0 (up to Galera Cluster version 3.9) | 1 (as of Galera Cluster version 4.0)
Dynamic No
Initial Version 1.0

This parameter determines which version of the EVS Protocol the node uses. In order to ensure backwards compatibility, the parameter defaults to 0 on Galera Cluster versions prior to 3.9. Certain EVS Protocol features, such as Auto Eviction, require you to upgrade to more recent versions. As of Galera Cluster version 4.0, the parameter defaults to 1.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="evs.version=1"

For more information on the procedure to upgrade from one version to another, see Upgrading the EVS Protocol.

gcache.dir

Defines the directory where the write-set cache places its files.

Default Value /path/to/working_dir
Dynamic No
Initial Version 1.0

When nodes receive state transfers they cannot process incoming write-sets until they finish updating their state. Under certain methods, the node that sends the state transfer is similarly blocked. To prevent the database from falling further behind, GCache saves the incoming write-sets on memory mapped files to disk.

This parameter determines where you want the node to save these files for write-set caching. By default, GCache uses the working directory for the database server.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcache.dir=/usr/share/galera"

gcache.keep_pages_size

Total size of the page storage pages to keep for caching purposes. If only page storage is enabled, one page is always present.

Default Value 0
Dynamic No
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcache.keep_pages_size=0"

gcache.mem_size

The maximum size of size of the malloc() store for setups that have spare RAM.

Default Value 0
Dynamic No

gcache.name

Defines the filename for the write-set cache.

Default Value galera.cache
Dynamic No
Initial Version 1.0

When nodes receive state transfers they cannot process incoming write-sets until they finish updating their state. Under certain methods, the node that sends the state transfer is similarly blocked. To prevent the database from falling further behind, GCache saves the incoming write-sets on memory-mapped files to disk.

This parameter determines the name you want the node to use for this ring buffer storage file.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcache.name=galera.cache"

gcache.page_size

Size of the page files in page storage. The limit on overall page storage is the size of the disk. Pages are prefixed by gcache.page.

Default Value 128M
Dynamic No
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcache.page_size=128M"

gcache.recover

Determines whether gcache recovery takes place on node startup. If gcache could be recovered successfully, the node can then provide IST to other joining nodes, which is useful when the whole cluster is being restarted.

Default Value no
Dynamic No
Initial Version 3.19

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcache.recover=yes"

gcache.size

Defines the disk space you want to node to use in caching write-sets.

Default Value 128M
Dynamic No
Initial Version 1.0

When nodes receive state transfers they cannot process incoming write-sets until they finish updating their state. Under certain methods, the node that sends the state transfer is similarly blocked. To prevent the database from falling further behind, GCache saves the incoming write-sets on memory-mapped files to disk.

This parameter defines the amount of disk space you want to allocate for the present ring buffer storage. The node allocates this space when it starts the database server.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcache.size=128M"

For more information on customizing the write-set cache, see the Best Practice Articles.

gcomm.thread_prio

Defines the policy and priority for the gcomm thread.

Default Value  
Dynamic No
Initial Version 3.0

Using this option, you can raise the priority of the gcomm thread to a higher level than it normally uses. You may find this useful in situations where Galera Cluster threads do not receive sufficient CPU time, due to competition with other MySQL threads. In these cases, when the thread scheduler for the operating system does not run the Galera threads frequently enough, timeouts may occur, causing the node to drop from the cluster.

The format for this option is: <policy>:<priority>. The priority value is an integer. The policy value supports the following options:

  • other Designates the default time-sharing scheduling in Linux. They can run until they are blocked by an I/O request or preempted by higher priorities or superior scheduling designations.
  • fifo Designates first-in out scheduling. These threads always immediately preempt any currently running other, batch or idle threads. They can run until they are either blocked by an I/O request or preempted by a FIFO thread of a higher priority.
  • rr Designates round-robin scheduling. These threads always preempt any currently running other, batch or idle threads. The scheduler allows these threads to run for a fixed period of a time. If the thread is still running when this time period is exceeded, they are stopped and moved to the end of the list, allowing another round-robin thread of the same priority to run in their place. They can otherwise continue to run until they are blocked by an I/O request or are preempted by threads of a higher priority.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcomm.thread_prio=rr:2"

gcs.fc_debug

Post debug statistics about replication flow every this number of writesets.

Default Value 0
Dynamic No
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcs.fc_debug=0"

gcs.fc_factor

Resume replication after recv queue drops below this fraction of gcs.fc_limit.

Default Value 0.5
Dynamic Yes
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcs.fc_factor=0.5"

gcs.fc_limit

Pause replication if recv queue exceeds this number of writesets. For master-slave setups this number can be increased considerably.

Default Value 16
Dynamic Yes
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcs.fc_limit=16"

gcs.fc_master_slave

Deprecated as of Galera 4.10 in favor of gcs.fc_single_primary.

gcs.fc_single_primary

Defines whether there is more than one source of replication.

As the number of nodes in the cluster grows, the larger the calculated gcs.fc_limit gets. At the same time, the number of writes from the nodes increases.

When this parameter value is set to NO (multi-primary), the gcs.fc_limit parameter is dynamically modified to give more margin for each node to be a bit further behind applying writes.

The gcs.fc_limit parameter is modified by the square root of the cluster size, that is, in a four-node cluster it is two times higher than the base value. This is done to compensate for the increasing replication rate noise.

Default Value NO
Dynamic No
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcs.fc_single_primary=NO"

gcs.max_packet_size

All writesets exceeding that size will be fragmented.

Default Value 32616
Dynamic No
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcs.max_packet_size=32616"

gcs.max_throttle

How much to throttle replication rate during state transfer (to avoid running out of memory). Set the value to 0.0 if stopping replication is acceptable for completing state transfer.

Default Value 0.25
Dynamic No
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcs.max_throttle=0.25"

gcs.recv_q_hard_limit

Maximum allowed size of recv queue. This should normally be half of (RAM + swap). If this limit is exceeded, Galera Cluster will abort the server.

Default Value LLONG_MAX
Dynamic No
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcs.recv_q_hard_limit=LLONG_MAX"

gcs.recv_q_soft_limit

The fraction of gcs.recv_q_hard_limit after which replication rate will be throttled.

Default Value 0.25
Dynamic No
Initial Version 1.0

The degree of throttling is a linear function of recv queue size and goes from 1.0 (full rate) at gcs.recv_q_soft_limit to gcs.max_throttle at gcs.recv_q_hard_limit Note that full rate, as estimated between 0 and gcs.recv_q_soft_limit is a very imprecise estimate of a regular replication rate.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcs.recv_q_soft_limit=0.25"

gcs.sync_donor

Should the rest of the cluster keep in sync with the donor? YES means that if the donor is blocked by state transfer, the whole cluster is blocked with it.

Default Value NO
Dynamic No
Initial Version 1.0

If you choose to use value YES, it is theoretically possible that the Donor Node cannot keep up with the rest of the cluster due to the extra load from the SST. If the node lags behind, it may send flow control messages stalling the whole cluster. However, you can monitor this using the wsrep_flow_control_paused status variable.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcs.sync_donor=NO"

gcs.vote_policy

When a cluster node fails to apply a writeset, it initiates voting on the outcome. Every node casts a vote, that is, a hash of the error message or 0, if there was no error. If a node votes “wrong”, the node is considered to be inconsistent and it shuts down. gcs.vote_policy decides on how the votes are being counted and how to choose the winner:

  • gcs.vote_policy = 0 - The outcome that has more votes is chosen as the winner. In other words, simple majority wins. In the case of a tie between 0 (success) and non-0 (error) outcomes, 0 (success) is preferred.
  • gcs.vote_policy > 0 - If success gets as many as or more votes that the parameter value defines, it is chosen as the winner, even if in minority. For example, if gcs.vote_policy=1, only the node that successfully committed a transaction would remain primary. Note that if gcs.vote_policy=1, an inconsistent primary may crash all the secondaries.
Default Value 0
Dynamic No
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gcs.vote_policy=0"

gmcast.listen_addr

Address at which Galera Cluster listens to connections from other nodes. By default the port to listen at is taken from the connection address. This setting can be used to overwrite that.

Default Value tcp://0.0.0.0:4567
Dynamic No
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gmcast.listen_addr=tcp://0.0.0.0:4567"

gmcast.mcast_addr

If set, UDP multicast will be used for replication, for example:

Default Value  
Dynamic No
Initial Version 1.0

The value must be the same on all nodes.

If you are planning to build a large cluster, we recommend using UDP.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gmcast.mcast_addr=239.192.0.11"

gmcast.mcast_ttl

Time to live value for multicast packets.

Default Value 1
Dynamic No
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gmcast.mcast_ttl=1"

gmcast.peer_timeout

Connection timeout for inactive connections.

Default Value PT3S
Dynamic No
Initial Version 1.0

GMCast module monitors liveness of the socket connections on a regular basis. Nodes exchange periodically replication and keepalives messages, which are expected to be received more frequently than gmcast.peer_timeout duration. The gmcast.peer_timeout defines the timeout after which an idle socket connection between two nodes is considered inactive and will be closed and reopened again. If a socket connection is closed due to timeout, the relaying protocol is activated as a side effect to keep messages delivered to all members in the cluster.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gmcast.peer_timeout=PT3S"

gmcast.segment

Define which network segment this node is in. Optimisations on communication are performed to minimise the amount of traffic between network segments including writeset relaying and IST and SST donor selection. The gmcast.segment value is an integer from 0 to 255. By default all nodes are placed in the same segment (0).

Default Value 0
Dynamic No
Initial Version 3.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gmcast.segment=0"

gmcast.time_wait

Time to wait until allowing peer declared outside of stable view to reconnect.

Default Value PT5S
Dynamic No
Initial Version 1.0

In case a node leaves or gets partitioned from the cluster, it is kept isolated from the rest of the cluster for a while to avoid distractions to membership protocol operation. Option gmcast.time_wait denotes the time period during which nodes in the primary component refuse connection attempts from nodes not in the primary component in the following way: The nodes which partitioned from the cluster are allowed to reconnect after gmcast.time_wait/2 seconds, the nodes which left the group completely are allowed to reconnect after gmcast.time_wait seconds.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="gmcast.time_wait=PT5S"

gmcast.version

This status variable is used to check which gmcast protocol version is used.

Default Value  
Dynamic No
Initial Version 1.0

This variable is mostly used for troubleshooting purposes and should not be implemented in a production environment.

ist.recv_addr

Address to listen on for Incremental State Transfer. By default this is the <address>:<port+1> from wsrep_node_address.

Default Value  
Dynamic No
Initial Version 2.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="ist.recv_addr=192.168.1.1"

ist.recv_bind

Defines the address that the node binds on for receiving an Incremental State Transfer.

Default Value  
Dynamic No
Initial Version 3.16

This option defines the address to which the node will bind in order to receive Incremental State Transfers. When this option is not set, it takes its value from ist.recv_addr or, in the event that that is also not set, from wsrep_node_address. You may find it useful when the node runs behind a NAT or in similar cases where the public and private addresses differ.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="ist.recv_bind=192.168.1.1"

pc.recovery

When set to TRUE, the node stores the Primary Component state to disk, in the gvwstate.dat file. The Primary Component can then recover automatically when all nodes that were part of the last saved state reestablish communications with each other.

Default Value TRUE
Dynamic No
Initial Version 3.0

This allows for:

  • Automatic recovery from full cluster crashes, such as in the case of a data center power outage.
  • Graceful full cluster restarts without the need for explicitly bootstrapping a new Primary Component.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="pc.recovery=TRUE"

Note

In the event that the wsrep position differs between nodes, recovery also requires a full State Snapshot Transfer.

pc.bootstrap

If you set this value to TRUE is a signal to turn a NON-PRIMARY component into PRIMARY.

Default Value  
Dynamic Yes
Initial Version 2.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="pc.bootstrap=TRUE"

pc.announce_timeout

Cluster joining announcements are sent every \(\frac{1}{2}\) second for this period of time or less if the other nodes are discovered.

Default Value PT3S
Dynamic No
Initial Version 2.0

When a node joins the cluster, it will initially broadcast join messages every half a second to inform other nodes about the joining attempt, but does not handle its own messages to avoid forming membership configuration containing only itself. This initial join message broadcasting will terminate if at least one other node is seen or if the timeout expires, whichever happens first.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="pc.announce_timeout=PT3S"

pc.checksum

Checksum replicated messages.

Default Value FALSE
Dynamic No
Initial Version 1.0

If set to true, a CRC16 checksum is computed and included into replicated messages on primary component protocol level. This checksum is redundant since the checksum does not take into account all protocol layers, and all messages are checksummed with stronger CRC32C algorithm when transferred between nodes.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="pc.checksum=TRUE"

pc.ignore_sb

Should we allow nodes to process updates even in the case of Split Brain? This is a dangerous setting in multi-master setup, but should simplify things in master-slave cluster (especially if only 2 nodes are used).

Default Value FALSE
Dynamic Yes
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="pc.ignore_sb=FALSE"

pc.ignore_quorum

Completely ignore Quorum calculations. For example if the master splits from several slaves it still remains operational. Use with extreme caution even in master-slave setups, as slaves will not automatically reconnect to master in this case.

Default Value FALSE
Dynamic Yes
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="pc.ignore_quorum=FALSE"

pc.linger

The period for which the PC protocol waits for the EVS termination.

Default Value PT2S
Dynamic No
Initial Version 1.0

When a node leaves the cluster, it will wait up to pc.linger duration to get itself removed gracefully from the cluster.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="pc.linger=PT2S"

pc.npvo

Control which primary component is allowed to continue in case of conflicting primary components after cluster partitioning.

Default Value FALSE
Dynamic No
Initial Version 1.0

If the cluster is configured to ignore quorum or split brain, the nodes may continue processing write sets independently after cluster partitioning, which leads to diverged states. When the cluster merges back to the original configuration, the pc.npvo controls which partition is allowed to continue after the merge. If the value is FALSE, the partition with the lowest view identifier (the most immediate successor to the view before partitioning) is allowed to continue. Otherwise the partition with the highest view identifier (the one which has gone through more configuration changes or has a representative with highest node identifier) is allowed to continue.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="pc.npvo=FALSE"

pc.wait_prim

Control whether a joining node is allowed to start in non-primary component or if it should wait for primary component.

Default Value TRUE
Dynamic No
Initial Version 1.0

This variable can be used to control if a node waits for connecting to the primary component when joining to the cluster.

With default value TRUE, the joining node waits for the primary component so that the first event delivered by the group communication system is the primary component view event, or times out with error (see pc.wait_prim_timeout).

With value FALSE, the joining node completes the initialization without waiting for the primary component, and may end up in non-primary component if no other nodes are seen in pc.announce_timeout. This is a useful setting mainly if it is desirable to start the cluster by starting all the nodes at once, and bootstrapping the primary component with pc.bootstrap after all nodes have been started.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="pc.wait_prim=FALSE"

pc.wait_prim_timeout

The timeout for waiting primary component.

Default Value PT30S
Dynamic No
Initial Version 2.0

The timeout after which an error is thrown if a primary view event is not seen when joining a new node into the cluster. This value is effective only if pc.wait_prim is set to TRUE.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="pc.wait_prim_timeout=PT30S"

pc.weight

Node weight for quorum calculation.

Default Value 1
Dynamic Yes
Initial Version 2.4

For detailed description about weighted quorum see https://galeracluster.com/library/documentation/weighted-quorum.html

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="pc.weight=1"

pc.version

This variable is used to control which PC protocol version is used.

Default Value  
Dynamic No
Initial Version 1.0

This variable is mostly used for troubleshooting purposes and should not be implemented in a production environment.

protonet.backend

This parameter is deprecated and will be removed in the future versions.

Which transport backend to use. Currently only ASIO is supported. This variable is deprecated and will be removed in the future versions.

Default Value asio
Dynamic No
Initial Version 1.0
Version Deprecated 4.14

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="protonet.backend=asio"

protonet.version

This parameter is deprecated and will be removed in the future versions

This status variable is used to check which transport backend protocol version is used.

Default Value  
Dynamic No
Initial Version 1.0
Version Deprecated 4.14

This variable is mostly used for troubleshooting purposes and should not be implemented in a production environment.

repl.commit_order

Warning

Do not change the default value 3, as other values may produce inconsistencies or even lead to server crashes.

Whether to allow Out-Of-Order committing (improves parallel applying performance).

Default Value 3
Dynamic No
Initial Version 1.0

Possible settings:

  • 0 or BYPASS All commit order monitoring is switched off (useful for measuring performance penalty).
  • 1 or OOOC Allows out of order committing for all transactions.
  • 2 or LOCAL_OOOC Allows out of order committing only for local transactions.
  • 3 or NO_OOOC No out of order committing is allowed (strict total order committing)

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="repl.commit_order=2"

repl.causal_read_timeout

The default timeout for causal read and sync wait operations.

Default Value PT30S
Dynamic No
Initial Version 1.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="repl.causal_read_timeout=PT30S"

repl.key_format

The hash size to use for key formats (in bytes). An A suffix annotates the version.

Default Value FLAT8
Dynamic No
Initial Version 3.0

Possible settings:

  • FLAT8
  • FLAT8A
  • FLAT16
  • FLAT16A

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="repl.key_format=FLAT8"

repl.max_ws_size

The maximum size of a write-set in bytes. This is limited to 2G.

Default Value 2147483647
Dynamic No
Initial Version 3.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="repl.max_ws_size=2147483647"

repl.proto_max

The maximum protocol version in replication. Changes to this parameter will only take effect after a provider restart.

Default Value 5
Dynamic No
Initial Version 2.0

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="repl.proto_max=5"

socket.recv_buf_size

The size of the receive buffer that used on the network sockets between nodes. Galera passes the value to the kernel via the SO_RCVBUF socket option. The value is either numeric value in bytes or auto which allows the kernel to autotune the receive buffer. The default was changed from 212992 to auto in 3.29.

Default Value auto
Dynamic No
Initial Version 3.17

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="socket.recv_buf_size=212992"

socket.send_buf_size

The size of the send buffer that used on the network sockets between nodes. Galera passes the value to the kernel via the SO_SNDBUF socket option. The value is either numeric value in bytes or auto which allows the kernel to autotune the send buffer.

Default Value auto
Dynamic No
Initial Version 3.29

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="socket.send_buf_size=212992"

socket.ssl

Explicitly enables TLS usage by the wsrep provider.

Default Value No
Dynamic No

socket.ssl_ca

Defines the path to the SSL Certificate Authority (CA) file.

Default Value  
Dynamic No
Initial Version 1.0

The node uses the CA file to verify the signature on the certificate. You can use either an absolute path or one relative to the working directory. The file must use PEM format.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options='socket.ssl_ca=/path/to/ca-cert.pem'

For more information on generating SSL certificate files for your cluster, see SSL Certificates.

socket.ssl_cert

Defines the path to the SSL certificate.

Default Value  
Dynamic No
Initial Version 1.0

The node uses the certificate as a self-signed public key in encrypting replication traffic over SSL. You can use either an absolute path or one relative to the working directory. The file must use PEM format.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="socket.ssl_cert=/path/to/server-cert.pem"

For more information on generating SSL certificate files for your cluster, see SSL Certificates.

socket.checksum

Checksum to use on socket layer.

Default Value 1 (before vs. 3), 2
Dynamic No
Initial Version 2.0

Possible Values;

  • 0 - disable checksum
  • 1 - CRC32
  • 2 - CRC-32C (optimized and potentially HW-accelerated on Intel CPUs)

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="socket.checksum=2"

socket.dynamic

Enable connection engine to accept both SSL and TCP connections.

Default Value false
Dynamic No
Initial Version 4.8

By enabling this parameter, it should be possible for Galera to communicate with both SSL and TCP connections. If SSL is enabled it will try to establish/accept SSL connection first and than fallback to TCP connection if necessary.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="socket.dynamic=true"

socket.ssl_cipher

Symmetric cipher to use for encrypted connections.

Default Value  
Dynamic No
Initial Version 1.0

This parameter defines which cipher to use for encrypted SSL connections. If left empty, the SSL library implementation default cipher is used.

The value format depends on used SSL implementation. For OpenSSL, see cipher list format description in https://www.openssl.org/docs/manmaster/man1/openssl-ciphers.html.

The default value was AES128-SHA until Galera version 3.24.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="socket.ssl_cipher=AES128-SHA256"

socket.ssl_compression

This parameter is deprecated and will be removed in the future versions.

Whether to enable compression on SSL connections.

Default Value YES
Dynamic No
Initial Version 1.0
Version Deprecated 4.14

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="socket.ssl_compression=YES"

socket.ssl_key

Defines the path to the SSL certificate key.

Default Value  
Dynamic No
Initial Version 1.0

The node uses the certificate key a self-signed private key in encrypting replication traffic over SSL. You can use either an absolute path or one relative to the working directory. The file must use PEM format.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="socket.ssl_key=/path/to/server-key.pem"

For more information on generating SSL certificate files for your cluster, see SSL Certificates.

socket.ssl_password_file

Defines a password file for use in SSL connections.

Default Value  
Dynamic No
Initial Version 1.0

In the event that you have your SSL key file encrypted, the node uses the SSL password file to decrypt the key file.

The excerpt below is an example of how this Galera parameter might look in the configuration file:

wsrep_provider_options="socket.ssl_password_file=/path/to/password-file"

socket.ssl_reload

Reinitialize SSL context.

Default Value  
Dynamic Yes
Initial Version 4.8

Parameter used to dynamically reinitialize the Galera SSL context. This is most useful if you need to replace a certificate that is about to expire without restarting the server. You need to change the certificate and key files at the relevant paths defined by SSL variables.

The excerpt below is an example of how this Galera parameter can be triggered from running database:

SET GLOBAL wsrep_provider_options = 'socket.ssl_reload=1';

Setting Galera Parameters in MySQL

You can set Galera Cluster parameters in the my.cnf configuration file as follows:

wsrep_provider_options="gcs.fc_limit=256;gcs.fc_factor=0.9"

This is useful in master-slave setups.

You can set Galera Cluster parameters through a MySQL client with the following query:

SET GLOBAL wsrep_provider_options="evs.send_window=16";

This query only changes the evs.send_window value.

To check which parameters are used in Galera Cluster, enter the following query:

SHOW VARIABLES LIKE 'wsrep_provider_options';