ARP Cache size and interval configuration

It is possible to increase the size of engine’s ARP cache by changing values of /proc/sys/net/ipv4/neigh/default/gc_thresh3 (The hard maximum number of entries to keep in the ARP cache. Defaults to 1024.) and /proc/sys/net/ipv4/neigh/default/gc_thresh2 (The soft maximum number of entries to keep in the ARP cache. Defaults to 512.)

To change the timeout value of the entries, you need to modify the gc_interval file, indicating the timeout value in seconds. For other kernal variables checkout the man page of arp (http://www.kernel.org/doc/man-pages/online/pages/man7/arp.7.html).

Not sure how to make the changes permanent in ubuntu, but in the reference, the following is stated.

The changes should be made to /data/run-at-boot  file in order for changes to survive boot. The run-at-boot file must be created with execute permission for root.

Add the following lines to /data/run-at-boot file in all of the engines of the cluster:

#!/bin/sh
echo 8192 >/proc/sys/net/ipv4/neigh/default/gc_thresh3
echo 4096 >/proc/sys/net/ipv4/neigh/default/gc_thresh2

The change activates after boot or by executing the run-at-boot script. The change should be activated as simultaneously as possible in all the nodes of the cluster, so that the functioning of the nodes in the cluster would be same all the time.

References:
1. http://stoneblog.stonesoft.com/2009/02/arp-cache-overflow/
2. http://www.ibm.com/developerworks/opensource/library/os-iptables/index.html