• 8 way replication with 8 nodes (each node having 7 other connections)
• 50k records
• Inserting the records into one cluster node to replicate to all the rest
  

The problems obvserved:

• Some nodes are faster at replication than others. In general, the time to complete replication is slower than expected. In my test environment I found that 50k records can take up to 2 hours for some nodes to complete. The fastest nodes complete in 1.5 hours. Because these records are brand new insertions, delta based replication wouldn't help here.
• When the replication is completed, some of the data.mdb files are larger than others (sometimes by an order of magnitude).
  

We would like to understand the reason behind these two problems above. First, the replication system seems unusually slow. Second, we need to understand why the data.mdb file grows sometimes far larger on one node than the rest of the nodes. For example, in our production environment, while most nodes were around 1GB in data size, one node stored in excess of 40GB in data.mdb. In my testing lab, my the 50k record insertion left most nodes with a data.mdb size of 150MB. On one of the nodes, the data size was 262MB.

Note that I've also tried alternative replication connectivity approaches to attempt to reduce the number of connections per server, but that did not improve replication performance or the varying data sizes in the end.

If updating to a newer version helps resolve the above observed problems, please let me know.

Any tuning or debugging advice here would be appreciated.

Thanks.

--

Brian Wright Sr. UNIX Systems Engineer 901 Mariners Island Blvd Suite 200 San Mateo, CA 94404 USA Email  brianw@marketo.com Phone +1.650.539.3530 www.marketo.com