Re-distribution (and Re-replication) happens when seeder nodes join or leave a space:

Re-distribution happens when a new seeder joins the space, re-replication happens when a seeder leaves the space.

When a record is re-distributed to a new joining seeder, the seeder that previously owned that record keeps it in memory as it is now the new replicator for that record.

Seeders always work to re-build the replication degree (until the number of seeders goes below the ‘min seeder’ space setting). This means for example with a replication degree of 1, if a seeder crashes the remaining seeders will re-replicate between themselves in order to reach a replication degree of 1 again such that when reached once again any one seeder can crash without any data being lost.

Re-distribution and re-replication in AS happens gradually and in the background and is throttled so as to avoid any interruption of service during the re-distribution/replication period. In order to achieve this, ActiveSpaces breaks up the re-distribution/replication in ‘phases’. During a phase the seeders re-distribute/replicate records between themselves (during that time on-going client requests are buffered) and then when the phase is completed wait for some period of time during which buffered client requests are processed and new requests can be serviced before starting the next phase, and so on until the re-distribution/replication is complete.

By default, the re-distribution/replication is quite throttled such as to have almost no noticeable impact on the performance and latency of on-going operations. However the aggressiveness/throttling of the re-distribution/replication can be tuned using the following space definition settings:

Phase Count: This is the number of phases that will be used during the re-distribution/replication. A phase count of 1 would provide the fastest re-distribution/replication time, but at the expense of incurring the most impact on on-going space operations (no client requests are serviced until the re-distribution/replication is completed).

The default value of ‘-1’ means that the total number of records that needs to be re-distributed/replicated is divided by 100,000 in order to compute the number of phases.

Phase Ratio: This is a percentage of the time spent doing re-distribution/replication versus the time spent servicing client requests.

Amount of time to wait before starting the next phase = Amount of time spent during a phase * Phase ratio percentage

The default value is 100 (%) meaning that if on average a re-distribution/replication phase takes X ms to complete, then re-distribution/replication will be paused for a 100% of X ms until the next phase starts.