Resolution:
GridServer 5 has the best scalability of any GridServer release to date. In our tests and in production environments, there are some configuration changes that can improve performance on large grids (> 1,000 Engines). This will improve Manager performance in situations, for example, when the Broker Monitor response time may take several seconds to update.

The following changes can improve performance on large Grids:

    * on the Admin -> Manager Configuration -> Services page increase the Assignment Timeout, to 60000 ms. Increasing this allows more time for an Engine to connect and pickup an assigned task when the Broker is under heavy load. This value should be increased if you see 'Task assignment expired:'… messages often.
    * On the Admin -> Manager Configuration -> Services page, increase the Task Reservation Timeout to 120.
    * Separate the administrative port (web browser port) from the HTTP messaging port, to all administrative access even under heavy messaging load. This is explained in KB article #235
    * On the Admin -> Manager Configuration -> Communication Page, change Messaging Retry Wait to 10000 ms; change Driver/Engine/Daemon Socket Timeout to 120 seconds; change Driver Heartbeat Timeout Factor to 15; change Engine Heartbeat Timeout Factor to 4.
    * Increase the heap size. See KB article #8 - "The Manager runs out of memory" for details.