Resolution:
This issue affects Linux kernels prior to 2.4.26, running on x86-64 hardware.  These kernels exhibit a defect calculating checksums, which prevents the Rendezvous daemon from properly joining multicast groups. (Network broadcast is not affected.)

We ran into this multicast problem with RedHat AS 3 - running AS 3 Update 3 to be precise.  We consulted RedHat and found that the Update 4 release solves the problem.

We found the multicast problem with the original AS 3-64 kernel - while trying to do an IGMP V3 request to join the multicast group, it fails with a bad checksum error.  We verified this by using the command "ethereal -V igmp" and running a tibrvlisten on the multicast service.  The problem is in the csum_partial.c kernel file.  There is a fix, which we found through Internet research from a user that had the same problem with SuSE Linux on AMD.  

Here's how to fix it:

1) cd /usr/src/linux-2.4/arch/x86_64/lib
2) make a copy of csum_partial.c in case
3) Change the following line in csum_partial.c from:
        return ~csum_partial(buff,len,0);
    to:
        return csum_fold(csum_partial(buff,len,0));
4) cd /usr/src/linux-2.4
5) make mrproper
6) make oldconfig
7) make dep bzImage
8) make modules
9) make modules_install
    10) make install
11) cp vmlinux to /boot/vmlinux-whatever_your_new_kernel_name
12) make the appropriate changes to /etc/grub.conf

After this change we observed that everything functioned normally.