Has anyone got a dual or quad socket Intel Xeon based server for testing? I've been testing on two AMD systems, one quad socket dual core and one dual socket quad core. There are a lot of different ways to tune these systems... slapd currently uses a single listener thread and a pool of some number of worker threads. I've found that performance improves significantly when the listener thread is pinned to a single core, and no other threads are allowed to run there. I've also found that performance improves somewhat when all worker threads are pinned to specific cores, instead of being free to run on any of the remaining cores. This has made testing a bit more complicated than I expected. I originally was just pinning the entire process to a set number of cores (first 1, then 2, incrementing up to 8) to see how performance changed with additional cores. But due to the motherboard layout and the fact that the I/O bridges are directly attached to particular sockets, it makes a big difference exactly which cores you use. Another item I noticed is that while we scale perfectly linearly from 1 core to 2 cores in a socket (with a dual-core processor), as we start spreading across multiple sockets the scaling tapers off drastically. That makes sense given the constraints of the Hypertransport connections between the sockets. On the quad-core system we scale pretty linearly from 1 to 4 cores (in one socket) but again the improvement tapers off drastically when the 2nd socket is added in. I don't have any Xeon systems to test on at the moment, but I'm curious to see how they do given that all CPUs should have equal access to the northbridge. (Of course, given that both memory and I/O traffic go over the bus, I'm not expecting any miracles...) The quad-core system I'm using is a Supermicro AS-2021M-UR+B; it's based on an Nvidia MCP55 chipset. The gigabit ethernet is integrated in this chipset. Using back-null we can drive this machine to over 54,000 authentications/second, at which point 100% of a core is consumed by interrupt processing in the ethernet driver. The driver doesn't support interrupt coalescing, unfortunately. (By the way, that represents somewhere between 324,000pps and 432,000pps. While there's only 5 LDAP packets per transaction, some of the client machines choose to send separate TCP ACKs, while others don't, which makes the packet count somewhere between 5-8 packets per transaction. I hadn't taken those ACKs into account when I discussed these figures before. At these packet sizes (80-140 bytes), I think the network would be 100% saturated at around 900,000pps.) Interestingly, while 2 cores can get over 13,000 auths/second, and 4 cores can get around 25,000 auths/second (using back-hdb), with all 8 cores it's only peaking at 29,000 auths/second. This tells me it's better to run two separate slapds in a mirrormode configuration on this box (4 cores per process) than to run a single process across all of the cores. Then I'd expect to hit 50,000 auths/second total, pretty close to the limits of the ethernet device/driver. -- -- Howard Chu Chief Architect, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

Has anyone got a dual or quad socket Intel Xeon based server for testing? I've been testing on two AMD systems, one quad socket dual core and one dual socket quad core. There are a lot of different ways to tune these systems...

slapd currently uses a single listener thread and a pool of some number of worker threads. I've found that performance improves significantly when the listener thread is pinned to a single core, and no other threads are allowed to run there. I've also found that performance improves somewhat when all worker threads are pinned to specific cores, instead of being free to run on any of the remaining cores. This has made testing a bit more complicated than I expected.

I originally was just pinning the entire process to a set number of cores (first 1, then 2, incrementing up to 8) to see how performance changed with additional cores. But due to the motherboard layout and the fact that the I/O bridges are directly attached to particular sockets, it makes a big difference exactly which cores you use.

Another item I noticed is that while we scale perfectly linearly from 1 core to 2 cores in a socket (with a dual-core processor), as we start spreading across multiple sockets the scaling tapers off drastically. That makes sense given the constraints of the Hypertransport connections between the sockets.

On the quad-core system we scale pretty linearly from 1 to 4 cores (in one socket) but again the improvement tapers off drastically when the 2nd socket is added in.

I don't have any Xeon systems to test on at the moment, but I'm curious to see how they do given that all CPUs should have equal access to the northbridge. (Of course, given that both memory and I/O traffic go over the bus, I'm not expecting any miracles...)

The quad-core system I'm using is a Supermicro AS-2021M-UR+B; it's based on an Nvidia MCP55 chipset. The gigabit ethernet is integrated in this chipset. Using back-null we can drive this machine to over 54,000 authentications/second, at which point 100% of a core is consumed by interrupt processing in the ethernet driver. The driver doesn't support interrupt coalescing, unfortunately. (By the way, that represents somewhere between 324,000pps and 432,000pps. While there's only 5 LDAP packets per transaction, some of the client machines choose to send separate TCP ACKs, while others don't, which makes the packet count somewhere between 5-8 packets per transaction. I hadn't taken those ACKs into account when I discussed these figures before. At these packet sizes (80-140 bytes), I think the network would be 100% saturated at around 900,000pps.)

Interestingly, while 2 cores can get over 13,000 auths/second, and 4 cores can get around 25,000 auths/second (using back-hdb), with all 8 cores it's only peaking at 29,000 auths/second. This tells me it's better to run two separate slapds in a mirrormode configuration on this box (4 cores per process) than to run a single process across all of the cores. Then I'd expect to hit 50,000 auths/second total, pretty close to the limits of the ethernet device/driver.