Robert Viduya
2016-05-17 20:04:49 UTC
We run a cluster of directory servers (4 masters, 2 hubs, 14 slaves) behind a set of F5 Bigip load balancers. Our Bigip admins recently decided to switch the boxes to "one-armed" mode and that services would have to use X-Forwarded-For headers or equivalent to get the actual client IP address. Obviously, LDAP has no equivalent.
So, I've hacked something together and I'm posting it looking for feedback. If the stuff is actually usable for other folks, that's good too.
On the load balancer side, the code is specifically for the F5 Bigips. But if other load balancers have similar abilities to trigger on events and can insert binary data into the datastream, it should be adaptable.
Essentially what I've done is defined a new LDAP Extended Operation with a payload that's a string containing the source IP address of the incoming connection. The load balancer sends this LDAP operation as soon as it opens a connection to the LDAP server, before any other traffic gets sent. On the directory server side, I've written an Extended Operation plugin that then logs the string. All we need is logging, so that's good enough for us. There's room for improvement there though, like making the address available for IP based ACls (which we don't use).
In order to insert an LDAP operation into the stream, the code on the load balancer needs to choose an LDAP message-id that hopefully the real client isn't going to use. Going on the assumption that the client will start at 0 and increment up, I had the code insert a message-id of 0x70000000. I initially thought I'd have to have the code on the load balancer look for the response message and throw it away so the client doesn't see it, but I've found that the clients just seem to ignore it with no ill effects, so I haven't bothered filtering the response out. The less the code on the load balancer does, the better.
There's a possibility that the client could send it's own xff extended operation, but since the load balancer always sends first, we can just ignore any subsequent log entries.
On the directory server plugin side, I needed to be able to log this to the access log, not to the error log. The slapi_log_access function isn't declared in the plugin header file, so I had to declare it manually.
Here's what our log entries look like now, for both 636 (SSL) and 389 (cleartext before starttls):
[17/May/2016:15:34:22 -0400] conn=230843 fd=156 slot=156 SSL connection from 130.207.167.12 to 130.207.183.16
[17/May/2016:15:34:22 -0400] conn=230843 op=0 EXT oid="1.3.6.1.4.1.636.2.11.11.1" name="forwarded-for extended op"
[17/May/2016:15:34:22 -0400] conn=230843 op=0 forwarded for 130.207.167.12
[17/May/2016:15:34:22 -0400] conn=230843 op=0 RESULT err=0 tag=120 nentries=0 etime=0
[17/May/2016:15:34:22 -0400] conn=230844 fd=156 slot=156 connection from 130.207.167.12 to 130.207.183.16
[17/May/2016:15:34:22 -0400] conn=230844 op=0 EXT oid="1.3.6.1.4.1.636.2.11.11.1" name="forwarded-for extended op"
[17/May/2016:15:34:22 -0400] conn=230844 op=0 forwarded for 130.207.167.12
[17/May/2016:15:34:22 -0400] conn=230844 op=0 RESULT err=0 tag=120 nentries=0 etime=0
We haven't switched our Bigips yet, so the "connection from" line still shows the actual client IP address.
F5 Bigip code fragments are called "irules" and are written in TCL. The tar file below contains two different irule files, one for cleartext streams and one for SSL streams. By SSL streams, I mean where SSL connections from the client are terminated at the load balancer and then re-SSLed to the ldap server. Sorry, I'm not writing the other kind.
So, I've hacked something together and I'm posting it looking for feedback. If the stuff is actually usable for other folks, that's good too.
On the load balancer side, the code is specifically for the F5 Bigips. But if other load balancers have similar abilities to trigger on events and can insert binary data into the datastream, it should be adaptable.
Essentially what I've done is defined a new LDAP Extended Operation with a payload that's a string containing the source IP address of the incoming connection. The load balancer sends this LDAP operation as soon as it opens a connection to the LDAP server, before any other traffic gets sent. On the directory server side, I've written an Extended Operation plugin that then logs the string. All we need is logging, so that's good enough for us. There's room for improvement there though, like making the address available for IP based ACls (which we don't use).
In order to insert an LDAP operation into the stream, the code on the load balancer needs to choose an LDAP message-id that hopefully the real client isn't going to use. Going on the assumption that the client will start at 0 and increment up, I had the code insert a message-id of 0x70000000. I initially thought I'd have to have the code on the load balancer look for the response message and throw it away so the client doesn't see it, but I've found that the clients just seem to ignore it with no ill effects, so I haven't bothered filtering the response out. The less the code on the load balancer does, the better.
There's a possibility that the client could send it's own xff extended operation, but since the load balancer always sends first, we can just ignore any subsequent log entries.
On the directory server plugin side, I needed to be able to log this to the access log, not to the error log. The slapi_log_access function isn't declared in the plugin header file, so I had to declare it manually.
Here's what our log entries look like now, for both 636 (SSL) and 389 (cleartext before starttls):
[17/May/2016:15:34:22 -0400] conn=230843 fd=156 slot=156 SSL connection from 130.207.167.12 to 130.207.183.16
[17/May/2016:15:34:22 -0400] conn=230843 op=0 EXT oid="1.3.6.1.4.1.636.2.11.11.1" name="forwarded-for extended op"
[17/May/2016:15:34:22 -0400] conn=230843 op=0 forwarded for 130.207.167.12
[17/May/2016:15:34:22 -0400] conn=230843 op=0 RESULT err=0 tag=120 nentries=0 etime=0
[17/May/2016:15:34:22 -0400] conn=230844 fd=156 slot=156 connection from 130.207.167.12 to 130.207.183.16
[17/May/2016:15:34:22 -0400] conn=230844 op=0 EXT oid="1.3.6.1.4.1.636.2.11.11.1" name="forwarded-for extended op"
[17/May/2016:15:34:22 -0400] conn=230844 op=0 forwarded for 130.207.167.12
[17/May/2016:15:34:22 -0400] conn=230844 op=0 RESULT err=0 tag=120 nentries=0 etime=0
We haven't switched our Bigips yet, so the "connection from" line still shows the actual client IP address.
F5 Bigip code fragments are called "irules" and are written in TCL. The tar file below contains two different irule files, one for cleartext streams and one for SSL streams. By SSL streams, I mean where SSL connections from the client are terminated at the load balancer and then re-SSLed to the ldap server. Sorry, I'm not writing the other kind.