You are here

DICE client LDAP configuration

Project ID: 
79
Current stage: 
Manager: 
Unit: 
What: 

Description: Investigation into DICE client LDAP configuration, to replace our existing home grown technology.

Deliverables: Recommendations for DICE client LDAP configuration, including LCFG configuration. Also, fully document all conclusions from investigations.

Why: 

Customer: All

Case statement: Initially, see this case statement for the "OpenLDAP Replication and Server Configuration" project, which this project has been spun off from. This project is only concerned with the client-side, specifically ways to replace our technology, in which all DICE clients run a full LDAP server. This approach has proven unreliable, particularly where clients run memory-intensive jobs. It is a pre-requisite of this project that any connection between clients and remote LDAP servers be secured. Since the original project was proposed, new techniques have arisen. In particular, there are three candidate approaches to providing secure client-side LDAP which require detailed investigation:

  • OpenLDAP proxycaching (using the openldap ldap backend and pcache overlays), which provide proxying to a remote LDAP server, with local caching (or not) of pre-configured queries.
  • nss_ldap - using kerberos-authenticated connections to a remote server
  • nss_ldapd - a fork of nss_ldap to address certain issues.
When: 

Status:

17/07/09
As discussed at the July development meeting, this project is to be put into a stalled state, although I will continue to monitor developments and run test clients. Unfortunately it just hasn't proved stable enough to consider deploying and debugging efforts have reached somewhat of a dead end.
05/03/09
As discussed at March development meeting, push back milestones to allow time for debugging/bug fixing/assessment for 2.4
03/03/09
See blog for latest. We need to decide now whether the project should be stalled, or whether another milestone for 2.4 testing is added.
18/12/08
Marked various milestones as done, as discussed at last devproj meeting, amended date for new proxy-crashes milestone to give myself a bit more time (factoring in Christmas break), amended subsequent milestones accordingly...
08/12/08
Latest project stuff here. Comments welcome.
08/12/08
2.4 - proxycache now seems stable, but ITS#5756 is a problem.
08/12/08
server testing - 2 servers currently being used - load so far is
minimal, but needs more testing with condor and also beowulf. It is ultimately our intention to run the LDAP slaves with extremely minimal logging (this vastly improves performance on a busy server). We will keep logging for the moment while testing is being done.
08/12/08
testing - proxycaching is currently running on all SL5 develop
machines, and in two student labs - AT-4.12 and AT-5-cl-s. 122 LCFG
profiles in total. Condor only recently added to labs and seems to
have increased number of crashes - this requires investigation - added
milestone.
06/11/08
I'm going to attempt to maintain a blog to discuss work on this project (and other ldap things)
26/08/08
Milestones revised as agreed at August development meeting
26/08/08
Work has been substantially delayed on this project due mainly to the
move to the Forum, but also because of SL5 server upgrades, exams in
labs and upgrades to SL5 of lab machines (as all desktops are moving
to SL5, there is little point in testing under anything else). Annual leave in September is also contributing to the delay.
The current focus will be to expand testing in labs, on beowulf
clusters and on some desktop machines.
26/08/08
During the last year, the OpenLDAP project has released version 2.4.
This has now become the recommended "stable" release. It is important
that we test and evaluate this version of OpenLDAP as well as the
latest 2.3 version. The versions we are currently using for testing
are 2.3.43 and 2.4.11. I have added a milestone to evaluate 2.4.
Briefly, the pros and cons of 2.4 are... Pros: contains functionality
that we would like, it is the focus for new development, it will be
better supported than 2.3. Cons: relatively untested in our
environment when compared to 2.3, judging by the bugs we've uncovered,
slapo-pcache is perhaps not tested much by the community. Ultimately
we will almost certainly move to 2.4 for all openldap use, the main
question is when and whether we do it for this project.
Bug reports have been submitted to the openldap project for problems
we have seen when using the pcache overlay with Openldap 2.4. See
openldap href="http://www.openldap.org/its/index.cgi?findid=5546">ITS 5546
and ITS
5665
for further information.
31/03/08
A bug report was submitted to the openldap project detailing the intermittent fault we were seeing - see openldap ITS 5404 for further information. Following this, a patch was produced and has been in testing since 10/03 with no recurrence of problem.
08/11/07
We have discovered an occasional problem with proxy-caching that requires further investigation before we implement it more widely.
06/11/07
Initial conclusions on the various client technologies
01/10/07
Following discussions with George concerning the timescales of this project, we have set an initial milestone for the November development meeting for conclusions on the various client technologies being tested. Subsequent milestones will be set following this.
14/09/07
Initial project proposal.

Timescales: We would expect the project to be completed by the end of the calendar year, in the worst case. Certain factors should be considered - particularly the relative open-endedness of testing and debugging. Also, expected paternity leave, etc. in November.

Priority: High priority.

Time:

How: 

Proposal:

Resources: Difficult to quantify. A large amount of testing of various configurations will be required, which may involve a lot of 'calendar time' without necessarily requiring a lot of person-time, i.e. leaving a configuration in place in a test-cluster for testing under real-world conditions. Requires experience of OpenLDAP and nss_ldap, as well as general Linux and C debugging skills.

Plan:

  1. Evaluate and test OpenLDAP proxy-caching and proxy-no-caching solutions.
  2. Evaluate and test nss_ldap with kerberos authentication
  3. Evaluate and test nss_ldapd
  4. Conclusions on best solution
  5. LCFG configuration for best solution
  6. Document conclusions and recommendations

Note that the first three items can effectively run in parallel

Other: 

Dependencies: None

Risks:

Milestones

Proposed date Achieved date Name Description
2007-11-06 2007-11-07 initial_conclus Initial conclusions on the various client technologies
2008-03-31 2008-03-31 proxy-debug Investigate occasional problem with proxy-caching and proxy-nocaching
2008-11-30 2008-11-30 proxy-moretest expand pool of testing machines to further test client-side operation
2008-11-30 2008-11-30 proxy-servertes assess server load generated by client-side testing
2009-02-11 2009-02-28 proxy-meeting inf-unit meeting to discuss recommendations
2009-07-31 proxy-recommend produce recommendations for dice ldap client configuration
2008-11-30 2008-11-30 proxy-2.4test test openldap version 2.4
2009-02-11 2009-02-15 proxy-crashes Investigate slapd crashes
2009-06-30 proxy-2.4morete More testing with openldap 2.4 to debug, file bug reports and assess stability/reliability.