You are here

OpenLDAP Replication and Server Configuration

Project ID: 
15
Current stage: 
Manager: 
Unit: 
What: 

Description: Investigation into OpenLDAP replication and proxy-caching server and client configuration.

Deliverables: A stable and robust OpenLDAP server and client deployment, using replication and proxying technologies, fully configured by LCFG resources.

Why: 

Customer: All.

Case statement: Our current replication configuration, in which we have one LDAP master (basilisk) with all clients as slaves replicating directly from the master, puts a rather heavy load on the master and has a single point of failure. Our replication technology is currently home-grown. It should also be noted that our current LDAP master server (basilisk) is no longer on warranty and so will be replaced as part of this project. We would like to investigate current replication and proxying technologies available in OpenLDAP, in particular the relatively new "syncrepl" replication. We should also keep Solaris requirements in mind throughout the project. The tentative initial plan on how our server/client set-up should look is to have one master server, with a slave at each site - these slaves would be kept up to date with the master using syncrepl replication. Each individual client would then run a proxy-caching LDAP server, falling back to the relevant site-slave. All of this requires detailed investigation before we propose a solution.

When: 

Status:

05/02/08
All LDAP slaves - panther, kingsmen, nautilus, mustang - are now configured as syncrepl consumers, with franklin (LDAP master) as syncrepl provider.
30/01/08
Header files now in release structure for configuring syncrepl provider and consumer. We're using copies of these headers in "live" for the moment, while the headers (in dice/options) make their way through the releases. Headers are:
  • dice/options/openldap-syncrepl-master-server.h (for configuring syncrepl master, or "provider")
  • dice/options/openldap-syncrepl-slave-server.h (for configuring syncrepl slave, or "consumer")
  • dice/options/openldap-syncrepl-server.h (included by above headers)

syncrepl consumers should define

#define _NO_OPENLDAP_SLAPREPL_SLAVE_SERVER
#define _OPENLDAP_SYNCREPL_RID 112

... prior to inclusion of syncrepl-slave-header, where _OPENLDAP_SYNCREPL_RID is set to a unique (to the provider) 3-digit number.

06/11/07
Implementing syncrepl replication between the ldap master and slaves requires some changes to be made to the openldap component.
06/11/07
We have been running a test syncrepl set-up for more than a month now (provider: hadrian, consumer: vammala) with minimal problems.
06/11/07
We have been running openldap version 2.3.38 on various machines (inluding the test syncrepl machines) for some weeks now and all appears OK.
01/10/07
As agreed at September's development meeting, this project is being altered to only include the master/slave replication technology, with the rest of the work, specifically the DICE ldap client configuration investigations, being spun off into a separate project. Subsequent milestones in this project have been revised accordingly.
18/07/07
This project has now been taken out of the 'stalled' state. All remaining milestones have been altered accordingly.
31/01/07
Conclusions on Replication
15/01/07
The 'investigate Solaris replication' milestone is not required. We don't run slapd on Solaris machines and have no intention of doing so - Solaris machines will continue to use the appropriate LDAP site-slave.
09/01/07
OpenLDAP version 2.3.32 is now the current 'stable' release - this includes patches to syncrepl, so the machines for testing replication have been upgraded to this.
11/12/06
Push-based syncrepl replication seems to be reliable - I have two 'consumers' being kept in sync with one 'provider'. The only issue is that slapd on the consumers needs to be restarted when the credentials expire. On my test machines, I'm restarting slapd daily, as creds last one day. We could think about having longer-life tickets.
11/12/06
Version 2.3.30 of openldap is out now, which contains various bug fixes (including some for syncrepl). It hasn't become the 'stable' release yet, but when it does so, will warrant investigation as to whether this should be deployed in favour of 2.3.27

Timescales: A date of end April 2007 has been allocated to this project.

Priority: High

Time:

How: 

Proposal:

Resources: Server hardware for the replacement of basilisk has already been purchased and installed in the AT machine room. Very difficult to quantify the person-time resources required for this project, this will become easier to estimate as the investigation is carried out. We would estimate 2-4 weeks of a person suitable experienced in OpenLDAP technologies, possibly more.

Plan:

  1. Investigate replication technologies...
    1. Investigate pull-based syncrepl replication
    2. Investigate push-based syncrepl replication
    3. Investigate delta-syncrepl replication (end Dec)
    4. Solaris replication (mid Jan)
    5. Conclusions on replication (end Jan)
  2. Investigate proxying...
    1. Investigate proxying configuration and set-up (end Feb)
    2. Conclusion on proxying (end Feb)
  3. Conclusions on recommended OpenLDAP server/client configuration
    (end Mar)
  4. LCFG configuration for chosen configuration (end Mar)
  5. Implement chosen set-up (mid Apr)
Other: 

Dependencies: In some respects, this project and the OpenLDAP project are inter-dependent in that all replication testing in this project will be done using the latest stable version, so will offer useful feedback into that project.

Risks:

URL: http://www.dice.inf.ed.ac.uk/units/infrastructure/Projects/06-OpenLDAP-R...

Milestones

Proposed date Achieved date Name Description
2006-09-30 2006-09-30 inv_pull_syncre Investigate pull-based syncrepl replication
2006-12-08 2006-11-30 inv_push_syncre Investigate push-based syncrepl replication
2007-01-15 2006-12-31 inv_delta_syncr Investigate delta-syncrepl replication
2007-01-15 2007-01-15 solaris_repl Solaris replication
2007-01-31 2007-01-31 repl_conclusion Conclusions on replication
2007-11-06 2007-10-15 test-syncrepl Test syncrepl with latest openldap version, to make sure it still works as it did when we tested it all.
2007-11-06 2007-10-15 test-2.3.38 Test latest version of openldap - 2.3.38 is the latest openldap release and is the recommended release - it contains syncrepl patches. Given that we need to run it on the LDAP master, it's important we fully test it first.
2008-02-05 2007-12-21 master-slaves Implement on master ldap/slaves.
2008-01-30 2008-01-15 lcfg-config LCFG headers to configure master/slaves.
2008-03-04 2008-02-15 conclusions Write up conclusions.