Hy everyone !
I am facing a major issued for the second time with OpenLDAP 2.3.11. I am aware it is a pretty old version of OpenLDAP, but, it has been working in production for almost a year now without any problem.
Here is the setup : Master -> Slave
For some strange reason, the contextCSN stops updating and therefore the Slave is not updated anymore. The strange thing is that the Master continues to successfully update/add data !
We are using a OpenLDAP 2.3.11, Berkeley DB 4.4.16 & OpenSSL 0.9.8a all running on Solaris 10.
Restarting the master solved the problem once, but now that it as failed again, I am very woried, because, restarting isn't a solution, considering that the application is in production.
Please see in attachement the configuration files for Master & Slave. I have no log on the master, and only "syncrepl logging" on the slave.
We consider restarting the master with "enable all logging" to see if wee can grab some informations....
Is this a bug of OpenLDAP, BerkleyDB or something else ?
Thanks in advance,
Adrien Futschik
Adrien Futschik wrote:
I am facing a major issued for the second time with OpenLDAP 2.3.11. I am aware it is a pretty old version of OpenLDAP, but, it has been working in production for almost a year now without any problem.
Within the 2.3 series the last release was 2.3.43! There were numerous syncrepl-related fixes since 2.3.11.
File CHANGES in OPENLDAP_REL_ENG_2_3 lists:
OpenLDAP 2.3.25 Release (2006/08/01) [..] Fixed slapd syncrepl contextCSN issue (ITS#4622) [..]
So you should definitely upgrade! I'd strongly recommend upgrade to 2.4.16 since the 2.3. series is no longer maintained. slapcat/slapadd is needed since the DB format changed.
We are using a OpenLDAP 2.3.11, Berkeley DB 4.4.16 & OpenSSL 0.9.8a all running on Solaris 10.
Note that there are also patches for the BDB releases.
Ciao, Michael.
Le jeudi 16 avril 2009 11:32:57, Michael Ströder a écrit :
Adrien Futschik wrote:
I am facing a major issued for the second time with OpenLDAP 2.3.11. I am aware it is a pretty old version of OpenLDAP, but, it has been working in production for almost a year now without any problem.
Within the 2.3 series the last release was 2.3.43! There were numerous syncrepl-related fixes since 2.3.11.
File CHANGES in OPENLDAP_REL_ENG_2_3 lists:
OpenLDAP 2.3.25 Release (2006/08/01) [..] Fixed slapd syncrepl contextCSN issue (ITS#4622) [..]
So you should definitely upgrade! I'd strongly recommend upgrade to 2.4.16 since the 2.3. series is no longer maintained. slapcat/slapadd is needed since the DB format changed.
We are using a OpenLDAP 2.3.11, Berkeley DB 4.4.16 & OpenSSL 0.9.8a all running on Solaris 10.
Note that there are also patches for the BDB releases.
Ciao, Michael.
First of all, thank you for your quick answer. I am aware I should migrate, but for the moment, the only solution I have would be to migrate to OpenLDAP 2.3.32, wich should solve issue (ITS#4622).
However, I am not sure at all that this bug is what we encounter. The contextCSN is not updated on the master, and therefore the replication stops. But When the master is restarted (contextCSN updating again), the replication restarts a well.
Is it possible to use a newer version of OpenLDAP as slave ? ie. Could I use OpenLDAP 2.3.11 as master and OpenLDAP 2.3.32 a slave for a while ?
Is this not recommended because syncrepl mecanisme changed a lot between those two versions ? I will give it a try anyway. OpenLDAP 2.4.x will be our next target, but needs some additionnal work...
Adrien Futschik
Adrien Futschik wrote:
I am aware I should migrate, but for the moment, the only solution I have would be to migrate to OpenLDAP 2.3.32,
Why? Please don't take this personally. But if that is because you strictly rely on Linux distribution packages I'd like to note that your operational concept is already flawed.
See also: http://www.openldap.org/faq/data/cache/1456.html
At my customers I normally have a build system matching the system in production where I can compile and test OpenLDAP, SASL and BDB myself installing into a separate prefix. Given that VMware machines are pretty cheap to set up this is feasible today even if you have various i386 OS running in production.
Is it possible to use a newer version of OpenLDAP as slave ? ie. Could I use OpenLDAP 2.3.11 as master and OpenLDAP 2.3.32 a slave for a while ?
Not sure. Please examine CHANGES of OPENLDAP_REL_ENG_2_3 yourself. There were also changes to make syncrepl compatible with OPENLDAP_REL_ENG_2_4. But this was in a more recent version.
Ciao, Michael.
Le jeudi 16 avril 2009 14:04:44, Michael Ströder a écrit :
Adrien Futschik wrote:
I am aware I should migrate, but for the moment, the only solution I have would be to migrate to OpenLDAP 2.3.32,
Why? Please don't take this personally. But if that is because you strictly rely on Linux distribution packages I'd like to note that your operational concept is already flawed.
Because the client I am working for uses source-compiled versions of OpenLDAP and is curently running 2.3.11 or 2.3.32. There is no newer package for the moment. We are working on a OpenLDAP 2.4.16 or newer, but, production is critical and we can not wait for this one to be released (by our team).
I know very well that many changes have been made between 2.3.11 and the latest 2.3.x and even between 2.3.32 and the latest 2.3.x, but we can not afford to migrate to 2.4.x right now.
See also: http://www.openldap.org/faq/data/cache/1456.html
At my customers I normally have a build system matching the system in production where I can compile and test OpenLDAP, SASL and BDB myself installing into a separate prefix. Given that VMware machines are pretty cheap to set up this is feasible today even if you have various i386 OS running in production.
The project that is using OpenLDAP 2.3.11 is running Solaris 10 SPARC from source.
Is it possible to use a newer version of OpenLDAP as slave ? ie. Could I use OpenLDAP 2.3.11 as master and OpenLDAP 2.3.32 a slave for a while ?
I have justed tested it. OpenLDAP 2.3.11 as master and OpenLDAP 2.3.32 as slave : it WORKS. The question is, is it reliable ?
Not sure. Please examine CHANGES of OPENLDAP_REL_ENG_2_3 yourself. There were also changes to make syncrepl compatible with OPENLDAP_REL_ENG_2_4. But this was in a more recent version.
I will examine CHANGES of OPENLDAP_REL_ENG_2_3.
Ciao, Michael.
Thanks for your help.
Adrien Futschik
--On Thursday, April 16, 2009 2:18 PM +0200 Adrien Futschik adrien.futschik@atosorigin.com wrote:
Le jeudi 16 avril 2009 14:04:44, Michael Ströder a écrit :
Adrien Futschik wrote:
I am aware I should migrate, but for the moment, the only solution I have would be to migrate to OpenLDAP 2.3.32,
Why? Please don't take this personally. But if that is because you strictly rely on Linux distribution packages I'd like to note that your operational concept is already flawed.
Because the client I am working for uses source-compiled versions of OpenLDAP and is curently running 2.3.11 or 2.3.32. There is no newer package for the moment. We are working on a OpenLDAP 2.4.16 or newer, but, production is critical and we can not wait for this one to be released (by our team).
I know very well that many changes have been made between 2.3.11 and the latest 2.3.x and even between 2.3.32 and the latest 2.3.x, but we can not afford to migrate to 2.4.x right now.
At a minimum, you should be using 2.3.43 on all your servers. You should be responsible for making sure your client uses the most up to date and stable release available for them to use, if you are given them support. Otherwise, you are failing in that support relationship.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
Le jeudi 16 avril 2009 19:05:58, Quanah Gibson-Mount a écrit :
--On Thursday, April 16, 2009 2:18 PM +0200 Adrien Futschik
adrien.futschik@atosorigin.com wrote:
Le jeudi 16 avril 2009 14:04:44, Michael Ströder a écrit :
Adrien Futschik wrote:
I am aware I should migrate, but for the moment, the only solution I have would be to migrate to OpenLDAP 2.3.32,
Why? Please don't take this personally. But if that is because you strictly rely on Linux distribution packages I'd like to note that your operational concept is already flawed.
Because the client I am working for uses source-compiled versions of OpenLDAP and is curently running 2.3.11 or 2.3.32. There is no newer package for the moment. We are working on a OpenLDAP 2.4.16 or newer, but, production is critical and we can not wait for this one to be released (by our team).
I know very well that many changes have been made between 2.3.11 and the latest 2.3.x and even between 2.3.32 and the latest 2.3.x, but we can not afford to migrate to 2.4.x right now.
At a minimum, you should be using 2.3.43 on all your servers. You should be responsible for making sure your client uses the most up to date and stable release available for them to use, if you are given them support. Otherwise, you are failing in that support relationship.
In case of this client no. It is long to explain, and therefore, I will not explain it here, but upgrading is'nt always an option.
For your information, between the time OpenLDAP releases a version and the time the projects might actualy use it, is about 6-7 month. That's because of the whole compilation/testing/validation/packaging/releasing process that the client is using. It has been so for many year now.
Thanks anyway.
Adrien Futschik
On Friday 17 April 2009 08:02:04 Adrien Futschik wrote:
Le jeudi 16 avril 2009 19:05:58, Quanah Gibson-Mount a écrit :
--On Thursday, April 16, 2009 2:18 PM +0200 Adrien Futschik
adrien.futschik@atosorigin.com wrote:
Le jeudi 16 avril 2009 14:04:44, Michael Ströder a écrit :
Adrien Futschik wrote:
I am aware I should migrate, but for the moment, the only solution I have would be to migrate to OpenLDAP 2.3.32,
Why? Please don't take this personally. But if that is because you strictly rely on Linux distribution packages I'd like to note that your operational concept is already flawed.
Because the client I am working for uses source-compiled versions of OpenLDAP and is curently running 2.3.11 or 2.3.32. There is no newer package for the moment. We are working on a OpenLDAP 2.4.16 or newer, but, production is critical and we can not wait for this one to be released (by our team).
I know very well that many changes have been made between 2.3.11 and the latest 2.3.x and even between 2.3.32 and the latest 2.3.x, but we can not afford to migrate to 2.4.x right now.
At a minimum, you should be using 2.3.43 on all your servers. You should be responsible for making sure your client uses the most up to date and stable release available for them to use, if you are given them support. Otherwise, you are failing in that support relationship.
In case of this client no. It is long to explain, and therefore, I will not explain it here, but upgrading is'nt always an option.
For your information, between the time OpenLDAP releases a version and the time the projects might actualy use it, is about 6-7 month. That's because of the whole compilation/testing/validation/packaging/releasing process that the client is using. It has been so for many year now.
Well, 3 of those steps (compilation,testing, packaging) should honestly take no more than 1 hour, if they are using any kind of decent build system ... (since OpenLDAP has had quite a comprehensive test suite since about 2.2.20, which can be run during the compilation/packaging).
Regards, Buchan
Adrien Futschik wrote:
For your information, between the time OpenLDAP releases a version and the time the projects might actualy use it, is about 6-7 month. That's because of the whole compilation/testing/validation/packaging/releasing process that the client is using. It has been so for many year now.
I already said: The operational concept is flawed. It might not be your fault but if you see real problems you should escalate the issue to change that process (with the help of a request to the change management).
Ciao, Michael.
--On Friday, April 17, 2009 1:14 PM +0200 Michael Ströder michael@stroeder.com wrote:
Adrien Futschik wrote:
For your information, between the time OpenLDAP releases a version and the time the projects might actually use it, is about 6-7 month. That's because of the whole compilation/testing/validation/packaging/releasing process that the client is using. It has been so for many year now.
I already said: The operational concept is flawed. It might not be your fault but if you see real problems you should escalate the issue to change that process (with the help of a request to the change management).
Another part of this that your management and/or process may be missing is the concept of point releases (2.3.32 to 2.3.43 for example) versus minor releases (2.3 to 2.4). Whereas going from one point release to another generally is backwards compatible and usually only consists of bug fixes, going between minor releases often involves incompatibilities in configuration, database structure, etc. So while the effort from going 2.3->2.4 is likely large, the effort from going to 2.3.11 or 2.3.32 to 2.3.43 is rather minute. Certainly, the effect of going between point releases in the 2.3 series has been rather well tested at this point.
As for fixes to the syncprov overlay since 2.3.11:
Fixed slapd-bdb empty suffix and syncprov issue (ITS#4171) Fixed slapo-syncprov LDAP response types (ITS#4183) Fixed slapo-syncprov unpublished control (ITS#4238) Fixed slapo-syncprov message id issue Fixed slapo-syncprov/pcache filter dup issue (ITS#4364) Fixed slapo-syncprov playlog nentries reset issue (ITS#4365) Fixed slapo-syncprov update latency issue (ITS#4385) Fixed slapd syncprov/glue interaction issue (ITS#4323, ITS#4417) Fixed slapo-syncprov MODs cause DELs (ITS#4423) Fixed slapo-syncprov/syncrepl sessionlog issue (ITS#4534) Added slapo-syncprov extra logging Fixed slapo-syncprov crash under glued database (ITS#4562) Fixed slapo-syncprov DEL propagation bug (ITS#4589) Fixed slapo-syncprov need new CSN with delete syncID sets (ITS#4534) Fixed slapo-syncprov startup when lastmod is off (ITS#4613) Fixed slapo-syncprov incomplete sync on restart issues (ITS#4622) Fixed slapo-syncprov to complain if defined outside of a database Fixed slapo-syncprov presence list (ITS#4813) Fixed slapo-syncprov contextCSN checkpoint again (ITS#4720) Fixed slapo-syncprov cookie parsing error (ITS#4977) Fixed slapd-glue/syncprov interaction (ITS#4623) Fixed slapo-syncprov uninit'd vars (ITS#5048,#5049) Fixed slapo-syncprov refreshAndPersist race condition (ITS#5177) Fixed slapo-syncprov csn update with delta-syncrepl (ITS#5493) Fixed slapo-syncprov psearch on closed connection (ITS#5401, ITS#5565)
So any number of these could be the source of the issue you are seeing and/or could be affecting your systems.
--Quanah
--
Quanah Gibson-Mount Principal Software Engineer Zimbra, Inc -------------------- Zimbra :: the leader in open source messaging and collaboration
On Thursday 16 April 2009 14:18:13 Adrien Futschik wrote:
Le jeudi 16 avril 2009 14:04:44, Michael Ströder a écrit :
Adrien Futschik wrote:
I am aware I should migrate, but for the moment, the only solution I have would be to migrate to OpenLDAP 2.3.32,
Why? Please don't take this personally. But if that is because you strictly rely on Linux distribution packages I'd like to note that your operational concept is already flawed.
Because the client I am working for uses source-compiled versions of OpenLDAP and is curently running 2.3.11 or 2.3.32. There is no newer package for the moment.
It should be almost no effort to build packages of 2.3.43 if you have packages of 2.3.32 ... depending on how you build them.
We are working on a OpenLDAP 2.4.16 or newer, but, production is critical and we can not wait for this one to be released (by our team).
I know very well that many changes have been made between 2.3.11 and the latest 2.3.x and even between 2.3.32 and the latest 2.3.x, but we can not afford to migrate to 2.4.x right now.
See also: http://www.openldap.org/faq/data/cache/1456.html
At my customers I normally have a build system matching the system in production where I can compile and test OpenLDAP, SASL and BDB myself installing into a separate prefix. Given that VMware machines are pretty cheap to set up this is feasible today even if you have various i386 OS running in production.
The project that is using OpenLDAP 2.3.11 is running Solaris 10 SPARC from source.
Is it possible to use a newer version of OpenLDAP as slave ? ie. Could I use OpenLDAP 2.3.11 as master and OpenLDAP 2.3.32 a slave for a while ?
I have justed tested it. OpenLDAP 2.3.11 as master and OpenLDAP 2.3.32 as slave : it WORKS. The question is, is it reliable ?
Well, the contextCSN disappearing from the master most likely won't change if you keep running the same code on the master, so I doubt this will help you.
Regards, Buchan
Le vendredi 17 avril 2009 09:47:27, Buchan Milne a écrit :
On Thursday 16 April 2009 14:18:13 Adrien Futschik wrote:
Le jeudi 16 avril 2009 14:04:44, Michael Ströder a écrit :
Adrien Futschik wrote:
I am aware I should migrate, but for the moment, the only solution I have would be to migrate to OpenLDAP 2.3.32,
Why? Please don't take this personally. But if that is because you strictly rely on Linux distribution packages I'd like to note that your operational concept is already flawed.
Because the client I am working for uses source-compiled versions of OpenLDAP and is curently running 2.3.11 or 2.3.32. There is no newer package for the moment.
It should be almost no effort to build packages of 2.3.43 if you have packages of 2.3.32 ... depending on how you build them.
Concerning OpenLDAP 2.3.43, I didn't see anything about contextCSN not being updated on a master in the changelog of OPENLDAP_REL_ENG_2_3, I am therefore not sure that it would solve the problem anyway.
Please, if anyone thinks that I have missed something there, don't hesitate !
Is it possible to use a newer version of OpenLDAP as slave ? ie. Could I use OpenLDAP 2.3.11 as master and OpenLDAP 2.3.32 a slave for a while ?
I have justed tested it. OpenLDAP 2.3.11 as master and OpenLDAP 2.3.32 as slave : it WORKS. The question is, is it reliable ?
Well, the contextCSN disappearing from the master most likely won't change if you keep running the same code on the master, so I doubt this will help you.
True, but then it will be "easier" to switch to 2.3.32. The idea, is to use 2.3.32 as a slave for a while and the switch it to master and get rid of OpenLDAP 2.3.11.
In case of this client no. It is long to explain, and therefore, I will
not
explain it here, but upgrading is'nt always an option.
For your information, between the time OpenLDAP releases a version and the time the projects might actualy use it, is about 6-7 month. That's because of the whole compilation/testing/validation/packaging/releasing process that the client is using. It has been so for many year now.
Well, 3 of those steps (compilation,testing, packaging) should honestly take no more than 1 hour, if they are using any kind of decent build system ... (since OpenLDAP has had quite a comprehensive test suite since about 2.2.20, which can be run during the compilation/packaging).
Wrong, different teams are taking part of the whole process, and testing a product is'nt only a question of unit testing ! Anyway, this was not my point at all, thanks for carring.
Adrien Futschik
openldap-technical@openldap.org
-
Adrien Futschik -
Buchan Milne -
Michael Ströder -
Quanah Gibson-Mount