Migrating to new ldap server and getting problems.
OS: Rocky Linux 8 (== RHEL/CentOS 8). Fully updated. LDAP software: symas-openldap-servers-2.6.1-2.el8.x86_64
cn=config and and data import (via ldif) on master. Everything look good. start/stop/restart is working. ldap access is working.
Adding "syncprov" on master (to enable producer/consumer mode). Now the slapd on the master/producer crashes on start!
Apr 26 18:31:27 apollo11 systemd[1]: Started Symas OpenLDAP Server Daemon. Apr 26 18:31:27 apollo11 kernel: traps: slapd[1379] general protection fault ip:7fa53a499f38 sp:7fa4f8e3d1b0 error:0 in back_mdb.so.2.0.200[7fa53a471000+40000] Apr 26 18:31:27 apollo11 systemd[1]: Started Process Core Dump (PID 1380/UID 0). Apr 26 18:31:27 apollo11 systemd-coredump[1381]: Resource limits disable core dumping for process 1377 (slapd). Apr 26 18:31:27 apollo11 systemd-coredump[1381]: Process 1377 (slapd) of user 0 dumped core. Apr 26 18:31:27 apollo11 systemd[1]: symas-openldap-servers.service: Main process exited, code=killed, status=11/SEGV Apr 26 18:31:27 apollo11 systemd[1]: symas-openldap-servers.service: Failed with result 'signal'. Apr 26 18:31:27 apollo11 systemd[1]: systemd-coredump@3-1380-0.service: Succeeded.
The syncprov ldif
dn: cn=module{0},cn=config changetype: modify add: olcModuleLoad olcModuleLoad: syncprov.la
dn: olcOverlay=syncprov,olcDatabase={1}mdb,cn=config changetype: add objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: syncprov olcSpCheckpoint: 100 10 olcSpSessionlog: 100
dn: olcDatabase={1}mdb,cn=config changetype: modify add: olcDbIndex olcDbIndex: entryCSN,entryUUID eq
Magnus Morén
--On Tuesday, April 26, 2022 6:56 PM +0000 Magnus Morén magnus.moren@hh.se wrote:
The syncprov ldif
dn: cn=module{0},cn=config changetype: modify add: olcModuleLoad olcModuleLoad: syncprov.la
dn: olcOverlay=syncprov,olcDatabase={1}mdb,cn=config changetype: add objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: syncprov olcSpCheckpoint: 100 10 olcSpSessionlog: 100
dn: olcDatabase={1}mdb,cn=config changetype: modify add: olcDbIndex olcDbIndex: entryCSN,entryUUID eq
Please provide the original config DB prior to the addition of the syncprov LDIF being applied, passwords redacted.
Regards, Quanah
--On Tuesday, April 26, 2022 12:00 PM -0700 Quanah Gibson-Mount quanah@fast-mail.org wrote:
--On Tuesday, April 26, 2022 6:56 PM +0000 Magnus Morén magnus.moren@hh.se wrote:
Please provide the original config DB prior to the addition of the syncprov LDIF being applied, passwords redacted.
I'm not able to reproduce:
root@ub18:/opt/symas/etc/openldap# ldapmodify -x -H ldap:/// -D cn=config -w secret dn: cn=module{0},cn=config changetype: modify add: olcModuleLoad olcModuleLoad: syncprov.la
dn: olcOverlay=syncprov,olcDatabase={1}mdb,cn=config changetype: add objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: syncprov modifying entry "cn=module{0},cn=config" olcSpCheckpoint: 100 10 olcSpSessionlog: 100
dn: olcDatabase={1}mdb,cn=config changetype: modify add: olcDbIndex olcDbIndex: entryCSN,entryUUID eq
adding new entry "olcOverlay=syncprov,olcDatabase={1}mdb,cn=config"
modifying entry "olcDatabase={1}mdb,cn=config"
root@ub18:/opt/symas/etc/openldap# root@ub18:/opt/symas/etc/openldap# root@ub18:/opt/symas/etc/openldap# ps -eaf | grep slapd root 41799 1 0 18:38 ? 00:00:00 /opt/symas/lib/slapd -d 0 -h ldap:/// ldaps:/// ldapi:/// root 41819 39679 0 18:39 pts/1 00:00:00 grep --color=auto slapd root@ub18:/opt/symas/etc/openldap# systemctl restart slapd root@ub18:/opt/symas/etc/openldap# ps -eaf | grep slapd root 41823 1 0 18:39 ? 00:00:00 /opt/symas/lib/slapd -d 0 -h ldap:/// ldaps:/// ldapi:/// root 41848 39679 0 18:39 pts/1 00:00:00 grep --color=auto slapd
Although this is Ubuntu rather than RedHat.
I would suspect that this may be related to the use of RockyLinux. Do you encounter the same issue with Alma Linux8?
Regards, Quanah
I have not tried with Alma Linux 8 yet (we are mainly a "Rocky/CentOS/RHEL site")
Did another quick-n-dirty test. Switched 2.6 => 2.5 (did not touch the config and database files from v2.6)
# wget -q https://repo.symas.com/configs/SOLDAP/rhel8/release25.repo -O /etc/yum.repos.d/soldap-release25.repo # yum erase symas-openldap-clients symas-openldap-servers (and deps) # vi /etc/yum.repos.d/soldap-release26.repo (disable repo) # yum install symas-openldap-clients symas-openldap-servers # systemctl start slapd # systemctl status slapd
This version is working!
________________________________ Från: Quanah Gibson-Mount quanah@fast-mail.org Skickat: den 26 april 2022 20:41 Till: Magnus Morén magnus.moren@hh.se; openldap-technical@openldap.org openldap-technical@openldap.org Ämne: Re: slapd (Symas 2.6.1) does not start with syncprov
--On Tuesday, April 26, 2022 12:00 PM -0700 Quanah Gibson-Mount quanah@fast-mail.org wrote:
--On Tuesday, April 26, 2022 6:56 PM +0000 Magnus Morén magnus.moren@hh.se wrote:
Please provide the original config DB prior to the addition of the syncprov LDIF being applied, passwords redacted.
I'm not able to reproduce:
root@ub18:/opt/symas/etc/openldap# ldapmodify -x -H ldap:/// -D cn=config -w secret dn: cn=module{0},cn=config changetype: modify add: olcModuleLoad olcModuleLoad: syncprov.la
dn: olcOverlay=syncprov,olcDatabase={1}mdb,cn=config changetype: add objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: syncprov modifying entry "cn=module{0},cn=config" olcSpCheckpoint: 100 10 olcSpSessionlog: 100
dn: olcDatabase={1}mdb,cn=config changetype: modify add: olcDbIndex olcDbIndex: entryCSN,entryUUID eq
adding new entry "olcOverlay=syncprov,olcDatabase={1}mdb,cn=config"
modifying entry "olcDatabase={1}mdb,cn=config"
root@ub18:/opt/symas/etc/openldap# root@ub18:/opt/symas/etc/openldap# root@ub18:/opt/symas/etc/openldap# ps -eaf | grep slapd root 41799 1 0 18:38 ? 00:00:00 /opt/symas/lib/slapd -d 0 -h ldap:/// ldaps:/// ldapi:/// root 41819 39679 0 18:39 pts/1 00:00:00 grep --color=auto slapd root@ub18:/opt/symas/etc/openldap# systemctl restart slapd root@ub18:/opt/symas/etc/openldap# ps -eaf | grep slapd root 41823 1 0 18:39 ? 00:00:00 /opt/symas/lib/slapd -d 0 -h ldap:/// ldaps:/// ldapi:/// root 41848 39679 0 18:39 pts/1 00:00:00 grep --color=auto slapd
Although this is Ubuntu rather than RedHat.
I would suspect that this may be related to the use of RockyLinux. Do you encounter the same issue with Alma Linux8?
Regards, Quanah
--On Tuesday, April 26, 2022 8:12 PM +0000 Magnus Morén magnus.moren@hh.se wrote:
I have not tried with Alma Linux 8 yet (we are mainly a "Rocky/CentOS/RHEL site")
AlmaLinux is a long standing stable CentOS like alternative to RHEL that's not still under development like Rocky Linux currently is. I'd be curious to know if that's the issue (or if you can reproduce it with Alma Linux). Does it happen 100% of the time with Rocky Linux or only sometimes?
Regards, Quanah
I have the same behaviour on two different test machines. But those are deplyed from the same vm template.
Probably going with 2.5 (ie Long Time Support Release). We do not use any 2.6 features anyway.
I will also try to test with the Alma dist.
________________________________ Från: Quanah Gibson-Mount quanah@fast-mail.org Skickat: den 26 april 2022 21:16 Till: Magnus Morén magnus.moren@hh.se; openldap-technical@openldap.org openldap-technical@openldap.org Ämne: Re: Sv: slapd (Symas 2.6.1) does not start with syncprov
--On Tuesday, April 26, 2022 8:12 PM +0000 Magnus Morén magnus.moren@hh.se wrote:
I have not tried with Alma Linux 8 yet (we are mainly a "Rocky/CentOS/RHEL site")
AlmaLinux is a long standing stable CentOS like alternative to RHEL that's not still under development like Rocky Linux currently is. I'd be curious to know if that's the issue (or if you can reproduce it with Alma Linux). Does it happen 100% of the time with Rocky Linux or only sometimes?
Regards, Quanah
Did a convertion of the Rocky Linux 8 -> Alma Linux 8 (https://techviewleo.com/how-to-convert-from-rocky-linux-to-almalinux/)
Same problem. slapd do not start!
# cat /etc/redhat-release AlmaLinux release 8.5 (Arctic Sphynx)
# systemctl start slapd
Apr 26 21:46:19 apollo99 systemd[1]: Starting Symas OpenLDAP Server Daemon... Apr 26 21:46:19 apollo99 systemd[1]: Started Symas OpenLDAP Server Daemon. Apr 26 21:46:19 apollo99 kernel: traps: slapd[1209] general protection fault ip:7f970713cf38 sp:7f96c58cd1b0 error:0 in back_mdb.so.2.0.200[7f9707114000+40000] Apr 26 21:46:19 apollo99 systemd[1]: Created slice system-systemd\x2dcoredump.slice. Apr 26 21:46:19 apollo99 systemd[1]: Started Process Core Dump (PID 1210/UID 0). Apr 26 21:46:19 apollo99 systemd-coredump[1211]: Resource limits disable core dumping for process 1207 (slapd). Apr 26 21:46:19 apollo99 systemd-coredump[1211]: Process 1207 (slapd) of user 0 dumped core. Apr 26 21:46:19 apollo99 systemd[1]: symas-openldap-servers.service: Main process exited, code=killed, status=11/SEGV Apr 26 21:46:19 apollo99 systemd[1]: symas-openldap-servers.service: Failed with result 'signal'. Apr 26 21:46:19 apollo99 systemd[1]: systemd-coredump@0-1210-0.service: Succeeded. ________________________________ Från: Magnus Morén magnus.moren@hh.se Skickat: den 26 april 2022 21:30 Till: Quanah Gibson-Mount quanah@fast-mail.org; openldap-technical@openldap.org openldap-technical@openldap.org Ämne: Sv: Sv: slapd (Symas 2.6.1) does not start with syncprov
I have the same behaviour on two different test machines. But those are deplyed from the same vm template.
Probably going with 2.5 (ie Long Time Support Release). We do not use any 2.6 features anyway.
I will also try to test with the Alma dist.
________________________________ Från: Quanah Gibson-Mount quanah@fast-mail.org Skickat: den 26 april 2022 21:16 Till: Magnus Morén magnus.moren@hh.se; openldap-technical@openldap.org openldap-technical@openldap.org Ämne: Re: Sv: slapd (Symas 2.6.1) does not start with syncprov
--On Tuesday, April 26, 2022 8:12 PM +0000 Magnus Morén magnus.moren@hh.se wrote:
I have not tried with Alma Linux 8 yet (we are mainly a "Rocky/CentOS/RHEL site")
AlmaLinux is a long standing stable CentOS like alternative to RHEL that's not still under development like Rocky Linux currently is. I'd be curious to know if that's the issue (or if you can reproduce it with Alma Linux). Does it happen 100% of the time with Rocky Linux or only sometimes?
Regards, Quanah
On 4/26/22 20:41, Quanah Gibson-Mount wrote:
Although this is Ubuntu rather than RedHat.
I would suspect that this may be related to the use of RockyLinux. Do you encounter the same issue with Alma Linux8?
FWIW: I'm running test VMs for my Æ-DIR on RockyLinux and AlmaLinux with SOLDAP 2.6.1 including multi-provider replication just fine.
Ciao, Michael.
On 26-04-2022 22:09, Michael Ströder wrote:
On 4/26/22 20:41, Quanah Gibson-Mount wrote:
Although this is Ubuntu rather than RedHat.
I would suspect that this may be related to the use of RockyLinux. Do you encounter the same issue with Alma Linux8?
FWIW: I'm running test VMs for my Æ-DIR on RockyLinux and AlmaLinux with SOLDAP 2.6.1 including multi-provider replication just fine.
AlmaLinux 8.5 x86_64 with OpenLDAP 2.6.1 with delta-sync works fine for me too.
Best, Patrick
Magnus Morén magnus.moren@hh.se schrieb am 26.04.2022 um 19:56 in
Nachricht AM9PR07MB7985D8C9214DBF6EAD7CF56594FB9@AM9PR07MB7985.eurprd07.prod.outlook.com
Migrating to new ldap server and getting problems.
OS: Rocky Linux 8 (== RHEL/CentOS 8). Fully updated. LDAP software: symas‑openldap‑servers‑2.6.1‑2.el8.x86_64
cn=config and and data import (via ldif) on master. Everything look good. start/stop/restart is working. ldap access is working.
Adding "syncprov" on master (to enable producer/consumer mode). Now the slapd on the master/producer crashes on start!
Apr 26 18:31:27 apollo11 systemd[1]: Started Symas OpenLDAP Server Daemon. Apr 26 18:31:27 apollo11 kernel: traps: slapd[1379] general protection fault
ip:7fa53a499f38 sp:7fa4f8e3d1b0 error:0 in back_mdb.so.2.0.200[7fa53a471000+40000] Apr 26 18:31:27 apollo11 systemd[1]: Started Process Core Dump (PID 1380/UID
0). Apr 26 18:31:27 apollo11 systemd‑coredump[1381]: Resource limits disable
core
dumping for process 1377 (slapd).
So maybe enable core dumps and show the backtrace.
Apr 26 18:31:27 apollo11 systemd‑coredump[1381]: Process 1377 (slapd) of
user
0 dumped core. Apr 26 18:31:27 apollo11 systemd[1]: symas‑openldap‑servers.service: Main process exited, code=killed, status=11/SEGV Apr 26 18:31:27 apollo11 systemd[1]: symas‑openldap‑servers.service: Failed
with result 'signal'. Apr 26 18:31:27 apollo11 systemd[1]: systemd‑coredump@3‑1380‑0.service: Succeeded.
The syncprov ldif
dn: cn=module{0},cn=config changetype: modify add: olcModuleLoad olcModuleLoad: syncprov.la
dn: olcOverlay=syncprov,olcDatabase={1}mdb,cn=config changetype: add objectClass: olcOverlayConfig objectClass: olcSyncProvConfig olcOverlay: syncprov olcSpCheckpoint: 100 10 olcSpSessionlog: 100
dn: olcDatabase={1}mdb,cn=config changetype: modify add: olcDbIndex olcDbIndex: entryCSN,entryUUID eq
Magnus Morén
--On Wednesday, April 27, 2022 9:41 AM +0200 Ulrich Windl Ulrich.Windl@rz.uni-regensburg.de wrote:
0). Apr 26 18:31:27 apollo11 systemd‑coredump[1381]: Resource limits disable
core
dumping for process 1377 (slapd).
So maybe enable core dumps and show the backtrace.
Yep, assuming it's not an issue with the VM software being used, this would be the next general step. I would note that installing the relevant debug symbol packages for slapd will be necessary as well.
It may be needed to attach to the slapd process with GDB prior to running the syncprov change as well, since core dumps often lose useful information that the running process has.
--Quanah
I am now running Symas openldap 2.5.11 with the same config and database in an "one producer / one consumer" setup. The same vm:s are used (Rocky Linux 8) and this setup is working.
I do not have the possibility to test the debug/core-dump until next week, but I will try to do it. (I probably need some intructions...)
Thanks for your help so far. ________________________________ Från: Quanah Gibson-Mount quanah@fast-mail.org Skickat: den 27 april 2022 17:06 Till: Ulrich Windl Ulrich.Windl@rz.uni-regensburg.de; Magnus Morén magnus.moren@hh.se; openldap-technical@openldap.org openldap-technical@openldap.org Ämne: Re: Antw: [EXT] slapd (Symas 2.6.1) does not start with syncprov
--On Wednesday, April 27, 2022 9:41 AM +0200 Ulrich Windl Ulrich.Windl@rz.uni-regensburg.de wrote:
0). Apr 26 18:31:27 apollo11 systemd‑coredump[1381]: Resource limits disable
core
dumping for process 1377 (slapd).
So maybe enable core dumps and show the backtrace.
Yep, assuming it's not an issue with the VM software being used, this would be the next general step. I would note that installing the relevant debug symbol packages for slapd will be necessary as well.
It may be needed to attach to the slapd process with GDB prior to running the syncprov change as well, since core dumps often lose useful information that the running process has.
--Quanah
Quanah Gibson-Mount quanah@fast-mail.org schrieb am 27.04.2022 um 17:06
in Nachricht <979B8C9BF3027ACA9450E58E@[192.168.1.20]>:
--On Wednesday, April 27, 2022 9:41 AM +0200 Ulrich Windl Ulrich.Windl@rz.uni-regensburg.de wrote:
0). Apr 26 18:31:27 apollo11 systemd‑coredump[1381]: Resource limits disable
core
dumping for process 1377 (slapd).
So maybe enable core dumps and show the backtrace.
Yep, assuming it's not an issue with the VM software being used, this would
be the next general step. I would note that installing the relevant debug symbol packages for slapd will be necessary as well.
It may be needed to attach to the slapd process with GDB prior to running the syncprov change as well, since core dumps often lose useful information
that the running process has.
In case you test your own software, non-optimized builds can be really helpful, specifically as gcc inlines functions massively at -O2 and higher (older versions did not).
--Quanah
openldap-technical@openldap.org
-
Magnus Morén -
Michael Ströder -
patrick+openldap.org＠laimbock.com -
Quanah Gibson-Mount -
Ulrich Windl