I'm experimenting with mdb to replace hdb on my test/development set of servers. I've built 2.4.32 and the server is running RHEL 6.3. This morning, I got a confusing error.
Attempting to perform a deletion of a member from a groupOfNames, the following was recorded in syslog:
mdb_id2entry_put: mdb_put failed: No space left on device(28) "cn=GroupNameNotImportant,ou=groups,dc=uvm,dc=edu"
and ldapmodify returned:
modifying entry "cn=GroupNameNotImportant,ou=Groups,dc=uvm,dc=edu" ldap_modify: Other (e.g., implementation specific) error (80) additional info: entry update failed
However, free shows memory left, df shows filesystem space left (in fact, it is only 22% used) and data.mdb is
-rw------- 1 ldap ldap 1547292672 Sep 8 06:22 data.mdb
while the maxsize parameter is set to:
maxsize 85899345920
So, I am confused about which device mdb is telling me is out of space.
What have I missed?
Frank Swasey wrote:
I'm experimenting with mdb to replace hdb on my test/development set of servers. I've built 2.4.32 and the server is running RHEL 6.3. This morning, I got a confusing error.
Attempting to perform a deletion of a member from a groupOfNames, the following was recorded in syslog:
mdb_id2entry_put: mdb_put failed: No space left on device(28) "cn=GroupNameNotImportant,ou=groups,dc=uvm,dc=edu"
and ldapmodify returned:
modifying entry "cn=GroupNameNotImportant,ou=Groups,dc=uvm,dc=edu" ldap_modify: Other (e.g., implementation specific) error (80) additional info: entry update failed
However, free shows memory left, df shows filesystem space left (in fact, it is only 22% used) and data.mdb is
-rw------- 1 ldap ldap 1547292672 Sep 8 06:22 data.mdb
while the maxsize parameter is set to:
maxsize 85899345920
So, I am confused about which device mdb is telling me is out of space.
What have I missed?
There are only two places in libmdb that return ENOSPC. Both of them imply that there's a bug in the library. It would be helpful to run under a debugger with breakpoints set on both locations, to see which has occurred.
Today at 4:34am, Howard Chu wrote:
There are only two places in libmdb that return ENOSPC. Both of them imply that there's a bug in the library. It would be helpful to run under a debugger with breakpoints set on both locations, to see which has occurred.
I see that is in mdb_node_add and mdb_update_key. If you will provide guidance (I'm not versed in gdb), I will see what I can find for you. I have preserved the environment, so I can do this debugging repeatedly.
Frank Swasey wrote:
Today at 4:34am, Howard Chu wrote:
There are only two places in libmdb that return ENOSPC. Both of them imply that there's a bug in the library. It would be helpful to run under a debugger with breakpoints set on both locations, to see which has occurred.
I see that is in mdb_node_add and mdb_update_key. If you will provide guidance (I'm not versed in gdb), I will see what I can find for you. I have preserved the environment, so I can do this debugging repeatedly.
Attach gdb to the running slapd, set breakpoints, then trigger the bug:
% gdb <path to slapd> <PID of running slapd> (gdb) list mdb_node_add (gdb) break 4952 <-- line number of 1st "return ENOSPC;" statement (gdb) break 5355 (gdb) continue
(Use the correct source line numbers for your source tree.) Trigger the bug. When gdb breaks:
(gdb) bt full
That's all.
On 09/08/2012 10:04 AM, Howard Chu wrote:
Frank Swasey wrote:
Today at 4:34am, Howard Chu wrote:
There are only two places in libmdb that return ENOSPC. Both of them imply that there's a bug in the library. It would be helpful to run under a debugger with breakpoints set on both locations, to see which has occurred.
I see that is in mdb_node_add and mdb_update_key. If you will provide guidance (I'm not versed in gdb), I will see what I can find for you. I have preserved the environment, so I can do this debugging repeatedly.
Attach gdb to the running slapd, set breakpoints, then trigger the bug:
% gdb <path to slapd> <PID of running slapd> (gdb) list mdb_node_add (gdb) break 4952 <-- line number of 1st "return ENOSPC;" statement (gdb) break 5355 (gdb) continue
(Use the correct source line numbers for your source tree.) Trigger the bug. When gdb breaks:
(gdb) bt full
That's all.
It is the one in mdb_node_add. Attached is the script captured gdb session. As you'll see in the capture, the line in mdb_node_add is 4806 in my source tree.
On Sep 8, 2012, at 1:34 PM, Howard Chu wrote:
Frank Swasey wrote:
I'm experimenting with mdb to replace hdb on my test/development set of servers. I've built 2.4.32 and the server is running RHEL 6.3. This morning, I got a confusing error.
Attempting to perform a deletion of a member from a groupOfNames, the following was recorded in syslog:
mdb_id2entry_put: mdb_put failed: No space left on device(28) "cn=GroupNameNotImportant,ou=groups,dc=uvm,dc=edu"
There are only two places in libmdb that return ENOSPC. Both of them imply that there's a bug in the library. It would be helpful to run under a debugger with breakpoints set on both locations, to see which has occurred.
I'm seeing the same error since I upgraded my test server to 2.4.32. CentOS 6 x86_64 But I have it in slapadd. The same ldif file works on 2.4.31.
# slapadd2.4 -b dc=ogilvy,dc=com -l all_slapcat.ldif .################# 88.75% eta 02m42s elapsed 21m16s spd 386.2 k/s 504f7987 => mdb_tool_entry_put: txn_commit failed: No space left on device (28) slapadd2.4: could not add entry dn="cn=testUnixHost1.ogilvy.com,ou=hosts,dc=OGILVY,dc=COM" (line=4000722): txn_commit failed: No space left on device (28) _################# 88.80% eta 02m41s elapsed 21m16s spd 616.2 k/s Closing DB... #
# df -h / Filesystem Size Used Avail Use% Mounted on /dev/sda1 253G 15G 226G 6% /
As a test to see if the ldif file would be corrupt, I loaded it with ldapadd. Created then a new ldif via slapcat and tried to load the new ldif with slapadd. Same error, same entry. If I edit the ldif and remove this "bad" entry, or an entry before or an entry after, the slapadd works.
I can't create a trace with the steps from the other reply. It exits with code 01 and there is no stack if I run bt full.
Do I have to set a different breakpoint to get something useful?
If I remove a bunch of index options from my slapd.conf file, the slapadd works.
-- Marco
openldap-technical@openldap.org
-
Francis Swasey -
Frank Swasey
-
Howard Chu -
Marco Schirrmeister