- openldap-bugs - openldap.org

Re: (ITS#7710) contextCSN values not updated by internal non-replicated ops
by michael＠stroeder.com 06 Oct '13

06 Oct '13

I've updated the test data with LDIF files and a README describing the LDAP operations to perform: http://www.stroeder.com/temp/openldap-testbed-its7710.tar.bz2 I consider this to be a serious issue which could be the cause for other replication issues including dead-locks we're experiencing. (Maybe there are similar issues in slapo-refint since dead-locks in our deployment arised when moving entries into different subtree.) Ciao, Michael. The README copied here for direct access: ---------------------------------------------------------------------------- 0. Start both servers by invoking start-slapd1.sh and start-slapd2.sh ---------------------------------------------------------------------------- 1. Add test entries on first server: $ ldapadd -H ldap://localhost:2071 -D "uid=diradm,dc=example,dc=com" -w testsecret -f 1_ldapadd.ldif adding new entry "dc=example,dc=com" [..] adding new entry "cn=replicas,ou=groups,dc=example,dc=com" You can now see contextCSN value of first server on both servers (as expected): $ for I in 1 2 ; do (echo ldap://localhost:207$I ; ldapsearch -LLL -H ldap://localhost:207$I -D "uid=diradm,dc=example,dc=com" -w testsecret -b "dc=example,dc=com" -s base "(objectClass=*)" contextCSN ) ; done ldap://localhost:2071 dn: dc=example,dc=com contextCSN: 20131006154300.921415Z#000000#001#000000 ldap://localhost:2072 dn: dc=example,dc=com contextCSN: 20131006154300.921415Z#000000#001#000000 ---------------------------------------------------------------------------- 2. Send a simple modification to second server: $ ldapmodify -H ldap://localhost:2072 -D "uid=diradm,dc=example,dc=com" -w testsecret -f 2_ldapmodify.ldif modifying entry "uid=michael,ou=users,dc=example,dc=com" You can now see contextCSN value of second server on both servers (as expected): $ for I in 1 2 ; do (echo ldap://localhost:207$I ; ldapsearch -LLL -H ldap://localhost:207$I -D "uid=diradm,dc=example,dc=com" -w testsecret -b "dc=example,dc=com" -s base "(objectClass=*)" contextCSN ) ; done ldap://localhost:2071 dn: dc=example,dc=com contextCSN: 20131006154300.921415Z#000000#001#000000 contextCSN: 20131006154406.940154Z#000000#002#000000 ldap://localhost:2072 dn: dc=example,dc=com contextCSN: 20131006154300.921415Z#000000#001#000000 contextCSN: 20131006154406.940154Z#000000#002#000000 ---------------------------------------------------------------------------- 3. Modification of group membership on first server: $ ldapmodify -H ldap://localhost:2071 -D "uid=diradm,dc=example,dc=com" -w testsecret -f 3_ldapmodify.ldif modifying entry "cn=testgroup1,ou=groups,dc=example,dc=com" Now the contextCSN values differ: $ for I in 1 2 ; do (echo ldap://localhost:207$I ; ldapsearch -LLL -H ldap://localhost:207$I -D "uid=diradm,dc=example,dc=com" -w testsecret -b "dc=example,dc=com" -s base "(objectClass=*)" contextCSN ) ; done ldap://localhost:2071 dn: dc=example,dc=com contextCSN: 20131006154449.514135Z#000000#001#000000 contextCSN: 20131006154406.940154Z#000000#002#000000 ldap://localhost:2072 dn: dc=example,dc=com contextCSN: 20131006154300.921415Z#000000#001#000000 contextCSN: 20131006154406.940154Z#000000#002#000000

1 0

Re: (ITS#7710) contextCSN values not updated by internal non-replicated ops
by michael＠stroeder.com 04 Oct '13

04 Oct '13

Hmmpf, the attachment did not come through. Download config here: http://www.stroeder.com/temp/openldap-testbed-its7710.tar.bz2

1 0

(ITS#7710) contextCSN values not updated by internal non-replicated ops
by michael＠stroeder.com 04 Oct '13

04 Oct '13

----==62f1203288d98ad3817191f9b713342b Content-Type: text/plain; charset=UTF-8; format=fixed Content-Transfer-Encoding: 8bit See attached a canned MMR config. The behaviour regarding contextCSN is really strange. You have to play around with some more modify requests changing the group membership of e.g. uid=michael,ou=users,dc=example,dc=com If contextCSN values are out-of-sync restarting one slapd brings them up to same values again, but the older(!) ones. Ciao, Michael. ----==62f1203288d98ad3817191f9b713342b Content-Type: application/octet-stream; name="openldap-testbed-its7710.tar.bz2" Content-Disposition: attachment; filename="openldap-testbed-its7710.tar.bz2" Content-Transfer-Encoding: base64 QlpoOTFBWSZTWS/IcOYADFR/tf7wKIR9/////+///v////6AQAwAGABAAAAIYCFvtXi4dW2a669a 8s1s1Rhe5yWddW7l1yqUkKAAbndwwp7MoLnuA0NGQQiklFraDW2yGgstttdd3rg600iUBSNY9UNd wkkITRpponop4mCamyNJpk0AaaExppqbU9RtT01GRkehqaaAaCRIAQhEYmTKp+TUyJ7U2qeaT1R6 nqPU00NAAAGgyAAAJSNEUPSHqPUDQAAAGgA0AAAAAAADQCTUiTKCeU8gT1JtE9IMgGjJoNAaDEBo A9IyAABocAAA0DQ0NDTIANAABoBoaAABkAABUkIICMQI000Enk0Jpkn6kw1Mpo08oeoNAepoAAaN A3uEH6CD7VzHMcwP95Q4z5/a/W+muViqEBA9LoELQ+yG5pE6QKpVVFoqkKJK5SAA91fdED90pwcj ETpHul0pWt3UnWxSVZXEfwDXYhmQxOBCoGLpYKVKk0rJDc2ZnDIqQ1GnDh6nMcQQEjCaq/CsnDpe UrShWFkREW4QMQZAEWQA55AZJCMqJJIIEDGqhBCCESHHIJzoHAnHqALsFkiISUw87bDPpM9dPSXQ y7v3nuQvnO5desdcbdm/ClPly8QLLcIX4EMjg8h/F8IDO24uMg2w/jVN4wlUfHBS1j0ECUvmUqVP QVDEXTUGGPgT0QsesuFc5zQaRpCLjDMEwNZ4BT/Jp/g+x0Wo7kxeHqYJoZrB4yBnDnDUJaS+Th9Q 4xhAypWBkMyAwptA9lq3M19qIP0Ra9xtGiPblilklhUlrmqMYOo54VKqSqqU0z+z9h9bvKfIeLcw e9r2K9bDkfDp0GoZNkWFkhi3S6x0Vc92K0oWN+jdf8ycsSz7RsESk2X1sLMLeqz6zZY43M17VXVq 1pdiXfUtZ3qUHm7JHmbTLDoaiHU7ivVlqqFYPqPmOZMUCYV+nad+OR1kJmm9MoXTaHGxnMesLYbL E7YZoYg2YDi62GjP4tqcyjk3+d0esUKmTJ5YPQdiw8tpZHpq/xkseLVTCnBuPJsdfZv4PPJ7B9wP rMHI5Z1nTG0zaq62mMr3+j3VXxFToMe5w5HpZIu9azi6ts7H54hREOR8bNKEkBIHYiSSCh6BJw2D S90wlAnbNgKAVDUon5lI+NnqeazJPpUu53rh+XV76v9ntq/L8XRjfuxJWuuilkxxR6zo6g7ssFPf IN10C4u6mGEJ1IpUxE6Sv1H+0qeS8hhK7XTcw7/sz9OefSkg+SwRqYZGoyJrGOHOo5jrm8QeKQDH wo2ex0aS5cpUt9PjkonyYPci5mmx8oeVfD6vKQ4foVmSqk8x3cMsUA3KyCgPYZBxQPJzsZjrHgNT Qasr/CMJ0HK76MW9Tm8V5K+D4LnlZZvV62RsLx/v3Z+HzZCkeRk8iDgplhScJ1gUz5bHh/ScFKr7 zDQxa0X5ky9fcbnL1yO8z4C6n7zmR28l3mxIZ7PSrEWZo2+Nn238PuM2TfRuN/he+/wp70fnqTHF weVXK2OqIlL816UcywkMAWxzB28o2mE5rG4i3bnE0MhdkMA5Iay+SuUp4Ul4ifpWykvCvAsVUt4A KNI/F16kei0SZc5Qd5Q6/6yQHu85v5g+qm+qZvm+O/p3dA1y0WhjKTF+eaILxbdQWU4fqKvAwERa JsS8KqBjxXfZucXYOuL4rKd9nm9Tg47a7uW/t5JlOztW2KldoIDIdXBf+Sxt1dR1s8Yi8Q5czzTV z0nefZem/iBYMHQ6HHUUiZMkaML1JLrI2fLY9vQXWZ0ekcPouOg/1lbMWdJYyXTfgUyIMK2QyQrk UQxE9Jhi28oQyfHdg47lDXkfIUqdhrGlhniPUu8qVx+USNyTs3A1hi8ZApK2nVqsZ5jExSVe812B OwaxUMRVtNYvIkbqDz9nrJlD12I4VhsO5IGxR4S/QimGZWxuWKLb8njUn3cFV7QoS8usdnk5h1HM 0fKzkYgNJiL9BSKXJLsK3+0cygeO5iJQ00+ZevPw9F6LU2eza8TqZvmh/pfk+7XXi+pc6HiUWet6 L1VP4LLKbNvm62G+jgj0UooUoUk/sE/+KRVIlUiRSpFUI6xE/C0ksRKGCRzzS0u97Xv2vX4V5393 oszeWvbkx5tmjb2eSCVIjaQ4H92x4H5hol43mrlI2t+TVu36sGezuBoADuG9qzbLuZVuJus6BbLd yceDbnLIxENweE4ygKCA4AXT9k1V1ie4WTzyiG4OuebNNnW1csjzsDcMcheIKmKD9508/BJPY2T5 DFZjxGkJYTKnHQJ3GW0LDvGn0d6mTvXWehTySR9ck/Goefn8W50YaKKU+tZub2i+x58bQq1UrNqb W4zND+lY6jR1u+hLqpLm5eWI5PXBcsZHhxhMCM4rJ6i8Ub1I/xBVPNeOQA85weMEekxAgtBg2RwT 0oBbSzCSvyfBm3SfhliBXv5P/J1ON21NkE6nxeDNtVOXW5lq2eR/fMk1nRN/p3PGvSk6kgVKQij1 AIVTaEKCrhCs1jCQGpASQvG9M89owOrReLSJ0gBEl1iClRk0hFG3DwxIiVgLZAQtVhzSLI2a+kq+ cOUetrAwwVdw3zHefbgzUySYOGo5zk7caSlq0c5E6pKzTqML5hMEUcVPCL1gkTXyscVzGS+3RKZe 6+uWLWXVgXANgK0gN5DMI6r42TJqVGcxUIXEJjXLxUuIUHiLoG5sBuUskbIYNDH2GRyN4IIiEUXi hQbt/Dic8kN8Pdd099yGUGKqymFXIkPE0aa4PVMDWZiISGAdjrn7BbObBQis4WBk2Zla80XkNEGR ZEWMFIKqqjzyG9LopmRI83WWbNLESKaEP534V0h5G3+OpxDyrZHlTn57VXbO0e/k4zpepraSD3pT CBkhyQefKdCFvDorirqw9DVDT0vzWmvqas0wIGixYlihRBUJECUDgcGeIKHjawys7tDlbftNhOhM 30VFD30YGnTJJFgdnAUyZUhMpLm5SGmjN8NcbrbkpvYbGnrPD53xy3nmPZjRowe3bNf4pRn416zZ mJYsuC/LqSCEkHFXEUSOVQm6m/WOTnJfVsNdOvGTgwRAkh6OxR1GCg5kMJfiKYyYvMEWP7hmJWrg kkFBCzyWOqNtjOfR+MLkQkWJGxZmKoBX3EMnHhIYrOQ1edltrDqrDFMLdmnoxwU1aEHDUhmxAumC DTodOvHWYCjbFxce8a80+VKWpj0c129vQd6y7jbu+kL3V7Ajl5pHUNgYKo4dJOuanVVtuqyymxXI qdjdIWG+hno8fV3RGXcztqiaouiYYuutfLKrv25DfLrIJzGxBV5oFFZUo8lihTZwO0T4FItqETw6 JSaUMIt+0lKjtMqa7W5jyx2wTtqhqXYVQIz9Juv5k0kNJbT8qq/JLRAaipJMkQkckIggqIGOfLnH O4FpXZYmYJjIcjpTJOlultcmGuJE4KbtS21iyFzBq0Ql2VAm11XW5nG94MpIVAY0aktVxTLcdRZA jO2WWNkE4HSmwTMMqDKgMm9piUKgyRsCDZIJKuM0K4apahMkymWkEj0mLVxLOJk5+dA9bN1VKBfj aeBTrfEylY4X3lUmkHCEEHUZUg50VDQjRuWgzsCIUuMEgMsApE3G5orRWktTCvKhIkGzCnAj7aaE csHoDFeNiCnFM9IdNuR0JdCwCLIRzGlnoKCxyFQyVIbOKrd7FCk5YiqqTd0gvaVuDWcbsqrJBc0k +CgyzIrBYCTdsVnOLmRZleouMWuINgW+GSA1SpMvJAKhGZOlCD9tVglWICsViSkCBhfjMwskyp4o BdlrUL7EHO+rJU5pVS1WnOJ830NByKLcWA5oBP1M344CXDLWJ1hkyALvROFqlxYnNtAoaSKWr24v yPVjayXOutHMpPNtkm9yshCkkxEwaE7Vyr9BQjkFVSeD2lYI8QuPm9PSDtt3OAHDh6YYYHdvdoPj 5xe0wlhqJdLZqt0Qmg0hydn1OtIpdd2ruK0URUeacJ7HESHVvc1HJOmfYZFzaBE1FE6NBS7sf3V3 8zOQmUd1SHJJGS0skhkkK5KZ98oYCxCLUIFWZNKRQhchfwglTk0jZC6HMD5LnEPPcoH1982w6aS9 oA8Qj4CnSqCPkSxrtnXXgFD3JI9STrPKfcPHHIJxdNo8sF0yIKBN3uuONRCZZxlXyArYJXMHQc4P DxG6fqHVIsm4JFJXDTKUlWGlS/ZFJ3DMsaDIMhMy3+kTUVWioqnrXvd1xPaiKjeHgdG85f0HHiUK r59IyUSVvjg4/ofefVlzVwfUdpw9Gk25Hr9DS4Y2GHF8ENjdTf3I4tZVP4nd4KWXOnsPzI3HpD/Z meE3P/DhJNhycq/fyjYc2g0DzADDMytBnsjCgaL4YwC+IiSXCeYcB9719G09r1vxPofRVU2Ijktc 7p09p+mVVrXtVH2t7j+Mm6O97bW977+JK4yS0Tkd5HxN7/WiILhX9MyxuDGwbVjle6Fry0IIgJgL GVc+sECisbEESYKQ0B1LhGsgtU1Sus5QiwSKZ2J+Wwd8HxD1ZN2MiNtJZFiD5g4M6M5xCCwY6w29 OY69p0kqWE96HX1z5mxf7soqOSO3ZVT7JymrjR3KbCT5HjH0ipueavidRqZ5qRSy0sUqKlHHgYuI YtnQWnzsdUoLNSBrBa6RQsqSGG7RS8pWhqxiSenexUpOSdfHnR1aOp/TJPj8opUjwdUk9W/7j4uh Oqdbvr4B7p7DRg0b1RHcsjks+1yGUldUkrrkjBq5OD+JLo/4v2v3p8D70/YfgQ+ZH208tvwvLFqm KRdap/QqIwJ6hr6/+ePcq6r4u+ja+ZKfe9yxZZKWMbRXBuwVzfFujHQPOwbAjnBMDAyrGynCB5Lo pwl8Co1NdpwD0l5xpyKmNcFhke0nEicuyFSD8ZUKKUEFEfNqZ67H4Q4F0S1uBGxUoe8NtEUEEgrB EFhGhDdkkM23BxtSqoNl51c2srr/QO9kiomumRax4wSdiDyXdxDXGk5vJnJ+14nrZ7BlZt6/RiTp afC/LqDEU2anVJDP3t0qL/Gjpuzem0hfsRPJD9LLvrE5c+WW7QtOLeh+5o+CnNTjitrylzDK1I07 pJKWp+xkLeNePuKvoaDvQ+3uL9DqSiqVSdp5qT6FkHedSCx1DCUc1Urx6f5x7E9rKJxgmBbjJDcz yfbKBuNL7/3If1pVaGhA1kS5rqKkAWi11BsT6xhKRTmYN1To6VyFYxUWkki35XHFxSjWKl5BhKKF rSMPIPTSGqGRp/L49rfl03PoZmUiflqqkitJsvvK2HC/WKubUYWuqku4Q0k4aSLTcnyWn7lCFFUS gqINAiihd4iQ0IQYAekUsX7qjAofaqOth2SH8OttH9TsarpiNXNdeQWVDsUmSGLE7FZJopjEuxf8 9Vi8d6vpaFozsllpzMsFYyVdMlgzINQUMUJExoY8DrggZcEQgFSCTA9QVldMRYiG7LsGQA7nySJJ Uy2aQEnDmGxLPQr18T3KfJ8pp6pJGgpKpPUUopG0plvcnvh0UqeqntYm1PCp11rO3td6yOC6GvbU 0tkvbCJLu41Q356FBkwvoUy8m9j1uwzN6pyN22UoeBgfw0SqObiYMzrv1Ib5kdukOLg4LcCnqVPO seXhKcEvliakCbSDElIaUMICDxOTOYU0SqbHbQ9xMzOUhla7kYjgkZICGMyjibghpnlery9Q9QaN XkUJBwTktnZYZY0cuIDjMFqjeWmSgSQw0OoUMzfFSs+nIrKaFnGsi6imUhLmWF2Gla1Ly7Gqt7Eb oq+UH/dCj9NKjVZrms3VyGUn9SEulfCOzhl1cVr1Gtra0uplHvZMnbBNJox9aXyegkO3ajYmEH2L kbnzptp0KYRoUgk2YOZ6eX0Dg8/PphOEblbXM4E4OkRP312kxHjNZy863N3hbNn29GD4lTIdHU7N BDiOlJhbS8tRZvW+wk19Y0t/NSh0DIt0xMbdhNh7CEUEF7w3sQQvIHxQ6qbcEFWQXFcmTbdLSkwO TVchkqIxJqoWZsM0qM02P8Dzdz+Nu9zVum4lbv31BTOOCVnIUoyQ/U/OcQG22+lSqO6D2Skvz6yx O1oIPkLz7waqu1FA7lJKVIdbIg7JUZeSi2d6W0fEbN5lozkk2ZPF+P8V19D1tzlSTHySRsKbXlHV 02qpSnQ+tXBRJd6uD+NPNqw9VnsK66GqL9G5iuguFCR8w0uyJEvpaqNBPG4Kq+Magjky4J0rlQmT JgQqFi0mWtJkYEqFlbb8IhswVoj4zR9BknUoVmXD6FBtIn6AhI9MTDjKKp+VmbR2HtiUpsPSpaU1 qWqQuIJQHZPSL2KqVUQLwSQnwDOk4By2a1Sk6u+J0PpfF7G9no20o8ZctU68Gj6Vp20zNlUXiHxL UqtotJ9fyxL4O85fyYm977bbPpqLKLqXKWejinI3OLY0ar930NudWHHerI/kU58BbMossmjicqc+ OSfB6clzm2RE7Ep03HwtJpuNLK983OY+6Rp8mFwVYPwcGzdsMK/l+qDt6t9HSGMaFwQHnzMN+ejn S8dqOHwTsWJbotp4tjaG5Ukfh6IdJoNcQ8s0G75rFkzmmIIKpliZt3XshjgCHBwue9E50DKyihAZ AmztAkpjR4GUmQG17YJiNJLs+8zbihxSx1xE3uxkhuDwMJQpOgZ5mtIxM40k1lcPMKffqls0qq2q lVV5GzzTkYb5nLeCYaCxKmmoisuVOmlsupZouzbjES0qG1vqVX4jHzKZoNdkj9xSkPcs/J0OHpRs Kar+tVrux4UqRKkknfSqlfkT7WHhPqNe5vnqPU9fUszKYUpYtelzJJFKjqkYJaKWUV4NFl1wFiyX PYoLrJlg46XqLj7CEsJoORcS3Pgpz6ngjkodr7nsWeyTpOHWp1u5kwdsjJ8Msu07Ei1aLLYFrJFP a0XkkXdkROuPtV7cpj0xsUkzVL0zGELqpKFHpLn3NqGtSb1BpV4l5CUpKWybHNLFxnsSqqvSvGqn FRiZ+DLIXavDBfWbi+VomHBW4kw3GewGYRxkYkho4CQ+dGj2yBnhLECyqtUZBqNHQ2A8Bhssdqpq DgosCXrOS0lhGqIljeGxZUKmJmL3xCIgMVGFJJDQ0xJFEK6qiGdkKA6xxxkPdT9Rh+zUub5D7bJK cL2Ww20wvLrjzC5gGEWgnUBm2MGRE2mtRmJQbbKCYgqSSLMSMIgcaYYXZE3Rdn4VIFgCGUDbCF44 U2PdK5bcTZTmpaSk1FnE1goUREopgsUVEYIpdmeF4F5ktilIqo9yyzCmdOdJtdpbNxkUfY2cYAM6 8shAwAbgMLeGFjK8YxlEuoDJLUUznrbpm1OX3Pd+PCrS0hoSGSjOJNST6UvfM/Sy/op59zZJFk0g DXvJ4QHFGNDe4e0w5RaMoSipQiCUUVFGQWcRxBkYI6EEyQYSBoBTtBR4gxCUMMsjEK0UukSdO0f0 TtPtd/v2MslqVS00Dnt1m5gOAWqQN4znSLku6k8pYqT9fnf8yMdPVMDSZMptUuljTSjveKz8jb/K sj5PplRwqJVSq0ab5tI3SSYlDNoUpZ9J+JU37X62TE3vtf2YqSQ/LKSKKHMpZHxcPUOyLPv40tKa IPpSG5r8n81xDSlQ9VZ6tXsdrhhHzbzikT0omhjIX5otTkQch/PLqRueqntyo7pR4iGsX8wVEonR TMGXeWtwqq2ySuctEcCGiWCCiKCIixhVBFgEmMlzhosTbZOE0tvTJzHNBZh351dZF15bdeYUWXN6 TjTg8V3Y1ZaJbidR1tDOlc82ad2M41VqpheRZEz4ccxjYuYG1TdICLEHz5YoaMpbHuR2xZgmSkio qM2ZIWKb51OsaO5vaPVJJglvzzsWkRtddiKFXiRMZM5td4qpchSVGEEQLJoQUXlrwnuTMVbmThYp 4GTrrJBM+I9ebroPy8r7I8iFswsaUHIxMRzORDq9zNd3XAmfAgCk8wZOcI6Smm6lWUsN+dKUY3mh hqya5obb6REwZIb2brTZhlNVp+x3sY2e47qO+ke1ZayrwRJd7LHn0+iTWYBQ1Q9ZpE+bR2MqfAsP 4zC5t3S15FVJH6bfRzWzVTsczm2MnKKbSd7h0YTryLcjPBwOUbmc6bJeL3vLxd+thSnWhfo4+DGH UriFTI4jrcEUqSgpCnNVi6yFsFIpSutyYKsusQ6jVwGcEzZ2go0LS2XghysQ1K6TETdM9I0cWj/M anWdhiWRxLFylKNJNcm3aXS1p7Jzq9UUuWTrUddsLqV+HqrIVMKPOM8xPWVUKaem3RqpMwWsEgoT DyJjIUqRqka5rKYUXUrEicdt2MiH9woyvLIvefrxGIJkNENYXbzcm5QgemizRcXUarxQKC95gNfd JhkZc0XWTwQCfmb70KEcyqSUGg4kmXHgkEDGlQJA0aIOwoELYwJgfaLHgpiylEDhw2pVLRGkREWr GGY2oBcyZJvjMkMAY3IeyuldHaVSIJmlkKgBzGVFwzQvYdxpO43PaZIMXei1USFl21eXk2qOpuY3 XF4JyxHE+DJB3tbaI2SSbaqbLOpJPhacFq/n4J2pJRU6yx6ZHU4O1dg0WuJwTq9qHlB5dHnxL0q1 QpO8T0iCgvsAxuKelSSjpnM5SdNh1z+bPe9r9+WcQ8HfLJM080admRETASmQdsMapgTv+EDPWZB4 WLRpjir9AhvXwg4RK9/osyUp7KLPQw+XF8GqlZaKnzWFLWYxY+BavmXWLsntXw1lUKniiyZqNDkd hdClzDaVTrksqjWhcvEXp+ZiSRa6xejm6/PxX0ezy07SdEZQkyWJgQSQQMungKBWqhlmigVRBk3T 2nNCvYtTYf22oKo1ZbGy0ncexzDFIdmjfO+d07XcrrjxhRpBNZ6Zj3nZPtrvLxnbOShIX/i7kinC hIF+Q4cw ----==62f1203288d98ad3817191f9b713342b--

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by hyc＠symas.com 04 Oct '13

04 Oct '13

Howard Chu wrote: > Željko Nejašmić wrote: >> On Fri, Oct 4, 2013 at 10:26 AM, <hyc(a)symas.com> wrote: >>> >>> Thanks. So far the output shows nothing out of the ordinary. Can you also >>> provide the output for: >>> ls -l /opt/openldap/var/openldap-data >> >> root@spr101:~ $ ls -ahl /opt/openldap/var/openldap-data >> total 100K >> drwxr-xr-x. 2 openldap openldap 4.0K Oct 4 07:54 . >> drwxr-xr-x. 4 openldap openldap 4.0K May 28 08:49 .. >> -rw-------. 1 openldap openldap 88K Oct 4 07:44 data.mdb >> -rw-------. 1 openldap openldap 8.0K Oct 4 07:54 lock.mdb > > Something is flaky with your installation. As I wrote in my previous email, > LMDB always sets the file size to match your mapsize, to avoid this SIGBUS > issue. Your mapsize is 38654705664 (36GB) but the data.mdb filesize is only > 88K, it should be 36GB. You must have some other process running that is > truncating the file; there is nothing in OpenLDAP's code that would cause this > behavior. I take this back, this is a bug introduced in liblmdb 0.9.8; it is only setting the filesize on a newly created DB. If you had an existing DB and added writemap to your configuration after the fact, the DB size wasn't getting set. The fix is simple, and is now in git branch mdb.master. > >> root@spr101:~ $ ps -efa | grep slapd >> openldap 44012 1 0 07:54 ? 00:00:01 >> /opt/openldap/libexec/slapd -h ldap:/// ldapi:/// -F >> /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap >> >> root@spr101:~ $ mount | grep sdb1 >> /dev/sdb1 on /opt type ext4 (rw,noatime,nobarrier,data=ordered) >> Do note that we did load testing on ext2 and ext4, with both being >> tweaked and not, and as far as I recall, every combination of >> parameters causes the problem; I will try to reproduce it on >> ext{2,3,4} with default mounting parameters. Almost forgot, we did >> tests using tmpfs also, with the same outcome. The whole /opt was used >> as a mount point for the tmpfs. >> >> root@spr101:~ $ fdisk -lu /dev/sdb1 >> Disk /dev/sdb1: 400.0 GB, 400027127808 bytes >> 224 heads, 56 sectors/track, 62284 cylinders, total 781302984 sectors >> Units = sectors of 1 * 512 = 512 bytes >> Sector size (logical/physical): 512 bytes / 512 bytes >> I/O size (minimum/optimal): 512 bytes / 512 bytes >> Disk identifier: 0x00000000 >> >> gparted doesn't seem to be complaining about SSD block alignment: >> root@spr101:~ $ parted /dev/sdb1 align-check opt 1 >> >> Another thing I'll be trying is making a partition with different >> alignment parameters to see if any change occurs; tried using a >> different scheme, but that hasn't produced a a change in behavior. >> >>> >>> Note from the mmap(2) manpage: >>> >>> Use of a mapped region can result in these signals: >>> >>> SIGSEGV >>> Attempted write into a region mapped as read-only. >>> >>> SIGBUS Attempted access to a portion of the buffer that does not >>> correspond to the file (for example, beyond >>> the end of the file, including the case where another process >>> has truncated the file). >>> >>> LMDB always sets the size of the file when using the WRITEMAP option, so the >>> SIGBUS condition should never happen. The pointers in your gdb output are all >>> within the bounds of the map and mapsize you configured. >>> >>> What filesystem type is the data directory stored on? This is looking more >>> like a kernel or filesystem bug than anything else. >>> -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by hyc＠symas.com 04 Oct '13

04 Oct '13

Željko Nejašmić wrote: > On Fri, Oct 4, 2013 at 10:26 AM, <hyc(a)symas.com> wrote: >> >> Thanks. So far the output shows nothing out of the ordinary. Can you also >> provide the output for: >> ls -l /opt/openldap/var/openldap-data > > root@spr101:~ $ ls -ahl /opt/openldap/var/openldap-data > total 100K > drwxr-xr-x. 2 openldap openldap 4.0K Oct 4 07:54 . > drwxr-xr-x. 4 openldap openldap 4.0K May 28 08:49 .. > -rw-------. 1 openldap openldap 88K Oct 4 07:44 data.mdb > -rw-------. 1 openldap openldap 8.0K Oct 4 07:54 lock.mdb Something is flaky with your installation. As I wrote in my previous email, LMDB always sets the file size to match your mapsize, to avoid this SIGBUS issue. Your mapsize is 38654705664 (36GB) but the data.mdb filesize is only 88K, it should be 36GB. You must have some other process running that is truncating the file; there is nothing in OpenLDAP's code that would cause this behavior. > root@spr101:~ $ ps -efa | grep slapd > openldap 44012 1 0 07:54 ? 00:00:01 > /opt/openldap/libexec/slapd -h ldap:/// ldapi:/// -F > /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap > > root@spr101:~ $ mount | grep sdb1 > /dev/sdb1 on /opt type ext4 (rw,noatime,nobarrier,data=ordered) > Do note that we did load testing on ext2 and ext4, with both being > tweaked and not, and as far as I recall, every combination of > parameters causes the problem; I will try to reproduce it on > ext{2,3,4} with default mounting parameters. Almost forgot, we did > tests using tmpfs also, with the same outcome. The whole /opt was used > as a mount point for the tmpfs. > > root@spr101:~ $ fdisk -lu /dev/sdb1 > Disk /dev/sdb1: 400.0 GB, 400027127808 bytes > 224 heads, 56 sectors/track, 62284 cylinders, total 781302984 sectors > Units = sectors of 1 * 512 = 512 bytes > Sector size (logical/physical): 512 bytes / 512 bytes > I/O size (minimum/optimal): 512 bytes / 512 bytes > Disk identifier: 0x00000000 > > gparted doesn't seem to be complaining about SSD block alignment: > root@spr101:~ $ parted /dev/sdb1 align-check opt 1 > > Another thing I'll be trying is making a partition with different > alignment parameters to see if any change occurs; tried using a > different scheme, but that hasn't produced a a change in behavior. > >> >> Note from the mmap(2) manpage: >> >> Use of a mapped region can result in these signals: >> >> SIGSEGV >> Attempted write into a region mapped as read-only. >> >> SIGBUS Attempted access to a portion of the buffer that does not >> correspond to the file (for example, beyond >> the end of the file, including the case where another process >> has truncated the file). >> >> LMDB always sets the size of the file when using the WRITEMAP option, so the >> SIGBUS condition should never happen. The pointers in your gdb output are all >> within the bounds of the map and mapsize you configured. >> >> What filesystem type is the data directory stored on? This is looking more >> like a kernel or filesystem bug than anything else. >> >> -- >> -- Howard Chu >> CTO, Symas Corp. http://www.symas.com >> Director, Highland Sun http://highlandsun.com/hyc/ >> Chief Architect, OpenLDAP http://www.openldap.org/project/ > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by nejasmicz＠gmail.com 04 Oct '13

04 Oct '13

On Fri, Oct 4, 2013 at 10:26 AM, <hyc(a)symas.com> wrote: > > Thanks. So far the output shows nothing out of the ordinary. Can you also > provide the output for: > ls -l /opt/openldap/var/openldap-data root@spr101:~ $ ls -ahl /opt/openldap/var/openldap-data total 100K drwxr-xr-x. 2 openldap openldap 4.0K Oct 4 07:54 . drwxr-xr-x. 4 openldap openldap 4.0K May 28 08:49 .. -rw-------. 1 openldap openldap 88K Oct 4 07:44 data.mdb -rw-------. 1 openldap openldap 8.0K Oct 4 07:54 lock.mdb root@spr101:~ $ ps -efa | grep slapd openldap 44012 1 0 07:54 ? 00:00:01 /opt/openldap/libexec/slapd -h ldap:/// ldapi:/// -F /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap root@spr101:~ $ mount | grep sdb1 /dev/sdb1 on /opt type ext4 (rw,noatime,nobarrier,data=ordered) Do note that we did load testing on ext2 and ext4, with both being tweaked and not, and as far as I recall, every combination of parameters causes the problem; I will try to reproduce it on ext{2,3,4} with default mounting parameters. Almost forgot, we did tests using tmpfs also, with the same outcome. The whole /opt was used as a mount point for the tmpfs. root@spr101:~ $ fdisk -lu /dev/sdb1 Disk /dev/sdb1: 400.0 GB, 400027127808 bytes 224 heads, 56 sectors/track, 62284 cylinders, total 781302984 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 gparted doesn't seem to be complaining about SSD block alignment: root@spr101:~ $ parted /dev/sdb1 align-check opt 1 Another thing I'll be trying is making a partition with different alignment parameters to see if any change occurs; tried using a different scheme, but that hasn't produced a a change in behavior. > > Note from the mmap(2) manpage: > > Use of a mapped region can result in these signals: > > SIGSEGV > Attempted write into a region mapped as read-only. > > SIGBUS Attempted access to a portion of the buffer that does not > correspond to the file (for example, beyond > the end of the file, including the case where another process > has truncated the file). > > LMDB always sets the size of the file when using the WRITEMAP option, so the > SIGBUS condition should never happen. The pointers in your gdb output are all > within the bounds of the map and mapsize you configured. > > What filesystem type is the data directory stored on? This is looking more > like a kernel or filesystem bug than anything else. > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by hyc＠symas.com 04 Oct '13

04 Oct '13

Thanks. So far the output shows nothing out of the ordinary. Can you also provide the output for: ls -l /opt/openldap/var/openldap-data Note from the mmap(2) manpage: Use of a mapped region can result in these signals: SIGSEGV Attempted write into a region mapped as read-only. SIGBUS Attempted access to a portion of the buffer that does not correspond to the file (for example, beyond the end of the file, including the case where another process has truncated the file). LMDB always sets the size of the file when using the WRITEMAP option, so the SIGBUS condition should never happen. The pointers in your gdb output are all within the bounds of the map and mapsize you configured. What filesystem type is the data directory stored on? This is looking more like a kernel or filesystem bug than anything else. Željko Nejašmić wrote: > On Thu, Oct 3, 2013 at 2:57 PM, <hyc(a)symas.com> wrote: >> >> Željko Nejašmić wrote: >>> On Wed, Oct 2, 2013 at 6:43 PM, <hyc(a)symas.com <mailto:hyc@symas.com>> wrote: >>> I ran the whole test suite for mdb, and as far as I can see, every test >>> returned OK. >>> >>> Found a new way to reproduce the SIGBUS using ldapadd on Ubuntu 12.04 firing >>> to openldap on RH 6.3: >>> ldapadd -h 172.17.101.150 -p 389 -D "cn=admin,dc=test" -w test -f test.ldif -- >>> ldif file is the same as the previous ldclt command. Doubt it matters, but the >>> ldif file is 1M adds. >> >> Can you provide your slapd config and this LDIF? > > > LDIF file is just 1M adds of the following format: > dn: uid=XXXXXXX,ds=USERS,o=STANDARD,dc=spr > objectClass: sprUser > > I've uploaded the config here: http://hastebin.com/nicecafaci.ldif > >>> On the RH box: >>> - compiled openldap with -g -O0 and previously used flags >>> gdb `find /root/openldap/ -type d -printf '-d %p '` --args >>> /opt/openldap/libexec/slapd -h "ldap:/// ldapi:///" -F >>> /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap -d 0 >>> >>> gdb output: >>> bt -- http://hastebin.com/hefikekaxi.sh >>> bt 10 full -- http://hastebin.com/vudocosuka.sh >> >> Can you also provide the gdb output for >> print *env > > (gdb) print *env > $1 = {me_fd = 15, me_lfd = 14, me_mfd = 15, me_flags = 940113920, > me_psize = 4096, > me_maxreaders = 126, me_numreaders = 2, me_numdbs = 13, me_maxdbs = > 130, me_pid = 43974, > me_path = 0x8c19a0 "/opt/openldap/var/openldap-data", me_map = > 0x7ff6f721f000 "", > me_txns = 0x7ffff7ffb000, me_metas = {0x7ff6f721f010, > 0x7ff6f7220010}, me_txn = 0x7ff6dc10afd0, > me_mapsize = 38654705664, me_size = 0, me_maxpg = 9437184, me_dbxs = 0x8eed10, > me_dbflags = 0x8ceb40, me_txkey = 1, me_pgstate = {mf_pghead = > 0x7ff6dc109d58, mf_pglast = 44}, > me_dpages = 0x0, me_free_pgs = 0xaf75e8, me_dirty_list = > 0x7ffff721f010, me_maxfree_1pg = 509, > me_nodemax = 2040} > >> -- Howard Chu >> CTO, Symas Corp. http://www.symas.com >> Director, Highland Sun http://highlandsun.com/hyc/ >> Chief Architect, OpenLDAP http://www.openldap.org/project/ > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by nejasmicz＠gmail.com 04 Oct '13

04 Oct '13

On Thu, Oct 3, 2013 at 2:57 PM, <hyc(a)symas.com> wrote: > > Željko Nejašmić wrote: > > On Wed, Oct 2, 2013 at 6:43 PM, <hyc(a)symas.com <mailto:hyc@symas.com>> wrote: > > I ran the whole test suite for mdb, and as far as I can see, every test > > returned OK. > > > > Found a new way to reproduce the SIGBUS using ldapadd on Ubuntu 12.04 firing > > to openldap on RH 6.3: > > ldapadd -h 172.17.101.150 -p 389 -D "cn=admin,dc=test" -w test -f test.ldif -- > > ldif file is the same as the previous ldclt command. Doubt it matters, but the > > ldif file is 1M adds. > > Can you provide your slapd config and this LDIF? LDIF file is just 1M adds of the following format: dn: uid=XXXXXXX,ds=USERS,o=STANDARD,dc=spr objectClass: sprUser I've uploaded the config here: http://hastebin.com/nicecafaci.ldif > > On the RH box: > > - compiled openldap with -g -O0 and previously used flags > > gdb `find /root/openldap/ -type d -printf '-d %p '` --args > > /opt/openldap/libexec/slapd -h "ldap:/// ldapi:///" -F > > /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap -d 0 > > > > gdb output: > > bt -- http://hastebin.com/hefikekaxi.sh > > bt 10 full -- http://hastebin.com/vudocosuka.sh > > Can you also provide the gdb output for > print *env (gdb) print *env $1 = {me_fd = 15, me_lfd = 14, me_mfd = 15, me_flags = 940113920, me_psize = 4096, me_maxreaders = 126, me_numreaders = 2, me_numdbs = 13, me_maxdbs = 130, me_pid = 43974, me_path = 0x8c19a0 "/opt/openldap/var/openldap-data", me_map = 0x7ff6f721f000 "", me_txns = 0x7ffff7ffb000, me_metas = {0x7ff6f721f010, 0x7ff6f7220010}, me_txn = 0x7ff6dc10afd0, me_mapsize = 38654705664, me_size = 0, me_maxpg = 9437184, me_dbxs = 0x8eed10, me_dbflags = 0x8ceb40, me_txkey = 1, me_pgstate = {mf_pghead = 0x7ff6dc109d58, mf_pglast = 44}, me_dpages = 0x0, me_free_pgs = 0xaf75e8, me_dirty_list = 0x7ffff721f010, me_maxfree_1pg = 509, me_nodemax = 2040} > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7717) Sudden memory increase leading to Master LDAP crash
by marcon.bruno＠free.fr 03 Oct '13

03 Oct '13

Full_Name: Marcon Bruno Version: 2.4.36 OS: Ubuntu URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (185.23.92.11) Hi. We have deployed OpenLDAP on our system, using the following architecture : One primary master One failover master (mirror mode) 4 replicas It works very well until the master LDAP receive two many writes /seconds. With 40 writes/seconds, the master LDAP work well about several minutes, then the memory process (slapd) started to increase very quickly, about 20Mo/s ! When all memory and swap is consumed, the OS kill Slapd. Notes : - Even in disabling all overlays except syncprov, the problem persists - If disabling only syncprov, there is no more problem - If all slaves and mirror master are stopped, there is no more problem This problem is very annoying. We are looking to a Microsoft solution, even if we have already deployed our solution but obviously, we would like to keep an OpenLDAP solution. The problem is very easy to replay : one master and one slave are sufficient. Then run 9 windows with a script adding an entry and modifying an entry.

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by hyc＠symas.com 03 Oct '13

03 Oct '13

Željko Nejašmić wrote: > On Wed, Oct 2, 2013 at 6:43 PM, <hyc(a)symas.com <mailto:hyc@symas.com>> wrote: > Without any other clues, this feels like ASLR is messing with us but that's > just a wild guess. I can no longer reproduce the SIGBUS in slapd regardless of > compile options, while ldclt itself keeps dying. If you can find some more > reliable way to reproduce the issue that would help. Perhaps using the client > in test060. > > > I ran the whole test suite for mdb, and as far as I can see, every test > returned OK. > > Found a new way to reproduce the SIGBUS using ldapadd on Ubuntu 12.04 firing > to openldap on RH 6.3: > ldapadd -h 172.17.101.150 -p 389 -D "cn=admin,dc=test" -w test -f test.ldif -- > ldif file is the same as the previous ldclt command. Doubt it matters, but the > ldif file is 1M adds. Can you provide your slapd config and this LDIF? > > On the RH box: > - compiled openldap with -g -O0 and previously used flags > gdb `find /root/openldap/ -type d -printf '-d %p '` --args > /opt/openldap/libexec/slapd -h "ldap:/// ldapi:///" -F > /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap -d 0 > > gdb output: > bt -- http://hastebin.com/hefikekaxi.sh > bt 10 full -- http://hastebin.com/vudocosuka.sh Can you also provide the gdb output for print *env > > I am begging to doubt that the problem could be on my end as the bt seems to > point to schema problems (although, I haven't analyzed it in great detail yet). > > > Zeljko -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by nejasmicz＠gmail.com 03 Oct '13

03 Oct '13

--001a11c253c0f33b8604e7d59850 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Wed, Oct 2, 2013 at 6:43 PM, <hyc(a)symas.com> wrote: > hyc(a)symas.com wrote: > > =C5=BDeljko Neja=C5=A1mi=C4=87 wrote: > >> Here you go http://hastebin.com/fukecejuje.tex > > > > Interestingly enough, I got the same result as you on an initial > compile/run > > of slapd. Unfortunately, with optimization, the backtrace wasn't all th= at > > useful. Recompiling back-mdb with just -g, no optimization, gets a > different > > result though - slapd is fine, and ldclt dies with a heap corruption or > > double-free. > > And now I cannot reproduce the original SIGBUS at all. But still getting > various crashes in ldclt. > > ldclt -h localhost -p 9011 -D cn=3Dmanager,dc=3Dexample,dc=3Dcom -w secre= t -b > ou=3Dpeople,dc=3Dexample,dc=3Dcom -e > object=3Dxx.txt,rdn=3D'uid:[A=3DINCRNNOLOOP(200000;999999;6)]' -e > add,commoncounter > -I 68 > ldclt version 4.23 > ldclt[30207]: Starting at Wed Oct 2 09:39:10 2013 > > *** glibc detected *** ldclt: double free or corruption (fasttop): > 0x00007fc2180021e0 *** > =3D=3D=3D=3D=3D=3D=3D Backtrace: =3D=3D=3D=3D=3D=3D=3D=3D=3D > /lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7fc223ff9b96] > /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x142d1)[0x7fc224f0b2d1] > /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x1ae74)[0x7fc224f11e74] > /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_Malloc+0x49)[0x7fc224f0d0b9] > /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x129f4)[0x7fc224f099f4] > /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x12068)[0x7fc224f09068] > /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_vsmprintf+0x38)[0x7fc224f09b28] > /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_smprintf+0x8c)[0x7fc224f09bec] > ldclt(+0x663a)[0x7fc22556663a] > ldclt(+0x74a4)[0x7fc2255674a4] > ldclt(+0x9d48)[0x7fc225569d48] > ldclt(threadMain+0x329)[0x7fc2255735d9] > /lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7fc224341e9a] > /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fc22406ecbd] > > Without any other clues, this feels like ASLR is messing with us but that= 's > just a wild guess. I can no longer reproduce the SIGBUS in slapd > regardless of > compile options, while ldclt itself keeps dying. If you can find some mor= e > reliable way to reproduce the issue that would help. Perhaps using the > client > in test060. I ran the whole test suite for mdb, and as far as I can see, every test returned OK. Found a new way to reproduce the SIGBUS using ldapadd on Ubuntu 12.04 firing to openldap on RH 6.3: ldapadd -h 172.17.101.150 -p 389 -D "cn=3Dadmin,dc=3Dtest" -w test -f test.= ldif -- ldif file is the same as the previous ldclt command. Doubt it matters, but the ldif file is 1M adds. On the RH box: - compiled openldap with -g -O0 and previously used flags gdb `find /root/openldap/ -type d -printf '-d %p '` --args /opt/openldap/libexec/slapd -h "ldap:/// ldapi:///" -F /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap -d 0 gdb output: bt -- http://hastebin.com/hefikekaxi.sh bt 10 full -- http://hastebin.com/vudocosuka.sh I am begging to doubt that the problem could be on my end as the bt seems to point to schema problems (although, I haven't analyzed it in great detail yet). Zeljko --001a11c253c0f33b8604e7d59850 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><br><div class=3D"gmail_extra"><br><br><div class=3D"gmail= _quote">On Wed, Oct 2, 2013 at 6:43 PM, <span dir=3D"ltr"><<a href=3D"m= ailto:hyc@symas.com" target=3D"_blank">hyc(a)symas.com</a>></span> wrote:<= br><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bord= er-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:soli= d;padding-left:1ex"> <div><a href=3D"mailto:hyc@symas.com" target=3D"_blank">hyc(a)symas.com</a> w= rote:<br> > =C5=BDeljko Neja=C5=A1mi=C4=87 wrote:<br> >> Here you go <a href=3D"http://hastebin.com/fukecejuje.tex" target= =3D"_blank">http://hastebin.com/fukecejuje.tex</a><br> ><br> > Interestingly enough, I got the same result as you on an initial compi= le/run<br> > of slapd. Unfortunately, with optimization, the backtrace wasn't a= ll that<br> > useful. Recompiling back-mdb with just -g, no optimization, gets a dif= ferent<br> > result though - slapd is fine, and ldclt dies with a heap corruption o= r<br> > double-free.<br> <br> </div>And now I cannot reproduce the original SIGBUS at all. But still gett= ing<br> various crashes in ldclt.<br> <div><br> ldclt -h localhost -p 9011 -D cn=3Dmanager,dc=3Dexample,dc=3Dcom -w secret = -b<br> ou=3Dpeople,dc=3Dexample,dc=3Dcom -e<br> object=3Dxx.txt,rdn=3D'uid:[A=3DINCRNNOLOOP(200000;999999;6)]' -e a= dd,commoncounter<br> -I 68<br> ldclt version 4.23<br> </div>ldclt[30207]: Starting at Wed Oct =C2=A02 09:39:10 2013<br> <div><br> *** glibc detected *** ldclt: double free or corruption (fasttop):<br> </div>0x00007fc2180021e0 ***<br> =3D=3D=3D=3D=3D=3D=3D Backtrace: =3D=3D=3D=3D=3D=3D=3D=3D=3D<br> /lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7fc223ff9b96]<br> /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x142d1)[0x7fc224f0b2d1]<br> /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x1ae74)[0x7fc224f11e74]<br> /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_Malloc+0x49)[0x7fc224f0d0b9]<br> /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x129f4)[0x7fc224f099f4]<br> /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x12068)[0x7fc224f09068]<br> /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_vsmprintf+0x38)[0x7fc224f09b28]<br= > /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_smprintf+0x8c)[0x7fc224f09bec]<br> ldclt(+0x663a)[0x7fc22556663a]<br> ldclt(+0x74a4)[0x7fc2255674a4]<br> ldclt(+0x9d48)[0x7fc225569d48]<br> ldclt(threadMain+0x329)[0x7fc2255735d9]<br> /lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7fc224341e9a]<br> /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fc22406ecbd]<br> <br> Without any other clues, this feels like ASLR is messing with us but that&#= 39;s<br> just a wild guess. I can no longer reproduce the SIGBUS in slapd regardless= of<br> compile options, while ldclt itself keeps dying. If you can find some more<= br> reliable way to reproduce the issue that would help. Perhaps using the clie= nt<br> in test060.</blockquote><div><br></div><div>I ran the whole test suite for = mdb, and as far as I can see, every test returned OK.</div><div><br></div><= div>Found a new way to reproduce the SIGBUS using ldapadd on Ubuntu 12.04 f= iring to openldap on RH 6.3:<br> </div><div>ldapadd -h 172.17.101.150 -p 389 -D "cn=3Dadmin,dc=3Dtest&q= uot; -w test -f test.ldif -- ldif file is the same as the previous ldclt co= mmand. Doubt it matters, but the ldif file is 1M adds.</div></div><br></div= > <div class=3D"gmail_extra">On the RH box:</div><div class=3D"gmail_extra">-= compiled openldap with -g -O0 and previously used flags</div><div class=3D= "gmail_extra"> gdb `find /root/openldap/ -type d -printf '-d %p '` --args /opt/ope= nldap/libexec/slapd -h "ldap:/// ldapi:///" -F /opt/openldap/etc/= openldap/slapd.d -g openldap -u openldap -d 0<br></div><div class=3D"gmail_= extra"> <br></div><div class=3D"gmail_extra">gdb output:</div><div class=3D"gmail_e= xtra">bt --=C2=A0<a href=3D"http://hastebin.com/hefikekaxi.sh" target=3D"_b= lank">http://hastebin.com/hefikekaxi.sh</a></div><div class=3D"gmail_extra"= >bt 10 full --=C2=A0<a href=3D"http://hastebin.com/vudocosuka.sh" target=3D= "_blank">http://hastebin.com/vudocosuka.sh</a></div> <div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">I am beggin= g to doubt that the problem could be on my end as the bt seems to point to = schema problems (although, I haven't analyzed it in great detail yet).<= /div> <div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra"><br></div><= div class=3D"gmail_extra">Zeljko</div> </div> --001a11c253c0f33b8604e7d59850--

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by hyc＠symas.com 02 Oct '13

02 Oct '13

hyc(a)symas.com wrote: > Željko Nejašmić wrote: >> Here you go http://hastebin.com/fukecejuje.tex > > Interestingly enough, I got the same result as you on an initial compile/run > of slapd. Unfortunately, with optimization, the backtrace wasn't all that > useful. Recompiling back-mdb with just -g, no optimization, gets a different > result though - slapd is fine, and ldclt dies with a heap corruption or > double-free. And now I cannot reproduce the original SIGBUS at all. But still getting various crashes in ldclt. ldclt -h localhost -p 9011 -D cn=manager,dc=example,dc=com -w secret -b ou=people,dc=example,dc=com -e object=xx.txt,rdn='uid:[A=INCRNNOLOOP(200000;999999;6)]' -e add,commoncounter -I 68 ldclt version 4.23 ldclt[30207]: Starting at Wed Oct 2 09:39:10 2013 *** glibc detected *** ldclt: double free or corruption (fasttop): 0x00007fc2180021e0 *** ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7fc223ff9b96] /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x142d1)[0x7fc224f0b2d1] /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x1ae74)[0x7fc224f11e74] /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_Malloc+0x49)[0x7fc224f0d0b9] /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x129f4)[0x7fc224f099f4] /usr/lib/x86_64-linux-gnu/libnspr4.so(+0x12068)[0x7fc224f09068] /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_vsmprintf+0x38)[0x7fc224f09b28] /usr/lib/x86_64-linux-gnu/libnspr4.so(PR_smprintf+0x8c)[0x7fc224f09bec] ldclt(+0x663a)[0x7fc22556663a] ldclt(+0x74a4)[0x7fc2255674a4] ldclt(+0x9d48)[0x7fc225569d48] ldclt(threadMain+0x329)[0x7fc2255735d9] /lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7fc224341e9a] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fc22406ecbd] Without any other clues, this feels like ASLR is messing with us but that's just a wild guess. I can no longer reproduce the SIGBUS in slapd regardless of compile options, while ldclt itself keeps dying. If you can find some more reliable way to reproduce the issue that would help. Perhaps using the client in test060. > All subsequent runs give the same result. Still looking into this; getting > away from the Ubuntu bundled libldap will probably help. >> >> >> Zeljko >> >> >> On Wed, Oct 2, 2013 at 11:34 AM, <hyc(a)symas.com <mailto:hyc@symas.com>> wrote: >> >> nejasmicz(a)gmail.com <mailto:nejasmicz@gmail.com> wrote: >> > Full_Name: Željko Nejašmić >> > Version: latest git pull of OPENLDAP_REL_ENG_2_4 >> > OS: RedHat 6.3 >> > URL: >> > Submission from: (NULL) (213.147.123.33) >> > >> >> > Hardware underneath all of that is: >> > 1) HP ProLiant BL460c Gen8, dual Xeon E5-2658 with attached >> storage blade >> > D2200sb with SSD raid, 256GB RAM -- RedHat 6.3 tests >> > 2) Intel server blade S2400BB, dual Xeon E5-2403, 48GB RAM -- >> Ubuntu 12.04 >> > tests >> > >> > If anything more is required to assist you in troubleshooting, please >> let me >> > know. >> >> Can you also post the gdb output for "bt 5 full" ? >> > >> > >> > Zeljko >> > >> > >> -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7716) slapd crashes after search immediately followed by (abandon+) unbind
by michael＠stroeder.com 02 Oct '13

02 Oct '13

michael.vishchers(a)7p-group.com wrote: > Full_Name: Michael Vishchers > Version: 2.4.23 > OS: Red Hat Enterprise Linux Server release 6.2 Did you check whether this problem still remains with the recent release 2.4.36? (I vaguely remember issues like this being fixed after 2.4.23 but I'm not sure.) Ciao, Michael.

1 0

(ITS#7716) slapd crashes after search immediately followed by (abandon+) unbind
by michael.vishchers＠7p-group.com 02 Oct '13

02 Oct '13

Full_Name: Michael Vishchers Version: 2.4.23 OS: Red Hat Enterprise Linux Server release 6.2 URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (178.15.66.50) slapd, running as a proxy to rewrite incoming connections based on user dn for later routing to different back ends, dies sporadically after receiving a (network delayed) search request that is "immediately" followed by an (optional) abandon request and an unbind request. We suspect that the abandon or unbind code tries to clean up data structures that belong to a not yet completely initialized search operation. The problem can unfortunately not easily be reproduced. Last time we had to wait at least two weeks before it appeared. It may be a timing problem between two or more threads. This is the stacktrace, core and other files could be provided if necessary. Program terminated with signal 11, Segmentation fault. #0 0x00007f299c3e71bc in ?? () #0 0x00007f299c3e71bc in ?? () No symbol table info available. #1 0x00007f2999bbd983 in rwm_op_rollback (op=0x7f2984002190, rs=<value optimized out>, ros=0x7f298c003570) at ../../../../servers/slapd/overlays/rwm.c:107 __PRETTY_FUNCTION__ = "rwm_op_rollback" #2 0x00007f2999bbe988 in rwm_op_search (op=0x7f2984002190, rs=0x7f2995bafaa0) at ../../../../servers/slapd/overlays/rwm.c:984 on = 0x7f299fb95210 rwmap = 0x7f299fb94f70 rc = <value optimized out> dc = {rwmap = 0x7f2984002190, conn = 0x7f298c003468, ctx = 0x12 <Address 0x12 out of bounds>, rs = 0x7f299e7963b0} fstr = {bv_len = 0, bv_val = 0x0} f = 0x0 an = 0x0 text = <value optimized out> roc = 0x7f298c003550 #3 0x00007f299e7fe02a in overlay_op_walk (op=0x7f2984002190, rs=0x7f2995bafaa0, which=op_search, oi=0x7f299fb95030, on=0x7f299fb95210) at ../../../servers/slapd/backover.c:659 func = 0x7f299fb95268 rc = 32768 #4 0x00007f299e8d29a1 in slapi_op_func (op=0x7f2984002190, rs=0x7f2995bafaa0) at ../../../../servers/slapd/slapi/slapi_overlay.c:647 pb = 0x7f298c1051b0 which = op_search opinfo = <value optimized out> rc = <value optimized out> oi = <value optimized out> on = <value optimized out> cb = {sc_next = 0x7f2995bae7e0, sc_response = 0x7f299e8d1fc0 <slapi_over_response>, sc_cleanup = 0x7f299e8d1ed0 <slapi_over_cleanup>, sc_private = 0x7f298c1051b0} internal_op = 0 preop_type = <value optimized out> postop_type = 503 be = 0x7f2995bae800 #5 0x00007f299e7fe02a in overlay_op_walk (op=0x7f2984002190, rs=0x7f2995bafaa0, which=op_search, oi=0x7f299fb95030, on=0x7f299fb9e8c0) at ../../../servers/slapd/backover.c:659 func = 0x7f299fb9e918 rc = 32768 #6 0x00007f299e7feb6b in over_op_func (op=0x7f2984002190, rs=<value optimized out>, which=<value optimized out>) at ../../../servers/slapd/backover.c:721 oi = <value optimized out> on = <value optimized out> be = 0x7f299fb940b0 db = {bd_info = 0x7f299fb95210, bd_self = 0x7f299fb940b0, be_ctrls = "\000", '\001' <repeats 17 times>, '\000' <repeats 14 times>, "\001", be_flags = 257, be_restrictops = 0, be_requires = 5, be_ssf_set = {sss_ssf = 0, sss_transport = 0, sss_tls = 0, sss_sasl = 0, sss_update_ssf = 0, sss_update_transport = 0, sss_update_tls = 0, sss_update_sasl = 0, sss_simple_bind = 0}, be_suffix = 0x7f299fb94ed0, be_nsuffix = 0x7f299fb94f00, be_schemadn = {bv_len = 0, bv_val = 0x0}, be_schemandn = {bv_len = 0, bv_val = 0x0}, be_rootdn = {bv_len = 0, bv_val = 0x0}, be_rootndn = {bv_len = 0, bv_val = 0x0}, be_rootpw = {bv_len = 0, bv_val = 0x0}, be_max_deref_depth = 15, be_def_limit = {lms_t_soft = 3600, lms_t_hard = 0, lms_s_soft = 500, lms_s_hard = 0, lms_s_unchecked = -1, lms_s_pr = 0, lms_s_pr_hide = 0, lms_s_pr_total = 0}, be_limits = 0x0, be_acl = 0x0, be_dfltaccess = ACL_READ, be_update_ndn = {bv_len = 0, bv_val = 0x0}, be_update_refs = 0x0, be_pending_csn_list = 0x7f299fc738f0, be_pcl_mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, be_syncinfo = 0x0, be_pb = 0x7f299fb9eaa0, be_cf_ocs = 0x7f299eb6da00, be_private = 0x7f299fb94240, be_next = {stqe_next = 0x0}} cb = {sc_next = 0x0, sc_response = 0x7f299e7fdd40 <over_back_response>, sc_cleanup = 0, sc_private = 0x7f299fb95030} sc = <value optimized out> rc = 32768 __PRETTY_FUNCTION__ = "over_op_func" #7 0x00007f299e794999 in fe_op_search (op=0x7f2984002190, rs=0x7f2995bafaa0) at ../../../servers/slapd/search.c:366 bd = 0x7f299eb72760 #8 0x00007f299e795177 in do_search (op=0x7f2984002190, rs=<value optimized out>) at ../../../servers/slapd/search.c:217 base = {bv_len = 55, bv_val = 0x7f298c11fef9 "vfsid=491722472236,ou=subscriber,ou=mmo,c=de,o=vodafone"} siz = 0 off = 0 i = <value optimized out> #9 0x00007f299e7920f9 in connection_operation (ctx=0x7f2995bafb70, arg_v=0x7f2984002190) at ../../../servers/slapd/connection.c:1109 rc = 80 cancel = <value optimized out> op = 0x7f2984002190 rs = {sr_type = REP_RESULT, sr_tag = 0, sr_msgid = 0, sr_err = 80, sr_matched = 0x0, sr_text = 0x7f2999bc4122 "Rewrite error", sr_ref = 0x0, sr_ctrls = 0x0, sr_un = {sru_search = {r_entry = 0x0, r_attr_flags = 0, r_operational_attrs = 0x0, r_attrs = 0x0, r_nentries = 0, r_v2ref = 0x0}, sru_sasl = {r_sasldata = 0x0}, sru_extended = {r_rspoid = 0x0, r_rspdata = 0x0}}, sr_flags = 0} tag = 99 opidx = SLAP_OP_SEARCH conn = 0x7f2996db74d0 memctx = 0x7f298c002820 memctx_null = 0x0 memsiz = 1048576 __PRETTY_FUNCTION__ = "connection_operation" #10 0x00007f299e892678 in ldap_int_thread_pool_wrapper (xpool=0x7f299fae9ae0) at ../../../libraries/libldap_r/tpool.c:685 pool = 0x7f299fae9ae0 task = 0x7f2988000a20 work_list = <value optimized out> ctx = {ltu_id = 139816582448896, ltu_key = {{ltk_key = 0x7f299e790d50, ltk_data = 0x7f298c002d40, ltk_free = 0x7f299e790e30 <conn_counter_destroy>}, {ltk_key = 0x7f299e7eaf70, ltk_data = 0x7f298c002820, ltk_free = 0x7f299e7eae50 <slap_sl_mem_destroy>}, {ltk_key = 0x7f299e7a6b70, ltk_data = 0x0, ltk_free = 0x7f299e7a6940 <slap_op_q_destroy>}, {ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0} <repeats 29 times>}} kctx = <value optimized out> keyslot = <value optimized out> hash = <value optimized out> __PRETTY_FUNCTION__ = "ldap_int_thread_pool_wrapper" #11 0x00007f299c91e7f1 in ?? () No symbol table info available. #12 0x00007f2995bb0700 in ?? () No symbol table info available. #13 0x0000000000000000 in ?? () No symbol table info available.

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by hyc＠symas.com 02 Oct '13

02 Oct '13

Željko Nejašmić wrote: > Here you go http://hastebin.com/fukecejuje.tex Interestingly enough, I got the same result as you on an initial compile/run of slapd. Unfortunately, with optimization, the backtrace wasn't all that useful. Recompiling back-mdb with just -g, no optimization, gets a different result though - slapd is fine, and ldclt dies with a heap corruption or double-free. ldclt -h localhost -p 9011 -D cn=manager,dc=example,dc=com -w secret -b ou=people,dc=example,dc=com -e object=xx.txt,rdn='uid:[A=INCRNNOLOOP(200000;999999;6)]' -e add,commoncounter -I 68 ldclt version 4.23 ldclt[10503]: Starting at Wed Oct 2 04:13:03 2013 *** glibc detected *** ldclt: double free or corruption (fasttop): 0x00007fa448003270 *** ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7fa463f8cb96] /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2(ldap_pvt_tls_set_option+0x1eb)[0x7fa46472e06b] /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2(ldap_int_tls_config+0x54)[0x7fa46472e234] /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2(+0x2b8b7)[0x7fa4647238b7] /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2(ldap_int_initialize+0x104)[0x7fa464723e84] /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2(ldap_create+0x29)[0x7fa4647074f9] /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2(ldap_initialize+0x2f)[0x7fa464707a7f] ldclt(+0x664a)[0x7fa4656fb64a] ldclt(+0x74a4)[0x7fa4656fc4a4] ldclt(+0x9d48)[0x7fa4656fed48] ldclt(threadMain+0x329)[0x7fa4657085d9] /lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7fa4642d4e9a] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fa464001cbd] All subsequent runs give the same result. Still looking into this; getting away from the Ubuntu bundled libldap will probably help. > > > Zeljko > > > On Wed, Oct 2, 2013 at 11:34 AM, <hyc(a)symas.com <mailto:hyc@symas.com>> wrote: > > nejasmicz(a)gmail.com <mailto:nejasmicz@gmail.com> wrote: > > Full_Name: Željko Nejašmić > > Version: latest git pull of OPENLDAP_REL_ENG_2_4 > > OS: RedHat 6.3 > > URL: > > Submission from: (NULL) (213.147.123.33) > > > > > Hardware underneath all of that is: > > 1) HP ProLiant BL460c Gen8, dual Xeon E5-2658 with attached > storage blade > > D2200sb with SSD raid, 256GB RAM -- RedHat 6.3 tests > > 2) Intel server blade S2400BB, dual Xeon E5-2403, 48GB RAM -- > Ubuntu 12.04 > > tests > > > > If anything more is required to assist you in troubleshooting, please > let me > > know. > > Can you also post the gdb output for "bt 5 full" ? > > > > > > Zeljko > > > > > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by nejasmicz＠gmail.com 02 Oct '13

02 Oct '13

--047d7b67749a03fb9b04e7c010b4 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Here you go http://hastebin.com/fukecejuje.tex Zeljko On Wed, Oct 2, 2013 at 11:34 AM, <hyc(a)symas.com> wrote: > nejasmicz(a)gmail.com wrote: > > Full_Name: =C5=BDeljko Neja=C5=A1mić > > Version: latest git pull of OPENLDAP_REL_ENG_2_4 > > OS: RedHat 6.3 > > URL: > > Submission from: (NULL) (213.147.123.33) > > > > > Hardware underneath all of that is: > > 1) HP ProLiant BL460c Gen8, dual Xeon E5-2658 with attached storag= e > blade > > D2200sb with SSD raid, 256GB RAM -- RedHat 6.3 tests > > 2) Intel server blade S2400BB, dual Xeon E5-2403, 48GB RAM -- > Ubuntu 12.04 > > tests > > > > If anything more is required to assist you in troubleshooting, please > let me > > know. > > Can you also post the gdb output for "bt 5 full" ? > > > > > > Zeljko > > > > > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > > > --047d7b67749a03fb9b04e7c010b4 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>Here you go=C2=A0<a href=3D"http://hastebin.com/fukec= ejuje.tex">http://hastebin.com/fukecejuje.tex</a></div><div><br></div><div>= <br></div><div>Zeljko</div></div><div class=3D"gmail_extra"><br><br><div cl= ass=3D"gmail_quote"> On Wed, Oct 2, 2013 at 11:34 AM, <span dir=3D"ltr"><<a href=3D"mailto:h= yc(a)symas.com" target=3D"_blank">hyc(a)symas.com</a>></span> wrote:<br><blo= ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c= cc solid;padding-left:1ex"> <div class=3D"HOEnZb"><div class=3D"h5"><a href=3D"mailto:nejasmicz@gmail.c= om">nejasmicz(a)gmail.com</a> wrote:<br> > Full_Name: =C5=BDeljko Neja=C5=A1mi&#263;<br> > Version: latest git pull of OPENLDAP_REL_ENG_2_4<br> > OS: RedHat 6.3<br> > URL:<br> > Submission from: (NULL) (213.147.123.33)<br> ><br> <br> > Hardware underneath all of that is:<br> > =C2=A0 =C2=A0 =C2=A01) HP ProLiant BL460c Gen8, dual Xeon E5-2658 with= attached storage blade<br> > D2200sb with SSD raid, 256GB RAM -- RedHat 6.3 tests<br> > =C2=A0 =C2=A0 =C2=A02) Intel server blade S2400BB, dual Xeon E5-2403, = 48GB RAM -- Ubuntu 12.04<br> > tests<br> ><br> > If anything more is required to assist you in troubleshooting, please = let me<br> > know.<br> <br> Can you also post the gdb output for "bt 5 full" ?<br> ><br> ><br> > Zeljko<br> ><br> ><br> <br> <br> --<br> =C2=A0 =C2=A0-- Howard Chu<br> =C2=A0 =C2=A0CTO, Symas Corp. =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 <a href=3D= "http://www.symas.com" target=3D"_blank">http://www.symas.com</a><br> =C2=A0 =C2=A0Director, Highland Sun =C2=A0 =C2=A0 <a href=3D"http://highlan= dsun.com/hyc/" target=3D"_blank">http://highlandsun.com/hyc/</a><br> =C2=A0 =C2=A0Chief Architect, OpenLDAP =C2=A0<a href=3D"http://www.openldap= .org/project/" target=3D"_blank">http://www.openldap.org/project/</a><br> <br> <br> </div></div></blockquote></div><br></div> --047d7b67749a03fb9b04e7c010b4--

1 0

Re: (ITS#7715) SIGBUS when mdb is configured with writemap
by hyc＠symas.com 02 Oct '13

02 Oct '13

nejasmicz(a)gmail.com wrote: > Full_Name: eljko Nejamić > Version: latest git pull of OPENLDAP_REL_ENG_2_4 > OS: RedHat 6.3 > URL: > Submission from: (NULL) (213.147.123.33) > > Hardware underneath all of that is: > 1) HP ProLiant BL460c Gen8, dual Xeon E5-2658 with attached storage blade > D2200sb with SSD raid, 256GB RAM -- RedHat 6.3 tests > 2) Intel server blade S2400BB, dual Xeon E5-2403, 48GB RAM -- Ubuntu 12.04 > tests > > If anything more is required to assist you in troubleshooting, please let me > know. Can you also post the gdb output for "bt 5 full" ? > > > Zeljko > > -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

(ITS#7715) SIGBUS when mdb is configured with writemap
by nejasmicz＠gmail.com 02 Oct '13

02 Oct '13

Full_Name: eljko Nejamić Version: latest git pull of OPENLDAP_REL_ENG_2_4 OS: RedHat 6.3 URL: Submission from: (NULL) (213.147.123.33) Using ldclt tool to stress test our OpenLDAP mirror sync setup I encountered a SIGBUS. Do note that the same issue occurs on only one node too, without sync. I've tested using the aforementioned tool and the same arguments on both Red Hat 6.3 (2.6.32-279.el6.x86_64) and Ubuntu Server 12.04 (Linux 3.2.0-54-generic x86_64) with the exact outcome. In both cases the OpenLDAP was compiled from sources (origin/OPENLDAP_REL_ENG_2_4), configured with --disable-{hdb,bdb}, --prefix=/opt/openldap, --enable-local=yes and using mdb as a backend, tweaked additionally with: * nometasync * writemap Without the writemap tweak, SIGBUS isn't happening. The command used was: ldclt -h 172.17.101.150 -p 389 -D "cn=xxx,dc=xxx" -w "xxx" -b "ds=USERS,o=STANDARD,dc=xxx" \ -e object=xxx.txt,rdn='uid:[A=INCRNNOLOOP(200000;999999;6)]' -e add,commoncounter -I 68 ...where the xxx.txt has the following content: objectclass: xxxUser The ldclt command uses 10 threads to do the add operation with the incrementing uid parameter on the base dn: ds=USERS,o=STANDARD,dc=xxx. Ulimits are: ulimit -a core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 2066206 max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 4096 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) unlimited real-time priority (-r) 0 stack size (kbytes, -s) unlimited cpu time (seconds, -t) unlimited max user processes (-u) 1024 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited At first sight, gdb seems to point to mdb_page_alloc: Starting program: /opt/openldap/libexec/slapd -h ldap:///\ ldapi:/// -F /opt/openldap/etc/openldap/slapd.d -g openldap -u openldap -d 0 [Thread debugging using libthread_db enabled] [New Thread 0x2aaaac764700 (LWP 20415)] [Thread 0x2aaaac764700 (LWP 20415) exited] [New Thread 0x2aaaac764700 (LWP 20416)] [New Thread 0x2ab3ad168700 (LWP 20417)] [New Thread 0x2ab3ad969700 (LWP 20418)] [New Thread 0x2ab3ae16a700 (LWP 20419)] [New Thread 0x2ab3ae96b700 (LWP 20420)] [New Thread 0x2ab3b8800700 (LWP 20421)] [New Thread 0x2ab3d4800700 (LWP 20422)] Program received signal SIGBUS, Bus error. [Switching to Thread 0x2ab3b8800700 (LWP 20421)] mdb_page_alloc (mc=<value optimized out>, num=1, mp=0x2ab3b87fd8b8) at ./../../../libraries/liblmdb/mdb.c:1759 warning: Source file is more recent than executable. 1759 np->mp_pgno = pgno; And the backtrace is: #0 mdb_page_alloc (mc=<value optimized out>, num=1, mp=0x2ab3b87fd8b8) at ./../../../libraries/liblmdb/mdb.c:1759 #1 0x00000000004afb19 in mdb_page_touch (mc=0x2ab3bc1103f0) at ./../../../libraries/liblmdb/mdb.c:1889 #2 0x00000000004b1c8c in mdb_cursor_touch (mc=0x2ab3bc1103f0) at ./../../../libraries/liblmdb/mdb.c:5597 #3 0x00000000004b3a85 in mdb_cursor_put (mc=0x2ab3bc1103f0, key=0x2ab3b87ff000, data=0x2ab3b87feff0, flags=32) at ./../../../libraries/liblmdb/mdb.c:5727 #4 0x00000000004f8586 in mdb_idl_insert_keys (be=<value optimized out>, cursor=0x2ab3bc1103f0, keys=<value optimized out>, id=13) at idl.c:534 #5 0x00000000004f9116 in indexer (op=0x2ab3bc10dbd0, txn=<value optimized out>, ai=<value optimized out>, ad=0x88e0e0, atname=0x88dfb8, vals=0x2ab3bc110120, id=13, opid=1, mask=4) at index.c:219 #6 0x00000000004f95d1 in index_at_values (op=0x2ab3bc10dbd0, txn=0x2ab3bc10e2f0, ad=<value optimized out>, type=0x88df50, tags=0x88e100, vals=0x2ab3bc110120, id=13, opid=1) at index.c:337 #7 0x00000000004f9627 in mdb_index_values (op=<value optimized out>, txn=<value optimized out>, desc=<value optimized out>, vals=<value optimized out>, id=<value optimized out>, opid=<value optimized out>) at index.c:386 #8 0x00000000004f96f9 in mdb_index_entry (op=0x2ab3bc10dbd0, txn=0x2ab3bc10e2f0, opid=1, e=0x8c18d8) at index.c:558 #9 0x00000000004ed77e in mdb_add (op=0x2ab3bc10dbd0, rs=0x2ab3b87ff950) at add.c:359 #10 0x0000000000487ac7 in overlay_op_walk (op=0x2ab3bc10dbd0, rs=0x2ab3b87ff950, which=op_add, oi=0x932280, on=0x0) at backover.c:671 #11 0x00000000004884a7 in over_op_func (op=0x2ab3bc10dbd0, rs=<value optimized out>, which=<value optimized out>) at backover.c:723 #12 0x00000000004281c0 in fe_op_add (op=0x2ab3bc10dbd0, rs=0x2ab3b87ff950) at add.c:334 #13 0x0000000000428a16 in do_add (op=0x2ab3bc10dbd0, rs=0x2ab3b87ff950) at add.c:194 #14 0x0000000000421259 in connection_operation (ctx=0x2ab3b87ffab0, arg_v=0x2ab3bc10dbd0) at connection.c:1155 #15 0x0000000000421a35 in connection_read_thread (ctx=0x2ab3b87ffab0, argv=<value optimized out>) at connection.c:1291 #16 0x0000000000516380 in ldap_int_thread_pool_wrapper (xpool=0x898160) at tpool.c:688 #17 0x000000384cc07851 in start_thread () from /lib64/libpthread.so.0 #18 0x000000384c8e767d in clone () from /lib64/libc.so.6 For context, the assembly land around the offending pointer dereferencing looks like: 0x4af949 <mdb_page_alloc+665> movslq %r13d,%rax 0x4af94c <mdb_page_alloc+668> lea (%rcx,%rax,1),%rax 0x4af950 <mdb_page_alloc+672> mov %rax,0x10(%r14) 0x4af954 <mdb_page_alloc+676> mov %rcx,0x0(%rbp) Hardware underneath all of that is: 1) HP ProLiant BL460c Gen8, dual Xeon E5-2658 with attached storage blade D2200sb with SSD raid, 256GB RAM -- RedHat 6.3 tests 2) Intel server blade S2400BB, dual Xeon E5-2403, 48GB RAM -- Ubuntu 12.04 tests If anything more is required to assist you in troubleshooting, please let me know. Zeljko

1 0

(ITS#7714) Making slapd easier to jail (enhancement?)
by blance3459＠hotmail.com 02 Oct '13

02 Oct '13

Full_Name: Barry Lance Version: 2.4.35 OS: Linux (Debian 7) URL: ftp://ftp.openldap.org/incoming/ Submission from: (NULL) (70.226.37.254) Operating System: Debian 7.1.0 (Wheezy) 64-bit Openldap version: 2.4.35 Configure Options: --prefix=/usr/local --enable-shared --enable-static --enable-debug --enable-dynamic --enable-syslog --enable-local --enable-slapd --enable-spasswd --enable-modules --enable-backends=mod --disable-ndb --disable-passwd --disable-perl --disable-shell --disable-sock --disable-sql --enable-overlays=mod --with-threads --with-cyrus-sasl --with-tls=openssl config: slapd.conf from make install Jail directory: /var/chroot/openldap Files copied into jail: /etc/openldap -> <jaildir>/etc/openldap /etc/nsswitch.cond -> <jaildir>/etc /etc/pam.d -> <jaildir>/etc/pam.d /etc/passwd (or fragment of) -> <jaildir>/etc/passwd /etc/groups (or fragment of) -> <jaildir>/etc/groups /etc/shadow -> <jaildir>/etc/shadow all other libs referenced by ldd slapd into respective <jaildir>/dir /usr/local/libexec/openldap -> <jaildir>//usr/local/libexec/openldap /lib/x86_64-linux-gnu/libnss_* -> <jaildir>/lib/x86_64-linux-gnu commandline: /usr/local/libexec/slapd -d -1 -f /etc/openldap/slapd.conf -h "ldap:/// ldapi:///" -n slapd -r /var/chroot/openldap -u ldap -g ldap The behavior I have experienced is as follows: 1, Launch slapd without user (-u), group (-g), and jail dir (-r) options is successful. Slapd is running under the current user id (root). 2. Launch slapd with user and group parameters, but without a jail directory successful and the root privilege is dropped to the username given Takeaway - slapd is able to read passwd and groups outside jail. This is definitely expected. 3, Launch slapd with user, group and jail dir options, slapd fails with a message that no such user exists in passwd. 4. Launching slapd given a jail directory, but no user or group options succeeds with the daemon jailed in jail dir, butrunning as root (undesirable). Takeaway - chroot code works, but passwd/groups cannot be accessed after it (as seen in (3)). (4) is expected given the code in servers/slapd/main.c attempts to get the real uid/gid and drop root permission only if the -u and -g options are given on the command line. Comparing servers/slapd/main.c to the code for a few other daemons (ntpd, named, isc-dhcp), the jailing process follows the chdir/chroot process as expected. The difference in these other daemon is that, in all cases, the real uid/gid are retrieved before the chroot code. By doing this, nsswitch.conf, passwd, groups, etc are all still available outside the jail. Once the chroot is completed, they then drop root permission to the non privileged user given on the command line. The code in servers/slapd/main.c gets the real uid/gid AFTER the chroot. As such, the authentication infrastructure (nsswitch.conf, passwd, groups, etc) must be duplicated in the jail. In my opinion, this makes jailing slapd more difficult (inconvenient) than the other daemons mentioned. The jailing code in slapd may work, but I was unable to make it go as a non-root user. Didn't find very much useful information available via Google with respect to troubleshooting my chroot issue. To test a few theories, I added some scaffolding code in main.c and user.c to see where the process was going bad for me. As expected after looking at the source, the failure was happening in the slap_init_user function of user.c. More specifically, the call to getpwnam was returning NULL (failure) and causing the corresponding Debug statement to print the error message I am seeing when attempting to jail as a non-privileged user. No surprise there. Not being that familiar with the code in these two files, I am reluctant to modify too much for fear of introducing unintentional side effects. But in testing I found a few ways of working around the issue. Initially, I thought it might be easiest to move the call to slap_init_user before the chroot code. But then I realized that cannot work, because this function drops root permission before returning which will then cause the chroot code to fail. The first, and simplest, workaround I found was that by making an initial call to getpwnam and discarding the result before hitting the chroot code seemed to make the subsequent call in slap_init_user succeed. Wierd. I can only speculate two possibilities for that. The most reasonable is that the getpwnam call before the chroot code loads some shared library(ies) into memory that I'm missing in my jail allowing the later call after the chroot call to succeed. The second, and least reasonable, is that the man page for getpwnam states the returned pointer is to a static passwd struct which when initialized before the chroot code, is returned despite the later failure in slap_init_user. Like I said, least likely. I don't have the knowledge to test either theory and it just works despite the wasteful additional call to getpwnam. This workaround seems to have the least amount of potential side-effects elsewhere in the code. The second workaround involved moving the code blocks for the getpwnam and getgrnam calls out of slap_init_user and into main just prior to the the chroot code. I would also then have to drop the root permission just after the chroot code block based on got_uid/gid variable values. I didn't dig far enough into the code to know if there is anywhere else that the code drops root permission. If so, this may cause one of those unintentional side effects. This also worked, but seemed like bad coding practice. My third, and final, work around I think fits in the best with the spirit of the existing code. In this workaround I split up the slap_init_user code in user.c into four seperate functions: slap_init_user, slap_init_group, slap_set_user, slap_set_group. The slap_init_xxx functions use two variables added to main of type uid_t and gid_t that are passed in by reference. Each function performs the respective passwd/group calls in slap_init_user, store the result, if successful, into the byref parameter(s) and return a got_uid/gid value back as an integer return value indicating if the uid/gid parameters contain useful id's. The slapd_set_xxx functions perform the actual drop of root permission. Back in main, the slap_init_user/group functions get called prior to the chroot code and the corresponding slap_set_xxx calls are added in the relevant code block that follows the chroot call if the "got" variables indicate the uid/gid values are useful. This allows the uid/gid to be looked up prior to chroot eliminating the need for security info in the jail, root permission is still held to chroot, and the dropping of root authority sill occurs immediately after the chroot call. At the end of the day, I think the third workaround allow the chroot code for slapd to perform in the same way as the other daemons I mentioned creating a more predictable user experience. I certainly can't speak for everyone, but to me, a predictable user experience is more definitely more convenient. In the same light, I wonder if moving the creation of the pid and args files before the chroot adds value for the user when dealing with the daemon via an init script. The init script might be a bit more simple without having to worry about prepending a jail path to the pid file detected from the config when the daemon is being run as such. Not a big issue. If you think the code I used in my workarounds add value to the project, I will gladly send them along. But, honestly, it is so rudimentary that I doubt my copy offers you much value since I'm sure you could duplicate it in less than 30 minutes. Thanks, Barry Lance

1 0

Re: (ITS#7710) contextCSN values not updated by internal non-replicated ops
by marco.pizzoli＠gmail.com 01 Oct '13

01 Oct '13

--047d7bdc109e14c94f04e7b17008 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi Michael, yes I was, but I'm quite sure I didn't have any memberOf-operation running in the meantime. Marco On Tue, Oct 1, 2013 at 7:36 PM, Michael Str=F6der <michael(a)stroeder.com>wro= te: > Marco Pizzoli wrote: > > Hi, this *could* be also the root cause of a problem I found some times > ago > > joking with a particular OL cluster scenario in which I failed to obtai= n > > all entries correctly populated. Please tell me if this could be relate= d. > > [..] > > All the members of the cluster were using the contrib/slapo-lastbind > > overlay. > > We're also using slapo-lastbind but deactivating does not make a differen= ce > when modifying group entries (tested today). > > Are you using slapo-memberof at all? > > Ciao, Michael. > --047d7bdc109e14c94f04e7b17008 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div><div>Hi Michael,<br></div>yes I was, but I'm quit= e sure I didn't have any memberOf-operation running in the meantime.<br= ><br></div>Marco<br></div><div class=3D"gmail_extra"><br><br><div class=3D"= gmail_quote"> On Tue, Oct 1, 2013 at 7:36 PM, Michael Str=F6der <span dir=3D"ltr"><<a = href=3D"mailto:michael@stroeder.com" target=3D"_blank">michael(a)stroeder.com= </a>></span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin= :0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div class=3D"im">Marco Pizzoli wrote:<br> > Hi, this *could* be also the root cause of a problem I found some time= s ago<br> > joking with a particular OL cluster scenario in which I failed to obta= in<br> > all entries correctly populated. Please tell me if this could be relat= ed.<br> </div>> [..]<br> <div class=3D"im">> All the members of the cluster were using the contri= b/slapo-lastbind<br> > overlay.<br> <br> </div>We're also using slapo-lastbind but deactivating does not make a = difference<br> when modifying group entries (tested today).<br> <br> Are you using slapo-memberof at all?<br> <br> Ciao, Michael.<br> </blockquote></div><br></div> --047d7bdc109e14c94f04e7b17008--

1 0

Re: (ITS#7710) contextCSN values not updated by internal non-replicated ops
by michael＠stroeder.com 01 Oct '13

01 Oct '13

Marco Pizzoli wrote: > Hi, this *could* be also the root cause of a problem I found some times ago > joking with a particular OL cluster scenario in which I failed to obtain > all entries correctly populated. Please tell me if this could be related. > [..] > All the members of the cluster were using the contrib/slapo-lastbind > overlay. We're also using slapo-lastbind but deactivating does not make a difference when modifying group entries (tested today). Are you using slapo-memberof at all? Ciao, Michael.

1 0

Re: (ITS#7710) contextCSN values not updated by internal non-replicated ops
by marco.pizzoli＠gmail.com 01 Oct '13

01 Oct '13

--20cf303ea4b89c8b7504e7abe440 Content-Type: text/plain; charset=ISO-8859-1 Hi, this *could* be also the root cause of a problem I found some times ago joking with a particular OL cluster scenario in which I failed to obtain all entries correctly populated. Please tell me if this could be related. Long story short: 4-way multimaster cluster --> cluster A 3-way multimaster cluster --> cluster B one of the members of cluster A has also configured, as provider, one of the member of cluster B. By modifying data on any of the members of cluster A I should be able to see the modification also on any member of cluster B. Correct? Well, this was failing sometimes on 1 or 2 members. I had load balancer health-checks continuously polling all of members from both A and B: - bind - search - unbind All the members of the cluster were using the contrib/slapo-lastbind overlay. So the internal authTimestamp attribute populated by an internal operation. Could it be that the contextCSN of one node of the cluster were newer than the one of the providers? I'm not too expert, just trying to be of help by sharing experiences. Thanks for reading Marco --20cf303ea4b89c8b7504e7abe440 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Hi, this *could* be also the root cause of a problem I fou= nd some times ago joking with a particular OL cluster scenario in which I f= ailed to obtain all entries correctly populated. Please tell me if this cou= ld be related.<div> <br></div><div>Long story short:</div><div>4-way multimaster cluster -->= cluster A</div><div>3-way multimaster cluster --> cluster B</div><div><= br></div><div>one of the members of cluster A has=A0also=A0configured, as p= rovider, one of the member of cluster B.</div> <div><br></div><div>By modifying data on any of the members of cluster A I = should be able to see the modification also on any member of cluster B. Cor= rect?</div><div>Well, this was failing sometimes on 1 or 2 members.<br> </div><div><br></div><div>I had load balancer health-checks continuously po= lling all of members from both A and B:</div><div>- bind</div><div>- search= </div><div>- unbind</div><div><br></div><div>All the members of the cluster= were using the contrib/slapo-lastbind overlay. So the internal authTimesta= mp attribute populated by an internal operation.</div> <div>Could it be that the contextCSN of one node of the cluster were newer = than the one of the providers?</div><div><br></div><div>I'm not too exp= ert, just trying to be of help by sharing experiences.</div><div>Thanks for= reading</div> <div>Marco</div></div> --20cf303ea4b89c8b7504e7abe440--

1 0

Re: (ITS#7713) Segmentation fault if the pagesize of the Operating system is not equal to 4096.
by sumantk2＠linux.vnet.ibm.com 01 Oct '13

01 Oct '13

On 10/01/2013 12:46 PM, Howard Chu wrote: > sumantk2(a)linux.vnet.ibm.com wrote: >> Full_Name: sumanth k >> Version: 2.4.35 to any recent version with mdb support >> OS: Linux - ppc64 >> URL: ftp://ftp.openldap.org/incoming/ >> Submission from: (NULL) (122.248.161.59) >> >> >> The pagesize of Linux for x86 and s390x architecture is 4096. Whereas >> in Powerpc >> <ppc64> , the pagesize is by default 65536. So when the pagesize is >> not equal to >> 4096, the segmentation fault occurs. So tried to compile the >> powerpc64 kernel >> with page size of 4096 and the problem disappears and runs smoothly. >> But by >> default the powerpc64 architecture runs with 65536 pagesize . So >> there is some >> problem in mdb_env_open2() function in mdb.c when the pagesize is not >> equal to >> 4096. > > Thanks for the report. I believe you should be able to instead change > the definition of MDB_PAGESIZE to 65536, instead of forcing your > machine to use 4096 byte pages. > Yes i have tried changing MDB_PAGESIZE to 65536 before reporting this and the segmentation fault still occured. > There are other problems though; we use an unsigned short for page > offsets. I'm not sure the assert that you tripped will succeed in this > case. > Will try changing datatype and see ..But not sure whether it affects some other part of the code. >> >> These are my observations : >> >> >> Compiled the source with -O0 optimization . >> /home/openldap-2.4.36/tests/../servers/slapd/slapd -s0 -f >> /home/openldap-2.4.36/tests/testrun/slapd.1.conf -h >> ldap://localhost:9011/ -d >> 0x4105 >> >> ./scripts/test000-rootdse: line 31: 19059 Aborted >> (core dumped) >> $SLAPD -f $CONF1 -h $URI1 -d $LVL $TIMING > $LOG1 2>&1 >> >> Core file : >> bash-4.2# cat .gdbinit >> b mdb_db_open >> b mdb_env_open >> b mdb_env_open2 >> b mdb_txn_begin >> b mdb_txn_renew0 >> b mdb_dbi_open >> b mdb_cursor_init >> b mdb_cursor_set >> b mdb_page_search >> b mdb_page_get >> b mdb_page_search_root >> r -s0 -f /home/openldap-2.4.36/tests/testrun/slapd.1.conf -h >> ldap://localhost:9011/ -d 0x4105 >> >> >> # Core was generated by >> `/home/openldap-2.4.36/tests/../servers/slapd/slapd -s0 >> -f /home/openldap-2.4.36'. >> Program terminated with signal 6, Aborted. >> #0 0x00001fffff7adb70 in .raise () from /lib64/libc.so.6 >> Missing separate debuginfos, use: debuginfo-install >> cyrus-sasl-lib-2.1.26-8.mcp8_0.2.ppc64 >> cyrus-sasl-md5-2.1.26-8.mcp8_0.2.ppc64 >> glibc-2.17-4.mcp8_0.6.ppc64 keyutils-libs-1.5.5-4.mcp8_0.2.ppc64 >> krb5-libs-1.11.3-1.mcp8_0.2.ppc64 libcom_err-1.42.7-2.mcp8_0.1.ppc64 >> libdb-5.3.21-8.mcp8_0.4.ppc64 libselinux-2.1.13-15.mcp8_0.3.ppc64 >> nss-softokn-freebl-3.14.3-1.mcp8_0.2.ppc64 >> openssl-libs-1.0.1e-4.mcp8_0.1.ppc64 >> pcre-8.32-7.mcp8_0.2.ppc64 zlib-1.2.7-10.mcp8_0.1.ppc64 >> (gdb) where >> #0 0x00001fffff7adb70 in .raise () from /lib64/libc.so.6 >> #1 0x00001fffff7afb64 in .abort () from /lib64/libc.so.6 >> #2 0x00001fffff7a455c in .__assert_fail_base () from /lib64/libc.so.6 >> #3 0x00001fffff7a464c in .__assert_fail () from /lib64/libc.so.6 >> #4 0x00000000101498cc in mdb_node_add (mc=0x3fffe89a7d40, indx=0, >> key=0x3fffe89a7d20, data=0x3fffe89a7d30, pgno=0, flags=2) at >> ./../../../libraries/liblmdb/mdb.c:6160 >> #5 0x000000001014882c in mdb_cursor_put (mc=0x3fffe89a7d40, >> key=0x3fffe89a7d20, >> data=0x3fffe89a7d30, flags=2) at ./../../../libraries/liblmdb/mdb.c:5877 >> #6 0x00000000101516a8 in mdb_dbi_open (txn=0x1000f675980, >> name=0x1027b9c8 >> "ad2i", flags=262152, dbi=0x1fffff0800a0) at >> ./../../../libraries/liblmdb/mdb.c:7902 >> #7 0x0000000010139cd4 in mdb_db_open (be=0x1000f4f83a0, >> cr=0x3fffe89a81c0) at >> init.c:207 >> #8 0x0000000010050e34 in backend_startup_one (be=0x1000f4f83a0, >> cr=0x3fffe89a81c0) at backend.c:224 >> #9 0x0000000010051588 in backend_startup (be=0x1000f4f83a0) at >> backend.c:325 >> #10 0x0000000010089a7c in slap_startup (be=0x0) at init.c:219 >> #11 0x000000001000a9c8 in main (argc=8, argv=0x3fffe89a8958) at >> main.c:991 >> >> >> >> Here is the error message : >> >> #/home/openldap-2.4.36/tests/../servers/slapd/slapd -s0 -f >> /home/openldap-2.4.36/tests/testrun/slapd.1.conf -h >> ldap://localhost:9011/ -d >> 0x4105 >> (...some messages...) >> 524a2fe1 mdb_db_open: database "o=OpenLDAP Project,l=Internet": >> dbenv_open(/home/openldap-2.4.36/tests/testrun/db.1.a). >> slapd: ./../../../libraries/liblmdb/mdb.c:6160: mdb_node_add: Assertion >> `mp->mp_pb.pb.pb_upper >= mp->mp_pb.pb.pb_lower' failed. < === fails >> in assert() >> ; >> Aborted (core dumped) >> >> >> >> Some of my observation : >> >> in this file libraries/liblmdb/mdb.c >> in X86 : rc = mdb_cursor_set(&mc, &key, &data, MDB_SET, &exact); the >> value of >> rc = MD_SUCCESS, but for ppc64 it is MDB_NOTFOUND. This is due to the >> fact that >> md_root != 2 >> >> The value of md_root=2 for env->me_metas[1]->mm_dbs in X86 , but some >> huge value >> in ppc64. The value of md_pad = 4096 in x86 and 65536 in ppc64 . >> >> in x86: >> >> Breakpoint 2, mdb_txn_begin (env=0x9c5050, parent=0x0, flags=0, >> ret=0x7fffffffdf98) at ./../../../libraries/liblmdb/mdb.c:2219 >> 2219 int rc, size, tsize = sizeof(MDB_txn); >> (gdb) p env->me_metas[1]->mm_dbs >> $1 = {{md_pad = 4096, md_flags = 8, md_depth = 0, md_branch_pages = 0, >> md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = >> 18446744073709551615}, {md_pad = 0, md_flags = 0, >> md_depth = 1, md_branch_pages = 0, md_leaf_pages = 1, >> md_overflow_pages = 0, >> md_entries = 4, md_root = 2}} >> (gdb) p env->me_metas[0]->mm_dbs >> $2 = {{md_pad = 4096, md_flags = 8, md_depth = 0, md_branch_pages = 0, >> md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = >> 18446744073709551615}, {md_pad = 0, md_flags = 0, >> md_depth = 0, md_branch_pages = 0, md_leaf_pages = 0, >> md_overflow_pages = 0, >> md_entries = 0, md_root = 18446744073709551615}} >> >> >> in ppc64: >> >> Breakpoint 2, mdb_txn_begin (env=0x10493ef0, parent=0x0, flags=0, >> ret=0x3fffffffe590) at ./../../../libraries/liblmdb/mdb.c:2219 >> 2219 int rc, size, tsize = sizeof(MDB_txn); >> (gdb) p env->me_metas[0]->mm_dbs >> $1 = {{md_pad = 65536, md_flags = 8, md_depth = 0, md_branch_pages = 0, >> md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = >> 18446744073709551615}, {md_pad = 0, md_flags = 0, >> md_depth = 0, md_branch_pages = 0, md_leaf_pages = 0, >> md_overflow_pages = 0, >> md_entries = 0, md_root = 18446744073709551615}} >> (gdb) p env->me_metas[1]->mm_dbs >> $2 = {{md_pad = 65536, md_flags = 8, md_depth = 0, md_branch_pages = 0, >> md_leaf_pages = 0, md_overflow_pages = 0, md_entries = 0, md_root = >> 18446744073709551615}, {md_pad = 0, md_flags = 0, >> md_depth = 0, md_branch_pages = 0, md_leaf_pages = 0, >> md_overflow_pages = 0, >> md_entries = 0, md_root = 18446744073709551615}} >> >> >> >> >> # From further investigation , the value of env->me_metas[1] is >> initialized in >> mdb_env_open2() at : >> >> p = (MDB_page *)env->me_map; >> env->me_metas[0] = METADATA(p); >> env->me_metas[1] = (MDB_meta *)((char *)env->me_metas[0] + >> meta.mm_psize); >> >> Here the meta.mm_psize in ppc64 is 65536 , hence there is some >> problem.. If the >> value of meta.mm_psize is 4096 , then everything works fine. >> >> >> As i dont have deep knowledge in openldap ,some help is needed. >> >> Thank you, >> Sumanth K >> >> > >

1 0

Re: (ITS#7710) contextCSN values not updated by internal non-replicated ops
by hyc＠symas.com 01 Oct '13

01 Oct '13

Michael Ströder wrote: > hyc(a)symas.com wrote: >>> Configuration: >>> - MMR with 3 nodes and normal syncrepl. >>> - slapo-memberof used to populate/update attribute 'memberOf' in user entries. >>> >>> If a group entry is modified and slapo-memberof updates the attribute 'memberOf' >>> in the member's entry the contextCSN value of the server where the LDAP request >>> was sent to is not updated on the other MMR replicas. >> >> I don't understand this sentence, can you rephrase? > > I looked into this a bit more: > > We have three MMR nodes, let's simply enumerate them as 1, 2 and 3. > > When modifying a group entry on 1 'memberOf' gets *locally* updated and the > contextCSN value for server 1 gets also updated. So far so good. > > Now the group modification are pulled by 2 and 3 and slapo-memberof installed > there *locally* updates attr memberOf and the *local* contextCSN value for 2 > and 3 are updated with new timestamp. But the updated contextCSN values of 2 > and 3 never get replicated to the other instances! So my monitoring check > shows a long-lasting replication gap. After next server restart all contextCSN > values get updated. > >> Also see ITS#6915. > > Sorry, glancing over ITS#6915 I don't understand what it means in this context. The point of #6915 is that internal ops are not supposed to affect the contextCSN. If you're seeing new contextCSNs on 2 and 3 solely due to slapo-memberof then some part of #6915 is still broken. We'll need a duplicate setup to reproduce this... -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/

1 0

Re: (ITS#7710) contextCSN values not updated by internal non-replicated ops
by michael＠stroeder.com 01 Oct '13

01 Oct '13

hyc(a)symas.com wrote: >> Configuration: >> - MMR with 3 nodes and normal syncrepl. >> - slapo-memberof used to populate/update attribute 'memberOf' in user entries. >> >> If a group entry is modified and slapo-memberof updates the attribute 'memberOf' >> in the member's entry the contextCSN value of the server where the LDAP request >> was sent to is not updated on the other MMR replicas. > > I don't understand this sentence, can you rephrase? I looked into this a bit more: We have three MMR nodes, let's simply enumerate them as 1, 2 and 3. When modifying a group entry on 1 'memberOf' gets *locally* updated and the contextCSN value for server 1 gets also updated. So far so good. Now the group modification are pulled by 2 and 3 and slapo-memberof installed there *locally* updates attr memberOf and the *local* contextCSN value for 2 and 3 are updated with new timestamp. But the updated contextCSN values of 2 and 3 never get replicated to the other instances! So my monitoring check shows a long-lasting replication gap. After next server restart all contextCSN values get updated. > Also see ITS#6915. Sorry, glancing over ITS#6915 I don't understand what it means in this context. Ciao, Michael.

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

openldap-bugs