openldap-technical

openldap-technical@openldap.org

3 participants
9898 discussions

ch_malloc of 0 bytes failed on Meta backend
by sumanbadra＠gmail.com 01 Mar '23

01 Mar '23

Hi, We have been using Meta backend with multiple AD domain controllers as targets. OpenLDAP version is 2.4.56 and installed on RHEL 8.6 Azure hosted servers. We have two such nodes with the same setup. I have been trying to find the root cause for the error "ch_malloc of 0 bytes failed on Meta backend" found on the two servers. Service was crashed on both servers after this error. After starting the OpenLDAP service, it has been working fine so far. My observations so far: - Our Azure team did not find any abnormalities on the servers w.r.t the memory utilisation by LDAP during the incident. - Stats around the time when issue occurred sar -r kbmemfree kbavail kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty 01:50:01 PM 25202860 29022816 7519888 22.98 13808 5385664 4736872 14.48 2349572 4157132 64 02:00:01 PM 25195656 29018052 7527092 23.00 13808 5388120 4767748 14.57 2349572 4163232 44 02:10:00 PM 25197340 29010572 7525408 23.00 13808 5386752 4783920 14.62 2352524 4158856 3336 02:20:01 PM 25201220 29016192 7521528 22.99 13808 5388296 4789868 14.64 2352516 4154632 52 02:30:01 PM 25196500 29013836 7526248 23.00 13808 5390708 4789412 14.64 2352516 4159140 36 02:40:01 PM 25181840 29002912 7540908 23.04 13808 5402364 4782228 14.61 2352528 4173032 144 02:50:01 PM 25170748 28994608 7552000 23.08 13808 5413236 4758840 14.54 2352516 4185172 1092 03:00:01 PM 25172224 28997852 7550524 23.07 13808 5414920 4729152 14.45 2352520 4182356 48 03:10:01 PM 24976028 28806984 7746720 23.67 13808 5428036 5007976 15.30 2353012 4375608 4940 - We did not set any cache specifically as meta does not store any data - Error shows 0 bytes rather than any specific number I would really appreciate if you can share any pointer which would help dig further. Thank you, Suman

2 2

lmdb partial update of existing records
by g.fanini＠gmail.com 28 Feb '23

28 Feb '23

What is the recommended way to update partial substrings of existing records in lmdb ? I could not find a way that allows to transfer not the entire record, thus wasting memory bandwidth, but only say overwrite a single byte within say a 1 mbyte record, is it acceptable/safe to modify : mdb_put(offset, partial_len) mdb_cursor_put(offset, partial_len) ... if (F_ISSET(flags, MDB_RESERVE)) data->mv_data = olddata.mv_data; else if (!(mc->mc_flags & C_SUB)) { if(!offset && !partial_len) memcpy(olddata.mv_data, data->mv_data, data->mv_size); else memcpy((char*)olddata.mv_data+offset, data->mv_data, partial_len);// allow partial update within existing record at offset } else { memcpy(NODEKEY(leaf), key->mv_data, key->mv_size); goto fix_parent; } ... further to that, is it possible to allow retrieving current record size, data with MDB_RESERVE, and what should be done if anything in case one opts not to write anything into the returned data } else if ((data->mv_size == olddata.mv_size) || F_ISSET(flags, MDB_RESERVE) )// allow retrieving current record size, data with MDB_RESERVE { /* same size, just replace it. Note that we could * also reuse this node if the new data is smaller, * but instead we opt to shrink the node in that case. */ if (F_ISSET(flags, MDB_RESERVE)) { if (!data->mv_size) *data = olddata;// allow retrieving current size, data with MDB_RESERVE by querying with size 0 else data->mv_data = olddata.mv_data; }

1 0

dynlist modul pageresult bug
by Andreas Ladanyi 28 Feb '23

28 Feb '23

Hi, is the pageresult issue of dynlist solved in the new 2.5.14 release of slapd or will it be solved in the near future ? cheers, Andi

2 1

Content Sync Refresh Required
by Geert Hendrickx 28 Feb '23

28 Feb '23

Hi We're on OpenLDAP 2.5.14, built from source on EL 8. After dropping the accesslog on an MMR master, we keep seeing on all replica's: > do_syncrep2: rid=xxx (4096) Content Sync Refresh Required and all replica's keep falling back to refresh mode every few minutes. It's strange because we do this more often (typically after batch updates, which leaves a large freelist in the accesslog upon pruning), but it's the first time we hit this issue. We stopped the master (at a time no updates are coming in), drop the accesslog, and start it again. Does it make sense to wait until this fixes itself? Or should I fix this manually somehow? I already tried mdb_copying the provider db to the replica's, but the same thing keeps happening. In the meantime replication is working very slowly... Geert

2 8

META Ldap
by bourguijl＠gmail.com 27 Feb '23

27 Feb '23

Dears, I've created a META configuration pointing to another backend ldap for which I'd like to use a generic user which will be used as unique user to fetch datas in backend requested by all users coming from the META proxy frontend. I did following dynamic configuration : dn: olcDatabase={2}meta objectClass: olcDatabaseConfig objectClass: olcMetaConfig olcDatabase: {2}meta olcSuffix: o=mobistar.be olcAddContentAcl: FALSE olcLastBind: TRUE olcReadOnly: FALSE olcRootDN: cn=directory manager,o=mobistar.be olcRootPW: secret olcSyncUseSubentry: FALSE olcMonitoring: FALSE olcDbOnErr: continue olcDbPseudoRootBindDefer: TRUE olcDbSingleConn: FALSE olcDbUseTemporaryConn: FALSE olcDbConnectionPoolMax: 16 olcDbBindTimeout: 1000000 olcDbCancel: abandon olcDbChaseReferrals: FALSE olcDbNoRefs: FALSE olcDbNoUndefFilter: FALSE olcDbProtocolVersion: 3 olcDbRebindAsUser: FALSE olcDbSessionTrackingRequest: FALSE olcDbTFSupport: no structuralObjectClass: olcMetaConfig entryUUID: c113f986-35b0-103d-9f4f-85924223dda7 creatorsName: cn=config createTimestamp: 20230131124432Z olcMaxDerefDepth: 15 olcDbNretries: 100 olcLastMod: FALSE entryCSN: 20230227112001.500938Z#000000#001#000000 modifiersName: cn=manager,cn=config modifyTimestamp: 20230227112001Z dn: olcMetaSub={0}uri objectClass: olcMetaTargetConfig olcMetaSub: {0}uri olcDbKeepalive: 0:0:0 olcDbTcpUserTimeout: 0 olcDbCancel: abandon olcDbChaseReferrals: FALSE olcDbNoRefs: FALSE olcDbNoUndefFilter: FALSE olcDbNretries: 100 olcDbProtocolVersion: 3 olcDbRebindAsUser: FALSE olcDbSessionTrackingRequest: FALSE olcDbTFSupport: no structuralObjectClass: olcMetaTargetConfig entryUUID: c113fc9c-35b0-103d-9f50-85924223dda7 creatorsName: cn=config createTimestamp: 20230131124432Z olcDbBindTimeout: 1000000 olcDbURI: "ldap://accmasterldapcorp.nonprod.priv.orange.be:389/o=mobistar.be olcDbIDAssertBind: mode=self flags=non-prescriptive,proxy-authz-non-critical b indmethod=simple timeout=0 network-timeout=0 binddn="uid=ldapproxyuser_acc,ou =test,ou=system,o=mobistar.be" credentials="secret" keepalive=0:0:0 tc p-user-timeout=0 entryCSN: 20230227142538.253522Z#000000#001#000000 modifiersName: cn=manager,cn=config modifyTimestamp: 20230227142538Z But when I do a ldapsearch with a user known in the backend from the META, it'll not take the olcDbIDAssertBind and it didn't found nothing. If I do the same request directly on the backend, I get what I'm searching for. Can you help me by giving me some advice about what I'm missing or what's erroneous ? Thx in advance, Jean_luc.

1 0

dynlist dont deliver memberof attribute
by Andreas Ladanyi 27 Feb '23

27 Feb '23

Hi, iam using one 2.5 Master / Provider / syncprov and some 2.5 Slaves / Consumers / syncrepl. I added the dynlist to generate memberOf attribute to slapd.conf on Master and all Slaves. Problem is only on some slaves the dynlist doesnt generate memberof attribute output when ldapsearch to a user. Iam using the objectClass labeledURIObject and attribute labeledURI to store the LDAP URI for dynlist to trigger / generate the DN of group membership for memberof attribute of the user. The labeledURI attribute is replicated successfully. User entry output on non working slaves with attribute labeledURI, memberof is missing: ldapsearch -x -LLL -ZZ -H ldap://non_working_slave -b 'ou=X,dc=department,dc=organization,dc=X,dc=X' '(&(uid=X))' results in the user entry with all objectClasses and all attributes except the memberof attribute. #start snip: ... objectClass: labeledURIObject ... labeledURI: ldap:///dc=department,dc=organization,dc=X,dc=X??sub?(&(objectClass=groupOfNames)(member=uid=XXXX,ou=account,ou=X,dc=department,dc=organization,dc=X,dc=X)) #stop snip slapd.conf: overlay dynlist dynlist-attrset labeledURIObject labeledURI memberOf The difference between working and non working slaves is the length of ACL list. The important ACL entry is: access to dn.sub=dc=department,dc=organization,dc=X,dc=X \ attrs=entry by peername=IP_subnet read by * break access to dn.regex=^[^,]+,ou=(account|group|groupOfNames),ou=X,dc=department,dc=organization,dc=X,dc=X$ by peername=IP_subnet read by * break i set no attrs= parameter at "access to dn.regex" rule to output all attributes. cheers, Andreas

1 1

ACL issue
by eric.innocente＠univ-avignon.fr 23 Feb '23

23 Feb '23

Hi, Using OpenLDAP 2.4 and this ACL : ===== olcAccess: {0}to * by dn="cn=admin,ou=ldap,dc=univ-avignon,dc=fr" write by * break olcAccess: {1}to attrs=userPassword by self write by anonymous auth by * none olcAccess: {2}to attrs=myAttribute by dn="cn=myUser,ou=ldap,dc=univ-avignon,dc=fr" read by * none olcAccess: {3}to * by * read ===== in the aim (rule {2}) to grant read access to attribute 'myAttribute' for myUser and no other user (except admin). As wanted, [R1] with authentified user myUser : [R1] ldapsearch -x -LLL -h <myLDAP> -b 'ou=people,dc=univ-avignon,dc=fr' -D 'cn=myUser,ou=ldap,dc=univ-avignon,dc=fr' -w <secret> "(uid=someUid)" myAttribute give me the dn and the required "myAttribute" : dn: uid=someUid,ou=people,dc=univ-avignon,dc=fr myAttribute: <attribute value> and [R2] with another authentified user : [R2] ldapsearch -x -LLL -h <myLDAP> -b 'ou=people,dc=univ-avignon,dc=fr' -D 'cn=anotherUser,ou=ldap,dc=univ-avignon,dc=fr' -w <secret> "(uid=someUid)" myAttribute does NOT give me the required "myAttribute", only the dn : dn: uid=someUid,ou=people,dc=univ-avignon,dc=fr BUT by replacing "read" by "none" in rule {3}, I get an error "No such object (32)" with either [R1] and [R2]. Since rule {3} should not be evaluated after matching rule {2}, I don't understand why modifying rule {3} modifies the behaviour. And by replacing "read" by "search" in rule {3}, I no longer get an error, but I do NOT obtain the required "myAttribute" and nor the dn, with neither [R1] nor [R2]. Does it mean that "read" in rule {3} was necessary to read the dn ? And that without reading the dn, rule {2} cannot be evaluated ? Please, help me ! Eric

2 1

Using Ppolicy in a provider peer cluster can trigger consumer refresh condition
by James Rawlins 22 Feb '23

22 Feb '23

Hi there, My ldap consists of a cluster of 3 providers that all replicate to each other, and a fleet of consumers replicating from them, and we have ppolicy installed on our providers and consumers both, though we're currently not using it to enforce any particular policies automatically. I've recently discovered a situation where a script could fail to login with a non-rootDN service account to all three provider instances in short order. The providers seem to be able to figure things out quickly, but the consumers sometimes detect some ContextCSN inconsistency and when this happens consumers to enter a REFRESH state caused by updates to the operational attributes on the service account's dn entry in all three providers at nearly the same time. In production this can cause latency due to the extra CPU and network traffic of refreshing all consumers at once. The only relevant documentation I've been able to find for this use case are from https://linux.die.net/man/5/slapo-ppolicy: Note that the current IETF Password Policy proposal does not define how these operational attributes are expected to behave in a replication environment. In general, authentication attempts on a slave server only affect the copy of the operational attributes on that slave and will not affect any attributes for a user's entry on the master server. Operational attribute changes resulting from authentication attempts on a master server will usually replicate to the slaves (and also overwrite any changes that originated on the slave). These behaviors are not guaranteed and are subject to change when a formal specification emerges. And the related ability for consumers to send updates up a replication chain using the chain overlay: ppolicy_forward_updates Specify that policy state changes that result from Bind operations (such as recording failures, lockout, etc.) on a consumer should be forwarded to a master instead of being written directly into the consumer's local database. This setting is only useful on a replication consumer, and also requires the updateref setting and chain overlay to be appropriately configured. tl;dr Ppolicy wrote operational attributes to a service account's dn to all of my provider instances at the same time when my SA used the wrong password to login to them all at once, and caused all my consumers to refresh at the same time. My question is: is the ppolicy overlay inherently unsafe for a provider cluster? Right now I'm considering these options to get rid of the risk of accidentally triggering a consumer REFRESH again: - Remove the ppolicy overlay from the replicated backend (I'm still checking if there's anything we actually use it for, but if it's inherently unsafe in this configuration then it's gotta go) - Move all of the service accounts to another database that ppolicy is not installed on. Thanks!

2 1

Re: Antw: [EXT] Re: Entering Multi-Byte Values for DirectoryString attributes
by Ede Wolf 22 Feb '23

22 Feb '23

> Hi! > > I'd say: Get the proper app (eg. BabelPad on Windows, see attachment) and then BASE64-encode the string. > > Regards, > Ulrich > Thanks, but windows is not an option. Again, base64 encoding is fine with echo -en "" | base64. With a browser I can search for the symbol and do copy paste into the ldif. All that works fine, even if my console font does not even support that symbol. I've done it. But feels a bit hackish. I just hoped, there was an option, to tell the server, when the ldif has this sequence, interpret it as an ecoding, not as a literal string. Now I know, this does only work for the dn (and maybe only openldap, will test, see other post), and for the rest I do have a work around.

2 3

Entering Multi-Byte Values for DirectoryString attributes
by Ede Wolf 22 Feb '23

22 Feb '23

Hello, This is probably more a ldif than an OpenLDAP question, but still, maybe somebody knows the answer: Is there a way to put multibyte characters into an attribute value and let the server know, these are not to be treated literally, but are utf8 character encodings? I've tried to dig into rfc3629 and 4517, but those were above my capabilities. It does of course work for the dn, it also works, if I provide base64 code to the attributes, but is there a way, to directly put them into a ldif an let the server know, these are character encodings? Also, rfc2849 only talks about not line breaking multi-byte characters. In this silly, but easy, example, both cn: and description: are entered literally, while the dn words as intended: dn: cn=A \F0\9F\99\82 Test,dc=example,dc=com cn: A \F0\9F\99\82 Test objectClass: person sn: Test description: %xF0%x9F%x99%x82 Test This is about understanding, not about the intention, to really put a smily into a dn. I am aware, this a potential recipe for disaster. Also, I am aware, the OpenLDAP kindly adds a proper cn value anyway, but that does not help here. And still would leave the description open. Also, as mentioned before: echo -en "A \xF0\x9F\x99\x82 Test" | base64 is a viable workaround, but a cumbersome one. So maybe there is an easier way Thanks Ede

4 11

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

openldap-technical