John Nietzsche writes:
WARNING: No dynamic config support for database ldbm. WARNING: The converted cn=config directory is incomplete and may not work.
Therefore, before switching from slapd.conf to a slapd.d directory, you must switch from database ldbm to database bdb or hdb.
ldbm is obsolete anyway and has been removed from OpenLDAP 2.4.
Read the relevant sections about tuning cache size etc. in the OpenLDAP Admin Guide Berkeley DB reference first though. Which it looks like you should have done in any case, with no indexes except for objectClass you are not going to have stellar performance.