Even if slapd is started with slapd.conf, the cn=config database exists.
Thanks, didn't know that!
So: make sure you have credentials for accessing cn=config, and just use ldapmodify to change the olcTLS* attributes as needed.
Since the paths don't actually change (and I have no means to make them change), can I do a dummy modification that would trigger cert reloading?