Thanks alot, for some reason my keyids were different. It's strange because I connected using the openssl s_server and s_client modes with the same key/certs and it connects. I was using the the scripts from a walkthrough to generate my CA and certs and key from http://web.singnet.com.sg/~garyttt/. I must of done some setup wrong or something for that script to not work, although the walkthrough did try to set a slave ldap server also, and I currently don't have one. Anyways used the CA.pl script that came with openssl and that worked.
Thanks again, Vinh