<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body>
    <p>Hi,</p>
    <p>OK good to see the syncrepl directives are the same/consistent.
      Note you're specifying a cert, key and cacert file specifically
      for syncrepl in your config.<br>
    </p>
    <p>RE: /etc/ldap/ldap.conf this is used by the client utilities e.g.
      openldap so what I think is happening is that as the certs
      presented by your LDAP servers isn't trusted on at least one of
      your servers (when acting as a client/performing queries) the
      client commands e.g. ldapsearch are failing to negotiate a TLS
      connection.</p>
    <p>There will also be global TLS settings which will determine which
      cert your servers will present to clients e.g.</p>
    <p>olcTLSCACertificateFile: /opt/openldap/certs/quovadis.bundle<br>
      olcTLSCertificateFile:
      /opt/openldap/certs/authorise-test.is.ed.ac.uk-QUOVADIS.crt<br>
      olcTLSCertificateKeyFile:
      /opt/openldap/certs/authorise-test.is.ed.ac.uk-QUOVADIS.key</p>
    <p>The ca-certificates.crt is a client-managed bundle file (usually
      managed by a ca-certificates package on the system?)</p>
    <p>Hope this explains/describes the situation and setup a bit
      better?</p>
    <p>I can supply a reference config for comparison if that's useful
      off-list although our config has a lot of custom attributes/schema
      entries and multiple databases as we use delta-syncrepl rather
      than standard syncrepl.<br>
    </p>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 21/05/2022 12:24, Tahir Hafiz wrote:<br>
    </div>
    <blockquote type="cite" cite="mid:CALmb6fsTG945=Pv3H31OQDp_A0zCUoYq9POJgZjhG9uWzGzzRA@mail.gmail.com">
      
      <div style="background-color:#fff2e6; border:2px dotted #ff884d"><span style="font-size:12pt; font-family: sans-serif; color:black;
          font-weight:bold; padding:.2em">This email was sent to you by
          someone outside the University.</span>
        <div style="font-size:10pt; font-family: sans-serif;
          font-style:normal; padding:.2em">
          You should only click on links or attachments if you are
          certain that the email is genuine and the content is safe.</div>
      </div>
      <div>
        <div dir="ltr">
          <div dir="ltr">
            <div>Hmmm....Thanks Mark<br>
            </div>
            <div><br>
            </div>
            <div>On sso1 ldap and on sso2 ldap
              (/etc/ldap/slapd.d/cn=config/olcDatabase={1}hdb.ldif) I
              see this:<br>
              olcSyncrepl: {0}rid=004 provider=<a class="moz-txt-link-freetext" href="ldaps://">ldaps://</a><a href="http://sso1.redacted.net/" moz-do-not-send="true">sso1.redacted.net/</a>
              binddn="cn=admin<br>
               ,dc=redacted,dc=net" bindmethod=simple
              credentials=REDACTED searchb<br>
               ase="dc=redacted,dc=net" type=refreshOnly
              tls_cacert=/etc/ssl/certs/GandiSt<br>
               andardSSLCA2.pem
              tls_cert=/etc/ssl/certs/redacted-2018.crt tls_key=/etc/ssl<br>
               /private/redacted.net.key interval=00:00:00:10 retry="5 5
              300 5" timeout=1<br>
              olcSyncrepl: {1}rid=005 provider=<a class="moz-txt-link-freetext" href="ldaps://">ldaps://</a><a href="http://sso2.redacted.net/" moz-do-not-send="true">sso2.redacted.net/</a>
              binddn="cn=admin<br>
               ,dc=redacted,dc=net" bindmethod=simple
              credentials=REDACTED searchb<br>
               ase="dc=redacted,dc=net" type=refreshOnly <b>tls_cacert=/etc/ssl/certs/GandiSt<br>
                 andardSSLCA2.pem</b>
              tls_cert=/etc/ssl/certs/redacted-2018.crt tls_key=/etc/ssl<br>
               /private/redacted.net.key interval=00:00:00:10 retry="5 5
              300 5" timeout=1<br>
            </div>
            <div><br>
            </div>
            <div>So those entries do look the same on both sso1 ldap and
              sso2 ldap for the file olcDatabase={1}hdb.ldif but on sso1
              ldap for
              <br>
              <div>/etc/ldap/ldap.conf I see: <br>
              </div>
              <div>
                <div>
                  <div><b>TLS_CACERT /etc/ssl/certs/ca-certificates.crt
                    </b><br>
                  </div>
                </div>
                <div><b><br>
                  </b></div>
                <div>And on sso2 ldap for /etc/ldap/ldap.conf I see<b>:
                    <br>
                  </b></div>
                <div>
                  <div>
                    <div><b>TLS_CACERT
                        /etc/ssl/certs/RapidSSLIntermediate.pem</b> <br>
                    </div>
                  </div>
                  <div><br>
                  </div>
                  <div><br>
                  </div>
                  <div>From what I can tell <b>ca-certificates</b> (on
                    sso1 ldap from /etc/ldap/ldap.conf) is a bundle of
                    different certificates,
                    <b>RapidSSLIntermediate.pem</b> (on sso2 ldap from
                    /etc/ldap/ldap.conf) is a single certificate and the
                    <b>GandiSt<br>
                       andardSSLCA2.pem</b> (listed on both sso1 ldap
                    and sso2 ldap, from file olcDatabase={1}hdb.ldif) is
                    a single certificate (Intermediate certificate from
                    the registrar Gandi). I'm not sure why the entries
                    are all different - should they all be the same cert
                    ideally?<br>
                    <br>
                  </div>
                  <div>I also don't understand how synch between the two
                    ldap servers is working if the TLS is broken.<br>
                  </div>
                </div>
              </div>
            </div>
            <div><br>
            </div>
          </div>
          <br>
          <div class="gmail_quote">
            <div dir="ltr" class="gmail_attr">On Fri, May 20, 2022 at
              10:41 PM CAIRNEY Mark <<a href="mailto:Mark.Cairney@ed.ac.uk" moz-do-not-send="true" class="moz-txt-link-freetext">Mark.Cairney@ed.ac.uk</a>>
              wrote:<br>
            </div>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px
              0.8ex;border-left:1px solid
              rgb(204,204,204);padding-left:1ex">
              <div dir="auto">Hi, 
                <div>It’s been a while since I looked at OpenLDAPs certs
                  (and it’s behavior differs slightly depending on which
                  SSL lib it was linked against). I think Debian linked
                  against openssl though. The TLSCACert should point to
                  a bundle containing all the roots and intermediates of
                  any certs in okay (it’s not fussy about order but
                  needs the full set. It’s more of a trust store than a
                  cert chain in that respect). The other thing is
                  ensuring that the server cert has the correct FQDNs in
                  it. We have a common cert with all the server CNs as
                  Subject AltNames to ensure consistency. </div>
                <div>This is all dredged from memory, as you can guess
                  from the sig I’m not currently sitting in front of an
                  LDAP Server to confirm! ;)<br>
                  <br>
                  <div dir="ltr">Sent from my iPhone</div>
                  <div dir="ltr"><br>
                    <blockquote type="cite">On 20 May 2022, at 20:50,
                      Tahir Hafiz <<a href="mailto:tahir.hafiz@gmail.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">tahir.hafiz@gmail.com</a>>
                      wrote:<br>
                      <br>
                    </blockquote>
                  </div>
                  <blockquote type="cite">
                    <div dir="ltr">
                      <div style="background-color:rgb(255,242,230);border:2px
                        dotted rgb(255,136,77)">
                        <span style="font-size:12pt;font-family:sans-serif;color:black;font-weight:bold;padding:0.2em">This
                          email was sent to you by someone outside the
                          University.</span>
                        <div style="font-size:10pt;font-family:sans-serif;font-style:normal;padding:0.2em">You
                          should only click on links or attachments if
                          you are certain that the email is genuine and
                          the content is safe.</div>
                      </div>
                      <div>
                        <div dir="ltr">Thanks Mark, I realised I never
                          restarted slapd on the sso2 ldap box when I
                          renewed the certs there and this what I see
                          upon restarting the slapd service:<br>
                          <div><br>
                          </div>
                          <div>May 20 19:35:36 sso2 slapd[17605]: slapd
                            starting</div>
                          May 20 19:35:36 sso2 slapd[17605]:
                          slap_client_connect: URI=<a class="moz-txt-link-freetext" href="ldaps://">ldaps://</a><a href="http://sso1.redacted.net/" target="_blank" moz-do-not-send="true">sso1.redacted.net/</a>
                          TLS context initialization failed (-1)<br>
                          May 20 19:35:36 sso2 slapd[17605]:
                          do_syncrepl: rid=001 rc -1 retrying (4 retries
                          left)<br>
                          May 20 19:35:36 sso2 slapd[17605]:
                          slap_client_connect: URI=<a class="moz-txt-link-freetext" href="ldaps://">ldaps://</a><a href="http://sso1.redacted.net/" target="_blank" moz-do-not-send="true">sso1.redacted.net/</a>
                          DN="cn=admin,dc=redacted,dc=net"
                          ldap_sasl_bind_s failed (-1)<br>
                          <div>May 20 19:35:36 sso2 slapd[17605]:
                            do_syncrepl: rid=004 rc -1 retrying (4
                            retries left)</div>
                          <div><br>
                          </div>
                          <div>And on the sso1 ldap box I still see this
                            upon restarting slapd there:<br>
                            <br>
                            May 20 19:36:26 sso1 slapd[16803]: slapd
                            starting<br>
                            May 20 19:36:26 sso1 slapd[16803]:
                            slap_client_connect: URI=<a class="moz-txt-link-freetext" href="ldaps://">ldaps://</a><a href="http://sso2.redacted.net/" target="_blank" moz-do-not-send="true">sso2.redacted.net/</a>
                            TLS context initialization failed (-1)<br>
                            May 20 19:36:26 sso1 slapd[16803]:
                            do_syncrepl: rid=002 rc -1 retrying (4
                            retries left)<br>
                            May 20 19:36:32 sso1 slapd[16803]: SASL
                            [conn=1001] Failure: no secret in database</div>
                          <div><br>
                          </div>
                          <div>So sso2 ldap box seems to be doing a bit
                            better (one less error).<br>
                            Is there a way I can troubleshoot which
                            certificates in the chain are causing the
                            issue - like some interesting commands?</div>
                          <div>I will try and look at the SASL config
                            too to see where the secret is missing.<br>
                          </div>
                          <div><br>
                          </div>
                          <div>Unfortunately, these seem to be quite
                            dated Ubuntu 14 boxes. <br>
                          </div>
                          <div><br>
                          </div>
                          <div>Thank you!<br>
                          </div>
                        </div>
                        <br>
                        <div class="gmail_quote">
                          <div dir="ltr" class="gmail_attr">On Fri, May
                            20, 2022 at 7:49 PM CAIRNEY Mark <<a href="mailto:Mark.Cairney@ed.ac.uk" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">Mark.Cairney@ed.ac.uk</a>>
                            wrote:<br>
                          </div>
                          <blockquote class="gmail_quote" style="margin:0px 0px 0px
                            0.8ex;border-left:1px solid
                            rgb(204,204,204);padding-left:1ex">
                            <div dir="auto">Hi, 
                              <div>I think there are 2 issues at play</div>
                              <div>1. The certs/CA isn’t right on one of
                                the servers (hence the TLS errors) do
                                the syncrepl bind is failing.</div>
                              <div>2. It looks like SASL can’t find a
                                password entry for the user in the bind
                                attempt. Might be worth checking your
                                SASL config to ensure the mechanisms you
                                expect are available and your
                                authzregexp is set up correctly.<br>
                                <br>
                                <div dir="ltr">Sent from my iPhone</div>
                                <div dir="ltr"><br>
                                  <blockquote type="cite">On 20 May
                                    2022, at 18:56, Tahir Hafiz <<a href="mailto:tahir.hafiz@gmail.com" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">tahir.hafiz@gmail.com</a>>
                                    wrote:<br>
                                    <br>
                                  </blockquote>
                                </div>
                                <blockquote type="cite">
                                  <div dir="ltr">
                                    <div style="background-color:rgb(255,242,230);border:2px
                                      dotted rgb(255,136,77)">
                                      <span style="font-size:12pt;font-family:sans-serif;color:black;font-weight:bold;padding:0.2em">This
                                        email was sent to you by someone
                                        outside the University.</span>
                                      <div style="font-size:10pt;font-family:sans-serif;font-style:normal;padding:0.2em">You
                                        should only click on links or
                                        attachments if you are certain
                                        that the email is genuine and
                                        the content is safe.</div>
                                    </div>
                                    <div>
                                      <div dir="ltr">
                                        <div>Hi, <br>
                                        </div>
                                        <div><br>
                                        </div>
                                        <div>We have two OpenLDAP
                                          servers (sso1 and sso2, ignore
                                          alpha - that one I think they
                                          decommissioned it years ago).</div>
                                        <div>sso1 and sso2 are meant to
                                          be in mirror mode (sometimes
                                          called multi-master mode).</div>
                                        <div>I had to switch sso1 off a
                                          while ago because it was no
                                          longer responding and didn't
                                          have much time to look at it
                                          back then.
                                          <br>
                                        </div>
                                        <div><br>
                                        </div>
                                        <div>I have now had some spare
                                          time to look at it, updated
                                          the web certs which had to be
                                          renewed and restarted the
                                          openldap server in question.
                                          <br>
                                        </div>
                                        <div>But I see the following
                                          error (and Google has not
                                          helped much on this one):<br>
                                          <br>
                                          <div>May 20 16:11:44 sso1
                                            slapd[9008]: slapd starting</div>
                                          <div>May 20 16:11:44 sso1
                                            slapd[9008]:
                                            slap_client_connect:
                                            URI=<a class="moz-txt-link-freetext" href="ldaps://">ldaps://</a><a href="http://alpha.redacted.net/" target="_blank" moz-do-not-send="true">alpha.redacted.net/</a>
                                            TLS context initialization
                                            failed (-1)</div>
                                          <div>May 20 16:11:44 sso1
                                            slapd[9008]: do_syncrepl:
                                            rid=003 rc -1 retrying (4
                                            retries left)</div>
                                          <div>May 20 16:11:44 sso1
                                            slapd[9008]:
                                            slap_client_connect:
                                            URI=<a class="moz-txt-link-freetext" href="ldaps://">ldaps://</a><a href="http://sso2.redacted.net/" target="_blank" moz-do-not-send="true">sso2.redacted.net/</a>
                                            TLS context initialization
                                            failed (-1)</div>
                                          <div>May 20 16:11:44 sso1
                                            slapd[9008]: do_syncrepl:
                                            rid=002 rc -1 retrying (4
                                            retries left)</div>
                                          <div>May 20 16:12:13 sso1
                                            slapd[9008]: SASL
                                            [conn=1001] Failure: no
                                            secret in database</div>
                                          <div><br>
                                          </div>
                                          <div>The weird thing is the
                                            OpenLDAP sso1 box is
                                            synching to sso2 ldap box,
                                            and I can connect to it with
                                            an ldap client on my home
                                            desktop and it now has the
                                            latest records so it is
                                            working as an ldap server
                                            but I'm not sure what the
                                            errors really mean.</div>
                                          <div>Are there any avenues I
                                            can explore on this or has
                                            anyone seen something like
                                            this before (N.B. I am no
                                            ldap expert) ?
                                            <br>
                                          </div>
                                          <div><br>
                                          </div>
                                          <div>Thanking you in advance,</div>
                                          <div>Tahir<br>
                                          </div>
                                        </div>
                                      </div>
                                    </div>
                                    <span>-- </span><br>
                                    <span>EdLUG mailing list</span><br>
                                    <span><a href="mailto:EdLUG@mailman.lug.org.uk" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">EdLUG@mailman.lug.org.uk</a></span><br>
                                    <span><a href="https://lists.edlug.org.uk/mailman/listinfo/edlug" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://lists.edlug.org.uk/mailman/listinfo/edlug</a></span><br>
                                  </div>
                                </blockquote>
                              </div>
                              The University of Edinburgh is a
                              charitable body, registered in Scotland,
                              with registration number SC005336. Is e
                              buidheann carthannais a th’ ann an
                              Oilthigh Dhùn Èideann, clàraichte an Alba,
                              àireamh clàraidh SC005336.
                            </div>
                            -- <br>
                            EdLUG mailing list<br>
                            <a href="mailto:EdLUG@mailman.lug.org.uk" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">EdLUG@mailman.lug.org.uk</a><br>
                            <a href="https://lists.edlug.org.uk/mailman/listinfo/edlug" rel="noreferrer" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://lists.edlug.org.uk/mailman/listinfo/edlug</a><br>
                          </blockquote>
                        </div>
                      </div>
                      <span>-- </span><br>
                      <span>EdLUG mailing list</span><br>
                      <span><a href="mailto:EdLUG@mailman.lug.org.uk" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">EdLUG@mailman.lug.org.uk</a></span><br>
                      <span><a href="https://lists.edlug.org.uk/mailman/listinfo/edlug" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://lists.edlug.org.uk/mailman/listinfo/edlug</a></span><br>
                    </div>
                  </blockquote>
                </div>
              </div>
              -- <br>
              EdLUG mailing list<br>
              <a href="mailto:EdLUG@mailman.lug.org.uk" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">EdLUG@mailman.lug.org.uk</a><br>
              <a href="https://lists.edlug.org.uk/mailman/listinfo/edlug" rel="noreferrer" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://lists.edlug.org.uk/mailman/listinfo/edlug</a><br>
            </blockquote>
          </div>
        </div>
      </div>
      <br>
      <fieldset class="moz-mime-attachment-header"></fieldset>
    </blockquote>
    <pre class="moz-signature" cols="72">-- 
/****************************

Mark Cairney
ITI Enterprise Services
Information Services
University of Edinburgh

Tel: 0131 650 6565
Email: <a class="moz-txt-link-abbreviated" href="mailto:Mark.Cairney@ed.ac.uk">Mark.Cairney@ed.ac.uk</a>

*******************************/</pre>
  </body>
</html>