openldap and the magic of latin-1 versus unicode in schema

I just got done upgrading the OpenLDAP instance on my Gentoo system from version 2.3.43 to 2.4.19 only to find it non-functional after following the steps outlined in the warning. Well, here is how I fixed it.

When I first attempted the upgrade, I was warned that things could break and suggested I follow these steps:

 *  1. /etc/init.d/slurpd stop ; /etc/init.d/slapd stop
 *  2. slapcat -l /root/ldapdump.1258652071.raw
 *  3. egrep -v '^entryCSN:' /ldapdump.1258652071.raw >/root/ldapdump.1258652071
 *  4. mv /var/lib/openldap-data/ /var/lib/openldap-data-backup/
 *  5. emerge --update \=net-nds/openldap-2.4.19
 *  6. etc-update, and ensure that you apply the changes
 *  7. slapadd -l /root/ldapdump.1258652071
 *  8. chown ldap:ldap /var/lib/openldap-data/*
 *  9. /etc/init.d/slapd start
 * 10. check that your data is intact.
 * 11. set up the new replication system.

Well, I did this, but at step 9 things went wrong and I ended up with some funky error:

backend_startup: bi_db_open(0) failed!

I ended up trying various things, but after playing with debug options I also noticed the following in the error output

config_build_attrs: error 21 on olcAttributeTypes value #1
config_build_entry: build "cn={0}webavis" failed: ""

Which led me to OpenLDAP Bug 5755.
In the end it turns out the problem was a schema file with some latin-1 characters in it. Reading the it sounded like changing it to UTF-8 might fix my issue.

I ended up tracking it back to the webavis.conf schema file, by running:

cd /etc/openldap/schema
for x in *.schema; do iconv -f iso-8859-1 -t utf-8 ${x} > ${x}.new; done
for x in *.schema; do echo ${x}; diff ${x} ${x}.new; done

The differences looked like this:

         DESC 'Virtual user of mail système'

I simply overwrote the webavis.conf file with cleaned up one and all my woes were gone.

Hear Hear for iconv and UTF-8!