[Bug / Issue] No umlauts could be found in standard search of version 5.5.6

[OpenEnd]

Hello Support,

we have upgraded from vb3.6.8 to vb5.5.6

We use German language pack (Du)

Using the SQL standard search machine now, it is not possible to search words, which includes umlauts. If you search for Verstärker (amplifier), you'll get no result. If you search instead for Verstaerker, you get listed all Verstaerker in yellow marked and also all Verstärker, but not yellow marked.

Same is für names including umlauts. Rüdiger you cannot find. Ruediger instead is possible.

Would it be better to use the Sphinx search machine?

What to do?

Greetz Charly

[Trevor Hannant]

What are the table collations in your database? What is the charset of your language pack here:

AdminCP > Languages & Phrases > Language Manager

[OpenEnd]

Hello Trevor,

the language pack "Deutsch (Du)" is installed.

Greetz Charly

[Wayne Luke]

We need to know what the specific character set is being used. This is something that can be altered by end users in the AdminCP regardless of what the language is called.

In addition to this, if your site is using a latin1 character set in the database, then these characters will not always be searchable on old content. Especially if you upgraded. vBulletin 3 and 4 do not support UTF-8 characters which means they do not support anything outside the basic 256 characters. Anything outside of this, including umlauts on modern computer systems are converted to HTML Entities when stored in the database. In older versions of vBulletin 5, we try to counter this by converting search terms to their HTML Entities as well but it does not always work.

To resolve this for future content, we recommend that customers convert their databases to a utf8mb4 character set (with a corresponding utf8mb4 collation) as this will support the thousands of UTF-8 characters in the modern specification including the Emojis that are popular on mobile devices. We have provided scripts to do this in the do_not_upload/dbtools folder of the vBulletin package.

In addition to this, if you use a MySQL collation with _ai attached, searches will be accent insensitive so any character with an umlaut or other accent will be treated the same as the base character.

[OpenEnd]

OK, I'll try to get the version of the character set from my data base. These database was updated by Trevor during the upgrade from vb3.6.8 to vb5.5.5. You know, we had a lot of trouble. After Upgrade to vb5.5.6 by VBulletin support forum worked, but now we found out this trouble with search machine.

Greetz Charly

[Wayne Luke]

You can see the collation of every table in the AdminCP under Maintenance -> Diagnostics -> System Information. Set the drop down to Table Status.

[OpenEnd]

My PHP admin site told me, that the database contents latin1_swedish_ci. This was installed durin upgrade from your side.
Search trouble comes up also with new entries.

Greetz Charly

[OpenEnd]

We need to know what the specific character set is being used. This is something that can be altered by end users in the AdminCP regardless of what the language is called.

In addition to this, if your site is using a latin1 character set in the database, then these characters will not always be searchable on old content. Especially if you upgraded. vBulletin 3 and 4 do not support UTF-8 characters which means they do not support anything outside the basic 256 characters. Anything outside of this, including umlauts on modern computer systems are converted to HTML Entities when stored in the database. In older versions of vBulletin 5, we try to counter this by converting search terms to their HTML Entities as well but it does not always work.

To resolve this for future content, we recommend that customers convert their databases to a utf8mb4 character set (with a corresponding utf8mb4 collation) as this will support the thousands of UTF-8 characters in the modern specification including the Emojis that are popular on mobile devices. We have provided scripts to do this in the do_not_upload/dbtools folder of the vBulletin package.

In addition to this, if you use a MySQL collation with _ai attached, searches will be accent insensitive so any character with an umlaut or other accent will be treated the same as the base character.

Is this easy to handle or do I need to order a specialist for SQL databases?

Greetz Charly