No announcement yet.

Latin 1 to UTF8

  • Filter
  • Time
  • Show
Clear All
new posts

  • [Bug / Issue] Latin 1 to UTF8

    On the weekend I finally got around to running the scripts to convert the old database to UTF-8. I've just had feedback from a member saying some of his old tutorials half the text is missing in the post.
    As a quick test I switched the language back to ISO-8859-1 and the text reappeared. Originally I thought this was related to the database however before I even knew about the DBTOOLS I changed the Language to UTF-8 a few months back and the same thing happened.
    Now the database has been converted it would seem to eliminate that as the issue and narrowed down to changing the language from ISO-8859-1 to UTF-8. Any suggestions on how to restore the posts other than switching back to ISO-8859-1 cut and pasting the missing text then switching back to UTF-8?
    I know it sounds confusing but this is whats happened.

  • #2
    Usually works the opposite way since ISO-8859-1 is a small subset of UTF-8 that only supports English and basic characters. Normally when things get cut off it is because the user has inserted UTF-8 quotes ( “, ”, ‘, ’ ) which are actually two characters with one of them translating to the end-of-file marker in ISO-8859-1.
    • What character is triggering the cutoff?
    • What is the character set and collation of your database tables now?
    • Are you using a Locale for your vBulletin Language? If you're using anything except English, you should be using a Locale.
    • When you switch to UTF-8 as your HTML Character Set in the AdminCP, do you change the Locale to a corresponding UTF-8 Locale? Usually it is the same as your original Locale with .UTF-8 at the end.

    Both the HTML Character Set and Locale entries are case sensitive.
    Translations provided by Google.

    Wayne Luke
    The Rabid Badger - a vBulletin Cloud demonstration site.
    vBulletin 5 API


    • #3
      I'm not sure what characters but I know entire sentences are missing in some cases posts.
      Collation is UTF8mb4_general_ci
      English only
      Locale is blank
      Click image for larger version

Name:	Firefox_Screenshot_2020-03-10T01-58-30.554Z.png
Views:	291
Size:	33.0 KB
ID:	4436525
      I've never touched these settings. UTF-8 was automatically changed after I ran the dbtools.

      I've got a backup of the database before the conversion, I was thinking of restoring it to my test site unless you have any other suggestions.


      • #4
        What are the links to the problematic posts?

        When you edit the posts is the text there?
        Translations provided by Google.

        Wayne Luke
        The Rabid Badger - a vBulletin Cloud demonstration site.
        vBulletin 5 API


        • #5
          Once I have time I'll restore the database to my test forum and show you. The missing text doesn't reappear unless you set the language to ISO-8859-1 and currently when you edit the post the text is gone also even in source mode.


          Related Topics