Announcement

Collapse
No announcement yet.

Error when saving a module with swedish characters in the title

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • [Bug / Issue] Error when saving a module with swedish characters in the title

    We have problems to save modules in Site Builder when giving them titles with Swedish characters like .

    When trying to save a error message is displayed:
    An unexpected error was returned: 'Invalid widget ID specified: 0'

    How can we solve this? The module titles is shown on our custom homepage in Site Builder so its important to get it solved.

    vBulletin version 5.4.4

  • #2
    I cannot recreate this issue using the latest version of vBulletin 5 and a UTF8MB4 database. Can you provide more information including all steps to recreate?
    Click image for larger version

Name:	2018-10-16_11-00-31.png
Views:	28
Size:	118.9 KB
ID:	4400564



    Translations provided by Google.

    Wayne Luke
    The Rabid Badger - a vBulletin Cloud demonstration site.
    vBulletin 5 API - Full / Mobile
    Vote for your favorite feature requests and the bugs you want to see fixed.

    Comment


    • #3
      Originally posted by Wayne Luke View Post
      I cannot recreate this issue using the latest version of vBulletin 5 and a UTF8MB4 database. Can you provide more information including all steps to recreate?
      Click image for larger version

Name:	2018-10-16_11-00-31.png
Views:	28
Size:	118.9 KB
ID:	4400564


      Our database is a "latin1 - default collation" InnoDB running on MySQL 5.7.20, PHP 7.2.7 and IIS 10 running.

      Im adding a search module and call it "Kp & slj" and try to save it. Then the error message appears.

      Can it be the database collation that is faulty? I also having another problem related to ordinary search. Ive posted in this thread about it:
      https://www.vbulletin.com/forum/foru...dish-charactes

      Maybe related?

      Comment


      • #4
        Yes. You're using UTF-8 characters so you should be using a UTF-8 character set and collation in your database. We recommend using the UTF8MB4 character set and corresponding collation (utf8mb4_general_ci or utf8mb4_swedish_ci in your case). If the vBulletin installer creates the database for you, it will use UTF8MB4 with the utf8mb4_genera_ci collation. UTF8MB4 normalizes the number of bytes per character at 4 each and allows new UTF-8 functionality such as the Emoticons popular on mobile devices.

        The character set controls how characters appear and are sent from the database. The collation affects sort orders when running queries and therefore searches in the software. In addition to the database, you need to set your Language options in the AdminCP to they use UTF-8 (new installs should do this) as opposed to ISO-8859-1 (upgrades from older versions) and you may have to set the Locale to a UTF8 compatible Locale under Languages & Phrases -> Language Manager -> Your Language -> Edit.
        Translations provided by Google.

        Wayne Luke
        The Rabid Badger - a vBulletin Cloud demonstration site.
        vBulletin 5 API - Full / Mobile
        Vote for your favorite feature requests and the bugs you want to see fixed.

        Comment


        • #5
          Originally posted by Wayne Luke View Post
          Yes. You're using UTF-8 characters so you should be using a UTF-8 character set and collation in your database. We recommend using the UTF8MB4 character set and corresponding collation (utf8mb4_general_ci or utf8mb4_swedish_ci in your case). If the vBulletin installer creates the database for you, it will use UTF8MB4 with the utf8mb4_genera_ci collation. UTF8MB4 normalizes the number of bytes per character at 4 each and allows new UTF-8 functionality such as the Emoticons popular on mobile devices.

          The character set controls how characters appear and are sent from the database. The collation affects sort orders when running queries and therefore searches in the software. In addition to the database, you need to set your Language options in the AdminCP to they use UTF-8 (new installs should do this) as opposed to ISO-8859-1 (upgrades from older versions) and you may have to set the Locale to a UTF8 compatible Locale under Languages & Phrases -> Language Manager -> Your Language -> Edit.
          Thanks! I will put up a testvb tomorrow and test it out. Do you know if i need to rebuild search index afterwards? The process takes around 2 days on my board.

          Comment


          • #6
            I recommend rebuilding the search engine. You should use the searchindex.php file located in the do_not_upload folder. You run it from the command line and let it go. It doesn't need to be monitored and won't time out.

            If that still takes too long, you should look into using Sphinx search. The instructions are in the do_not_upload folder.
            Translations provided by Google.

            Wayne Luke
            The Rabid Badger - a vBulletin Cloud demonstration site.
            vBulletin 5 API - Full / Mobile
            Vote for your favorite feature requests and the bugs you want to see fixed.

            Comment


            • #7
              Originally posted by Wayne Luke View Post
              I recommend rebuilding the search engine. You should use the searchindex.php file located in the do_not_upload folder. You run it from the command line and let it go. It doesn't need to be monitored and won't time out.

              If that still takes too long, you should look into using Sphinx search. The instructions are in the do_not_upload folder.
              I ran into major issues when trying to make a testvb installation to test the solution for the search problem. Please also take a look at https://www.vbulletin.com/forum/foru...in-site-broken

              I do not dare to make the database changes if i dont have tested it in an test environment before and the search issue is a major thing that must be solved.

              Comment


              • #8
                Originally posted by Wayne Luke View Post
                Yes. You're using UTF-8 characters so you should be using a UTF-8 character set and collation in your database. We recommend using the UTF8MB4 character set and corresponding collation (utf8mb4_general_ci or utf8mb4_swedish_ci in your case). If the vBulletin installer creates the database for you, it will use UTF8MB4 with the utf8mb4_genera_ci collation. UTF8MB4 normalizes the number of bytes per character at 4 each and allows new UTF-8 functionality such as the Emoticons popular on mobile devices.

                The character set controls how characters appear and are sent from the database. The collation affects sort orders when running queries and therefore searches in the software. In addition to the database, you need to set your Language options in the AdminCP to they use UTF-8 (new installs should do this) as opposed to ISO-8859-1 (upgrades from older versions) and you may have to set the Locale to a UTF8 compatible Locale under Languages & Phrases -> Language Manager -> Your Language -> Edit.
                Hmm, it seems to be a VERY complex task to convert the DB, tables and columns from latin1 to utf8mb4. Our forum have been upgraded over a lot of years and latin1 have always worked. Is this solvable in another way? We also have issues searching for words or phrases with in them. All words/phrases with is missing in "words" database table so they cant be searched

                Is it some easy fix for this? Or can the conversation from latin1 to utf8mb4 be done in some easy way? Please help with this.

                Comment


                • #9
                  Yes, keep your database as Latin1 and make sure that the new database uses the same character set and collation as the old database. I can't guarantee that you'll be able to use anything other than English though.

                  If you had a support contract, we could log into your server and do the conversion for you. It takes about an hour to complete. If you migrate to vBulletin Cloud, we will do the conversion as part of the Migration process.
                  Translations provided by Google.

                  Wayne Luke
                  The Rabid Badger - a vBulletin Cloud demonstration site.
                  vBulletin 5 API - Full / Mobile
                  Vote for your favorite feature requests and the bugs you want to see fixed.

                  Comment


                  • #10
                    Originally posted by Wayne Luke View Post
                    Yes, keep your database as Latin1 and make sure that the new database uses the same character set and collation as the old database. I can't guarantee that you'll be able to use anything other than English though.

                    If you had a support contract, we could log into your server and do the conversion for you. It takes about an hour to complete. If you migrate to vBulletin Cloud, we will do the conversion as part of the Migration process.
                    I think that we have support contract but we have problems to give telnet access etc.

                    I did a clean test install of vBulletin 5.4.4 with utf8mb4 and utf8mb4_swedish_ci and the problem with in module names is gone.

                    But, the search function still not working! Actually you have the same problem here on the vBulletin forum. If i post some Swedish fish names here like gdda gs frna etc they will not be searchable. If you exclude the from the searchword like gdda gs frna they will be searchable.

                    In the DB table "words" they will appear like gdda gs frna (they missing the ). The posts shows the correct characters (the node data seems ok) but the indexing of the search words do not work. Why is the searchindex missing the parts of Swedish words?

                    See attached images:

                    Comment


                    • #11
                      Search Engine Issue

                      https://tracker.vbulletin.com/vbulle...ssues/VBV-1333
                      https://tracker.vbulletin.com/vbulle...sues/VBV-17331

                      I need the software to fully support UTF-8 before I can push these issues more. That work is currently ongoing. The biggest issue is upgrades because vBulletin 3 and 4 tried to fake UTF-8 support by converting these characters to ASCII HTML Equivalents. So if you're upgrading, we'll need a tool to convert all these entities into actual characters. Not a small endeavor while maintaining database integrity. Every single post as has to be scanned, modified, and resaved.
                      Translations provided by Google.

                      Wayne Luke
                      The Rabid Badger - a vBulletin Cloud demonstration site.
                      vBulletin 5 API - Full / Mobile
                      Vote for your favorite feature requests and the bugs you want to see fixed.

                      Comment


                      • #12
                        Originally posted by Wayne Luke View Post
                        Search Engine Issue

                        https://tracker.vbulletin.com/vbulle...ssues/VBV-1333
                        https://tracker.vbulletin.com/vbulle...sues/VBV-17331

                        I need the software to fully support UTF-8 before I can push these issues more. That work is currently ongoing. The biggest issue is upgrades because vBulletin 3 and 4 tried to fake UTF-8 support by converting these characters to ASCII HTML Equivalents. So if you're upgrading, we'll need a tool to convert all these entities into actual characters. Not a small endeavor while maintaining database integrity. Every single post as has to be scanned, modified, and resaved.
                        But i have done a fresh new install (not an upgrade) in /testvb/ folder to test if old data is the issue. No old data in that test. Only one test post containing i the posts words. The same issue appears there. How can we proceed?

                        Comment


                        • #13
                          You can't until either VBV-1333 or VBV-17331 is implemented. You can probably use a custom Google Search for your site instead of the built-in vBulletin search.
                          Translations provided by Google.

                          Wayne Luke
                          The Rabid Badger - a vBulletin Cloud demonstration site.
                          vBulletin 5 API - Full / Mobile
                          Vote for your favorite feature requests and the bugs you want to see fixed.

                          Comment


                          • #14
                            Originally posted by Wayne Luke View Post
                            You can't until either VBV-1333 or VBV-17331 is implemented. You can probably use a custom Google Search for your site instead of the built-in vBulletin search.
                            What??? Can i help you to proceed with the vbvs? This is vital for my site. Google search is not an alternative.

                            Comment


                            • #15
                              Originally posted by Wayne Luke View Post
                              You can't until either VBV-1333 or VBV-17331 is implemented. You can probably use a custom Google Search for your site instead of the built-in vBulletin search.
                              The first vbv is from 2012 and the second from 2018. When can we see a solution for this?

                              Comment

                              Related Topics

                              Collapse

                              Working...
                              X