Remove or merge duplicate users en masse

[Sherry C]

Hi,

I'm in the process of importing our Discus forum into vBulletin. Because of the nested structure of our Discus board, I've had to create separate tab files of the top levels that are currently in the Discus forum for import into the vBulletin installation. This works just fine except for one major problem. Every time I import one of these top level tab files, and I have to do the Import Users module, which for the "Discus 4 tab file" option does not offer the option to "Associate Users". (I can't use the "discus_file" option because of the nested structure - the deeper threads/posts don't display if I try to use this import.) I had been under the impression that when I used the "cleanup" module for successive imports that the duplicate Users issue would be resolved. I was sorely mistaken

Ergo, where the old board had 808 users, I now have over 3000, with multiple entries of duplicate users! These are exact duplicates - not the "import-username" and "username" that others have reported. Therefore, I can't use the "Merge Users" command, or a similar bulk sql script that I found elsewhere on one of the vBulletin forums. I could go through and delete the dupes, but I'm not sure now if they are differentially associated with the posts that I've imported (I've been selectively seeding the new install with only some of the old posts). I'm about 1/3 of the way done with the number of imports I'll need to do, so this is not a trivial point for me at all - we're talking about many hours worth of work if I have to manually delete these duplicate users.

Is there a way to mass merge/associate these duplicates with themselves? Will deleting/eliminating them screw up the user ids for the posts?

Help (please oh please) would be tremendously appreciated!

Thanks
Sherry

[Jerry]

Which version of Discus are you importing from ?

[Sherry C]

Hi Jerry,

I imported from Discus 4.10.1.

I actually ended up just going in to the database manually using the Plesk DB admin, and sorting the records to show the most duplicates at a time and deleting the dupes that way. I then emailed all the users explaining what had happened, and we've gone on from there. I ended up having over 5000(!) users after doing all the "chunk" imports. Some of the usernames did indeed have the "imported" tag in front, but there were also a number of them that were exact duplicate records of usernames and email addys, but different userid numbers. I didn't count them, but I'd guesstimate that about 30% of the dupes were the exact duplicates - not "import-XXX" dupes.

After I manually removed the duplicates (via Plesk DB admin) and updated the board (using vBulletin), all the correct usernames (802 of them) were visible using the vBulletin "users - show all" command, and our users have been able to re-register with their usernames. On the board, all the imported posts maintained the correct user names, but it does appear that not all of these posts are associated with the user's username. This is what I would expect, given what I ended up doing. Those "disassociated" post usernames just show as "unregistered". It's not a horrendous bother, as we're not particularly worried about keeping score for total lifetime posts or anything like that

If I had been able to use the "discus_file" import, I don't think I'd have had the problem because that tool did have the "Associate users" module. However, as I said before, it didn't work properly for our board, apparently because of the deeply nested structure of the Discus forum. I tried both Discus import tools many times in many different ways before settling on the method I did.

We've been live since Saturday on the vBulletin installation, and so far everything has worked great. It just would have saved me a tremendous amount of set up time if there were a bulk "merge" function, or (better) if there had been an "Associate users" module when doing the initial import.

ttfn,
Sherry

[Jerry]

I'm guessing if you've been doing it in chunks, the users will be imported again and again each time which would generate the duplicates.

Discus is always a pain, though I try and do them all in one hit.

[Sherry C]

Hi Jerry,

I'm guessing if you've been doing it in chunks, the users will be imported again and again each time which would generate the duplicates.

Discus is always a pain, though I try and do them all in one hit.

Exactly what happened. I did try to do the import as one big hit, but the threads that were "deep" in the structure didn't display properly. So I ended up having to export each 'top' forum and then importing those. That worked, but created the duplicate users. Anyway, it's all done now.

Thanks,
Sherry