OK I was looking around and I noticed that if you click edit to view the path to the model file in CK some of the files that should exist in both skyrim and oblivion dont begin with TES4 . Take for instant TES4SoulGem2Lesser2LesserSoul . If you click edit on this the model points to Clutter\SoulGemLesser01.nif
In Skyrim SoulGemLesser and SoulGemLesserFilled point to Clutter\SoulGem\SoulGemLesser01.niff
So I'm just guessing that the converter missed a subfolder for some files. I went in TESVEdit and added the sub folder to all of the TES4 SoulGems and now they show up in game, yea Sticking through the tables, but with out changing any IDs.
Now to my question.
Would this be a proper way of fixing stuff like this?

Each type of record was
Each type of record was converted separately, so I probably forgot to add tes4 when converting SLGM records. Considering there are only a couple of them, not a big problem to fix manually.