Pages

292 posts / 0 new
Last post
ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
[TUTORIAL] Converting Tamriel Rebuilt to Morroblivion

This tutorial is now old and out of date.  An updated tutorial for June 2020 is available here: https://morroblivion.com/forums/morroblivion-mod-releases/tutorial-converting-tamriel-rebuilt-2002-for-morroblivion-v065

 

[REL/WIP] Morrowind-to-Morroblivion ModExporter 2017-Dec-31

Description:
The ModExporter is designed to load a TES3 (Morrowind) based ESP/ESM file and export it as a TES4 (Oblivion) ESP which can be played in Morroblivion. It is based off the OpenMW code base. Although it was designed with Tamriel Rebuilt in mind, it can be used with many other Morrowind-based mods. Currently, it will convert landscapes, static meshes, NPCs and weapons. There is partial conversion of creatues, armor, clothing (meshes are substituted instead of converted). Dialog and quests are converted, but not fully usable due to incomplete conversion of scripts. A script translator and byte compiler are partially complete. Future versions will have more complete conversion of scripts and thus more usable dialog and quests.

Below is a tutorial to take you through conversion of Tamriel Rebuilt, the entire process should take about 30 minutes on a modern computer (circa 2017).

Source Code:
https://github.com/ponyrider0/openmw

Credits:
OpenMW developers/source-cdoe for doing all the heavy lifting of loading and parsing the TES3-based files. The Unofficial Elder Scrolls Pages (uesp.net) community for sharing valuable information regarding game engine and file formats of Morrowind and Oblivion.
TESxEdit developers/source-code for definitions of the TES4-based file format.
Zilav for insights and helpful advice in emails and forum posts related to TES4Edit script writing and Oblivion interior and exterior cell cluster organization.
TESRenewal.com community for encouragement, suggestions and help testing and bug reporting throughout the development process.
EDIT: And also... 
Galadrielle for original ESPConverter and the long list of contributors to Morroblivion!
Puddles for his work on the Tamriel Rebuilt creature and replacers!

Requirements:
ModExporter-2017-Dec-31
Mods-for-ModExporter-2017-Dec-31
Morroblivion v064 + Unofficial Patch 2017-Dec-18
Tamriel Rebuilt 17.09 + Tamriel Data
OpenMW 0.42 (the Win64 version is required)
Oblivion Character Overhaul 2
Morroblivion Symphony of Creatures

Instructions:

Building the ESPs:
------------------
1. Install Morrowind + Tamriel Rebuilt 17.09.  For details on how to install TR, go here: https://www.youtube.com/watch?v=TvKoRflsg4Q
2. Install OpenMW 0.42.
3. Launch OpenMW and import your Morrowind settings.
4. If you have a folder named "C:\Oblivion.output", then make sure it is empty. This is where exported mesh and texture resources will be placed.
5. Unzip the contents of "ModExporter-2017-Dec-31.7z" into the main executable directory for OpenMW.
6. Launch "openmw-modexporter.exe" and click "Edit A Content File".
7. Select Morrowind.esm from the drop down.
8. Put a checkmark next to Tamriel_Data.esm. Click Open.
9. After it's finished loading and the main editor window has loaded, select the menu command: File->Export ESM.
10. In the Export Module dialog window, navigate to your Oblivion\Data directory and click Save. It will be automatically saved as Tamriel_Data.esp.
11. Wait for export to complete, then close the modexporter and relaunch. This time export TR_Mainland.esm to TR_Mainland.esp.
12. Repeat the same steps to export TR_Preview.esp to TR_Preview.esp.
13. Open Wrye Bash and make sure load order is Tamriel_Data.esp, followed by TR_Mainland.esp, followed by TR_Preview.esp. Move all 3 files after Morroblivion_Unofficial_Patch.esp.
14. Install the "Mods-for-ModExporter-2017-Dec-31.7z" with Wrye Bash. Place Morrowind_Compatibility_Layer.esp before Tamriel_Data.esp. Place TR_Fixes.esp and TR_OCO2_facelift.esp after TR_Preview.esp.
15. Create a Bashed Patch and move it after the last TR related esp file. Build the Bashed Patch.

Building the Resources:
-----------------------
1. Landscape and Icon textures should have been exported to C:\Oblivion.output during the ESP export steps above. Leave them there for now.
2. Go to the OpenMW executable directory where you ran the "openmw-modexporer.exe".
3. Open the Morroblivion.ini from that directory.
4. Find and edit the "Morrowind=" line so that it points to the directory where you have installed Morrowind + TR 17.09. Make sure to put a "\" at the end of the line.
5. Go back to the OpenMW executable directory, and look for these files which should have been generated during the ESP export steps above:
ModExporter_NIFConv_Tamriel_Data.bat, ModExporter_NIFConv_TR_Mainland.bat, ModExporter_NIFConv_TR_Preview.bat
6. Double-click "ModExporter_NIFConv_Tamriel_Data.bat". It will start two windowed processes to convert NIF files in parallel. Let it run until you see a message that conversion is complete for both windows. Press any key to close the windows.
7. Repeat step 5 for "ModExporter_NIFConv_TR_Mainland.bat". Then repeat again for "ModExporter_NIFConv_TR_Preview.bat".
8. Now go to the "C:\Oblivion.output\Data" directory.
9. Create a new 7-zip or other Wrye Bash compatible archive from the directories in c:\Oblivion.output\Data.
10. Drag that new archive into the Installers tab of Wrye Bash to let Wrye Bash copy it into the Bash Installers directory.
11. Right-Click on the new archive inside Wrye Bash and click Install.

Building the LOD Resources:
---------------------------
1. After you have archived and installed the resources in C:\Oblivion.output in the above steps, delete the contents of C:\Oblivion.output so you can start with a clean directory.
2. Go back to the OpenMW executable directory, and look for these files:
ModExporter_NIFConv_Tamriel_Data_LOD.bat, ModExporter_NIFConv_TR_Mainland_LOD.bat, ModExporter_NIFConv_TR_Preview_LOD.bat
3. Double-click "ModExporter_NIFConv_Tamriel_Data_LOD.bat". It will start a windowed process to convert NIF files. As above, let it run until you see a message that conversion is complete for both windows. Press any key to close the windows.
4. Repeat step 5 for "ModExporter_NIFConv_TR_Mainland_LOD.bat". Then repeat again for "ModExporter_NIFConv_TR_Preview_LOD.bat".
5. Now go back to the "C:\Oblivion.output\Data" directory.
6. Create a new 7-zip or other Wrye Bash compatible archive from the directories in c:\Oblivion.output\Data. Make sure it has a different name from the archive you made above.
7. Drag that new LOD archive into the Installers tab of Wrye Bash to let Wrye Bash copy it into the Bash Installers directory.
8. Right-Click on the new LOD archive inside Wrye Bash and click Install.

Generating Distant Lands and Objects:
-------------------------------------
In order to register the LOD meshes and generate distant lands, you will need to run TES4LL and TES4LODGen. Instructions for using these programs can be found on their webpages at Oblivion Nexus.

https://www.nexusmods.com/oblivion/mods/40549
https://www.nexusmods.com/oblivion/mods/15781

Downloads

 

Edited by: DaggerfallTeam on 11/06/2021 - 12:50
TonyB500
TonyB500's picture
Member
Offline
Last seen: 6 years 4 months ago
Joined: 12/31/2012 - 01:39
Karma: 36
Tried this and followed very

Tried this and followed very step, but when I got to step 10, TES4Edit wouldn't save TR_Mainland.esp. It kept saying "Errors have occurred. At least one file was not saved."

Also, how do you select an esm as an active file in the Construction Set? Because as far as I know, that's not possible. So I saved them as esp's on steps 5 and 8 because I didn't know what else to do.

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
TonyB500 wrote:

TonyB500 wrote:

Tried this and followed very step, but when I got to step 10, TES4Edit wouldn't save TR_Mainland.esp. It kept saying "Errors have occurred. At least one file was not saved."

Also, how do you select an esm as an active file in the Construction Set? Because as far as I know, that's not possible. So I saved them as esp's on steps 5 and 8 because I didn't know what else to do.

Hi TonyB500, thanks for trying the tutorial!  Hopefully this suggestions will help:

For Step 5, make sure you did not skip the step to change filetype to Plugin when in the Morrowind Enchanted Editor -- this is basically the same as saving it as an ESP.  After you change filetype, Morrowind CS should detect it as an ESP format whether or not the filename has an .esm at it's end.  However, your way should work as well.

For Step 8, your way of changing to an ESP first should work fine here too.  I forgot to specify in the tutorial that I'm using the Oblivion Construction Set Extender -- that's why I can save directly as an ESM.  I will update the tutorial to show that. Thanks!

For Step 10, the TES4Edit errors can happen if the Oblivion Construction Set has not fixed the raw output from the ESP Converter.  Try loading and saving within the Oblivion Construction Set again.  If you are still having problems, TES4Edit should make a TES4Edit_log.txt if you can at least exit without crashing.  Post it here and I'll try to figure out where the problem is.

Good luck!  and also Congratulations! If you've made it to step 10, then you've already completed the most difficult parts which are preparing the ESM/ESPs and converting with the ESP Converter.

TonyB500
TonyB500's picture
Member
Offline
Last seen: 6 years 4 months ago
Joined: 12/31/2012 - 01:39
Karma: 36
Thanks for offering to help!

Thanks for offering to help! Unfortunately, when I tried running the construction set extender, I got an error stating that I was missing a file called MSVCP140.dll. So I couldn't try what you said in attempting to save TR_Mainland as an esm instead of an esp.

Also, here's my TES4Edit log. And one other thing to note is that I recently moved all of my my Steam games to my Games folder and from my Program Files (x86) folder. However, I couldn't get TES4Edit to redirect to where I now had Oblivion installed. So I had to copy my Oblivion.esm and the other files I was using to the original Program Files folder so TES4Edit could detect them.

EDIT: Okay, SOMEHOW I figured out how to get TES4Edit to agree with me (I rran it through Wrrye Bash and it worked perfectly). But now that I've come to step 13, this is the error I get:

AttachmentSize
Plain text icon tes4edit_log.txt3.31 MB
Image icon esp_converter_error.png40 KB
ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
1. I believe MSVCP140.dll is

1. I believe MSVCP140.dll is part of the Visual C++ runtime distributable.  Try installing the Visual Studio 2015 redistributable: https://www.microsoft.com/en-us/download/details.aspx?id=48145.  You can look at the other files it installs here.

2. Regarding TES4Edit intsallation problems: I think it's looking at the old registry value when you installed.  Run Registy Editor (type Windows-R to open the run dialog, then type 'regedt32' and click OK), then click Edit->Find and type 'oblivion'.  Then let it run for a very long time until it finds a registry entry.  Make sure that the associated entry on the right panel: 'installed path' is set to your current Oblivion directory, for example, mine is set to 'C:\SteamLibrary\steamapps\common\Oblivion\'.  Now press 'F3' and let it run until it finds another entry and change that one as well.  Once it finds the last entry, you can exit the program.

3. tes4edit_log.txt:  Yes, these errors are consistent with using TES4Edit on the raw output from ESP Converter.  Here are alternative steps which will work with the original CS:

  1. From Wrye Bash: right click on the TR_Mainland.esm and select 'Copy to esp'.  If you renamed the .esm to .esp by hand, then rename it back to .esm and use Wrye Bash's 'Copy to esp' function.
  2. Once it Wrye Bash has converted it to an ESP, load TR_Mainland.esp into the CS - making sure you remember to click 'Set As Active File'.  The Status field should change from 'Plugin File' to 'Active File'.  If it does not, then you haven't made it active even if there's a check mark next to it.
  3. Once it's finished loading, click File->Save and exit.  This should repair all the corrupted entries in the ESP so that TES4Edit can now process it without errors.
Sorry, I must have been mistaken - it doesn't look like the regular CS is able to load the raw output of ESP Converter...  You will need to use the Construction Set Extender to Load and Save to fix the corrupted records.  

Let me know if this helps!

TonyB500
TonyB500's picture
Member
Offline
Last seen: 6 years 4 months ago
Joined: 12/31/2012 - 01:39
Karma: 36
Thanks, but I think i solved

Thanks, but I think I solved the issue regarding TES4Edit. made sure everything was installed properly and then ran it through Wrye Bash. It finally detected where it was. Made it to step 13 this time, but ESP Converter didn't...well, convert. It gave me the message "Master File TR_Data.esm is missing!", and I have no idea why. And yes, that file is still in my Morrowind Data Files directory. Put an image into my edited post above.

Also, do I change Master 3=TR_Data_e.esp to Master 3=TR_Data_e.esm? Or leave that as esp? Oh, and don't worry about the error regarding the extraregions file. It turns out that I spelled it differently than you did since yours was spelled "extrarewgions", and that's what I put in the Morroblivion.ini file instead of "extraregions". ^^;

So yeah. Only problem now is ESP Converter saying TR_Data.esm is missing when it isn't.

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
Sorry, missed a file on Step

Sorry, missed a file on Step 11!  Open TR_Preview.esp in the Morrowind Enchanted Editor again, go to the Headers, and rename TR_Data.esm to TR_Data_e.esm.  Save and exit.  Then try step 13 again.  You're almost done!

TonyB500
TonyB500's picture
Member
Offline
Last seen: 6 years 4 months ago
Joined: 12/31/2012 - 01:39
Karma: 36
Now at step 16. And thought I

Now at step 16. And thought I was home free...and then I got another error preventing TES4Edit from saving TR_Preview.esp. Once again, no idea what went wrong. >.<

Here's the new log. From what it says, the problem lies with whatever record 1Travel "Voyager" is.

EDIT: Wrye Bash now won't even list TR_Preview.esp and says it's corrupted. Forcing me to redo everything from step 12 to get a new TR_Preview.esp. Not going back to step 16 until I know what went wrong.

EDIT 2: Okay, tried step 16, but this time made backups. No corruption this time, but TES4Edit still won't save the file cause of whatever that "Voyager" file error is.

AttachmentSize
Plain text icon tes4edit_log.txt1.66 MB
ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
The message about TR_Preview

The message about TR_Preview.esp being corrupt sounds like an error with the ESP Converter which caused it to abort early.  The file size of the raw output for each file should be about 120-140 MB.  If it's less than that, it may have aborted early.  Make sure you get the expected ESP Converter message about it being done.

The errors in your tes4edit_log.txt are still consistent with running TES4Edit without first Loading/Saving the TR_Mainland inside the Oblivion CSE.  If the CSE is loading the file correctly, it should take about 5-6 minutes (during that time, you should see a small message box at the center of the screen that says "Loading Plugins Please Wait").  If it finishes within a few seconds, then it did not load. 

Once the file finishes loading into the CSE, that message box disappears and you should see the name of the file (TR_Mainland.esm or .esp) across the title bar.  If you don't see the name, then it did not load correctly.  Once you see the filename in the title bar, click File->Save.  Saving should also take a few minutes, with the mouse cursor spinning until it is completed.  Once done, you can exit.  To be sure that the CSE Load/Save step actually worked, check the file size:  Before this step, it should be 120-130MB.  After this step, it should be 60-70MB.

And make sure you do this same step with the TR_Preview.esm/esp too!  Also in step 16, make sure your are right-clicking the TR_Preview.esp and not TR_Mainland.esp.

Sorry if I sound overly detailed with my instructions -- just making sure we're on the same page.  Good luck!

TonyB500
TonyB500's picture
Member
Offline
Last seen: 6 years 4 months ago
Joined: 12/31/2012 - 01:39
Karma: 36
Perhaps then the issue lies

Perhaps then the issue lies with being unable to have the Oblivion Construction Set load the esm's as main files. When I try to make them active files it won't let me (even after loading the CSE through Wrye Bash, and it won't load outside of Wrye Bash), forcing me to save them as plugins. So when I save the plugins in Wrye Bash in steps 9 and 15 I'm saving over those plugins. Which I think is likely the issue I am having.

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
Sorry, I had assumed CSE was

Sorry, I had assumed CSE was loading appropriately and it was only TES4Edit that you needed to load through Wrye Bash.  Make sure that the the CSE is loading and not just the plain CS.  If it's not loading from the included LaunchCSE.bat, then it might not be launching correctly through Wrye Bash.  I bet that's the problem: CSE is not working yet.  

Make sure that you have the obse_loader.exe in your Oblivion directory -- CSE needs OBSE but requires the executable loader instead of the obse_steam_loader.dll that the steam version of Oblivion uses.  Download the latest loader here.   If CSE is working, you will be able select Master Files and set them to Active just like a plugin file. 

Check your Oblivion directory to see if there's a "Construction Set Extender.log".  Send that to me in addition to "obse_editor.log" and "obse.log" and I'll try to figure out why it's not loading.

TonyB500
TonyB500's picture
Member
Offline
Last seen: 6 years 4 months ago
Joined: 12/31/2012 - 01:39
Karma: 36
Just for the sake of posting

Just for the sake of posting an update, I did manage to get everything working. Figured out what went wrong and what I had and didn't have installed correctly. And everything got converted and it works! Well, sorta. I mean, all the locations are set up and they look amazing! Here's hoping that, maybe one day, Tamriel Rebuilt will be able to be fully converted to Morrolivion. :)

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
Progress Update:

Progress Update:

I've been working on several things: 

First, I'm testing some techniques to port the hundreds of creatures and thousands of creature references into Morroblivion using Brainslasher's suggestions here: https://tesrenewal.com/comment/81745#comment-81745

I've also gotten several nice in-game World Maps created by merging the Color- and Normal-space LOD maps with some other maps from the TR website.  The cool thing is they even work with regular Morroblivion. 

Finally, I've been having a lot of fun adapting ICE's Morroblivion in Tamriel mod which let's you travel on foot between the base Oblivion and Morroblivion game worlds.  It now will transport you from Cyrodil to Morrowind as you cross the Velothi mountains.  I'm working on a mod which will allow access by way of the 4-5 mountain gate passes between Morrowind and the other provinces.

EDIT: attached is a preview of one of the map types I've been making.  I'll post the finished product as a mod release.

AttachmentSize
Image icon tamrielrebuiltworldmap_satellitemap.jpg365.02 KB
Diana TES GotH
Diana TES GotH's picture
Member
Offline
Last seen: 3 years 7 months ago
Joined: 11/25/2015 - 21:49
Karma: 165
I may not be able to complete

I may not be able to complete my guide until you have finished because everything you keep sharing, I want to include...lol 

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
Thanks!  I really enjoy

Thanks!  I really enjoy reading your guides and find them a great resource, so that is some high praise to be included in them!

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
I've updated the original

I've updated the original post with a new ESP fix file as well as the link to some worldmaps I created.  Next up on my todo list is a TR creatures addon, and then maybe activation of the transport NPCs.

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
At the end of the original

At the end of the original post, I've uploaded a patch ESP to be loaded after Ice's Morroblivion in Tamriel.  It modifies the travel script to move the transition zone to the edge of the Tamriel Rebuilt map.

Requirements:
You will need the "Morroblivion in Tamriel" mod and the "Under the sign of the dragon - Tamriel Heightmaps" mod in order to use this ESP.

Known Issues:
Due to the transition taking place on land (and in mountainous regions), you may materialize within a static object -- it will look as if you are only able to move in a small area.  Use the "tcl" console command to get out of the object.  This issue will be fixed by a new version which restricts transition destinations to Gates/Passes on the Morrowind side.

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
I've made a version 1.3 of my

I've made a version 1.3 of my patch for Morroblivion in Tamriel.  This version will move you to one of 5 mountain gate passes whenever you try to enter Morrowind from the western border.  This fixes the issue of materializing inside a static object.  If you'd rather cross into Morrowind in a freeform style rather than going through a gate, then keep using the v1.2 file.  Download links are at the end of the original post.

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
Tamriel Rebuilt 1609 is

Tamriel Rebuilt 1609 is released!  link
There are many changes to the ESMs and BSAs due to integration of assets with Project Tamriel.  This has resulted in a huge increase the number of records, necessitating a thorough analysis of all the referenced and unreferenced records in the ESMs.  Creation of barebones Morrowind/Bloodmoon/Tribunal.ESMs is not enough to meet the record limits of the ESP Converter.  I will likely move back to an older system of partial conversion of the ESMs rather than converting the full ESM at once.  Hopefully, I'll be able to streamline the process into another tutorial for conversion.  Until then, converting 1603 is still the best way to experience Tamriel Rebuilt in Morroblivion... and I'll continue to make more add-ons for it (including creatures, dialog, transport NPCs, and some preliminary quests).

Diana TES GotH
Diana TES GotH's picture
Member
Offline
Last seen: 3 years 7 months ago
Joined: 11/25/2015 - 21:49
Karma: 165
Does a person need to

Does a person need to download the patches for TR Worldmap if he/she is not converting Tamriel Rebuilt to Morroblivion?

 

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
dmdbd1970 wrote:

dmdbd1970 wrote:

Does a person need to download the patches for TR Worldmap if he/she is not converting Tamriel Rebuilt to Morroblivion?

No, the patch is just to make the Morroblivion in Tamriel mod's transport scripts compatible with the Tamriel Rebuilt conversion.  The Worldmaps should work just fine as standalone graphic replacers.

DaggerfallTeam
DaggerfallTeam's picture
Moderator
Offline
Last seen: 3 months 8 hours ago
Joined: 06/03/2014 - 15:53
Karma: 471
Donor
Does this work with Tameriel

Does this work with Tameriel Rebuilt 1609 ?

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
Daggerfall Rebuilt Dev wrote:

Daggerfall Rebuilt Dev wrote:

Does this work with Tameriel Rebuilt 1609 ?

Unfortunately, not yet.  TR 1609 merged it's ESM library into a common library shared by several other projects which significantly changed the organization of the records.  My instructions currently only work for 1603.

Sifonseal
Sifonseal's picture
Offline
Last seen: 6 years 9 months ago
Joined: 06/01/2017 - 01:34
Karma: 1
How goes the progress towards

How goes the progress towards creating a system that works with the newer versions of TR?

I am sure there are countless lurkers like myself who keep checking in here frequently, I just figured I would ask on behalf of all my lurker buddies ^^.

It goes without saying that this is an amazing project you are working on and one I am very excited to see more of ^^!

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
Hi Sifonseal, thanks for your

Hi Sifonseal, thanks for your vote of interest!  I've mentioned my project plans on a couple other threads but didn't realize that I had never posted an update on this main thread -- my apologies for that, hehe.  In terms of the status of this project, I am still in the design/planning stage.  Hopefully, more progress will be made once I finalize the current versions of my other Morroblivion projects.

The current direction I am taking for converting current and future versions of TR is to develop a new ESM converter that is fully automated.  My plan is to extend the OpenMW CS to support exporting to the TES4/Oblivion ESM file format.  A side benefit of supporting TES4 ESM file-format is that this could be a stepping-stone towards creation of an Open Oblivion engine.

I know I had mentioned making an additional ESP fix for the 1603-based conversion to enable silt-stride and other transport to the mainland, but since my previous ESP-fix already enables fast-transport to several dozen areas on the mainland as well as land travel between Cyrodil and Morrowind, I don't think that's necessary.  If anyone has any ideas, comments or suggestions, please feel free to post them here.  Thanks!

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
Good news: looks like things

Good news: looks like things might take less time than I thought.  Apparently, there's been progress made since I last rage-quitted trying to set up the Windows build environment for OpenMW.  This script supposedly will download and build OpenMW and its dependencies for you: https://gist.github.com/ace13/de7f30454e99b3952405.  I will try it out later this weekend.  Having a working build environment will probably shed several weeks off my schedule (probably months when you factor in rage-quitting).

Puddles
Puddles's picture
Moderator
Offline
Last seen: 4 years 8 months ago
Joined: 11/17/2016 - 14:09
Karma: 259
My only suggestion as someone

My only suggestion as someone who's screwed around with stuff they're way in over their head on and then regretted the size of it after the fact...maybe a way to create an .esm with just the resources and nothing placed anywhere, and then .esps of each affected region? After that, could just merge them.

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
What you're describing is

What you're describing is actually very similar to how the original tutorial worked.  But it required over 6 hours of the user's time (not even counting the software computation time) to edit the Morrowind/Tamriel Rebuilt ESM/ESPs.  This is mainly because the Morrowind Enchanted Editor is from an era when ram was limited, and so the response time between operations is mind-numbingly slow when compared to modern tools like TES4Edit.  Even with the tutorial time now drastically reduced from multiple days to less than 120 minutes of total time (user time + software computation time), I don't see much interest in people using the tutorial.

** If I'm wrong about this and many people have actually performed the tutorial, then please let me know.  I would love to be proved wrong on this point, and it would definitely encourage me to spend more time on this project -- especially if those people could offer suggestions on making the tutorial steps faster. **  

From the very few responses I've gotten, I think most people are waiting for me to come up with a completely automated conversion process.  And I think that's perfectly reasonable for people who don't have more than a handful of hours per week to devote to Morroblivion... but it will take some time for me to get there.

FYI, this thread discusses most of the original conversion process and how it slowly improved to the steps of the current tutorial.  And if anyone is interested in using the tutorial, you can still download TR1603 on nexus mods -- just scroll down through the Old Versions.

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
I have to give a major thanks

I have to give a major thanks to the ace13 and everyone else that worked on the new OpenMW scripts for setting up a Windows Build environment.  The rumors are TRUE, following the directions on the site I linked to above really does automatically download and configure all dependencies to OpenMW.  The major leap forward in ease of setup comes from the fact that ace13 and/or other contributors have done all the hard work of configuring, building and packaging the dependency libraries like Boost, Bullet, OSG and MyGUI.  I've made at least 3 attempts at building OpenMW before rage-quitting in the past: the first rage-quit was while trying to configure and build Boost,then second was Bullet or OSG and the last time I rage-quitted, I had overcome all my trials with building Boost, Bullet and OSG only to be denied access to OpenMW by MyGUI.  With the new preconfigured build libraries, the script went straight from downloading the dependencies to configuring the makefiles and generating the Visual Studio project files!  All I had to do was open the OpenMW.sln and click Build!!  This has literally saved me weeks or months of effort. 

Of course, it would have been too much to ask for everything to work perfectly... after building OpenMW-CS, I did a test-load of TR_Mainland.esm -- it took about 45-60 minutes just to load!  It would seem that the ESP importer/loader functions in OpenMW-CS need some significant optimization before they are usable for converter Tamriel Rebuilt...  That's the next step.  After that, I'll work on making a TES4 ESP importer.  Once that's done, the final step will be making the TES4 ESP exporter!  ( I need to create an importer first so that I can match records from the original Morrowind.esm with the FormIDs in the Morrowind_ob.esm from Morroblivion.  That way, the converted ESP will actually be compatible with Morroblivion. )

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
I think I've found the issue.

I think I've found the issue... OpenMW-CS is making disk reads for each subrecord, and there are hundreds of thousands of Dialog subrecords which are just less than 20 characters each... Uh yeah, reading from disk 20 bytes at a time will kill your performance... and your disk drive.   I'll try to figure out how to at read the entire record or more from disk at a time then parse, rather than "parse, read subrecord, parse, read subrecord" like it's doing now.

Correction: My mistake, OpenMW-CS developers went the route of over-riding the low-level internal buffer routines for C++ streams to read 4K at a time instead of 20 bytes.    My apologies if I offended anyone's programming prowess.  I will continue to track down the cause of the performance hit... my current suspicion, is that OpenMW-CS may be trying to evaluate and build the entire Dialog tree as it goes along during the ESM load function rather than delaying until the user tries to read Dialog records.

Update: I guess I should have ran a performance profile from the start rather than reading source code.  Visual Studio is pointing to the CellRef::InsertRecord function as taking up 22% of all processing time and InfoCollection::getTopicRange's calls to ciEqual as taking up another 19%....  Okay, replacing 3 lines of C++ using iterator functions with the equivalent ANSI C using plain old array indexing[] has reduced the processor usage from 19% to 6%.  Time to look at the InsertRecord function...  It looks like two issues: CellRef::InsertRecord() is actually called by CellRef::AppendRecord() and both are operating on a std::vector.  Using the insert method to append is inefficient, but I was only able to gain 5% when changing insert to push_back.  The expensive operations here are resizing the vector by one record and allocating memory for the new record.  I don't know why instantiating a CellRef object is so expensive -- the constructor zeros 3 integers.  There are some wasted steps due to allocating/copying/freeing several local const CellRef variables just to pass it down 3 function calls to get to the vector insert function.  But trying to fix it is turning out to be a huge headache due to the use of templates and virtual functions to declare appendRecord().  I think my best bet is to focus on resizing the vector by multiple records at a time....

Update 2: I had a productive weekend tweaking the vector functions and learning about the ESM loader internals at the same time.  After looking at the big picture, I see that the main issue is a design flaw: during the ESM load process, the engine is trying to use a vector (dynamically resizing array) to mainly insert 100,000+ new subrecords and to occasionally over-write base subrecords.  This is the worst case scenario for a vector, since it is constantly resizing and copying all of its contents to a new location just to insert a new record.  The optimizing compiler does a great job of skipping the redundant instructions, turning an operation to load the TR_Mainland.esm which takes 67 minutes in a Debug build to just 3 minutes in a Release build.  Unfortunately, developing a large conversion program in Release mode is unacceptable to me.

Update 3: Day 4 - conversion to std::list is complete.  Loading TR_Mainland.esm in debug build is shortened from 67 minutes to 16 minutes 41 seconds.  A few more optimizations and will be ready to work on the actual TES3->TES4 conversion process.

Update 4: Day 7 - I've optimized the current code base as far as I can without doing significant redesign of how the CS works.  OpenMW-CS is now 300% faster than the current master Release build and 600% faster than current master Debug build.  Now that I'm more familiar with the source code, I've decided to add another step before doing the TES3->4 conversion:  I'm going to work on making a TES3-format ESP exporter first, so that I can more thoroughly test export functionality.

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
I updated the TR_Fixes patch

I updated the TR_Fixes patch to tr_fixes_011.7z (link in first post).  It fixes a mapmarker bug and replaces some missing book and scroll models with Morroblivion equivalents.  Thanks to Diana TES GotH for testing and feedback!

Puddles
Puddles's picture
Moderator
Offline
Last seen: 4 years 8 months ago
Joined: 11/17/2016 - 14:09
Karma: 259
Okay, finally going to sit

Okay, finally going to sit down and try to do this.

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
Let me know if you need any

Let me know if you need any help.  Also, It would be cool if you could work your conversion magic on some of the awesome looking TR creatures!  ....Unfortunately, the Tamriel Project won't allow distribution of any of their assets, so the only thing you could do is post a tutorial or an ESP patch with your converted creatures.  I'll try to incorporate creature conversion with the new automated exporter I'm working on.

Puddles
Puddles's picture
Moderator
Offline
Last seen: 4 years 8 months ago
Joined: 11/17/2016 - 14:09
Karma: 259
They won't allow distribution

They won't allow distribution of their assets?

"Tamriel_Data is free to use as modder resource, as long as proper credits are given. See the included readme file for usage permissions."

Puddles
Puddles's picture
Moderator
Offline
Last seen: 4 years 8 months ago
Joined: 11/17/2016 - 14:09
Karma: 259
Hrm...my version of TR_Data

Hrm...my version of TR_Data.esm doesn't have the bolded regions:

-Open the Regions group and checkmark: (1)Asskaedh Coast, (2)Grey Meadows, (3)Northen Velothi Mountains, (4)Roth Roryn and (5)Velothi Mountains.

Edit: nvm was using the newest version

 

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
Puddles wrote:

Puddles wrote:

They won't allow distribution of their assets?

"Tamriel_Data is free to use as modder resource, as long as proper credits are given. See the included readme file for usage permissions."

Tamriel_Data is a modder resource, when used with the Morrowind game engine... some of their art assets are based on meshes/textues in the original Morrowind ESMs so there is concern about legal grey zone if Tamriel_Data is used outside of the Morrowind game engine.  There's a lot of anxiety over attracting attention of Bethesda lawyers and getting a C&D order or being blacklisted from Nexus and Official Bethsoft forums.

However, I think you might be able to talk them into allowing the conversion and distribution of the creatures/models which are visually unique to Tamriel Rebuilt and not based on any Morrowind assets.

Puddles
Puddles's picture
Moderator
Offline
Last seen: 4 years 8 months ago
Joined: 11/17/2016 - 14:09
Karma: 259
I keep getting  these "Object

I keep getting  these "Object reference "Ex_T_menhir_L01" missing in master file" warnings in addition to the ones you said are normal opening TR_Mainlands.esm (which is actually an .esp file with an .esm suffix). Is this normal, too? And...lots of other different warnings.

Edit: It also didn't convert any of the nifs at all, lol. um...

And, hm... @ the creatures bit. I'll do some digging. I mean, we're already using vanilla Morrowind creature textures in Morroblivion, I'm pretty sure. Even if the original textures were lightly seeded to look different...

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
Sorry, which step are you on

Sorry, which step are you on and what program are you using when you get the "ex_t_menhir_l01" error?  Make sure you didn't miss any steps when you were editing and renaming all of the Morrowind master files.  

Regarding no nif files being converted: make sure you have the ESP Converter's Morroblivion.ini correctly edited with the correct *.esm and *.bsa filenames.

About Morroblivion using original textures/meshes: this is why it was blacklisted from Nexus and Official Bethesda forums.  Morroblivion may have resolved the issues and avoided getting into further troubles with Bethesda's laywers but it is still banned from Nexus and official forums.  This is one of the situations that the Tamriel Rebuilt project wants to completely avoid.  So they are unlikely to approve anything containing the word "Morroblivion".

Puddles
Puddles's picture
Moderator
Offline
Last seen: 4 years 8 months ago
Joined: 11/17/2016 - 14:09
Karma: 259
The huge cascade of errors

The huge cascade of errors are during step 5 when I load the TR_Mainland_e.esm into the CS. I get a warning about 1 or more files "not being able to find the right masters", and then...get like 20 minutes of warning messages.

Puddles
Puddles's picture
Moderator
Offline
Last seen: 4 years 8 months ago
Joined: 11/17/2016 - 14:09
Karma: 259
Here's the warnings file.

Here's the warnings file.

AttachmentSize
Plain text icon warnings.txt43.79 KB
ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
Puddles wrote:

Puddles wrote:

I keep getting  these "Object reference "Ex_T_menhir_L01" missing in master file" warnings in addition to the ones you said are normal opening TR_Mainlands.esm (which is actually an .esp file with an .esm suffix)....

 

The huge cascade of errors are during step 5 when I load the TR_Mainland_e.esm into the CS. I get a warning about 1 or more files "not being able to find the right masters", and then...get like 20 minutes of warning messages.


These two errors / pop-ups are fixed by clicking "CANCEL" when you get the popup (which is actually "YES TO ALL" even though it says "Cancel" on the button).  You will never see any of these pop-ups if you follow this step as soon as you launch TES Construction Set:

When you see the Warning box about "TR_Data_extraregions.esm", click Cancel.

I will modify the Tutorial to make this step more clear.  ***EDIT: you may also need the following line in the Morrowind.ini file in your Morrowind directory:

[General]
allowyestoall=1

Also, check your Morrowind_e.esm -- based on the warnings.txt, I think you may have forgotten to delete the pathgrids in step 1.

Puddles
Puddles's picture
Moderator
Offline
Last seen: 4 years 8 months ago
Joined: 11/17/2016 - 14:09
Karma: 259
And...now the esp converter

And...now the esp converter itself (not the nif converter) is crashing midway through converting the TR_Previews_e.esp.

AttachmentSize
Plain text icon morroblivion.txt369.48 KB
ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
The most likely reason for

The most likely reason for ESP Converter to crash half-way through the conversion process is that it ran out of memory trying to process the records, i.e. you missed a step somewhere in ESM/ESP preparation and did not delete all the records/record-groups that were listed... (I will update file sizes in tutorial).

Puddles
Puddles's picture
Moderator
Offline
Last seen: 4 years 8 months ago
Joined: 11/17/2016 - 14:09
Karma: 259
1 kb under your basis for the

1 kb under your basis for the TR_Data_e.esm, hm...pretty sure I followed the instructions right. Hopefully that's not too big a margin of error. Let's try all this again.

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
I just found these two

I just found these two modified NIF files in my dev directory -- I think they are supposed to fix the two NIFConverter crashes and produce correct Oblivion meshes.  You'll also have to put these lines into the statics.ini:

TR_ex_nec_cause_03 = "morro \ TR \ x \ trUexUnecUcauseU03.nif"
TR_in_v_monastery = "morro \ TR \ i \ trUinUvUmonastery.nif"

Use the two lines above to figure out where to place these meshes in your Morrowind.TR_input directory tree.

​EDIT: I'll replace the two original files with heavily decimated "thumbnail" versions (eventually).  Until then, you can at least extract these from the BSAs and figure out the compatibility workaround yourselves.  Sorry, I don't remember what the actual fix was.

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
This is an addon for TR

This is an addon for TR_Mainland which adds the creature records and populates the mainland using Brainslasher's technique: https://tesrenewal.com/comment/81745#comment-81745.  Unfortunately, not all the leveled lists for creatures are converted and their corresponding map references are thus missing too.  All creatures are currently assigned centurion spider models, but that can be easily fixed by editing the 100+ records (this could be done manually in a weekend if someone wants to do it).  The bigger hurdle is filling in the thousands of missing map references (this could take weeks to months if it is done manually).

To be clear: this package only contains an ESP add-on which requires both Morroblivion v064 and the manually converted TR_Mainland.esp.  There are no creature models or other assets provided.  All creature records have been assigned to the default Morroblivion centurion spider model (aka Lore Creature Expansion).

AttachmentSize
File tr_mainland_creatures_v0_91.7z32.75 KB
Puddles
Puddles's picture
Moderator
Offline
Last seen: 4 years 8 months ago
Joined: 11/17/2016 - 14:09
Karma: 259
I'll have a replacer up

I'll have a replacer up within a week or so for most of them, I hope. Once that's done I'll do a second using Symphony's models where possible.

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
I'm posting TR_Fixes_012 -

I'm posting TR_Fixes_012 - this version adds several more fast-travel enabled mapmarkers to the western half of mainland Morrowind, and it now offers Transport to Cyrodiil via Pathfinder NPCs stationed at the 5 Imperial Fort Passes between Morrowind and the rest of Tamriel. Corresponding Pathfinders are stationed outside the North and East facing gates of Bruma and Cheydinhal.  AND, the Almalexia-Mournhold palace gates will now open directly into the Mournhold interior city.

Also, several missing meshes are now replaced with Morroblivion/Oblivion default ones.

Note: neither the transport to Cyrodiil or to Mournhold will start the corresponding mainquests. These are pure transport scripts.

ponyrider0
ponyrider0's picture
Contributor
Offline
Last seen: 1 year 2 months ago
Joined: 07/22/2016 - 05:01
Karma: 1214
Update:

Prior Updates: https://tesrenewal.com/comment/85231#comment-85231

Update 5:
I've been making progress with understanding the OpenMW source code pertaining to loading and saving files.  I'm now able to export the OpenMW native files into working TES3 ESM/ESP files that are compatible with Morrowind and the Enchanted Editor.  I now need to cleanup my experimental code and set up the framework for Exporting to multiple formats.  Obvious formats are TES3 and TES4, but there is also potential for office spreadsheet and database compatible formats.  The last two could be very useful for wiki pages and translations.

FYI, the sourcecode is here: https://github.com/ponyrider0/openmw/tree/experimental-expor...

Puddles
Puddles's picture
Moderator
Offline
Last seen: 4 years 8 months ago
Joined: 11/17/2016 - 14:09
Karma: 259
This has taken me a while,

This has taken me a while, but I'm posting a beta (read: beta) that substitutes appropriate creatures or adds in new ones (either converted or cobbled together by me) using Oblivion resources. I am absolutely certain there are some bugs. The creatures I converted from Tamriel Rebuilt have "solid" collision because I can't get the ragdoll to work, and might be a bit buggy (you can walk on top of the larger ones after killing them, as if they were static objects). I communicated briefly with the TR staff during their sunday meetings over a few weeks and have taken steps to avoid converting anything that's based off of vanilla Morrowind. The Ornada and Mole Crab were made from vanilla Morrowind armor pieces, so I altered Oblivion creatures as substitutes, in particular. Butterflies have been removed because the TR is phasing them out of future additions. There's a few things left to work on, but as-is everything should be playable without running into Centurion spiders everywhere.

Puddles
Puddles's picture
Moderator
Offline
Last seen: 4 years 8 months ago
Joined: 11/17/2016 - 14:09
Karma: 259
Okay, I've finally had a

Okay, I've finally had a chance to actually play this...I've noticed some land tears along the two southern-most entrances into Morrowind, along the borders. They're nothing terrible, though. Native Chuzei Bonemold Cuirass is broken/invisible.

Edit: I think Ruineous Keep is probably one of the creepiest forts I've ever visited, even with the retro graphics... x_X

Edit 2: I have the worst luck. Over an hour and the only exterior creature I've seen is Wild Guar...

Pages