First, I need to give a BIG thank you to llde for his tips on how to build the obse library and plugins!! (original post by him here: https://tesrenewal.com/comment/84757#comment-84757) Following your directions, the build succeeded on the first attempt! Awesome!
Now that I'm finally able to build OBSE plugins, here are some of the things that I have on my todo list:
- A better Universal Silent Voice plugin which plays one of Oblivion's generic greeting dialogs in the appropriate race and gender whenever you start dialog.
- Custom EquipItem function to allow for an arbitrary number of clothing/armor slots.
- Also Custom EquipItem: Disable the cast-on-equip bug when Equipping OnTarget items like Ring of Fireball
- 
Implement Morrowind-style combat: miss-miss-miss-miss-hit
llde's original post for convenience:
llde wrote:
ponyrider0 wrote:
Will hopefully be optional.
GreetingDose not Need to Be Optional If Controlled Blade\Strength\Health Skills
roxon_55 wrote:
The main motivation for Morroblivion is for me that I can play Morrowind with the Oblivion combat system, because it is more realistic and more fun.
It would be a pity which one the other features then can not use.
Greeting
I've been combing over the internet for more information on Oblivion's internals to do the Universal Silent Voice update and the EquipItem slot implementation, but not having much luck. The one bit of luck was that the memory address to hook into the EquipItem function was easy to find in the OBSE source code. I briefly tried to set up a debugger/disassembler environment with Visual Studio 2015 but it is beyond my simple brain... Visual C++ 5 was so much more intuitive to use, and it already supported cross-compiling to RISC, mobile and Version Control back then: 20 frickin years ago. Gah!
Well, anyway, with regards to a USV update, I'm going to have to reach out to Ely to see if I can get source code or at least some more direction as to what to modify. If I can get get it working, my goal would be to dynamically choose the best MP3 source files for any NPC -- whether it is Oblivion voices, Voice-acted voices or CG voices -- that way, we won't have to deal with huge overlapping, disorganized directories of mp3s. Maybe I could even dynamically generate the CG voices and forego a library of mp3s entirely. Too bad Microsoft's voice technology is far behind what's offered by Apple or even Amazon.
And as for the EquipItem hack, right now, I've made a enough progress with the script-only version of More-Armor-Slots that I think I can back-burner this project for the time being.
For the disassembler I recommend Ida Pro with the script from JRoush. Note that many parts of the code is horribly complicated, and many things were not decoded.
You might want to reach Alenet or Tiawar for the EquipItem.
For the Universal Silent Voice update: I wanted to do something similar myself, however never found the correct entrypoint.
In my plan, I would have update USV to fix the "localized race folder" beheviour. In practice when a dialogue start the game search for the corresponding sound in the specific race folder but it search for it by the "Name" field and not by the "Editor ID". I wanted to do a sort of fallback cascade: Check Localized name -> Check English Name -> Check EditorIds.
And only after checking all three it would have loaded the empty sound. How this sound to you?
If you is able to contact Elys and obtain a copy of the source code or at least the entrypoint I may be able to help you on this.
Cool -- your ideas for USV sound similar to mine. On top of your voice file search functionality, I would also like to add an optional ability to search different races (if no altmer, then look for dunmer, then look for imperial, etc.) In the case of the computer-generated voice files, they are all under the imperial race, so doing a fallback to that for every other race would allow for a less complicated CG-voice setup with full voicing of dialog. Of course, if we could generate voices from a voice-synth SDK in real-time, that would be even more awesome. It's too bad MacOS is far more advanced than Windows in this regard -- they have voice libraries for hundreds of languages, multiple accents for English language and even multiple voices for each accent.
I'll try to contact Elys through the Nexus contact page and will let you know what happens.
Good news!
I got a reply back from Elys with very helpful information / source code. I will try to re-implement as a C++ plugin and will then post my code here....
Update: It's actually (in my opinion) a very elegant solution, the silent-voice function hook seems to be at a point where the game engine code has just failed at loading the voice file from disk. Elys then overwrites the memory address in the stackbuffer (ESP + 0x64) where the filename should go and then is probably jumping back to the start of that function to re-attempt to load the filename from disk.
I will hopefully have a working re-implementation of USV written in C++/asm in a week. From there, we will need to figure out how to read the original filename. Hopefully, this original filename is intact at the memory address that Elys uses to overwrite with the filename of the silent MP3 (ESP + 0x64). The original filename will then allow us to generate alternative filename paths to search for the voice file.
PS - I don't think dynamically generating synthesized voice files on the fly (in "real-time") is feasible at the moment with affordable hardware... the steps needed would be to synthesize the full voice file, encode into mp3, write to disk, then pass the filename to the game-engine. On my computer, just loading the voice file from disk alone is enough to cause stuttering and the loss of the first few milliseconds of audio as the game-engine skips ahead to maintain proper animation sync.... Of course, if I write directly into buffered disk cache instead of the physical disk and then if I'm able to pass the memory address directly to the game-engine then we might shave off enough milliseconds to make it work (the bottleneck/latency would then be how fast the CPU can generate the synthesized voice and encode into mp3). --hehe, unfortunately, this method also assumes that we can access and read the dialog text to be synthesized.
Shouldn't be too much work to reimplement in C++. Or in a memory safe language like Rust.
Also digging inside OBSE sourcecode it seems that TESDialogue structure is not aviable.
Alright, I had some time today and made a quick working test version. FYI, the path-name of the original missing voice file is indeed at that memory address that I mentioned above! All we need to do now is parse the original pathname, modify it to what we want and write it back into memory.
Update: I just confirmed that I can successfully replace the string with a filename found inside a BSA! I've attached a test plugin with preliminary source code. The source code files are made for llde's version of obse source. Just drop it into the obse_plugin_example directory.
If you want to try out this plugin, all you need to do is put the test_usv.dll into \Data\OBSE\Plugins directory. Then remove Elys_USV.dll. When you run Oblivion, all missing voice files will be replaced by a line from Socucius Ergalla. A test_usv.log file will be generated in the \Oblivion main folder with output of all the missing original filenames that it redirected.
To do:
1. Implement hooks for dialog subtitles and lip files.
2. Parse original pathname string.
3. Replace original pathname with redirected path.
This is Running test_usv in Plugins.
Plugin List plus test_usv.log in second z7 file
Edit:
Removed Clip.
haha -- it looks/sounds funny when put into a video, but the plugin is working as intended. This version is just to demonstrate that I could rewrite Elys_USV -- intercepting missing voice files and replace it with a test voice file of my choosing. I chose Socucius Ergala instead of using a silent MP3 so you can more easily see that it's my plugin and not Elys_USV that is running. Now I need to finish the next few steps on my to-do list before it's actually usable in gameplay (sorry, should have made that more clear).
No Prob will wait for the next 1.
Test_USV_2:
- Completed implementing remaining hooks (dialog subtitles, general subtitles, .LIP file loading).
- Unfortunately, I can't figure out a CTD when calling the Oblivion internal CheckFile() routine to check for existence of a LIP file. I wrote a workaround replacement with PathFileExists() but this is worthless since it doesn't check inside BSAs.
- I'm going to put the lip files aside and focus on implementing the voice file redirection code for now.
UPDATE: okay, I really didn't put the CheckFile routine aside... I kept searching through all the offsets in OBSE and found an indirect match with the offset for the FileFinder object... BINGO: FileFinder->FindFile() looks like an exact match for what Elys calls CheckFile()... now I just need to figure out what I did wrong when calling it.... FIXED!!!
UPDATE2:
Test_USV_3: fully re-implements Elys_USV.dll. You will still need Elys_USV.mp3 and Elys_USV.lip, just remove the Elys_USV.dll and replace with test_usv.dll. The \Oblivion\test_usv.log will contain information of all missing MP3/LIP files. I'm going to start working on a generic greeting behavior when you talk to someone (ala Baldur's Gate) and a race name redirection for the computer-generated voices mod. I'll probably rename the plugin to something else too, since I've advanced out of just "testing".
Wow, does this mean the huge computer generated voice file can now be optimized and further more improved upon with better voices?
AWESOME!!!
Test_USV_4:
- The first version to now extend Elys_USV.dll!!
- This version will search for missing greetings and insert a generic Oblivion greeting, otherwise the silent voice will take over.
- Unfortunately, the generic greetings will go by quickly (about 3-4 seconds), so you may miss dialog if it's a long greeting -- but the real question is: why aren't you using the Protocolled Dialogs Mod??? Seriously, go install it right now. It gives you a dialog history window for the current conversation.
To Do:
- Implement fallback system to look in the Imperial race directory for users of the computer-generated voices mod.
- Implement random generic greetings instead of one phrase per race.
- Create framework to receive information from ESP script.
ALSO: As I mentioned above, OBSE allows plugins to communicate with ESP scripts. That means we could potentially get information on race, sex, disposition, class, and the actual dialog text... And that means: real-time synthesized voices may be back on! Of course, not anytime soon.
Test_USV_5:
- Things are getting interesting! Users of the computer-generated voice mod rejoice! Missing voice files will now fallback to searching the Imperial race -- so you can get rid of all those symlinks and just keep an Imperial race folder!
- If no voice files are found in the Imperial race, the fallback cascade continues: next, the plugin checks for generic greetings. If no generic greetings are present, it falls back to Elys_USV.mp3/lip files.
- I haven't thought of a good name for the plugin yet, so it stays in "test" form until I do, hehe.
To Do:
- Nothing for now! I'm going to put creating the ESP script communications system on hold, so I can focus on finishing all of my mainquest script patches. After that, I'll clean up the source code and maybe rename it to something other than "test".
- If anyone wants to pick up where I left off, feel free! Post your progress on this thread.
Thanks again to Elys, llde and the entire Morroblivion/TESRenewal community!
( Kool ) More Life to Morroblivion, did not capture the first section
Thanks,, dont let that Spark go Out.
Do you want test-usv.log
Edit:
Question: Doe's the Dialog that has no Audio Listing, Record it self in the Log. There is some Dialog up in Solstheim has no Audio.
Thanks, roxon_55.
If you find dialog without audio, it should be recorded in the log with Ely_usv.mp3 as the replacement voicefile like this:
You can look at the original filepath that the elys_usv.mp3 replaces to find out more about where that dialog came from. In the above example, it is "cm partners.esm" and the quest is "cmpartnersquest", the topic is "cmshare" and the actual dialog line has the (non-load order) Form ID 00010c97. In the future, one thing we could do with this plugin is update it to detect mod names and try to redirect it to another mod directory. That way, we can store all related audio in one directory for simplicity.
Hi,
When using:
strcpy(pSoundFile, NewSoundFile);
You have to make sure than the length of the string pointed by pSoundfile is at least bigger or equal to the length of the string pointed by NewSoundfile.
This was not an issue for the original USV because due to the sound path structure used by Oblivion, I knew that the Silent file pathname would be shorter in any case.
But if the code happens to overwrite the string with a longer one, you risk ended up with corrupted memory if the excess characters overwrite memory already used for something else.
In that case it might be better to create a new temporary string and replace the pointer directly. This might have some consequences and possibly require more work but I have no idea anymore since I forgot all about when I was inspecting the Oblivion code.
Hi Elys!
Yes, thanks for pointing that out -- I was cringing as I wrote that code but got caught up with the excitement that things were actually working that I forgot to go back to address this issue.
Maybe I'm reading this incorrectly but, from your inline assembly code, it would seem like Oblivion is passing a fixed-length array
char[100]as an argument to the Voice function (hence the entire string being on the stack rather than just a pointer). Would it be safe for us to just pop the pSoundFile off the stack and push NewSoundFile onto the stack?Oh wait, nevermind about the char[100]. haha, my mistake: I was counting down and not up when advancing memory addresses in my head. But still, pSoundFile is being assigned the memory address: ESP + 64h rather than the actual value stored at that address, correct? Sorry for that off-topic newbie Intel Assembly question. In any case, NewSoundFile is actually a static fixed-length array of char[MAX_PATH], so we should be able to pass a pointer to it without much extra work.
I don't have Oblivion installed anymore but just looking at the assembly code I made in SilentVoiceHook, yes you are right, I remembered incorrectly, the string is on the stack and not passed by pointer.
Now if the string is pushed as a fixed max length record (so regardless of the actual string length), then it will be safe indeed to just keep overwriting as it is now.
It's most likely. There just needs to be sure of that, which you can easily know by looking at the hooked function prototype in Oblivion disassembly, or by setting a breakpoint and look at the stack in action.
Cool, thanks again for your help!!
UPDATE (with corrections):
Alright, I've traced through the disassembly code in the sections of the Oblivion.exe which call LipsHook and SilentVoiceHook.
In the case of LipsHook, the game engine allocates 4+256+4 bytes of stack space using command [sub esp, 108h] which is used to store one DWORD at [esp+104h] and then 256 bytes used only for storing the string of the filename path starting at [esp+4h] before the allocated space is finally removed from the stack with [add esp, 108h] at the end of the function.
In the case of SilentVoiceHook, the game engine allocates a chunk of stack space [sub esp, 158h] and stores one DWORD at [esp+154h] and the filename string at [esp+50h]. Another 20 bytes or [14h] is added to the stack prior to SilentVoiceHook being called, where pSoundFile can be found at the adjusted address of [esp+64h]. Just as with LipsHook, the memory address from the start of the string at [esp+50h] to the first DWORD at [esp+154h] is not used to store any other information.
In other words, the engine appears to be using a fixed-length array of 256 characters. I will modify the code to observe this maximum string length. No need to worry about memory corruption.
Latest Source Code with 256 char limit sanity checks here:
test_usv_7:
- This version fixes a bug where the generic greeting would play multiple times in a row for mult-part greetings.
Also, I think a good way to make this plugin easily customizable is with an INI file where users can store original/replacement string pairs. Then the plugin can just go down the list of string pairs until it finds a working match. for example:
The plugin will sequentially go down the list of string pairs, replacing the first term with the second term and then testing to see if the resulting file exists. Once we have that INI reading function implemented, maybe we should call the plugin Voice_Redirector or USV_Redirector. Or maybe just call it USV_Update. Any comments, feedback, suggestions?
Future To-Do:
- Implement INI string search/replacment system (see notes in previous post).
- ESP Script communication to receive information on speaker, disposition, class, quest variables and actual dialog text.
- Insert dynamic length silence, probably with a command similar to this:
- Find someone to work on all of the above, and maybe move this source code to github.
Any way of making the log file longer than 5kb while you are upgrading.
Go with the New Name..Voice_Redirector it is what it is.
The log can grow larger than 5kb, but right now it gets over-written with a new log every time you start the game. I can either move the old one into a backup file like "test_usv.log0", or just continually add more messages to the end of test_usv.log each time the game is run. Let me know what sounds best to everyone.
I will have the first version of the "voicefile_redirector.dll" plugin posted on Github within the next day or two. FYI, I found a really convenient WinAPI function that spits out a buffer of multiple null-terminated strings with a final null to mark end of buffer: GetPrivateProfileSection(). The null-terminated strings are already in this format:
Which means I can just parse these strings as orig_string=replacement_string and quickly implement the INI-customizable voicefile redirection function. Hopefully I can get this implemented within the next 1-2 weeks.
Update:
github repository is here: https://github.com/ponyrider0/voicefile-redirector