Sword of Moonlight > Devs

EXIT: Long time no update! (Sorry about that)

(1/4) > >>

Holey Moley:

--- Quote from: http://www.swordofmoonlight.net/archives/sword-of-moonlight/2021/12/long-time-no-update/ ---I’m pleased to announce I’ve completed the most grueling round of work I’ve ever done on Sword of Moonlight and all the new files are online for the taking.

I did two long release cycles back-to-back except the first was not published because it didn’t have anything to offer to the general public. What it was instead was a big task to bolt on an OpenGL based alternative mode for SOM because new VR peripherals for PC don’t work with Direct3D 9. I chose OpenGL because I’ve used it plenty in the past and the newer systems like Vulkan and Direct3D 11 or 12 are completely different in terms of their design and are much more complicated, so it would be an even bigger effort to write code for those and it would be very alien to my past experience.

Once I completed that task the OpenGL performance wasn’t good enough, so I wanted to experiment with the dynamic vertex buffers, since SOM only uses dynamic buffers, which OpenGL is not good for. But to do that it’s necessary to rebuild all of the model files, like the MDO files, and remove the MDL files, so I realized that the time had finally come to do a project I’ve long been considering to unify SOM’s model files and build an art system into SOM to make it aware of the art development process.

SOM’s animated model files inherited a lot of quirks from the PlayStation’s design and the model format used by Sony’s SDK. These were unsuited for PC. SOM has a better format called MDO but it’s not animated, so you can say that its team were relying on this Sony style format as a crutch that shouldn’t have been published to an unsuited system. As a result this format has to be converted into a MDO like layout when the game loads, which is bad for performance, and both formats had used very small vertex chunks, which seem to be almost a clerical error, that would make them 32 times smaller than the target vertex buffer, which always remained until now only 1/32nd full, the rest going to disuse.

Working on this ended up being the culmination of all of my time working on the model conversion tools since I originally started working on SOM. These tools were the first things users at the time asked for, however I developed them and their reception was lukewarm at best, so don’t give people what they ask for I guess is the moral of this story, or don’t expect them to be grateful. I never know what to expect in terms of details when I start a programming task, it’s hard to predict, but this project turned into a never-ending slog very fast, to the point for the first time in my career I did what video game developers call “crunch” for weeks on end. I didn’t do this because I had to hit a deadline. I did it because if I hadn’t I would grow crazy forever pushing this rock up a proverbial Sisyphean ramp… and at least if you focus singularly on a task it tends to help to distract from the difficulty and monotony of the task. That was my thinking anyway. In the end it took hundreds of hours and add that to the hundreds of hours that I’ve already poured into the conversion tools, which had to be entirely battle tested for this effort, which itself proved to be a big part of the work.

In practical terms how to think about this development is SOM now will act as if it works directly with your art files, but in fact what it does is monitor changes to your art files and when they change it rebuilds its proprietary runtime files to reflect your changes. This can take a little bit of time, so it will be experienced as a pause with a busy cursor, or the splash screen taking longer to lower.

To make all of SOM’s components comply with this change it was necessary to upgrade its level editor. I had to rewrite all of that tool’s graphics from scratch. But this is good because it finally brought it up to the same standard as the rest of SOM, where before it had been noticeably lacking; another thing I long wanted to address. Its mouse input model has also been upgraded, and the screens that use gray and white squares has been upgraded to show an impression of the tiles underneath them so that they’re not so disorienting as before.

In addition I had to do forensics on all of SOM’s built-in models to reverse them back to the state of art files and archive them in an MM3D format I’m using for my King’s Field II project. You can download pretty competent 3D modeling software from my Github to work with them. I made some corrections to them where I found major non-cosmetic issues, however I haven’t yet made the effort to fix their many blemishes, although this work has moved that goal within reach. I will try to speak to more details in the forum post attached to this blog post.
--- End quote ---

Note, for this update it's very important to use SVN Update to refresh all of your files because that's the main part of the change. I usually post a link to a new SomEx.dll file like this (http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip) one, but this file will be updated with the rest (although you'll still be prompted to upgrade it, which you should in case it's patched.)

For English readers, there are new buttons in the language package here (http://www.swordofmoonlight.net/text/English/Neutral.zip) which aren't extremely essential, but they offer a "Convert" feature in SOM_PRM and SOM_MAP which enables you to manually convert all of your art files in bulk. In SOM_MAP this is useful for making demos since you can tell it to generate data folders with just the runtime files that you need for a set of maps and nothing more, and use that as the basis for your game demo without including art files. (Edited: There's also some new error messages in them.)

This manual conversion system is a little bit different in that it also copies over runtime files where it doesn't find art files, so it's not strictly the same as rebuilding your cache to avoid running into load hiccups/waits, but if all of your models are represented by art files then it will be the same thing. All but a few weird MDL files that my x2mdl tool couldn't faithfully recreate--because they look like solid control points to it--are converted to art files, although I've not yet gotten to the level geometry files. The game never sees level files (it sees the compiled MPX files) so technically they're not part of the runtime file set, but I only neglected them now because I couldn't take anymore (delay) before publishing, and I hope I get them done in time for the next release.

The Convert button in SOM_PRM is supposed to let you select a subset of items on each screen, however it doesn't yet have a multi-selection model, so it just has to convert everything on said screen until I can get around to it. As for SOM_MAP I think it should generate everything. At first I thought it might be lacking, but I realized it was simple to also include SOM_SYS's starting inventory in its output, and also some odd models like the arm and shadow and gold coin. So it should build out a complete demo. In SOM_MAP this button is in the compile window called Prepare in the English language package.

For the new MM3D models you can use the program here (https://github.com/mick-p1982/mm3d/releases) called portable-mm3d.zip. I updated it last night but intend to do a little more work soon to improve the appearance of the texture model mode with respect to lighting and selection because with SOM's models its lighting is overpowered to the point that it may as well not exist. I've put a lot of work into this modeling software, but I didn't develop it from scratch. The original version was called Misfit Model 3D. I found a newer version (fork) called Maverick Model 3D. I'm calling my version (fork) Multimedia 3D. The others aren't compatible with these files.

The art system can work with any format in theory, but I'm not sure if other exchange formats are able to fully represent SOM's feature set, even though it's relatively simplistic. 3D file formats are just weird and nonstandard... to this day there's no standard thinking in terms of how to encode animations. The main page to watch for these things is this (http://www.swordofmoonlight.net/bbs2/index.php?topic=319.0) one. The x2mdl tool that I've developed originally as a tool for making MDL files is based on Assimp. It's an older version of Assimp but I don't think it's changed that much in the meantime. In theory what formats it accepts should be viable art formats. At one point I spent a good amount of time rewriting Assimp from scratch. That code lives in a code/Daedalus folder in the code downloaded with SOM. Unfortunately I never switched fully over to it, but I hope to one day because the x2mdl code (code/x2mdl) is very messy even by my own standards. I just can't commit to that right now, and what I didn't port much of to the rewrite was the actual importer code. That said I don't stand by the importers other than the main ones I've worked with and developed myself. I recommend the MM3D path if you can stand it, but if someone wants to work with a different format I will do all I can to make that work. I chose MM3D for SOM because I'm fully in control of it and it's a very close fit.

Something all of Assimp's loaders lack is support for soft animations. That's one reason I forked it since there was no movement happening in that direction. I've only implemented soft animation paths for the import code I've written myself. Another reason I forked it is I strongly disagree with how its importers work since they throw away connectivity data and don't work efficiently the way most proprietary formats work. (Not only are they inefficient for performance they're not good for writing maintainable/legible/brief code reflecting the structure of a target format.)

EDITED: I'm going to take a much needed break of sorts for a while. I hoped I could do another release before New Years but this one spiraled far into December, so I'm not expecting that to happen. I really want to pick up my King's Field II project so the next goal I have in mind is to implement level streaming, a la KFII and KFIII, that is always on everyone's lips when talking about King's Field but From Software neglected to include in SOM. Their team implemented very little more than the bare necessities to make a half passable PC version of the original King's Field (1994) that they only published in Japan. I'm not sure what will be required to implement streaming, but it feels like the right time to do it. I hope it's much simpler than this project turned out to be. Because it will deal a lot with level data it should be an opportune time to try to finish this art system with respect to level geometry. Level files also include the MHM clipping format which I intend to extend to static objects at the same time, hopefully included in the next release. For animated models my plan is to put clipping data in the MDL format. BTW, I forgot to say that how the new system unifies the model format is to combine MDO and MDL so that what you get is a hybrid where everything is a MDO file except there's animation data inside the MDL file, so neither has been discarded IOW.

Holey Moley:
patches

I noticed the new MM3D files made from MDO files had UVs that weren't on the origin (0,0 to 1,1) but offset by 1, so I've gone ahead and repaired them and x2mdl too. I almost decided to remove x2mdl.exe because it's too heavy to repair like this, however instead I've packaged it with x2mdl.dll so it will be updated at the same time. This required patching Somversion.dll because I'd never actually tested it with more than one file. Luckily I'd programmed everything this requires but just had to fix a few things since I'd not tested. (Edited: I actually did remove x2mdl.exe, so it requires updating x2mdl.dll to get it back.)

The x2mdl versions in the TOOL folder don't have this fix, although it's not likely to be a problem. The patch links follow in case anyone already downloaded these versions.

http://csv.swordofmoonlight.net/Somversion.dll/1.0.4.6.zip
http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip

SVN Update is required to fix these MM3D files, but it's justnot even a cosmetic fix unless the intent was to edit these files... as in the bad UV mapping doesn't change their appearance since they wraparound.

Holey Moley:
Again, the software for viewing/editing MM3D files like these (http://svn.swordofmoonlight.net/Sword-of-Moonlight/data/enemy/model/) can be got here (https://github.com/mick-p1982/mm3d/releases) and I've just updated it to fix an old problem where the sense of material/lighting was inverted so that there's now visible lighting in the texture mapped mode and the selection triangles are lit up red.

The old software was set up to set materials by default to 20% ambient and 80% diffuse, which is more like how you set a light, whereas a material should be 100% on both by default. Likewise it set its lights (which aren't currently customizable) up like materials to compensate. I only recently noticed it was doing that, which explained the lighting problems.

I was thinking about adding some controls to adjust its lighting and background color (bright background is not good for SOM's additive blend materials) but I've stopped at this fix for now because I'd rather be working on SOM in the next few days. I'm pretty eager to get back to work in some capacity.

Some of the new models that I edited I dropped the ambient material level down on them so I could see their lighting. It won't effect how they appear in SOM but it will make them look a little darker now until they're corrected.

Holey Moley:
PLEASE MAKE IT STOP (PATCH)

http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip

x2mdl.dll is updated for SVN Update

I shot myself in the foot with one of those patches that needed different behavior for MDL output than MM3D output (control point stuff never ends) and also ran into a problem (probably related now that I think about it) that was applying a CP rotation to the whole model for soft body models like the Slime. Basically a huge class of models I didn't notice. I really should do more testing before publishing things. I never learn. I really don't like posting these notices, but it seems the correct thing to do.

I think I'll delete the earlier patch notices to save readers the trouble...


--- Quote from: Good info from the older patches ---A problem with this new system is if a bug gets into the cache or if something goes wrong and x2mdl.dll doesn't finish then the cache has to be invalidated. The simplest way is to remove the shortcut file if you know the model name (try SOM_PRM to see the model) or delete the files from the (%TEMP%\Swordofmoonlight.net\art) cache folder. I think eventually the tools like EneEdit will have a menu option for managing this, especially if it becomes a problem.
--- End quote ---

EDITED: Some good news, the optimized release mode build of x2mdl.dll runs way faster than the debug version, so the wait isn't really noticeable on single models, and probably won't be a big deal for converting lots of models when a game loads either.

Holey Moley:
I got my KF2 project switched over to using this system today. It was very convenient. Because with it I'm putting the MM3D extension in the profiles it required some additional work that I've quietly patched. The x2mdl program also needed some changes to support some unique things that would only be of interest to anyone wanting to make custom art with it. It's been patched too.

I'll be eager to get the MSM and MHM files working to be so convenient also.

Navigation

[0] Message Index

[#] Next page