simple machines forum

I got my KF2 project switched over to using this system today. It was very convenient. Because with it I'm putting the MM3D extension in the profiles it required some additional work that I've quietly patched. The x2mdl program also needed some changes to support some unique things that would only be of interest to anyone wanting to make custom art with it. It's been patched too.

I'll be eager to get the MSM and MHM files working to be so convenient also.

Important MM3D fix

EDITED: Yesterday I uploaded an EXE file to https://github.com/mick-p1982/mm3d/releases instead of the full ZIP file with some extra documentation. I've also fixed a problem with materials not having their alpha blending mode initialized that didn't show up in debug builds (this even applies to opening files so it's kind of a big deal.) (There were two places where the value was initialized--a big part of my working on this MM3D project has been refactoring it's really chaotic and duplicitous code.)

Again, the software for viewing/editing MM3D files like these (http://svn.swordofmoonlight.net/Sword-of-Moonlight/data/enemy/model/) can be got here (https://github.com/mick-p1982/mm3d/releases) and I've just updated it to fix an old problem where the sense of material/lighting was inverted so that there's now visible lighting in the texture mapped mode and the selection triangles are lit up red.

The old software was set up to set materials by default to 20% ambient and 80% diffuse, which is more like how you set a light, whereas a material should be 100% on both by default. Likewise it set its lights (which aren't currently customizable) up like materials to compensate. I only recently noticed it was doing that, which explained the lighting problems.

I was thinking about adding some controls to adjust its lighting and background color (bright background is not good for SOM's additive blend materials) but I've stopped at this fix for now because I'd rather be working on SOM in the next few days. I'm pretty eager to get back to work in some capacity.

Some of the new models that I edited I dropped the ambient material level down on them so I could see their lighting. It won't effect how they appear in SOM but it will make them look a little darker now until they're corrected.

I’m pleased to announce I’ve completed the most grueling round of work I’ve ever done on Sword of Moonlight and all the new files are online for the taking.

I did two long release cycles back-to-back except the first was not published because it didn’t have anything to offer to the general public. What it was instead was a big task to bolt on an OpenGL based alternative mode for SOM because new VR peripherals for PC don’t work with Direct3D 9. I chose OpenGL because I’ve used it plenty in the past and the newer systems like Vulkan and Direct3D 11 or 12 are completely different in terms of their design and are much more complicated, so it would be an even bigger effort to write code for those and it would be very alien to my past experience.

Once I completed that task the OpenGL performance wasn’t good enough, so I wanted to experiment with the dynamic vertex buffers, since SOM only uses dynamic buffers, which OpenGL is not good for. But to do that it’s necessary to rebuild all of the model files, like the MDO files, and remove the MDL files, so I realized that the time had finally come to do a project I’ve long been considering to unify SOM’s model files and build an art system into SOM to make it aware of the art development process.

SOM’s animated model files inherited a lot of quirks from the PlayStation’s design and the model format used by Sony’s SDK. These were unsuited for PC. SOM has a better format called MDO but it’s not animated, so you can say that its team were relying on this Sony style format as a crutch that shouldn’t have been published to an unsuited system. As a result this format has to be converted into a MDO like layout when the game loads, which is bad for performance, and both formats had used very small vertex chunks, which seem to be almost a clerical error, that would make them 32 times smaller than the target vertex buffer, which always remained until now only 1/32nd full, the rest going to disuse.

Working on this ended up being the culmination of all of my time working on the model conversion tools since I originally started working on SOM. These tools were the first things users at the time asked for, however I developed them and their reception was lukewarm at best, so don’t give people what they ask for I guess is the moral of this story, or don’t expect them to be grateful. I never know what to expect in terms of details when I start a programming task, it’s hard to predict, but this project turned into a never-ending slog very fast, to the point for the first time in my career I did what video game developers call “crunch” for weeks on end. I didn’t do this because I had to hit a deadline. I did it because if I hadn’t I would grow crazy forever pushing this rock up a proverbial Sisyphean ramp… and at least if you focus singularly on a task it tends to help to distract from the difficulty and monotony of the task. That was my thinking anyway. In the end it took hundreds of hours and add that to the hundreds of hours that I’ve already poured into the conversion tools, which had to be entirely battle tested for this effort, which itself proved to be a big part of the work.

In practical terms how to think about this development is SOM now will act as if it works directly with your art files, but in fact what it does is monitor changes to your art files and when they change it rebuilds its proprietary runtime files to reflect your changes. This can take a little bit of time, so it will be experienced as a pause with a busy cursor, or the splash screen taking longer to lower.

To make all of SOM’s components comply with this change it was necessary to upgrade its level editor. I had to rewrite all of that tool’s graphics from scratch. But this is good because it finally brought it up to the same standard as the rest of SOM, where before it had been noticeably lacking; another thing I long wanted to address. Its mouse input model has also been upgraded, and the screens that use gray and white squares has been upgraded to show an impression of the tiles underneath them so that they’re not so disorienting as before.

In addition I had to do forensics on all of SOM’s built-in models to reverse them back to the state of art files and archive them in an MM3D format I’m using for my King’s Field II project. You can download pretty competent 3D modeling software from my Github to work with them. I made some corrections to them where I found major non-cosmetic issues, however I haven’t yet made the effort to fix their many blemishes, although this work has moved that goal within reach. I will try to speak to more details in the forum post attached to this blog post.

Note, for this update it's very important to use SVN Update to refresh all of your files because that's the main part of the change. I usually post a link to a new SomEx.dll file like this (http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip) one, but this file will be updated with the rest (although you'll still be prompted to upgrade it, which you should in case it's patched.)

For English readers, there are new buttons in the language package here (http://www.swordofmoonlight.net/text/English/Neutral.zip) which aren't extremely essential, but they offer a "Convert" feature in SOM_PRM and SOM_MAP which enables you to manually convert all of your art files in bulk. In SOM_MAP this is useful for making demos since you can tell it to generate data folders with just the runtime files that you need for a set of maps and nothing more, and use that as the basis for your game demo without including art files. (Edited: There's also some new error messages in them.)

This manual conversion system is a little bit different in that it also copies over runtime files where it doesn't find art files, so it's not strictly the same as rebuilding your cache to avoid running into load hiccups/waits, but if all of your models are represented by art files then it will be the same thing. All but a few weird MDL files that my x2mdl tool couldn't faithfully recreate--because they look like solid control points to it--are converted to art files, although I've not yet gotten to the level geometry files. The game never sees level files (it sees the compiled MPX files) so technically they're not part of the runtime file set, but I only neglected them now because I couldn't take anymore (delay) before publishing, and I hope I get them done in time for the next release.

The Convert button in SOM_PRM is supposed to let you select a subset of items on each screen, however it doesn't yet have a multi-selection model, so it just has to convert everything on said screen until I can get around to it. As for SOM_MAP I think it should generate everything. At first I thought it might be lacking, but I realized it was simple to also include SOM_SYS's starting inventory in its output, and also some odd models like the arm and shadow and gold coin. So it should build out a complete demo. In SOM_MAP this button is in the compile window called Prepare in the English language package.

For the new MM3D models you can use the program here (https://github.com/mick-p1982/mm3d/releases) called portable-mm3d.zip. I updated it last night but intend to do a little more work soon to improve the appearance of the texture model mode with respect to lighting and selection because with SOM's models its lighting is overpowered to the point that it may as well not exist. I've put a lot of work into this modeling software, but I didn't develop it from scratch. The original version was called Misfit Model 3D. I found a newer version (fork) called Maverick Model 3D. I'm calling my version (fork) Multimedia 3D. The others aren't compatible with these files.

The art system can work with any format in theory, but I'm not sure if other exchange formats are able to fully represent SOM's feature set, even though it's relatively simplistic. 3D file formats are just weird and nonstandard... to this day there's no standard thinking in terms of how to encode animations. The main page to watch for these things is this (http://www.swordofmoonlight.net/bbs2/index.php?topic=319.0) one. The x2mdl tool that I've developed originally as a tool for making MDL files is based on Assimp. It's an older version of Assimp but I don't think it's changed that much in the meantime. In theory what formats it accepts should be viable art formats. At one point I spent a good amount of time rewriting Assimp from scratch. That code lives in a code/Daedalus folder in the code downloaded with SOM. Unfortunately I never switched fully over to it, but I hope to one day because the x2mdl code (code/x2mdl) is very messy even by my own standards. I just can't commit to that right now, and what I didn't port much of to the rewrite was the actual importer code. That said I don't stand by the importers other than the main ones I've worked with and developed myself. I recommend the MM3D path if you can stand it, but if someone wants to work with a different format I will do all I can to make that work. I chose MM3D for SOM because I'm fully in control of it and it's a very close fit.

Something all of Assimp's loaders lack is support for soft animations. That's one reason I forked it since there was no movement happening in that direction. I've only implemented soft animation paths for the import code I've written myself. Another reason I forked it is I strongly disagree with how its importers work since they throw away connectivity data and don't work efficiently the way most proprietary formats work. (Not only are they inefficient for performance they're not good for writing maintainable/legible/brief code reflecting the structure of a target format.)

EDITED: I'm going to take a much needed break of sorts for a while. I hoped I could do another release before New Years but this one spiraled far into December, so I'm not expecting that to happen. I really want to pick up my King's Field II project so the next goal I have in mind is to implement level streaming, a la KFII and KFIII, that is always on everyone's lips when talking about King's Field but From Software neglected to include in SOM. Their team implemented very little more than the bare necessities to make a half passable PC version of the original King's Field (1994) that they only published in Japan. I'm not sure what will be required to implement streaming, but it feels like the right time to do it. I hope it's much simpler than this project turned out to be. Because it will deal a lot with level data it should be an opportune time to try to finish this art system with respect to level geometry. Level files also include the MHM clipping format which I intend to extend to static objects at the same time, hopefully included in the next release. For animated models my plan is to put clipping data in the MDL format. BTW, I forgot to say that how the new system unifies the model format is to combine MDO and MDL so that what you get is a hybrid where everything is a MDO file except there's animation data inside the MDL file, so neither has been discarded IOW.

new MM3D release

https://github.com/mick-p1982/mm3d/releases/tag/win32-dec-2021 is a new version of the makeshift 3D modeling software I've developed which is meant to work with the new SOM model files I'm going to be publishing any day now. I've added notes above.

Strike-out: Please use this link https://github.com/mick-p1982/mm3d/releases for newer versions.

I had a really bad evening with it because for some its app always starts slow (something about wxWidgets) but now I got it to start fast but it would show a busy cursor (IDC_APPSTARTING) for like 5 seconds at the start. It was really dumb because the app worked fine. I spent hours trying to make it go away through programming fixes until it went away for mysterious reasons, at the very end... that I don't want to speculate on further. I just want to relate a frustrating evening sitting in the cold. It doesn't help I'm missing a cat lately.

There will be new and improved versions of the other tools/apps here in the coming days. They need to go up after the new release does. There's now (shortly) a version of x2mdl built-into SOM (technically a new DLL file) so using x2mdl/cpgen should be a thing of the past, but the renaming trick for x2mm3d.exe is still pretty useful to convert Assimp's formats to this MM3D format. The new system isn't technically limited to MM3D but I haven't worked with other formats, most don't have everything SOM needs. There will probably be issues with control-points that would need to be ironed out in order to use different formats. CPs are the most annoying part honestly.

There are some new convenient command-line switches for x2mdl but I haven't yet staked out ones for changing the output mode and the output location.

Important patch

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.5.zip

This is a must patch to fix some new/significant animation problems in this latest "demo" I somehow didn't notice before publishing it.

The big one is some animations like SOM's skeletons were drifting off center. (The reason for this has to do with converting 8-bit data to 255 versus -1 to detect the root joint that's supposed to stay pinned to 0,0,0.) A more humorous bug is the new scale factor was not updated for the arm_bicep extension so the arm had been missing above the elbow. (I had fixed it for the new MDL+MDO animations but at that point hadn't yet decided to move plain MDL over to a common scale factor.)

EDITED: I've reuploaded (SomEx.dll) to add a fix (see next paragraph) and added a warning for a change to the games as a side-effect of this patch.

The fix has to do with "neighbors" on different elevations and cells without tiles (perhaps because their tile is on another layer) having been incorrect in the initial patch yesterday. There may still be some issues in this area.