simple machines forum

MM3D upgrade

First, off-topic, I've uploaded a minor SomEx.dll patch just now that fixes a problem with scaling animations. I'm not going to announce it as usual.

The past few days I've been working upgrading MM3D's modeling capabilities by addressing several of its tools that original author neglected to add texture map support to. That made these tools pretty much useless since texture mapping is the most difficult thing, and it's not worth ruining your texture map to edit a model.

I've mostly been using it for animation, but lately I've not used Blender since I started the new round of work on my KF2 project. I have been making some edits, but I'd hit enough walls that I decided it was time to roll up my sleeves and do something about this. The demo and source code for this is on Github.com. I really enjoy working on MM3D this way. I've been pleased with my decision to take it under my wing.

Anyway, this is just my activities of late. Hopefully I will be back to working on KF2's NPCs by this time tomorrow.

Attachments

 KF2 underbelly.png 

Ever wondered what the fountain looks like from underneath? Well I'm kidding of course, there's nothing there, but I had to improvise this. Both because you can actually stand on the bottom of the sluice gate canal (in the original the standing elevation hardly changes) and you can crouch down quite far with SOM. Lately I've also designed a lip inside the dragon doors to make their ability to open without jamming/breaking passably believable (one door must begin opening a little sooner--really faster--than the other) and I just extended the mouth of the canals. I'm not sure I like how that turned out, I'll probably go back and redesign it. It seems more distracting than I expected. I put a slope in there so the water wouldn't crash down. The lighting draws a bit too much attention. It has to be extended (somehow) for the same reason as the bottom side of the fountain.

Yes the yellow water is flowing, even though the canals are empty. There's actually an animation that just pulls the water up like with a crane. I guess the game short-circuited it, or didn't use it, or something. When the yellow water appears it just fades in. I'm thinking about making it possible to remove the dragon statues just for the heck of it, and making an animation where the water comes out as a thin thread and then expands out. Likewise the canals would become shallow as the closing animation recedes.

Edited: I worked on the obelisks yesterday... I did some tricks to help it look better by extending the texture map on the base and mirroring the sides so the squares appear level. I think the reason the base is how it is is the artist might not have tried turning the quad faces into 4 triangles instead of 2. It sounds really basic, but it does always surprise me when I try to map a trapezoid and it comes out completely distorted. I guess that's just how barycentric coordinates work, which I think is what's used to interpolate triangles.

If I spend even a little bit of time in this chamber, I start seeing everything as yellowed. Even the moon becomes bright yellow, because it's not blue. It's an odd feeling. After leaving the room the regular (not blue) walls are all very ripe yellow. It's a funny thing, the color schemes for the regular marble/stone walls is somewhat yellow. However textbook saturation enhancement formulas assign weights to red, green, and blue. I've always wondered what justifies these weights (I mean you can read up on it) but it's odd that red and green (especially) are weighted much harder than blue. As a result applying a saturation function beyond a very little tends to turn things yellow... unless I suppose the scene is predominantly green, in which case it may become ultra green because green's contribution to this yellowing is significantly more than red's. It's I guess a pure function of our physiology. I wonder if it applies though to everyone equally. I'm sure studies have been done, but I'm sure also it's just a compromise. But I sometimes wonder what if those weights were different, if it would allow for saturation to be pushed further before it breaks or not. Anyway... I wonder if it's somewhat linked to this color compensation/exposure our eyes play on us, since it looks very similar.

[Edited: I guess this pic has come some way since Reply #120 at the top of this page! BTW, I believe the fountain was the last "object" on this map. Next is NPCs, and then comes monsters/monks.]

dragon doors

I've been having an interesting time with the heavy french doors outside the fountain room. One surreal experience I had last night was realizing there is a huge snafu in them when the frame rate plummeted. I'd already seen some phantom polygons in them, but it turned out there are 3 copies of the doors stacked on top of each other!! Some poor artist was working overtime that day (Edited: I've fixed up the frames too in order to mesh with the walls.)

That led me down a pretty strange hour or so of peeling off polygons from the doors, almost physically. It was the most tactile experience I've ever had with a 3D model, an odd feeling, so I just kept at it for the unique experience of it. I thought for a second about programming a solution to tease the copies apart, and then this morning I realized I could've worked a lot faster by pulling out a vertex, and then using a function that selects all of the connected polygons, and their connections, and so on... but that would've ruined my fun. Bu I did do that to confirm I didn't miss anything, comparing the vertex and polygon counts to the real model's.

Then I got swept up today working on a scheme to hide the interior sections of the door while they're closed, because if not they appear as shimmering cracks in the walls, which is something all our 3D technology can't solve it seems, since the edges coincide perfectly. It's a unique challenge of low-poly animations I suppose, although I imagine it probably does come up in high-poly scenarios as well. I found a sign extension bug in doing this that was causing Y scaling to make the MDL file appear to have X scaling (and so corrupting the decoding) (this is in my rewritten animation routines) and, unrelated, I had to disable the 60 fps upscaling when scale values on either side of the interpolated frames are set to 0, to prevent artifacts when using scaling to hide polygons.

Lastly I just added a fix to reset objects with closing animations to the first frame of their opening animation, or else I'd have to also put this scaling trick into the closing animation, and I reckon that's busy work and a source of mistakes, better to have the player behave consistently. (It shows the first frame of the opening animation when the door, etc. hasn't yet been opened, so I reckon it should show that after closing as well.)

Another problem arises with these doors that also affects the wooden doors. That is it's really not possible for a pair of doors to not have any space between them, at least if they're flat. And you can see this in the animation, it's quite clear the polygons clip into each other.

With the wooden doors it seemed sensible to put a notch in them. With these doors I'm less inclined to do that because they're more impressive in design. I figure that I'll have to invent some solution by remodeling them. I wonder if simply having a crescent (inner) contour would be sufficient. That might keep with the moon theme of this area. Although "dragon doors" appear in KF games often. I think that there should be sensible design that's also functional while maintaining their mystique. It'll be interesting to see what I can come up with.

Optional x2mdl (dll) patch (#2)

http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip
http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip (maybe slightly improved)

I got lucky with an accident that helped me to identify a pretty wild bug in x2mdl when converting the MM3D files (that format specifically) into SOM's files. It will likely crash if the MM3D file doesn't assign all of the model parts to groups (i.e. materials/smoothing groups) (which is how I found it, by accidentally not doing so with a portion of model) but the real kicker is a "loop" I wrote was using the wrong "variable" to track its progress, which could potentially be slow things down.

(I've never made a bug like this before, but basically in one standard kind of loop you have a number that is incremented until it gets to another higher number. What I did is mistype the "variables" holding the numbers so that the higher number is incremented and the other retains the value of the first item that is looped over. In that case the CPU just keeps hitting that item until the higher number "wraps around" in memory so that it becomes lower, and that can take billions of instructions (probably around 1B in this case) which is in theory a gigahertz if an instruction is one cycle. In practice though there's probably several instructions involved, although this loop was very simple.)

I'm just glad I found it. I'm sure it would turn up eventually, but I actually neglected last night to investigate it since I forgot to make a note, but luckily I remembered and went back and reproduced the crash (in SOM_MAP) to figure out what happened.

x2mdl (dll) patch

http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip
http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip (maybe unnecessary, can't remember)

I'm still patching this leaky raft... namely I didn't test the texture downsizing path (to fit MDL's 256x256 limit) and some other nontrivial stuff I won't go into details about. One serious problem led SOM_MAP to convert non level geometry models into level geometry. (A new time optimization is it no longer generates MDL textures when there aren't animations, since it won't be outputting a MDL file. Eventually I want to omit textures from MDL files altogether since they're not shown in the game. But for now--transitioning time--the files still have textures so they can be opened up and looked at.)

Attachments

 KF2 shrine.png 

Here's a screen from what I'm working on. I disabled the dither in this screenshot... I wanted to get more detail into the icon in the back of the room. I scooted it over 0.25 meters in order for it to be centered. I always wonder if its art direction is meant to be askew or not whenever I think I'm correcting something like that. I mean, it gets to your head, in a perfectly symmetrical room, is there a reason the icon is askew? Or were artists working overtime? The latter certainly seems to be the case.

Another thing you may not notice in this room is these braziers actually have some lava flows in their basins. You can't see it in this screen, but it's very easy to walk up to them and see it in this version... whereas in the original you really have to work to get the (very exaggerated) walking effect to peek over the lip of the basins, usually for only a brief moment... however somehow I was able to get it to perch, but still it's only possible to see just the tip of the molten lava.

These flames are all synchronized in this screen. I think there's supposed to be some randomness to them, although it's probably synced to the spawn frame, which in this case is identical since I started the level in this room, via the level editor.

I can't fathom how long it took the artists to imagine all of these set pieces. It takes me what seems like a long time just to set them back up. I'm learning partially how much work it would take to mount a new KF production. The answer is a lot. It's a lot of coordination and task management. I think it would be really hard to involve other team members, but necessary. At least by doing a first outing with this project a lot of the difficult programming challenges can be met in advance.

P.S. I also finished the Harvine room in the first zone today, and rebuilt the magic crystals, thinking my new set up is better at accurately capturing their double (and asymmetrical) sided texture map and that I may have made a mistake on the fire and water ones before. I did the wind, earth, and holy (light?) ones today. I'm not 100% positive this rendition of this crystal is accurate because I plundered the one in my emulator game.

Patch #2 (today)

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip

I think I may have found the reason for random crashing at start in this release in this patch. I also tested the new background loading system today on my KF2 project and found I neglected to test it on an elevation change scenario. It turned out a "shock absorber" effect caused a big glitch. (The crashing I think was caused by the [Number] based extensions in the Ex.ini file, since it is used on the background loading "thread" when it wasn't designed for sharing between threads. There may be other sources of this kind of problem, I now know what to keep an open mind about. Of course SOM isn't a multi-thread system, even though many of its tools are loaded down with "critical sections" because it's written with MFC code that seems to have produced ridiculously bloated applications. When designing the system I had to be careful about what kind of bad interactions could happen. Luckily loading code is pretty isolated.)

Edited: Oh yeah... as for background loading KF2. There's still a hiccup. I don't know if I'll be able to work it out of the demo I'm working on or not. It could very likely have nothing to do with disk access... maybe just memory cache hits. On beefier CPUs it might not hiccup. Hopefully I can profile it some more at some point, or think of some strategies to avoid extra work.

Minor usability patch (SOM_MAP)

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip

Sometime back in 2017 I noticed when SOM_MAP's palette area moves on its own (after clicking a tile) that it wipes out the information displayed in text boxes under the work area.

But what I somehow didn't notice until yesterday is this also wipes out the stored information used to place copies of the selected tile (instead it reverts to as if you clicked the palette area... forgetting things like elevation and rotation) which is obviously important for work.

So I'm going ahead and putting this out there in case it helps anyone in the meantime. An example of when this would become a problem is if you were using the tile sets packaged with SOM and mix-and-matching tiles from different sections of its KF sample project. It could also happen if the palette just happens to be on the border of the tiles you're working with. I guess it goes to show how little time I spend building game maps with SOM that I didn't notice this for more than a decade of working with it!

Important Patch (Critical)

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip
http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip

Late last night I ran into yet another problem saving layer MAP files. I can't ever seem to complete that project... I really feel like at some point I need to rewrite the MAP file input/output code from scratch, but maybe this is the last time... knock on wood. (What was happening was some code I wrote that juggles some data between the files failed to give READ permissions to the file in question... which explained why some junk would appear at the end of layer MAP files... but the real problem I suppose is I added some error handling code that reported errors, and I thought it would catch my attention in debug modes, but it was incomplete. I think this was the same problem the last time I published a patch the other week... i.e. new error code causing the load/save operation to give up where previously it would've carried on.)

What's a wonder is how this didn't become a problem sooner. (This is a common refrain for bugs... really the problem bugs are the ones that get masked by shear chance instead of backfiring immediately.)

Last night I also found numerous new problems I introduced to x2mdl around textures mainly... and I chanced upon a realization that som_db.exe ignores the last frame in hard (skeletal) animations, and so I've adapted x2mdl to reflect this behavior by clipping the final frame down to 0 time... this means the last frame is shown at the end of the animation but for 0 amount of time. I.e. you won't see it unless you freeze the animation (or set the cursor) on the end. This is actually how animators usually work, so this is a benefit. The only problem is now I have to go back and regenerate all of the hard animation MM3D files from the last release a while ago. (Note, technically this topic/thread is still maintaining that release.)

story time (j/k)

Today was a town/groceries day, but I managed to squeeze in a project to work on programming KF2's drawbridges. It was somewhat eventful (no pun intended.)

How SOM uses switches you could accomplish this a few ways. However in practice it probably wouldn't work for a number of reasons. How I wanted to make it simple, so I set up a global event that is constantly updating. It just checks the state for both drawbridge systems and updates them. Alternatively the event could be triggered by a button press, for each bridge.

The latter approach might require a timer, however the switch animation seems to be synchronized with the drawbridge's, and it seems like SOM doesn't let you flip the switch back until it finishes. Although I haven't looked at the code for this.

Edited: Actually the PlayStation game doesn't work like this. In it the bridges don't begin moving until the switch is fully transitioned, and the bridges move more slowly (and creakily) than the switches do. (I wonder if SOM should work like this... the way it works is more like an analog--1-to-1---switch, whereas the way KF2 works is like a digital switch... both have merit.)

For the approach I chose it wouldn't work because every event frame the animation would reset. So I worked on the object engage (animation) instruction to make sure it finishes playing the animation unless the state is flipped (I should probably add some protection to make sure the animation completes while I'm at it.)

But then it turned out there was another problem, because after the animation plays, the "object" automatically switches over to its resetting animation. For doors that means it closes itself.

This applies to either approach. So in some ways it probably turns out that to have the object remain open that the global event approach is the only way that can work. Of course I had to modify the programming, essentially to have the animation reset the wait timer, which for doors is set to 120. I think at the old 30 fps this holds the doors open for 4 seconds. However a while back I modified doors to behave like KF2's. That's better because they don't close on your face, and the timing is moot because they now will close at an opportune time.

This was a little confusing at first until I figured out that while waiting, it's actually displaying the first frame of the closing animation. I had programmed a strategy when I realized this, and so I knew I should work with the given system and take that into account, and take a new tack.

Then I noticed something, that wasn't a fluke. The drawbridge wouldn't quite open up fully after closing. This was a hint of something I've kind of suspected but never quite understood. It turns out (it seems) that som_db.exe will pretty much ignore the final frame in animations. I never could quite confirm it because usually for a door like animation the final frame isn't that important in the scheme of things... however, for the drawbridge it needed to be level in the final configuration.

This was kind of annoying, since I had to factor all of this into x2mdl. And while I was in the process I found a lot of other recent problems with it... as you do. I'll have to patch it, etc. again tomorrow.

It's actually nice to have the player (som_db) ignore the final frame, because the hard animations use "step" sampling, which is in a way ambiguous on the final frame, and I realized it was causing a problem with "round-tripping" animations between art and runtime files. I figured it was doing the best it could, but I was missing this important piece.

The existing animations will all have to be patched/revised, at least for the hard animations. I'll have to regenerate them. I looked at the player, and it's a little confusing because when the animation advances, it sets it to plus 1... which is what you'd expect. And it rejects it if that value is less than the length of the animation. But what I expect is that "plus 1" is really amounting to rejecting if less than the runtime minus 1, in a roundabout way. For the soft animations the subroutine that gets the runtime actually adds 1 to their length, which is I think a way of bringing both systems into alignment. Soft animations don't use "step" sampling.

If I wasn't clear, this is actually a good development for art work because it ensures the end of the animations are the same. It's a little confusing though because in the player the final frame never appears. It's more or less assumed either it matches the first for looping, or the closing animation will reproduce it.

Another interesting note is SOM doesn't really have a system for KF2's drawbridges and trap doors. What I've used is the double-door model. It doesn't actually capture the incline of these, since it's still working as if it's an upright door. However it pretty much captures everything, except that really you're walking on top of the door, which is assumed to be flat. I could try to improve it some time by factoring in an incline formed by the control-points. But I'm planning to add an even more flexible system, simply by enabling objects to use an MHM like clipping model before very long. I'm thinking this is something I'd like to get into this next demo. A good way to exhibit it would be to be able to climb inside the fountain I think.

I've been able to convert the new map's objects and characters fairly briskly (still very much in progress) but I keep finding myself midday in the middle of fixing some bug or programing this or that, often into the night right up until bedtime. One interesting moment came up last night. I'd been kind of unhappy with how the new map looked for some reason I couldn't place. I thought maybe its per tile ambient lighting isn't set up yet. Then last night I realized there was something wrong with the texture upscaling system (set up because KF2's textures are pretty small by modern standards.) It turned out for some reason (I forget) I'd set it to ignore the sky model's textures (maybe because they're always at a fixed distance that might not qualify, I don't know) and so now that the art system I just set up was sourcing its textures out of the same folder they were having the effect disabled. It looks so much better restored that I felt really good that it was back on track again.

It sounds very simple, but whenever something breaks, and it's for some left field reason, it's kind of stressful for programmers. We have to think on our toes, or dread that wait until every logical possibility is exhausted, and you've been there before, you know it's going to be some curve ball... you just don't know how long until it will reveal itself. Lately I've been having a problem that seemed to creep up when I worked on the MPX preloading system (I'm not sure what to call it) that I thought was a side effect of switching D3D over to multi-thread (which I've recently disabled in the latest patches, it seems to be working now without it.) but it turns out not. What's happening is it's crashing at start up every now and then, always inside an Intel driver. I've been watching it but I haven't got any leads on it. I'm a little worried about it. It's not good. I haven't figured out if it's isolated to my KF2 project. In theory it could be an Intel bug, but anyway, it is something that's bothering me lately. I just don't have any way to approach it. I worry it could be worse (always crash) on other drivers.

Update: I think I figured out the crash, or the basic reason for why crashes started with the introduction of the new background loading system. I wasn't open minded enough to realize some other subsystems could be split across multiple "threads" that aren't designed for that. (The first major such find was the [Number] system that enables some functional programming in the Ex.ini files. I just got lucky that eventually the CPU state post-crash made some sense... or at least gave me an idea.)