simple machines forum

Please login or register.

Login with username, password and session length
 

News:

Remember to make your own backup of posts before submitting.

 
 
Pages: [1] 2

Author Topic: Exit: Passing between worlds  (Read 8828 times)

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« on: February 01, 2022, 12:10:08 AM »

An important milestone has (finally) been reached by Sword of Moonlight by realizing the original King’s Field single environment experience of no “loading screens” after hopping into the game. This is something King’s Field fans always gloat about since it was pretty novel back in the day, and even to this day lately I’ve played a little bit of some PlayStation 4 games and am finding myself shocked how much of the time is really just spent on minutes long loading screens. In my opinion I would happily remove the high fidelity visuals in these games for practical seconds long loading screens. Of course, no load screens would be ideal.

I believe From Software published Sword of Moonlight so that it would be carried away by its public. This is a crucial facility no King’s Field Making Tool is complete without. I believe Sword of Moonlight would not have been published in this form except as a challenge to take it up. We should not forget that Sword of Moonlight is modeled on a sword after all.

I wasn't sure whether to make this a release or a demo because I still need to battle test it by building out a full 2 map system using my KF2 project (it's long over due for another map to be added to its demo on itch.io and I've been postponing that until it could properly replicate KF2's no hiccups experience.) Both because I was thinking of making it a demo and I kept shooting myself in the foot with patches to the current release, all the way up to today, I've decided to just write this blog post and refer it to the current patch, which I hope is finally stable. Below are links to the patch files. I'm also going to be posting any future patches to this topic/thread to mark a point of departure.

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip
http://www.swordofmoonlight.net/text/日本語/最新.zip (Japanese)
http://www.swordofmoonlight.net/text/English/Neutral.zip

I've been working on this since the last weeks of December, last year. Although in that month I mainly did preliminary work (some studying/planning) by adding layer elements to events that take map coordinates. Technically I think it was possible to just refer to the base layer and set its elevation setting to coincide with a desired layer's elevation. I've tried to cover layers fully in the event system in this patch/quasi-release/project/task. This also includes new elevation limits and a rotation option for action based events and trip zone events. Note, I think in terms of activating things everything is now covered so that activation doesn't span "infinitely" above and below. IOW the default range is now limited to the object or character model bound to the event. (Also there's a "breaking-change" for the circle trip zone which had added its radius to said object's radius, which is no longer done. I've added an option to re-enable this, however I have a feeling the restriction to the object's base/height is more likely to break old games. I had a thought today to roll all of that together.)

One new event "module" (I think I'm calling these modules now, even though I have another plan for literal C# modules for SOM, but that will offer a plugin system for events too, so they will technically be modules too) was added, filling out the last remaining slot in the list (before it requires a scroll-bar) as if fated to fill in this slot (why was a slot left unfilled?) much like the layer system filled in the spaces left by the layer system From Software ripped out prior to publishing. Anyway, this event module/command/operation/instruction (call it what you want) sets a secondary map to start loading in the background. This requires a multi-core CPU. They've been standard for many years now and current Windows systems require them to function, so unless there is demand it will just be a minimum requirement from now on to use this. I did some timing tests and I think that a single-core CPU just won't be practical given how slow Windows is to access files (I'm a little surprised it can stream the BGM file, and this may be a bottleneck) at 60+ frames per second, so if demand arises the only likely thing I can do is add a way to disable this so that loading screens come back.

As a starting point this system functions just like the King's Field games would have functioned (I think all of them do so this way anyway) by connecting two maps at a time. Technically more than two "maps" (i.e. levels) will stay in memory for a few minutes, but two maps will remain in memory. Memory just means they're not on your disk drive. I'm pretty sure from what I've seen online that KF2 duplicates parts of its maps that intersect on the threshold. Currently this duplication is required with this existing system. I am interested in pushing it further down the road to actually display the other map against the current map. Logistically there is only one active map. Things like events only run on one map's events at a time. Under the current system the maps switch out instantaneously. If a more literal stitching system combined their images however, there would still just be one set of events. I do plan to try to carryover monsters, however presently, any monsters visible on map change will just blink out. This version isn't fully complete. To avoid that you can set up doorways (gates) like you will find in most KF games' inter-map corridors.

Edited: To get this new style transition working the new "relative" setting has to be checked on the map change event module. Technically this makes it so the "event" can cover every tile inside its trip zone, but it's also doubling as a "new mode" switch, since the old mode is limited to a single tile, almost like a teleportation booth. (I've thought about upgrading the old mode but it's not clear-cut that the new semantics can work with it without losing something in the process.) One other small thing is you need to rebuild the MPX file (compile/prepare it) since MapComp needs  to add some new information to it. (What tile/layer things belong too. This is information it used to emit but was disabled.) Note, it now defaults to "relative" mode.

The fog and sky and BGM is smoothly transitioned, however there's a larger flaw in this version than monsters blinking out, that is there's no provision for transitioning lighting. I have a plan for this, but it's going to have to wait until later this year, and will be a big project. In the meantime there are some creative workarounds for this if someone is really keen on using this in the next few months. What can be done is map geometry can have its diffuse lighting component overpowered either by maximizing the "emissive" component or using the new per-tile ambient system my KF2 demo is using to overpower the ambient component. At that point you will be stuck with having to fake lights by baking any lighting into your textures. But this approach can create a liminal corridor which both maps (with different global lighting) can agree on.

The solution I have planned involves more per-tile settings for most of SOM_MAP's settings. Even ones you wouldn't expect like the fog and names and music. This system should allow for little pocket regions within the maps themselves. (Edited: There will be an option to set the sky model on a per-tile basis, but it will most likely only apply to the current tile/layer and so anytime the sky is swapped out will have to be obscured from view somehow. The sky transition system blends two skies, and if it were possible to blend per-tile skies there would be a "combinatorial explosion" and tiles are square, so it would be tricky to smooth out jagged diagonals between tiles. That said I won't completely rule out tackling this one day.)

Oh, one more small limitation is you should stick to WAV files for music. The transcoding system I set up a while back isn't super efficient, but it will no doubt incur a hiccup if you try to mix it with a new nonstop map transfer. I need to roll that into the new art system cache x2mdl.dll is using so that the conversion to WAV is done one time and saved until the cache is emptied. I will do that when I add sound effects to the art system.

Another little something worth mentioning is I've tweaked the opening movies to load the first map in the background so that the game will load faster, even if players just try to cancel it ASAP... it only needs a second or two, which it can take to do the fade out effect. I'm also going to add this to the Continue screen, but I haven't yet. It will need to start loading while its Yes/No menu is shown. Also if you use do_start/start_mode=1 ("KF1" style) it will load the first map as soon as the title screens begin. Ideally SOM can be a no loading experience like the old 16-bit consoles which used ROM cartridges :713:

Edited: For the record, I want to take some time off to work on art for my KF2 project. So I'm not going to be working on SOM features for a while, and when I return to it I plan to focus on a few things that will make a new KF2 demo shine. One other odd thing that made it into this patch is a new way to turn off the menu system on the item pickup screen. This is how most of From Software's games actually work. SOM works differently IOW. But this new system still feels a little bit unfinished. Obviously it needs animation, but I wonder if it needs something else. (A sound perhaps?) Anyway, you can try it by setting it to Off in the Options menu, or pressing one of the auxiliary face buttons on the pickup screen. Even in an unfinished state it's actually good to get rid of the old Options setting, since it was totally nonsensical (and implemented completely wrong I found out) and so would otherwise be interpreted as the mark of unprofessional product.

https://www.youtube.com/watch?v=TgQUZcwd-JQ
« Last Edit: February 01, 2022, 05:34:58 PM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #1 on: March 01, 2022, 09:55:34 PM »

Important Patch

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip
http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip

This patches a serious SOM_MAP bug that might be in the current build/patch which will fail to save your MAP file if encountered. It's down to random chance, so you want to update this to not lose work.

I'm not 100% certain this needs patching, but I can't test it because my map that encountered the bug depends on the new art system, which isn't available in the current build/patch. (The bug happens when the buffer happens to end at the last tile in a string of blank tiles at just the right spot... probably because of a sanity check I recently added.)

Another thing I happened to notice today is bugs in a few places around the vertical limits I added to event triggers/activation a while ago. (I really messed up in multiple places here: 1) was a rounding mistake with negative values; 2) monsters were checking an NPC parameter to determine if they're alive or not; 3) square triggers were using the round trigger's parameters.)

enhancements

Everything seems pretty stable right now, so also in this patch is support for the last two model formats under the new art system, which I've been using lately and it's really a vast improvement for building maps with custom art.

I haven't yet added some new advanced features to these formats, but they're as good (better) than the existing x2msm/x2mhm option. One thing to look out for is untextured models are still treated as wall-to-wall CPs, except for when converting to MM3D, which I've added some code to its path to tease them apart in order to be able to convert MHM files. If you put x2mdl into MHM mode it won't do this, but the art system doesn't do that, so models either need to be textured, or make the MHM geometry slightly transparent to defeat the CP detectors. (I don't know if I can fix this really, the best that can be done is to only treat lone triangles as CPs, but sometimes low-poly art might have long triangles too... luckily MSM files should be texture mapped because they don't have materials even if you wanted to do something more like the original KF.)

Another thing it doesn't do (yet) is split models up into MSM and MHM parts. IOW the art system will just output both an MSM and an MHM file, so you don't just use one of them if you just need it for MSM or MHM for time being.

Edited: Another secret I added today (I guess SOM can hold secrets if KF does) is holding the Alt key when clicking in the workshop tools (PrtsEdit, ItemEdit, etc.) will open up the model in your 3D editing software associated with your art file's file format. I intend to make this available from outside the workshop tools too. (I also had a good idea to make A and S and E set elevation values in SOM_MAP. A for add, S for subtract, and E for equal/elevation. That way you can do it left handed! Unfortunately I still have to work on this...)
« Last Edit: March 01, 2022, 10:02:30 PM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #2 on: March 02, 2022, 02:19:13 AM »

EDITED: For 1hr the following patch was bad (I hate when this happens) because I forgot I disabled the fix to try to test it before realizing it wouldn't work with the art system absent.

Patch Part 2

http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip
http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip (see EDITED in bold)

The SomEx.dll link is because I botched the earlier patch, and I'm merging the warning I posted into this post.

As for x2mdl, I noticed a problem with my wall writing decals in my KF2 project and was convinced x2mdl was outputting bad colors in the textures somehow, and so I finally did something I've been meaning to do to optimize how x2mdl outputs textures from back in the day when they needed to be downscaled to 256x256 in 16-bit color for MDL files. (Originally it was actually using D3D to do this color conversion, but it failed once on a GPU or device that didn't support it.)

Well, it turned out at some point when I had lost the original texture, and only the TXR file was correct. After I started using the new art system that TXR was no longer used. So IOW my scramble to fix x2mdl was for nought there. However, I also realized it was still downscaling to 256x256 and MDO files actually support much larger textures than this. I'm not sure how many mipmap levels som_db allocates, but with do_mipmaps it doesn't even matter. Moratheia has 2048x2048 textures that I know of, so at least that big (which is too big) probably works without SomEx.dll.

So I've fixed that problem. The only textures in SOM's larger than this is 3 of its 4 sky textures I enlarged to 512x512 in the original release for this release. I didn't realize they were getting hacked back down to 256x256. To see that benefit it's necessary to clear out the shortcut/lnk files in the map/model folder for sky files (called sky) or delete that folder from the temporary cache.

I've also updated these files in the TOOL folder. I meant to do that earlier, but I'm glad I forgot to now.

I think x2mdl should run a little faster now when converting models. It actually was pretty optimal already when updating textures by themselves, and didn't have this 256x256 problem then (actually that's probably why I missed the sky kneecapping come to think of it) however, in practice it still generates the textures when converting the models, and usually that comes first unless the model's texture(s) already exist with shortcuts. Anyway, now it's almost as good, but still allocates a texture in video memory in case it needs to be downscaled for the MDL file... although I can fix that too come to think of it (next patch.)
« Last Edit: March 05, 2022, 04:42:35 AM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #3 on: March 02, 2022, 08:49:17 PM »

off-topic

Quote
(I also had a good idea to make A and S and E set elevation values in SOM_MAP. A for add, S for subtract, and E for equal/elevation. That way you can do it left handed! Unfortunately I still have to work on this...)

Sorry this really isn't on-topic, but just as a note... I worked on this this morning, which will be in the next patch, whenever that is.

I first looked at SOM_MAP's hotkeys. They're a little unusual, but they all work. I've never learned them I guess because they're not intuitive. I remember seeing them in the manual. Here are some of the more odd ones...

Ctrl+1 through Ctrl+3 choose between the 3 different views or tabs. This also works in SOM_PRM.

Ctrl+I opens the events menu. This one seems odd, so does putting hotkeys on the right side of the keyboard. I wonder why I of all letters? Maybe I should add a Ctrl+E option come to think of it. Maybe I is for Japanese something?

Ctrl+T opens a game (a test).

Ctrl+W opens the MPX compiler menu, but doesn't compile it.

Ctrl+P is the same as Ctrl+Click used to be, which copies the tile into the current "pen" and syncs the palette viewer.

Ctrl+Left (and Right) does horizontal paging. Up/Down don't, but I think PageUp/Down does. I guess I should add those come to think of it.

new SOM_MAP hotkeys

New ones I've added (mainly oriented to using a mouse with left hand on keyboard) are A to add 0.5 to elevation, S to subtract. E to assign elevation. Note, these 3 don't currently work with Undo, but I will probably fix that if it ever starts to annoy me...

C does same as Ctrl+P. V does same as Enter/Space but doesn't depend on the Ctrl key or CtrlLock button. I recommend using V now, but I also made Ctrl+Enter/Space work with CtrlLock off, which it hadn't previously.

Ctrl+D now deletes tiles. F and G flip the tile, equivalent to 2 turns. They're the same actually, except you can press Shift to make them do 1 turn, like Z and X. Useful to avoid moving your finger between them. Plus F and G are closer to V.

« Last Edit: March 02, 2022, 08:57:06 PM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #4 on: March 06, 2022, 11:04:43 PM »

Important Patch (Critical)

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip
http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip

Late last night I ran into yet another problem saving layer MAP files. I can't ever seem to complete that project... I really feel like at some point I need to rewrite the MAP file input/output code from scratch, but maybe this is the last time... knock on wood. (What was happening was some code I wrote that juggles some data between the files failed to give READ permissions to the file in question... which explained why some junk would appear at the end of layer MAP files... but the real problem I suppose is I added some error handling code that reported errors, and I thought it would catch my attention in debug modes, but it was incomplete. I think this was the same problem the last time I published a patch the other week... i.e. new error code causing the load/save operation to give up where previously it would've carried on.)

What's a wonder is how this didn't become a problem sooner. (This is a common refrain for bugs... really the problem bugs are the ones that get masked by shear chance instead of backfiring immediately.)

Last night I also found numerous new problems I introduced to x2mdl around textures mainly... and I chanced upon a realization that som_db.exe ignores the last frame in hard (skeletal) animations, and so I've adapted x2mdl to reflect this behavior by clipping the final frame down to 0 time... this means the last frame is shown at the end of the animation but for 0 amount of time. I.e. you won't see it unless you freeze the animation (or set the cursor) on the end. This is actually how animators usually work, so this is a benefit. The only problem is now I have to go back and regenerate all of the hard animation MM3D files from the last release a while ago. (Note, technically this topic/thread is still maintaining that release.)
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #5 on: March 08, 2022, 12:40:14 AM »

Minor usability patch (SOM_MAP)

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip

Sometime back in 2017 I noticed when SOM_MAP's palette area moves on its own (after clicking a tile) that it wipes out the information displayed in text boxes under the work area.

But what I somehow didn't notice until yesterday is this also wipes out the stored information used to place copies of the selected tile (instead it reverts to as if you clicked the palette area... forgetting things like elevation and rotation) which is obviously important for work.

So I'm going ahead and putting this out there in case it helps anyone in the meantime. An example of when this would become a problem is if you were using the tile sets packaged with SOM and mix-and-matching tiles from different sections of its KF sample project. It could also happen if the palette just happens to be on the border of the tiles you're working with. I guess it goes to show how little time I spend building game maps with SOM that I didn't notice this for more than a decade of working with it!
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #6 on: March 08, 2022, 05:03:41 AM »

Patch #2 (today)

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip

I think I may have found the reason for random crashing at start in this release in this patch. I also tested the new background loading system today on my KF2 project and found I neglected to test it on an elevation change scenario. It turned out a "shock absorber" effect caused a big glitch. (The crashing I think was caused by the [Number] based extensions in the Ex.ini file, since it is used on the background loading "thread" when it wasn't designed for sharing between threads. There may be other sources of this kind of problem, I now know what to keep an open mind about. Of course SOM isn't a multi-thread system, even though many of its tools are loaded down with "critical sections" because it's written with MFC code that seems to have produced ridiculously bloated applications. When designing the system I had to be careful about what kind of bad interactions could happen. Luckily loading code is pretty isolated.)

Edited: Oh yeah... as for background loading KF2. There's still a hiccup. I don't know if I'll be able to work it out of the demo I'm working on or not. It could very likely have nothing to do with disk access... maybe just memory cache hits. On beefier CPUs it might not hiccup. Hopefully I can profile it some more at some point, or think of some strategies to avoid extra work.
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #7 on: March 10, 2022, 11:21:49 PM »

x2mdl (dll) patch

http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip
http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip (maybe unnecessary, can't remember)

I'm still patching this leaky raft... namely I didn't test the texture downsizing path (to fit MDL's 256x256 limit) and some other nontrivial stuff I won't go into details about. One serious problem led SOM_MAP to convert non level geometry models into level geometry. (A new time optimization is it no longer generates MDL textures when there aren't animations, since it won't be outputting a MDL file. Eventually I want to omit textures from MDL files altogether since they're not shown in the game. But for now--transitioning time--the files still have textures so they can be opened up and looked at.)

Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #8 on: March 13, 2022, 10:13:30 AM »

Optional x2mdl (dll) patch (#2)

http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip
http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip (maybe slightly improved)

I got lucky with an accident that helped me to identify a pretty wild bug in x2mdl when converting the MM3D files (that format specifically) into SOM's files. It will likely crash if the MM3D file doesn't assign all of the model parts to groups (i.e. materials/smoothing groups) (which is how I found it, by accidentally not doing so with a portion of model) but the real kicker is a "loop" I wrote was using the wrong "variable" to track its progress, which could potentially be slow things down.

(I've never made a bug like this before, but basically in one standard kind of loop you have a number that is incremented until it gets to another higher number. What I did is mistype the "variables" holding the numbers so that the higher number is incremented and the other retains the value of the first item that is looped over. In that case the CPU just keeps hitting that item until the higher number "wraps around" in memory so that it becomes lower, and that can take billions of instructions (probably around 1B in this case) which is in theory a gigahertz if an instruction is one cycle. In practice though there's probably several instructions involved, although this loop was very simple.)

I'm just glad I found it. I'm sure it would turn up eventually, but I actually neglected last night to investigate it since I forgot to make a note, but luckily I remembered and went back and reproduced the crash (in SOM_MAP) to figure out what happened.
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #9 on: March 29, 2022, 07:40:59 PM »

Routine patch

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip
http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip (minor)

The x2mdl.dll patch just adds some hints to error popups and an option to subdivide MSM input. The latter is is needed to make object models that blend into the level geometry, a/k/a secret doors.

The past few days I've been trying to crack down on a bunch of recent bugs I've been neglecting debugging. That's what this patch is. I'm not going to lie, here lately it's been crashing for me a lot, usually at the start, in ways that seem random. I'm hoping all the problems are covered by this patch. I haven't had any this morning. A lot of them were really weird, and I've only recently hit upon a few reproducible cases for a few of them.

I'm pretty sure there are still unsolved problems. It's always demoralizing when a problem keeps raising its head. Most of these crashes are around new features. There was a constellation of bugs that all kind of seemed related but were not. I've fixed a least 3 like this, so at least they're being teased apart. Divide and conquer. At least this patch addresses several of them, which is better than nothing. It also fixes a number of places where recent patches broke new features. I have to be bug free before I can release a demo later in April. I'm going to have to work like a demon if that demo is going to not have a bunch of loose ends, but I intend to publish it in April, however far I get.
« Last Edit: March 29, 2022, 08:00:47 PM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #10 on: April 13, 2022, 08:36:09 AM »

Significant patch

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip

I think this patch is a pretty stable representation of the map transfer and load balancing work I've been wrestling with of late. I'd also (for some reason) been plagued lately by a random crash/corruption type bug that I think I've figured out (knock on wood) which seems to have been related to a long problem around map geometry vertex buffers. For some reason the multi-thread scenario seemed to cause it to manifest, however the root cause was (if correct) two-fold, first in some transparent map geometry code of my own, and second in that MapComp doesn't decompose the MSM geometry to fit into som_db.exe's buffers, so they can overflow and produce similar effects. I've observe this in the Moratheia 2.1 demo, and I've added temporary code to som_db.exe to excise the offending geometry until I can try to extend these buffers.

FWIW the limit on MSM geometry is 896 vertices, and 896 triangles on a per texture basis. In Moratheia 2.1 the naked trees seem to go over this limit, even though they certainly don't look like 896 vertices to me. I'm not sure what that could mean, but some of their polygons are lost with the patch I've applied. (It could be that every triangle has different normals and UVs for every vertex, requiring duplications.)

The Moratheia 2.1 demo is a big help to me because it always has unique problems to solve whenever I work on something, that would otherwise go unnoticed most likely.

Both of these problems have been around for a long time, so I really don't understand why the addition of background loading agitates them. For the MPX geometry (MSM) index buffer, following it is a huge gulf in the data section of the player's EXE image. I'm beginning to wonder why it exists. Any buffer overruns would corrupt this memory. I wonder if it's a default heap chunk or something. The vertex data goes into a D3D vertex-buffer. If it overruns it overwrites internal D3D memory.

I've checked in my source code for safe keeping. This is the first patch that can do seamless map transfers. It's probably more stable than some of the recent patches.

It adds a do_fix_trip_zone_range=no [Bugfix] extension that the Morathaeia 2.1 demo depends on because it sets its trip zones underground for some reason, and their bound object height doesn't reach to the surface. The alphafog_skyflood_constant [Detail] extension default is changed to 1.5 from 8 because I had to remove some code that capped it to 1 in the vertex shader because it made the math depend on the sky model's geometry.
« Last Edit: April 13, 2022, 08:59:09 AM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #11 on: April 14, 2022, 10:35:54 AM »

emergency patch-patch

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip

Yesterday's patch is still having thread synchronization issues I hoped had gone away with that overrun bug fix. A real problem is changing the resolution, etc. in Moratheia would lose mipmaps and sometimes freeze. I'm not sure why this doesn't happen in my KF2 project... maybe because Moratheia has some really big monster textures that take a while to process.

I hope this patch solves these problems, but it does so bluntly by wrapping the whole mipmap/colorkey/upload subroutine in a global "critical section" object (thread synchronization) which isn't ideal. But this should stop the bleeding until I can think of something else.

It also removes a "critical section" object that was kind of superstitiously preventing some routines that seemed to lock up from coinciding with the screen buffer present/flip step. I really want these problems to just go away since they stand to make using background loading a nonstarter if they don't. (At least not without further work to decouple texture loading from Direct3D.)

blue is the new black

Without writing too much about this, this patch is trying a new strategy for lighting up black pixels so they don't look like holes in the monitor. There's a do_black extension if you believe your monitor can do black gracefully. Anyway, looking at the blue night sky in my KF2 project I decided the 1,1,1 black pixels were a kind of haze and so I tried different strategies and settled on just adding 1 blue pixel value to every pixel. This has upsides and downsides. One is blue is the dimmest component and so guarantees black is the darkest color and so brightness gradients should be preserved. But also it seems to me to have some visual effects if I'm not just imagining it. I think in theory if pixels are pure red/green then just turning on 1 blue has a similar impact on the character of those pixels as filling in pure black pixels does. But it's really hard to describe... Mortheia's final dark area seems to be the most dramatic difference I've seen, where somehow it seems much darker. It's a strange effect to me because I don't really see any pixels in Moratheia that are completely lacking in blue. Although its entire game has a very green/yellow/red palette. You wouldn't think one blue would make a difference. I suppose the darker the pixel the bigger difference relative to the other components adding a blue will produce.

In my KF2 project I can't see a difference, other than desired effect. I think I'm going to add a little blue tint to its fog to help things blend a little better. It seems to help with its water horizon regardless of the black strategy.
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #12 on: April 14, 2022, 11:59:34 AM »

Repatch (EDITED) :censored:

EDITED: Okay, I think I had a miserable/crisis morning of this. I neglected to see if the fix from a couple hours ago still worked to remove loading hiccups as advertised and it did not.

So I had to just try everything I could think of to figure why the critical-section set up I had wasn't working. The first thing I found was a typo I made and I really thought that explained it all, but there were still strange further troubles to work through. It seems I needed to extend the guard region and what I had written wasn't exactly a correct double-locking pattern. I should've just relented and looked at some old code of mine. I've done this before.

It wasn't fun but I'm glad for the Moratheia project as ever for helping me discover unique problems.

EDITED: I really hope this stuff is finally out of the woods... knock on wood. This work can be great fun but it can be frustrating too. This latest around of work has been beset with bugs (many seemingly unrelated) and I've been constantly on edge about whether it's going to work out in the end or not.
« Last Edit: April 14, 2022, 12:10:45 PM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #13 on: April 21, 2022, 08:52:58 PM »

Replatch/Reupload (upload failure) patch

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip

For the record, I just found out (by chance) I must have (somehow) botched the last patch (7 days ago) most likely by forgetting to replace the DLL file inside the ZIP file before uploading, I'm not sure??? Maybe I imagined I uploaded the patch files because the dates on my host were the same as the ones in my TOOL folder. Maybe Windows failed to update the DLL file in the ZIP file, I don't know.

while I'm here posting (off-topic news)

The UV animation project I'm doing these days is coming along as fast as it can. It just takes a long time. I wonder sometimes if I'm going about things the wrong way when something takes too long. I found out I messed up this patch (human fallibility still gets to me in this area) because of a problem in the saved (Slime) model file... something I forgot needed to be updated with the addition of new features (I've stuck a note in the source code for future features.) Hopefully I'll have a render in the modeling software going before day's end. That will mark the end of work on the editor side.
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts

Holey Moley

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • twitter.com/m__7761
look out honey, 'cause I'm using technology
Holey Moley says,
« Reply #14 on: April 29, 2022, 09:07:30 PM »

Important Patch

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.6.zip
http://csv.swordofmoonlight.net/x2mdl.dll/1.0.0.2.zip

[SVN Update is required for new sky model animations.]

EDITED: For the record, I had to make one more fix shortly after writing this post (I should've test it) so if the sky is stuck, please try to redownload/reapply the SomEx.dll patch.

This patch fixes a problem that writes corrupted item data to save files. What happened is some code that tries to eliminate removed items that were edited so the save file doesn't crash ended up running before the save file is loaded (I probably misplaced it when reorganizing code for changing maps in this release.)

Somehow another problem slipped through that fails to play sound effects when using the new "standby" preloading system. This was owing to some checks that ensured some mmio APIs (Win32 multimedia subsystem) were running on the main thread, which is not the case when preloading. I honestly don't know how this didn't raise its head months ago.

As for x2mdl, it has a new system for directly specifying texture map animations in the MDO format. The only way to use this is with the MM3D editor (https://github.com/mick-p1982/mm3d/releases/tag/win32-dec-2021) (see Model->Utilities) and is limited to 1 key-frame of "translation" data only. This isn't limited to one texture or even specified on a per-texture basis. It uses groups of triangles instead, which can all be animated independently. Items can now be animated like this, as can NPCs, including monsters.

This reminds me, I have to now quickly update the SVN repository (SVN Update) because with this change sky models will no longer be animated so artists can manually control this. I almost forget, since I was putting fixes ahead of features. Give me a few minutes to do this.

Note, if a project loads an old MDO file the old sky animation system will kick in, however there isn't an opt out option for this, so if using the new art system it's required to specify the sky animation with the art file. The reason for this is not specifying an animation is meaningful since it tells the player a stationary sky is wanted.

Animation info: For those curious, to reproduce SOM's sky animations, set V to -0.025 (40 seconds) for a 1 frame animation at 1 FPS. For objects, set V to -1 (1 second) I believe. Of course you can set these to any values you please now. These values are negative because OpenGL and Direct3D differ on this. Note, the editor plays back these animations when playing animations (via a toggle) but the skies don't have animations, so to see it in action it might be necessary to make a dummy animation.

I'm going to enable using MDL files for skies sometime. If so it will just loop like a looping object until something can be done to control it (MDL animations move the geometry, whereas texture animations just move/cycle the texture map.) I'd like to explore more animation options in the future, however I chose to keep with tradition for the moment.
« Last Edit: April 29, 2022, 09:45:40 PM by Holy Diver »
Formerly "Holy Diver" ("Holy") [Holy will be back as soon as I'm back to full form]

Holey Moley has 2726 posts
Pages: [1] 2