simple machines forum

Please login or register.

Login with username, password and session length
 

News:

Remember to make your own backup of posts before submitting.

 
 
Pages: 1 [2]

Author Topic: MDO+MDL & MSM+MDL (2021)  (Read 2450 times)

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #15 on: October 09, 2021, 01:56:00 PM »

follow-up

I was able to set up ambient/directional lighting and transparency/sorting today, however one problem that arises from lighting is SOM_MAP's little scene viewer is most likely surrounded by a vast white desktop, so its little dark box appears much darker than it really is to your eyes as they cope with the contrast difference (and probably your monitor is playing some contrast tricks too) so I can't publish it as is.

With my KF2 project it's not so bad, but dull, whereas with the little test project (someone else made) in the other day's post it has a pretty standard even darker "dungeon" atmosphere that feels too dark in SOM_MAP.

I think my plan is to use the mouse wheel to be able to raise the ambient setting artificially, assuming everybody has a wheel, which is currently how the lighting level is adjusted in SOM_PRM. I would like to provide a fallback for no wheel (or even use the wheel to zoom someday) but I'm not having any bright ideas.

Eventually I intend to try to have right-clicking the viewer pull up the lighting window so you can adjust the lights with a 3D view handy. I'll see if that makes the cut. Right-clicking in SOM_PRM currently pulls up a color selection menu for the background fill, so it's not that far off. (Ultimately I intend to add a right-click menu that will instead offer these options and a way to open the Exselector companion tool. It's going to pose a problem for turning tiles in SOM_MAP, but that's the only place right-click is used. On the other hand it will prevent accidental turnings by default, which is probably a good thing.)

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #16 on: October 20, 2021, 02:02:16 AM »

update

Off-topic: On SOM_MAP I did a second demo update that improved mouse input and added graphics to the other two map screens that had just used flat squares.

On-topic: I have ObjEdit, EneEdit and NpcEdit working with MDL+MDO. It was kind of a chore because I'm trying to have them use the same code som_db.exe uses since their MDL code is identical to it, except every program has their code at different places, so everything has to collated in triplicate to do this. I'd like to find a more organized approach, or just hollow them out and implement them from scratch, but that seems like overkill for now, especially since I'm not animating them, so I've just done what's expedient if drudgework. I'm hoping I will get some inspiration about what to do after everything is further along.

Now that everything can display MDL+MDO the next step is to set up the automatic conversion process and finally convert everything. I'm a little concerned that my MM3D project is still in demo status and only hosting debug builds, but it's functional. I will be preparing a new release for it. At minimum I've added a blend modifier to its material system to match SOM and Assimp's.

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #17 on: October 24, 2021, 12:04:36 AM »

update

I've been taking it slow to ease into this next stage of this project. It's very scary territory. I've done some preliminary work on x2mdl to be able to load it as a DLL and have it do cpgen's job too.

I've been thinking in terms of how to communicate the new level of indirection to developers, without instructing them (I believe SOM should be something you can find your way around without a tutorial) what to do but to also have something flexible enough that you can manage it manually when you need to (I'm especially thinking about how to make standalone games that only have the converted data and don't have source art of caching/update components) but to also be simple enough that it's not a lot of work to maintain or a major interface presence.

And I've been thinking about what the hell to call this system, as in how to label buttons, etc. For authors who don't understand the technical breakdown of game development infrastructure it's hard to explain why there's a need for a "tool chain" in the first place, but this is not an intuitive jargon I think either. The goal is to make the "tool chain" invisible and that's actually what "tool chain" means too usually.

How you experience it by default is when the game starts up the tool chain needs to kick in if any of the art is modified going by "modification" timestamps in the files, i.e. Windows Explorer's "Properties" stats and sorting options. When that happens there will probably be a decent delay since it's going to be doing the heavy duty task of converting the models that you normally do manually with x2mdo for example, but at least you don't have to do it manually anymore.

That's the basic scenario. Except if you want to make a standalone game or a demo for example, then there should be a way to do this for every model in advance, because otherwise you could only trigger the system by playing the game, which isn't always practical. And this adds an extra step to preparing a game/demo because you'll really want to copy your DATA folder out of the cache, which will be in your %TEMP% files by default.

So after giving it thought, I decided that to have such a manual option the best place to put it would be in SOM_MAP's "Maps..." screen, since that way you can choose from the maps you want to rebuild your cache for. That would be nice if you want to publish a demo with only a small selection of maps. That way you can be sure you're only including DATA files for the things that the maps use.

I decided I didn't want to make this a screen because I think SOM_PRM would benefit from having its own version of this, and even SOM_SYS perhaps. But I'm less inclined to put one in SOM_SYS because I think it only makes sense on the first screen with the character setups. Anyway I wouldn't want to have a copy of a complex screen in both tools. So I think just a push button is best. But I think it can open up a standard directory selection screen so you can choose a custom directory, but it will open up to the %TEMP% directory. Actually SOM files already have a TEMP variable for this.

Opening up to a directory will double as a cue to what's going on, since that's where the files will go to. But there won't be any other instructions than this.

Finally as for what to call it? I think I've settled on calling it "Artifact" as a kind of jargon. Other names could be something like Cache or Data but I want to focus on the idea of source art, which also helps distinguish it from map models. So I think my goal is to add an Artifact button to the bottom of SOM_PRM that will rebuild the cache for the selected items on each tab in SOM_PRM. And put another one in SOM_MAP that will rebuild items used by the selected maps. This way if you were to need to prepare an entire game it should be enough to select all the maps and rebuild them (maybe to your game's folder) and then go into SOM_PRM and select all the "items" to get the items that aren't on any maps. That should cover everything. Also if there is no selection it will rebuild everything. So in that scenario selection is optional.

Calling these "artifacts" now will come in handy later on since that's what I plan to call it when there's more of an online sharing framework for crowdsourcing art. I call them digital artifacts generically even in other projects I have outside of SOM. I think that's an especially apt description when you get into restoring old games, since that's literally working with archeological excavations which is where the word "artifact" is used most exclusively.

I'm not sure how long this will take to wrap up. It's not a very fun thing to work on. I just hope it goes by fast. Like I've already said the thought of changing how SOM distributes its art files is nerve-racking. I have to be sure I get it right since if this cache system breaks anything it will make SOM next to useless until fixed, and there's potential for data loss to go unnoticed.

I probably won't report anything else until there's a working system and I switch over to verifying as much as I can and converting everything that is eligible. Only MDO and MDL files will be considered in this round. When there's nothing left to do I will release it. This will be the main focus of the next release since the other things I've been working on lately aren't really something you can use out of the box. Plus the enhancements for SOM_MAP from last week.

Technically shields will be in that release but they're still kind of immature and I don't have animations for the shields that come with SOM. I've been a little risk averse to modifying SOM's built-in art work. I also don't want to add shield animations without adding heavy attack animations to its weapons. I'm not sure I'm qualified to do animation work at that level since I'm not sure I'm experienced enough or a good enough animator.

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #18 on: October 26, 2021, 08:01:08 AM »

Today I've been writing some more formal helper code (lib/swordofmoonlight.h) for the MPX model format, that's kind of the last one to be fully explored since it's the combined level data model.

I realized the SOM_MAP function I described the other day should be using the MPX file and not the MAP file in order to mirror what the game will do when it loads. It adds a wrinkle of maybe needing to build the MPX file first.

This is actually good timing because next is streaming maps which will require working with the MPX files, so it's probably good to ease into it a little now to break up the monotony. Sometimes this stuff is exciting but sometimes it's monotonous too.

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #19 on: October 31, 2021, 10:07:28 AM »

EDITED: So it turns out there's a lot more models embedded in a map when you factor in all of the possible sources, it gets to the point you gotta simulate a game pretty much.

So I've been having to map out most of the remaining file formats as part of this process. It includes all of the SFX generated by monsters/NPCs, all of the items they "drop" and eventually all of things generated by events, including everything sold in shops. I intend to cover everything. I've already done the SOM_PRM ones, except for shops, and SFX and items on maps, except for events. It's been a good motivator to roll up my sleeves and finally sit down with these formats that I usually just spot fix as necessary since it's too much to think about tacking all at once. I'm not filling them out fully, but now whenever I come across something new it will be easy to fill it in.

Changing subjects I've changed the script editor buttons to say "Script" and ended up going with "Convert" for the new art/artifact system, since I figure that's the easy way to think about it and what seemed to look good. In SOM_MAP I'm changing to using check-boxes for everything and single button that launches MapComp (so the other buttons are now check-boxes) and it looks a little weird. I had to do this because I wanted to use descriptive text that would not fit on a button label. SOM_MAP says "Convert Map Content" instead of just "Convert" so it's more clear about what it's doing. It also runs MapComp since the MPX file needs to be up-to-date.

EDITED: The main reason it "looks weird" is the main button needs to be in the middle so it's centered on the mouse, so these check-boxes have to be arranged around it somehow. I've also changed the group label to say Background Task (MapComp.exe) so it's very explicit what it's doing and matches the progress window. The progress windows now have new messages/titles for the art processing/caching stages. SOM_MAP says Background Task (x2mdl.dll) in stage two. Stage one is indexing everything by loading the MPX files one by one. The English buttons says Execute... now.

I've written code for the bulk of the difficult logic for checking timestamps and talking to x2mdl. It's getting to the point where it will all have to come together soon. I've been spending fewer hours working on it lately. This is a pretty boring task to work on. Sometimes spending too many hours on boring tasks can lead to burn out. A boring task just has a lot of monotonous labor that's not so simple you can do it on autopilot but doesn't have anything interesting going for it either, and nothing to look at and nothing to do for days on end. But it's going to be a big addition for SOM so I'm pretty excited to see it through. The new source code will be in som.art.cpp, and it's a decent amount of code. The default cache will be %TEMP%/Swordofmoonlight.net/art. It's pretty cool to have the concept of "art" built into it. It's becoming aware of the art side of project development.
« Last Edit: October 31, 2021, 10:14:01 AM by Holy Diver »

Holy Diver has 2627 posts

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #20 on: November 08, 2021, 02:03:32 PM »

Update

Today after loads of work (and staying up too late) the first model was generated/loaded from the timestamped cache. TBH I don't know why it takes so much work to coordinate this stuff. But it's a milestone at least.

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #21 on: November 14, 2021, 06:13:22 AM »

Update

This is proving very arduous, high endurance, low fun challenge. I'm trying to muscle through it. Yesterday I did so many different things my head was spinning. And in the end I found myself trying to fix one of the last problem MDL files after losing all my familiarity with the x2mdl conversion code I worked out (I want to say) last year. It was the candle knight (Death Fighter) and its unique problem was it doesn't have an abdomen. I really felt sick at my stomach thinking about how I could solve this problem (get to the bottom of it) now that I couldn't solve back when I was fully entrenched (developing) this difficult code. But the pressure of knowing I had to figure it out now got me to stick with it, and I managed to come through. I'm glad I managed it as an end of a long day task... instead of it becoming a full day task in its own right. (This code is really hanging by a thread, but at least it only has to cover the original models that go with SOM.)

I'm at the point of replacing the runtime models with art models... which is how I got stuck on the candle knight. An unexpected benefit is the shadow (kage.mdl) model will now be able to use a proper texture instead of 15-bit color embedded in the MDL file, so it will make that shadow much smoother. (Fixing this shadow was a key feature of the recently defunct "Keygen" system, so I'm glad to have a new solution for it.)

There's some issue around lighting surface-normals. The MM3D editing software doesn't keep track of normals, but it does have a facility for storing them in the format. Since the MM3D files need to be convertible to runtime files they may never actually see the MM3D software, so they need to have some fallback normals. Another problem is MM3D will forget the normals, and it uses smoothing groups instead, so this is a real fault line where the normals may get messed up... not that the existing normals are ideal or anything. But it's hard to choose some one-size-fits-all smoothing group settings. I'm thinking of adding a "preserve normals" setting to my MM3D fork so that it won't override the imported ones unless you manually disable it. (An import normals feature would be good too, to repair them. Edited: Another thing I hope to work on is some code for removing redundant key-frames since it's really hard to edit animations without sparse key-frames.)

Looking back I can't believe how much time I've invested in this and all of the groundwork up to this point. I'm constantly surprised how much work it is to bring to life a 3D art system[1] like this. I wonder if it's the same difficulty in every case or if there's just something about SOM that makes it very handicapped in this regard. I know I go the distance to do everything extremely high quality, so it's done right the first time.

[1]This time I mean SOM in general, not just this new "art" system.
« Last Edit: November 14, 2021, 08:02:08 AM by Holy Diver »

Holy Diver has 2627 posts

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #22 on: November 25, 2021, 02:54:33 PM »

follow-up

I've been setback a few days ever since I started working with the player component and keep running into new issues mostly revolving around animation and little quirks around SOM's interpretation of Sony's TMD file format.

The animation troubles took me by surprise because I hadn't had any issues with my KFII project that was already using the new features. The reason this project never ends is it has to be battle tested against every model file SOM has, that is a lot.

Yesterday I worked a lot with the Reaper (tree) enemy model. I was surprised it has a walking animation because it's a tree. I wonder if its walk animation should be reassigned to the previously unknown chasing animation since it's really weird for a tree to be on the prowl by default like a regular monster, but it would make sense for it to chase you once provoked.

Its model seems to reveal a problem with the new soft animation path in the cpgen.exe tool for making the CP files. I guess it must be a decoding problem. I have to pore over it today. I'm not looking forward to it, it was already a headache last night to diagnose all of the problems it was having, not limited to cpgen (I really didn't expect cpgen to be a source of problems, but the soft animation code is still relatively untested. I'm just surprised my KFII project didn't turn up anything.)

There's another problem with most of the objects' animations that I think I understand as because som_db.exe discards the repositioning component in the root of the skeleton. I think instead the root CP should be subtracted from it. The soft animations already do this. So I'm hopeful making this change will make this a nonissue in the future. Without such a change it's necessary to add a dummy CP to the base of every model, i.e. one that doesn't have any geometry underneath it.

When I got sidetracked I was working on the feature that lets the NPCs use an alternative "texture" (i.e. like a "palette swap") that's an example of all the little bases that have to be covered. I'm pretty sure SOM_MAP doesn't reflect this feature, so I have to add it to SOM_MAP before I'm done.

I've still yet to test the SOM_MAP and SOM_PRM features that generate a standalone cache that I started out this work with. I'm not sure what else I've so far not finished. The wait when starting the player if a bunch of models have to be converted is a lot. I think I have to work on a visualization something to communicate what's happening. I'm hoping I can repurpose Windows' interactive file-transfer feature for this, since it has a nice presentation, but I have to look into if it's customizable or not... i.e. if it's only good for basic file copy/move operations, or if it can implement some custom logic instead, like conversion. There's also a problem that it's possible for conversions to fail for any reason (a major one being if the program is perceived as being stuck and is forcibly killed) so I wonder if there should be improved facilities to deal with this kind of contingency. I would also like a way to do a bulk conversion right after installing SOM but I can't think of anything that would feel natural for this right now. It would be nice to not have to wait on each model to be converted on first encountering it. I'm sure that eventually something like this will come to pass.[1]

Edited: [1]This turns out to not be so bad since the slowness of the conversion is mainly owing to debugging settings. (Specifically--probably--C++'s STL doing checks every time a subscript is dereferenced.)
« Last Edit: December 28, 2021, 01:43:30 AM by Holy Diver »

Holy Diver has 2627 posts

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #23 on: November 28, 2021, 03:12:39 AM »

I got out of the woods last night with the player components animation problem. I was in town today getting a COVID booster. Didn't get much else done. Another thing, it turns out MapComp has to read the lamp model files to set the light source's origin. Just another base to cover I hadn't thought about. I really can't wait to be done with this.

In other news...

I learned this morning (not from local media) my home town (same "town" I visited today) scored 0 (along with just 7 other US cities) on this (https://www.hrc.org/resources/municipalities/search?q=72401) thing (sorry the main page on this website is one of those unnavigable/trying single page mobile sites.) It's really hard to believe since it's a university town with a big international student body and by no means the most strident city in our state... but I guess laws are laws.

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #24 on: December 07, 2021, 05:59:25 AM »

progress update

Developing this release cycle has been very hard on me... it seems like it's just been unlimited subtasks... I don't understand it. Now there's just one last thing to do I think, namely to investigate some troubles with converting some SFX models and add code to output scaling animation data from x2mdl.

In other news, lately I've been feeling kind of weird, like the WWW seems to be generating an endless array of media on sites like YouTube and I just don't care about anything much in the world media wise right now. Working on SOM has been a refuge while I go through this. It may be my new normal, I'm not sure. I'm just starting to feel like I don't care about anything in the world, not in detached or dysfunctional way... it just doesn't have much meaning for me, it's too much of too little all blending into nothing that seems to matter[1]. I don't know why SOM matters for that matter, but the idea of making its little ghosts come alive in the computer does seem to hold some magic for me. There's something very cool about that in the abstract that continues to feel novel. I've been trying to read more. Reading is something I've tried to avoid for a while since it's so time intensive, but having lost interest in shallower things it seems like it could be good timing. Something about the current world seems stuck on a loop and oversaturated with media that just never turns the page.

[1] Edited: A big part of this is definitely age related. I don't want to give the impression that nothing should matter, but at a certain age (some time ago for me) it does all seem to blend into an opaque soup you may come to regard with a degree of overstayed sameness. (I'm probably lucky to have SOM instead of midlife crises.)
« Last Edit: December 28, 2021, 01:45:32 AM by Holy Diver »

Holy Diver has 2627 posts

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #25 on: December 13, 2021, 11:23:44 AM »

RELEASE NOTICE

Everything is published for the new release (1.2.3.6) except for a formal announcement. I ended up discovering some good problems I missed with the SfxEdit tool and had to work on the update tool to hide the splash screen so the updater isn't stuck hidden behind it (I imagine some people have gotten stuck here but didn't bother to report it... although it may be a new scenario to update a component in the middle of use.)

I couldn't come up with a plan for communicating that models are being converted on the fly, which can cause a game load screen to go for a long time (like if you delete your %TEMP% folder for example where the files are stored) so the splash screen just hanging around for longer than normal is the only indicator (unless x2mdl.dll has to be updated, then there's none afterward.)

For tools it's not a problem because only ever one or a few things need to be converted unless you're doing a conversion job, which has a progress bar indicator. Otherwise just the normal busy icon pops up to indicate the delay.

This will seem annoying at first, but it only happens the first time you look at a model until it's modified again. (Edited: It's actually not a long wait with the regular "release" build. I was using a "debug" build.)

EDITED: BTW I've updated all of the files in the TOOL folder so that everything is up-to-date on initial install, and the only available version of SomEx.dll offered is the new version since the others can't work with the new system. Only problem is those are Japanese tools, so for English you need the new language package as always to get the new buttons: http://www.swordofmoonlight.net/text/English/Neutral.zip (I'm sorry I still haven't been able to automate this outside the first time setting the language. You can also use the setup tool to update this file.)
« Last Edit: December 28, 2021, 01:49:05 AM by Holy Diver »

Holy Diver has 2627 posts
Pages: 1 [2]