simple machines forum

Please login or register.

Login with username, password and session length
 

News:

Remember to make your own backup of posts before submitting.

 
 

Author Topic: Modern MDL convention (document)  (Read 484 times)

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« on: July 15, 2020, 01:10:55 AM »

This might go here (http://www.swordofmoonlight.net/bbs2/index.php?topic=311.0) but I want to write this down in a quasi formal fashion in the form of a document. At least here I can refer to it until it finds a home.

I'm trying to be able to edit MDL animations that use the rigid (skeleton) animation format but there is a problem in its representation that comes down to a matter of interpretation.

How som_db.exe advances through these animations in the beginning results in omitting the first two frames of the animations. I don't know if that's intentional in all cases, it could suggest that the format didn't originate with From Software's team and the implementation of the player is flawed.

Skipping the first frame makes sense since the data isn't yet positioned into an initial pose suitable for the animation. I can't rightly think of a reason to skip two. I think it's an oversight likely, however the artists would have worked around this since som_db.exe initializes doors to first appear on the start of the third frame, and the start of their second frame is unsuited for this. (Note, if you ignored the first frame, since it's a dummy frame, you could call these two the second and first frames respectively.)

The first frame of the first animation is special in that it includes an extra byte that relates the body parts within a hierarchy of coordinate systems surrounding each part. That's what's called a "skeleton".

This is a very weird choice of design, but the code actually checks for this (00 frame) as it advances through every frame, so it's safe to say it's a deliberate decision.

Although every animation's first frame defines a kind of skeleton, it's only the first animation's that possesses the hierarchy byte. This again is odd. It suggests that the animations are self contained (as if each was its own file at some stage) and yet still the hierarchy itself is not.

Because the body parts aren't all situated at 0,0,0 (a pivot) there are auxiliary animation channels that contain inverse transformations that move the parts into the 0,0,0 space. This requires extra work and doubles the number of animation channels, even though I suspect these channels are fixed by the first frame.

All of these quirks combine to make a situation where interoperability with these files is treacherous. The format is too loosely defined for its own good. Note that none of this applies to the alternative soft animation format. Making things more confusing the soft format only omits the first frame.

Ever since I started looking more deeply at this format these things have weighed on me. The x2mdl tool I've shared that was used on some projects like Moratheia is actually very naive by comparison and I can see where a lot of issues from the past arise now.

My plan as it stands now for dealing with this--if only so the original data files can be maintained--is to establish a new convention that I will describe now.

Firstly, in order to have a new convention it's necessary to be able to distinguish a new convention MDL file from an old style one. I'm not going to mark the new files, but the new files will be distinguished by not having those auxiliary animation channels. They will have one channel per part.

I partly suspect that the special frame with the extra hierarchy bytes is not meant to be part of the first animation. I've modified som_db.exe to only skip two frames on the first animation. It's probably only a coincidence but I haven't seen any problems arise from this modification and it seems to work better. (I've noticed the skeleton monster's attack animation ends badly but it does so without the change too, hence why editing these models is worthwhile.)

So this new convention draws a distinction between the animations and this special 00 frame, but it's still embedded within the first animation since doing otherwise would be incompatible.

This 00 frame should just establish a skeleton like most modeling packages possess. Indeed the problem is the old MDL files produce unnatural skeletons, so you can put the animations into most animation systems but the skeletons won't look recognizable.

Fortunately it seems like it's straightforward to interpret the auxiliary frames as if they're an inverted skeleton. Really they're just inverse transforms to 0,0,0 but that's basically a skeleton. I don't think they include rotation or scaling data but they might, and the new skeleton can do so. This will let them be converted to new convention files.

The inverse transforms will be rolled into the first frames of the animations. This eliminates the extra channels but means the data may be more chaotic (I'm not sure) and possibly more lossy. These inverses will be derived from the skeleton in the 00 frame, since that's standard practice. That means to undo this inversion the channel data needs to be "multiplied" (linear algebra) by the skeleton transforms (the inverse of the inverse is the original.)

This should produce models that can more or less be round-tripped through editors, although it's possible each time there will be a little bit lost because MDL files roughly round off to the millimeter (really 1/1024 m) What's more important is a least the models can be improved on.
« Last Edit: July 15, 2020, 08:38:01 PM by Holy Diver »

Holy Diver has 2452 posts

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #1 on: July 15, 2020, 08:33:53 PM »

Unfortunately this is a major job. Here's the first screen of a MDL with a reconstructed skeleton. Half the channels/nodes are removed, and the hand and sword are fused into one node. (Edited: these are good things in case you're wondering.)

This outputs a model that uses the same conventions as the Assimp library, so it's transform stack can be displayed a meaningful skeleton, as you can tell from the red lines.

I think this part was the scariest. The rest should be downhill but there's still a lot of work to do. This is just phase 1 of 3. It took me a solid day and I expect the rest to take as long too.

A problem the x2mdl tool faces is it depends on old code that makes use of the Assimp library. I can't easily publish its source code because of this. Half the code that makes it work is custom code inside the Assimp library code.

I have a project from years ago that is a rewrite of the core components of the Assimp library. It's just not as developed to the point it's useful for my day to day work. It doesn't have a viewer facility like this, and it only has limited import support for SOM's formats.

So before I can really publish an x2mdl project--or whatever its successor will be--I will have to finish that program. I mean, I developed it so that one day I could publish model tools easily.
« Last Edit: July 17, 2020, 08:20:40 AM by Holy Diver »

Holy Diver has 2452 posts

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #2 on: July 16, 2020, 08:15:23 AM »

Man, this kicked my butt a lot more today. Turns out I'm still in phase 1! A number of MDL files were more complicated and the pelvis part in the screenshot actually has some animation data in it. The skeleton (monster)  has lots of animation in its non joint pieces, but this stuff only comes into play in the death animations so far.

I programmed a kludge for the pelvis, but I may try something different to include the skeleton, or just give up on its death animation. Remodel it or something. The original has a problem in its attack animation, so it'd be nice to remodel it some.

The pelvis pivot is actually on the floor, right on top of the base (CP) pivot, which is not so great since it's hard to tell them apart if they coincide. I may just fuse the others together, I don't know, but the "pelvis" is special because there's an implicit CP that needs to move independently of the body. I tried moving the pelvis skeleton between the legs, which works except in the death animation, it can't be automated. (Artists may want to hand edit these.)

It's annoying how time consuming this is. I might just give up and go with the arm and door models, since that's what I was trying to do in the first place anyway, and they seem to work alright. Anyway, this is a huge, 2wk long maybe, project. Not great.
« Last Edit: July 16, 2020, 07:52:26 PM by Holy Diver »

Holy Diver has 2452 posts

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #3 on: July 18, 2020, 08:06:24 PM »

After a full week of butt kick hell I present to you, voila:



This animation in particular proved the most difficult since it works like the skeleton's death outro. It may be more complicated by the fact that this model has a rotation in its initial pose. There may be some more wrinkles among the MDL files but I think if so they'll be few and minor trouble to fix. But I'm not adding code for hypothetical files since this work is purely to salvage the original MDL files. It's been multiple days of work, but it doesn't matter, since I'm committed to SOM. I do regret the time it's taken off my KFII project's deadline and that I'm trying to publish a new demo.

This just completes phase 1 of this project. Hopefully the rest is not as time consuming and frustrating. I should add that it's a blessing that these models are in this convoluted format (notice how crazy the transform stack is and how many unnecessary nodes there are) since they could've been in a format that could have also been disorganized but also impossible to automatically extract a meaningful skeleton from, in which case it could've been a great loss. So, in a way this is another bright spot along SOM's golden path. The MDL format isn't an editing format. In run-time formats it's common to discard unnecessary information, but usually these days skeletal model formats are organized in terms of an explicit skeleton.

Going from how these files work it seems like the tool they were developed in could represent a skeleton but it could also "parent" skeletons or objects to objects instead of joints in the skeleton. That produces these weird spiky results. I don't know if it's just how the software works or if it represents the habit of SOM's artist(s) or some post-processing software.
« Last Edit: July 18, 2020, 08:42:34 PM by Holy Diver »

Holy Diver has 2452 posts

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #4 on: July 23, 2020, 11:04:55 AM »


Signs of progress in the editing software (attached). I added a feature to change how some of the "bones" are displayed. There was a line mode, but now it's possible to mix "bones" with "lines" in case the line mode is more appropriate. Having giant bones dragging on the ground between this guys legs was a bit disturbing. (It has to be set up by hand since the conversion tool isn't able to make that call.)
« Last Edit: July 23, 2020, 11:21:12 AM by Holy Diver »

Holy Diver has 2452 posts

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #5 on: August 01, 2020, 04:33:36 PM »

I thought I was almost done here until yesterday I realized I still had to ensure the new MDL files are in this "modern" configuration, and it turned out to be a bit of work that I was just able to see-through today.

I also to the opportunity to find out about some of the last details of the format to see if it was possible to mark the new files in some way. I ended up setting the 4th bit in the first byte of the file to mark it. That makes it easier to not attempt to convert them as if they're in the old-fashioned style.

I also found out that there is possibly some extra data in the rigid animation data that almost looks like a way to extend the format. But I tried to use it to make extra space in the file and it didn't fully work. That means either some part of som_db.exe doesn't respect this extensibility feature, or it could be that there is actually supposed to be something in that extended region and I just don't know what goes there. The program didn't crash so I can't get an easy hint at what code is failing.

Luckily setting the 4th bit seems safe. Interestingly the first 2 bits are tested for rigid animations, instead of just the first bit. That suggests maybe there's a third mode. When I set the second bit the animation plays but the geometry attached to the skeleton is garbled like a mess of triangles.

I had hoped that extracting the skeleton in the style would be simple, but it seems like it's pretty difficult to do. But the process is identical to what x2mdl already does before outputting the MDL file, just in reverse. So it's not disproportional, but it's not cleverly simple either. In order to prepare the data it's necessary to convert it all into global space, just as if you were playing back the animation, and then apply the skeleton to that data, and then convert it back into local space. The algorithm for doing that is pretty involved, but it's the price to pay for the files being backward compatible.

I've chosen to not store the skeleton inverted, but if I did it could be simpler. I just think that the skeleton section should be straightforward and recognizable as a skeleton and you can reason that if the model is stored in a way that can displayed without animating it (as SOM_MAP does) that somehow the animation can be mapped back onto the skeleton.

Converting a MDL file into another format should be a last ditch effort. It's not meant to be an art format. Models should be saved in a master format.

I also developed a back door for repairing any files made with old versions of x2mdl. Until this year my understanding of the MDL format was incomplete.

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #6 on: September 17, 2020, 07:13:58 PM »

I figured out the significance of the extra frames, the hard way, last night :xd:

It does confirm it would be unnecessary on all but the first animation.

I spent most of the day cooking, but on such days I always try to do a half of day of regular work at the end of the day and always end up going overboard because I forget I spent half the day cooking!

So I was really exhausted at the end of the day and I thought I was losing my mind because the MDL I was working on was having its vertices fused so it looked like something out of a Cronenberg film. I thought it was a bug in my conversion software, so I kept going over it without making any progress, until finally right before bed I converted the output back into my MM3D format to see what it looked like and it was fine.

So, my usual knack for blind spots caught me, it was som_db.exe (GAME.EXE) that was the source of the trouble, and it stands to reason that the game would have to process the MDL files at some stage since they're not in a Direct3D friendly format. It just never really occurred to me, even though I think I've seen this problem before. It's even possible I saw it only a little while ago working on the exact same model, but if so my memory is more short-lived than I usually think of it.

So the weird extra frame exists only as a strategy to prevent this fusion from occurring, and it's honestly a daffy strategy for an animated format like MDL since it's already indexed. The player should be trusting the existing indexing.

Today I have to locate this code and correct it to go by the vertex index instead of the position. It should be straightforward unless it's trying to create an index that covers all sections of the model at once. I don't think that's possible, but it could explain why to go by position instead of index.

It so happens I need to find the code that omits control points since for some weird reason it only omits the standard red control points, and the cyan CP (presumably) for soft models. But for some reason it doesn't omit a cyan CP for hard models! It seems like code would be much simpler if it omitted all non-texture-mapped primitives, instead of going by color!

And I need to find the code that converts the PlayStation like UV coordinates to regular ones so I can perfect this in my x2mdl tools. These three things ought to be nearby in the same code region, which for some reason I've never looked at before.

Holy Diver

  • Website System
  • Administrator
  • *****
  • Offline Offline
    • MaleView Profile
    • About/Support Me/Sword of Moonlight
look out honey, 'cause I'm using technology
Holy Diver says,
« Reply #7 on: September 20, 2020, 11:02:31 AM »

So, the texture coordinates (UVs) in MDL files are just calculated by dividing the 8-bit values by the texture's size. This means it's not possible to go all the way to the edge of the texture.

That's how the PlayStation works, but I don't think SOM centers the coordinates in the middle of the pixels that would be correct. The PS doesn't care because it doesn't have a filter.

I can be pretty certain because I was able to make the secret doors and compartments in my KF2 project cover the texture to match the walls, but I found out in SOM_MAP that the polygons that did that were white, but that's because I changed SOM_MAP to accept untextured models when I was starting out with my project. So what was really happening is these polygons were failing to be assigned a texture, except the default behavior in that case is to assign them to the first texture (without the change I made) so this wouldn't work with two textures.

Also the reason it worked at all is my textures were out-of-bounds, which is possible if the textures are smaller than 256 and with this bug behavior falling back on one (the first) texture.

I thought it could be possible to rotate the polygon UV map since only the first corners are tested, which I think would work well in one dimension, except for the 256 limit.

Since KF2 is based on PlayStation it works okay with MDL but I'd like to prefer MDO in the future, but don't want to make a big project of it just yet, so I'm thinking about using 7 bits in the "TSB" fields to store if each UV component goes to 1.0 in the meantime. There's not room for 4-sided polygons, but they should have one corner that can't be on the edge in both dimensions, so I think they could be rotated if desired, but right now x2mdl only outputs textured triangles, so I may just not worry about it unless it sees a lot of use.

Something else is how MDL files are prepared for Direct3D is in theory a problem for load times because it is "quadratic" in its performance, so I will optimize it. But I think this might relate to how I've observed the model data is broken up into small chunks, which is a problem in itself. I think maybe the chunking is just to avoid this quadratic problem. I saw that the memory is allocated in fixed chunks, so I suspect this is the case. If so it shows that the chunks aren't for z-sorting or anything.