simple machines forum
Sword of Moonlight => Archive (pre 2023) => Projects, demos, and games Information => Topic started by: Verdite on January 12, 2016, 08:47:25 PM
-
Welcome to the map piece tinkering thread.
I put this under projects as it's a joint effort between Holy and I.
Holy at the helm with ideas and scrutiny, and myself bearing the monotony of conversions*, modifications, etc. :coffee:
So rather than bounce emails back and forth about tinkering with the map models in order to achieve map pieces that'll work with the new AA system, I've opened this thread to hopefully improve workflow and discussion. I'm expecting him to make edits to this (top) posts when I've missed something, and to the Required modifications section. I don't mind editing in his ideas for convenience.
I'll place a :smug: beside my posts in the ideas section.
EDITED: I guess I am :saint: then. *The "monotony of conversions" should cease to be a thing before the end of 2016, if I meet all of my goals for the year.
Project summary
Verdite says: :smug:
Modification of the original SOM pieces is required in order to achieve an accurate representation of the new AA system (and show it off.)
Required modifications
Holy answers RFC: :saint:
Eliminate all ("AA"-related) cracks for tiles that obviously mate with one another. Details in Replies.
Ideas and brainstorming
:smug: 12/01/16
Uniform separate step model to apply to models of same height. Same will apply for all map pieces in other sets.
(http://oi65.tinypic.com/x0tngn.jpg)
:saint:
I cannot grasp what this image is demonstrating (it's a little small for me to be honest) but I am automatically against tiles that bury themselves into the ground like potatoes or carrots. I am not especially crazy about SOM's original stock artwork, but it is very sacrosanct, if only because it represents From's original product. I honestly do not think we should invest too much into it right now. It will not produce the best showcases for SOM. I want to get defects out, but after that, I'd rather you focus on your own artwork, or help with the project to port KF2 to SOM. It's high time for SOM to make an impression on outsiders, one way or another.
17/1/16 :smug:
The image depicts a 'foot' model to go below map pieces in order to create steps. The extrusion of the roof edges is required, else there will be an unsightly gap.
Absolutely agree, I hope we can get this project done and dusted soon. I'd like to work on finishing up Moratheia after this. Hopefully with a new x2mdl in the pipeline to give me a boost.
17/1/18 :saint:
I'm not good at imagining these things. There won't be a new x2anything, you'll just save/export to DAE and the player/editors will automatically do any necessary conversion/caching, probably even without stop/restarting the running programs. Just remember, if you make new tiles for the original sets, they must have proper icons, that are easy to make sense about.
Public and joint effort files
17/01/16
Set 1 ready for AA testing. Not yet stepped.
Set 1 msm files. (http://1drv.ms/1U5hYeU)
12/01/16
Pristine, converted parts 0-78 are available to review here. (http://1drv.ms/1ONPar9)
File is saved as an .mqo. This will require Metasequoia, or MetasequoiaLE to view.
Please make sure you place the Texture folder into your Metaseq directory, then merge it with the existing Texture folder.
-
Here is my advice about the "AA" cracks (if you want to do other changes you can pursue that, but this is the most pressing issue as far as I am concerned)
First it's refreshing to have a thread here. I've been feeling very irrelevant lately...
SOM_MAP can show you where cracks will appear. There are NEW buttons that have upright triangles on them, that when pressed show where the vertices are. Of course you can see this in your modeling software, but it's probably easier to compare tiles side-by-side in SOM_MAP.
Usually the cracks appear between two tiles, because their vertices do not match. But in some tiles there are like intra-tile cracks, caused by the tile's own vertices not matching. Without do_aa this doesn't yield cracks because you can think of the edges meeting in a T junction.
You see in this T, if the top crossbar of the T doesn't have a vertex in the middle, where the itersection is, then it will crack.
Two places where cracks are likely to appear:
Consider stoop tiles. I want to call them steps, but they are steps for giants, or terraces if anything. They just form a zigzag each one being placed usually 1/2 meter higher than the previous. These will always crack. I'm not sure what logistics are required to prevent cracking, but it seems like just adding extra cuts to their walls at every 1/2 meter interval would do the trick.
I think we would benefit from developing some standard measurements, but ultimately it depends on the tile sets.
Second place cracks may appear:
What a crack is, is a place in the model where you can see through its veneer. It's okay for this to happen if there is part of the model that lies behind an open section such as this.
Consider for example a teacup. The handle of the cup doesn't have to mate with the cup, in terms of vertices and edges. It can simply disappear into the cup. Or be lined up perfectly with the exterior of the cup. It will not crack because the cup part is behind the handle.
A crack appears when there is nothing visibly backing the parts that do not mate exactly. The reason for the cracks is the vertices are always moving slightly, and so a visible seam will open up, unless every vertex is matched. If the vertex moves, and there is no matching vertex, its only a problem if that opens up a view to "the void"; or the empty space inside of 3D models, or a view into the backdrop, or a compartment that should not be visible.
EDITED: I also want to add. That some tiles may have protrusions, like little windowsills or shelves. I'm not sure what to call them. Fluting. Anyway, I don't know how From's artists modeled these. But I reckon it's more than likely that they don't have anything behind them. In that case, it's probably better to make these parts separate from the wall behind them, because otherwise they introduce complicating vertices along the edge of the wall. So if you have a wall like this:
-------
A
====
B
-------
And A and B are separate in the model. My advice is to break the ==== part out, and make A and B one piece (a single quadrangle) and just have ==== sit over that piece. It's probably not a problem if ==== doesn't run into the outer edges of the tile, but I do not know what is better practice to be honest.
PS: Please do not try to fix the black parts of the textures that manifest as holes. I want to fix those myself so I can ensure that the procedure is mathematically sound, since this is not as simple as it seems, because it isn't merely pure-black parts that form holes, and neither is it any pixel where one of the color components is pure-black (basically it's any one where all components are between 0 and 7, and so care must be taken to select the correct replacement color, with respect to adjacent pixels)
[To be honest I haven't even decided. It's very tempting to say only pure-black is a hole, and repair the fallout from that decision instead. It isn't merely tempting. I realize that this is probably what I will decide to do. The only reason I hesitate at all is it will break all of the non-tile model textures, and will be a breaking change from classic SOM. But the second problem doesn't concern me so much, because that's a dead end regardless.]
-
Checked over the whole of set 1, and amended 3 major offenders. The worst offender was 0030, which had to be rebuilt from scratch along with pixel perfect UV replication.
Files up for testing in the Public and joint effort files section, to make sure we're 100% on the same page.
I'll step up set 1 after checking over set 2 on my next bout.
Cheers,
:smug:
-
There will definitely be growing pains around the AA technology.
Ultimately it might make sense to automatically insert the extra cuts in the polygons when generating MPX files. Still internally the models must be free of cracks. And I don't want to automate this unlessuntil we get to a point where per-vertex-lighting is never to be used again.
I was thinking about this while waking up today. There are a few different ways to approach the MSM files. I'll leave it up to you to decide what is easiest.
Strikeout: I thought there were more, but I realized one wouldn't work. So there's probably only two approaches.
The trouble is for detached pieces of a model, unless the triangles dig into each other, there is always the theoretical possibility of a a visible crack if you can line the view up so you are looking perfectly along the separation between the parts so that the perspective is cancelled out...
And having the triangles dig into one another is an option, but I worry it will cause Z-buffer artifacts, and will not produce lines that are as clean as they'd otherwise be. That is because the Z-buffer will determine the formation of the line--not Z-fighting, which is a separate kind of artifact. I don't have enough experience with this to say one way or another.
That said, the simplest way I could think of to do the terraces, would be to make the walls separate from the floors, and extend the edges of the walls until they touch the bottom of the steps on one side, and the top of the eave on the other side. This would introduce no new polygons, but might result in UV-mapping issues since the part of the texture below the wall might bleed into it. And since the wall is separate, in theory if you can get the view lined up directly along the wall, there's a small chance of seeing between the wall and the floor.
EDITED: I tried to think of another way, but I couldn't, other than cutting every 1/2 meters along the side as already suggested. That introduces more polygons, but x2msm does the same more or less. It may be more work to balance all of the new UVs unless your modeling software is very good at that, or is able to manage vertices without UVs (which is something I do not know if DAE can natively support or not.)
PS: I will try to setup a test project to try these new tiles out today or tomorrow. I kind of hoped you'd just understand what the problem is and work on it until all the cracks go away in the player. But I'm curious to see, if only because always working alone takes its toll on me. Reviewing others' work makes me feel like I'm in some kind of company.
-
Update
I setup a test project. I can see at least one place where you've altered a tile. What you've done is in accordance with what I described as "the simplest way" in the above post. However you only did the alteration on the top. The bottom half of the wall remains. To do the bottom in the same way part of the wall will be hidden behind the step portion, and the triangles cannot be cut...
I don't like hidden sections of polygons, but if you want to do it this way that is fine in this case, as it is a difficult one. Polygons could be added so the part that would be hidden could be minimized, but I worry how it might affect vertex-based lighting to have skinnier sections like that.
The other approach described is likely to yield more even vertex-based lighting, since it would simply carve the walls up into regular grids. What to do is up to you.
PS: I don't know how x2msm's splitting algorithm works. The only way I know to avoid it for certain is to make the polygons small enough that it will not try to split them further. As it splits it may introduce new vertices. So if the splits are regular, then they should be mirrored in the other tiles, but I won't be surprised if this becomes a complicating factor.
PPS: About half of your tiles I looked at no longer match the rotation of their icons/originals. They will have to be spun around at some point. Please don't lose faith.
-
Hello, I'm pretty much back to normal so I felt like dropping a thought off here before I set about making fixes...
I've been thinking over where I went wrong on the lower half of the model, as I followed the same design / AA fix principals for the whole model. I've attached a comparison image to show where I made amendments, if they are wrong please let me know.
-
Hello, I'm pretty much back to normal so I felt like dropping a thought off here before I set about making fixes...
I've been thinking over where I went wrong on the lower half of the model, as I followed the same design / AA fix principals for the whole model. I've attached a comparison image to show where I made amendments, if they are wrong please let me know.
I am not sure. In the case of this model, the windowsill like pieces may cause a problem. I cannot tell if they are separate or not, but if they are, then the full height of the wall can be a single section. They are clearly detached from the wall that forms the back of the niche.
However, I get the impression that maybe you think my reply above was in regard to this tile. If so. Just know that it wasn't. My main concern is the walls that form the terraces. They pose the biggest source of do_aa related cracks. So maybe you'd like to post those pieces instead?
I expect the proper way to do this is to cut the walls every 1/2 meters. So if you can separate the windowsill-like sections on this piece on display, that would ensure that they would not interfere with the 1/2 meter cuts.
BTW: I do not know if the triangles are built into the MSM files or not. They are capable of storing quadrangles and larger polygons. Unfortunately even though the Assimp library used by mdo2x is supposed to support polygons, I could not get it to accept the output of the loaders I developed for SOM, so they had to be triangulated. I think actually the problem was that Assimp only supported "verbose" models, and I wasn't about to output such models, and so they had to be run through a post-process to convert to "verbose" and that process did not support non-triangle data. These are things I am presently working to overcome, by taking the Assimp code-base into my own hands on behalf of SOM/the open-development 3D scene at large (edited: MDO cannot store non-triangle data. MDL can store quadrangles, but I don't think polygons, or I don't think I've ever come across a file with polygons anyway. It won't matter after converting to DAE, since these proprietary formats will only exist in temporary folders. And that will make it much easier to iterate on the formats, since the temporary cache can just be cleared if new features make old files incompatible. DAE will never be incompatible)
But what I mean to say is, even though it's extra work, assuming the pristine files you've cataloged so far will be the future art files for these tile sets, it's good to have the unnecessary diagonal edges taken out of the picture as you have done here.
I realize it's not easy, but neither is it impossible. You just have to reason about the AA. Imagine every way the tiles can be arranged (don't consider arrangements that introduce missing sections) and make sure there are no unmatched vertices for any of those arrangements. If you cannot learn its requirements you simply won't be able to produce models that SOM is able to use ever again :evil:
In order to make this compatible with vertex-based lighting, generally it will be necessary to add vertices, and then propagate those vertices further into mating tiles. Which is why cutting at every 1/2 meters is best, since this will produce more even lighting patterns (until we ultimately transition away from vertex-based lighting. The pressures are too great not to)
-
Update
I've come back to SOM after a long break, working on COLLADA. And it may be only a short break from COLLADA :sweatdrop:
I've been looking at this problem again. The landscape hasn't really changed. I can't really use COLLADA to get SOM's models into Blender. But after a break, and having forgotten the challenges involved, it occurred to me to use the old mdl2x tool, ran through Assimp. And luckily there's an export function in Assimp's command-line tool now, so I didn't have to make a program. Which might've been a nonstarter.
I used Assimp to convert from X to OBJ, and imported the OBJ into Blender. Then I remembered there were problems getting non-triangle polygons out of Assimp. I wanted to make a collection of SOM's models not in its proprietary formats. Eventually COLLADA would be used. But I thought X was good starting point. I added a trick to mdl2x so it can communicate to the SOM loaders that it wasn't doing post-processing. Assimp had required exploded triangles just for its validator post-process to work, and I think its loaders all outputted such triangles; not bothering to preserve polygons.
But then I realized that it was for naught if the X->OBJ step triangulated the models, but surprisingly it didn't. So maybe that problem finally got addressed. I certainly didn't have the sway to move the Assimp gatekeepers back in the day.
Also, x2msm would not work with the models produced by mdl2x. I didn't know in the past if I couldn't make x2msm (and friends) work or if it just silently ignored certain kinds of X files. I remembered I had some X files in a folder that were given to me long ago. They worked. So I sat about trying to figure out what the difference was. And someone had asked me to make a workaround to get text X files out of mdl2x, and that made it practical to hand edit the text until it would work...
So what I determined is x2msm requires a single mesh to be global and not inside the scene graph. And it ignores any additional meshes. I guess embedded meshes appear to it as if the file is empty. Modern day apps would use D3DXLoadMeshFromX, which is a horrible API that shouldn't exist, since it encourages this kind of ignorance. But I'm not sure that's the API that x2msm would've used. Possibly so. Possibly it depended on the retained-mode DLL in those days.
It also refuses to acknowledge files that have names that don't limit themselves to a-z A-Z 0-9 and _.
I think the biggest benefit of this may be that I can use mdl2x to mess around with x2mhm now. I intend to do that before long. I've always been mildly curious about MHM and now it's time to really figure it and MPX out. An impediment to that is I could never quite make out what was what in the MHM files, and I didn't have systematic way to make tweaks to the X files. I guess if I'd thought about it I could've used those same contributed X files and hand edited them, but I seem to have no shortage of blind spots.
So in the second attachment is the result of bringing the MSM files into Blender. The diamonds I added to note a few models that don't have PRT files and so don't appear in SOM_MAP. These have names following the same pattern as the rest of this set...
The theory I formed about the two doorways (after I made this screenshot) is I think they are same doorways, that would've been arranged back-to-back. My guess is it might more closely resemble how secret passages are done in the PlayStation games, but that it was scrapped because it was too complicated for a consumer product. (6 whole tiles dedicated to secret doors does seem like bad presentation. Really the door should've been built-into the whole apparatus and not treated as an object, so it would match the lighting.)
There are two ceiling pieces that seem to have receptacles for ceiling hung traps. Just a theory. And 4 pieces that I thought was a tunnel system, but I see that they are transitional only, and I'm not sure what they would connect to except for each other or possibly one or more of the other sets.
These missing pieces might have analogues in King's Field that didn't make it into the port in order to streamline the editor itself.
Sadly this morning I learned that the split-level pieces that transition to different elevations are impossible to fix for do_aa with x2msm. I played with them a lot to try to get x2msm to subdivide them so they would match the regular walls. I learned in the process how x2msm's subdivision algorithm works:
What x2msm does is not clever at all. It doesn't subdivide based on anything at all, except for a 1x1x1 meter grid that has an origin at 0,0,0. My guess is it has a multilevel subdivision tree system only to repair the cuts made by this initial subdivision. I can't recall if it subdivides down to triangles or not.
This is not a good system for attractive per-vertex lighting. Or shadowing as it were.
But it means that two tiles that differ in elevation by 1/2 a meter can never line up on that 1x1x1 grid.
This means for do_aa that I must develop something for generating new MSM files. I'm already planning on doing something that does this, so I might just get on with that, or I might use x2mdl. I think I'm more likely to do the former, unless I already added MSM to x2mdl at one point. Then I might consider it.
On the plus side, I'm somewhat relieved that there's no cause for me to have to replicate x2msm's subdivision algorithm for any reason. It's not a good algorithm, so if anything I should create a new one and run all of the MSM files through that. But right now, in the immediate term, if I work on an MSM outputting tool, I will want to use it to fix these problem tiles, and to manually subdivide them. So the new MSM tool would not require any subdivision step. Not for now. The only long term value to subdivision I think, is that it avoids baking the high-resolution vertices into the model that artists work with. Which would just be clutter for them. That may very well become a non-issue shortly, if artists will be advised to switch to using COLLADA.
I've added a second image. Only as reply to the images provided by Verdite before, two posts up.
I've illustrated the necessary changes (I made a day or two ago) to the first set of SOM's tiles. The number of changes are relatively small in truth. It's possible the other sets will require more changes. But it's also very possible that this is the most involved set with regard to changes.
I think based on recent emails that Verdite has a better understanding of the do_aa extension today. I put the niche front and center to show how it needed to be changed. It can only be seen from behind. I didn't keep before images. So I've drawn diagrams around the changed areas.
In the niche the back wall has been made into a single piece, so that its edges match adjoining walls whether they are niche walls or plain walls.
For the bridge piece in the back, the floor part is extended all the way across, so that it matches adjoining floor pieces.
The other piece is the split-level wall. This is the hard case that can't be addressed with x2msm. Because its subdivision step spoils it. Its wall must match the other walls. That's complicated by the walls meeting at a different height.
I've changed it by making one of the walls seem to have a webbed connection to the floor, that is out of view. I regret that this is not so attractive from inside of SOM_MAP. I tried a different arrangement with two disjoint flat panels also, while trying to make sense of x2msm. It looks a little better, but is still confusing I think. A better way would be to split it up. But if you do that either A) all connecting walls must be split, which would be a lot of changes. More than I can do. Or B) You have to hope that x2msm would do the splitting for you. But it's now known that cannot be the case. Maybe an alternative subdivision algorithm could do something like that. But it can get tricky because adding uniform splits can be like unraveling knitting.
EDITED: I also did not mention that, theoretically it's possible to create degenerate polygons. But I don't know if modeling software would be happy with that. It's something that an x2msm replacement could do with instructions of some kind embedded in the models. Another possibility to make SOM_MAP looks better, is similar, but just short of degenerate triangles. To do this the webbed parts could be cut back. But that disrupts the UVs and lighting, and so is very hard to manage with UV maps, and might not look so good with per-vertex lighting. I'm sure I theorized this in earlier posts.
So, in conclusion. I've put at least two full days into this. And I'm beginning to be able to do things with Blender. I'm by no means impressed with Blender. But I'm relearning it. Which is something I'd hoped I'd have the opportunity to do. Only last week or so I threw up my hands with working with Blender, and regretted the possibilities. But after I gave up trying to import COLLADA, I made some progress. If only short term.
My work on an alternative to Blender for SOM is taking longer than I hoped. So to try to accomplish some things, including do_aa, sooner than later, I am just trying to make ends meet again.
P.S. The diamond marked 5 looks like it might connect to the lofted ceiling beside it. But it's not. It's like the ceiling beside it. Recessed for some reason. But seemingly of cruder construction.
-
This afternoon I've tried to figure out how to interpret a COLLADA file as one of SOM's file formats. Blender's built-in COLLADA exporter is not so hot. I've been asked before to come up with a system for not having to set MSM tiles at 0,0,0 to export them...
I first tried to use a joint, or bone, or whatever Blender calls them; but everything I tried produced laughable results with the COLLADA exporter, that wouldn't be usable. Probably it's completely useless for animated MDL models. I don't know how the hell Second Life or anything could use it...
But what seems to work is actually pretty nifty for MSM management. Blender has an "Empty" widget. And it can be used as a parent to the tiles. I've attached an image of it. I've made it use the Image mode, and set the tile's icon to be the image. The color is the same always, and must be changed in the theme Preferences. It is black at first, which is pretty much impossible to see.
EDITED: Also set them to show Axes, a Lable, and X-Ray mode.
Because the tile has a container, it can be moved around. And the container here should even remember which direction is north for the tile, unless I misunderstand it.
(I think the tiles need standard directions. And some of the legacy tiles may have to be fudged to achieve this.)
The real problem is I would like to be able to drag-and-drop the DAE files, without providing any context. I also don't want to label things with MSM and MPX and so on. Because the goal of this exercise is to abstract away from runtime/proprietary formats. And those are just awkward acronyms that people shouldn't have to memorize.
Blender has next to nothing in terms of frills. So its COLLADA exporters can't provide any real information even if it wanted to. Not unless they cooperate with Add-ons.
If MDL can be eliminated, that just leaves MDO, MHM, MSM, and MPX. My strategy for MSM is to look for the word "tile" in all permutations in the name of a node, and if it doesn't have model data, assume it's a container. For MHM: anything that doesn't have a texture is automatically MHM. Textures are about the only reliable feature. Textures are required...
MDO/MDL is everything else except MPX. Hopefully "tile" doesn't clash with names. Maybe "plan" can be used as an MPX container. And something else for other objects, that trumps both "tile" and "plan"? Content?
Strikeout: I'm actually considering emitting MDO files for tiles too, so that MPX can eventually have transparent elements and lighting. I may keep MHM as a non animated MDL format, so objects can use MHM too. That just means figuring out a way to mark tiles for emitting MSM.
EDITED: The icon file is not exported. It's just for reference. I'm not sure if it could be used if it was, since the icon is more of a PRT thing.
-
*Exacerbated smiley* 1 set down. 4 or 5 to go. Then 4 rounds of copies. (Apparently at least the final set isn't a straight palette swap. So I can't assume anything.) Of course, the work is small potatoes to the technical hurdles.
I'm so tired of manipulating Windows Explorer :grumble:
Either Blender or its COLLADA exporter outputs relatively small floating-point errors, but not minuscule. On top of the dragging and dropping I would drop into the text editor and edit these errors away. Of course they could very well be in the mesh data itself if Blender's math is just sloppy.
I wonder if they are enough to open up seams themselves. If it becomes a problem the new conversion process will have to round the data off. It's features like this that modeling software always seems to lack.
I worked a new component that I'm calling COLLADA-SOM.dll for the time being. I use it by dropping DAE files onto SOM.exe. It saves them to a folder like C:\Users\Michael\AppData\Local\Temp\Swordofmoonlight.net\file----C--Users-Michael-Sword_of_Moonlight-data-map-msm if the file comes from the map/msm folder (or anywhere on your computer. Theoretically anywhere on the Internet.)
Historically CSV files are dropped on SOM.exe to trigger the updater. This extends the idea of updating to the 3-D models themselves. Eventually I'll either change its Open function to accept DAE files in the filter, or add a new menu item. That will include a COLLADA viewer also.
This workflow is going to make everything much more manageable. It's very good, but will be disruptive. I guess some of the little side projects I've done in the past that expected to use SOM's files and file tree won't make a whole lot of sense after all this.
The image shows these problem tiles. They come pre-divided up. For now I'm just modifying the MSM files. This will overwrite files in the DATA folder. If you know someone with a modified DATA folder, remind them to make their own Data folders with the new SOM_EDIT Settings function. I know it's not so simple because some of the accessory tools expect to use the DATA folder.
-
The second tile set doesn't seem to have any do_aa issues. It broadly works on a 1m basis or if it has any that aren't so, they are so irregular as to not mate with any other tiles.
The third only has the 1/2m rise (technically a fall) pictured here. I actually did this one last night; but I simplified the wall so that it was like a plain flat wall, since that's what it is. But I realized afterward that that might clash under a lamp light. So I went back and just did the subdivision as MSM would if it understood elevation transitional tiles.
It looks crazy, but I reckon just let the chips fall where they may. Vertex-based lighting is always a bit of a crap shoot. The level walls would look pretty similar fully subdivided.
I added little handles to this Blender file, while importing the MSM models. It's a pretty fast process. Not completely painless because Blender links the icon when it is copied. So it has to be un-linked every time. It would be nice if selecting multiple files in the import menu would import multiple files, but it doesn't.
The best tool for these split-level tiles is the Bisect tool. It's basically completely useless. I thought Blender was poorly designed. But I realize now that its developers are just lazy bastards. There's no craftsmanship to Blender whatsoever. And so I guess it's hard to criticize something so poorly constructed, because you can never know if it's because of deprivation or genuine ineptness and carelessness. In any case, it doesn't inspire confidence.
Bisect would be perfect if I had a system for determining the plane normal to bisect along. But Blender won't even tell you the value of a face's normal. Bisect shouldn't even be expecting a hand inputted normalized value in the first place. Which is half of why it's an awful, half-assed piece of work.
In trying to see a normal I came across this (https://wiki.blender.org/index.php/User:Mont29/Foundation/Split_Vertex_Normals/Custom_Split_Normals_Manual) and this (https://docs.blender.org/manual/en/dev/modeling/modifiers/modify/normal_edit.html) which might just possibly be capable of softening things like the outdoor set's tiles. A plot of grass shouldn't look like a pyramid. But there's really so little data to it, that I reckon From's staff just gave up on the surface normals.
I didn't finish bringing in the rest of the cave set, because it's actually spread out all over the place MSM wise. It suggests that the staff started with the cave set. It's very disorganized. I'm trying to organize them into Blend files for future modeling work. I would like to tackle the lighting normals as soon as possible, and the UV maps scream for extensive work.
I realized that MSM files should probably be MDO files too. They need transparency information. I can't think of any good reason for the artificial division. I thought that MHM could be a lightweight MDL file going forward. That means if a model isn't animated, it can make do with MHM. This means that the new system only needs to know if a model is a map tile or not. And that is just so you can drag and drop it to get an MSM file back.
This simplifies the present use scenario. So instead of using words like "Tile" to mark things. I think I've decided to get away from English, and use symbols instead. But the problem is that COLLADA 1.4.1 names are NCName (this was a mistake) meaning that they can't include ASCII symbols. But they can include most Unicode symbols. So I think I've chosen to use WHITE LARGE SQUARE as a tile symbol. This means you can use any name you want to. (This symbol can just say, make a MSM file, and so it's probably the only one we'll need. Eventually something like <keyword>msm</keyword> can be used instead. But I'm trying to make this easy to do with only Blender.)
I would've put the square in the model's name, so it's not visible in the little fuchsia handles. But it's convenient to be able to copy the handle, without having to copy-paste the square into the names. Since the model names come from the exporter.
Set 4 has many tiles that require attention. And 5 is going to need some creativity too. It has its inset plaques and paneling. Then comes the outdoor set. It doesn't look too work intensive.
I expect to have this done by next week. After I am going to write a disposable little program that reads in instructions to generate palette swapped MSM files. Eventually this practice will be obsolete. But I'm writing the program because I think it will need to be run again in the future. If I do it by hand, I'll have to do it again and again, so I'm not going to.
The same program will fill in the holes in the bitmaps and output TXR files. It will be usable enough for other's projects to use.
-
Some "porn" for SOM freaks :hearton:
I couldn't do this without the model database inside this website from years ago. It seems to be broken since my gracious host moved the website onto a different server earlier in the year. I need to work on it. I think it shows models that were viewed before, but it won't generate new models on demand. Fortunately I can get by on icons for now, as long as I'm doing tile work.
EDITED: Some of these islands (pictured) are not lined up on the grid correctly. The Viaducts actually connect to the Arches. I later put them together.
-
:confused: Hit another crazy snag.... *drum roll* (:drummer:) x2msm doesn't do its subdivision thing on polygons below 0!
I'm unsure how to proceed. I will get the problem cases for sure. I might not bother with cliff like pieces that seem designed to fall off of.
The Balcony tiles (pictured previously) are 2.5m below 0.
-
Had a pretty awful half workday today.
I learned about Blender's "Auto Smooth" setting; confirming that Blender is truly the devil. I'm guessing you don't so much learn about it, as be driven to the brink of madness before offhandedly figuring out about this thing you never asked for and is nowhere to be found anywhere in the vicinity of any of the lighting related functions. (And the cherry on top is once you learn about it you get to spend the rest of your life disabling it and or reencountering it on a mesh by mesh basis as near as I can tell.)
Blender is anti-software. I want to take in its pain and learn from it how not to do things ever. I hope I can turn the pain into something good...
This problem manifested as messed up lighting where these edited models were concerned. Inside SOM and inside Blender.
There a second problem rolled up into it though, and at first they seemed like the same problem...
Actually this turned out to be a relatively minor problem. It was an MSM. The wall with a niche in the 4th set. But the problem is either not in the second copy of that set, or is somehow not as apparent. It would come out when the shadows are produced by a lamp light source.
I didn't realize the problem was in the original MSM. Because I was comparing it to its twin. Which didn't have the problem, or not so badly.
Let's just say that it looks so bad that if you tried to park a light near this tile, you'd either have to change the tile or remove the light.
Anyway, I thought this was a larger problem with my workflow. But really there were two bad vertices in this MSM's model. One was nearly on the 1m grid cut mark. So x2msm would merrily create a sliver of a triangle between it and the cut. That really throws off the shadow system it would seem.
The discrepancy is actually not very small. It's off by like 0.005, which is not simply rounding error. And it's probably too large to plug too. You're talking about rounding to the centimeter. The MDL files can do millimeter precision. But still, the effect was dramatic.
The other vertex was off by considerably less. Like 0.00005. It was not on the grid. But should've been equal to a sister vertex. It seems that the sister casted an inescapable shadow on the off vertex, or vice versa.
I did not expect any problems in the lighting department. But here were two. That seemed related, or the same, but were not.
Good news is the 4th set is finished. Although some of its twins are not strictly identical, like the 5th set. I will have to be very careful that I don't obliterate these differences. One example of small differences is that wall with a niche.
I did change the lighting for wall with a column set in it. It looks round instead of octagonal now. Which I think is probably the desired effect. In other cases, it's hard to justify straying from flat lighting.
-
For the record, I ran into a new problem days ago (I've been in town for the weekend.)
This problem is because 1) Blender is highly inaccurate. And 2) mapcomp.exe's shadow generation system doesn't have any kind of threshold. So that a number like 0.000001 will cast a shadow on a 0.000000. It probably has some sense of an epsilon (very small threshold) by some mechanism, but it is very small if so, and it really should be treating these numbers as equal.
Blender introduces major errors when you use its Parent->Child relationship system. You can create this relationship, and then you will have errors, that cannot be seen in the numbers. You can then set these numbers equal to themselves to erase the errors. (A typical Blender WTF??? moment.) But that's impractical.
I don't know what artists do to make custom made SOM art pieces ... but it seems like we've been very lucky up to now. Call it some kind of providence, even if nothing much has come out of SOM user communities thus far.
I've thought from the beginning that something like this would be in order. I didn't expect this though.
My naive approach to rounding didn't work. It would produce acceptable shadows, but was insufficient for do_aa. Although it might have worked if every file had the technique applied to it, but here I am mixing existing MSM with new MSM.
Thankfully though I was able to research and learn about less naive ways, that do seem to do the trick.
So I've implemented this rounding for the new pathway hosted by SOM.exe (COLLADA-SOM.dll) and also for mdl2x, since I'm using it to convert DAE files to X files, for x2msm. Except in the latter case it's designed to work with Visual Studio 2010 which didn't have a rounding function, so I had to improvise one, that I assume will work.
So for now, I'm back on track.
Actually I ran into this problem with the 4th set's split-level tiles. The tile that serves as a middle piece for those, has a texture that is completely mismatched. So I'm going to try to repair that texture; to map it to the right section of the image, or change the image if it's using the wrong image.
I didn't intend to do texture work. But I think if I find these total mismatch problems, I should try to address them immediately.
-
:confused: I've almost completed the exteriors set. It's been a big eye-opener. I think I only finished by the skin of my teeth. MSM and mapcomp.exe have a lot of problems, that I do not believe artists can solve.
The exteriors set it unique for a lot of reasons. When I implemented some code to round numbers to try to get nearly identical vertices to not cast shadows on each other, I chose a much too severe parameter for this. It wasn't noticeable for the interiors because they are mostly square things. But the exterior set first raised eyebrows. In fact the same tile that first made it clear there were problems was the last tile I got around to, and it was still nothing but trouble. It--the cave-like entrances that transition to interiors--just wasn't going to work without cracks or bad shadowing one. I struggled to seal its cracks. And decided that was good enough under the circumstances. But I woke up realizing there was one last option. And just barely got it to work with shadows by 1) recompiling the converter program to remove the shadow fix, and 2) manually removing the errors Blender introduces, that I designed the shadow fix to avoid having to do.
Artists can't do this kind of stuff to make ends meet. And I don't want to signal to the program to apply that fix. And I'm not even sure it's the best way to do that. And there's no perfect fix, since any kind of rounding always sorts points into sides, and there's always a possibility of points lying on the threshold, and so being parted.
The parting happens mainly with organic shapes with odd curves. These tend to have values that are not round, rational numbers. And because they are organic, it's inconvenient to go around sawing off their coordinates. And x2msm slices and dices models, and so its slices always run the possibility of falling on a threshold, and so it doesn't make sense to try to round away its outputs.
It's funny how the cracks seem to be much more forgiving than the self-shadowing problem. It's almost as if 3D hardware knows how to partition numbers, but I don't it does actually. Mapcomp should have included a test for these cases. It was just shear luck (or our misfortune) that From Software's art staff never encountered the problem.
To begin with the exterior set, I chose to not approach the half meter rise tiles the same way. Because their shape was so different. Instead I treated all of the wall pieces as if they were rises, and so did not use x2msm with them. I later came to half regret this, because many tiles have walls that connect to things, and it soon became evident that every tile would have to be modified.
I used the same program to extract the models as to convert them to X files to prepare them for x2msm. And it applied that shadow fix to the exteriors set. I didn't learn that I should've chose a different rounding parameter until I'd already made extensive edits, and so could not turn back. That altered the shape of the models, so that except for the simple square floors, they were mostly incompatible with the originals. I checked to see how many vertices this applied to. They were not minor differences. You could stick your fingers through them if they were life size models. But very few vertices were affected. Mainly only the amorphous pieces were, at their curviest points. Where the changes occurred, generally it took long, ugly irrational like numbers and made them round numbers that are easier to reckon with. So in a way it's probably a good thing.
This morning I decided to be more conservative, as I've tended to do in this project, after having more time to consider my actions. So I tried to go back and use x2msm for the entire set, except for the half meter rises. That way the set would be more consistent with the rest of the tiles, for lighting purposes. Even though the x2msm cuts generally don't look good on this set, because they tend to occur very close to the lines that are already there, and the set has lots of curves already and so is basically tessellated--or subdivided--out of the box...
For this I used a trick I learned: Blender has goofball Knife Project tool. It came in very handy with the ramparts. When I realized the job was messier than I thought. So I made a grid that matches x2msm's cuts, but sheared it (Blender doesn't even have a Shear transform???) and used it to slice up the tile in one swoop. But as it so happens, my original decision to not try to do this unwittingly was the right idea, because slicing these curvy pieces (with Blender at least) changes their topology as flat faceted meshes, and that alters their surface-normals. This is not what x2msm does. x2msm simply blends the original surface normals. This is something Blender cannot do, unless there is a fancy "modifier" for it, and if so I'm skeptical it would be supported by the COLLADA exporter.
AND SO AGAIN this is territory that artists can't really manage in. The interior pieces were able to do this because they just happen to be flat. So for the exterior set, it's necessary to not only do a manual subdivision of the split-level tiles, but also for every tile that connects to them, or can connect to them. And for consistency anything that looks similar, whether it can or can't.
In any event, I'm doing one last thing for the exterior set. I am making it able to be stacked on top of itself. To create terraces; without cracks. I recall the Dark Destiny project had an icy environ that worked similarly.
I feel like I've just barely gotten away with this campaign. I hope to have a release ready before months end. I have to take the modified tiles and somehow replicate them (without redoing the same work) across the 5 level's of the KING'S FIELD remake's cemetery. I will probably try to build the SAMPLE project with the modified tiles, since that's the easiest way I can think of to do a quick overview of all of them to see if I missed anything as thoroughly as I can be. (I can't exactly create such maps, easily, and this is more or less what the SAMPLE is; so why reinvent the wheel.)
My take away from this experience, is I think the next major initiative on the horizon is going to be mapcomp.exe. It needs a complete replacement if SOM is going to be competent. It's not artist friendly, and fails in many ways: self-shadow artifacts; lighting doesn't match Direct3D; wrong surface normals between tiles; seams should be automatically healed if MPX allows it. These are just the basic geometry things.
I would say I'm rearing to go at this, except that I don't yet understand MHM and MPX and so must apply myself to that first, and I have other duties with COLLADA-DOM. So I don't know the timeline exactly. But I can feel that this is going to be my orientation major features wise in the coming months. Maybe it will be done before year's end, if not in the beginning of next year.
I've decided to implement mapcomp's replacement inside SomEx.dll, and so inside mapcomp, although there probably won't be anything for the original mapcomp code to do. My justification for putting the code in this DLL that every tool and the player loads up is A) it's easy, and there is a lot of project bookkeeping code that needs to be identical across components, and B) the same code could be used by the player to do something like rebuild the MPX inside the player, or update the lighting in real-time, either so an open instance of SOM_DB could be used to see changes made in SOM_MAP as they are made; or a game could actually change the MPX on the fly to do dynamic lighting for example, or to compile the map on the fly, so it's created on-demand and possibly according to the player's preference or latest available technology. It doesn't have to lift heavy COLLADA files, so it can be done from within the player....
By compiling on-demand it's also possible to update parts of the map piece-wise and recompile it. Really the MPX is secondary, like the CP files. IOW it doesn't include original information; and so is ideally a temporary file.
EDITED: Another factor for choosing SomEx.dll is I just couldn't think of a name for a new standalone mapcomp replacement; but also there will probably have to be something on the MSM side too. It's hard to say what exactly until more is known about MPX. A lot depends on if an MPX is just one big model or if it is organized more like an archive of the various input files.
-
Here (attached) is a more-or-less pristine set of X files for every MSM file, including curiosities that don't have a PRT file. (A few of which at least could have PRT files of their own.)
There are now textures in the following folder.
http://svn.swordofmoonlight.net/Sword-of-Moonlight/data/map/msm/bmp/
I assume it will be reachable after it is deleted, but I don't know. It is Revision 322. It needs an SVN client unless you intend to download them all by hand, or with an FTP like client.
I'm hesitant to include X files in the regular download, because there's only a few odd X files among SOM's files, and it's a dying format. I've used the old mdl2x program a lot in this project, but I've modified it to work with x2msm, and it may not even be up to doing SOM's "tinman" style MDL files now. It's always been more of an experiment. (Not to be confused with x2mdl.)
Around set 5-1 and the exteriors set mdl2x was made to try to normalize vertices to reduce shadow casting errors, and I did this too severely. So these are not exactly pristine. I checked 5-1 against 5-2 today, and did not find differences. But I didn't look long either. There are big differences in the exteriors set. But they should be self consistent at least, as a set. And there are slight differences in all of the interiors sets after 5-1 due to the aforementioned normalization, which I left in place, toned-down. I think this difference is so slight as to not be perceived.
This is more for historical reference. Since I've already made many changes. It's easier to compare against X than MSM itself. Needless to say. The download files keep a history, so it's not as if the old MSM are truly lost (short of installing SOM from an old disc.)
Update: there's now an MHM download in the next post. Reply #17.
-
Here's the MHM files in the X format. It seems like there's probably a lot of duplicates among these, and many times they do not differ from the MSM files.
Off-topic: I uploaded 0030.msm today after realizing I neglected to revert it back after using it as a reference. It should've been in the recent update.
-
I've been removing micro features from the MHM files, and sometimes make the PRT files share the identical MHMs so they can eventually be removed.
I just discovered a problem with arched walls/ceilings that I introduced because I was concerned with how the cave ceilings knock the PC around so. Basically as if they are sliding down a hill, touching the ceilings is quite violent.
I don't understand the problem. It is creating an attractive effect for the 4th set's arched bridge, seeming to propel the PC in the opposite direction.
I'm making the space under the bridge wider. Otherwise it's not really noticeable. But now I guess the best thing to do is to remove the arches from the cave ceilings. Since while interesting, they are really just micro features that are there because the art staff didn't take the time to prepare proper MHM files (generally preferring to just use the raw MSM geometry.)
In the meantime I've disabled this ameliorating addition. Probably the better path forward is to not put arches in MHM files unless there really is cause for a hard obstacle.
-
Good news! I was experiencing a problem yesterday that I thought had been solved. So I got led into the old/new clipping extension code from awhile back.
The problem is a glitch in SOM that pushes you down where a slope meets a floor tile. I think I accidentally compromised the fix, so that in some circumstances the problem remained. (I was experiencing it while hugging walls and crossing the threshold.)
It took me a long time to familiarize myself with this code. It's by no means straightforward. Working with SOM is a bit like retrofitting the SDF-1 in the famous Macross anime series: to wit, it might as well be an alien artifact and it's just tinkering around its edges most of the time.
(EDITED: Sometimes tinkering around the edges has its draw backs, but I prefer to ease my way in. The clipper definitely seems disadvantaged because it seems to stop at the first MHM face that is hit, and can't proceed until moving past it. It seems like it could be a better system if it could collect all of the faces that would be hit and consider them in concert ... but that would require going deeper into the clipper and reprogramming whatever is in there.)
BUT IN ADDITION to correcting this, at the same time I began to be dismayed by the bumpy ride you can receive when moving around corners, especially when hugging the walls.
So, today I tried to address that. The result is just some smoothing that is done when the clipper gets violent. It can't be done for long, because the violence may be justified; in which case counteracting it could prevent it from doing its job. So the counter force is lessened over time, so it acts like a stabilizer.
The result is more or less as if the sharp corners in the MHM files are rounded off. It only applies in the horizontal plane.
I am going to do a small release, and am holding off until I finish the MHM work, in case anything comes up. I have done things like remove the beveled edges from the windows, and sometimes that makes the clipper extensions not behave as they should; resulting in standing up in a window on the outside of a wall for example. Most of the time it behaves correctly, ducking into the window as necessary (or not entering if it's too small to enter.)
The exterior MHMs require a lot of work. There are holes in them. Some have extra faces that seem to act as guards. But some that should follow suit don't. I think I'm changing them so there is a very short wall along the ground. As long as there is nothing to stand on above the wall--at ground level.
MHMs definitely have quirks. I don't know how much good it would do to describe them right now.
-
If you've ever fallen through the ground in a game of SOM, maybe it was this? :batman:
UPDATE: There were not problems with a matching (two triangle) MHM model. So it seems like this is not intentional. (Of course it doesn't look intentional, but get a load of the 1m set...)
-
Oh my god... the 1m hill set is way worse :confused:
The perspective shot is from below/behind to try to show both of the interstitial boundaries together.
And yes, the straight piece can't stay inside its tile! :box: :box: :box:
-
I rebuilt the 1m exterior hill tiles. They are pretty smooth now. It's actually very hard to make a climbable 1m slope fit into triangle tile. 1m is the limit without increasing the angle requirement.
I had to somewhat remodel the 45 degree hills. The only criticism now might be too smooth. But there's an occasional stumble, maybe just due to gravity effects. I might look into an old extension designed to lessen these.
At the base of the wall there is a buffer that is a flat vertical wall. This used to be underground and many meters deep. But I brought it above ground. I don't know if the depth is important, but it shouldn't be...
It is a hack to get the player to work, but in theory it is also needed because the surface normal coming off the skewed wall is skewed itself, and I think players would expect the wall to repel them backward as it was a flat wall instead of a skewed wall.
I deleted two posts. I went down a rabbit hole where I thought I discovered a new bug in the player, because the original MHM models (previous post) look like they are working hard to work around a bug. I investigated the player until I had enough information to suspect the MHM models I prepared. And it turned out one had a zero-length extra (5th) side in a polygon that was the cause of the havoc. Though I feel almost certain that the problem affected more than the one tile once upon a time... it's possible though that edits I made inadvertently removed more such extra sides (doubled up vertices) that may or may not have been in the originals (I can't think how I could have created such a model with Blender's editing tools.)
This may have been providence though, because it got me under the clipper's hood. Which will be important later. And maybe sooner than later; because there is a problem--or a missing feature--that affects this set...
The top 3 polygons in the wall tile are not final. It seems it's impossible to put slopes on the top of a wall, forming a kind of uneven cliff side. I flattened them from a steep slop to a straight wall to see if it would work as a wall or not.
I'm digging my heels deep into this problem.
There is a secondary problem around the base of the wall. For reasons I don't know it is bumpy when skirting the wall. It remains an imperfection in this set. Interestingly the 0.5 m tiles don't have this problem. I'm unsure what the difference it. Perhaps the point where the slope turns on itself.
P.S. When I disable the extensions to see how the original code performs it is so bumpy I think something must be wrong. I don't remember it being like that in the beginning. At the same time I don't believe anything's changed. (EDITED: I tried an old game, and yes the bumps are there, but the faster movement makes them less apparent. The old bob animation is really distracting.)
-
I've been to the mountaintop! and I didn't fall off the mountainside!!
Big things coming in the next release. It's kind of a mess though; so it could take as long as it has already to organize and finish the remaining MHM modeling work. It will be done before the end of the month. Even if I have to rush it.
The NPCs are going to get the same treatment. Maybe the magic missile stuff too. Until now I've only modified the PCs clipping behavior. But I think if the NPCs aren't incorporated into this it could cause problems.
-
This morning I got to the bottom of a problem I didn't quite understand. It had to do with walking off of flat pieces onto slopes, below them...
The trouble is a flat piece is a platform. So until the clip radius clears the platform, it can't plop down onto the slope. Naturally that's not how the exterior set works, conceptually.
There's not much I could think of to do about that. I just made it so the clipper looks below the current elevation with a laser precision, and prefers to take a slope. If it finds one, it keeps using that laser precision. It's probably imperfect. It's a tricky problem.
The end result is a much smoother experience going down slopes. Which is important, because they are smooth going up. I hadn't really noticed the problem because the clip radius for purpose of climbing is normally small anyway these days...
It expands when dashing however, or really depending on the "slip" extension, and also when pushing against things, whether that's appropriate or not (I'm not really sure at this point.)
So these are the times when the bump can be noticeable. So the bump (plop) would occur when pushing against a rail while walking down a slope. Because the pushing expanded the radius. The radius could not be expanded, but that wouldn't solve the problem from the end of running down the slope.
I redid the exterior walls yesterday. I decided to make them simple. That wouldn't work before I started this, but it does now. I think maybe it was because the 1m slope was going outside its boundary that it would pass through the 1m slope wall without the extra gutter wall hidden below its surface.
I didn't provide any vertical walls in front of the slope walls. This means that pushing against them reflects down the slope. So pushing against the wall at a straight angle will push down the slope.
I ultimately didn't think it was worth adding extra shaping polygons, even if it might have been safe. To make the MHM models look nice the shaping polygons would have to be on every wall, matching, slope or not. And I'm not sure, but the steep slopes maybe don't push away at the right distance, so if not repeated throughout the slope walls might push further than the rest. It should be investigated one day I suppose.
But really, the only reason it should push back is friction. And so I reason that if running or falling there is less friction, and so the behavior is correct. So what's really called for is a model of friction. I would like to model friction at least for pushing into things anyway. So I think the better plan is to pass this problem down to the future.
The main reason I'd like some friction is for climbing, I think it would be nice if there was a lot less sliding along the surfaces when pushing into them. The sliding can feel very unpleasant, as if you don't push precisely forward you'll slide along the wall very fast, possibly to your demise, but at the very least, unpleasantly.
In any event, the clipping experience should be only this much better as soon as the next release drops.
P.S. I expect this extra precision around slopes can only help with the problem of slopes abutting cliffsides.
-
I've uploaded some MHM/MSM work I did weeks ago. I was going to do it all in one release, but there are noticeable holes in the exteriors set without these, and I probably won't get around to doing a full pass or new release anytime soon.
Modified: C:\Users\Michael\Sword of Moonlight\data\map\mhm\ykoh21.mhm
Modified: C:\Users\Michael\Sword of Moonlight\data\map\mhm\ykoh22.mhm
Modified: C:\Users\Michael\Sword of Moonlight\data\map\mhm\ykoh23.mhm
Modified: C:\Users\Michael\Sword of Moonlight\data\map\mhm\ykoh49.mhm
Modified: C:\Users\Michael\Sword of Moonlight\data\map\msm\yko20.msm
Modified: C:\Users\Michael\Sword of Moonlight\data\map\msm\yko60.msm
Modified: C:\Users\Michael\Sword of Moonlight\data\map\msm\yko61.msm
Modified: C:\Users\Michael\Sword of Moonlight\data\map\msm\yko62.msm
Modified: C:\Users\Michael\Sword of Moonlight\data\map\msm\yko63.msm
Modified: C:\Users\Michael\Sword of Moonlight\data\map\msm\yko64.msm
Modified: C:\Users\Michael\Sword of Moonlight\data\map\msm\yko65.msm
Modified: C:\Users\Michael\Sword of Moonlight\data\map\msm\yko66.msm
Modified: C:\Users\Michael\Sword of Moonlight\data\map\msm\yko67.msm