simple machines forum

Please login or register.

Login with username, password and session length
 

News:

Remember to make your own backup of posts before submitting.

 
 
Pages: [1] 2 3 4 5 6 7 8 9 10
 1 
 on: Today at 12:04:36 AM 
Started by Holy Diver - Last post by Holy Diver
update

I've been taking it slow to ease into this next stage of this project. It's very scary territory. I've done some preliminary work on x2mdl to be able to load it as a DLL and have it do cpgen's job too.

I've been thinking in terms of how to communicate the new level of indirection to developers, without instructing them (I believe SOM should be something you can find your way around without a tutorial) what to do but to also have something flexible enough that you can manage it manually when you need to (I'm especially thinking about how to make standalone games that only have the converted data and don't have source art of caching/update components) but to also be simple enough that it's not a lot of work to maintain or a major interface presence.

And I've been thinking about what the hell to call this system, as in how to label buttons, etc. For authors who don't understand the technical breakdown of game development infrastructure it's hard to explain why there's a need for a "tool chain" in the first place, but this is not an intuitive jargon I think either. The goal is to make the "tool chain" invisible and that's actually what "tool chain" means too usually.

How you experience it by default is when the game starts up the tool chain needs to kick in if any of the art is modified going by "modification" timestamps in the files, i.e. Windows Explorer's "Properties" stats and sorting options. When that happens there will probably be a decent delay since it's going to be doing the heavy duty task of converting the models that you normally do manually with x2mdo for example, but at least you don't have to do it manually anymore.

That's the basic scenario. Except if you want to make a standalone game or a demo for example, then there should be a way to do this for every model in advance, because otherwise you could only trigger the system by playing the game, which isn't always practical. And this adds an extra step to preparing a game/demo because you'll really want to copy your DATA folder out of the cache, which will be in your %TEMP% files by default.

So after giving it thought, I decided that to have such a manual option the best place to put it would be in SOM_MAP's "Maps..." screen, since that way you can choose from the maps you want to rebuild your cache for. That would be nice if you want to publish a demo with only a small selection of maps. That way you can be sure you're only including DATA files for the things that the maps use.

I decided I didn't want to make this a screen because I think SOM_PRM would benefit from having its own version of this, and even SOM_SYS perhaps. But I'm less inclined to put one in SOM_SYS because I think it only makes sense on the first screen with the character setups. Anyway I wouldn't want to have a copy of a complex screen in both tools. So I think just a push button is best. But I think it can open up a standard directory selection screen so you can choose a custom directory, but it will open up to the %TEMP% directory. Actually SOM files already have a TEMP variable for this.

Opening up to a directory will double as a cue to what's going on, since that's where the files will go to. But there won't be any other instructions than this.

Finally as for what to call it? I think I've settled on calling it "Artifact" as a kind of jargon. Other names could be something like Cache or Data but I want to focus on the idea of source art, which also helps distinguish it from map models. So I think my goal is to add an Artifact button to the bottom of SOM_PRM that will rebuild the cache for the selected items on each tab in SOM_PRM. And put another one in SOM_MAP that will rebuild items used by the selected maps. This way if you were to need to prepare an entire game it should be enough to select all the maps and rebuild them (maybe to your game's folder) and then go into SOM_PRM and select all the "items" to get the items that aren't on any maps. That should cover everything. Also if there is no selection it will rebuild everything. So in that scenario selection is optional.

Calling these "artifacts" now will come in handy later on since that's what I plan to call it when there's more of an online sharing framework for crowdsourcing art. I call them digital artifacts generically even in other projects I have outside of SOM. I think that's an especially apt description when you get into restoring old games, since that's literally working with archeological excavations which is where the word "artifact" is used most exclusively.

I'm not sure how long this will take to wrap up. It's not a very fun thing to work on. I just hope it goes by fast. Like I've already said the thought of changing how SOM distributes its art files is nerve-racking. I have to be sure I get it right since if this cache system breaks anything it will make SOM next to useless until fixed, and there's potential for data loss to go unnoticed.

I probably won't report anything else until there's a working system and I switch over to verifying as much as I can and converting everything that is eligible. Only MDO and MDL files will be considered in this round. When there's nothing left to do I will release it. This will be the main focus of the next release since the other things I've been working on lately aren't really something you can use out of the box. Plus the enhancements for SOM_MAP from last week.

Technically shields will be in that release but they're still kind of immature and I don't have animations for the shields that come with SOM. I've been a little risk averse to modifying SOM's built-in art work. I also don't want to add shield animations without adding heavy attack animations to its weapons. I'm not sure I'm qualified to do animation work at that level since I'm not sure I'm experienced enough or a good enough animator.

 2 
 on: October 20, 2021, 02:02:16 AM 
Started by Holy Diver - Last post by Holy Diver
update

Off-topic: On SOM_MAP I did a second demo update that improved mouse input and added graphics to the other two map screens that had just used flat squares.

On-topic: I have ObjEdit, EneEdit and NpcEdit working with MDL+MDO. It was kind of a chore because I'm trying to have them use the same code som_db.exe uses since their MDL code is identical to it, except every program has their code at different places, so everything has to collated in triplicate to do this. I'd like to find a more organized approach, or just hollow them out and implement them from scratch, but that seems like overkill for now, especially since I'm not animating them, so I've just done what's expedient if drudgework. I'm hoping I will get some inspiration about what to do after everything is further along.

Now that everything can display MDL+MDO the next step is to set up the automatic conversion process and finally convert everything. I'm a little concerned that my MM3D project is still in demo status and only hosting debug builds, but it's functional. I will be preparing a new release for it. At minimum I've added a blend modifier to its material system to match SOM and Assimp's.

 3 
 on: October 18, 2021, 02:05:28 AM 
Started by Holy Diver - Last post by Holy Diver
2021 update

These tools have now come a little ways further and recently here (http://www.swordofmoonlight.net/bbs2/index.php?topic=327.msg3225#msg3225) is news that SOM_MAP has finally gotten the same treatment (coming in the next release) so that the tools are now all on the "same page" graphically.

There's also improved antialiasing and supersampling and EneEdit and NpcEdit have gotten whole second pages (screens) in their editors. SfxEdit is still unfinished as the SFX system is still a difficult part of SOM, even with "Ghidra" SFX is hard to map out.

I'm finding today that EneEdit and NpcEdit appear to work identically to som_db.exe, so that it should be possible to use very similar techniques to quickly enhance them. You can think of them almost as som_db.exe minus all the game stuff. There may even be some vestigial stuff in them, depending on how that was accomplished and how much the compiler removed from them. In screen shots of an in house version of SOM that were (inexplicably) shown on SOM's most recent product page there was even an animation player. Since they're the same as som_db probably the animation can be driven in the same way. I wonder if there's actually parts of them that retain an animation UI. Their code is easier to work with than the main tools since they just use plain C almost like som_db.exe, whereas the main tools use MFC. Note, MFC is actually C++ but so are these tools. Although the DLC tools use the C++ library (at least in places) for memory management, som_db.exe seems to use C's malloc/free exclusively.

 4 
 on: October 14, 2021, 09:19:03 PM 
Started by Holy Diver - Last post by Holy Diver
Attachments * SOM_MAP Grayscale.png 
Bonus patch

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.5.zip

A few days ago I had an idea to work on giving definition to the other two map views that shown flat gray tiles, since they can be very disorienting, especially on a full map. A lot of unexpected things came of exploring/implementing this.

Before I forget, SOM_MAP now loads instantly. This is because it had kept two sets of icons, one turned 90 degrees. To make the turned versions it had used the notoriously bad GetPixel/SetPixelV APIs. I was surprised that using those to make turned icons could be the bulk of its load time, compared to opening/processing so many files (on an SSD drive) but it was. In the end I actually got rid of the second set altogether and used its memory to hold the gray versions of the tiles. To do this was a bit of a squeaker because the sideways tiles needed the PlgBlt API that doesn't offer any "ROPs" like the others, which is actually a problem for inverting the white tiles, but that's done the same way SOM inverts the multi-selection tiles. The 4 colored circle icons also had to be switched over to TransparentBlt to act like an overlay. Really this just removed the 4 corner pixels because there's very slight (unintentional) aliasing artifacts that are a different color, which I intend to remove from the language packs at some point.

Then because of this enhancement I had to finally add the right-button dragging function to the "checkpoints" screen. But I found that to be difficult without fully overhauling the new click logic (CtrlLock) on the main screen. And I found that even though I changed the button name to CtrlLock to be plainspoken, that it wasn't actually functioning as if the Ctrl button were automatically held down. I worked on this most of the morning today. It now works perfectly I think. And the Ctrl key state shouldn't be able to get stuck anymore.

The checkpoint screen now works with just the left button. It works better this way I think. It flips the first tile and sets drug over tiles to the same result. I think it would be nice to have a multi-select (Shift) mode for this. I can't tell if something is broken, but multi-select only seems to work with the keyboard arrow keys. I.e. shifting with the mouse doesn't form a multi-selection, but it can start one. I have to look into this, but I did all of this very fast and don't want to spend more time on it. It may be rough too, but I hope not. I just checked for keyboard input on the checkpoint screen, it seems to invert the tile, so that's good, now the mouse and keyboard use the same inversion logic.

Again, the big unexpected find/news here is load time in my book, but the rest is a big improvement too.

 5 
 on: October 11, 2021, 04:00:08 PM 
Started by Holy Diver - Last post by Holy Diver
EDITED: I've reuploaded (SomEx.dll) to add a fix (see next paragraph) and added a warning for a change to the games as a side-effect of this patch.

The fix has to do with "neighbors" on different elevations and cells without tiles (perhaps because their tile is on another layer) having been incorrect in the initial patch yesterday. There may still be some issues in this area.

 6 
 on: October 10, 2021, 06:15:50 PM 
Started by Holy Diver - Last post by Holy Diver
Attachments * SOM_MAP Direct3D 9.png 
New SOM_MAP demo!

WARNING: Installing this patch changes the timing of when textures are processed that can manifest as hiccups in the frame rate, that I intend to address in the next release. Since I was thinking about demoing SOM_MAP I hadn't thought about it when I posted this demo announcement. It may linger until the release after since that's going to deal with dynamic loading/streaming maps that will require some infrastructure for trickle loading stuff mid play.

http://csv.swordofmoonlight.net/SomEx.dll/1.2.3.5.zip
http://csv.swordofmoonlight.net/Exselector.dll/0.0.0.1.zip

This has been a long release period, so I'm putting out another demo that can really make SOM_MAP a much better experience, so I highly recommend it to anyone actively using SOM.

In the scheme of things this work on SOM_MAP is only about a 1wk or 1.5wk stop on the way, a little diversion, before moving onto the next leg of the next release's goal.

I will say something about it in a second, but I'm also uploading a new DLL called Exselector.dll that I don't recommend, but it can be used to try OpenGL with SomEx by setting directx_version_to_use_for_window=32 in the [Window] section of the Ex.ini file. There's not much use in this, but it's proof I actually worked on it for the past 3mo :rolleyes:

EDITED: Exselector.dll is so huge because of ANGLE. To try angle use 95 instead of 32, but it will be very bright because of a bug (and slow) but the bug was announced fixed 5 days ago, so I just have to download the new source code (assuming it's publicly available) and rebuild ANGLE (which is a pretty big task) to fix this problem, in which case I'll reupload a new copy of the same file to the same place that will fix the brightness problem.

I think I'm going to check-in some CODE too, since it's been a while since I last did so. Speaking of that, when I uploaded these DLL files, it was the first time I uploaded a large file with my new fiber Internet service, which really took me by surprise by how fast it went!

There are some critical fixes in this demo for using the new layer system. In the screenshot I've attached you can see 2 layers (of 7 total) for the first time visible together in SOM_MAP. And also notice there are "objects" on display that belong to the outer 8 tiles, which is a new first that can be toggled with the new "neighbors" button.

It's hard to tell in this image but it also has accurate lighting for a change. Because lighting can tend to look much darker in SOM_MAP than in the game (even though the pixels are the same, except for some effects) I've increased the starting brightness some so that it's not necessary to manually do it every time you turn on SOM_MAP. To control the lighting you need a mouse with a middle button, and ideally a fast wheel. I think intend to add the ability to hold the middle button down and drag to adjust the brightness, but it's not there yet, so a wheel is required. Pressing the middle-button will reset the brightness to the default, but this will be less than the extra added brightness I mentioned, since the default reflects the real lighting as you see in-game.

I'm going to add an extension to control this startup brightness with. The starting boost also applies to if you disable lighting. I've reduced the disabled lighting in SOM_MAP so it's not blinding, and I would do the same for the game if I knew where to look in MapComp to make the change (as soon as I stumble across it.) SOM_MAP reflects the disabled lighting, although I'd like to have a way to disable it just in SOM_MAP and not in the game too.

Finally you can also see transparency in the water. Those are MSM models (not objects) and how material properties are being assigned to the level geometry is to make a MDO file in the DATA/MAP/TEXTURE folder that has the properties you want. Then you can either name that file to match the texture (TXR) you want to assign those properties to, which will take precedence, and if not that, then just assign the textures you want to the polygons in that MDO file. The actual model(s) can just be a square swatch. There's a way to assign a UV animation to them too by attaching a fin to the square, but I'm not going to go into that here. Transparency also works correctly for the other elements. It did not before. To do it correct it needs to draw the models in two passes (transparency second) and sort the transparent elements.

The 3D parts of SOM_MAP are recreated in Direct3D 7 code, so that after this all of the tools are using D3D7, which is then translated to D3D9, so it's really D3D9. Originally SOM_MAP and SOM_PRM were programmed in D3D6. You no longer see SOM_PRM's model viewer since a few years ago because it now uses the DLC editors to display the models, which are written in D3D7. You know I found out just a month or so ago (or less) that D3D7 was either not yet out to the general public when SOM was published, or had just came out, so From Software's staff either had early access or really scrambled to upgrade the player component to D3D7.

The new D3D9 path has most of the graphics effects of the player component. In this screenshot it even has the chunky (2x2) dither effect that I don't necessarily recommend for the tools, but I haven't tried to write anything to have the tools use a different dither size than the game does. My KF2 project is using it. Everything is antialiased and I really had to get creative to make the image sharp and fix a Microsoft bug in the line drawing system that the grid and RGB axes use that would kick in when getting the "camera" too close to its lines.

 7 
 on: October 09, 2021, 01:56:00 PM 
Started by Holy Diver - Last post by Holy Diver
follow-up

I was able to set up ambient/directional lighting and transparency/sorting today, however one problem that arises from lighting is SOM_MAP's little scene viewer is most likely surrounded by a vast white desktop, so its little dark box appears much darker than it really is to your eyes as they cope with the contrast difference (and probably your monitor is playing some contrast tricks too) so I can't publish it as is.

With my KF2 project it's not so bad, but dull, whereas with the little test project (someone else made) in the other day's post it has a pretty standard even darker "dungeon" atmosphere that feels too dark in SOM_MAP.

I think my plan is to use the mouse wheel to be able to raise the ambient setting artificially, assuming everybody has a wheel, which is currently how the lighting level is adjusted in SOM_PRM. I would like to provide a fallback for no wheel (or even use the wheel to zoom someday) but I'm not having any bright ideas.

Eventually I intend to try to have right-clicking the viewer pull up the lighting window so you can adjust the lights with a 3D view handy. I'll see if that makes the cut. Right-clicking in SOM_PRM currently pulls up a color selection menu for the background fill, so it's not that far off. (Ultimately I intend to add a right-click menu that will instead offer these options and a way to open the Exselector companion tool. It's going to pose a problem for turning tiles in SOM_MAP, but that's the only place right-click is used. On the other hand it will prevent accidental turnings by default, which is probably a good thing.)

 8 
 on: October 08, 2021, 06:55:43 PM 
Started by Holy Diver - Last post by Holy Diver
If anyone cares, I found a couple glitch type bugs in the new layers system around selecting the map elements, so I think I will upload a SOM_MAP demo tomorrow if I can get some lights working and do a little bit extra with transparency by the end of the day. (Edited: SOM_MAP doesn't have  a system for deferring transparency for a second pass, but it doesn't even do that for the individual models.)

So far I have added the ability to see elements on neighboring tiles and see all of the layers in the same column.

I don't think anyone else is using the layer system (beside myself) so I'm not too worried about these glitches, but I'd still like to show off these new developments, and it's not quite enough to make it a new release. Technically this work on SOM_MAP is a minor diversion to what I'm preparing.

It's a big help to be able to see the layers, especially if you were using them to not for floors but to intersect so that you can for example control the ceiling height independent of the floor height.

Unfortunately there's no way to edit the layers at the same time. It might be nice to be able to adjust their elevations at the same time. Other than that there's no real reason to. The layers are on separate MAP files, so switching layers makes it necessary to save the current MAP file.

 9 
 on: October 05, 2021, 03:01:45 PM 
Started by Holy Diver - Last post by Holy Diver
Attachments * SOM_MAP Diretect3D 9.png 
EDITED: It turns out what I thought was wire-frames data was bounding-boxes, i.e. wire cubes fit to the extents of the models.

I got those MDL files drawing, but ended up spending the most time on a bug in Microsoft's ID3DXLine line-drawing system, that only half works because if you walk through the lines they glitch out if one point on the line is in front of you and the other is behind you.

I left that bug in ItemEdit's item drop setup screen but it couldn't stand for SOM_MAP. All told this project has taken a solid week. I sometimes reflect on how maintaining SOM is unconscionably time consuming but it keeps me humble at the same time.

I may add a checkbox to turn on these boxes. I'm not sure how useful they are though. There's already little triangle checkboxes, so I figure a square icon beside them won't hurt. It can sometimes help to see things, especially from underneath/behind.

EDITED: Here's a preview of the image quality. Dither can be disabled with do_not_dither [Editor] but is on by default.

 10 
 on: October 05, 2021, 03:39:32 AM 
Started by Holy Diver - Last post by Holy Diver
update

For main work I just have to rewrite the MDL drawing code (for SOM_MAP) I've already annotated it in Ghidra. Yesterday as everything was coming together I was still disappointed because it was very blurry compared to the original look.

I think maybe my Intel chipset is too aggressive with mipmapping, but I don't know what's going on exactly, and assume mipmapping it pretty standardized these days. (Edited: in the game it seems to draw from mipmaps even where it shouldn't.) To remove most of the blur I ended up resorting to not using mipmapping in the tools, and it's surprising how much of an absolute improvement that is. I'm a little torn on mipmapping, I wonder if it's really ideal (yes anisotropic filtering is maxed out at 16x) or if there's a hard visual penalty for applying it in favor of performance.

The problems that arise from pulling back on mipmapping even a little happen with moving images, so in a tool, it's not really an issue, since it only moves while you rotate the view, etc.

As a result the tools can look super crisp. I also disabled the do_smooth effect with supersampling (that's currently forced on in tools) that I'd left in for the tools because I thought it helped, but made them less crisp. Some months later I noticed that the high contrast reduction effect was being applied to to the tools, when it was only meant for games, and I think actually that's why using the smoothing effect helped back then, but when I disabled it in the tools (fixed the bug) I wasn't thinking about this experience at the time.

I'm going to look at increasing the mipmap bias instead of disabling mipmapping entirely. There are some visual artifacts as a result of disabling it, especially apparent on SOM's textures that have seam issues in need of fixing.

Both of these enhancements should apply to SOM_PRM's viewers too. Also it turns out I didn't need to do any of this to be able to display MDO files for NPCs in SOM_MAP, but I wanted to do it anyway, now seemed like a good time, ever since adding the new viewers to SOM_PRM. This will make the entire experience have a consistent look and feel. But it turned out that all of the models in SOM_MAP use the same data structure (though different drawing routines) so it would not have been difficult to stick a MDO model in place of a MDL model (or even in place of an MSM model.)

Also there's actually a mysterious 4th model format associated with the mystery "5th category" from my previous reply (Reply #11) but it turns out I should have looked at that subroutine more closely, because there's actually only one model being drawn in that category, whatever it is. It can also be represented by a MDL file. Unfortunately it's much harder to decipher the tools because they use the (bad) MFC (object oriented wrapper for Win32) that's very circuitous and doesn't do anything directly, it makes tracing stuff backwards to their source virtually impossible. There's so much bloat in MFC 80% of the code is just meaningless ritualistic paper shuffling to wade through. (It's not a good programming paradigm either.)

Edited: Being a single model the most likely explanation for the "5th category" is the sky box, but it's hard to see why it would be included in a level editor. Also SOM uses MDO for its sky boxes, but it uses MDL or the "4th model format" but it's possible that's just a variant of MDO, although it could be a long lost format too... although its type ID comes after MDO and MDL's.

Pages: [1] 2 3 4 5 6 7 8 9 10