Jump to content
OWA

W3D Hub 2018 Roadmap

Recommended Posts

Wow... 2018... I gave up map making after my last computer kicked the bucket, but with the announcement of new tools I think I might give it another try. 

Looking forward to all the amazing stuff!

(Reborn roboot sounds Epic)

(RA 2 is making excellent progress!)

(keeping fingers cross for soviet migs)

Share this post


Link to post
Share on other sites

Reborn... Reborn this year? :v Colour me excited. I do generally hope there will be more media updates this year, if possible.

Share this post


Link to post
Share on other sites

I am looking forward to what is ahead this year. I wish you all at W3D the best of luck in getting those plans and goals accomplished.

Share this post


Link to post
Share on other sites

The roadmap is looking really good, and I can't wait to see some of the new stuff in action.

I'm really glad that there's tools being worked on to replace the ancient LevelEdit and Max 8 W3D exporter, should make modding W3D a whole lot easier with the more modern toolchains (especially 3ds max, it's such a pain building stuff in a newer version of Max and having to backport it to 8... not fun.)

Share this post


Link to post
Share on other sites

Great roadmap guys, and yes I am absolutely interested in new tools. I might not have mentioned this, ever, so this may come as a surprise to some, but I actually, really, bloody hate the current tools and their bucket full of issues! I figured I'd mention it, you know, in case I never have before.

A new W3D asset plugin would be great as well. Saberhawk was dreaming big, too big perhaps, when he intended to effectively change how materials and meshes are set up. Sure, we need additional material features such as normal mapping and so on, but surely this can be achieved by updating the tools and engine separately rather than making some giant unwieldy mega system akin to current-era, massively funded, engines? The only merit I saw in his proposals, was making sure the engine didn't call materials of the same name more than once, which would mean materials would probably best be made in a universal collection, stored somewhere, and referenced through the 3D modelling plugin.

I'd also be interested in a Blender version. As mentioned, Blender is "free", while current-era 3DS Max is "holyshitthatsexpensiveeverymonth" and realistically only accessible to students these days who get free access. From a purely financial perspective 3DS is making itself obsolete, which is a darn shame.

Lastly, the "freelance" work style is more or less what me and Pushwall already arranged in not nearly so many words. I am currently working on a different project which has some slight urgency attached to it (here is a small preview of that, this is a demonstration level background, and not an intended play space), but I fully intend to do further contributions for APB/W3D after that.

By the way, you are missing something in your roadmap: We found a way to clone Jonwil&Team!

Edited by Raap

Share this post


Link to post
Share on other sites
29 minutes ago, Mojoman said:

@Raap working on No Man's Sky 2 confirmed.

I don't know that game, so no. :)

(Edit: I just realized that cactus's in real life always grow straight, woops!)

Edit2: I have long intended to make a project of my own, and while my current focus is merely a prototype on another engine, if I could find a dedicated C++ programmer to help me out then I wouldn't shy away from using W3D as a platform over something else. There are a few 'ifs' linked to that though, aside of locating a programmer. My main concern with W3D, as always, is the network code and specifically the infantry component, as well as the severe limitations of how W3D handles infantry meshes. But I think we can all mostly agree that W3D has a pretty strong heavy ground vehicle departement, in contrast.

Closing note; Working with a non-W3D engine has benefits, but I've certainly learned to appreciate the open-book nature of W3D by working with something that is not W3D. If only the platform had more hands on deck to push it forward.

Edited by Raap

Share this post


Link to post
Share on other sites
6 hours ago, Raap said:

as well as the severe limitations of how W3D handles infantry meshes.

What limitations are you referring to here, if I may ask?

Share this post


Link to post
Share on other sites
4 hours ago, OWA said:

What limitations are you referring to here, if I may ask?

I think Pushwall summarizes it pretty well so let me quote him on this;

Saberhawk's perfdocs insists that WWskinned meshes should have "as few polygons as needed to look right" because they are deformed and uploaded to the video card every frame, so we may be looking at considerable performance drops if we get a bunch of more modern character models with like 10000 polys instead of the current 500-1000ish.

It was a big pain getting the current infantry to hold their weapons in a remotely sensible fashion (remember pre-Delta spies/thieves/RS doing kung fu with their pistols? Rocket soldiers with floating LAWs? Medics with broken wrists? All that?) I'd hate to have to redo that on any new infantry who have a different hand structure, arm length, arm position, and 20 times as many vertices :/

(...)

Every part of the infantry needs wwskin or else the unskinned parts will just float in the same spot no matter what animation your character is in. 

(...)

A bunch of separate accessories = more draw calls. Combined with the mega polycounts and high texture resolution of more modern models, that'd probably be disaster combined with the way wwskin works. A spam of accessories with different textures are kind of a problem that the oldest infantry in the game suffer from (allied sniper has separate meshes/textures for body, head/hands, bandolier, holster and pouches... and really I can get rid of the holster now that they don't have pistols) but the performance problems of that would just be exacerbated with high polycount wwskin models that are made out of dozens of different accessories, for every infantry instead of just the oldest ones (the new ones like Allied inf and engineers have a mere two meshes/textures).

TLDR: Infantry handling in terms of literally every aspect on W3D is just plain horrid.

Share this post


Link to post
Share on other sites

And this is why I hate this forum software: I cannot edit the above post properly anymore... The "Edit" isn't what Pushwall said but something I added below the quote, but this forum added it to the quote post-submission and in a form that is completely un-editable. Just to clarify, I hate miss-quoting. Try moderator editing that TLDR line... You cannot even access it because it is embedded into the quote frame itself. This is what I asked source view for a while ago!

... Anyhow, infantry models are a pet annoyance for me. I consider it the doorway into any W3D game, it is the first thing players get to see. In a perfect world, WWskin is removed and a new animation system entirely is created. With WWskin out of the way, the door is opened for much more infantry detail and mechanics.

Edited by Raap

Share this post


Link to post
Share on other sites
5 hours ago, Raap said:

And this is why I hate this forum software: I cannot edit the above post properly anymore... The "Edit" isn't what Pushwall said but something I added below the quote, but this forum added it to the quote post-submission and in a form that is completely un-editable. Just to clarify, I hate miss-quoting. Try moderator editing that TLDR line... You cannot even access it because it is embedded into the quote frame itself. This is what I asked source view for a while ago!

... Anyhow, infantry models are a pet annoyance for me. I consider it the doorway into any W3D game, it is the first thing players get to see. In a perfect world, WWskin is removed and a new animation system entirely is created. With WWskin out of the way, the door is opened for much more infantry detail and mechanics.

I've found a fixed a bug that was preventing members from editing their own posts, so that should be working now. We did investigate adding source view, but with IPB deprecating BBCode and the only way to enable a source view is to expose the HTML viewer, then it could be a bit of a security risk.

You'll be delighted to know that WWSkin has been completely annihilated by the new exporter. When the new tools are ready we will be using Max's native skin modifier for rigging, which is a whole lot better. Currently WWSkin allows us to interpolate between two bones, but the Max Skin modifier could allow us to interpolate between more than that which could be really good for characters.

As for performance concerns, AR uses high poly (for the W3D Engine) characters for the Conscript and GI and the performance impact is negligible.

Posing infantry has always been awkward because we lack animators, but if we were to recruit just one competent animator, then a lot of our weapon pose issues would be solved.

Share this post


Link to post
Share on other sites
6 hours ago, OWA said:

I've found a fixed a bug that was preventing members from editing their own posts, so that should be working now. We did investigate adding source view, but with IPB deprecating BBCode and the only way to enable a source view is to expose the HTML viewer, then it could be a bit of a security risk.

You'll be delighted to know that WWSkin has been completely annihilated by the new exporter. When the new tools are ready we will be using Max's native skin modifier for rigging, which is a whole lot better. Currently WWSkin allows us to interpolate between two bones, but the Max Skin modifier could allow us to interpolate between more than that which could be really good for characters.

As for performance concerns, AR uses high poly (for the W3D Engine) characters for the Conscript and GI and the performance impact is negligible.

Posing infantry has always been awkward because we lack animators, but if we were to recruit just one competent animator, then a lot of our weapon pose issues would be solved.

How do you intend to solve character animation in W3D though? From my understanding using video card memory is a very outdated method and newer cards don't care for it much - correct me if I am mistaken here. As for the actual animations, well by eliminating WWskin on the modeling side, and utilizing a more mainstream solution, wouldn't that, theoretically speaking, open the door to motion captured animations?

I'm also not grasping how you can make W3D understand non-WWskin animated meshes, I presume that engine-side developments are to be paired with this?

As for the forum, let me show you the bug I hit consistently:

VZrSnZd.png

Post-submission anything below certain quotes become embedded into an area within the quote that cannot be edited or touched, or not even deleted, the post is just completely broken when this happens. Using Chrome.

Edited by Raap

Share this post


Link to post
Share on other sites
6 minutes ago, Raap said:

How do you intend to solve character animation in W3D though? From my understanding using video card memory is a very outdated method and newer cards don't care for it much - correct me if I am mistaken here. As for the actual animations, well by eliminating WWskin on the modeling side, and utilizing a more mainstream solution, wouldn't that, theoretically speaking, open the door to motion captured animations?

I'm also not grasping how you can make W3D understand non-WWskin animated meshes, I presume that engine-side developments are to be paired with this?

I'm not sure about the memory-allocation stuff, so I can't really answer that one.

So as far as I'm aware, W3D looks for skinned meshes within the file format and WWSkin is used to translate skinned weights into the W3D file format. So if the Max Skin Modifier is able to be exported and translated to skinned weights within the W3D file format, we eliminate the need for WWSkin. This does open up W3D for motion capture, CAT rigging, Inverse Kinematics and all of that good stuff. :)  Basically the key is how the skinning data is exported to the W3D file format rather than how the engine interprets it.

I could be totally wrong, though. @cfehunter knows more about the technical side of things so I'll try and get him to comment.

Share this post


Link to post
Share on other sites
3 hours ago, OWA said:

I'm not sure about the memory-allocation stuff, so I can't really answer that one.

So as far as I'm aware, W3D looks for skinned meshes within the file format and WWSkin is used to translate skinned weights into the W3D file format. So if the Max Skin Modifier is able to be exported and translated to skinned weights within the W3D file format, we eliminate the need for WWSkin. This does open up W3D for motion capture, CAT rigging, Inverse Kinematics and all of that good stuff. :)  Basically the key is how the skinning data is exported to the W3D file format rather than how the engine interprets it.

I could be totally wrong, though. @cfehunter knows more about the technical side of things so I'll try and get him to comment.

I will take your word for it! Ultimately only the end result is most relevant to me, not the road taken to get there (unless said road is a muddy trench through which traversing is difficult).

I look forward to testing the new editor... The W3D plugins will have to wait since I'm still stuck with 3DS Max 8 for now. :)

Share this post


Link to post
Share on other sites

@OWA Surely someone out there has developed a plugin of sorts to have a source-code viewer without the security risk?

Share this post


Link to post
Share on other sites
20 hours ago, Raap said:

How do you intend to solve character animation in W3D though? From my understanding using video card memory is a very outdated method and newer cards don't care for it much - correct me if I am mistaken here. As for the actual animations, well by eliminating WWskin on the modeling side, and utilizing a more mainstream solution, wouldn't that, theoretically speaking, open the door to motion captured animations?

It depends on the workload. For renegade GPU skinning is probably the right choice as the GPU isn't the performance bottleneck and most skinned objects are using common skeletons.

We can always move to CPU skinning, but our min-spec is a single core CPU and in that particular case it would absolutely murder the frame rate.

Share this post


Link to post
Share on other sites

Very fair point on the CPU usage, and I imagine we'll not see any changes there, anytime soon. :)

I have two more comments regarding the plugins and editor. Firstly, could you suit up both the plugin and editor with as much informative tooltips as possible? If you want to attract new contributors to W3D projects, it would help if they knew what the various material settings did. Good tooltips often include examples such as saying "enabling this causes X and disabling this causes Y". Maybe make a pass on obsolete features as well. AFAIK "shininess" and "translucency" both do nothing, and in the shaders/texture tabs there is a trainload of settings with dubious purposes.

Secondly, the editor itself, would it be possible to upgrade the script attachment window? Search function, easier to use input fields, the ability to link objects in reference directly instead of using ID's (meaning as well to add 'go to object' and 'add selected as reference' buttons), easier translation database references (we really should end the practice of adding strings as script parameters), etc.. All mostly to make scripting easier and allow for bug tracking from one window frame instead of script attachments on several individual objects... Hell, maybe make scripts smarter in general by detecting bad input and flagging it before you can export a level.

It wouldn't be visual scripting by any means, but just general QoL improvements.

Edited by Raap

Share this post


Link to post
Share on other sites

I can't really comment on the the editor.

 

Shininess is supposed to impact specular intensity, but when the engine was moved over to DX9 with a DX8 emulation shader, instead of being actually DX8, the specular was broken and it hasn't been fixed since.

Translucency likewise actually should work, but it'll only effect specific material types.

Share this post


Link to post
Share on other sites

Having such things fixed and better documented would help, like what types of materials are affected by translucency? Not even I know that and I worked with this engine for literally over a decade.

Share this post


Link to post
Share on other sites
On 4/5/2018 at 8:27 AM, Raap said:

A new W3D asset plugin would be great as well.

A new asset plugin is somewhat required.

On 4/5/2018 at 8:27 AM, Raap said:

Saberhawk was dreaming big, too big perhaps, when he intended to effectively change how materials and meshes are set up. Sure, we need additional material features such as normal mapping and so on, but surely this can be achieved by updating the tools and engine separately rather than making some giant unwieldy mega system akin to current-era, massively funded, engines? The only merit I saw in his proposals, was making sure the engine didn't call materials of the same name more than once, which would mean materials would probably best be made in a universal collection, stored somewhere, and referenced through the 3D modelling plugin.

Let's dig in on the engine details a bit before we call things giant unwieldy mega systems with no merit, shall we?

On 4/5/2018 at 9:34 AM, Raap said:

My main concern with W3D, as always, is the network code and specifically the infantry component, as well as the severe limitations of how W3D handles infantry meshes.

So, skinned mesh rendering? The engine originally shipped with CPU-side rigid (i.e. one bone) skinning. It was stuck at this for a while because the contemporary exporter only supported rigid skinning, which doesn't work great for actual skin. The BFME version of the exporter added support for two-bone soft skinning by duplicating the rigid skinning system and storing an extra set of vertices optimized for rigid skinning along with the extra bone IDs and weights required. This is incredibly memory bandwidth and space intensive and doesn't scale very well to soft skinning with more than two bones. It does look better though, and the engine was updated to read and use this "new" type of skinning data.

On 4/6/2018 at 9:34 AM, OWA said:

So as far as I'm aware, W3D looks for skinned meshes within the file format and WWSkin is used to translate skinned weights into the W3D file format. So if the Max Skin Modifier is able to be exported and translated to skinned weights within the W3D file format, we eliminate the need for WWSkin. This does open up W3D for motion capture, CAT rigging, Inverse Kinematics and all of that good stuff. :)  Basically the key is how the skinning data is exported to the W3D file format rather than how the engine interprets it.

I could be totally wrong, though.

You are. The purpose of WWskin was to make the content in the content authoring tool behave like content in the engine does. Things like inverse kinematics (i.e. feet following terrain) require engine level support when not faked via baked hierarchy transform animations. Which is what mocap data generally is, until you get into facecap, and already supported by the engine.

4 hours ago, cfehunter said:

It depends on the workload. For renegade GPU skinning is probably the right choice as the GPU isn't the performance bottleneck and most skinned objects are using common skeletons.

We can always move to CPU skinning, but our min-spec is a single core CPU and in that particular case it would absolutely murder the frame rate.

k. PS: Whatever you do, don't log how long MeshModelClass::get_deformed_vertices(Vector3* dst_vert, Vector3* dst_norm, uint32 stride, const HTreeClass* htree) takes as it hits the slow weighted skinning case many times each frame in a multiplayer match. 

2 hours ago, Raap said:

Firstly, could you suit up both the plugin and editor with as much informative tooltips as possible? If you want to attract new contributors to W3D projects, it would help if they knew what the various material settings did. Good tooltips often include examples such as saying "enabling this causes X and disabling this causes Y". Maybe make a pass on obsolete features as well. AFAIK "shininess" and "translucency" both do nothing, and in the shaders/texture tabs there is a trainload of settings with dubious purposes.

Getting rid of confusing features like the W3D fixed function emulation übershader generator would really help.

1 hour ago, cfehunter said:

Shininess is supposed to impact specular intensity, but when the engine was moved over to DX9 with a DX8 emulation shader, instead of being actually DX8, the specular was broken and it hasn't been fixed since.

Translucency likewise actually should work, but it'll only effect specific material types.

An audit was performed on a large library of .w3d files and per-vertex specular lighting wasn't found, so I didn't bother to both create test cases and implement accurate emulation of those test cases to prevent the übershader from exploding much more than it already had. 

23 minutes ago, Raap said:

Having such things fixed and better documented would help, like what types of materials are affected by translucency? Not even I know that and I worked with this engine for literally over a decade.

I wish I knew what all the various material settings really did. The options in the exporter GUI don't always match up with the data contained in exported files. Old exported files need to keep working, they don't just disappear. Exported files contain lots of features not exposed by the exporter, such as alternative materials. The rendering system runtime data is very different than the stored data. For example, all texture coordinates are stored upside down and need to be flipped. Why? Because Renegade originally used SurRender 3D which followed OpenGL math conventions instead of Direct3D ones. These quirks in the file format can't really be fixed without updating all the required related tools, like the level editor. Which we couldn't really update because the GUI used MFC6 which is now two decades obsolete and incompatible with newer compilers. The engine also performs plenty of fixups for "oh noes, we exported the file but the settings are wrong and we can't export a new version" and fog.

After all of that, the post-processed mesh finally makes it to the step of generating #defines for the shaders that it needs to compile to render triangles with that particular combination of settings. This happens with the source code found here. https://gist.github.com/saberhawk/199235c38a2b1051f06f0340cec9b5b4#file-ff_generator-cpp-L22. The rest of the gist contains the übershader code. There are some optimizations to prevent compiling a particular shader twice and to load compiled shaders instead of compiling them as required, but there are still a ridiculous number of combinations. 

And then the actual D3D11 rendering engine just uses whichever shader (i.e. compiled .fx code, not the "shader" in the W3D exporter) and a combination of constant buffers and textures, some static and some dynamic. Which is so much more powerful than the limited options from the exporter, and what those options eventually end up in anyways so they can work. Is it really that much of a dream for tools to just use that directly instead of maintaining the fixed function emulation übershader generator?

Share this post


Link to post
Share on other sites

I think it's safe to say that if W3D hub had the manpower/resources, a "redo" would have been pursued on quite a few things. 

All I can do is highlight my own issues as a user of the tools. The inner workings, put simply, are well beyond my skill set.

But then again, this is why updates like this should occur more frequently. I doubt that there is no one out there that would be willing to contribute to W3D as a platform, as long as things remain clean and controlled.

So perhaps "W3D Hub" should promote a development platform for non-profit multiplayer games more aggressively than just the games themselves. The games can be a showcase of engine flexibility, but to attract the right people, you got to change your strategies. 

Edited by Raap

Share this post


Link to post
Share on other sites
Just now, Raap said:

I think it's safe to say that if W3D hub had the manpower/resources, a "redo" would have been pursued on quite a few things. 

All I can do is highlight my own issues as a user of the tools. The inner workings, put simply, are well beyond my skill set.

But then again, this is why updates like this should occur more frequently. I doubt there there is no one out there that would be willing to contribute to W3D as a platform, as long as things remain clean and controlled.

Frankly, it would be easier to port the game code to a new engine than re-write the core game systems *and* keep them compatible with stock renegade.

That said, there's plenty that can be tweaked.

Share this post


Link to post
Share on other sites
20 hours ago, saberhawk said:

You are. The purpose of WWskin was to make the content in the content authoring tool behave like content in the engine does. Things like inverse kinematics (i.e. feet following terrain) require engine level support when not faked via baked hierarchy transform animations. Which is what mocap data generally is, until you get into facecap, and already supported by the engine.

I figured as much. Though fake bake was generally what I was alluding to, since the artist would still need to export with a skin modifier in the stack. It's just a lot easier to convert IK to be used with the much more broader-used Skin modifier rather than W3D engine-specific WWSkin.

Share this post


Link to post
Share on other sites

YES FINALY TIBERIAN SUN REBORN 2.0 AND IT MAY COME OUT THIS YEAR ITS LIKE CHRISTMAS IS ALL YEAR ROUND FOR ONE HOLE YEAR  GREAT JOB WITH THE GAMES COMING OUT AND IN IMPROVENTS!!!!!!!!!!!!!!!!!!!!!!!!!!!! great logo as well 

Share this post


Link to post
Share on other sites

If you wish some new music for Tiberian Sun Reborn 2.0, I can help =) By the way... keep working also on W3D-import and WWSkin, that would be incredibly cool! Nice updates for CLE too, I am always waiting for some new tools for C&C Renegade in general after I joined modding community in 2006.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×