Fell Seal: Arbiter's Mark

RTX Remix - Q&A with NVIDIA

  • Comment
In case you haven't heard yet, NVIDIA's new modding tool - RTX Remix - got announced during their September keynote. It's a fantastic-looking application that has the potential to revolutionise graphical modding for old games. We're joined by Nyle Usmani, a Product Manager from the team behind the tool, to give you the opportunity to have your burning questions about RTX Remix answered directly from the source.



If you're not up to speed yet, here's a clip from the keynote and the video above provides an in-depth look at how it works.

NVIDIA RTX Remix is a free modding platform built on NVIDIA Omniverse that vastly expands the modding capabilities of compatible DX8 and DX9 games. With Remix, modders can use a combination of AI texture tools and their own artwork to replace textures, objects and materials with high-quality versions. It also allows modders to inject modern lighting techniques like ray tracing, DLSS 3 and NVIDIA Reflex into the game. 

Perhaps the biggest and most exciting thing about RTX Remix is that it effectively enables modding for titles that were either harshly limited or almost impossible to mod beforehand!


So how does it work? Modders can start the game with RTX Remix running alongside it and at the press of a button, all the render data for textures, geometry and lighting data is captured and converted to the Universal Scene Description (USD) format which is then loaded into the application. From there, modders will be able to replace assets, update textures and tweak the lighting to their heart's content. It also features AI-assisted upscaling tools to supplement the modder's workflow. Once they're ready to share what they have created, all the changes can be bundled into an RTX mod.

For players, you simply install the RTX mod near your game executable and when you boot up, the changes made in Remix will be automatically applied. That's not all though. RTX Mods will also include an in-game configuration menu that allows the player to tweak lighting, reflectivity and other values on the fly to achieve their desired effect.

RTX Remix will require modders to be using a ray tracing capable NVIDIA GPU to create the content (RTX 20 series or above) but players can install and play RTX mods with any GPU that supports Vulkan Ray Tracing. 

You can see a great example of what's possible with RTX Remix by looking upcoming Portal RTX trailer or the Morrowind RTX Mod showcased during the keynote. There's no confirmed release date yet, but we're optimistic that you'll be able to get your hands on RTX Remix sometime in early 2023.



NVIDIA is excited to see how modders will take advantage of RTX Remix and has joined us in this post to answer your questions from the comments. Before we get to that though, here's a word from Nyle.

Thanks Nexus Mods. Modders, we are working hard to make sure you don’t have to wait long to start creating with RTX Remix. When we made Quake II RTX and saw all of the amazing things the community built off of it, it was eye-opening. We knew our next project had to be more than an RTX remaster–it had to more importantly enable the community to apply their undeniable creativity to the diversity of classic content that is out there. From the games that have been difficult to mod, to community favorite titles and existing mods–RTX Remix has the possibility to reimagine all of these experiences in beautiful path tracing with high quality assets. We can’t wait to see what jaw dropping remasters and remixed mods you will create with this powerful tool. And with that, please feel free to ask whatever questions are on your mind, but please understand that I’m here to share more on RTX Remix and Portal With RTX and am not in a position to speak to other topics (GPU availability, price, performance for example). We can’t unveil everything just yet since RTX Remix is still deep in development, but we will do our best to answer all that we can.

To ask a question, simply leave a comment on this article - please take a minute to scan the existing comments so that we avoid too many repeated questions. Nyle will answer as many as he can, but please keep in mind the topics they can and can't discuss! 

Don't worry about digging through the comments for answers though, I'll be adding a copy of each question and answer to the spoiler in the pinned comment below.

98 comments

Comments locked

A moderator has closed this comment topic for the time being
  1. NVNyle
    NVNyle
    • member
    • 8 kudos
    Locked
    Sticky
    Thank you everyone, it’s been a real pleasure to finally get to address some of the more technical and in-depth questions from the modding community. We hope you have fun with “Portal With RTX” next month, the first RTX Mod being released. Everything in Portal With RTX was made with the help of RTX Remix and is glimmer of what’s possible with this modding platform. Please sign up to be notified when RTX Remix is out so that you can get to creating, and we can marvel at all of the creative things you build!

    Until next time,

    Nyle Usmani
    NVIDIA Product Manager
  2. Pickysaurus
    Pickysaurus
    • Community Manager
    • 649 kudos
    Locked
    Sticky
    Your Questions Answered 

    Spoiler:  
    Show


    I'm thinking about remastering Unreal Tournament 2004 - a highly beloved game! But seeing as it's a multiplayer arena shooter, I wonder how that'd work out. We'd have to remaster all the custom maps. When connecting to a server that's playing a map you don't have, would the game be able to detect the assets from RTX Remix that accompany the map and download them? Probably not right? What if we're connecting to a server that doesn't use RTX, or someone without RTX tries to connect to a server that does use it? Would they be able to play together? If you go from an RTX map to a non-RTX map, would the renderer handle it properly?
    Argonil

    Hey Argonil,

    We look forward to seeing your future experiments with RTX Remix! On to your questions:
    I'm thinking about remastering Unreal Tournament 2004 - a highly beloved game! But seeing as it's a multiplayer arena shooter, I wonder how that'd work out. We'd have to remaster all the custom maps. When connecting to a server that's playing a map you don't have, would the game be able to detect the assets from RTX Remix that accompany the map and download them? What if we're connecting to a server that doesn't use RTX, or someone without RTX tries to connect to a server that does use it? Would they be able to play together? 
    Without commenting on Unreal Tournament 2004 specifically, here’s how it should work.

    As long as players have the RTX Mod with remastered assets, they will see the maps with updated visuals. The server is uninvolved in the rendering process and is not relevant to whether users will see updated visuals–it’s all about the RTX Mod users have locally stored on their PC. Because the RTX Remix runtime is only influencing the visuals, players should in theory be able to play against each other even if some players are seeing the vanilla game while others are using RTX Mods.

    To correct a misconception, you may not have to remaster every custom map. In old games, many custom maps reuse the same textures and assets. RTX Remix should replace all of those textures and assets if the mod has replacements available, even if the level geometry is being seen for the first time.

    As for dynamically downloading RTX Mods in game for players who do not have those assets already–that type of functionality would have to be modded into the original game by the modder so that remastered assets are downloaded and put into the RTX Remix Runtime's asset folder. RTX Remix wouldn’t be involved in architecting such a system.

    Another thing to note–Anti-cheat software may prevent RTX Remix from working in certain competitive multiplayer titles.
    If you go from an RTX map to a non-RTX map, would the renderer handle it properly?
    The renderer would fall back to showing classic assets, wherever it has no replacement assets to point to.



    ?Will Remix include a library where we could use its renderer and insert it into other games ourselves by describing the scene's contents instead of relying on the fixed pipeline translation layer? Some games have enough reverse engineering work on them that it'd end up being easier to use something like this instead of backporting to fixed pipeline.

    Will it be possible to make custom implementations of the shading of surfaces (e.g. shader programming) to be able to recreate specific materials more easily?

    How will the USD replacement functionality work with skinned assets? Or will it only support static mesh replacement? Will assets that use vertex blending be exported with their matching bones and weights correctly? And what about games that use morphers for facial animations?

    What kind of scripting functionality will be available to control what lighting or other scene settings get used in a game? Would this provide a way to modify the scene in real time to match certain systems of the game (like a game with a dynamic day/night system)?

    Will there be any systems in place to deal with replacing assets in games that are capable of streaming and swapping different level of detail meshes?
    dariort64

    Hey DarioRT64, big fan of your mods!

    Very detailed questions!
    Will it be possible to make custom implementations of the shading of surfaces (e.g. shader programming) to be able to recreate specific materials more easily?
    This topic has come up a lot and we are deep in research, but there’s nothing to announce today.
    How will the USD replacement functionality work with skinned assets? Or will it only support static mesh replacement? Will assets that use vertex blending be exported with their matching bones and weights correctly?
    It depends on the implementation used by the game. For CPU based skinning, we get the post-skinned data in Remix, and while it renders correctly, it's a bit more challenging to replace these assets. It’s something we would like to look at closer in the future.

    For GPU based (fixed function) skinning, we actually have access to the pre-skinned mesh and the skeleton (including the collision volumes for raytracing) which gives us a great deal of flexibility to modify and reskin characters in RTX Remix.

    We saw some (incorrect) speculation that we did not include NPC’s in our Morrowind RTX showcase because we weren’t able to properly mod them. But Morrowind actually passes us pre-skinned meshes and the skeleton so it was perfectly in our capabilities to modify the NPCs. In order to give the artists time to do an incredible job, we narrowed the scope of the project to an indoor environment without NPCs.
    Will there be any systems in place to deal with replacing assets in games that are capable of streaming and swapping different level of detail meshes?
    Variable level of detail during streaming is an interesting case. Here's how it should work: you would need to take multiple captures at each level of detail, and RTX Remix would regard each asset at each level of detail as a unique instance. For example, imagine an asset has three levels of detail depending on the distance of the player–at 5 meters it’s high quality, at 15 meters it’s middle quality and at 25 meters it’s low quality. That object would actually be regarded as three separate objects depending on the distance of the player.

    For the modder, they have two options. Recreate all three assets with different levels of detail to give you a similar feeling as the original game. Or map a high quality asset to all three instances, which would help make the asset feel more at home in a modern graphical context.

    Hope that helps Dario. Can't wait to see what you can do with RTX Remix!



    One thing that concerns me is the requirement that the games have a Fixed Function pipeline, which excludes a great majority of games people would want to use these tools on.
    Notably for Nexus, New Vegas and Skyrim.
    What I'm really, really hoping is the case, is that it will only take some extra work to make games like that fully RTX-ready, perhaps missing out on some automated stuff that Remix would otherwise do, but I'm afraid that it will simply not work.

    Additionally, I saw in the previews that they were always shown off indoors, and with no NPCs or other dynamic objects. It can obviously handle at least some dynamic stuff given Portal with RTX's cubes, pellets, turrets, and things, but with Morrowind, there were no NPCs, and the outdoors was never shown outside of a vanilla shot for comparison.
    How does Remix handle games which have a dynamic day/night cycle, and which load/unload objects dynamically in a single area (Such as exploring the worldspace in Morrowind and travelling to different cells)? How is it able to differentiate between indoors/outdoors? Can you specify an object or vector as relating to the sun/moon, and have it adjust RTX parameters dynamically to match?

    Another thing is how it would deal with games that use culling. Will objects behind you stop casting their shadows because the game engine is no longer rendering them? Can it catch the whole scene in one go with that regardless, or would it require multiple 'snapshots'?
    DarianStephens

    Hey DarianStephens,

    I love technical questions like this.
    How does Remix handle games which have a dynamic day/night cycle, and which load/unload objects dynamically in a single area (Such as exploring the worldspace in Morrowind and traveling to different cells)? 
    I want to preface some of these specific questions by saying lots of aspects of RTX Remix are still in motion, and many games behave in unique ways depending on a host of variables including their lighting model. Not every title has been tested.

    For dynamically loaded objects, as long as the capture properly includes each object that is dynamically loaded in and out of an area, the replacements should also dynamically load.

    As far as day/night cycles go, that’s a trickier question. Assuming the celestial bodies are coded as directional lights, the ray traced conversion should retain that feeling of dynamic day and night cycles with more realistic shadowing and skies. From our outdoor tests in Mount and Blade: Warband, we’ve seen some stunning sunsets and sky lighting with realistically rendered light rays peeking through clouds–it’s entrancing to see that kind of thing in games from the 2000s!

    If it's a lit skybox it should also work. But developers in those days took a lot of shortcuts to simulate a lit sky and there might be certain approaches to doing classic lighting that we haven’t seen in our testing, and therefore haven’t yet addressed.
    How is it able to differentiate between indoors/outdoors?
    There shouldn’t be a need to differentiate between outdoors/indoors in a ray traced lighting model since lights begin to behave realistically. An indoor environment would simply possess more meshes and geometry that lights bounce off of, or that occlude lights to cast more shadows. Both outdoor and indoor environments would have to be relit with the context of realistic lighting, which is something we show in our RTX Remix overview video: https://youtu.be/Vg52-HZhrFc?t=163.
    Can you specify an object or vector as relating to the sun/moon, and have it adjust RTX parameters dynamically to match?
    On adjusting RTX parameters dynamically, it might help if you can provide us with an example of a game that we can research that would require something like that, to help illustrate the use case (I've sent you a DM). As for RTX Remix’s options with respect to celestial bodies, classic games can specify a directional light (vector + light parameters) and RTX Remix will ray trace it accurately. Or the original game could have a big mesh up in the sky and we can put an emissive texture on it, or even attach a big sphere light to it. There are a lot of options as it pertains to lighting outdoor environments.
    Another thing is how it would deal with games that use culling. Will objects behind you stop casting their shadows because the game engine is no longer rendering them?
    Culling is another case where there might be different interactions with RTX Remix depending on the game.

    As you’ve noted, if culling causes objects to cease to exist in the original game, then the RTX Renderer would also cease rendering them. Portal has built in settings that let us disable a lot of the more aggressive culling, which lets us avoid most of those issues. It still culls lights very aggressively, so we had to implement persistent static lights even after the game stops drawing them. There are other workarounds of course–for example, we can simply attach new static lights to level geometry and disable the original lights to retain the original game’s feel without breaking the immersion of full ray tracing.

    If other games don't allow customizing the culling settings, and the issue can't easily be addressed by traditional modding, then we may need to look into implementing a different system. For example, some kind of caching to improve object permanence. There are many paths to a solution and we will be paying attention to see if this becomes an issue for classic games.
    Can it catch the whole scene in one go with that regardless, or would it require multiple 'snapshots'?
    As it is currently implemented, a snapshot will capture a single frame of content which includes everything rendered, whether it is in view or out of view of the player, and every object will be cataloged (alongside the geometry, textures, lighting, cameras). Some games that render the entire level might only require a single capture, while other games that have aggressive culling might require more. RTX Remix will understand duplicate instances of the same asset through a game, so a modder may already have certain assets captured from a prior room when they begin a capture in a new room/area. For a game, you will have to take multiple captures, although a single area might only need a single capture.

    We did not get to every one of your questions but hopefully this demystifies things quite a bit. I should point out, another poster asked about skinning and NPC's, so definitely take a glance at the summary that Nexus Mods will post to see how RTX Remix handles that.



    ?Can it inject non-RT lighting and graphical features such as Global Illumination, Volumetric Lighting, and Screen Space Reflections?

    LZKiller7

    Hey LZKiller7,

    Btw, Killer7 was one of my favorite games on the Gamecube (always hoped the Smiths would make it into Super Smash Bros).
    Can it inject non-RT lighting and graphical features such as Global Illumination, Volumetric Lighting, and Screen Space Reflections?
    RTX Remix really started as a way to make full ray tracing (i.e. path tracing) accessible for modders, so that faking reflections or light could be a thing of the past. Big publishers already design their games with ray tracing in mind–we think the modding community should have that same power at their disposal.

    As such, in place of non-RT global illumination, volumetric lighting, and screen space reflections, RTX Remix offers highly customizable light accurate global illumination, volumetrics, and on/off screen reflections.

    There are some post effects that are non-RT that allow modders to tweak the experience like bloom, tone mapping, motion blur, but the focus is to empower modders with higher fidelity lighting.



    Does RTX Remix have an API which can be used to provide additional information to it, like weather parameters, lighting conditions, engine-provided material information.

    How are skinned/animated objects tracked? Are they replaceable?
    Doodlezoid

    Hey Doodlezoid, interesting question.

    Currently there is no such API but if there is interest for something like this, we can look to see if there’s anything we can do.

    How are skinned/animated objects tracked? Are they replaceable?
    We've answered a similar question about skinning in this comment section. Please take a look at that answer for more information :).



    ?This is truly the most powerful tool out there. And possibly the best thing is that it's actually free !
    I do have some questions though:
    Spoiler:  
    Show

    Performance: If adding more graphical features, effects, textures, triangles(from meshes) to the game, wouldn't it tax on the FPS more ? Or would the new optimizations such as DLSS 3 and reflex counter balance it?

    Is the RTX Remix only gonna offer DLSS 3 support or DLSS 2 as well ? 

    If we can edit meshes in the remix, then we should be able to also manually edit textures through outside software too, such as photoshop and others ?

    How powerful is the AI upscaler ? Can it be used for possible seamless texture generation ? 

    Will there be a documentation/tutorial on everything about the RTX Remix? 

    Lastly, are there any limitations that RTX remix would have?


    Thanks Nvidia for this amazing tool ! 
    RokHel

    Thank you for the kind words RokHel! We can't wait to see the response when modders actually get their hands on it. On to your (very good) questions:
    Performance: If adding more graphical features, effects, textures, triangles(from meshes) to the game, wouldn't it tax on the FPS more ? Or would the new optimizations such as DLSS 3 and reflex counter balance it?
    One of the benefits of turning RTX On in older games is that there’s a lot of GPU performance headroom to take advantage of to modernize graphics. Particularly with performance boosting technology like NVIDIA DLSS available from RTX Remix, you can achieve both beautiful graphics and a smooth (60+ FPS) experience. However, the extent of visual bells and whistles and quality of assets a modder chooses to include is up to their discretion.
    If we can edit meshes in the remix, then we should be able to also manually edit textures through outside software too, such as photoshop and others ?
    Absolutely. We recognize the best mods incorporate high quality custom assets, and that’s why we built RTX Remix on Omniverse–so that any major Digital Content Creation tool (including Adobe Photoshop, Blender, or Adobe Substance) could be used to edit the assets.We demonstrate using outside software in our RTX Remix Overview video: https://youtu.be/Vg52-HZhrFc?t=231For a full list of Omniverse connected applications, take a peek at the bottom of this page:https://www.nvidia.com/en-us/omniverse/?ncid=pa-srch-goog-574267&gclid=CjwKCAjwwL6aBhBlEiwADycBIKRo7oBbr4boAgA8BV6Rj7O1hybawzWGf26QhnVQmg-06dBMqUKrnBoCSFAQAvD_BwE#cid=ov01_pa-srch-goog_en-us?To add some clarification, a modder would modify meshes in a DCC tool and bring them into the game scene via the RTX Remix application. They might use the AI Texture Tools to super rez the textures and add physically based materials, but modifying the mesh within the RTX Remix application isn’t supported.
    Will there be a documentation/tutorial on everything about the RTX Remix? 
    Yes, we will be creating documentation and tutorials so modders can get up and running quickly with the RTX Remix.  


    Will the retexturing be made based on a target res we can choose? Just to not waste storage and VRAM since mipmapping is needed

    VishVadeva50

    Great question VishVadeva50,

    AI Super Resolution currently upscales up to 4x the original resolution of the asset, but the degree of upscale can be configured. Custom textures that a modder might have made manually will just be at whatever resolution they were created to be. During the RTX Mod export process, the exporter will handle the compression and mip generation for you (picking optimal settings for example). However, even the export process can be defined by the modder should they desire.



    Does RTX Remix work with all directX 8 + 9 games or will it be on a case per case basis? 
    What other games have you tested the tool with other than Portal and Morrowind?
    jayserpa

    Does RTX Remix work with all directX 8 + 9 games or will it be on a case per case basis? 

    Hey jayserpa,

    The goal is to have RTX Remix capable of modifying as many DirectX8 and 9 games with fixed function pipelines as possible. Compatibility will be an ongoing effort.  We will target a number of titles for initial release and with each update, we will broaden that compatibility further.

    For the technically minded, availability of the following APIs is a good predictor of compatibility:

    • SetTransform(View/Projection/World, ...) - so we can understand the placement of objects and cameras in the world
    • SetLight(...) - so we can understand lights in the world (their type, intensity etc).

    The absolute minimum we currently need is View/Projection matrix to set up the camera correctly.
    What other games have you tested the tool with other than Portal and Morrowind?

    We aren’t ready yet to provide a game compatibility list for RTX Remix, but we are listening to the community’s feedback on games to focus on. I’m often in touch with modders, speaking to Nexus Mods, reviewing all of the comments people are making around RTX Remix and taking good notes on the games you all are excited to mod!



    ?Hey, thanks for this amazing tool! I'm really looking forward to seeing what people make.

    One thing I'd like to know is if there are plans to expand the functionality of RTX Remix to newer version of DirectX at some point in the future. While DirectX 12 supports raytracing, DX10 and DX11 do not. For example, modders working on the original Mass Effect releases could make use of this tool, but those who mod the Legendary Edition remaster could not, since LE is DX11.
    Audemus

    Hey Audemus,

    Thank you for the informative question.

    To make RTX Remix work, we have to be able to understand, parse, and re-create everything the original game does. This is already a huge task just with the fixed function DX8/DX9 pipeline. As games start to rely on shaders, it becomes even harder to accurately capture light data, texture info, and asset models.

    Each step of customizability introduced by graphics APIs exponentially increases the amount of stuff RTX Remix has to know how to handle.  And as you climb to more sophisticated APIs, it likely would require more and more specific code for a specific game to work with RTX Remix. 

    DirectX8 and 9 games with fixed function pipelines were a target for us because we can predict how things should be rendered. Compatibility could be improved for older versions of DirectX and possibly future versions of DirectX but it is a significantly more complex problem going forward.

    In the future, we ourselves could expand compatibility or the community could work to introduce fixed function pipelines into existing classic games (like Modder Sajid here did in anticipation of RTX Remix):
    https://twitter.com/Sajidur78/status/1581411284805120000?s=20&t=nvCiA9qcWRO6KSoKNrp4vQ.




    Any plans to expand to, and add support for OpenGL (the fixed function pipeline portions of it, obviously. Namely 1.0 - 2.0)?
    SknTheLisper

    Hey SknTheLisper!

    Great question. 

    We are certainly interested in exploring OpenGL support but we have nothing to announce today. There are some technical hurdles such as DXVK-OpenGL support and OpenGL not exposing separate matrices for model and view transforms. All of it could be surmounted with time but our initial focus is DX8 and 9 titles with fixed function pipelines.

    It would be really cool to see OpenGL modding with RTX Remix though!


    How much disk space can we expect an RTX mod to use? E.g. Portal is 4 GB, how much space does Portal RTX use and would that be an accurate reflection of the space required for a remaster of a game of its length using RTX Remix? I'm worried that a large game like Morrowind would use hundreds of GB.

    Argonil

    Hey Argonil, thank you for the question. Ultimately it depends on the art team and what they target for any given mod. Portal with RTX is coming in under 14 gigabytes, but is not fully complete, which could go either way - either larger if we’ve got more content to update or smaller if we undertake more optimizations.


    Will Reflex work for games with largely-procedural scenes? A game I've been waiting to try Remix on was Empire Earth: The Art of Conquest, for example, which procedurally generates its maps and is a fairly-open RTS.

    BellCubeDev

    Hey BellCubeDev,

    I'm not sure if your question regarding Reflex was a typo and meant to be about Remix? With respect to Reflex, the largely-procedural scenes in a game like Empire Earth: The Art of Conquest is unlikely to matter.

    For RTX Remix, the question has a fairly complicated answer. It depends on if a mesh/texture will be procedurally generated the same way every time–if it is, it's essentially treated as a static mesh/texture from our perspective.

    With the RTX Runtime (component of RTX Remix), we look at draw calls and motion vectors to see if an asset is present at runtime, and we replace it if we are able to identify it. As long as the modder captures even one scene with the generated asset/texture, we will be able to replace it. If the asset has a true dynamic mesh (like trees blowing in the wind or a never before seen asset), then RTX Remix will be unlikely to identify the object, and therefore won’t be able to replace it.

    Hope that helps.


    The main (and unsolvable if the game's source code is unavailable. By definition, the engine is not even aware of any changes) problem of modding graphics like this is that, not only the model collision must be respected, but also physics and the game logic attached to dynamic objects as well. Otherwise this serves little to no functional purpose - just an eyecandy to take screenshots with.

    In the Morrowind demo, it's apparent that scene had several objects changed/removed/added. Are they real objects, or merely ghosts with no collision that NPCs are going to mindlessly pass through?
    dsp2003

    Thanks for the question dsp2003:

    For Remix to replace assets with accurate collision behavior, the redone assets must match the meshes of the existing objects they are replacing. This is because we are intercepting draw calls from the game, and replacing assets at runtime. In the Morrowind case, since it was a non-playable technical demo, we opted to let our artists stretch their legs, and make a visually appealing spectacle with new assets that would lack collision in the playable game.


    However, I want to point out, adding “ghost” assets into an existing game–especially older games–has a lot of viable uses. In many classic games, it is common for there to be tables covered with non-interactive props, ceiling architecture, or even foliage–all things users can add to a scene without breaking immersion, since the player never has the expectation to collide with them in the first place. Classic games often feature flat textures to represent 3D models that are already non-interactive. These textures benefit from being modeled with a 3D mesh to accurately cast shadows and interact properly with a realistic lighting model.

    Also, because RTX Remix does not care about the source of the draw calls, a modder could use RTX Remix alongside another tool that can build assets with collision, and then reskin those objects via RTX Remix without restrictive memory constraints and with full ray tracing and DLSS 3 enabled in the game.


    I think the Nvidia team has done great work on remix, my question is will this only work on games with mod support like morrowind or can we use it with almost any game ?
    AiBlastafari

    Hey AiBlastafari (great name!):
    The goal is to have RTX Remix capable of modifying as many DirectX8 and 9 games with fixed function pipelines as possible. This means there will be titles like Morrowind and Mount and Blade: Warband that are highly modded that will be supported, as well as titles that have been difficult to mod before. To set expectations – compatibility will be an ongoing effort.  In Beta, we will target a number of titles and with each update to RTX Remix, we will broaden that compatibility furt6er.




  3. gaun217
    gaun217
    • premium
    • 2 kudos
    Do you have plans to make this compatible with older versions of DX or DX11(though I'm assuming the answer to the latter is that it is too new)?
  4. Etherial1
    Etherial1
    • premium
    • 10 kudos
    What about for games that most people are gonna use this for in nexus' sake new vegas and skyrim that don't have a fixed function pipeline?
    1. hicks233
      hicks233
      • member
      • 14 kudos
      It won't work.
  5. BlairRafeKnox
    BlairRafeKnox
    • member
    • 0 kudos
    Question. What about cards such as  GTX 16 (series) and  GTX 1080ti that have Turing-architecture, can they also with RTX remix?   
  6. KlLLMaster
    KlLLMaster
    • member
    • 0 kudos
    Hi, I’ve got a few questions;
     
    1. This is more of an expansion of the previous question posed by dariort64, more specifically the part about LOD’s.
    Would it be possible to completely subvert/ignore the base games LOD system and simply set up our own LOD hierarchy through RTX Remix?
    This way the artist could go through and set their own performance target with mesh specific LOD distances catered to their custom assets.
     
    2. This question ties into the first question a little. Would it be viable to give artists the ability to create “proxy” meshes for
    the path tracer to use in reflections or to manually target a specific LOD to be used in reflections.
    (or maybe just use the original games geometry for reflections)
     
    3. Is it possible to override a mesh/s culling to prevent it from ever being culled or set custom parameters that would dictate when an
    object will be culled in order to facilitate more efficient and or better reflections in the path-tracer?
    This would allow artists to decide what geometry stays rendered in the scene, regardless
    of what the base game intends.
     
    4. I know that you guys will be validating games on you’re end, but will end-users be able to attempt to use RTX Remix with any DX8/9 game
    they chose?
    I know similar questions have already been asked but I couldn’t find any definitive answers on this.
     
    5. This question is more of a stretch but have you guys ever looked into a “virtualized geometry system” similar to UE5’s Nanite solution?
    It would completely eliminate the need for LOD’s (and allow absurd triangle densities), but I am well aware that that is in no way a trivial task.
    What Epic managed to achieve with Nanite is borderline voodoo magic.
  7. sabi123456
    sabi123456
    • supporter
    • 2 kudos
    I wanted to update the model in the game Shogo: Mobile Armor Division which says it is DirectX 6.0 and above but I don't know if it would work or not with RTX remix. this game was my childhood would love to upgrade this.

    so in short my question is. would it work on games that have a lower direct x but also say lower direct x and above for example DirectX 6.0 and above?
  8. Argonil
    Argonil
    • premium
    • 218 kudos
    I'm thinking about remastering Unreal Tournament 2004 - a highly beloved game! But seeing as it's a multiplayer arena shooter, I wonder how that'd work out. We'd have to remaster all the custom maps. When connecting to a server that's playing a map you don't have, would the game be able to detect the assets from RTX Remix that accompany the map and download them? Probably not right? What if we're connecting to a server that doesn't use RTX, or someone without RTX tries to connect to a server that does use it? Would they be able to play together? If you go from an RTX map to a non-RTX map, would the renderer handle it properly?
    1. NVNyle
      NVNyle
      • member
      • 8 kudos
      Hey Argonil,

      We look forward to seeing your future experiments with RTX Remix! On to your questions:
      I'm thinking about remastering Unreal Tournament 2004 - a highly beloved game! But seeing as it's a multiplayer arena shooter, I wonder how that'd work out. We'd have to remaster all the custom maps. When connecting to a server that's playing a map you don't have, would the game be able to detect the assets from RTX Remix that accompany the map and download them? What if we're connecting to a server that doesn't use RTX, or someone without RTX tries to connect to a server that does use it? Would they be able to play together? 
      Without commenting on Unreal Tournament 2004 specifically, here’s how it should work.

      As long as players have the RTX Mod with remastered assets, they will see the maps with updated visuals. The server is uninvolved in the rendering process and is not relevant to whether users will see updated visuals–it’s all about the RTX Mod users have locally stored on their PC. Because the RTX Remix runtime is only influencing the visuals, players should in theory be able to play against each other even if some players are seeing the vanilla game while others are using RTX Mods.

      To correct a misconception, you may not have to remaster every custom map. In old games, many custom maps reuse the same textures and assets. RTX Remix should replace all of those textures and assets if the mod has replacements available, even if the level geometry is being seen for the first time.

      As for dynamically downloading RTX Mods in game for players who do not have those assets already–that type of functionality would have to be modded into the original game by the modder so that remastered assets are downloaded and put into the RTX Remix Runtime's asset folder. RTX Remix wouldn’t be involved in architecting such a system.

      Another thing to note–Anti-cheat software may prevent RTX Remix from working in certain competitive multiplayer titles.
      If you go from an RTX map to a non-RTX map, would the renderer handle it properly?
      The renderer would fall back to showing classic assets, wherever it has no replacement assets to point to.
  9. dariort64
    dariort64
    • member
    • 0 kudos
    Will Remix include a library where we could use its renderer and insert it into other games ourselves by describing the scene's contents instead of relying on the fixed pipeline translation layer? Some games have enough reverse engineering work on them that it'd end up being easier to use something like this instead of backporting to fixed pipeline.

    Will it be possible to make custom implementations of the shading of surfaces (e.g. shader programming) to be able to recreate specific materials more easily?

    How will the USD replacement functionality work with skinned assets? Or will it only support static mesh replacement? Will assets that use vertex blending be exported with their matching bones and weights correctly? And what about games that use morphers for facial animations?

    What kind of scripting functionality will be available to control what lighting or other scene settings get used in a game? Would this provide a way to modify the scene in real time to match certain systems of the game (like a game with a dynamic day/night system)?

    Will there be any systems in place to deal with replacing assets in games that are capable of streaming and swapping different level of detail meshes?
    1. NVNyle
      NVNyle
      • member
      • 8 kudos
      Hey DarioRT64, big fan of your mods!

      Very detailed questions!
      Will it be possible to make custom implementations of the shading of surfaces (e.g. shader programming) to be able to recreate specific materials more easily?
      This topic has come up a lot and we are deep in research, but there’s nothing to announce today.
      How will the USD replacement functionality work with skinned assets? Or will it only support static mesh replacement? Will assets that use vertex blending be exported with their matching bones and weights correctly?
      It depends on the implementation used by the game. For CPU based skinning, we get the post-skinned data in Remix, and while it renders correctly, it's a bit more challenging to replace these assets. It’s something we would like to look at closer in the future.

      For GPU based (fixed function) skinning, we actually have access to the pre-skinned mesh and the skeleton (including the collision volumes for raytracing) which gives us a great deal of flexibility to modify and reskin characters in RTX Remix.

      We saw some (incorrect) speculation that we did not include NPC’s in our Morrowind RTX showcase because we weren’t able to properly mod them. But Morrowind actually passes us pre-skinned meshes and the skeleton so it was perfectly in our capabilities to modify the NPCs. In order to give the artists time to do an incredible job, we narrowed the scope of the project to an indoor environment without NPCs.
      Will there be any systems in place to deal with replacing assets in games that are capable of streaming and swapping different level of detail meshes?
      Variable level of detail during streaming is an interesting case. Here's how it should work: you would need to take multiple captures at each level of detail, and RTX Remix would regard each asset at each level of detail as a unique instance. For example, imagine an asset has three levels of detail depending on the distance of the player–at 5 meters it’s high quality, at 15 meters it’s middle quality and at 25 meters it’s low quality. That object would actually be regarded as three separate objects depending on the distance of the player.

      For the modder, they have two options. Recreate all three assets with different levels of detail to give you a similar feeling as the original game. Or map a high quality asset to all three instances, which would help make the asset feel more at home in a modern graphical context.

      Hope that helps Dario. Can't wait to see what you can do with RTX Remix!
  10. DarianStephens
    DarianStephens
    • premium
    • 122 kudos
    One thing that concerns me is the requirement that the games have a Fixed Function pipeline, which excludes a great majority of games people would want to use these tools on.
    Notably for Nexus, New Vegas and Skyrim.
    What I'm really, really hoping is the case, is that it will only take some extra work to make games like that fully RTX-ready, perhaps missing out on some automated stuff that Remix would otherwise do, but I'm afraid that it will simply not work.

    Additionally, I saw in the previews that they were always shown off indoors, and with no NPCs or other dynamic objects. It can obviously handle at least some dynamic stuff given Portal with RTX's cubes, pellets, turrets, and things, but with Morrowind, there were no NPCs, and the outdoors was never shown outside of a vanilla shot for comparison.
    How does Remix handle games which have a dynamic day/night cycle, and which load/unload objects dynamically in a single area (Such as exploring the worldspace in Morrowind and travelling to different cells)? How is it able to differentiate between indoors/outdoors? Can you specify an object or vector as relating to the sun/moon, and have it adjust RTX parameters dynamically to match?

    Another thing is how it would deal with games that use culling. Will objects behind you stop casting their shadows because the game engine is no longer rendering them? Can it catch the whole scene in one go with that regardless, or would it require multiple 'snapshots'?
    1. Wyntilda
      Wyntilda
      • premium
      • 5 kudos
      As a Fable modder (which relies quite a bit on shaders and is very dynamic; for example, areas can have different objects and light sources depending on context) I have pretty much all the same questions, so thank you for speaking up! I'm worried the hype from news sources is going to lead to a lot of disappointed gamers when they discover their favourite DX8 or DX9 games aren't fully supported by RTX Remix, and while some of this tech is neat and convenient, it seems like it will still manually require a lot of work, especially for games with more quirks. The only demos we've seen are a Morrowind interior space and Portal, which are both pretty limited and simplistic.
    2. NVNyle
      NVNyle
      • member
      • 8 kudos
      Hey DarianStephens,

      I love technical questions like this.
      How does Remix handle games which have a dynamic day/night cycle, and which load/unload objects dynamically in a single area (Such as exploring the worldspace in Morrowind and traveling to different cells)? 
      I want to preface some of these specific questions by saying lots of aspects of RTX Remix are still in motion, and many games behave in unique ways depending on a host of variables including their lighting model. Not every title has been tested.

      For dynamically loaded objects, as long as the capture properly includes each object that is dynamically loaded in and out of an area, the replacements should also dynamically load.

      As far as day/night cycles go, that’s a trickier question. Assuming the celestial bodies are coded as directional lights, the ray traced conversion should retain that feeling of dynamic day and night cycles with more realistic shadowing and skies. From our outdoor tests in Mount and Blade: Warband, we’ve seen some stunning sunsets and sky lighting with realistically rendered light rays peeking through clouds–it’s entrancing to see that kind of thing in games from the 2000s!

      If it's a lit skybox it should also work. But developers in those days took a lot of shortcuts to simulate a lit sky and there might be certain approaches to doing classic lighting that we haven’t seen in our testing, and therefore haven’t yet addressed.
      How is it able to differentiate between indoors/outdoors?
      There shouldn’t be a need to differentiate between outdoors/indoors in a ray traced lighting model since lights begin to behave realistically. An indoor environment would simply possess more meshes and geometry that lights bounce off of, or that occlude lights to cast more shadows. Both outdoor and indoor environments would have to be relit with the context of realistic lighting, which is something we show in our RTX Remix overview video: https://youtu.be/Vg52-HZhrFc?t=163.
      Can you specify an object or vector as relating to the sun/moon, and have it adjust RTX parameters dynamically to match?
      On adjusting RTX parameters dynamically, it might help if you can provide us with an example of a game that we can research that would require something like that, to help illustrate the use case (I've sent you a DM). As for RTX Remix’s options with respect to celestial bodies, classic games can specify a directional light (vector + light parameters) and RTX Remix will ray trace it accurately. Or the original game could have a big mesh up in the sky and we can put an emissive texture on it, or even attach a big sphere light to it. There are a lot of options as it pertains to lighting outdoor environments.
      Another thing is how it would deal with games that use culling. Will objects behind you stop casting their shadows because the game engine is no longer rendering them?
      Culling is another case where there might be different interactions with RTX Remix depending on the game.

      As you’ve noted, if culling causes objects to cease to exist in the original game, then the RTX Renderer would also cease rendering them. Portal has built in settings that let us disable a lot of the more aggressive culling, which lets us avoid most of those issues. It still culls lights very aggressively, so we had to implement persistent static lights even after the game stops drawing them. There are other workarounds of course–for example, we can simply attach new static lights to level geometry and disable the original lights to retain the original game’s feel without breaking the immersion of full ray tracing.

      If other games don't allow customizing the culling settings, and the issue can't easily be addressed by traditional modding, then we may need to look into implementing a different system. For example, some kind of caching to improve object permanence. There are many paths to a solution and we will be paying attention to see if this becomes an issue for classic games.
      Can it catch the whole scene in one go with that regardless, or would it require multiple 'snapshots'?
      As it is currently implemented, a snapshot will capture a single frame of content which includes everything rendered, whether it is in view or out of view of the player, and every object will be cataloged (alongside the geometry, textures, lighting, cameras). Some games that render the entire level might only require a single capture, while other games that have aggressive culling might require more. RTX Remix will understand duplicate instances of the same asset through a game, so a modder may already have certain assets captured from a prior room when they begin a capture in a new room/area. For a game, you will have to take multiple captures, although a single area might only need a single capture.

      We did not get to every one of your questions but hopefully this demystifies things quite a bit. I should point out, another poster asked about skinning and NPC's, so definitely take a glance at the summary that Nexus Mods will post to see how RTX Remix handles that.
  11. LZKiller7
    LZKiller7
    • member
    • 0 kudos
    Can it inject non-RT lighting and graphical features such as Global Illumination, Volumetric Lighting, and Screen Space Reflections?
    1. NVNyle
      NVNyle
      • member
      • 8 kudos
      Hey LZKiller7,

      Btw, Killer7 was one of my favorite games on the Gamecube (always hoped the Smiths would make it into Super Smash Bros).
      Can it inject non-RT lighting and graphical features such as Global Illumination, Volumetric Lighting, and Screen Space Reflections?
      RTX Remix really started as a way to make full ray tracing (i.e. path tracing) accessible for modders, so that faking reflections or light could be a thing of the past. Big publishers already design their games with ray tracing in mind–we think the modding community should have that same power at their disposal.

      As such, in place of non-RT global illumination, volumetric lighting, and screen space reflections, RTX Remix offers highly customizable light accurate global illumination, volumetrics, and on/off screen reflections.

      There are some post effects that are non-RT that allow modders to tweak the experience like bloom, tone mapping, motion blur, but the focus is to empower modders with higher fidelity lighting.
  12. doodlum
    doodlum
    • premium
    • 3,319 kudos
    Does RTX Remix have an API which can be used to provide additional information to it, like weather parameters, lighting conditions, engine-provided material information.

    How are skinned/animated objects tracked? Are they replaceable?
    1. NVNyle
      NVNyle
      • member
      • 8 kudos
      Does RTX Remix have an API which can be used to provide additional information to it, like weather parameters, lighting conditions, engine-provided material information.
      Hey Doodlezoid, interesting question.

      Currently there is no such API but if there is interest for something like this, we can look to see if there’s anything we can do.

      How are skinned/animated objects tracked? Are they replaceable?
      We've answered a similar question about skinning in this comment section. Please take a look at that answer for more information :).