Add model module: FBX/GLTF import representation#46
Draft
jayrulez wants to merge 23 commits into
Draft
Conversation
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
jayrulez
force-pushed
the
model
branch
3 times, most recently
from
3f60aae to
8cc9361
Compare
…ernion to Core Drop the bgfx rendering stack entirely: delete the Rendering and Scene modules, the bx/bimg/bgfx submodules and their ThirdParty build tooling, the shader compilation (cmake/Shaders.cmake), and rewire Runtime/Engine/Draconic so the engine builds on Core + Shell + SDL3 only. RenderSample becomes a shell-only placeholder that opens a window and pumps events until closed. Core/Math kept its one bx dependency (Transform's TRS matrix build), so replace it with native math: add core.math.matrix4 (4x4 row-major, row-vector convention: identity/translation/scale/rotation/perspective/ortho/lookAt, multiply, transpose, transformPoint/Direction/Point2D, determinant, inverse) and core.math.quaternion (fromAxisAngle, Hamilton product, conjugate/dot/inverse/normalize/rotateVector/slerp, rotationMatrix). Rewrite Transform to position/rotation(Quaternion)/scale with an inline toMatrix() and a lerp helper, removing bx from Core entirely. Added missing math helpers used by these types (sin/cos/tan/acos/sqrt, scalar and Vector2/3/4 nearlyEqual, Vector3::zero/one) and added Matrix4/Quaternion/Transform unit tests.
Introduce the rhi module: a backend-agnostic GPU render-hardware-interface, as the primary module 'rhi' with partitions for enums, texture formats, core types, forward declarations, resource/descriptor structs, resources, command recording, extensions (mesh shader / ray tracing), queue, swap chain, and the Backend/Adapter/Device factory. Interface only - concrete backends (Null, Validation, Vulkan, DX12) land as separate modules. Add a small Status/ErrorCode facility to Core (core.status, re-exported via core.defs) for the interface's status-code returns; value-or-error paths use std::expected. Uses this engine's conventions throughout: draco::rhi namespace, camelCase methods/functions/constants (types and enum values stay PascalCase), and std container types (std::vector/span/optional/u8string). Wired via add_modules_library(Rendering/RHI) into Runtime.
Headless no-op RHI backend implementing every rhi interface with stubs: a NullDevice that creates/destroys resources, a mappable NullBuffer backed by host memory, no-op command encoders, and a createNullBackend() entry point. Unsupported extensions (mesh shaders, ray tracing) degrade to NotSupported rather than crash. Intended for headless testing, CI, and tools. Includes a doctest suite covering adapter enumeration, device/resource creation, buffer mapping, and extension degradation. Allocations use new/delete for now (the engine's Slice-based allocator does not compose with polymorphic base-pointer frees; revisit with the GPU backends).
The validation backend (rhi.validation) wraps any RHI backend and checks API usage before forwarding: encoder state machines (recording / in-pass / finished), non-null resource arguments, pipeline-bound-before-draw, viewport/scissor set before draw, monotonic fence signaling, swap chain acquire/present balance, and per-device live-resource leak tracking. Validated wrappers unwrap to the inner objects before forwarding, so an inner backend only ever sees its own types.
The Vulkan backend (rhi.vk) implements the full RHI surface on Vulkan 1.3+ dynamic rendering: adapter enumeration with preference ordering, logical device and queue setup, all resource/pipeline/bind-group creation, command encoders (render/compute/bundle passes), swap chains with platform surfaces (Win32/X11/Wayland), transfer batches, and the mesh-shader and ray-tracing extensions. Requires the Vulkan loader (find_package(Vulkan)); the loader is a private dependency since the headers live in the module's global fragment. Includes a platform-agnostic smoke test that initializes the backend, enumerates adapters, and creates a device. It never touches surfaces, so it runs anywhere a loader is present and skips the device step gracefully when no adapter is available.
The DX12 backend (rhi.dx12) implements the full RHI surface on D3D12: adapter enumeration via DXGI, device/queue setup, all resource/pipeline/bind-group creation, descriptor heap management (CPU staging + GPU-visible ring), command encoders (render/compute/bundle passes with copy-on-bind descriptor staging), swap chains, transfer batches, and the mesh-shader and ray-tracing extensions. Windows only: registration is guarded by if(WIN32) and links the D3D12/DXGI system libraries privately. Includes a platform-agnostic smoke test (built only on Windows) that initializes the backend, enumerates adapters, and creates a device. Not yet compiled — this box has no D3D12 toolchain; the port is to be validated on Windows.
- DxBackend: sortAdaptersByPreference casing to match Device.cppm - DxRenderBundleEncoder: PascalCase delegate calls to camelCase - Math.test: qualify std::cos to resolve overload ambiguity
The shaders module wraps DXC (IDxcCompiler3) to compile HLSL to SPIR-V or DXIL. It loads dxcompiler at runtime (LoadLibrary/dlopen) rather than linking it, so nothing hard-links the compiler. The public compile types (ShaderStage, CompileOptions, CompileResult, ShaderFlags + variant key) are self-contained: the compiler produces bytecode and has no RHI dependency — the RHI-coupled variant cache is a separate module (shaders.system, not yet ported). Draco::DXCHeaders (ThirdParty/DXC) carries the DXC include dir plus DRACO_DXC_PATH, the absolute path the compiler falls back to for the runtime library. The prebuilt DXC runtime libraries themselves are not committed here. Test compiles a trivial vertex shader to SPIR-V (checks the magic number) and confirms a bad shader reports an error instead of crashing — passes against the vendored DXC on Linux.
The sample framework (samples.rhi.framework) is the shared base for the RHI samples, relocated into Samples/cpp/RHI so it is scoped to RHI samples only. SampleApp brings up an SDL3 window (shell.desktop), a Vulkan or DX12 backend (validation-wrapped), device, graphics queue, and swap chain, then runs an event/resize/render loop calling the subclass's onInit/onRender/onResize/onShutdown. DepthBuffer is a recreate-on-resize depth target; ShaderHelpers compiles HLSL to a ShaderModule, picking SPIR-V or DXIL by device type and applying Vulkan binding shifts. The framework re-exports rhi + shell so samples import one module. DX12 is compiled in only on Windows (DRACO_HAS_DX12). Individual samples are added on top of this.
Ports the 30 numbered RHI samples (Triangle through RenderBundles) plus the Smoketest, verbatim apart from the namespace (draco::) and camelCase API changes. They exercise the RHI across vertex/index buffers, textures, uniform buffers, compute, bind groups, blending, instancing, depth, mipmaps, MSAA, MRT, wireframe, samplers, blit, queries, readback, multi-queue, bindless, batch upload, mesh shaders, ray tracing, stencil, cube maps, occlusion queries, multi-draw-indirect, dynamic offsets, 3D textures, procedural RT, resolve, and render bundles. Each sample is one Main.cpp on the shared framework; a draco_add_rhi_sample() helper wires each executable with the whole-archive SDL link so its static video/input drivers survive. Math types are qualified under draco::math (Vector, Matrix4, Quaternion, degToRad). Build only: they open an SDL3 window and need a GPU/display, so they are not run in CI.
Commit the prebuilt DXC runtime libraries (dxcompiler + dxil, for win-x64 and linux-x86_64) that the shaders module loads at runtime via DRACO_DXC_PATH. Tracking them directly keeps a fresh checkout self-contained: the samples compile HLSL at runtime and need dxcompiler present.
The shader system (shaders.system) is a compile-on-demand cache for shader variants on top of the RHI-free shaders compiler. A shader is registered by name per stage; getVariant(name, stage, flags) turns the flags into #defines, compiles the permutation via DXC, creates a GPU ShaderModule, and caches it by (nameHash, stage, flags). Compile failures are not cached, so a later request retries after a fix. invalidateShader drops and destroys a shader's variants and bumps a monotonic version the PSO cache can poll. Variants are keyed in a std::unordered_map via a hash functor for the variant key. Needs the RHI to create modules, so it is a separate library from the base shaders module. Test compiles real SPIR-V via DXC and creates modules on the Null backend, covering flags-to-defines, caching, distinct variants, and invalidation.
The graphics module is the shared RHI render host: GraphicsDevice (backend + adapter + logical device + graphics queue + CPU frame-in-flight ring) hands out RenderWindows, each owning a surface/swapchain and a per-window ring of command pools and fences. FrameContext is the per-window, per-frame hand-off — the host owns acquire, fence sync, submit, present, and the backbuffer state transitions. Split into three modules in one target: graphics (backend-agnostic core, imports only base RHI + Shell), graphics.null (headless factory over the Null backend), and graphics.gpu (Vulkan/DX12 factory, validation-wrapped on request). DX12 is compiled in only on Windows via DRACO_HAS_DX12; validation defaults on for dev builds and off under NDEBUG. Value-or-error returns use std::expected; owned handles use std::unique_ptr. Headless test over the Null backend + null shell passes (5 cases, 38 assertions).
The render graph (rendergraph) sits over the RHI: passes declare resource accesses, the graph builds a dependency DAG, culls unused work, topologically sorts, allocates and aliases transient textures from a pool, and inserts the right barriers automatically via a subresource state tracker. compile() runs without an encoder, so the graph logic is testable headless. Includes a validator, DOT-graph debug export, and a per-pass GPU/CPU profiler. Value-or-error uses Status; owned resources use new/delete and std::unique_ptr; callbacks use std::function; maps/sets use std::unordered_map/set. A few local stand-ins (mapFind, appendFormat, getTicks) are marked with NOTE comments to replace once core grows the corresponding facilities. Thirteen headless test executables over the Null backend cover handles/types, descriptors, pass building, dependency and culling logic, transient generation, barriers, bundle passes, validation, debug export, and profiling.
core.color adds two color types in the draco namespace: Color (linear RGBA float, the runtime currency) and Color32 (packed 8-bit RGBA for compact storage of image pixels and vertex colors). Both pack to/from 0xRRGGBBAA, with named constants (White, Black, Red, Green, Blue, Transparent), float/byte conversions, and sRGB/linear transfer functions. Exported from the core umbrella. Unit test covers packing round-trips, constants, conversions, and the sRGB transfer.
Image loading belongs in the image module, not core. Drop core.io.loader.image (its stb-based ImageData/load) and core's stb_image dependency; the image module owns stb load/save now. Nothing consumed the core loader.
The image module is CPU-side image data and manipulation, independent of the RHI: PixelFormat, ImageData/OwnedImageData, procedural generators (solid/checkerboard/gradient/normal maps), nine-slice, and a shelf-packing atlas builder. A separate image.io module does concrete load/save through stb_image / stb_image_write (its stb_impl TU holds the implementations); stb is a private dependency. Vendors stb_image_write and stb_truetype alongside stb_image. Pixel access uses core's Color32. Headless tests cover image data, manipulation, normal maps, atlas packing, and a save/reload round-trip through stb.
The texture module is the descriptor layer between CPU image data (image) and the GPU (rhi): pixel-format types and image-to-RHI format conversion (FormatUtils), plus TextureData describing a texture's dimensions, format, and mip/layer layout. No GPU resources are created here; it is the format/layout currency the texture resource factory and renderers build on. Headless test covers the texture-data descriptors.
The VectorN types are alignas(16) SIMD vectors (a Vector3 occupies 16 bytes), which pads tightly-laid-out data such as GPU vertex streams past its canonical stride. Add packed Float2/3/4 that stay byte-tight (8/12/16 bytes) and convert implicitly to and from the aligned VectorN, plus the Float3 dot/cross/length/lengthSquared/normalize helpers. Add AABB, an axis-aligned bounding box with an inverted empty() seed, expand(), center(), and size(). Both are consumed by the mesh format and will serve the material and model representations. Covered by unit tests.
The engine's GPU-upload-ready mesh format, distinct from the importer's loaded-file representation. Aggregates four partitions: - types: primitive topology, submesh ranges, and the two vertex streams. The static stream is a tightly-packed 48-byte vertex (core's packed math::FloatN); the parallel skinning stream is 24 bytes. - index_buffer: a format-tagged 16/32-bit index buffer stored as raw bytes with a streaming append cursor. - mesh: StaticMesh (static stream + index buffer + submeshes + bounds + normal/tangent/bounds ops) and SkinnedMesh, which IS-A StaticMesh and adds only the parallel skinning stream. A SkinnedMesh is substitutable wherever a StaticMesh& is expected; virtual isSkinned()/skinningStream() hooks let a base-ref consumer discover and bind the skinning stream, and toSkinned() downcasts safely without RTTI. - primitives: procedural quad, cube, plane, and UV sphere generators. Covered by unit tests for stream layouts, the index buffer, the geometry ops, static/skinned substitutability, and the primitives.
A material is data: a shader name + variant flags, a list of declared properties, a PipelineConfig, and default values. The system infers the GPU bind-group layout from the property list, so a new material or shader needs no renderer changes. Partitions of the `materials` module: - types: property kinds, blend/depth/cull presets, and the predefined vertex layouts, plus a value-or-null map lookup helper. - pipeline: PipelineConfig, the content-hashable render-state key, and VertexLayoutHelper mapping the layout presets to RHI vertex attributes. - material: Material, the shared immutable template (property list + defaults + stable name backing). - builder: a fluent builder that lays out the uniform buffer (std140-ish 16-byte alignment) and assigns bindings, plus a PBR preset. - instance: MaterialInstance, per-use overrides with dirty tracking that notifies its sink on a clean-to-dirty transition. - system: MaterialSystem, which owns instance GPU resources, caches inferred layouts by content hash, and re-preps only dirtied instances. Also adds the render-side PSO cache (`materials.pso`): maps a PipelineConfig to a compiled RenderPipeline, pulling shader variants from the ShaderSystem and rebuilding via version polling when a shader reloads. Covered by unit tests against the Null backend (builder layout, inferred bind groups, instance overrides/dirty notification, and the PSO cache build/cache/reload path).
Single-header (ufbx also ships its .c) FBX and glTF parsers, exposed as INTERFACE targets that add their include directory. The implementations are compiled in the model module's own TUs.
The importer-side representation of a loaded 3D file: meshes, materials, skeleton (bones/skins), animations, and textures, distinct from the engine's runtime mesh format. One `model` module whose partitions keep their folder separation: - the interface partitions (vertex formats, mesh/mesh-part, material, bone, skin, animation, texture, and the Model aggregate) live at the top level; the static/skinned import vertices use core's packed math::FloatN so they stay 48/72 bytes. - io: a loader registry that selects a format by file extension. - fbx / gltf: the FBX(+OBJ) and glTF loaders, built on ufbx and cgltf, which load embedded/external textures through the image module. The loaders and IO registry fold into the same module as the interface (rather than separate modules), importing the sub-partitions through the Model partition's re-exports. Covered by a unit test for the core data types.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The model module is the representation of a loaded 3D file - meshes, materials, skeleton (bones/skins), animations, textures - distinct from the engine's runtime mesh format.
interface (top level) - vertex formats, mesh/mesh-part, material, bone, skin, animation, texture, and the Model aggregate. The static/skinned import vertices use core's packed math::FloatN to stay 48/72 bytes.
io - a loader registry that dispatches by file extension.
fbx / gltf - the FBX(+OBJ) and glTF loaders built on ufbx/cgltf, loading embedded/external textures through the image module.
Third parties: vendors ufbx and cgltf as INTERFACE targets (include dir only); their implementations compile in the module's own ufbx_impl/cgltf_impl TUs.