feat: generating npz files using only C++#151
Open
yhisaki wants to merge 1 commit into
Open
Conversation
yhisaki
force-pushed
the
convert-data-by-cpp
branch
from
00cb7b3 to
665bb4b
Compare
- Updated `frame_processor.cpp` to include NPZ frame writing capabilities. - Modified the `process_sequence` function to utilize `ConverterPaths` for directory management. - Enhanced frame data saving logic to support NPZ format. - Introduced `cnpy.cpp` for handling NPZ file operations, including loading and parsing. - Cleaned up `neighbor_processor.cpp` by removing unused includes and defining constants. - Simplified tests in `test_converter_options.cpp` to validate default options and removed redundant tests. - Streamlined `test_frame_writer.cpp` by removing unnecessary helper functions and tests. - Removed bin file processing from `parse_rosbag_by_cpp.py` to focus on NPZ generation. Signed-off-by: Y.Hisaki <yhisaki31@gmail.com>
yhisaki
force-pushed
the
convert-data-by-cpp
branch
from
665bb4b to
5b5e969
Compare
yhisaki
changed the title
NPZ Diff SummaryComparison of the same frame written by two different
Both files contain the same 18 keys. No key is missing on either side. 1. dtype / shape differences
2. float value differences (numerical, epsilon-level)
All differences are at the float32 epsilon level (~1.2e-7), consistent with floating-point ordering / rounding differences rather than logic differences. The other 14 float keys are bitwise identical: |
|
SakodaShintaro
requested changes
Jun 19, 2026
SakodaShintaro
left a comment
SakodaShintaro
left a comment
There was a problem hiding this comment.
These changes breaks the training pipeline.
And, it seems to be too big for the purpose.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
By using cnpy, it is now possible to generate npz files using only C++.