[RNE Rewrite] feat: add semantic segmentation task by barhanc · Pull Request #1275 · software-mansion/react-native-executorch

barhanc · 2026-06-22T14:29:42Z

Description

Adds semantic segmentation task and required native operations for it.
Adds computer-vision example app screen for semantic segmentation.
Fixes the performance regression in the argmax introduced in [RNE Rewrite] feat!: add classification task and common utilities for manipulating tensors in CV tasks #1264. The initial implementation from PoC was more efficient for the default axis=-1 case as the internal loop was over contiguous elements. The linked PR changed it so that it was more efficient for axis=0 case, however since the default is axis=-1 this caused a performance regression in the semantic segmentation task.

Introduces a breaking change?

Yes
No

Type of change

Bug fix (change which fixes an issue)
New feature (change which adds functionality)
Documentation update (improves or adds clarity to existing documentation)
Other (chores, tests, code style improvements etc.)

Tested on

iOS
Android

Testing instructions

Build the computer-vision example app.
Test the semantic segmentation screen.

Screenshots

Related issues

Closes #1242

Checklist

I have performed a self-review of my code
I have commented my code, particularly in hard-to-understand areas
I have updated the documentation accordingly
My changes generate no new warnings

Additional notes

… for semantic segmentation

…formats

barhanc · 2026-06-23T13:00:59Z

Two models are missing from models.ts: DeepLabV3 and FCN, since the forward returns auxiliary tensor. I will reexport them but don't know under what tag should I put them, isv0.10.0 ok?

msluszniak · 2026-06-23T13:05:09Z

Two models are missing from models.ts: DeepLabV3 and FCN, since the forward returns auxiliary tensor. I will reexport them but don't know under what tag should I put them, isv0.10.0 ok?

Yeah, v0.10.0 sounds ok.

msluszniak · 2026-06-23T13:07:53Z

+        if (dst->dtype_ != rnexecutorch::core::types::DType::uint8) {
+            throw jsi::JSError(rt, "applyColormap: dst must be uint8");
+        }
+        if (dst->numel_ != src->numel_ * 4) {


I guess 4 might be replaced here and in other places in this file as constexpr size_t numChannels = 4; or so.

msluszniak · 2026-06-23T13:15:57Z

+            dstData[i * 4 + 0] = lut[idx][0];
+            dstData[i * 4 + 1] = lut[idx][1];
+            dstData[i * 4 + 2] = lut[idx][2];
+            dstData[i * 4 + 3] = lut[idx][3];


maybe this one when you introduce numChannels? Loop will be automatically unrolled on constexpr numChannels. Just add correct formatting.

Suggested change

dstData[i * 4 + 0] = lut[idx][0];

dstData[i * 4 + 1] = lut[idx][1];

dstData[i * 4 + 2] = lut[idx][2];

dstData[i * 4 + 3] = lut[idx][3];

for (size_t c = 0; c < numChannels; ++c) {

dstData[i * numChannels + c] = lut[idx][c];

}

msluszniak · 2026-06-23T13:17:19Z

        int32_t *dstData = reinterpret_cast<int32_t *>(dst->data_.get());
-        std::vector<float> maxVals(inner);

        for (size_t o = 0; o < outer; ++o) {
-            const float *srcSlab = srcData + o * axisDim * inner;
-            int32_t *dstRow = dstData + o * inner;
-
            for (size_t i = 0; i < inner; ++i) {
-                maxVals[i] = -std::numeric_limits<float>::infinity();
-                dstRow[i] = 0;
-            }
-
-            for (size_t d = 0; d < axisDim; ++d) {
-                const float *srcRow = srcSlab + d * inner;
-                for (size_t i = 0; i < inner; ++i) {
-                    const float val = srcRow[i];
-                    if (val > maxVals[i]) {
-                        maxVals[i] = val;
-                        dstRow[i] = static_cast<int32_t>(d);
+                float maxVal = -std::numeric_limits<float>::infinity();
+                int32_t maxIdx = 0;
+                for (size_t d = 0; d < axisDim; ++d) {
+                    const float val = srcData[o * axisDim * inner + d * inner + i];
+                    if (val > maxVal) {
+                        maxVal = val;
+                        maxIdx = static_cast<int32_t>(d);
                    }
                }
+                dstData[o * inner + i] = maxIdx;


Why do we eventually change this one. Didn't it give performance gain?

The change from #1264 caused a performance regression for default axis=-1 case as in the PR description.

Maybe we can add some comment in here to not "optimize" it wrongly in the future then?

msluszniak · 2026-06-23T13:18:18Z

+import { validateModelSchema, SymbolicTensor } from '../../../core/modelSchema';
+import { wrapAsync } from '../../../core/runtime';
+
+import { type ImageBuffer } from '../image';


Suggested change

import { type ImageBuffer } from '../image';

import type { ImageBuffer } from '../image';

feat: add semantic segmentation task and native image operations

f084902

barhanc self-assigned this Jun 22, 2026

barhanc added refactoring feature PRs that implement a new feature labels Jun 22, 2026

barhanc linked an issue Jun 22, 2026 that may be closed by this pull request

[RNE Rewrite] CV - add semantic segmentation pipeline implementation #1242

Open

barhanc added 3 commits June 22, 2026 16:48

refactor(cv): optimize applyColormap and document layout requirements…

bc774e9

… for semantic segmentation

refactor(cv): validate colormap input parameters and document layout …

ff18eb8

…formats

feat: add semantic segmentation screen

359a6e4

barhanc marked this pull request as ready for review June 23, 2026 12:58

barhanc requested a review from msluszniak June 23, 2026 12:59

msluszniak reviewed Jun 23, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RNE Rewrite] feat: add semantic segmentation task#1275

[RNE Rewrite] feat: add semantic segmentation task#1275
barhanc wants to merge 4 commits into
rne-rewritefrom
@bh/semantic-segmentation

barhanc commented Jun 22, 2026 •

edited

Loading

Uh oh!

barhanc commented Jun 23, 2026 •

edited

Loading

Uh oh!

msluszniak commented Jun 23, 2026

Uh oh!

msluszniak Jun 23, 2026

Uh oh!

msluszniak Jun 23, 2026

Uh oh!

msluszniak Jun 23, 2026

Uh oh!

barhanc Jun 23, 2026

Uh oh!

msluszniak Jun 23, 2026

Uh oh!

msluszniak Jun 23, 2026

Uh oh!

msluszniak Jun 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	import { type ImageBuffer } from '../image';
	import type { ImageBuffer } from '../image';

Conversation

barhanc commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Introduces a breaking change?

Type of change

Tested on

Testing instructions

Screenshots

Related issues

Checklist

Additional notes

Uh oh!

barhanc commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

msluszniak commented Jun 23, 2026

Uh oh!

msluszniak Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

msluszniak Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

msluszniak Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

barhanc Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

msluszniak Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

msluszniak Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

msluszniak Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

barhanc commented Jun 22, 2026 •

edited

Loading

barhanc commented Jun 23, 2026 •

edited

Loading