EBMV @ ECCV 2026 · Research Project

EventShift

Reliability-Aware Residual Adaptation for Day–Night RGB–Event Semantic Segmentation

Shao-Kai Liu1 · Yun-Tze Tsai1 · Chia-Yu Lin2 · Chia-Ming Lee1,2 · Chih-Chung Hsu1,2

1 National Yang Ming Chiao Tung University · 2 National Cheng Kung University · Taiwan

Event-Based and Multimodal Vision Workshop · ECCV 2026

Keep RGB as the semantic anchor. Let events make a bounded correction—only when illumination calls for it, where evidence is reliable, and by no more than the feature can safely absorb.

+4.58night mIoU over the matched RGB continuation46.81 → 51.39
−0.16day mIoU change while adding event correction55.03 → 54.87
+2.21balanced day–night average improvement50.92 → 53.13
+1.34DSEC11 mIoU under the matched schedule78.36 → 79.70
Day and night segmentation examples comparing RGB, published RGB-event models, EventShift, and ground truth
The trust conflict. Events can recover nighttime structure, but unrestricted fusion may disturb reliable daytime semantics. EventShift aims for a stronger operating point across both regimes.
01 · Problem

The modality worth trusting changes with the light.

RGB carries stable appearance semantics in daylight. Events reveal motion and boundaries when frames go dark. Treating both as equally authoritative all the time creates the wrong trade-off.

Daylight

Protect the RGB anchor.

Raw event activity is abundant even in daylight, but it is not uniformly useful for semantic decisions. Strong fusion can fragment small or appearance-heavy classes.

55.03matched RGB day mIoU
Low light

Spend event evidence selectively.

When RGB visibility falls, reliable event structure becomes valuable. The model should increase correction without giving the event stream an unrestricted semantic path.

+4.58night mIoU with EventShift
CIELAB lightness and chromaticity distributions for CoSEC day, CoSEC night, and DSEC11
Photometric evidence for the shift. CoSEC night frames occupy a much darker region than CoSEC day; DSEC11 lies closer to the daytime lightness distribution.
02 · Method

Three controls bound event authority.

EventShift changes one RGB feature state through a small residual update. The correction is governed at the image, spatial, and feature-magnitude levels.

When

An image-level low-light budget reduces correction in bright, high-contrast frames and opens more capacity when RGB evidence weakens.

Where

A learned allocator is biased by density, temporal balance, polarity balance, support, and multi-window edges.

How much

RMS normalization and clipping explicitly cap each sample’s residual relative to the RGB feature scale.

EventShift architecture with an RGB semantic path, bidirectional event voxel and reliability-aware bounded residual adapter
RGB remains the semantic path. A 10-channel bidirectional event tensor and reliability prior guide only a bounded residual correction at res3 and res4.
03 · Evidence

The same adapter behaves differently by illumination.

Switch between the measured day and night cases. The low-light budget and realized residual both rise when the RGB frame becomes less reliable.

Conservative by day

RGB already carries a useful scene representation, so EventShift spends a smaller correction budget.

Low-light budget0.56
Residual / RGB RMS4.1%
Measured day and night EventShift behavior showing RGB, events, reliability, allocation, residual, prediction and ground truth
04 · Results

Night improves without collapsing day.

The main evidence is a matched-backbone CoSEC19 ablation: every row uses the same Swin-L Mask2Former backbone, split, and training recipe.

Matched RGB vs. full EventShift

mIoU under the same CoSEC validation protocol.

RGB · day
55.03
Ours · day
54.87
RGB · night
46.81
Ours · night
51.39
Shared 0–60 mIoU scale.
Central finding
Bound the residual first; then spend it where event evidence is trustworthy.

Over bundled residuals, MW edge alone adds +1.56 and reliability/support adds +1.76 D–N average points; full EventShift reaches +1.78. The priors overlap, so their gains should not be read as independent or additive.

Matched-backbone CoSEC19 fusion ablation. Values are mIoU; D–N Avg. is the arithmetic mean of day and night.
VariantEvent useOverallDayNightD–N Avg.
RGB continuationnone55.1555.0346.8150.92
Input RGB + eventinput52.2052.1346.6149.37
Naive event adapterdirect52.1151.9747.1349.55
Bundled residualbounded55.1554.9347.7751.35
+ MW edgeedge prior55.0354.8450.9852.91
+ reliability/supportallocation55.0954.8651.3653.11
Full EventShiftbounded + all priors55.1054.8751.3953.13
Official challenge · Test phase
2nd41.11 mIoU

The challenge entry is a separate composite system built on the same reliability-aware residual principle. Its aggregate also includes the REAL subset, so it is context—not a direct comparison with the controlled CoSEC-only study.

Additional evaluation · DSEC11

Matched-schedule improvement

78.3679.70

mIoU on a separately trained internal validation split—not zero-shot transfer. Pixel accuracy also rises from 95.33 to 95.73 under the matched schedule.

How to read these results

  • Primary causal evidence: the matched Swin-L fusion ablation above.
  • Cross-architecture comparisons: useful context, but backbones, parameter counts, initialization, and source taxonomy differ.
  • Overall mIoU: recomputed from pooled confusion, so it is not the arithmetic mean of day and night.
  • Adapter size: 0.47M parameters, or 0.22% of the 215.93M-parameter model.
05 · Qualitative

Inspect what the correction changes.

The visual evidence focuses on the same matched mechanism: keep daytime semantics intact while recovering supported nighttime structure.

06 · Scope

Strong evidence, with clearly bounded claims.

The paper treats EventShift as a design principle supported by controlled experiments—not as a universal guarantee across every sensor, split, or deployment regime.

Controlled validation split

CoSEC uses 2,236 training and 327 validation samples. It supports controlled diagnosis rather than an official sequence-disjoint benchmark.

Proxy and ablation limits

The reliability prior is an uncalibrated deterministic proxy, and the individual contribution of res3 versus res4 injection is not isolated.

Fixed-latency representation

The symmetric multi-window tensor uses up to 100 ms look-ahead. Strictly causal deployment calls for past-only windows and an explicit latency–accuracy study.

Read the full study.

Definitions, implementation settings, class-level failures, challenge context, DSEC11 protocol, and complete qualitative evidence are documented in the final author version.

EBMV @ ECCV 2026Final author versionPDF · 5.0 MB
Open PDF

Paper figure

Original