# Proteus vs Real-ESRGAN: Temporal Axis Matters More Than Detail

Marcus Vance · August 6, 2026

> Proteus vs Real-ESRGAN: Temporal Axis Matters More Than Detail. On Replicate, a Real-ESRGAN run costs about $0.0052. That still-frame...

| Takeaway | Detail |
| --- | --- |
| The PSNR gap is a temporal artifact, not a detail win. | On frozen frames, Real-ESRGAN is sharper; Proteus's advantage depends on sequence order, and Replicate's $0.0052 per-run price makes a still-frame check cheap. |
| A video benchmark rewards sequence stability, not texture detail. | Topaz's REDS-Val puts Proteus ahead, but $1 of Real-ESRGAN runs can test many frames and reveal the frozen-frame reversal. |
| Don't let a temporal-consistency score hide the sharper still-frame model. | Real-ESRGAN wins the frozen-frame test even though Proteus wins the moving sequence; at the per-run price, you can run many Real-ESRGAN stills to verify it. |
| Detail comparisons should freeze the video before judging sharpness. | The temporal smoothing that boosts Proteus's PSNR is not extra detail; Real-ESRGAN's $0.0052 per-run cost makes frame-by-frame checks easy. |

On Replicate, a Real-ESRGAN run costs about $0.0052. That still-frame price makes it easy to forget that video is a different problem: the benchmark that puts Proteus ahead rewards consistency along the temporal axis, not per-frame detail. Freeze any frame from an SD clip and Real-ESRGAN is the sharper model.

Topaz's REDS-Val benchmark underlines the point. Proteus posts a higher PSNR than Real-ESRGAN on video, but that margin is a temporal-consistency artifact. The two models are not competing on texture generation; they are competing on whether the sequence stays stable. When the sequence is collapsed to a still, the detail ranking flips.

The lesson is not that Real-ESRGAN is a better video model. It is that the temporal axis matters more than detail in this comparison. A short clip at Proteus's per-frame speed spends a long GPU run to produce a score that says more about smoothness than about sharpness. For evaluating real quality, freeze frames first; the price of a Real-ESRGAN test is only $0.0052, but the insight is worth more than the dollar.

![Concrete relevant scene alley ruin](https://static.mm-ais.com/article-images-ai/proteus-vs-real-esrgan-temporal-axis-mat-ai-05fa50df.jpg)
Concrete relevant scene alley ruin

## Mechanism

Proteus does not beat Real-ESRGAN-v4 because it sees more detail per frame; it beats it because it sees five frames at once. The +1.2 dB PSNR gap on 480p REDS-Val footage is a temporal-consistency effect, and the architecture makes that unambiguous. Feed Proteus a single still, and its entire motion path produces nothing; feed it a pan, and the same model earns the gap above. The delta is not detail — it is the absence of flicker.

Proteus treats 480p video as a sliding five-frame window. A modified SPyNet flow network estimates motion at quarter resolution, then DCNv3 deformable convolution fuses the warped frames before any super-resolution happens. Temporal alignment precedes spatial upscaling, so the SR backbone never sees a raw, unaligned frame. Real-ESRGAN-v4 has no motion path: it is a single-image RRDB U-Net trained with a high-order degradation pipeline — second-order blur, sinc kernels, noise, JPEG compression — as described in the arXiv tech report "Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data." Because it cannot distinguish camera motion from actual texture, it treats a pan as a series of independent textures, and each frame receives a slightly different hallucinated detail pattern. That per-frame flicker is exactly what PSNR and SSIM penalize.

The verifiable knob is the training loss. Proteus's loss is L1 + LPIPS + 0.4 · temporal-warping L1, where the temporal term compares the current output to the flow-warped previous output. That 0.4 weight is the entire separation between a video model and an image model. Set it to 0, and the architecture still receives five frames, but the gradient pressure to remain temporally stable vanishes.

The ablation confirms it. With the temporal-warping term set to 0, Proteus's PSNR drops 0.9 dB and its warped-frame standard deviation rises. Same model, same backbone, same parameters — minus temporal state, it behaves like a still-image upscaler. That 0.9 dB is the same order of magnitude as the 1.2 dB advantage over Real-ESRGAN, which tells you most of the gap is temporal memory, not spatial reconstruction quality.

After the fusion stage, Proteus passes the aligned frame to a SwinIR-v2 backbone with 8×8 window attention. That yields a much larger effective receptive field than Real-ESRGAN's U-Net path — but only after the temporal alignment stage. The spatial backbone is not why Proteus wins on pans; the motion compensation in front of it is.

| Component | Proteus | Real-ESRGAN-v4 | Winner on 480p video |
| --- | --- | --- | --- |
| Input | Sliding 5-frame window | Single independent frame | Proteus — motion can be exploited; Real-ESRGAN is blind to it |
| Motion estimation | Modified SPyNet at 1/4 resolution | None | Proteus — pans are aligned before SR; Real-ESRGAN flickers |
| Frame fusion | DCNv3 deformable convolution | None | Proteus — warped frames contribute; Real-ESRGAN has no warp |
| Backbone | SwinIR-v2, 8×8 window attention | RRDB U-Net | Proteus — larger receptive field, but only after alignment |
| Temporal loss weight | 0.4 · temporal-warping L1 | 0 | Proteus — stability is explicitly enforced; Real-ESRGAN has no temporal term |
| Ablation at weight 0 | −0.9 dB PSNR, higher warped-frame std | N/A | Temporal term explains most of the video gap, not the backbone |

The practical test: run Proteus on a single 480p frame, and the five-frame window collapses to one — the flow network has nothing to estimate, the DCNv3 fusion degenerates to an identity alignment, and the temporal-warping loss penalizes nothing. The model quietly behaves like an image upscaler. That is the canonical decision rule in action: moving sequence → Proteus; single frame → Real-ESRGAN. Everything else is the headline hiding the mechanism.

![Mechanism — Proteus vs Real-ESRGAN](https://static.mm-ais.com/article-images-ai/proteus-vs-real-esrgan-temporal-axis-mat-ai-99d55ce3.jpg)

## Evidence

According to Topaz Labs benchmark report v1.3 (Jan 2026), on REDS Val 480p clips Proteus scores PSNR 27.42 dB / SSIM 0.889; Real-ESRGAN-v4 scores 26.22 dB / SSIM 0.870 — a +1.20 dB / +0.019 SSIM margin with a paired t-test p

Canonical: https://ai-videoupscale.com/blog/proteus-vs-real-esrgan-temporal-axis-matters-more-than-detail.php
Markdown: https://ai-videoupscale.com/blog/proteus-vs-real-esrgan-temporal-axis-matters-more-than-detail.php/index.md