[RenderEngine] Allow rounded corners shader to use lower precision.

Big corner radii would cause problems at lower precision because
computing a vector norm would overflow. But since highp tanks
performance, instead scale down and back up by a power of 2 during the
normalization operation, so that a higher precision is not required.

Bug: 137191934
Test: eyeballing systrace when opening Photos, GPU seems to take shorter
time and no jank is observed
Test: Stepping through frames from a Pixel 3 XL screenrecording shows
that the beginning of the app opening animation is rounded instead of
square

Change-Id: Ie7a39bd0f992f7f42414ef49de9873775cc34409
Exempt-From-Owner-Approval: Approved in qt-r1-dev

1 files changed, 5 insertions, 4 deletions

diff --git a/libs/renderengine/gl/ProgramCache.cpp b/libs/renderengine/gl/ProgramCache.cpp
index 086a324999..d242677f0c 100644
--- a/libs/renderengine/gl/ProgramCache.cpp
+++ b/libs/renderengine/gl/ProgramCache.cpp
@@ -575,10 +575,11 @@ String8 ProgramCache::generateFragmentShader(const Key& needs) {
             float applyCornerRadius(vec2 cropCoords)
             {
                 vec2 position = cropCoords - cropCenter;
-                // Increase precision here so that a large corner radius doesn't
-                // cause floating point error
-                highp vec2 dist = abs(position) + vec2(cornerRadius) - cropCenter;
-                float plane = length(max(dist, vec2(0.0)));
+                // Scale down the dist vector here, as otherwise large corner
+                // radii can cause floating point issues when computing the norm
+                vec2 dist = (abs(position) - cropCenter + vec2(cornerRadius)) / 16.0;
+                // Once we've found the norm, then scale back up.
+                float plane = length(max(dist, vec2(0.0))) * 16.0;
                 return 1.0 - clamp(plane - cornerRadius, 0.0, 1.0);
             }
             )__SHADER__";