How to Solve Video Color Inconsistencies Across Browsers
A technique utilizing the canvas element to play videos in their intended color space.
💡 Our Bright Idea
While creating a new website for a startup I was working for, it was decided early on that we wanted to tell the story of what the product was and how it helps remote teams be awesome using the medium of videos. These videos would be embedded right into the page itself using native HTML 5 video elements, they would autoplay (no controls shown), and they would appear seamlessly integrated into the rest of the page design. Primarily that the background color of each video would perfectly match the background color of its containing element on the page.
⚔️ The Great Mint War of 2018
For the most part, this worked out okay in practice, with the bulk of our video content being juxtaposed against the main dark-gray background of the site. However, there was one place where this design goal quickly broke down into the hot mess now known as the Great Mint War of 2018.
When we placed our main header animation video into its minty green sea of tranquility, it revolted violently in Chrome and Safari, yet remained at peace in Firefox. We were understandably perplexed.
🎨 Web Browser Color Profiles
After some initial digging around in the caverns of the Interwebs, we found some information leading us to believe that perhaps it was the color profile that was used to export the video from Adobe Premiere or Adobe After Effects that was the culprit. But then why was Firefox okay? After some more digging, it seemed that some browsers utilize their own color profiles and forcibly thrust them upon whatever MP4 file which dares wander into their dominion.
🗜 Video File Downsizing
Not being well-versed in the realm of video codecs, color profiles, and the like, we thought that perhaps we could simply remove whatever wayward data was causing such bad behavior by simply downsizing the video file — this, of course, was a good idea to do anyway as we quickly discovered that each video file shaved off a good 80–90% of its file size! The service that we ended up using, and which I recommend, is called VideoSmaller.
With our now 195 KB, 20-second video in hand (Yes! 195 KB video. Did I mention how awesome VideoSmaller is?), we tried our header video setup once more. But, alas, the colors still did not match in Chrome or Safari (yet they remained ever-correct in Firefox).
💡 A Newer Brighter Idea 💡
Completely bewildered and frustrated, we regrouped and came up with a new plan. If the colors are just going to be off in those browsers, we could at least set the background color of the parent element to the same background color from the video… right? I mean, how hard could it be?
Immediately, a couple of us had the same bright idea: let’s just grab a frame from the video, paint it to a canvas element, then grab the pixel data from a spot we know will have the background color, then apply that color as the background color of the containing element. Easy peasy!
To test this idea out, I set up a CodePen with the expected background color and the video embedded in the page. Then I searched around for some quick ideas on how to get images from the video painted to a canvas element. After a few minutes of sleuthing and coding, I had the beginnings of a solution. Except, I couldn’t get the pixel color information to show up in my console log.
🛑 Another Set of Hurdles 🚫
I asked around on the team to see if anybody could help me solve the mystery of why I wasn’t able to get my seemingly simple experiment to yield any fruit. Turns out there was a error being logged to the real browser console (not the simple CodePen implementation I was staring so longingly at):
Unable to get image data from canvas because the canvas has been tainted by cross-origin data.Gah-what!? A tainted canvas preventing me from using getImageData? Another round of Google-sleuthing clued me into what the problem was, and why it was, but it didn’t really help me figure out how to solve the problem. I had no idea how to properly ask for the video source from the CodePen side, nor did I find any simple-to-understand means of adding the necessary CORS header on the Amazon S3 or CloudFront side.
Something I did notice, however, is that when we painted a frame of the video to the canvas element, the colors were correct. I had a new hunch that even if we figured out the CORS issue and were able to successfully grab a pixel’s color data, we’d still get the wrong color.
🌈 The “Aha!” Moment, and a Scheme 🤔
We did, however, focus on this quite possibly wrong solution for longer than we may care to admit. Until at last we had a breakthrough, “Aha!” moment:
When we paint the video frames to the canvas, the colors match … in every browser!
“Could we just re-paint the entire video to the canvas element?” I pondered. Yes, yes we can! It didn’t take me very long to expand the simple proof-of-concept in CodePen to repaint every frame of the video to the canvas element in near-real-time. And just like that, we had video colors perfectly matching background colors in every browser! 🎉
📈 But What About Performance?
There is definitely a hit to the CPU while it constantly paints frames of video into a canvas element. So much so, in fact, that we’re already looking into other alternatives (my alternative of choice would be to export a few simple PNG images and animate the whole thing using pure CSS).
Currently we are using a setTimeout of about 30ms between calls to drawFrame (roughly 33.333 FPS). We tried using requestAnimationFrame, thinking that it would potentially be smoother and kinder to the CPU than constant timeouts, but it actually ended up being a lot choppier and didn’t seem to alleviate the CPU enough to continue using it.
We also tried using the same technique on all of the videos on our page, since it really does fix the color inconsistency issue, but that proved to be way too heavy-handed. The colors were fixed (which made the pixel-focused designer in me so happy 😃) but the cost of our relatively simple page on the CPU was unacceptable (and made the user-focused designer in me so sad 😢).
💭 Your Thoughts
So, what do you think of our solution? Have you struggled with color inconsistencies across browsers? What solutions have you come up with in the past to deal with problems like this?
About
🧔🏻 Dallas works at GitLab as a Fullstack Engineer. He lives in College Place, WA, with his wife, 👩🏻 Nicole. They enjoy taking walks with their dog, 🐶 Ruby.
