Unveiling the Power of Invisible Watermarking: A Meta Journey
At Meta, we've embarked on an exciting path to harness the potential of invisible watermarking, a technique that's revolutionizing content provenance across our platforms. From detecting AI-generated videos to verifying the original poster, invisible watermarking is a game-changer. But here's where it gets controversial: scaling this technology is no small feat. We're excited to share our journey, from overcoming deployment hurdles to optimizing performance.
The Challenge: Scaling Invisible Watermarking
Invisible watermarking is a sophisticated media processing method, embedding a subtle signal that's invisible to humans but detectable by software. It's a powerful tool for content tagging, but bringing it to production at scale is a complex task. We faced challenges with deployment environments, bitrate increases, and visual quality regressions. But we didn't shy away; instead, we adapted our approach to real-world use cases.
Understanding the Basics
Digital watermarking, steganography, and invisible watermarking are related, but each has its unique purpose and characteristics. Invisible watermarking stands out for its robustness, surviving edits and offering a persistent solution for content attribution and protection. It's a superior alternative to traditional methods like visual watermarks or metadata tags, which can be easily lost or distracting.
The Scaling Journey: From GPUs to CPUs
Early digital watermarking research focused on modifying spectral properties of images, but these methods fell short for dynamic content. Today, state-of-the-art solutions like VideoSeal use machine learning, offering improved robustness against social media edits. However, deploying these solutions comes with computational challenges. GPUs seemed like an obvious choice, but they're specialized for large-scale model training and inference, not video transcoding. This led us to explore CPU-based solutions.
GPU to CPU: A Shift for Optimization
Our initial attempts with GPUs revealed low utilization and bottlenecks. Data transfer overhead, inference latency, and model loading time were major challenges. Recognizing these limitations, we turned to CPU-only inference. By optimizing threading parameters and sampling techniques, we achieved remarkable results. The end-to-end performance on CPUs was within 5% of GPU performance, and we could run multiple FFmpeg processes in parallel without increased latency. This breakthrough led to a more operationally efficient solution.
Validating CPU Scalability
To validate our CPU solution, we conducted load tests in a distributed system. The results confirmed that our CPU-based approach could perform at scale, comparable to our local tests. This achievement allowed us to provision capacity efficiently, outperforming GPU-based solutions in operational efficiency.
Optimization Challenges and Trade-offs
Scaling invisible watermarking presented optimization challenges, requiring a balance between latency, watermark detection accuracy, visual quality, and compression efficiency. Optimizing one metric could negatively impact others. For instance, a stronger watermark for higher accuracy might lead to visible artifacts and increased bitrate. We had to strike a delicate balance.
Managing Bitrate Impact
Invisible watermarking introduces increased entropy, leading to higher bitrate for video encoders. Our initial implementation showed a 20% increase in bitrate, impacting user experience. To mitigate this, we developed a novel frame-selection method, reducing the bitrate impact while improving visual quality and maintaining detection accuracy.
Ensuring Visual Quality
Maintaining the invisibility of the watermark was crucial. Despite high-quality metric scores, we observed visual artifacts. We addressed this by implementing custom post-processing and crowdsourced manual inspections. This subjective evaluation was key, as traditional metrics couldn't capture the unique artifacts introduced by invisible watermarking. We fine-tuned our algorithm, balancing visual quality and detection accuracy.
Key Takeaways and Future Directions
Our journey provided valuable insights. CPU-only pipelines, with the right optimizations, can match GPU performance at a lower cost. We learned that traditional video quality scores are insufficient for invisible watermarking, requiring manual inspection. The bar for production use is high, and we had to expand our knowledge to minimize BD-Rate impact while maintaining excellent detection accuracy.
We successfully deployed a scalable watermarking solution with impressive latency, visual quality, and detection accuracy. As we continue to improve, our North Star goal is to enhance precision and copy-detection recall. We envision invisible watermarking as a versatile 'filter block,' seamlessly integrated into various video use cases, offering a robust content provenance solution without compromising the user experience.
And this is the part most people miss: invisible watermarking is not just a technical feat; it's a game-changer for content attribution and protection. We invite you to share your thoughts and experiences in the comments. How do you see invisible watermarking shaping the future of content provenance?
