Existing video removal tools are surprisingly good at magic. You can paint over a stray tourist in your vacation footage, and AI will replace them with a reasonably convincing background. But if that tourist was leaning against a wall, or blocking the sun, or holding a leash, the illusion falls apart. The shadow stays. The wall looks weirdly untouched. The dog on the leash is now being held by an invisible ghost.The problem is that current models treat video editing as a 2D pixel-filling task. They are great at \u201cinpainting\u201d textures, but they have zero concept of causality. The new paper from researchers at Netflix and INSAIT, titled VOID (Video Object and Interaction Deletion), argues that we should stop thinking about inpainting and start thinking about counterfactuals.Instead of asking \u201cwhat pixels go in this hole?\u201d, VOID asks: \u201cIf this object had never existed, what would the physics of this scene actually look like?\u201dFrom pixel-filling to causal reasoning\u201cFigure 1: Removing an object and its interactions can require rewriting the entire scene. On the left, when the middle three blocks are removed, VOID correctly models the domino effect halting so that the yellow block never falls. On the right, when the hands are removed, VOID correctly models the spinning tops continuing without interruption.\u201d\u200a-\u200aall images are from the original research paper.VOID works by identifying the \u201ccausal ripples\u201d an object leaves behind. When you select an object for removal, the system doesn\u2019t just mask those specific pixels. It uses a Vision-Language Model (VLM) to analyze the scene and identify downstream effects\u200a\u2026 \u200alike shadows, reflections, or objects that would have moved differently if the target wasn\u2019t there.This analysis produces a \u201cquadmask.\u201d Unlike a standard binary mask (remove\/keep), the quadmask identifies four distinct zones: the object to remove, the background to preserve, the areas affected by the object\u2019s interaction, and the overlap where the object was physically touching something else. This gives the underlying diffusion model\u200a- a modified CogVideoX transformer\u200a – \u200aa roadmap of what needs to be fundamentally rewritten versus what just needs a texture patch.The two-pass pipeline and the \u201cjelly\u201d problemGenerating new physics is harder than filling in a static background. In early tests, the researchers found that when the model tried to simulate new motion\u200a (\u200alike a ball falling to the ground after the person holding it was removed\u200a) \u200athe moving object would often deform, turning into a \u201cjelly-like\u201d version of itself.To fix this, VOID uses a two-pass generation strategy. The first pass predicts the general counterfactual trajectory (where the object goes). The second pass then uses \u201cflow-warped noise\u201d derived from that trajectory to stabilize the object\u2019s structure. By aligning the diffusion noise with the predicted motion, the model maintains the rigidity of the object even as it simulates an entirely new physical path.\u201cFigure 3: Void: Interaction-Aware Counterfactual Video Generation. A user provides an input video and clicks on an object to mask it for removal. A VLM-based pipeline expands the mask to identify other areas that will be affected. VOID\u2019s first pass then predicts a counterfactual trajectory. The optional second pass stabilizes object deformation using flow-warped noise derived from the initially predicted motion.\u201dThe training data for this was built using Kubric and HUMOTO. Since you can\u2019t easily film \u201cground truth\u201d for a person being both present and absent in the same physical moment, the team relied on 3D simulations. They rendered pairs of videos: one with a physical interaction (like a collision) and one where the initiating object never existed, forcing the physics engine to calculate the alternate timeline.Examples from the paper.Community reaction: A100s and ethical anxietiesThe technical community on Reddit and X has been quick to pick up on the project, which Netflix has notably open-sourced on GitHub. While the results are impressive, the hardware barrier is steep. On r\/DefendingAIArt, u\/Anal-Y-Sis pointed out that the model requires a GPU with 40GB+ VRAM, effectively gating local use to A100-class hardware for now.On the more mainstream side of the fence, the ability to \u201crewrite history\u201d so convincingly has triggered immediate concerns about media integrity. In a thread on r\/whennews, u\/just4browse suggested that the tech will be used to mass censor media by deleting inconvenient objects or actors with zero trace of their original influence.Others view it as a necessary physics upgrade for the next generation of creative tools, moving us closer to video editors that act like world simulators.The end of the \u201cclean\u201d plate?I think the most significant part of VOID isn\u2019t the inpainting quality\u200a – \u200ait\u2019s the VLM-guided causal reasoning. For decades, VFX artists have relied on \u201cclean plates\u201d (filming the background without the actors) to handle object removal. VOID suggests a future where the clean plate is unnecessary because the AI can infer the \u201ccorrect\u201d physics of an empty room better than a human can recreate it.However, the reliance on synthetic data (Kubric) is a double-edged sword. While it provides perfect causal pairs, it\u2019s a far cry from the messy fluid dynamics or complex lighting of a real film set. I expect we\u2019ll see \u201cphysics-aware\u201d artifacts in early deployments\u200a, where a model might handle a falling ball perfectly but fail to understand how that same ball would splash if it hit a puddle.The real test for VOID will be how it scales as the backbone models (like CogVideoX) improve. If the underlying model understands the world better, VOID\u2019s \u201ccounterfactual\u201d approach only becomes more powerful.<\/p>\n","protected":false},"excerpt":{"rendered":"