The Moment the Black Box Arrived
There was a point in our planning cycle when someone suggested we stop running prospecting and retargeting as separate budget lines on display. Both could run inside a single campaign, they said, if we passed back the right signals to the DSP.
The room was not convinced.
The discomfort was specific. In the old model, we knew who we were targeting. Prospecting line items went to new audiences. Retargeting line items went to people who had already visited the site. Each had its own budget, its own optimisation goal, its own line in the plan. The new model asked us to give that control to the platform and trust that the algorithm would find the right people across both.
The mathematics behind our discomfort was real. Retargeting was inflating prospecting CPAs. People who were shown a prospecting ad, came back a week later after seeing a retargeting ad, and then converted were being credited entirely to retargeting. Prospecting received nothing. Which meant the activity responsible for putting those people into the funnel in the first place looked inefficient on paper. Meanwhile, the learning that should have been compounding inside our prospecting campaigns was being diluted, because the pool of genuinely unconverted prospects was smaller than the data suggested.
This was the moment I started to understand what the black box era actually meant. It was not simply that the platforms were running our ads for us. It was that the decisions we had always made manually, who to target, in what sequence, at what budget, were moving to the platform side. And the platform would make those decisions well or badly depending on what we fed it.
Most brands have lived through a version of that moment. Many are still working out what it means
What This Chapter Is About
The first four chapters of this series were about evaluation. What methods do we have. How they work. What they can and cannot tell us. When to use which one. Chapter 4 closed with the argument that triangulating three sources of evidence beats relying on any one of them.
Chapter 5 asks a different question.
Once you have made an investment decision, how do you make sure the capital you have committed is finding the right people as efficiently as possible? That question now sits, largely, on the platform side. The platforms are not just distributing your ads. They are running active targeting decisions in every auction, responding to signals you provide, optimising toward outcomes you define. What you give them shapes what they do with your money.
This chapter covers three pieces of that reality. This article is about the signals themselves: what they are, how to think about them, and what responsible signal design looks like. M017 will look at how to measure systems you cannot see inside. M018 will examine what happens to your measurement when the act of using AI changes the underlying system.
The progression runs from fuel to trust to calibration.
The Admission Nobody Makes
Let me say the thing out loud.
Most in-week optimisation decisions are driven by platform-reported attribution. Last-touch dominates. In planning meetings and budget reviews, it rarely surfaces explicitly. But in the gap between MMM cycles and incrementality studies, which is most of the time, platform numbers fill the space. When performance is being reviewed week by week and decisions about where to shift budget need to be made quickly, teams use what is in front of them. And what is in front of them is the platform dashboard.
The challenge is not that platform numbers are useless. The challenge is that they are known to be limited, and people use them as if they are not.
Here is what that looks like in practice. Platform results point one direction. Cross-platform attribution from GA4 or Adobe points another. An experienced team understands why: platform attribution assigns credit within the platform’s own visibility. It cannot see what other channels contributed to the same journey. So the number it reports overestimates its own role. Most teams know this. They know that if they acted on platform numbers alone they would make the wrong call. So they try to bridge the two, adjusting what they see in the platform against what they see in their analytics suite before making a decision.
Not all teams have the time or capability to do that bridging. When they do not, the platform number wins by default. Budget shifts in ways that reflect how the platform reported the week, not what actually happened commercially. Over time, those micro-decisions compound.
The Platforms Are Not the Problem
It would be easy to frame this as the platforms overcrediting themselves. That framing is too simple, and it is not quite honest.
Platforms do not lie. They measure what they observe. The structural limitation is that they observe only what happens inside their own ecosystem. Meta cannot see the Google Search click that preceded the Meta view. Google cannot see the Meta impression that primed the intent. Each platform assigns credit to the journey as it appears from its own vantage point, which is necessarily incomplete.
This is not a commercial distortion. It is a visibility constraint. The same limitation applies to every walled garden in the media ecosystem, regardless of size or measurement maturity. A platform reporting on its own performance is always reporting on a partial picture.
The responsibility for resolving that limitation sits with the brand. Not because the platforms are acting in bad faith, but because only the brand holds the full view. The brand is the one entity that sees across all channels, all conversions, all customer journeys. If that view is never assembled and never connected to what the platforms receive, the platforms will optimise against whatever fragment they are given.
That is not a platform problem. It is a signal design problem.
The Signal Stack
Signals follow intent. Every interaction a potential customer has with your brand carries some information about how likely they are to convert, and how valuable they are likely to be if they do.
A visit that does not bounce quickly signals genuine interest in what was shown. A basket add signals intent for a specific product at a specific moment. Sharing personal details signals willingness to commit. A completed purchase is the confirmation of that journey.
Each stage down the funnel carries more signal weight. But the dimension that is most commonly missing from how brands think about signals is value. Two customers who both reach the purchase stage are not equal if one is buying a high-margin product and the other is buying a low-margin one. Two new customers are not equal if one is likely to stay for three years and the other is likely to churn in six months.
For a subscription business, this is especially concrete. You know, from historical data, what a new customer in a given product category is likely to generate over three years and five years. Finance produces NPV estimates. Those estimates are the most commercially accurate expression of what an acquisition is worth, and they are almost never what gets passed to the platform. What gets passed is a conversion event. A binary signal that says something happened, without saying how much it matters.
The goal of responsible signal design is to combine intent depth with value and lifetime dimension into a single coherent input. A signal that tells the platform not just that a conversion happened, but what kind of customer was acquired and what they are likely to be worth. The richer and more complete that signal, the better the algorithm builds models of who to find more of
Building the Infrastructure
Before server-side tracking, conversion APIs, and enhanced consent, signals were delivered via client-side pixels and tags. The problem with that model was structural. Browser-based tags could be blocked. Ad blockers, browser privacy settings, and the gradual deprecation of third-party identifiers all created gaps in what reached the platform. And client-side infrastructure was not built to carry rich value data securely.
The result was that the algorithms were optimising against a partial picture. Not a slightly incomplete one. A materially incomplete one. And because they did not know what they were missing, they were confident in their models in ways they should not have been.
Addressing this required a decision to invest in the underlying infrastructure before any of the platform capabilities it would eventually enable. At one point in my time overseeing digital marketing at a large subscription business, we worked through a Google-funded programme to build exactly this foundation. The programme, run over six to eight months with external delivery support, covered five specific capabilities: Enhanced Conversions, Conversion APIs, Value-Based Bidding, Advanced Consent Mode, and server-side GA4. Server-side Google Tag Manager underpinned all five, decoupling our data flows from browser dependency entirely.
Every capability that was selected had to defend itself against at least one of three commercial outcomes: improving return on media investment, recovering reach that had been lost due to privacy and signal gaps, or building a more defensible measurement foundation. The programme was not a technology project. It was a commercial decision with a technology component.
The most significant capability was value-based bidding. Passing lifetime value to the platform meant the algorithm could optimise not for conversion volume but for the type of customer the business actually wanted to acquire. Getting there required more than standing up a server-side tag. It required building a pipeline from Finance, bringing the latest NPV estimates into the data layer so they could flow server-side with each conversion event. It also required commercial alignment across the business: shifting from volume optimisation to value optimisation changes product mix, because the algorithm concentrates spend toward higher-value customers and away from lower-value ones. That is a business decision, and it needed sign-off as one.
Meta’s Step Toward Honest Measurement
Meta’s Incremental Attribution, broadly available from April 2025, is worth understanding precisely, because it is often described inaccurately.
The mechanism is a holdout model. A portion of the target audience is withheld from seeing the ad. Both groups, exposed and unexposed, are tracked for conversions. The difference in conversion rates between the two groups is the incremental lift attributed to Meta. The critical point is that this comparison is channel-agnostic. The unexposed group is withheld from Meta ads only, not from other channels. They are free to convert via paid search, organic, direct, or any other route they would naturally take. The difference between the two groups captures what Meta’s presence actually caused, not just what Meta observed.
This is a meaningful advance from standard attribution. Meta’s own data from 37 Conversion Lift Studies run across 30 advertisers and eight verticals in 2024 showed a 46% lift in incremental conversions when campaigns optimised against this signal rather than standard attribution. The signal tells the algorithm to find people who will genuinely convert because of the ad, not people who would have converted anyway.
The residual caution is this: it is a model-based estimate, not a transparent controlled experiment. Meta does not disclose the exact statistical assumptions used. And it answers a specific, bounded question: does Meta drive incremental conversions within the current media mix? It does not answer whether the system needs Meta at all.
That is a different question, and it requires a different instrument. A geo or matched market test removes Meta from an entire market and observes what happens to total business outcomes. Does volume fall by the amount Meta was claiming? Do other channels absorb the demand? Those are the questions that matter for material budget allocation decisions, and only a properly designed geo test provides the answer.
Use Meta’s incremental attribution as a directional signal and a daily optimisation input. Use geo tests when the commercial stakes are high enough to warrant the investment.
Where Platform Signals Belong
None of the above is an argument for dismissing platform signals. It is an argument for using them in the right role.
In-week optimisation legitimately belongs to the platform. Day-to-day bid adjustments, audience expansion, creative rotation, delivery pacing across placements: all of these decisions are better made by an algorithm processing real-time signals than by a human reviewing a weekly report. The brand’s role in that process is not to override the algorithm. It is to make the algorithm’s inputs as complete and commercially accurate as possible, then give it the structure and freedom it needs to learn.
That means simple account structures that do not fragment signal. Broad audience definitions that give the algorithm room to find the right people rather than constraining it to predefined segments. Multiple placements so the algorithm optimises across contexts rather than within one. And signals, as discussed above, that reflect real commercial value rather than proxy events.
MMM and incrementality studies operate at a different cadence and answer different questions. They are the governance layer, not the operational layer. Expecting them to guide in-week decisions is asking the wrong question of the wrong instrument.
The Multi-Platform Problem
Google and Meta lead the industry on measurement maturity. Their incrementality tools, their signal matching infrastructure, their feedback loops between conversion data and bidding systems, are more developed than most of the ecosystem. But most brands run activity across more than two platforms.
Relying only on the signals and measurement from the two most sophisticated platforms means making decisions about other channels using whatever those other channels report about themselves. Which compounds exactly the problem this article has been describing.
No single platform provides cross-portfolio truth. Each walled garden reports on its own contribution. The structural solution is a central measurement layer: MMM, incrementality studies at the portfolio level, or cross-platform attribution, not as replacements for platform signals but as the framework that makes signals from different platforms comparable and actionable at the allocation level.
Platform signals drive in-week execution. Central measurement governs allocation across the portfolio. Both are necessary. Neither substitutes for the other
Signal Sovereignty
The platforms will use whatever they are given. If the signal is incomplete, they optimise against an incomplete picture. If the signal stops flowing because of browser blocking, consent failures, or gaps in the data pipeline, the algorithm works blind. The cost of that does not show up in any report.
This is the uncomfortable truth about signal loss: it is invisible. Sales left on the table because the algorithm found the wrong people never appear in a dashboard. The only visible symptoms are indirect: a gradual inability to spend at the usual rate, a quiet drift in the quality of acquisitions, a delivery system that fills its impression quota but misses the customers it should be finding.
The investment case for signal infrastructure is therefore difficult to make from the data that exists, because the data that exists reflects the world with the gap already in it. The counterfactual, what would have happened with better signals, is not observable. Which is precisely why most brands underinvest in it. The cost of inaction is invisible. The benefit of action is only measurable after the fact.
Measurement sovereignty sits with the brand. Not because the platforms are untrustworthy, but because the brand is the only entity with the full commercial picture. Platform measurement informs. It does not govern. Keeping that distinction clear is the brand’s responsibility, not the platform’s.
Closing: The Box Is Not Blind
Platform AI is not a black box in the sense of being arbitrary. It is a black box in the sense that you cannot see its internals. You see the inputs. You see the outputs. What happens in between is opaque.
But the inputs are yours. The signals are yours. The conversion data, the value data, the consent infrastructure, the account structure that gives the algorithm room to learn: all of this sits with the brand. The box is only as black as the signals that enter it.
The next question is harder. How do you measure what the box is doing when you have no visibility into its workings? How do you build confidence in a system you cannot inspect? How do you govern something you do not control?
That is M017. And it matters more now than it ever has.
If you are working through how to build signal infrastructure that reflects real commercial value, or trying to make sense of what platform signals are and are not telling you, this is the problem KaiSignals works on.
