The deck is 84 slides. The recommendation sits on slide 71. A decision is expected by Friday. Three questions I’d ask before I signed.
- What would have to be true for this recommendation to be wrong?
If the vendor can’t name two or three specific conditions – a covariate they couldn’t source, an adstock they had to assume, a holdout window they couldn’t test – the model has been presented as more certain than it is. I’d push back. Every honest model has a fault line. Ask where this one sits.
- What happened to last quarter’s recommendation?
If the vendor recommended a 15% shift into paid social last quarter, what did the next read say about that shift? Vendors rarely volunteer this. The pattern I look for is whether prior recommendations survived contact with the next data refresh, or whether each quarter’s read quietly contradicts the last one.
- Which number in this deck would you stake your bonus on?
Not a rhetorical question. A senior analyst should be able to point at one specific estimate – a channel’s return on investment, an elasticity, a saturation point – and say, this one I’m confident in, and here is why. The numbers they hedge around are the numbers I’d hedge around too.
None of this requires a statistics degree. It requires the vendor to show their working. If they cannot, the deck is not yet ready to be signed off, regardless of what Friday’s diary says.
If you’ve a deck like this in front of you and any of the three questions is awkward to answer, drop a line to hello@themmmdoctor.com – happy to take a look.