In the worst case, you’ll need five consultations from the builder.
It's easy for the more maths orientated to think the answer is twenty-four times, arguing that the total number of possible tower arrangements — 4×3×2×1 — is also the number of consultations required. But that isn’t true at all. Every single answer you receive from the builder tells you how many tower blocks are in the right order, and this information can be used to minimise the number of consultations by learning about the structure as you go.
Here's one such example for your perusal:
Think of the blocks as lettered and start by stacking them ABCD.
If you're still not convinced, here is a handy diagram showing every possible route:
Might want to let prospective contestants for 'The Price is Right' know this, in case they play 'Race Game'.