‘Mamma Mia!’ actor recalls ‘unbearable’ day on set — and how Christine Baranski reacted to complaints