Thank you very much for taking the time to provide such a detailed and helpful reply.
The point about HDI versus the central interval is what struck me the
most — I hadn’t realised that ArviZ’s default of 94 per cent concealed
that ambiguity regarding the distribution of the residual tail. I’ll switch straight away
to central percentiles (95 per cent) for the next revision.
The explanation about the sign flip was also helpful: I was interpreting
the instability as a problem with the pipeline, rather than as the expected behaviour
of an effect compatible with zero — that distinction matters a great deal
in terms of how I’m presenting the results.
I’m also implementing the bootstrap you suggested to estimate the
frequentist variance of the estimator — it seems to me to be the right check to provide
a second, independent perspective on stability, in addition to the
sensitivity check I’d already carried out.
If you’re up for it, once I have the corrected version incorporating these changes,
I’d be keen to hear your second opinion — I fully understand if you don’t
have the time, but the quality of this feedback has made me want to give it some thought.
Thanks again,
Alex