The additional height corresponds to the difference in height between vertical-align: baseline and vertical-align: bottom. This is known as the "descender line." The seemingly arbitrary "5 extra pixels" can be attributed to this factor. If the font size is increased by a factor of 10, this gap will also increase by 10 times.
It appears that in cases where overflow: hidden is not applied, the baseline of an inline-block element aligns with the baseline of its last line of text.
This observation suggests that overflow: hidden actually enforces the baseline of the entire inline-block element to be at the bottom of the element. Even in the absence of visible text, the parent container of the inline-block reserves space for the descender line. In the provided example, due to the container having a height: 100%, the overflow of the descender line remains unseen.
The persistent presence of this space, despite no visible text being present, may be due to the fact that the inline-block element establishes an inline formatting context, leading to this occurrence. Had the element been a block, such a context would only be created upon encountering inline elements or text.
While just a hypothesis, this explanation seems plausible. It also clarifies the effectiveness of @Jonny Synthetic's resolution: by adding overflow: hidden to the parent container, the surplus space assigned for the descender line is concealed.
Credit goes to @Hashem Qolami for providing the jsbins which inspired this theory.