I think it is probably the user agent of the browser inflating the text size on orientation change, check out text-size-adjust - CSS: Cascading Style Sheets | MDN
If you set that to 100% on the html/body tag I think it will solve it, make sure to add vendor prefixes for other browsers as well if you want to cover them!
html {
text-size-adjust: 100%;
-webkit-text-size-adjust: 100%;
-moz-text-size-adjust: 100%;
}