I’ve asked the TSA dudes while going through security this exact question and they confirmed that, yes you can take fully frozen water bottles through security.

You can also just read on their website that, as long as it’s fully frozen when you’re at the security screening, you’re fine. https://www.tsa.gov/travel/security-screening/whatcanibring/items/ice