Your “all-in-one” router is a modem/router combo and you need the modem half. Just plug the Wyze Router into one of the LAN ports (not WAN) on your existing router using the provided Ethernet cable. You can leave or shut off the WiFi on your existing router. I would. And match the SSID and password so you don’t have to reset all of your Wyze devices.
I assisted my Wifes cousin in WVA who had frontier. We put in their own router and turned off their WiFi / Rtr and it worked fine. I did have to call them to get the logon info as her cousin didn’t know it.
Make sure when you set it up that you turn of the DHCP host on the Frontier router and let the Wyze Router do it.