Update: 11/29 1:00AM PST - After few days of monitoring our metrics, we are confident calling the issue resolved. Both our CS team and internal metrics show that we are back to normal business as of 11/28.
Root cause of the issue: Our service provider made a service upgrade intended to better support the US market. However, during the deployment there was a bug introduced into the system which caused camera login failures for Wyze Cams. That caused connection outside the home network to fail since the device showed as offline for streaming. Around midnight on 11/26, the issue was identified and fixed. However a device restart needs to happen for each camera to re-login. Given how many customers we have, it took about 2+ days for most cameras to be back online.
Again, we are very sorry for the service disruption! Thanks for your patience and continuous support. We have learned a lot from the two service issues last week and that’s definitely not where we want to set the bar. We are determined to give our customers a much better experience and we sincerely hope we can use our actions to earn your trust again.
If you are still having trouble connecting, this may be due to another issue and we’d love to work with you on it. In that case, please send us a PM or fill out this form to contact customer support:
https://support.wyzecam.com/hc/en-us/requests/new
Best,
Wyze
History:
Update 11/26 6:27PM PST - Still under investigation but there is improvement.
Many people are saying that their connections are working better but while the situation is improving our connection percentage isn’t quite to where it usually is. We’re still looking into this. If a reboot hasn’t resolved this problem, please let us know!
If you aren’t near your Wyze Cam, you may be able to use the in-app restart option. You can find this feature in Camera Settings.
Update: 11/26 8:25AM PST - Provider updated the issue resolved.
We received an update from our provider that they have identified the root cause and fixed the server issue on their side. You should NOT need to use our temporary solution anymore. If your device is not working, you need to either power cycle your camera or go to Camera Settings → Restart Device. We would like to verify from our customers that the solution works now before we say the issue is resolved.
Update: 1:32PM PST - Temporary Solution.
We are still working on the root issue. Before we are able to get to the bottom of it, we are offering a temporary solution.
You can use this weblink https://user.wyzecam.com/upgradedevices to upgrade your camera firmware. (after log-in, click on user information -> upgrade devices). A manual firmware flash won’t work for this but the weblink should. We have a couple of other versions of the link if there ends up being a redirect issue or something but this is the most mobile friendly version. If you cannot see the menu, please try requesting the desktop version on your phone, using your phone in landscape mode, or switching to another browser (Chrome seems to be working pretty well for this but we’ve heard some reports of Safari having trouble).
Please note that this is a temporary solution. We are committed to find the root cause and implement a long-term solution. So, if you are not willing to use this temporary solution, we appreciate your patience while we working to fully resolve the issue.
For details about this temporary solution, please see below.
Our P2P (peer-to-peer) service provider uses global load balancing to optimize live stream connection reliability and speed.
After we launched Wyze Cam, we heard concerns from our community about traffic going outside of the United States due to the global load balancing. We then implemented “Server Restriction” to limit traffic to only US-based servers. Although we don’t think the server restriction is the root cause, it is related to this issue.
By removing the server restriction, a user should be able to view the live stream outside of their local network.
What does this mean?
- For the Live Stream, the P2P servers are still North America Only. So the live streaming video still stays inside North America.
The change is that in order to make device registration successful, a global list of servers (not only US-based) will be pinged, and the fastest server will respond for device registration. Before making this change, it would only use US servers.
- For Alert Videos there is NO CHANGE. Your alert video will still go to US-based AWS servers (Owned by Wyze) with the same encryption and security level.
Starting 11/24, we have noticed that some cameras can not be viewed live stream, outside of local network. The symptom is close to 11/19 issue, but the root cause has not been determined.
We are working on this issue, and will provide update as it progresses.
Sorry for the inconvenience.
