AWS Outage Impacting Multiple Wyze Services - 12/15/21

12/16/21 3:35 AM PT - Thank you for your patience! At this point, the majority of Wyze devices have reconnected. We see that some have not successfully connected with the cloud since the initial outage and these devices may need a bit of help. If your device still isn’t connecting, please first try power cycling the device (removing the power or turning it off and then turning it back on). If that doesn’t work, you may want to try factory resetting the device to restore connection. A factory reset will require you to set up the device again. If that doesn’t work, please contact Wyze Customer Support.

If you are unsure of how to power cycle or reset your affected device, please check our support articles. We see that Wyze Video Doorbell has had a lower reconnect rate than other Wyze devices. Here is a troubleshooting article with instructions for rebooting and resetting this product. Please note that you can power cycle Wyze Video Doorbell by flipping the breaker and that you do not need to disconnect wires.

12/16/21 1:26 AM PT - Most Wyze devices appear to have connected again but we are not back to pre-outage levels yet. We’re continuing to work on this and monitor our metrics.

12/16/21 12:15 AM PT - We’re seeing good improvement trends! We’re not fully recovered yet and some products (such as Wyze Video Doorbell and Wyze Plug) are recovering at a slower rate than others. We’re still seeing some AWS IoT errors when the cloud tries to send a command to a Wyze device through IoT. We’ll let you know when we have another update.

12/15/21 11:15 PM PT - AWS tells us that we should be seeing further improvement shortly. We are slowly seeing improvement in our metrics as time progresses and are continuing to work with AWS to restore service to all of our customers.

12/15/21 10:16 PM PT - We’ve heard from AWS that they’re still working on the backlog and are now reaching a point where they can speed up the process. We’ve received more confirmations from customers about some of their devices coming online so we are seeing improvement confirmed in our community. Thank you for your patience!

12/15/21 9:10 PM PT - AWS says that they’re still working on the backlogs and are increasing the speed slowly. We’re seeing reports from customers saying that some devices have come back online but not all of them yet. We’ll keep you posted.

12/15/21 8:18 PM PT - AWS tells us that they’re seeing the operations slowly getting processed but that there’s still some backlogs to clear out. Thank you to everyone that’s been keeping up with these updates!

12/15/21 7:03 PM PT - AWS is still working on mitigation for the remaining issues. We’ll let you know when we hear more.

12/15/21 6:17 PM PT - AWS is working on scaling for us and plan to onboard our infrastructure when they have things stable enough for it. We are continuing to work with them on this issue. Some Wyze devices are seeing a more rapid recovery than others.

12/15/21 5:24 PM PT - We’ve been told that AWS has still not fully recovered and that the AWS team is continuing to work on it. Thank you for your patience.

12/15/21 4:11 PM PT - AWS is continuing to work on this outage. They’re trying to balance resolving the issue without accidentally overloading systems and haven’t found the right balance yet. We’ll continue to keep you updated as we hear more. We again apologize for any inconvenience you may have experienced as a result of today’s outage.

12/15/21 2:34 PM PT - AWS has let us know that we are not recovered yet. IoT-related functions such as showing device status, changing settings (like on/off, bulb colors, etc.), using Rules, Wyze Home Monitoring function, the Home Monitoring tab in the Wyze app, and third-party integrations such as Google Assistant and Alexa are still impacted. However, Wyze Cam Event notifications and live streaming as well as basic app functions such as viewing the device list in the Home tab have stabilized. We are continuing to work with AWS on these issues. Please make sure you do not reset your devices during this time to avoid complicating the recovery for your devices. We are sorry for any impact that this outage has had on you and will resolve it as soon as possible.

12/15/21 12:16 PM PT - We know that some of our services such as Wyze Bulb voice commands and other products and services continue to be impacted by the AWS outage. AWS is continuing to work on restoring service. Thank you.

12/15/21 10:36 AM PT - AWS says that portions of their service have been recovered and they are working on remaining components such as new components. We’ll continue to update you as we hear more.

12/15/21 9:44 AM PT - The main issue remaining on our end is difficulty connecting to AWS IOT. AWS reports that they are aware of this and are actively investigating this issue.

12/15/21 9:23 AM PT - Amazon is seeing recovery on their end. Our services will take some time to recover because of built-up commands from during the outage. We are working on this with the help of AWS and appreciate your patience.

9:00 AM PT - There is currently a widespread outage with Amazon Web Services (AWS) that is impacting our services such as the online/offline status of devices in the Wyze app, the ability to change settings such as arming or disarming Wyze Home Monitoring devices in the Wyze app, assigning Cam Plus licenses, using our Centercode test platform, and using our forums. If you are a Wyze Home Monitoring subscriber, please use your Keypad during this time to arm or disarm your service. We are working on getting our systems back up as soon as possible. We apologize for any inconvenience this has caused.

(Sorry, folks. Basically just rolled out of bed and caught up on this one. I apologize for my delay! :sweat: )

[Mod Note]: Corrected date in title.

24 Likes

Thanks, Gwendolyn. FYI, wrong date / typo in headline.

See also

10 Likes

Did I mention how I just rolled out of bed? Thanks! :sweat_smile:

10 Likes

12/15/21 9:44 AM PT - The main issue remaining on our end is difficulty connecting to AWS IOT. AWS reports that they are aware of this and are actively investigating this issue.

8 Likes

You should send out an email when this occurs. We shouldn’t have to search on the forums to check if an outtage is occuring.

65 Likes

You should set up a automatic email alert system for this kind of thing or a manual email alert. This way customers get the email saying “xyz happened … here’s the update”

54 Likes

Agreed on the notifications. Email, push, text, some sort of opt-in notification is necessary because we use our Wyze cams for security, and basically if we are without monitoring, we need to know about that, and not just assume there’s no activity.

33 Likes

I got a home monitoring email about an hour ago, and a Cam Plus email a few minutes ago. So emails are going out. Thousands to millions take time to process.

Push notifications are out because they are affected by the same AWS services.

4 Likes

Email blast shouldn’t be under AWS. If AWS IOT fails, email blast under AWS will be iffy.

4 Likes

Point of information, I got a successful Wyze Alexa detection announcement two minutes ago. Events still unreachable. Cameras working fine.

I think the most frustrating thing about this is that Wyze devices cannot be used offline on a local network. Every time I ask Google to turn something on, I need permission from Wyze to do so. I wasn’t aware of this years ago when I first bought into the Wyze ecosystem. Now I’m at my studio and embarrassed in front of a client having to crawl behind racks just to turn on devices because my Wyze plugs aren’t working. This is absolutely the last straw for me. I will be turning to another smart plug and cameras that can be used locally without a need for the permission of Wyze to use. Sorry Wyze. You did good for awhile but have failed me for the last time.

26 Likes

Same here:

2 Likes

wish I would have read this earlier. Would have been nice if Wyze sent out an email or something. Not all of my devices are coming back online. I uninstalled 4 smart plugs and have been trying to re-install them for like the last hour.

15 Likes

You need the permission of Wyze to set up your devices. If they are down. So are you.

1 Like

Amazon is not owning up to the ongoing outage. They claim it was fixed. Things take time to recover but…

Salesforce.com also took some big hits from this.

3 Likes

Yeah sucks. I guess I will try again later.

1 Like

Hindsight but for next time. If this outage was noted earlier, your customers wouldn’t have done a hundred things each to figure out if the problem was at their end or yours hence the “built up commands” you have to clear. There has to be a system better than relying totally on just you to post the outage when you get up in the morning. Someone at Wyze must have been awake! You’re great but you’re only human.

10 Likes

An e-mail, Twitter, Facebook, etc. to your customers would have been greatly appreciated!!

I have two Wyze Thermostats that were showing not connected. I assumed I was the problem so reset my router, then reset one thermostat, and still nothing. Now I can’t turn the heat on with the one that I reset. Fortunately I didn’t touch the other one and used it to turn on the heat in at least one floor.

8 Likes

When all of a sudden things start whacking out and not working right, best thing you can do is investigate but just sit back and wait , starting to fix and troubleshoot can just cause more problems

5 Likes