Service Advisory 02/16/2024

[johnyjuice]

I’m still having trouble with my V3 pan cam

[PanCamJeff]

Here is the update that was sent out via email, received early this morning…

Wyze Friends,

On Friday morning, we had a service outage that led to a security incident. Your account and over 99.75% of all Wyze accounts were not affected by the security event, but we wanted to make you aware of the incident and let you know what we are doing to make sure it doesn’t happen again.

The outage originated from our partner AWS and took down Wyze devices for several hours early Friday morning. If you tried to view live cameras or Events during that time, you likely weren’t able to. We’re very sorry for the frustration and confusion this caused.

As we worked to bring cameras back online, we experienced a security issue. Some users reported seeing the wrong thumbnails and Event Videos in their Events tab. We immediately removed access to the Events tab and started an investigation.

We can now confirm that as cameras were coming back online, about 13,000 Wyze users received thumbnails from cameras that were not their own and 1,504 users tapped on them. Most taps enlarged the thumbnail, but in some cases an Event Video was able to be viewed. All affected users have been notified. Your account was not one of the accounts affected.

The incident was caused by a third-party caching client library that was recently integrated into our system. This client library received unprecedented load conditions caused by devices coming back online all at once. As a result of increased demand, it mixed up device ID and user ID mapping and connected some data to incorrect accounts.

To make sure this doesn’t happen again, we have added a new layer of verification before users are connected to Event Videos. We have also modified our system to bypass caching for checks on user-device relationships until we identify new client libraries that are thoroughly stress tested for extreme events like we experienced on Friday.

We know this is very disappointing news. It does not reflect our commitment to protect customers or mirror the other investments and actions we have taken in recent years to make security a top priority at Wyze. We built a security team, implemented multiple processes, created new dashboards, maintained a bug bounty program, and were undergoing multiple 3rd party audits and penetration testing when this event occurred.

We must do more and be better, and we will. We are so sorry for this incident and are dedicated to rebuilding your trust.

If you have questions about your account, please visit support.wyze.com.

Wyze Team

[Seapup]

The 4 variations of emails are posted here:

Update on Investigation of 2/16/24 Security Issue

We have sent emails out to all affected and unaffected Wyze users from the security issue that occurred on 2/16/24. The first email went to all unaffected users. The second email went to users whose event thumbnails were made available to others but not tapped on. The third email went out to users whose event thumbnails were made available to others and were tapped on. The fourth email went out to users who had thumbnails made available to them that were not their own, but their thumbnails w…

[tynwistle]

I don’t have any cameras, but my system was taken down on the 16th, and it took me 3 days to recover with zero help from Wyze. I don’t think we are getting the full story here.

[itbedave]

“To make sure this doesn’t happen again
…”. Didn’t you say that at least once before???

Servers don’t get “mixed up.” They execute commands written by humans.

Blaming AWS and Servers isn’t a good look Wyze.

[richkut]

First of all, kudos to you for such an extensive troubleshooting guide!

HOWEVER, Wyze product owners SHOULD NOT have to ever be in a position to have to perform such extensive tasks in order to get their equipment working again (on Wyze’s plate, not yours–you did great!)!!

Seriously, how many users even have the time to get into this so deeply? How many users have the technical ability to handle all of this? Why should users be forced to jump through hoops such as these, lest their equipment will not work if they don’t?

Have we not paid Wyze for the privilege of having our equipment and associated systems they offer us work as advertised? Has your car manufacturer recently asked you to repair your car by yourself after handing you a set of written instructions? I doubt it (LOL)!!

Bottom line is that if the system goes awry (which all systems do sooner or later), the equipment should be self-healing (e.g., self-reboot and analyze, as well as reconfigure themselves to the way they were prior to the outage, etc.).

Here is an example: if my ISP (AT&T–fiber) has an outage and I lose my internet signal, when it comes back on their gateway and my wireless router AUTOMATICALLY sense the return of the signal and return themselves back to the operating condition that they were prior to the outage. This is the way it should be!! If Win 11 craps out and I get the blue screen of death, USUALLY a self reboot corrects things!

So putting us Wyze users in the position of having to figure out all of these tech issues just to get our equipment back up and running is INSANE!!!

Personally, I am fortunate enough to have a tech background and my equipment came back on WITHOUT USER INTERACTION) within about a day or so, and since I knew it was a system problem and not on my end I “patiently waited it out”. I also am retired, so time spent on this issue is not nearly as critical an item for me as it is for people that have 40+ hours committed to their jobs.

Regardless of the price point, though, since Wyze DOES offer home monitoring in addition to just online cameras, they should be responsible enough to keep a better eye on things, be significantly better in their communications with all concerned parties and design all of their equipment to self-heal as much as is technically possible, offloading such problems to their staff and not to the general users who so rely on them.

Thanks for the opportunity to offer my take on this mess, and best of luck to everyone in getting their gear to once again work as advertised and expected. I know that it is a huge struggle for the average user who just wants their equipment to work properly without first having to get a formal tech education, but remember that the squeaky wheel eventually get the oil!!

[stargazeriinh]

Thanks for your advice here @richkut ! However, I found that a lot of people here like conflict and to waste their time dealing with all issues in hopes of finding answers and communicating them online here. It’s only when you can step back and review what you face do you realize what you purchased and what you’re getting into… LOL
For me… I’m all done till I can see that WYZE, actually Wyzed up !
@stargazeriinh

[peepeep]

@itbedave

Servers don’t get “mixed up.”

Hey itbe’

I am lay-plus, not an IT maven. Lay people are used to hearing about ‘buffer overflow’ which corrupts memory. Bad actors try to create buffer overflow so as to gain entry to a system and do mischief, as I understand it.

So, they are trying to mix up a system in order to mess it up further - to their benefit and/or other dark desires.

Open to correction, just representing the lay-plus as best I can.

[itbedave]

That’s not at all what they are claiming. Good theory though.

[bryonhu]

I think I got a thumbnail pic from one of your cams Peepeep, hope Mrs. Peepeep isn’t too upset it was leaked by Wyze

[peepeep]

Should I be ashamed if I’m a little aroused?

[N0tanks87]

Well as of today 2-19-24 i keep getting error 7 on 2 of my battery cam pros and then fail to connect. Anyone have any ideas? Ive reset the cams. Power cycled them by removing the batteries while i reset the router. Nothing is helping.

[dwhitney67]

Same here.

My v3 cam is connected to the WiFi, but I cannot get the Wyze app to connect to it.

After performing the sequence of powering down the camera, resetting it, etc., it becomes apparent that repeating these steps is useless.

The problem is with the app, not the device(s). Maybe the Wyze AWS server is down?

[N0tanks87]

It could be. I hope.thats all. I just ordered a couple of v3’s. Hopefully they will show up and work lol. Did anyone else get the email updat from wyze just now?

[Antonius]

I am using the iOS app and all of my V3 cameras are on line and working. How did you try to reset your camera. If you look at device info on your camera what does the firmware version say. I got my problem V3 on line Saturday. Take a look here annd see if it will help you out.

V3 remains offline after 2024/02/16 outage - Device Info reports Wyze Cam Floodlight firmware

This has been reported and discussed elsewhere, but figured the issue is more than worthy of its own thread. For those of you that still have V3 cameras offline, you may want to check the firmware version listed within Settings | Device Info. It appears that several of us have V3 cameras that will not come online after the outage. Prior to the outage, all of my V3 cameras had the same version of firmware. After the outage, one of my V3 cameras now lists firmware that is used for the original…

[N0tanks87]

Does anyone else get error code 7 when trying to view the live stream on battery cam pro?

[mjmgnferro]

Did you try just reinstalling the cameras through the app top left corner plus sign

[N0tanks87]

I have on one of them. The other is to high up to get too without a ladder. Didnt work on the one i tried it on. Maybe i need to uninstall the app and reinstall and see if that helps.

[dwhitney67]

I can’t get the app (on my iPhone) to connect to the camera, thus I’m stuck. It is not possible to query the camera for the firmware version until the app issue is remedied.

[Antonius]

I did it the other day with the camera off line. Open the app, tap your camera, tap the settings gear top right.
Go down to device info and see what it states.
Have you tried clearing the cache from the app and force closing the app and opening it again. I am using iOS app 2.49.2 (2) and it works just fine.