Wyze AI Newsletter - Aug 2021

Wyze AI
1

Hello all,

Greetings and happy August! As a company based in Kirkland, we are enjoying the golden time of year full of sunshine (yeah, even though we are still working from home and it’s sometimes smoky out there). Hope you are having good weather and a great day when opening up this post!

In this month’s edition:

  • :confetti_ball: Model keeps improving especially for Pet Detection!
  • :trumpet: Face Recognition v2 is close to beta testing with new UI!

Wyze AI Model Updates

In the past few weeks, tens of thousands of images were added to the model training to recognize people, pets, vehicles, and packages in different scenarios. Specifically, performance of Pet Detection is improved more than 10%, and comes very close to our other types of object detection!

image
image634×937 26.6 KB

Amazing!

Also, as the doorbell reaches more Wyze users, we are training a special model for the doorbell specifically to perform better in the porch scenario with portrait view. Stay tuned!

New AI System Testing

First, I want to thank everyone for participating in the new system testing. During the test, we found several key issues from the back end and fixed them. And we made the bug fixing process simpler than before. Your device should already get the latest fixes automatically. If you still see any abnormal things on your testing device, feel free to shoot us a message at aifeedback@wyze.com.

We’ll expand the test to more people in two weeks or so. You can also sign up for testing here if you want to test it with us.

Face Recognition v2 Testing

We are wrapping up the last part of testing! If you are using our beta app, you should already be able to see the new interface of face recognition in the latest beta app. Starting from 2.24, we’ll gradually roll it out to more people to test. No need to use a signup form, it will be there shortly!

Sound Detection Testing

Phase one of Sound Detection Proof of Concept (PoC) is moving to its end and we are moving into making it a real production-level feature. During the PoC testing, we encountered a very interesting problem and we have no answer to it yet. The question is:

When a baby makes noises (not crying, more like yelling, murmuring, etc.), do you want the Wyze Cam to pick it up or not? This is really a headache for us now. The feedback from parents we asked has so far been a split. Let us know your thoughts in the comments!

image
image1200×800 41.4 KB

The testing is still ongoing. You can find the sign-up sheet for sound detection here.

AI Data Bounty Hunter

We’re still looking for the data examples below as we build sound detection for Wyze parents! If you have any examples of the following videos, please send them to us at your convenience to help us train the model.

  • Videos with babies inside the frame
  • Videos with audio of a dog barking
  • Videos with audio of a baby crying

You can submit the video using the banner at the bottom of your Event video.

Before You Go…

This month’s open topic is, “How would you imagine Wyze AI helping your Wyze experience beyond Wyze Cams?” If you have any ideas like, “recommend some popular rules” or, “automatically turn off the lights if I go to bed,” feel free to leave your comments below. We’ll pick the most creative ones and reward them with some perks!

Best,
Shawn from the Wyze AI team

8 Likes
3

I love these AI Newsletters!

V3’s and VDB’s don’t work with face recognition testing yet?

@WyzeShawn It sounds like you are confirming that only the V2’s do Facial detection currently. In the Reddit AMA one of your team members seemed confused why the V3’s weren’t working for facial detection and said we should submit support tickets or logs if V3’s weren’t also working with facial detection testing (they aren’t). The doorbells don’t seem to work either. Why is there an option active to enable facial detection for V3’s and Video Doorbell if it doesn’t work for them yet? Do you have any time estimate on when it might be working for other cameras?

Crying Detection Suggestions

For Crying vs other sounds, it seems the best option would be to train it to recognize the difference and have them as different options. Maybe one that is crying only, and another to select all baby sounds. At the very least you should probably have a submission option for non-crying sounds so that people don’t keep submitting any baby noise as “crying” just because it is something that person wants to get notifications for, but isn’t actually crying or what many others want notifications for (maybe they just want alerts about an emergency like make sure the baby isn’t hurt). It could confuse the AI. I have other camera brands that do “Crying detection” and they frequently trigger from the TV when someone is yelling or laughing and mistake that as crying. I would suggest definitely using several examples of yelling and laughing to train the AI that is not crying so that your model is better than competitors at reducing false detections.

If you MUST pick between baby crying or ALL baby noises, and can’t separate or do both, I would definitely suggest only the crying. This is for a few main logical reasons:

  1. A LOT of people using crying detection expect it to clearly tell them when the baby needs attention or needs to be checked on, or that they may be hurt. That’s what I use your competitor’s Crying detection for. If you added in babbling, murmuring, yelling, giggling, etc into that I would never use it. Well, maybe YELLING might be okay in some instances if it is angry upset type of yelling, because that serves the same purpose as crying sometimes. But adding in any other baby noises would be nerfing the whole point of this detection notification IMO. It would be helpful for SORTING and finding events of the baby, but in theory, the facial detection should help with this. I know the face detection DOES recognize baby faces because it created a profile of my baby’s face. So people should be able to use that for a lot of sorting purposes anyway.
  2. Adding other sounds will increase false detection issues and not be as clear.
  3. If someone plans to use this as a baby monitor and wants all baby sounds, then presumably they could just enable sound detection and sound detection notifications in general. That’s how my current baby cam monitor (Owlet) works anyway. It has sound detection levels and assumes any noise in the baby room is the baby. If I get other sounds, then I reduce the sensitivity. So anyone needing to be alerted of “Any baby sound” should just do the same thing since most baby cams just work this way. Then in addition they can have crying detection to get different alerts for that, or to easily filter to find events of crying instead of just other baby noises. Doing things this way will still allow the parents who want all noises to be able to support getting what they want, while not ostracizing the group who only wants crying/emergency noises which will otherwise be impossible to limit if you include all noises in the detection.

The other question is whether this is just for babies or any kind of crying? If mostly babies, then what age range counts as a baby? When do they graduate past the age of the detection training? Some companies train for ALL types of crying, including adults, while others train the AI to just the youngest infants, but not older toddlers such as those of sentence speaking age. That should be clarified too in your AI training.

Allowing AI Detection and AI Notification to be separate individual options.

Here is a current major problem with the AI. I want the AI to detect Person, pet, vehicle, package, and everything else possible on my front cam so I can sort and find videos. But I only want NOTIFICATIONS of “PERSON” or maybe “PERSON or PACAKAGE” detections. This is currently impossible. If I tell it to detect my pets or vehicles for easy sorting/filtering of events to find something that happened, it will send me constant unwanted notifications of all vehicles and pet events, when I don’t want notifications of those. If I deselect those from the AI Events section, Now I won’t get notifications for those anymore, but I also can never filter to find videos with pets or vehicles. This is very frustrating that the notifications are an ALL or NOTHING selection. I can’t have it detect EVERYTHING but only notify me of people. PLEASE fix this. It is currently the most frustrating thing about Wyze’s AI system. The only real solution is to use multiple cams, and have one that only detects the things I want notifications for, and another right next to it to detect EVERYTHING, but not notify of anything. I shouldn’t have to use 2 cameras next to each other everywhere just to solve that issue when I could use the second camera somewhere else instead if Wyze would add a separate notification option for each AI option instead of all in the same “Wyze AI Evens” that requires all or nothing. I hope that wasn’t too confusing.

(I will think through and list more answers to this month’s open topic in a separate post).

Thanks for your time Shawn! :slight_smile:

8 Likes
4

  • I have put a couple of Motion Sensors outdoors - yes I know they are not weather proof, but wanted something to turn lights on and then off after motion ended. So for a new Rule is to add Person Detection and Person Detection End so that I can trigger my lights to go on and off when Person detection has ended for x minutes.

  • as @carverofchoice stated above, the V3 and VDB is not providing any face recognition even though I stand in front of the devices and it lets me know that a person has been detected.

  • add the ability to recognize multiple faces. Frequently, my wife and I or kids go in and out of the garage, this is where my V2 Camera is, I find it seldom recognizes anyone when more than one person is present.

5 Likes
5

I very well could be wrong but I think he means he 2nd version of facial detection not that it is only on v2’s. I will try and figure that out but that is what I am thinking.

3 Likes
6

Thanks, that would be great to know. I have only heard of one person getting one single facial detection on a V3, and then not again as far I know. Everyone else I know has said they’ve never gotten facial detection to work on any V3’s even though it has always been working for them on V2’s. It just seems weird there would be an option for it, but never working.

7

I have been able to verify it is version 2 of Facial Detection and it should be working on most/all devices that have Cam Plus on them not just the v2’s

1 Like
8

Can anyone else verify if Facial detection has ever worked on V3’s or Video Doorbell for them? Almost everyone I’ve talked to so far has said it’s only ever worked on V2’s…so I would love to find out anyone else’s experience related to facial detection on other devices.

Here is a survey I hope people will answer so we can get a feel for what people are experiencing with this.

PLEASE CHECK AS MANY OF THE FOLLOWING AS ARE TRUE FOR YOU

  • Facial Recognition has only ever worked on my V2 cams.
  • Facial Recognition works on my V2’s, but it works on my other Cam models too.
  • Facial Recognition regularly works on my V3 cams.
  • Facial Recognition regularly works on my Video DoorBell.
  • Facial Recognition worked randomly once on my V3.
  • Facial Recognition worked randomly once on my Video DoorBell.
  • I haven’t paid attention to which cams it works on or not, so I am not sure.
  • I am not in the Facial recognition pilot, so I don’t know.
0 voters
1 Like
9

so I have facial recognition on ALL of my cams, V2 V3 and doorbell. but the quality of the pictures I used weren’t the best. so the filter has not worked. capturing a picture from a moving object isn’t the optimum prospect for these cams. I wish we were able to upload a GOOD picture to use as the base picture, with facebook and everything this would be exceedingly simple to use as identifiers.

1 Like
10

As you know, Facial Recognition, for me, has only worked on my V2 Camera. I have stood in front of the VDB, it reported of a person, picture is clear and of my face. No Facial Recognition on the VDB. Did the same on my Porch V3 cam, and the same no Facial Recognition.

2 Likes
11

AI possibilities:

Default Thermostat Schedule Improvements

Have it offer to track and analyze all our motion and contact sensors and specific indoor cams with person detection and then after a period of time (month? longer?) consider what patterns there are for when people are home or away and suggest a new default Thermostat schedule based on predictable patterns of when people are predictably home or not home. I know it currently tracks based on our manual input…but some of us are too lazy to manually change the thermostats when we leave or return, so the AI figuring out when we leave or return could give more accurate suggestions than relying on me to not be lazy and give manual input (which is rare).

Electrical Grid Machine Learning

There are a bunch of companies now offering devices that you connect to your electrical box, and it tracks the watt usage and patterns going through it and can figure out what devices in the house are on or left on. If you made something that could track the wattage use and patterns going through it, your AI could be trained to figure out what things are on or off and offer it at a significantly better price than a lot of these other companies that are selling expensive options. One I looked at getting is that Sense Energy Monitoring. This type of thing using your AI capabilities would fit nicely into the Home Monitoring system aspect you are trying to build and it could make recommendations about TONS OF STUFF, almost anything in the home, and send alerts if someone leaves on the curling iron too long, etc.

Detect Smokey Environment or Flames regardless of detector sounds

I know this one is camera-related, but I’ll mention it anyway. Alerts if the camera/AI recognizes the room or environment is getting smokey/foggy or sees flames (not just reliant on sensors or sounds).

Adjust Lighting Brightness to the Time of Day

AI could automatically adjust the lighting to the time of day or possibly the degree of cloudiness (estimated light outside). I like my lights brighter in the daytime, but as it is darker, I don’t need them to be as bright when it’s darker outside. Though that might SEEM to be backward for some people at first thought…at night a little light goes a long way, but during the day I need MORE light for it to make any difference in the room (try it and see what I mean).

Room Sensor/Light Rules/Automation Suggestions

Allow the AI (machine learning) to guess what sensors should or could be part of a group to work together in some way. For example, if it notices that certain motion sensors are often triggered in succession, or maybe let’s say I have a family room with multiple motion sensors, the AI could suggest that we make all those motion sensors into a group and have that group determine whether or not to leave certain lights on or off (it should/could notice that particular lights status of on or off has a high correlation with particular sensors being triggered or not. It could then offer something like Making all my family room motion sensors be part of a group, and as long as any of those sensors in the group have detected motion in the last X minutes, it will leave those lights on, and if none of them have detected motion for X amount of time (whatever it figure out means nobody is in the room anymore), then it will turn the lights off. Right now we can’t have multiple sensors act this way (in a group as though they are all a single sensor), but the AI could figure out what kind of pattern most likely means nobody is in the room anymore or not. Currently, a single motion sensor is not accurately able to do this from farther away than like 5-6 feet unless the person is constantly moving substantially (like always walking), but if they are sitting or laying down, the sensor is only accurate from like 5 feet away. Therefore the AI could figure out how to use multiple sensors to actually make this room presence detection more accurate/possible.

Wyze Robot Vacuum Suggestions

The AI could suggest when is a good time to run my Robot vacuum, either because of how long it’s been or because nobody is in that area and it hasn’t cleaned there in a while, etc. I know we can set up schedules, but I don’t do this because of various obstacles. I need to run the vacuum when Baby isn’t around, when people aren’t sleeping, and preferably nobody is in the room, which the AI could figure out for me and remind me now is a good time to run the vacuum.

Allow the Wyze AI to connect through Home Assistant, Smart Things, etc = endless possibilities.

If you allow your AI to connect to third-party things through Home Assistant or other options, your AI can basically do anything with nearly unlimited data entry from any competitor too. Imagine being able to leverage data from ANY AND ALL smart devices, including any that you didn’t have to expend all the R&D on yourself, and leverage all those other devices or things with your own ecosystem to make it better than almost anyone elses. You could potentially collect data on competitors or anything else given the right permissions and all of that would be priceless to you, let alone the value all us customers get from it. Instead of Home Assistant being a pain in your pocketbook, it could turn your weakness into a giant strength in countless ways. I could use Wyze’s AI to control or give suggestions about a 3rd party thermostat, let alone the AI get insights about when I’m home or where I am from some of those third-party devices or control them through the bridged API’s. In my opinion, this is the biggest no-brainer if you are trying to expand your valuable with HUGE moonshots of gargantuan potential and unlimited options and resources for your machine learning. Take off all the handicaps and play nice with all the 3rd party options. I guess the new MATTER initiative might help with some of this, we’ll have to see.

Outside Data Input such as Weather

Linking the AI up to outside data input. For example, Sprinkler Plus already gets info on when it will rain and how much. If the AI had access to that data, it could be used as a rule to remind us of other things related to the weather. Maybe warn us that a window is open (based on the contact sensor there) when it’s about to rain or currently raining. It could tell nobody is currently using or in the garage, but the door was left open at night and prompt us to close it.

(More may be forthcoming as I put some thought into it)

4 Likes
12

This is exactly the same thing for me and my VDB.

2 Likes
13

The face recognition is working on v2, Pan, v3, DB now.
If you cannot see the toggle, it’s because the account isn’t whitelisted yet.
If you enabled Face Recognition but failed to capture the faces, can you shoot me some clips to my email so that I can take a look with the team? thanks
my email: sniu@wyze.coim

2 Likes
14

Sorry guys if I made it confusing. Face Recognition v2 is actually referring to the version 2 system, not Wyze Cam v2. I’ll get it right next time!

2 Likes
15

Thanks, I was told (by you) months ago that I was white-listed for everything with facial recognition, and the others, and I do see the toggle for facial recognition on all the cameras (V2, V3, Pan, Doorbell) but it has never detected faces on a V3 or VDB, and most people I’ve talked to report the same thing.

I will do some clear videos of my face on a V3 and include the video (and my account) to your email and show screenshots that the options are selected and the AI still does not ever detect a face no matter how clearly visible it is. Maybe it will help you guys figure out why it’s not working for most people. I’m sure some others can do this too to help.

1 Like
16

Can you also send me your Wyze account email? Just to double check why the Face Recognition toggle is not shown.

1 Like
17

I just PM’d you Shawn.

1 Like
18

Wyze must be working on this, I just got my first ever V3 Facial recognition while trying to run some tests again for Shawn! I have run 2 dozen tests just today (face right in front of my V3’s) and the very last one finally caught my face for the first time despite probably hundreds of attempts before this! Going to run some more tests now and see if I can replicate or if it was a weird fluke. But it is exciting I was finally able to get 1 single face recognition from a V3. I think everyone who’s been submitting to Shawn has helped this. Now I need to rerun my tests before I submit to shawn. Great work community!

1 Like
19

Okay @WyzeShawn I sent you an email with 3 examples, My Video Doorbell, and 2 different v3 cams.

  • I show you (via screenshot) I have the Face detection pilot and that the toggle is on.
  • I show you (via screenshot) in the event screen that it did not recognize my face (it’s the top/first event for each example per camera).
  • I include a full copy of the cam plus event videos.
  • I include screenshots of my my profile to show it exists already and there are 48 source examples of my face in there.
  • I included logs for each device.
  • I included the firmware version (latest beta firmware) for each cam
  • a screenshot of my App version,
  • Screenshots showing that the same event when uploaded to Google photos, recognizes my face clearly (so it’s obviously not quality of the video that is the problem since my face is easily recognizable),
  • Included my Wyze Account info for you to see Face detection is active
  • A link to the beginning of @spamoni 's testing in the Face Recognition use Thread so you can see what he’s been discovering and that it could be an Android-related issue.

I am willing to do any other tests or provide whatever else you need, but I tried to cover everything I could think of. I hope it helps.

2 Likes