AMA SUMMARY FOR: "We're the Wyze AI team - ask us anything (again)! - 7/14/22"

[carverofchoice]

(Thanks to the AI Team for taking the time to give us all these updates and answers!)

We’re the Wyze AI team - ask us anything (again)! - 7/14/22

AI recognition of Moving Vehicles (not parked vehicles) and lights/shadows triggering motion detection

@WyzeZhongwei

Thanks for raising these important points, and yes, the team is taking bold efforts to address them!
For the parked vehicle issue, we are testing many different solutions to address it at both the notification and detection level. It’s actually much more challenging than it might seem to accurately detect when a vehicle is moving if you consider all edge cases and want to assure an overall good experience. But in short, we are accelerating a feature that can suppress notifications of parked vehicles, and are actively evaluating comprehensive solutions to differentiate moving vehicles vs. parked vehicles.
For the lighting condition triggering motion detection, we will soon be beta testing a new feature that can support event triggering/recording with AI tags. This AI-Recording works like you can set to trigger events only when there are interested AI-tags present. We hope this feature paired with detection-zones can mitigate the problem you are facing.

Ignore stationary vehicles

@WyzeWenjiang

We are working on moving vehicle detection, which may be launched by the end of this year or the beginning of next year. If you’d like a quick solution for now, we recommend adjusting your motion sensitivity or set up detection zones, if you haven’t already done so.

ETA on alerts for only moving vehicles (not stationary ones)?

@WyzeWenjiang

Thanks for the question! We are targeting the end of this year or the beginning of next year.

Plans to speed up notification arrival?

@WyzeWenjiang

We released new AI systems on most of the existing cameras to speed up notification speed earlier this year. We are still working hard on many aspects to improve the end to end notification experience. Please keep your firmware updated and we will keep improving it.

You used to have an Edge AI job position posted, are you now actively developing local/edge AI solutions?

@WyzeLin

Admittedly, it is a significant challenge to run AI on cameras at our price as we are limited to less-powerful chips. However, we are investing in Edge AI technology which can also be used to optimize our Cloud AI. Meanwhile, we are optimizing our AI to run simple models on existing cameras, which by themselves are not as good as more complex cloud models but when both work together they can deliver the best trade-off and value to our users.
You might also find my answer to another question interesting: https://www.reddit.com/r/wyzecam/comments/vz3mhl/comment/ig6cjkt/?utm_source=share&utm_medium=web2x&context=3

[This is the next answer just below]

Are there plans for onboard AI for future Wyze Cams? That would speed up the notifications and save Wyze money without using the cloud for events.

@WyzeLin

It is pretty challenging to make cameras affordable at our price and at the time same time equip them with powerful chips for onboard AI. That said, this is on our radar and we believe this will become possible as technology advances. Meanwhile, we are optimizing our AI to run simple models on the existing cameras, which alone are not as good as more complex cloud models but when both work together they can deliver the best trade-off and value to our users.

Will you work on processing AI detections on a local device to get notifications faster?

@WyzeShawn

The idea of processing AI on a specific device is on the roadmap and undergoing evaluation right now. Currently there are several aspects we need to consider, incl. speed, compatibility, cost efficiency etc. We’ll be able to update you more on our progress later on. Thanks!

Instead of motion in the detection zone causing the AI to detect the entire screen, how is progress on having the AI only scan detection zone areas of the screen too?

@WyzeWenjiang

We recently released this feature to many of our cameras (v2/pan/v3). Users will just need to make sure their firmware is up to date. With this feature, the motion region is compared with the detection zone, and the AI will only tag the video if there is an overlap.

@WyzeWenjiang

Followup comment:
[This feature] is in the general release firmware. Not all new features are mentioned in the firmware release since some of them are only service related.

[So to help forum users know why something wasn’t detected, we may now we need to ask to see their detection zones, since that could be the reason an object in view was excluded from detections.]

Tell us about the feature to show us what the AI detected and tagged as an object?

@WyzeWenjiang

Yes! We are currently testing a new feature that shows detected objects on the camera feed’s thumbnail. Stay tuned.

Another community member who couldn’t attend this wanted me to ask for him: “Does the [event submission] feedback actually help when submitting the correct feedback [or detections]? I have spider webs that will be labeled as people all the time and submitting the correct feedback for the last 8 months or so has not really improved much.”

@WyzeZhongwei

Thanks for asking, and we really appreciate your friend’s help by submitting their feedback! User feedback is the foundation upon which we build and improve our AI models. All our AI models are derived from user-submitted data!
We are sorry to hear their issue hasn’t yet been resolved by our latest model updates. We continuously update our models using the most recent user-submitted feedback, and we will also be rolling out new features to help make it easier for users to diagnose their AI issues. Sometimes the incorrect tag may not come from where people might initially think.
We treasure all user-submitted data and it is a massive help when users provide detailed information by checking/un-checking related tags when your AI results are wrong.

Tree limb shadows being detected as a vehicle in the backyard

@WyzeShawn

Thanks for sharing your experience. We understand the inconvenience when AI makes the wrong predictions, and our team works constantly to improve our AI models.
A bit more about why sometimes AI fails and what we are doing to improve: The shadows of many things present similar (at least to computers) visual shape/structure of our target tags under certain circumstances, so the AI models are tricked as it aligns their knowledge. What we can do is to keep expanding their knowledge by feeding more diverse data and applying more advanced techniques. We already curated many new data sets to address the specific failures we heard from our users, and we observed improvements with recent model updates.
We will keep working on this! Note: When you submit the videos of failures through the button at the bottom of the event viewing page, please make sure uncheck the incorrectly predicted tags.

Floodlights triggering motion alerts

@WyzeZhongwei

This is a very interesting use case, thank you for sharing it. AI features can definitely jump in here to improve the experience in your case. This is actually aligned with our new AI-Recording feature I roughly mentioned in another thread.
For the floodlight scenario, with AI-Recording you can specify only person as your interested tag, and when you walk by the floodlight, the light turns on and the camera captures a video. Then when you walk away, and later when the light turns off, no video will be recorded as there’s no person in scene.

What are the chances of getting an upgrade to the detection zone (for the AI, motion, etc)? This second version of the D.Z. being a grid is a HUGE improvement over the original singular box detection zone, but it is still not as precise as many competitors’ methods allow and as many of us would really prefer. We can’t do precise lines at angles, etc. This could also be more important if you come out with higher resolution cameras as your VP mentioned in his AMA that you would have some cameras in the future with higher resolution (in that case, each D.Z. square would potentially be covering twice as many or even more pixels now).

@WyzeShawn

Thanks for sharing your thoughts on the detection zone, we share the same feelings as you. I’ll pass your feedback along to our camera team for their further consideration!

Ability to add or remove multiple Faces at a time (instead of individually).

@WyzeShawn

Thanks for the feedback! Admittedly we deprioritized this item at first. However, as time goes on and more faces are picked up by the camera, this problem is becoming more critical in some cases. We are working on the next iteration of Friendly Faces, and this is at the top of our list of problems to be fixed. We plan to roll out the new design sometime later this year.

When is friendly faces coming to the doorbell?

@WyzeShawn

Thanks for your interest in Friendly Faces! Right now, Friendly Faces is a feature included in our Cam Plus Pro service. The team is working on bringing the service to doorbell shortly. Stay tuned with us!

Will the Matter Initiative have any influence over how you handle AI or anything else related to your team?

@WyzeLin

The Matter Initiative is definitely a good one to make smart home devices more connectable under the same standard. At Wyze, we believe smart home needs to have smartly connected devices. Where you could, for example, set up a rule in the Wyze App to set a Color Bulb to red when a package is detected by the Front Doorbell. With more devices added to the system under the same standard, more automations can be set up to make your home smarter. That is also why we are working on building our smart rule engine to make setting up automations an easy and enjoyable experience for our beloved users

Did you finish “Optimizing the pipeline” to allow the AI to analyze the entire event for all faces and detections now (not just the first one)?

@WyzeWenjiang

Thanks for this followup question! We released the new pipeline early this year and all Friendly Faces users are currently using the new pipeline. The entire video is analyzed by AI for recognizing faces in this new pipeline.

Is your AI using neural networks or rules based logic? And Can you give clarity into what are some hurdles you need to overcome in order to reach some of your milestones for the coming year?

@WyzeLin

We are using neural networks for many of our AI features, such as person/vehicle/package/pet detection by cameras. We also have hand-crafted algorithms using rule-based logic for some use cases, such as part of the on-device motion detection.
The biggest challenge for us is to reduce false detections from diverse corner cases, similar to training self-driving cars like Tesla. Despite that, our models have improved significantly over the past few years (though it’s not 100% accurate, and sometimes make mistakes).
A big challenge for us is our Cam Plus Pro, where we use person detection to trigger alarms. Although it is already much better than traditional motion sensors that have been used for decades, it sometimes makes mistakes, especially when the camera is installed facing a cluttered background.
We are collectively working on products and tech to include humans in the loop to verify the AI detection results. That way when AI makes mistakes, the user can help to correct them, while behind the scenes the AI will still help to significantly reduce the review burden.

Wyze Anything Recognition / Smart Vision updates?

@WyzeShawn

Thanks for staying tuned with Smart Vision! We opened up the first batch of closed group tests earlier this year for 1 month. Based on the feedback we received, the team started working on the next iteration intensively in June. The next round of tests will be with a bigger audience group and will be rolled out soon. Pay close attention to our email communications and communities, and sign up ASAP for a chance to give it a shot!

Current Face recognition limits are 300 pixels to maintain a 90% confidence interval = about 6ft away. How will you expand and improve these limits (higher resolution, or other new innovations that Steve McIrvin mentioned)?

@WyzeZhongwei

New hardware equipped with higher specs will help to produce better quality images that will directly benefit our Friendly Faces experience. Before we cut to the future improvements, I’d like to elaborate a bit more on the justification of current working limits.
Face recognition techniques generally have two major components: 1) Face detection, and 2) Face identity verification. Our limits on distance/pixel coverage are mainly to address the challenge of identity verification, even if faces can be easily detected beyond current limits. We have to capture certain levels of details to be able to reliably recognize the facial identity.
With upcoming new hardware and algorithms, we hope we can relax these limits and achieve a new level of experience. Our team is actively testing the potential of this, and preliminary results are promising. Stay tuned!

What other AI detections are you considering having as global options (trained for everyone to have)?

@WyzeShawn

Great question! Ultimately, we are counting on you all to tell us what you want us to build. Right now, things on our roadmap include expanding the horizon of object detection to baby care, pet/animal monitoring, as well as going deep vertically in the security scenarios with finer granular sub categories of existing object detection. Let us know what’s on your mind!

I would love more animal categories (ie: predators or individual species) instead of just everything being “pets”

@WyzeShawn

Absolutely! We want to protect our family, including fluffy members. Subcategory of animal detection is in our backlog right now, if you can submit the video through Wyze App (the link is at the bottom of the event viewing page), we’ll include these videos into our model training. The more data we have, the smarter the model could be!
In addition to that, you can also train the model by yourself! Stay tuned with our Wyze Smart Vision product!

Current Doorbell concerns: Person notification delay, Notifications outside of detection zone. iOS thumbnail previews stopped working.

@WyzeShawn

Thanks for sharing this feedback. Those problems are fixed on our cameras, however it will be highly resource intensive to port the work over to the doorbell considering its complexity.
That said, we’ll pass the request to the team and review it again.

Why can’t we edit subscriptions for Cam Plus purchased through the web and the app all in the same place?

@WyzeShawn

That totally makes sense. However, we are fighting directly against Apple/Google’s policies for subscription payments and management in the app. They make it really hard and in many cases flat out prevent us from streamlining access to manage web licenses. They also in most cases don’t allow us to direct our customers to external links to manage web subscriptions. On top of that, they also don’t provide an API to manage app subscriptions on the web.
You can check some of your favorite apps that use subscriptions like Spotify, Netflix, and others - they have entirely given up on trying to do subscriptions in the app and can’t even link to their own websites to tell people where to go when you are looking for where to go to manage your subscription. This is a hard one but we’re constantly trying to improve here while also testing out what the app stores will allow us to do. There are also governments that have ongoing regulatory action against Apple and Google that are designed to allow more freedom that we’re hoping will lead to positive change here.
Right now the team is working on improving the UI on the services.wyze.com to provide a better experience when you need to make changes to licenses. At the same time, we’ll keep spending efforts in the app to figure out the most viable solutions.

[Seapup]

Many thanks for doing this carverofchoice! As always… well done!