Here’s a copy of the #ama for everyone who just wants to read everything here in the forums and without having to sort through all the irrelevant comments just to find the 26 Answers given.
If you want to look at the original source, you can search for the answer in the following link and then link to the originally typed answer directly:
https://www.reddit.com/r/wyzecam/comments/13bwqw9/reddit_ama_592023/
Wyze App and Firmware Engineers | 05/09/2023
“We are the Wyze App and Firmware Engineering Teams - ask us anything!” Link
Hosted by:
- u/WyzeMJ, SVP of Engineering
- u/WyzeMitchell, Firmware Development Lead
- u/WyzeMaximK, Principal Software Engineer
| Question | Reply |
|---|---|
| Dark mode on Android and iOS | We are working on these features already, initially for the camera live view screens, camera group, and HMS/CPT camera screens. We are aiming for as early as 2.43 but sometimes delays can happen. -u/WyzeMaximK |
| "App Engineering Question for Maxim Komleu: Wyze users are frequently told about some App wishlist requests that some things we ask for aren’t easy to program/implement due to “the way the app is structured,” and that to implement some of them would require waiting for a whole rewrite of the app from the ground up to implement several of these requests. Can you elaborate for us on what makes this so difficult compared to other apps so that some things just aren’t possible or as easy to do right now as some other people with programming experience have thought it should be? What kind of design debt is causing these limitations? How is it structured differently than you would like? | It is not just engineering question, but also reasonability, marketing, and so on. Sometimes, it is not a problem to implement something, but it conflicts with company projections or conflicts with other features and deadlines for them. As an example, we were receiving a lot of requests for dark mode support. We finally did that, but before the release we needed to work on other more critical problems. - u/WyzeMaximK |
| “Also along these lines, knowing if there are hurdles from the backend architecture/infrastructure vs the way the Wyze app has been built to manage so many devices over time vs having multiple apps. The only secondary app is the Wyze Car app and that might be because it was originally “just” an April Fools joke product but became real later on after seeing continued interest. Are microservices being used or planned for to improve development processes?” | In terms of back-end technical architecture, we have adopted a technical framework of microservices. However, this will not improve our hardware development speed. But we will try to speed up from the hardware development process. - u/WyzeMJ |
| “Why are rich notifications never consistent? I have four cameras, and it’s a complete toss-up on any given moment which notifications will have the rich notification image.” | We have two types of notifications. A general notification and a rich one. The general notification is sent when an event happens but is still processing to get a more immediate response. The rich notifications are sent after the cloud has processed the event and the recording is finalized. - u/WyzeMaximK |
| “Notification when a camera or the Wyze server goes off line would be awesome also landscape for tablet please” | We are looking for the solutions and the main concern on our side we do not want to flood users with false events. We are currently working on adding comprehensive landscape layout and it is a high priority UI feature. We are actually working on portions of this that could be released as early as 2.43 or 2.44 to improve UI. - u/WyzeMaximK |
| “Firmware Development question for Mitchell Hansen: Recently, we were told that some newer Wyze devices like the OG cams have switched from Linux on RISC-V to FreeRTOS on ARM. All we have really heard about this is that it is part of the reason the OG’s can reboot really fast, load the live stream really fast, and send out slightly faster notifications compared to other cameras. Can you tell us more about your experience of the strengths and weaknesses between using Linux on some and FreeRTOS on some? Do you have a preference? Why? Will Wyze mostly switch to FreeRTOS, or keep alternating releasing devices with both? Wyze has spent a LOT of time developing new features on the Linux based devices that may not already be on the FreeRTOS devices. How hard is it to convert those features over to the other system? Any other thoughts about Linux vs FreeRTOS?” | In general, Linux systems are far easier to develop and maintain with the caveat that you spend some of your system resources just running the Linux image. You get a lot of really nice things with that though, like a root console, systemd (or systemd like things), the ability to run multiple binaries (!!GDB!!), NFS, POSIX, etc. We ultimately have to answer to you, the customer, though when it comes to OS and HW selection. If a device has specific product requirements that necessitate a baremetal OS like FreeRTOS and others, we will select the baremetal OS. So far this decision has been made for all of our cameras that run on battery. For devices that are always-on and wired, they will probably continue to run Linux. Regarding the feature parity of the systems. 10.x for the V3 and other similar cameras was essentially a complete rewrite that made all of this feature code generic. Expect to see more or less feature parity between Linux and FreeRTOS cams going forward (subject to HW capabilities though). - u/WyzeMitchell |
| “FIRMWARE QUESTION (Re:Wyze Robot Vacuum): I don’t know if the Firmware Team on here is mostly related to cameras, or for any firmware, but I’ll ask anyway. In December, Wyze updated the firmware for Robot Vacuums on the production firmware tract to get multi-floor mapping, spot cleaning, DND, and improved algorithm, etc. It would be nice to hear if there is a new ETA for the firmware merger on the Wyze Robot Vacuum for those who were on the Beta Firmware Tract. We have seen comments posted publicly stating Wyze was preparing behind the scenes for the logistical considerations related to the firmware update potentially requiring small percentage of units needing to be returned for a manual update/correction and Wyze wanted to have that ready to go as smoothly as possible. Is this firmware merger still the plan, and is there an ETA? Why does the firmware upgrade only have problems from some that were on Beta? And if that were the case, couldn’t you just do something like have all the beta firmware update to a firmware that rolls them back to a previous version, then do a second step to update to the same firmware as production? I am sure if it was possible, that’s what you would do (I am not experienced with doing firmware myself, so I am just thinking based on what SEEMS logical). I am curious and asking why it isn’t possible to just roll back to an earlier firmware that would then update successfully to the newest one?” | Hi! This is Gwendolyn popping in to answer this one since I’m more involved with this process. I hate to say it, but I don’t have a specific ETA. We got almost to the finish line when we were told that the folks we had trained to help with this were no longer likely to be available. Now we’re having to completely set up a new team, do the negotiations, and train them. What’s going on is that the beta builds that some folks were kind enough to test had an issue that made changing away from the firmware currently on the vacuum have a pretty significant, though not guaranteed, brick risk. We spent a bunch of time working on methods to reduce this risk both to help with potential costs and because we know no one wants to have a high risk update. Vacuums fit an important place in people’s lives and we understand this. The firmware that was released to the public didn’t have this problem so we were able to eventually do the update with the extra features. We can’t have the beta folks do an easy change so we wanted to set up a process that would make this as painless as possible for all of you. What’s going to happen when we’re able to get the process stabilized is that we’ll send out an email explaining everything and the fact that the update will actually bring minimal benefit ,especially since we don’t currently plan further Wyze Robot Vacuum firmware updates. Generally, the beta firmware already has the features and fixes on the public firmware. But we made the promise and we intend to keep it. If folks want to do the update, they’ll be able to opt-in and we’ll be sending out the updates in batches so we can keep everything moving smoothly and quickly. Most vacuums will take the update and be okay. But some of them are expected to have trouble. For the vacuums that fail the update, you’ll be able to ship them to us. Your vacuum will be manually repaired by our team. When it’s functioning normally, we’ll send it back to you. All of this will be free of charge. We know that this has been a massive delay. And it’s admittedly pretty frustrating to have been right there about to send the email when we got the halt notice. But we haven’t given up and we’ll get this handled. We’re sorry and appreciate your patience. -u/WyzeCam |
| “What is the reason 24/7 recording is disabled for the outdoor battery cams when they are plugged in? Mine runs on solar and I’ve never had battery dip below maybe 90%…” | I don’t believe that we will be enabling 24/7 recording for the outdoor battery cam at this time due to technical reasons. Keep your eyes on this space though! This is subject to change. =) - u/WyzeMitchell |
| “To put some of the other Dark mode comments in the form of an AMA relevant question for you, Cofounder Dave Crosby publicly shared in this Facebook thread 3 months ago that Wyze was working on the design process to convert the Camera UI for all the other cameras to DarkMode now, without having to wait for an entire rewrite of the app. He even showed us a prototype of the progress you have made. Firstly, thank you for working on this! We really appreciate it! Related to the above prototype example, Dave said, There’s a number of surprisingly difficult technical challenges to this, but we’ve started the design process. Still scoping it out. Since this is your field of expertise, can you tell us more about what the technical challenges are to this that he was referring to? He said you have started the design process (again, thank you). Since you are probably the most familiar with what this is going to take to get it done, can you tell us any [non-binding] estimates or hopes on what you are personally thinking of for a rough timeline? We acknowledge things could work out better than you expect, or there could be other weird obstacles that arise, but it would be nice to have some kind of estimate based on how much work you’ve already put in to figure out how easy it would be to finish now. He also said “Still scoping it out,” and I am wondering what that implies. Does that mean that you are unsure IF it can reasonably be done in the current app build, or whether he just mean scoping out in the sense of still unsure HOW LONG it is going to take?” | The problem is that adding theme management into the entire app was not expected at the initial stage of app development. And even when the app was starting development, Apple had not released the functionality yet. It’s always more complicated to update something that is already working. We have to think about the current user experience, keep it untouched, avoid regressions, and place something more on top of the current implementation. But we are working on it. We have a solution for the problem and we’re targeting the 2.43 release. Of course, the initial release will not be able to cover all UX. But it’s a start, and we will continue working to bring a comprehensive user experience. - u/WyzeMaximK |
| “This question could be for any or all of the AMA guests (App, Firmware, Engineering): How do you balance the need for new features with the need for stability?” | We are simultaneously strengthening our manual testing capabilities and automation testing abilities, hoping to provide stability and rapid iteration in the fast-paced cycle of testing and development. u/WyzeMJ |
| “What are some of the new features or improvements you are working on or planning to implement for your existing or upcoming Wyze products?” | For the hardware part, new camera products will continue to be launched in the second half of this year. For the software part, we are providing more AI detection capabilities for our subscription services, while upgrades to the security, performance, and experience of the app are also under development. u/WyzeMJ |
| “There have been several community developed projects for the wyze cams like WyzeHacks, wz_mini_hacks, and docker-wyze-bridge, however, it seems as though each project is slowly getting killed off with every new product and firmware release. WyzeHacks no longer seems to work at all, mini hacks only works on older cameras running older firmware, and wyze-bridge doesn’t work with the newer OG cameras. Curious to hear if these firmware changes were intentionally made to block these community projects or not?” | We would be lying if we said we were not aware of these projects, but we aren’t actively targeting these projects with our security fixes. Often, these open source ‘hack’ projects target areas that we already have on the roadmap to adjust to make your products even more secure for more general reasons. We would love it if users were able to deploy software securely to their own cameras, but it’s not an easy problem to solve while maintaining security for the rest of our users. - u/WyzeMitchell |
| “App-wide Landscape support for tablets, in my case an iPad. This means starting with the Home screen that lists devices. I was probably the first to formally request this feature, which was eventually added to the forum Wishlist, back in early 2018. Due to a physical disability, it’s difficult for me to rotate the iPad (or my head) 90° every time I launch the app, just to see the Main Menu normally. A majority of people, me included, operate their iPad in Landscape orientation most of the time. Note that of the 86 apps I currently have installed, only 2 don’t support Landscape. Please take care of this.” | We are definately aware of the problem and it is a high priority feature. Landscape mode for the Home page is targeting for 2.42. Then we are going adopt landscape for the Events screen, and next for devices settings, because those screens are the most complicated. - u/WyzeMaximK |
| “Can you tell us a little about the usefulness of submitting App logs to you? Does EVERY App log get looked at by someone in an organized way (looked at and passed on to the relevant teams), or do they only get looked at if there is a support ticket that refers them to someone or if the community managers pass them on to someone specifically? Can you tell us how the App log process works? I know we need to submit a support ticket with the log number if we want a response about something, but if we don’t care about a response, and just want to let you know about something happening, is it still helpful to submit a log for things even without a support ticket, or do those just get lost or overlooked and buried without getting looked at? I would love to understand how you use logs or how it works. Thanks.” | We receive tens of thousands of logs a month. I believe the backend interface shows something in the order of millions of logs uploaded since Wyze shipped its first device. Due to this… Logs that are uploaded and not attached to a specific ticket, support request, or Community Manager are lost to the black hole that is the log backend. Additionally, depending on the issue you are experiencing, log uploads may not be incredibly useful. Offline issues, for example, rarely have useful logs because the device needs to be online in order to upload them! In that case, microSD card logs are incredibly useful, and we are forever in your debt if you are able to connect with a Community Manager to go through the process. FYI. We (on the Engineering Team) do read this subreddit and the forums on occasion, and will often pursue fixes related to the discussions our community is having. We may not reply to your posts directly, but I would highly recommend always including a log ID in your post anyway along with a general time that you encountered the issue at in case we scoop it up. - u/WyzeMitchell |
| “More questions for Mingjun Yin (u/WyzeMJ) - SVP of Engineering: What are some of the engineering principles or values that guide your work at Wyze and how do they align with the company’s mission of making advanced technology more accessible for everyone along with the original “too good to be true” approach and “being friends with users” value? How do you manage and coordinate the engineering teams that work on different Wyze products such as cameras, doorbells, plugs, lights, scales, vacuums, etc.? What are some of the current or future projects or initiatives that you are most excited about or enjoy the most?” | Thank you for asking. Actually, the Wyze engineering principles have been updated since the beginning 2023. Engineers are expected to be a rational group of people on the team, and we are striving to be both courageous and wise. Pushing the limits with an engineering approach is an expression of our Wyze value of being Too Good To Be True (TGTBT). We are also emphasizing the development work for meeting the real needs of our users. We hope that through our efforts, our friends can feel how much the Wyze Engineering Team cares about their feelings. For the generic part of our engineering, we have a platform department. And for the different products, we have vertical plug-in technical architecture and teams to maintain the plug-ins. I’m excited to be looking for innovative ideas for AI-powered hardware that fits user needs. - u/WyzeMJ |
| “The iOS app is not in a healthy state, and quite frankly almost un useable, what is being done to fix this issue and make iOS users not feel forgotten?” | We are aware of issues and are constantly fixing, improving, and bringing in new features. We are constantly monitoring the app stability and gathering user feedback. We have many products and features, and acknowledge that it takes time to make everything work well together. Please work with Community Managers to send reports and logs to the Engineering Team so we can target the issues affecting your experience. - u/WyzeMaximK |
| “Version number Question (Firmware and potentially App too): There seems to be some similarity and even some duplicates of Firmware version numbers and structure used between products. Is there a specific meaning behind the structure of the firmware number positions? ex: 4.50.3.4240 has 4 number positions. Besides generally being sequential, is there specific meaning or organization behind them?” | Firmware versions change kind of per product, but generally, the last number is going to be our internal build number. Second to last is the major version and then the first two are product-specific. Annoyingly, that last number can’t exceed 4 characters for legacy reasons. So expect it to roll over back to 0000 at some point. - u/WyzeMitchell |
| "For both the App and the Firmware teams. In a letter, the Founders said the following: “in 2023, because this year, much like our first year in business, will be the year of the CAMERA. With your help, we will revisit everything about the Wyze Cam experience. We will focus hard on hearing all your feedback and making the best camera hardware and software in existence. Product launches in 2023 will also be focused on cameras, with few exceptions. Get ready for some big camera announcements in just a few weeks! And we will of course continue to support and update all Wyze products.” With that context in mind, please answer the following: What efforts have you made toward these goals so far this year that most added to that, and what we can expect for the second half of the year? What feedback are you hearing loud and clear from users related to your team’s particular responsibilities or area of influence? | Soon the app teams will be releasing many improvements and fixes to make apps more efficient, We are working on several high demand features (frequently asked by users) which we will try to release this summer and early autumn. The application is the business card of the company, this is the first thing that the user sees and starts using, because of that, the app team was working hard and we still have a lot of work to do to make the app look and operate better, we constantly are researching new features, and improving current functionalities, we are carefully monitoring user feedback to satisfy most of our users. The app is one big system, we can not make something specific working well and forget about rest of the functionalities, e.g. if we do dark mode, we do it across all app, if we do landscape we do for whole app and so on. - u/WyzeMaximK |
| “Wyze Cam V3 is made even more powerful by the wide range of accessories you have built for it. Why do they often forget they are connected to an accessory and require a reboot to fix? What does the reboot do to resolve the problem? Could the resolution the reboot provide be coded into the device itself to prevent frequent reboots?” | The accessory code is a large and fickle beast (50k lines of code) that spans multiple different implementation environments (App, Cloud, FW). We are continuing to apply improvements to this code, but some fixes expose other issues, or some improvements to our generic code break important communications about the accessories to the App. Reboot usually solve the issue as in this large blob of code there are hundreds (thousands?) of potential states that it can get into, that it may not be able to recover from. Know that we are actively pursuing fixes to these issues, and your reports to our community managers (with logIDs!!) is incredibly valuable to getting them fixed! We are also working on more diverse QA test cases to try and catch more of these breakages before they are visible to users - u/WyzeMitchell |
| “In regards to how connectivity is coded into the firmware… why do wyze devices more frequently than not, fail to reconnect after an internet outage? A manual power cycle is required whereas maybe the cameras recover but outdoor plugs don’t. I was curious if this is intentional and if so why? Or actually a bug that needs fixed.” | Connectivity is a difficult topic, and I know that it’s a bit of a meme that “it’s your WiFi” whenever you’re having connectivity issues. But WiFi routers are not only incredibly diverse, but also built to all sorts of different price points.There are WiFi routers that we will be able to connect to without any issue using a certain combination of parameters and timeouts, but will then struggle to connect to another router using those same values. There are also many different types of “offline” that a device can encounter. It’s sometimes really difficult to determine which kind of offline state we are in! Is the device having trouble getting DNS? Is your ISP down? Or maybe the Wyze cloud? Perhaps the environment has a lot of RF noise? All of these conditions require a specific flow in the connection code with timeout parameters that may or may not work on a specific brand of router. It may not be your WiFi but sometimes your WiFi doesn’t help what would otherwise be a minor issue to overcome. We are actively working on improving connectivity though and watch our fleet metrics like a hawk. It is incredibly important to us. - u/WyzeMitchell |
| “In January, a beta firmware was released for the V3 Pro’s that broke their ability to connected to a WPA2/3 mixed network. This firmware was halted for 3-4 months where then another firmware was released where this issue still persisted. My question is, why is this firmware being pushed to all wyze camera’s when you knowingly know it breaks their ability to connect to these mixed networks? Even if they don’t use the WPA3 standard, many devices are starting to. Users have to have a less secure network in order for the cameras to connect.” | This is a fun issue… When the Wyze Cam v2/v3 launched it supported non-secured networks. We broke this at some point and it caused a lot of internal discussion about whether or not we wanted to continue to support this since it’s not a great security practice to use open networks. After lots of discussion, we decided to keep supporting it since it would break users’ camera connectivity (extra bad, because users who have unsecured networks probably aren’t super tech savvy or won’t have access to the network itself to adjust it). But when adding the support back in, there is a technical problem that either breaks open network compatibility or WPA3 networks… I’m rewriting this code so it can support both types right now and have some fixes coming. Stay tuned! - u/WyzeMitchell |
| “Since the Android and iOS apps always have some slight differences because of how they program and compile and have different API options and limitations, other slight differences, how do you decide which differences are okay and which are not? Where do you draw the line? Also, would you consider adding a feature specific to one and not the other if it helped make the overall user experience and usability more identical overall? For example, Navigating “Back” often feels different for Android and iOS users. Android users are able to universally easily swipe left or right from the side of the screen to go “back” in the Wyze app which is a really great experience, and while this is possible to do in iOS as well on lots of apps, we can’t do it on the Wyze app currently because iOS requires each app developer to allow/implement this on their specific app. How difficult would it be to program something like that so that both apps have similar navigation and user experience? Would you consider doing something like that so that the user experience is more similar for everyone? Why or Why not? And related to having to keep up with both iOS and Android, how do you do this? Do you program everything for one of them first and then port the updates to the other? Do you have a program that just compiles the same code one way for Android and then converts it the other way for iOS? How do you make sure both apps are kept up the same?” | It is a trade off between total unification and following UX guidelines of the OS. Navigation is a good example, iOS provides navigation only over UI where Android has hardware buttons. We are unifying most of the user experience but for some features it is not possible to totally unify due to system restrictions and limitations e.g. VoIP and PiP. - u/WyzeMaximK |
| “What are the challenges to providing app or firmware fixes for items which seem to be relatively simple? Like allowing the grouping of different devices in a group. Individuals would like to label things as Living Room, Family Room, etc. Why do some of the things like this take so long?” | We are typically pretty quick to fix these simple things in the Beta environment, but “simple” fixes typically create silly bugs, at least from my experience. (See my previous post about the WPA3 breakage!). There are a lot of legacy reasons for our slow cadence in regard to the production firmware… The 10.x release was essentially a complete rewrite of the code, which meant that the tiny fixes that could have been made to 9.x took forever to get into production. This is changing though!! 10.x is out! The weight has been lifted! Our release cadence will not only increase going forward, but also include smaller changes that should cause less breakage - u/WyzeMitchell |
| “For both App and Firmware AMA Guests: If your budget and team size suddenly increased 10 times from what it is now, and the founders gave leeway to basically do whatever you wanted with just half of the increased resources to see if you can come up with some awesome moonshot or improvements related to your team (Firmware, app, etc), what things would you consider looking into doing or trying that you think would be really interesting if there weren’t other constraints to worry about.” | RTSP =) - u/WyzeMitchell |
| “Maybe for Maxim or Mingjun. I find that the app layout is still very primitive. I find it hard to use, as someone who has tons of Wyze devices. Why is this not being addressed? (since, like, the start?) It seems like a UX and usability expert can solve this issue. Also, just the frontend in general. Do you not have resources for this? Just asking, from one engineer to another. asking this specifically as this is my field. e.g. why can’t i group doorbells and cameras? or why does the app always requires a cache clean on both app level and device level (because it says offline on thumbnail), or why does it not scale well on bigger devices?” | We are working on the problem and are gradually on improving the usability of the app. We have gathered a lot of user feedback and accumulated tons of data. Currently, we are focusing on Home and Event page layout, especially for adopting landscape layout, and we are also working on dark mode because it is the most requested feature. We are redesigning the UI and this can be found initially on new incoming products and later in the year we will adopt the new UI for older products. - u/WyzeMaximK |
| “Do wyze cams use the NTP (Network Time Protocol) for time or simply only device time? I can have a view of 4 cams up and the time can be off on each upwards of 10-15 seconds” | Yes! They do, but… Our hardware has a fairly hilarious amount of clock drift in addition to the presence of latency when viewing playback that can also contribute to the “desync." They will NTP I think every 1-2 hours and resync, but they most likely won’t ever perfectly line up. - u/WyzeMitchell |