Single cam making over 120 queries/minute to AWS kinesis?!?

This just started last night. Drilling into the data, a single v2 cam is doing over 120 DNS queries per minute to a specific AWS kinesis video server in the US West AZ. (Each bar in the graph represents 10 minutes.) Drilling into the data, these are coming from a single cam (the blue part in the client graph) that has been up for months. No other cams are doing this. That 5-hour block last night is especially remarkable since it was pitch dark and the cam registered exactly zero events during that time. Why would the cam be hammering kinesisvideo when there are no events to upload?

Maybe that server wasn’t successfully resolving during that time and that cam was already assigned to use it, so it kept flailing trying to get an address to connect to. These are just queries to your DNS server, not necessarily connections or even attempted connections to that kinesis server. …

2 Likes

what is the Firmware Version of the v2 you are seeing this on? What DNS Servers are you using?

@Customer All the other cams are hitting the same kinesisvideo URL (at very, very low rates). Moreover, I also did a recursive DNS lookup to get the authoritative answer. All same. So I think we can eliminate stale caching on my end. But if I catch another big block like that, I will flush and see if the hammering stops.

@spamoni All of my v2 cams (including this one) are running 4.9.8.501. Also, this is not the same v2 cam that I had trouble updating/reconnecting a few days ago. I’m using OpenDNS. My PiHole is caching with a modest TTL, so I periodically see the cam’s DNS request filled by OpenDNS. Even if the cam tried to ignore the DHCP-provided DNS and go to a different DNS server, it couldn’t (without extra effort) because I’m DNAT’ing outbound port 53 to OpenDNS. This is all SOP on my network; it’s been this way for years.

1 Like

Impressive setup. I used OpenDNS for years as well. I am on a new router and have not enabled it yet. Not sure I will as the new router has a lot of features for safe guarding the network.

I was checking to make sure you were on tyhe latest FW as there were some issues in the past. Since you are, I cannot say exactly what is causing it, especially since it seems to be on one camera.

Any slight movement will cause the camera to react and send to the cloud for processing. Is this particular camera overlooking a high motion area or around trees and shrubs? could be something related to that,

You could test some of this by turning off Motion Detection and Notification for a short period of time and see if it calms down

1 Like

I forgot to mention. I turned off motion and sound detection around 5am – about an hour after the first interval of hammering – and the second interval of hammering started shortly afterward.

That camera is watching a ‘dead space’ – no windows, no appliances or equipment with lights, or anything else I can think of. During the second interval of hammering, I pulled up the live feed and put a NIR-pass filer (blocks visible spectrum) in front of the camera to check for unexpected IR sources. None. Then I put a book in front of the camera. The hammering continued. Until it stopped for no apparent reason about an hour later. I have since removed the book and the NIR-pass filter. No hammering since.

Just weird… I’ll share if happens again or I learn more.

That is weird. Try rolling back to the previous FW and see if it still exists, if you want. I am actually more curious than anything else

I may try that if the issue reoccurs. So far, it hasn’t.

Update: 2021-11-18 - the issue reoccurred today. Same cam. 120 req/min over a 50 minute period from 0820 to 0910 local time. Event detection was on and logged only two events during that timeframe. No other hammering has been seen in the previous 24 hours, despite many captured events.

So far I’ve been too lazy to monitor any traffic other than DNS queries but I might if this continues. It’s just odd that the cam would be consistently looking up kinesis twice per second even if it did get triggered. Right now, I feel that the hammering is not predictable enough to warrant rolling back the firmware. My assessment may change if this goes on. :slight_smile:

Update: 2021-11-19 - no DNS hammering in the last 24 hours. I guess the fun ends.

1 Like