Smart TVs are spying on you. How much do manufacturers really know?

As the proliferation of smart home devices continues, users are growing increasingly worried about manufacturers who are collecting data and effectively spying on their in-home behaviors. It’s been a big topic of concern for many years with smartphones that are always in our pockets and purses, but the risk is no longer limited to just that.

According to a new study (PDF) that was jointly published by students from the US, the UK, and Spain, manufacturers of smart TVs actually know a lot more about their users than you’d expect, all thanks to something called Automatic Content Recognition (ACR).

What data is collected by smart TVs?

According to the study’s results, smart TVs collect a lot of data during use, including data as far as linear TV shows watched, streamed movies and TV shows watched, games played, how long certain content is watched, among other bits of data.

All of this data is used to create a unique digital fingerprint, which is transmitted back to the manufacturer’s servers for storage and analysis. For individual users like you and me, this means manufacturers can draw precise conclusions about viewing behaviors on a per-smart-TV basis.

Related: Why you should be using a VPN with your smart TV

The only requirements for this are that the smart TV remains connected to the internet (which is essential for smart TV operation) and that the user has agreed to the terms and conditions (which usually happens when the device is first set up).

Perhaps most unnerving is that even if a smart TV is only used as a connected display via HDMI and doesn’t run any internal apps or software, it can still collect and understand what you’re watching: “ACR network traffic exists when watching linear TV and when using smart TV as an external display using HDMI.”

Related: How to use your TV as a monitor

Notably, there was one special case when ACR transmissions weren’t being measured: when using certain third-party apps, including Netflix and YouTube. (This presumably has to do with special agreements or technical restrictions. For example, Netflix prevents screenshots from being taken in its app in order to protect its content.)

How your viewing data is collected

According to the study, smart TVs aren’t just analyzing network traffic to determine what you’re watching — they’re taking screenshots of the display at regular intervals. (This is reminiscent of Microsoft’s Windows Recall feature, which was sharply criticized over privacy concerns.)

With LG’s smart TVs, screenshots occur approximately every 10 milliseconds (100 per second); with Samsung’s smart TVs, screenshots are taken approximately every 500 milliseconds (2 per second). Then, after 15 seconds, all collected screenshots are sent to the manufacturer via ACR as another data point on your viewing behavior.

Although the study only looks at devices from LG and Samsung, previous reports suggest that virtually all smart TV manufacturers are doing the same thing. It’s yet another step towards less privacy at home.

Further reading: Worried about spying TVs? Try a home theater PC