AI and Privacy: What Companies Know About You — Surveillance, Data and the AI Act

Data: the New Oil of the Digital Economy

The comparison between data and oil is well-known but imperfect in one crucial way: oil gets used once and runs out. Data, the more it's used, the more valuable it becomes — and it never depletes. Every digital interaction you perform generates information that can be stored, analyzed, and monetized indefinitely. The scale of this is staggering: by the mid-2020s, the world was producing over 2.5 quintillion bytes of data per day.

To understand why this matters, you need to understand what companies actually collect. We're not talking about just name and email. The spectrum of captured data is far broader — and for most people, completely invisible:

What's Actually Being Collected

Most people accept terms of service without reading them. What's written in those terms typically includes permission to collect:

• Identity data: name, email, date of birth, gender, occupation, government ID numbers when provided
• Location data: real-time GPS, history of places visited, commute patterns, home and work addresses inferred from routine
• Inferred financial data: estimated income based on spending behavior, purchase history, product categories researched
• Digital behavior data: searches performed, websites visited, time on each page, scroll behavior, where your cursor pauses
• Media consumption data: videos watched, music played, series followed, pause points, content skipped
• Social data: contacts, communication frequency, who you call most, groups you belong to
• Calendar and schedule data: appointments, recurring meetings, events
• Biometric data: heart rate (smartwatches), sleep quality, physical activity levels, breathing patterns, body temperature
• Voice data: commands to virtual assistants, speech patterns, vocabulary used
• Visual data: photos, videos, facial recognition in uploaded images
• Device data: phone model, operating system, battery level, network used, other apps installed

How AI Learns About You

Collecting data is just the first step. What transforms a list of raw information into real power is the artificial intelligence applied on top of it. Modern machine learning algorithms can identify patterns that no human analyst could spot manually — and do it at a scale and speed that makes the process virtually imperceptible to the user.

The Digital Profile You Don't Know You Have

AI doesn't just record what you do — it predicts what you're going to do. That's the qualitative leap that makes the technology so valuable (and potentially invasive). Some concrete examples of how this inference works in practice:

• Purchase prediction: a US retailer became infamous for sending baby product coupons to a teenager — before her family knew she was pregnant. The algorithm detected shifts in her buying patterns (prenatal vitamins, unscented lotions, larger clothing sizes) and inferred the pregnancy weeks in advance.
• Mood prediction: streaming platforms adjust recommendations based on time of day, consumption pace, and history — knowing that on Friday at 11pm you probably want comedy, and Monday morning you'll prefer a short documentary.
• Churn prediction: companies know you're about to cancel a subscription before you've made the decision — and act preemptively with personalized offers or content.
• Emotional state prediction: research shows that typing rhythm, types of errors made, and word choice in messages can indicate whether a person is anxious, depressed, or stressed — with surprisingly high accuracy.

What Big Tech Really Knows About You

Let's move from abstract to concrete. Each digital service you use has access to a specific type of information — and the cross-referencing between them is what creates the most complete profile. Understanding where this data comes from is the first step toward making more conscious choices.

The critical point — one most people miss — is that this data rarely stays with the company that collected it. The data economy involves a vast network of data brokers who buy, sell, and combine information from multiple sources to build even more complete profiles. Companies like Acxiom, Experian, and Oracle Data Cloud maintain files with hundreds or thousands of attributes on billions of people — the majority of whom have never heard of these companies.

Digital Surveillance: Commercial, Governmental, and the Blurry Line Between Them

The word "surveillance" evokes cameras and authoritarian governments — but the most pervasive form of monitoring in modern life is actually conducted by private companies, with the (formal) consent of users themselves, for primarily commercial purposes. Understanding both types is essential for a realistic picture of the landscape.

Commercial Surveillance

It's the business model that sustains the free internet. When you don't pay for the product, you frequently are the product — or more precisely, access to your behavior is. Companies use AI to:

• Real-time personalized advertising: the auction system that decides which ad you see takes under 100 milliseconds — during that time, data about your profile is bought and sold among dozens of companies
• Consumption prediction: anticipating your needs to serve offers at the exact moment you're most likely to buy
• Dynamic pricing: the same product can have different prices depending on who's buying, based on estimated income and payment history data
• Content recommendation: keeping you on platforms longer, maximizing available ad inventory

Government Surveillance

Government use of AI to monitor citizens spans a wide spectrum — from broadly accepted applications (traffic cameras) to deeply controversial practices (mass communications surveillance). The main forms include:

• Urban camera monitoring: cities worldwide have installed camera networks connected to real-time facial recognition systems. London has over 900,000 cameras; Beijing leads globally with estimates reaching hundreds of millions
• Social media analysis: monitoring mentions, groups, and online behaviors to identify suspicious activities
• Communications interception: the Snowden revelations in 2013 showed intelligence agencies were collecting metadata from communications at a global scale
• Social scoring: the most extreme model, implemented in parts of China, uses behavioral data to assign scores that affect access to services, transportation, and opportunities

Facial Recognition: How It Works and Where It's Being Used

Facial recognition is probably the most discussed surveillance technology of recent years — and for good reason. It's one of the most powerful biometric identification methods ever created, operating in real time and at a distance, without requiring any cooperation from the person being identified.

The Technology Behind Recognition

1. Face detection: the system identifies that a face is present in the image, separating it from the background
2. Alignment: the image is normalized to a standard position, compensating for angle and lighting
3. Feature extraction: the AI maps distances between facial landmarks — distance between the eyes, nose width, jawline shape, lip curvature, orbital socket depth
4. Vector generation: these features are converted into a set of numbers (an embedding) representing each person's unique "facial signature"
5. Comparison: the vector is compared against a reference database for identification (1:N) or verification (1:1)

Where Facial Recognition Is Being Used

A documented and serious problem: facial recognition systems have significantly higher error rates for Black people, women, and older individuals compared to young white men — a reflection of biases in the data used to train them. This isn't theoretical: in the United States, at least three Black men were wrongfully arrested based on incorrect facial recognition identifications between 2020 and 2023.

Real Benefits of AI for Digital Security

It's important not to frame all technological surveillance as negative. There are genuinely beneficial applications of AI in security — and ignoring them would be intellectually dishonest.

• Banking fraud detection: algorithms analyze transaction patterns in milliseconds and block suspicious operations before damage occurs. Modern systems achieve fraud detection accuracy above 99% in controlled scenarios
• Identity theft prevention: AI monitors the dark web for leaked credentials and alerts users before information is exploited
• Cyberattack prevention: machine learning-based intrusion detection systems identify attack patterns that would go unnoticed by human analysts
• Secure biometric authentication: facial and fingerprint recognition are significantly more secure than traditional passwords for most everyday use cases
• Online child abuse detection: AI tools are used by platforms to identify and remove abuse content — a task impossible to do manually at internet scale

The Real Risks: Beyond Fear of Big Brother

The risks of mass data collection by AI aren't just philosophical or futuristic — they affect real lives today, in concrete and documented ways.

Data Breaches: When Security Fails

Concentrating data in large platforms creates extraordinarily valuable targets for attacks:

• The Equifax breach in 2017 exposed financial data of 147 million Americans, including Social Security numbers, birth dates, and addresses
• The T-Mobile breach of 2021 exposed data of over 50 million customers, including phone numbers, addresses, and driver's license numbers
• A healthcare data breach can include diagnoses, medications in use, and hospitalization history — information that can be weaponized against someone in insurance, employment, and personal relationships

Algorithmic Discrimination

Algorithms aren't neutral. They reflect the data they were trained on — and that data carries decades of human bias. The result is systems that reproduce and amplify existing inequalities:

• Credit: algorithms that deny loans more frequently to residents of certain neighborhoods — a digital proxy for racial discrimination, even without explicitly using race
• Hiring: resume screening systems that penalize words associated with minority groups, including certain names or historically Black universities
• Insurance: pricing that uses digital behavior data to charge more from groups who are systematically disadvantaged
• Content: algorithms that amplify misinformation and extremist content because it generates more engagement — and engagement is what's being optimized

Deepfakes and the Authenticity Crisis

Generative AI has made it possible to create videos, audio clips, and images with quality sufficient to fool humans and, often, other detection systems:

• Financial fraud: scams using synthetic voice or video of executives to authorize transfers (a Hong Kong bank transferred $25 million after a deepfake video call in 2024)
• Political disinformation: fake videos of politicians saying things they never said
• Extortion: use of intimate deepfakes as a blackmail instrument
• Evidence manipulation: the ability to dismiss any genuine recording as a possible deepfake

Children's Privacy: The Most Vulnerable Group

Children are among the most vulnerable targets of data collection — and simultaneously among the least protected in practice:

• Connected smart toys that record conversations and transmit them to servers — often without adequate encryption
• Educational platforms that collect detailed behavioral data on children in school settings
• Recommendation algorithms optimized to maximize screen time for children, ignoring developmental impacts
• Targeted advertising using psychological manipulation techniques on users who don't yet have the cognitive capacity to recognize them

The AI Act in Europe: The World's First Comprehensive AI Law

While the GDPR (General Data Protection Regulation) addressed data specifically, the EU went further with the AI Act — which entered into force in 2024 and represents the world's first comprehensive legislation on AI as a technology. It regulates not just data, but the development and use of AI systems themselves, through a risk-based classification framework.

Risk-Based Classification

The AI Act has global implications: any company that wants to operate in the European market must comply, regardless of where it's headquartered. This creates a regulatory effect similar to the GDPR — American and Asian companies need to adapt their products for the European market, and those privacy and transparency improvements often end up benefiting users in other countries too.

GDPR and Global Data Protection Standards

The EU's GDPR, in force since 2018, remains the global gold standard for data protection. It established principles — purpose limitation, data minimization, consent, rights to access and erasure — that have since been replicated or referenced by data protection laws in over 130 countries. If you're in the US, federal data protection law is still fragmented, but California's CCPA and CPRA provide robust state-level protections, and several other states have followed. The global direction is clear: stronger protections, not weaker ones.

Algorithmic Transparency: the Black Box Deciding Your Life

Why did that piece of content appear in your feed today? Why was a credit application approved or rejected? Why did a resume pass or fail the first screening round? In many cases, the answer involves AI — and that AI frequently operates as a black box: a system that produces results without explaining how it arrived at them.

The concept of explainability (or XAI — Explainable AI) is becoming central to AI regulation debates precisely because automated decisions have real, sometimes irreversible consequences for real people. The denied loan can kill a small business. The resume filtered out by an algorithm can cost a major career opportunity. The content amplified by an algorithm can shape political views at scale.

Speculation: The Future That May or May Not Arrive

Some scenarios circulating in privacy and surveillance debates sound like science fiction — but deserve serious analysis, separating what's genuinely in development from what remains speculation.

What's Actually in Development

• AI embedded in smart glasses: devices like the Meta Ray-Bans combine camera, microphone, and AI — and US students demonstrated in 2024 that it's possible to identify faces in real time and pull up personal information about them in seconds using only those glasses
• Smart cities with AI: Singapore, Dubai, and some Chinese cities already have integrated networks of cameras, sensors, and AI managing traffic, security, and public services
• Autonomous vehicles as data collectors: a self-driving car collects multiple terabytes of data per hour of use — including detailed mapping of everything around it

What's Still Speculation (but Technically Plausible)

• Continuous emotional profiling: systems that monitor facial expression, tone of voice, and body language in real time to infer emotional state — in stores, workplaces, or public spaces. Technically feasible today; ethically prohibited in Europe under the AI Act
• Total digital memory: devices that continuously record everything you see and hear, creating a complete archive of your lived experience. Companies have tested primitive versions; full scale remains impractical
• Reading intentions before action: brain-computer interfaces that detect intentions before they're executed. Neuralink and others are in experimental stages — surveillance application is technically conceivable but extremely distant

How to Protect Your Privacy: A Practical Guide

Total privacy protection in the digital age is practically impossible without giving up fundamental conveniences. But there's a spectrum between "no protection" and "off the grid" — and most people can significantly improve their position with relatively simple actions.

Basic Steps (Do These Now)

• Review app permissions: open your phone settings and check which apps have access to camera, microphone, location, and contacts. Revoke anything that isn't necessary
• Use two-factor authentication: on all important accounts. Prefer authenticator apps (like Google Authenticator or Authy) over SMS, which can be intercepted
• Manage location sharing: switch to "only while using the app" instead of "always" for every app that doesn't need continuous location access
• Use strong, unique passwords: a password manager (Bitwarden, 1Password, Dashlane) solves the problem of remembering dozens of complex passwords
• Keep devices updated: security updates fix known vulnerabilities that are actively exploited
• Review ad settings: in iPhone and Android privacy settings, you can limit ad tracking

Intermediate Steps

• Use a privacy-focused browser: Firefox with uBlock Origin or Brave Browser block trackers by default
• Switch to a non-tracking search engine: DuckDuckGo and Startpage don't build user profiles
• Use a VPN on public networks: especially on hotel, airport, and coffee shop Wi-Fi
• Be mindful near virtual assistants: if you don't use them regularly, disable them or switch to manual activation mode
• Exercise your data rights: under GDPR (EU), CCPA (California), and many other laws, you can request deletion of your data — typically via email to the company's Data Protection Officer

The Future of Privacy: What's Coming

The tension between data collection and privacy won't resolve on its own — it will intensify. But there are technical and regulatory developments that could change the equation in the coming years.

• On-device AI: processing that happens directly on the device, without sending data to the cloud. The iPhone with Apple Intelligence and Pixel phones with Gemini Nano are early examples. Reduces exposure significantly
• Federated learning: a technique that trains AI models without centralizing data — data stays on devices and only learning gradients are shared. Used by Google to improve predictive typing without seeing what you type
• Confidential computing: data processed in hardware enclaves that even the server operator can't access
• Sovereign digital identity: models where you control your digital credentials and choose what to share with each service, without relying on a centralized intermediary
• Growing regulation: the European example with GDPR and AI Act is being followed by other regions — state laws in the US expanding, federal debates intensifying, and global frameworks emerging through bodies like the OECD and UN