What App Privacy Testing Reveals About Your Data

Olaf Pursche: When you analyze an app from a privacy perspective, what do you look for, and what can technical testing reveal that a privacy policy cannot?

Eric Clausing: AV-TEST looks at an app from several perspectives. Alongside a formal review of the privacy policy, in which we gather and assess all information provided to the user regarding data collection, processing, and sharing, we conduct both static and dynamic technical analyses. These examine, among other things, integrated tracking components, permissions, and the actual data flows generated by the application.

During the dynamic analysis, the app is operated under realistic usage conditions while its communications are captured and analyzed. This allows us to determine what data is actually collected and transmitted, when this happens, and which recipients are involved. By combining the results of the formal review with those of the static and dynamic analyses, we can identify inconsistencies, omissions, or discrepancies.

That is the crucial difference: a privacy policy describes what an app says it does; technical testing shows what it actually does. Only by comparing these two perspectives can we obtain a reliable picture of an app’s real-world behavior and its implications for user privacy.

Where is the line between data collection that is necessary for an app to function and data collection that creates unnecessary privacy risks?

Clausing: The line is best determined by looking at the specific function of an app. A navigation app needs location data, for example. A cycle-tracking app needs certain health information if it is going to generate meaningful predictions. But that does not mean it also needs access to an advertising identifier or other unrelated information.

Our analysis therefore always asks three basic questions: What data is collected? What data is actually transmitted? And is that data necessary and proportionate to the app’s functionality? The relevant question is not simply whether a piece of data has a legitimate purpose, but whether collecting and transmitting it is genuinely necessary and whether the same functionality could be achieved in a less intrusive way.

In our analysis of 17 women’s health apps, for example, 15 accessed the Android Advertising ID or incorporated advertising-related technologies. This is clearly not necessary for the core functionality of a cycle-tracking app. In many cases, these technologies are associated with advertising or other forms of monetization. This is particularly noteworthy where users are already paying for access to the app’s full functionality.

Monetization itself is, of course, not inherently a problem – developers need to generate revenue. However, monetization practices can also have implications for user privacy. The important question is therefore not whether an app collects data at all, but whether the data it collects and shares is genuinely necessary and proportionate to what the app is supposed to do.

Your W Social analysis found little tracking activity but also data transmission before users could accept the privacy policy. What does this reveal about the gap between stated privacy practices and actual app behavior?

Clausing: The example shows very clearly why technical analysis is necessary. An app may appear formally correct at first glance; however, a more detailed analysis and assessment of its actual behavior may uncover practical shortcomings. This is precisely the kind of behavior that technical testing can make visible.

It is important, however, to distinguish between technical findings and legal assessment. AV-TEST can objectively establish what happens technically. Whether a particular process constitutes a legal violation is a matter for legal assessment. The technical evidence provides the basis for making that assessment.

How much control do app providers have over data flows created by third-party services, and where are the biggest risks?

Clausing: An app provider has quite a significant degree of control over third-party data flows: they decide which services to integrate, what data is passed to them, and how those services are configured. However, once data leaves the app, the provider also becomes dependent on the functionality and data practices of the external service.

The biggest risk is therefore not the use of third-party services itself, but a lack of transparency and control over what is actually being transmitted. SDKs may collect or transmit considerably more information than is necessary for the function they provide, and developers may not always have a complete picture of the resulting data flows.

This is why privacy testing should not rely solely on privacy policies or SDK documentation. The app's actual network traffic and behavior need to be examined. In practice, there can be a significant gap between what a third-party component is supposed to do and what it actually transmits.

Ultimately, choosing a third-party component is also, in a sense, choosing its data flows. The best approach is therefore to minimize external services, limit the data they receive, configure them restrictively, and regularly verify their actual behavior.

Does the privacy risk change when apps process particularly sensitive information such as health, sexual, or location data?

Clausing: Yes. The level of privacy risk changes significantly when an app processes particularly sensitive information. The more intimate the information, the greater the potential consequences if it is unnecessarily collected, shared, misused, or compromised.

Health and dating apps are especially interesting in this regard because seemingly separate pieces of information can become highly revealing when combined. Menstrual cycles, fertility information, sexual health, precise location, or even patterns of app usage can reveal aspects of a person's private life that they may never have intended to disclose.

At the same time, the sensitivity of the data makes the context and purpose of its collection particularly important. Collecting information that is genuinely necessary for the core functionality is fundamentally different from transmitting the same information to third parties for analytics, advertising, or other secondary purposes.

Does keeping app data in Europe automatically make an app more privacy-friendly, or is digital sovereignty more complex than data residency?

Clausing: Where data is processed and stored is certainly important, particularly when sensitive personal data is involved. Keeping data in Europe can provide important legal and regulatory safeguards, but European data residency alone does not automatically make an app privacy-friendly.

Digital sovereignty is much broader than the physical location of a server. You also need to consider who operates the infrastructure, which companies have access to the data, where those companies are based, which third-party services are involved, and under which legal jurisdictions they operate. A European server operated by a non-European provider can therefore raise very different questions from a genuinely European-controlled infrastructure.

Overall, I would see data residency as one important factor rather than a guarantee of privacy. Where the data is stored matters, but who can access it, under which conditions, and what happens to it afterward matters just as much.

Can an app comply with data protection rules and still collect or share more data than good privacy engineering would require?

Clausing: Yes. Compliance and privacy engineering address different questions. The legal question is: Is a particular form of processing permissible? The technical question goes further: Is it actually necessary for the functionality of the product?

That distinction is important. Data collection can be legally permissible under certain circumstances and still be technically unnecessary.

Advertising and tracking technologies are a good example. If an app can provide its core functionality without an advertising identifier or certain trackers, we should ask why that data is being collected in the first place (with “monetization” being the answer here in many cases).

Good privacy engineering therefore needs to go beyond mere compliance.

What does Privacy by Design mean for apps handling sensitive data, and which decisions should developers make from the outset?

Clausing: Privacy by Design means treating privacy as a fundamental design requirement from the very beginning, rather than something that is added after the app has been built. This is particularly important for health apps, where the data involved can be extremely intimate.

Developers should therefore ask at the outset what data is actually necessary for the app's core functionality, whether it needs to leave the device at all, and which third-party services are genuinely required. Every SDK and external service should be evaluated not only for its functionality, but also for the data it collects and the recipients it introduces.

For highly sensitive applications, advertising and unnecessary tracking should not simply be the default. Developers should deliberately minimize data collection, avoid unnecessary third parties, and build strong privacy protections into the architecture from the start. The most privacy-friendly data flow is often the one that never happens in the first place.

What do AI-powered features change from a privacy perspective, particularly around data collection, profiling, and further processing?

Clausing: AI significantly expands the possibilities for further processing and deriving new information. We therefore need to ask not only what data is collected, but also what can be inferred from it, where processing takes place, and which external AI services are involved.

At the same time, AI is changing software development itself. Vibe coding is a good example: People who may never have written a line of code themselves can now develop and publish apps. This opens up major opportunities, but it also creates risks for users and significant liability risks for publishers. Ultimately, the provider remains responsible for the app they publishe, regardless of whether the code was written by a person or generated by AI.

This makes independent technical testing even more important. AV-TEST also supports startups with app testing, ideally during development, so that security and privacy issues can be identified and addressed before an application is released.

What are your three priorities for developers building more privacy-friendly applications?

Clausing: First: Minimize data. Collect only the data that is genuinely required for the specific functionality. Data that is never collected cannot be unnecessarily transmitted or misused.

Second: Control third parties and data flows. Developers need to know which SDKs they use, what data they process, and where that data is transmitted.

Third: Test the application’s actual behavior regularly. A privacy policy is not a technical test. Apps, SDKs, and backend services change, so their technical behavior needs to be reviewed regularly as well.

Privacy should increasingly be understood as a technical quality characteristic of a digital product. For Europe in particular, this is an important part of digital sovereignty: We need to be able to understand and control what our digital products do, where data flows, and which technical dependencies they rely on.

Eric Clausing has been part of the AV-TEST team since 2015 and is responsible for the conception, implementation and further development of all security and privacy tests and certifications in the area of the Internet of Things.

Olaf Pursche is an independent IT security consultant, trusted advisor, journalist and keynote speaker. Since 2025, he has headed the Security Competence Group of the eco Association, prior to which he advised eco on IoT security. For 15 years, he was responsible for IT security for all European editions of COMPUTER BILD and wrote for BILD, Welt, iX, kes and IT-Sicherheit, PC Professionell and other magazines and newspapers. He was a member of advisory boards at the BSI, the GDV and other expert committees. He was also responsible for communications, press relations and marketing for 10 years as CCO of the AV-TEST Institute and as Head of Marketing & Communications at the SITS Group.