2024

Exploring ChatGPT App Ecosystem: Distribution, Deployment and Security
Exploring ChatGPT App Ecosystem: Distribution, Deployment and Security

Chuan Yan, Mark Huasong Meng, Liuhuo Wan, Tian Yang Ooi, Ruomai Ren, Guangdong Bai

ASE'24: The 39th IEEE/ACM International Conference on Automated Software Engineering 2024

As the flagship large language model (LLM) product of OpenAI, ChatGPT has gained global attention for its remarkable ability to handle complex natural language understanding and generation tasks. Inspired by the success of the mobile app ecosystems, OpenAI enables third-party developers to create ChatGPT plugins to fur- ther expand ChatGPT’s capabilities. These plugins are distributed through the OpenAI’s plugin store and are easily accessible to users. With ChatGPT as the powerful backbone, this app ecosystem has illustrated great business potential by offering users personalized services in a conversational manner. Nonetheless, this ecosystem is still in its nascent stage and undergoing dynamic evolution. Many crucial aspects regarding app development, deployment, and secu- rity have yet to be thoroughly studied in the research community, potentially hindering a wider adoption by both developers and users. In this work, we conduct the first comprehensive study of the ChatGPT app ecosystem, aiming to unveil its landscape to our research community. Our study focuses on the distribution and deployment models in the integration of LLMs and third-party apps, and assesses their security and privacy implications. We in- vestigate the runtime execution mechanism of ChatGPT apps and accordingly propose a three-layer security assessment model from the perspectives of users, app developers, and store operators. Our evaluation of all 1,038 plugins available in the store reveals their uneven distribution of functionality, underscoring prevelent and emerging topics. Our security assessment also reveals a concerning status quo of security and privacy in the ChatGPT app ecosystem. We find that the authentication and user data protection for third- party app APIs integrated within LLMs contain severe flaws. For example, 173 plugins have broken access control vulnerabilities, 368 plugins are subject to leaking manifest files, and 271 plugins provide inaccessible legal document links. Our study for the first time highlights the immaturity of the ChatGPT app ecosystem. Our findings should especially raise an alert to OpenAI and third-party developers to collaboratively maintain the security and privacy compliance of this emerging ecosystem.

Exploring ChatGPT App Ecosystem: Distribution, Deployment and Security
Exploring ChatGPT App Ecosystem: Distribution, Deployment and Security

Chuan Yan, Mark Huasong Meng, Liuhuo Wan, Tian Yang Ooi, Ruomai Ren, Guangdong Bai

ASE'24: The 39th IEEE/ACM International Conference on Automated Software Engineering 2024

As the flagship large language model (LLM) product of OpenAI, ChatGPT has gained global attention for its remarkable ability to handle complex natural language understanding and generation tasks. Inspired by the success of the mobile app ecosystems, OpenAI enables third-party developers to create ChatGPT plugins to fur- ther expand ChatGPT’s capabilities. These plugins are distributed through the OpenAI’s plugin store and are easily accessible to users. With ChatGPT as the powerful backbone, this app ecosystem has illustrated great business potential by offering users personalized services in a conversational manner. Nonetheless, this ecosystem is still in its nascent stage and undergoing dynamic evolution. Many crucial aspects regarding app development, deployment, and secu- rity have yet to be thoroughly studied in the research community, potentially hindering a wider adoption by both developers and users. In this work, we conduct the first comprehensive study of the ChatGPT app ecosystem, aiming to unveil its landscape to our research community. Our study focuses on the distribution and deployment models in the integration of LLMs and third-party apps, and assesses their security and privacy implications. We in- vestigate the runtime execution mechanism of ChatGPT apps and accordingly propose a three-layer security assessment model from the perspectives of users, app developers, and store operators. Our evaluation of all 1,038 plugins available in the store reveals their uneven distribution of functionality, underscoring prevelent and emerging topics. Our security assessment also reveals a concerning status quo of security and privacy in the ChatGPT app ecosystem. We find that the authentication and user data protection for third- party app APIs integrated within LLMs contain severe flaws. For example, 173 plugins have broken access control vulnerabilities, 368 plugins are subject to leaking manifest files, and 271 plugins provide inaccessible legal document links. Our study for the first time highlights the immaturity of the ChatGPT app ecosystem. Our findings should especially raise an alert to OpenAI and third-party developers to collaboratively maintain the security and privacy compliance of this emerging ecosystem.

Investigating Documented Privacy Changes in Android OS
Investigating Documented Privacy Changes in Android OS

Chuan Yan, Mark Huasong Meng, Fuman Xie, Guangdong Bai

FSE'24: Proceedings of the ACM on Software Engineering, Volume 1, Issue FSE 2024

Android has empowered third-party apps to access data and services on mobile devices since its genesis.This involves a wide spectrum of user privacy-sensitive data, such as the device ID and location. In recent years, Android has taken proactive measures to adapt its access control policies for such data, in response to the increasingly strict privacy protection regulations around the world. When each new Android version is released, its privacy changes induced by the version evolution are transparently disclosed, and we refer to them as documented privacy changes (DPCs). Implementing DPCs in Android OS is a non-trivial task, due to not only the dispersed nature of those access control points within the OS, but also the challenges posed by backward compatibility. As a result, whether the actual access control enforcement in the OS implementations aligns with the disclosed DPCs becomes a critical concern. In this work, we conduct the first systematic study on the consistency between the operational behaviors of the OS at runtime and the officially disclosed DPCs. We propose DopCheck, an automatic DPC-driven testing framework equipped with a large language model (LLM) pipeline. It features a serial of analysis to extract the ontology from the privacy change documents written in natural language, and then harnesses the few-shot capability of LLMs to construct test cases for the detection of DPC-compliance issues in OS implementations. We apply DopCheck with the latest versions (10 to 13) of Android Open Source Project (AOSP). Our evaluation involving 79 privacy-sensitive APIs demonstrates that DopCheck can effectively recognize DPCs from Android documentation and generate rigorous test cases. Our study reveals that the status quo of the DPC-compliance issues is concerning, evidenced by 19 bugs identified by DopCheck. Notably, 12 of them are discovered in Android 13 and 6 in Android 10 for the first time, posing more than 35% Android users to the risk of privacy leakage. Our findings should raise an alert to Android users and app developers on the DPC compliance issues when using or developing an app, and would also underscore the necessity for Google to comprehensively validate the actual implementation against its privacy documentation prior to the OS release.

[Paper]

Investigating Documented Privacy Changes in Android OS
Investigating Documented Privacy Changes in Android OS

Chuan Yan, Mark Huasong Meng, Fuman Xie, Guangdong Bai

FSE'24: Proceedings of the ACM on Software Engineering, Volume 1, Issue FSE 2024

Android has empowered third-party apps to access data and services on mobile devices since its genesis.This involves a wide spectrum of user privacy-sensitive data, such as the device ID and location. In recent years, Android has taken proactive measures to adapt its access control policies for such data, in response to the increasingly strict privacy protection regulations around the world. When each new Android version is released, its privacy changes induced by the version evolution are transparently disclosed, and we refer to them as documented privacy changes (DPCs). Implementing DPCs in Android OS is a non-trivial task, due to not only the dispersed nature of those access control points within the OS, but also the challenges posed by backward compatibility. As a result, whether the actual access control enforcement in the OS implementations aligns with the disclosed DPCs becomes a critical concern. In this work, we conduct the first systematic study on the consistency between the operational behaviors of the OS at runtime and the officially disclosed DPCs. We propose DopCheck, an automatic DPC-driven testing framework equipped with a large language model (LLM) pipeline. It features a serial of analysis to extract the ontology from the privacy change documents written in natural language, and then harnesses the few-shot capability of LLMs to construct test cases for the detection of DPC-compliance issues in OS implementations. We apply DopCheck with the latest versions (10 to 13) of Android Open Source Project (AOSP). Our evaluation involving 79 privacy-sensitive APIs demonstrates that DopCheck can effectively recognize DPCs from Android documentation and generate rigorous test cases. Our study reveals that the status quo of the DPC-compliance issues is concerning, evidenced by 19 bugs identified by DopCheck. Notably, 12 of them are discovered in Android 13 and 6 in Android 10 for the first time, posing more than 35% Android users to the risk of privacy leakage. Our findings should raise an alert to Android users and app developers on the DPC compliance issues when using or developing an app, and would also underscore the necessity for Google to comprehensively validate the actual implementation against its privacy documentation prior to the OS release.

[Paper]

Are Your Requests Your True Needs? Checking Excessive Data Collection in VPA App
Are Your Requests Your True Needs? Checking Excessive Data Collection in VPA App

Fuman Xie, Chuan Yan, Mark Huasong Meng, Shaoming Teng, Yanjun Zhang, Guangdong Bai

ICSE '24: Proceedings of the IEEE/ACM 46th International Conference on Software Engineering 2024

Virtual personal assistants (VPA) services encompass a large number of third-party applications (or apps) to enrich their functionalities. These apps have been well examined to scrutinize their data collection behaviors against their declared privacy policies. Nonetheless, it is often overlooked that most users tend to ignore privacy policies at the installation time. Dishonest developers thus can exploit this situation by embedding excessive declarations to cover their data collection behaviors during compliance auditing. In this work, we present Pico, a privacy inconsistency detector, which checks the VPA app's privacy compliance by analyzing (in)consistency between data requested and data essential for its functionality. Pico understands the app's functionality topics from its publicly available textual data, and leverages advanced GPT-based language models to address domain-specific challenges. Based on the counterparts with similar functionality, suspicious data collection can be detected through the lens of anomaly detection. We apply Pico to understand the status quo of data-functionality compliance among all 65,195 skills in the Alexa app store. Our study reveals that 21.7% of the analyzed skills exhibit suspicious data collection, including Top 10 popular Alexa skills that pose threats to 54,116 users. These findings should raise an alert to both developers and users, in the compliance with the purpose limitation principle in data regulations.

[Paper]

Are Your Requests Your True Needs? Checking Excessive Data Collection in VPA App
Are Your Requests Your True Needs? Checking Excessive Data Collection in VPA App

Fuman Xie, Chuan Yan, Mark Huasong Meng, Shaoming Teng, Yanjun Zhang, Guangdong Bai

ICSE '24: Proceedings of the IEEE/ACM 46th International Conference on Software Engineering 2024

Virtual personal assistants (VPA) services encompass a large number of third-party applications (or apps) to enrich their functionalities. These apps have been well examined to scrutinize their data collection behaviors against their declared privacy policies. Nonetheless, it is often overlooked that most users tend to ignore privacy policies at the installation time. Dishonest developers thus can exploit this situation by embedding excessive declarations to cover their data collection behaviors during compliance auditing. In this work, we present Pico, a privacy inconsistency detector, which checks the VPA app's privacy compliance by analyzing (in)consistency between data requested and data essential for its functionality. Pico understands the app's functionality topics from its publicly available textual data, and leverages advanced GPT-based language models to address domain-specific challenges. Based on the counterparts with similar functionality, suspicious data collection can be detected through the lens of anomaly detection. We apply Pico to understand the status quo of data-functionality compliance among all 65,195 skills in the Alexa app store. Our study reveals that 21.7% of the analyzed skills exhibit suspicious data collection, including Top 10 popular Alexa skills that pose threats to 54,116 users. These findings should raise an alert to both developers and users, in the compliance with the purpose limitation principle in data regulations.

[Paper]

Analyzing Excessive Permission Requests in Google Workspace Add-ons
Analyzing Excessive Permission Requests in Google Workspace Add-ons

Liuhuo Wan, Chuan Yan, Mark Huasong Meng, Kailong Wang, Haoyu Wang

ICECCS '24: 28th International Conference on Engineering of Complex Computer Systems 2024

In the digital era, business collaboration platforms have become pivotal in facilitating seamless remote work and virtual team interactions. These platforms, typified by Google Workspace, offer an integrated suite of tools~(such as Google Docs, Slides, and Calendar) that significantly enhance business operations. They often extend their functionality through the integration of third-party applications, known as “add-ons”. Google Workspace exemplifies this trend, blending traditional business solutions with advanced, add-on-driven capabilities. While this greatly augments productivity and collaboration for online personal or team work, concerns about the excessive use of data and permissions have been raised by both users and legislators, as add-ons can utilize the granted permissions to access and manipulate files managed by business collaboration platforms. In this work, we propose an end-to-end approach to automatically detecting excessive permissions among add-ons. It advocates purpose limitation that the requested permissions of the add-on should be for its specific functionality and in compliance with the actual needs in fulfilling the functionality. Our approach utilizes a hybrid analysis to detect excessive permissions, including analysis of the add-on’s runtime behavior and source code, and state-of-the-art language processing techniques for textual artifact interpretation. This approach can serve the users, developers and store operators as an efficient and practical detection mechanism for excessive permissions. We conduct a large-scale diagnostic evaluation on 3,756 add-ons, revealing that almost half of existing add-ons contain issues of excessive permissions. We further investigate the root cause of excessive permissions and provide insights to stakeholders. Our work should raise the awareness of add-on users, service providers, and platform operators, and encourage them to implement solutions that restrict the excessive permissions in practice.

Analyzing Excessive Permission Requests in Google Workspace Add-ons
Analyzing Excessive Permission Requests in Google Workspace Add-ons

Liuhuo Wan, Chuan Yan, Mark Huasong Meng, Kailong Wang, Haoyu Wang

ICECCS '24: 28th International Conference on Engineering of Complex Computer Systems 2024

In the digital era, business collaboration platforms have become pivotal in facilitating seamless remote work and virtual team interactions. These platforms, typified by Google Workspace, offer an integrated suite of tools~(such as Google Docs, Slides, and Calendar) that significantly enhance business operations. They often extend their functionality through the integration of third-party applications, known as “add-ons”. Google Workspace exemplifies this trend, blending traditional business solutions with advanced, add-on-driven capabilities. While this greatly augments productivity and collaboration for online personal or team work, concerns about the excessive use of data and permissions have been raised by both users and legislators, as add-ons can utilize the granted permissions to access and manipulate files managed by business collaboration platforms. In this work, we propose an end-to-end approach to automatically detecting excessive permissions among add-ons. It advocates purpose limitation that the requested permissions of the add-on should be for its specific functionality and in compliance with the actual needs in fulfilling the functionality. Our approach utilizes a hybrid analysis to detect excessive permissions, including analysis of the add-on’s runtime behavior and source code, and state-of-the-art language processing techniques for textual artifact interpretation. This approach can serve the users, developers and store operators as an efficient and practical detection mechanism for excessive permissions. We conduct a large-scale diagnostic evaluation on 3,756 add-ons, revealing that almost half of existing add-ons contain issues of excessive permissions. We further investigate the root cause of excessive permissions and provide insights to stakeholders. Our work should raise the awareness of add-on users, service providers, and platform operators, and encourage them to implement solutions that restrict the excessive permissions in practice.

2023

On the quality of privacy policy documents of virtual personal assistant applications
On the quality of privacy policy documents of virtual personal assistant applications

Chuan Yan, Fuman Xie, Mark Huasong Meng, Yanjun Zhang, Guangdong Bai

PETS'24:The 24th Privacy Enhancing Technologies Symposium 2024

An app ecosystem built around virtual personal assistant (VPA) services becomes flourishing in recent years, fueled by the booming of the Internet of Things (IoT). A large number of functionality-rich VPA applications (or apps for short) have been released through app stores, and become easily-accessible by users through their smart speakers. In response to the increasingly stringent data protection regulations around the world, VPA service providers require app developers to include a privacy policy that declares their data handling practices. These privacy policies serve as the de facto agreement between developers and users, and may be taken as the basis in resolving conflicts in the event of a data breach. Therefore, it is essential that privacy policy documents are crafted in a clear, easy-to-understand, and unambiguous way. In this work, we conduct the first systematic study on the quality of privacy policies in the VPA app domain. Based on our review of literature and documents from standard working groups, we identify four metrics that enable the quality of the privacy policy to become measurable, including timeliness, availability, completeness and readability. We then develop QuPer, which extracts the meta features (e.g., update history) and linguistic features (e.g., sentence semantics) from privacy policies, and assesses their quality. Our analysis reveals that the status of the quality of privacy policies in the VPA app domain is concerning. For instance, only 1.17% of privacy policies completely cover all contents that are regarded as privacy concerns by legislation (e.g., GDPR article 13) and relevant literature. Our findings are expected to raise an alert among the VPA app developers and provide them with guidelines for creating high-quality privacy policy documents. We also encourage app store operators to implement a vetting process that ensures the quality of privacy policies before apps are released to the public.

[Paper]

On the quality of privacy policy documents of virtual personal assistant applications
On the quality of privacy policy documents of virtual personal assistant applications

Chuan Yan, Fuman Xie, Mark Huasong Meng, Yanjun Zhang, Guangdong Bai

PETS'24:The 24th Privacy Enhancing Technologies Symposium 2024

An app ecosystem built around virtual personal assistant (VPA) services becomes flourishing in recent years, fueled by the booming of the Internet of Things (IoT). A large number of functionality-rich VPA applications (or apps for short) have been released through app stores, and become easily-accessible by users through their smart speakers. In response to the increasingly stringent data protection regulations around the world, VPA service providers require app developers to include a privacy policy that declares their data handling practices. These privacy policies serve as the de facto agreement between developers and users, and may be taken as the basis in resolving conflicts in the event of a data breach. Therefore, it is essential that privacy policy documents are crafted in a clear, easy-to-understand, and unambiguous way. In this work, we conduct the first systematic study on the quality of privacy policies in the VPA app domain. Based on our review of literature and documents from standard working groups, we identify four metrics that enable the quality of the privacy policy to become measurable, including timeliness, availability, completeness and readability. We then develop QuPer, which extracts the meta features (e.g., update history) and linguistic features (e.g., sentence semantics) from privacy policies, and assesses their quality. Our analysis reveals that the status of the quality of privacy policies in the VPA app domain is concerning. For instance, only 1.17% of privacy policies completely cover all contents that are regarded as privacy concerns by legislation (e.g., GDPR article 13) and relevant literature. Our findings are expected to raise an alert among the VPA app developers and provide them with guidelines for creating high-quality privacy policy documents. We also encourage app store operators to implement a vetting process that ensures the quality of privacy policies before apps are released to the public.

[Paper]

2022

Scrutinizing privacy policy compliance of virtual personal assistant apps
Scrutinizing privacy policy compliance of virtual personal assistant apps

Fuman Xie, Yanjun Zhang, Chuan Yan, Suwan Li, Lei Bu, Kai Chen, Zi Huang, Guangdong Bai

Proceedings of the 37th IEEE/ACM international conference on automated software engineering 2022

A large number of functionality-rich and easily accessible applications have become popular among various virtual personal assistant (VPA) services such as Amazon Alexa. VPA applications (or VPA apps for short) are accompanied by a privacy policy document that informs users of their data handling practices. These documents are usually lengthy and complex for users to comprehend, and developers may intentionally or unintentionally fail to comply with them. In this work, we conduct the first systematic study on the privacy policy compliance issue of VPA apps. We develop Skipper, which targets Amazon Alexa skills. It automatically depicts the skill into the declared privacy profile by analyzing their privacy policy documents with Natural Language Processing (NLP) and machine learning techniques, and derives the behavioral privacy profile of the skill through a black-box testing. We conduct a large-scale analysis on all skills listed on Alexa store, and find that a large number of skills suffer from the privacy policy noncompliance issues.

[Paper]

Scrutinizing privacy policy compliance of virtual personal assistant apps
Scrutinizing privacy policy compliance of virtual personal assistant apps

Fuman Xie, Yanjun Zhang, Chuan Yan, Suwan Li, Lei Bu, Kai Chen, Zi Huang, Guangdong Bai

Proceedings of the 37th IEEE/ACM international conference on automated software engineering 2022

A large number of functionality-rich and easily accessible applications have become popular among various virtual personal assistant (VPA) services such as Amazon Alexa. VPA applications (or VPA apps for short) are accompanied by a privacy policy document that informs users of their data handling practices. These documents are usually lengthy and complex for users to comprehend, and developers may intentionally or unintentionally fail to comply with them. In this work, we conduct the first systematic study on the privacy policy compliance issue of VPA apps. We develop Skipper, which targets Amazon Alexa skills. It automatically depicts the skill into the declared privacy profile by analyzing their privacy policy documents with Natural Language Processing (NLP) and machine learning techniques, and derives the behavioral privacy profile of the skill through a black-box testing. We conduct a large-scale analysis on all skills listed on Alexa store, and find that a large number of skills suffer from the privacy policy noncompliance issues.

[Paper]