Friday, June 20, 2025

Privacy Breach and Artificial Intelligence

Introduction

The term ‘AI’ is a wide phenomenon and it encompasses wide range of technologies. AI basically refers to the evolution of computer systems that would do the given tasks which is unique characteristic of human intelligence like problem-solving, decision-making, language understanding, and perception.

This is very well put by an eminent Professor of Law Ryan Calo and he very nicely says: “There is no straightforward, consensus definition of artificial intelligence. AI is best understood as a set of techniques aimed at approximating some aspect of human or animal cognition using machines.”[1]

Thus, AI though taken as a misleading term for machine learning, it appears that the train seems to have left the station and now there is no turning back. AI has become the magic buzzword in the present times. Mere use of the term ‘AI’ opens new vistas which attracts the investors, money, excitement and attention. Hence, in the tech industry the use of AI has become inevitable and it is being slammed on almost every piece of code. Therefore, it has become very difficult to identify what really falls or does not fall in the category of AI. AI is actually both a term and a metaphor. Metaphors are like a lens through which we see and interpret things, making a juxtaposition that shape our understanding and processes. As Ryan Calo insightfully observes, “Every metaphor is, in its own way, an argument.”[2]

Metaphors have the dual capability to both clarify and distort our perception. However, though the metaphors may give technology human-like qualities, it also strips it of its human essence. Yet the human element is an integral part of technology.

It was further observed by Kate Crawford and he remarked that human involvement is deeply embedded in AI at almost every stage.[3] The data used to train AI algorithms originates from human activities, thoughts, and conversations, and it is often curated by humans too. Humans are instrumental in designing and training algorithmic models.[4]

As Eric Siegel observed, the most effective machine learning is “supervised machine learning” which involves training data that is labelled. The algorithm “learns” from the labels and can confirm it is getting something right or wrong. It is humans who label this data. AI, while seemingly ethereal, is profoundly physical, rooted in intense human labor—often arduous and taxing—and reliant on substantial energy and material resources.[5]

In short, Artificial intelligence is basically intelligence displayed by machines, primarily the computer systems. It is an area of research in computer science which evolves and examines the methods and softwares that aid the machines to understand the ecosystem and use learning and intelligence to take actions that multiplies their prospects of achieving the targets. Those machines could be called as AI.

Some of the renowned applications of AI are: advanced web search engines for example., Google Search; recommendation systems which is used by You Tube, Amazon, and Netflix; interacting via human speech such as, Google Assistant, Siri, and Alexa; autonomous vehicles like Waymo; generative and creative tools like Chat GPT and AI art. However, many AI applications are not perceived as AI: “A lot of cutting edge AI has filtered into general applications, often without being called AI because once something becomes useful enough and common enough it’s not labelled AI anymore.”[6]

The other subfields of AI research revolve around certain goals and the use of explicit tools. The classical goals of AI research comprise of reasoning, knowledge representation, planning, learning, natural language processing, perception, and support for robotics. In order to attain these goals, AI researchers have refashioned and consolidated an expansive range of techniques, including search and mathematical programming, symbolic logic, simulated neural networks, and the methods which are based on statistics, operations research, and economics.[7]

Artificial intelligence was founded as an academic discipline in 1956,[8] and the purported field went through multiple cycles of optimism,[9] which was followed by spells of failures and losses of funding, known as AI winter.[10] There has been an upsurge in funding and interest since 2012 when the concept of deep learning evolved and surpassed the existing AI techniques.[11] After this, in the year 2017, the growth advanced with the transformer architecture,[12] and there has been a great shift by early 2020s hundreds of billions of dollars were being invested in AI also called as the “AI boom”. The extensive use of AI in the 21st century led to numerous unintended repercussions and detrimental effects both in the present and in the future, which gave to a new topic of discussion with respect to the regulatory policies and benefit that would ensure the safety and benefits of the technology.

Privacy vis-à-vis Data Protection

Privacy is a right of the individuals to take their voluntary decisions, calculate their behaviour in the given environment and be prudent in their social interplay. Governments have contended that in some conditions, like prisons, any assumption of privacy is not valid, but in many circumstances, the presupposition of privacy is required to be curtailed. The control of Government on privacy has been taken as a method of preventing the crime; still it is seen more as a means to ‘control’. The idea of limiting the privacy provides the government with such informations about which the people living in a society ought to be controlled. In the Supreme Court of United States privacy has often been considered as a function of context in which the physical environment of the person is the determining factor of what privacy they should be expecting. But there are other considerations as well which includes person’s age and his mental capacity.

Data Privacy and Data Protection

Both the terms are often used interchangeably as they seem very similar but in reality, they are different as far as their meanings are concerned as it depends on their usage. Despite the differences they are closely interconnected and complement each other in the actual process.

Data privacy means the proper handling of data, determining whether or what data is to be shared with third parties by the organization. It basically implies an authorized access. For example, a Bank keeping the customer’s account details, their monetary transactions as private in order to keep customer’s identity safe and protected as much as possible by minimizing any kind of external risks. The Personal Health Information (PHI) and Personally Identifiable Information (PII), the financial information, medical records, social security or ID numbers, names, birth dates, and contact information are covered under the data privacy laws.

The EU General Data Protection Regulation (GDPR) defines personal data as: any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person.[13] Other countries also broadly define personal data. For example, under South Korea’s Personal Information Protection Act, personal information means “information pertaining to any living person that makes it possible to identify such individual by their name and resident registration number, image, etc.,” and specifically includes “information which, if not by itself, makes it possible to identify any specific individual if combined with other information.”[14] The Economist wrote in 2015, “the ability to compare databases threatens to make a mockery of [data] protections.”[15] Hence, data protection, means keeping the important data safe i.e.; protecting data against unauthorised access of all kinds of data whether it is personal or organizational data. For example; bank protecting all the records self-bank records as well as customer information from unauthorized accesses to keep everything safe and to ensure everything is under the control of bank administration. Data protection is basically the process taken to ensure the privacy, availability, and integrity of sensitive data, and is frequently used interchangeably with the word ‘data security’. These security measures are essential for organizations that gather, handle, or retain sensitive data. They work to avoid data corruption, loss, or harm. At a time when data collection and storage are growing at an unprecedented rate, a strong data protection strategy is critical. The major purpose of data protection is not just to preserve sensitive information, but also to keep it accessible and trustworthy, hence maintaining confidence and compliance in data-driven processes. There are certain Principles of Data Protection such as Data Availability; Data Lifecycle Management; Information Lifecycle Management.

The Digital Era and Privacy

Since the advent of the digital era, personal data has become a prized possession. Following this an enormous amount of data is generated and it is shared online on a regular basis thus enabling businesses, governments, and organisations to arrive at better decisions. Nonetheless, this data might be classified and people may not be willing to share. This is the point where the concept of privacy shows up. It’s a well known fact that privacy implies the right to keep the personal information undisclosed and unshackled from unaccredited access. Privacy is also seen in the light of human rights and it is considered an essential right that would ensure an individual’s control over their personal data and its usage.

In the present times, the importance of privacy has risen than ever before. In fact, the amount of personal data being collected and scrutinized continues to go upwards. In the context of AI, privacy is essential to ensure that AI systems are not used to manipulate individuals or discriminate against them based on their personal data. AI systems that rely on personal data to make decisions must be transparent and accountable to ensure that they are not making unfair or biased decisions. Hence, privacy is a fundamental right which is essential for human existence and it is granted to the citizens by the Constitution of India.

Privacy Challenges in the Age of AI

AI presents a challenge to the privacy of individuals and organisations because of the complexity of the algorithms used in AI systems. As AI becomes more advanced, it can make decisions based on subtle patterns in data that are difficult for humans to discern. This means that individuals may not even be aware that their personal data is being used to make decisions that affect them.

The Violation of Privacy

The development of technology always has been a sword with two sharp edges. So on the one hand AI provides with certain potential benefits while, on the other hand, it throws us with significant challenges. For example; the biggest challenge is the violation of privacy which could be followed by the idea of getting into the wrong hands and being misused causing identity theft or cyberbullying.

Bias and Discrimination

The next potential threat posed by AI technology is of bias and discrimination. The AI systems are based on training so there are chances of bias as they can be unbiased only in the case where they are trained which results in the system also functioning the same way. This could be disadvantageous and affect number of factors. Therefore, it becomes mandatory that the AI systems are trained on various data and are regularly audited in order to prevent bias. If we see this situation initially we are unable to find a link between bias and discrimination in AI and privacy as it is not apparent but the reality is that they are very closely interlinked. To understand this situation better we can take an imaginary situation wherein a company uses the AI system to screen the job applications. Here if the system is biased against a gender, say for example women or people of specific race it may very unfairly exclude them from the process of selection. This will harm the applicants and would perpetuate systemic inequalities.

Job Displacements for Workers

Another crucial issue related to AI is the employment issue and economic disturbance. This is because AI systems are very advance and they are capable of performing tasks that were previously done by individuals which has led to the job displacement in few sectors. This economic disturbance has escalated the financial insecurity of people. This is the situation which might compel the individuals to abdicate their privacy to make ends meet. For example, there is a worker who has lost his job as a result of automation. They are trapped in a very bad situation where they are struggling to survive and thus are forced to take the option of gig economy to make money. Here in order to get the job they are required to provide personal information to the platforms which would include their location, work history, and ratings from the previous clients. Though this may be essential to get the job, it is a serious issue of concern as far as privacy is concerned because this data might be shared with third parties or might as well be used for targeting ads. This is the sad reality of every sector where the job applicants may not be aware about their data being collected from them and used in this way. Thus, we see how privacy is at stake in the era of AI technology.

The Data Abuse Practices

One of the most significant challenges posed by AI technology is of the potential threat of misuse by bad actors. This implies the ability of AI to create the fake images and videos which could be used to spread false information or even maneuver public opinion. Besides, AI could be used to design extremely polished phishing attacks, which can deceive people to disclose the confidential information by clicking on benign links. This kind of dispersion of fake images and videos causes serious damage to an individual’s privacy. The AI technology has been growing immensely in present times and poses new challenges which needs to be addressed in order to ensure that it is used in an ethical way. The reason behind the challenges are that it should be used ethically and responsibly. The AI software is an ever evolving subject which should be used with great precaution and care so that the privacy of individuals is not affected.

Underlying Privacy Issues in the Age of AI

Privacy of late has become a complex phenomenon especially after the advent of AI. The huge amount of data that is being amassed and assessed by various companies and the governments puts the individual’s exclusive particulars at a substantial risk than ever before. Among these are intrusive surveillance which in a way destroys an individual’s liberty and complicates the authority leading to imbalance in the system; and then there is unaccredited data collection which again makes the individuals prone to cyber-attacks. These issues are mostly constituted of the power of big organisations which have extensive quantity of data for chucking out and considerable supremacy over how that data is assembled, assessed and used. There are some implications of each of these problems which are as follows:

The Power of Big Tech on Data

The Big organisations have become very powerful institutions in the world in the recent times with a huge impact over the global economy and the society at large. Since the rise of AI and a shift in metaverse their power is only going to increase further. With the rise of AI and the impending shift to the metaverse, their power is only set to increase even further.

If we look at the global scenario, the Big Tech companies such as Google, Amazon, and Meta have an access to the large amounts of data which gives them extraordinary power to have an impact over the customer’s psychology and shaping the global economy. These companies are also taking part in politics more than ever as they have the capacity to exert influence on public opinion and even determine the government policy.

As we are moving towards the metaverse, where individuals will be living, working and interacting in a virtual environment, it will be adding to the dominance of these Big Tech companies. The metaverse would be generating the data usage by over twenty times which is more than internet at present, giving a much better opportunity to the Big Tech companies to have an anchorage over their data and influence as well.

These Big Tech companies are now set to create a new kind of virtual ecosystems wherein they will have more authority over the buyer behaviour and this will provide the companies capitalize on their platforms and have a bigger impact on society. However, with these developments these companies also have the responsibility of collection and data usage in a righteous manner. Besides, they should make the various platforms all-inclusive and handy to all rather than being commanded by a few influential leaders.

AI and Surveillance

The most contentious usage of AI technology happens to be in the area of surveillance. The AI-based surveillance system has the prospects of transforming the legal system and the defence system but at the same time it poses a serious threat to privacy and civil liberties. The critics have been arguing that due to these aspects it raises concerns as it can be used to track and control people at the cost of their right to privacy and liberties. The AI systems are not always transparent. Mostly, they don’t let the individuals have any idea of being observed and what are the underlying factors for being checked in this manner. This lack of transparency would ruin the credence of people in the agencies whether it is law enforcement or security agency. It will rather make them edgy and apprehensive. Hence, in order to account for the concerns, there is a need to regulate the AI systems. For this, there is a need to develop lucid blueprint and strategies for the application of these systems as well as the setting up of unconstrained supervision and evaluations.

So, we derive from the above discussions that the hackers are always on a look out for new mechanisms and AI needs to be pliant and ready to learn from various methods of frauds. This means there is a need for adaptability. In addition to this, the banks have acquired a good knowledge and have set their AI systems to curb different kinds of frauds at the same time they are also trying not to make their customers unhappy. Then, there is healthcare system that is gravely affected as AI is also applied to track the health records of patients. In fact, both the examples of banking and healthcare exemplify how AI is plausibly most functional to the firms that utilize it in terms of supplementing and building up their procedures for the purpose of safeguarding and shielding their data. In the recent times, the collection of personal data has become very easy with the advent of some important technologies which includes smartphones, surveillance cameras and the most crucial thing which has taken everything under its wings is the omnipresence of “The Internet”. This has made possible to track every step an individual takes whether he visits the clubs, restaurants, workplace or anything else. People feverishly collect the data as a process of selfoptimisation and also as an attempt to change their lives for better. They upload most of these data to cloud computers which significantly increases the possibility to track the private informations of the individuals. The individuals most often deliberately ignore the fact that by uploading their private data they happen to transfer the copyright of those data to the providers and owners of the given platform. The platforms such as Facebook and others not only own the data but they also tend to use it and even sell the same to others. The most important factors that led to the success of Google in collection of the personal information is that people are unable to hide their interest while searching for informations online. This is because one cannot search for any information on Google without entering the key words in the search box.

The self-driving cars or the robotic cars consistently apprise via “telemetry data” as to how the car’s performance has been which aids these companies to anthologize the reports with respect to the specific car with respect to the safety perspective of the car. For example, Tesla gathers a report quarterly about how many kilometres its vehicles drove and how much it engaged the auto pilot or did not engage him at all.

The Inadvertent Usage of Private Data

The availability of the private data gives the AI systems an edge in performing good but there are some major threats associated with it. One of the major problems is the usage of data for the non-intended purposes. The users mostly have a little or no idea about their data being treated, employed or auctioned off. The programmatic advertisement has been quite a success and it denotes how the personal data could be used for making the individuals buy goods. This kind of marketing gained acceptance from the individuals in the society and they have thus formed a certain kind of mechanisms for their protection. We all know somewhere in our minds that we cannot trust the advertisements completely but we lack that kind of evaluative awareness towards AI.

Of all the things, there is a severe threat from the privacy perspective, that happens to be the ability of AI to imitate people is fraught with danger. In the year 2016, Adobe came up with its VoCo system which could copy the voice of anybody after having listened to it for almost 20minutes. It didn’t stop here. The videos began to be manipulated such as exchange of faces became a very crucial issue which took the problem to another level. It began to be used for taking revenge mostly in making porns. The vengeful ex-partners now could mount the face of their old flame onto the actors of the porn movies and later share them on the internet. These “deep fakes” tend to use the neural networks for the purpose of manipulating the videos. Apart from this, another issue is of the people entering into the contracts for different online services. The terms and conditions of these services on social networks are not an easy read but their consequences cannot be underestimated.

Another issue is of “Auto Insurance Discrimination”. Here the lack of privacy and the amount of data gathered might lead to different set of rules to be applied to various groups. For example, the insurance companies value the existing data for the purpose of predicting the monetary value of the policy and thereafter assigning the premiums. The Auto insurance companies assess the behaviour of drivers and then assigns them the risks classes. The AI usage could lead to bias and discrimination here.

The Future Perspective

The idea of Artificial Intelligence is unique and it as a matter of fact holds a great deal for the future generation. Despite the vast study of approximately fifty years a lot is yet to be discovered and learnt in this field. I believe that everyone who shares some space in the field of science should have the basic knowledge of the theories of AI as everyone has his specific role to play in the formation of safe and secure future of intelligent technology. There are numerous movies of science fiction such as Wars, I Robots, Knight Rider, The Matrix, The Terminator, With Folded Hands, Transformers, Ghost in the Shell, Blade Runner and so on which might provide the base for arguments, issues and learnings.

Conclusion

The paper talks about the crucial role of AI and its efforts to implement data privacy and security. AI has become an important artillery for fighting against the cyber-attacks. Furthermore, in this paper we have studied and observed the implications of AI across different sectors such as banking and healthcare which shows the versatility and efficacy of its functionalities. Hence, we see that AI has a significant role in alleviating the challenges with respect to data privacy and security. So, AI will have a big role in securing the digital assets as there is a huge dynamism in cyber offences like cyber threats and the business data footprints which increase by way of pattern-based identification of the forthcoming danger. Despite the advantages of AI in various fields like healthcare, banking there remains a little gap where more attention towards careful stewardship is required. Besides, with the escalation and submergence of the digital era in the present times we can be hopeful to see a better advancement of technologies and their increasing bond with policies and ethical standards so that AI stands true in its affirmations with respect to the data privacy and security while maintaining unassailable coherence.

Footnotes

1 Ryan Calo, Artificial Intelligence Policy: A Primer and Roadmap, 51 U.C. Davis L. Rev. 399, 404 (2017)

2 Ryan Calo, Robots as Legal Metaphors, 30 Harv. J. L. & Tech. 209, 211 (2016)

3 KATE CRAWFORD, ATLAS OF AI: POWER, POLITICS, AND THE PLANETARY COSTS OF ARTIFICIAL INTELLIGENCE (2021); Rebecca Croot of, Margot E. Kaminski & W. Nicholson Price II, Humans in the Loop, 76 Vand. L. Rev. 429 (2023)

4 Margot E. Kaminski, Binary Governance: Lessons from the GDPR’s Approach to Algorithmic Accountability, 92 S. Cal. L. Rev. 1529, 1538-39 (2019)

5 CRAWFORD, ATLAS OF AI, supra note X, at 53-87; IVANABARTOLETTI, AN ARTIFICIALREVOLUTION: ON POWER, POLITICS, AND AI 81-93 (2020)

6 AI set to exceed human brain power Archived 2008-02-19 at the Wayback Machine CNN.com (July 26, 2006)

7 Russell & Norvig (2021), Luger & Stubblefield (2004), Poole, Mackworth & Goebel (1998) and Nilsson (1998)

8 Dartmouth Workshop: Russell & Norvig (2021, p. 18), McCorduck (2004, pp. 111–136), NRC (1999, pp. 200–201) The proposal: McCarthy et al. (1955)

9 Successful programs the 1960s: McCorduck (2004, pp. 243–252), Crevier (1993, pp. 52–107), Moravec (1988, p. 9), Russell & Norvig (2021, pp. 19–21)

10 AI Winter, Lighthill report, Mansfield Amendment: Crevier (1993, pp. 115–117), Russell & Norvig (2021, pp. 21–22), NRC (1999, pp. 212–213), Howe (1994), Newquist (1994, pp. 189–201)

11 Deep learning revolution, AlexNet: Goldman (2022), Russell & Norvig (2021, p. 26), McKinsey (2018)

12 Toews, Rob (3 September 2023). “Transformers Revolutionized AI. What Will Replace Them?”. Forbes. Archived from the original on 8 December 2023. Retrieved 8 December 2023.

13 GDPR, supra article 4(1) (emphasis added)

14 Article 2(1) South Korea Personal Information Protection Act. Official English translation available at http://law.go.kr/engLsSc.do?menuId=0&subMenu=5&query=%EA%B0%9C%EC%9D%B8%EC%A0%95%EB%B3%B4%EB%B3%B4%ED%98%B8%EB%B2%95#.

15 “We’ll See You, Anon,” The Economist (13 Aug. 2015), available at https://www.economist.com/science-and-technology/2015/08/13/well-see-you-anon.

Also Read:
Rights of undertrial prisoners in India
How To Send A Legal Notice In India

Dr. Vidyottma Jha
Dr. Vidyottma Jha
ADVOCATE, SUPREME COURT OF INDIA
RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Most Popular