AI Ethics and Governance: Societal Impact and Accountability

To balance commercial innovation with responsibility, businesses must prioritise AI ethics and governance—the combination of principles, policies, technical controls, and accountability structures that shape how AI systems are designed, deployed, monitored, and corrected. As we explored in Part I of this series, AI bias can emerge in unexpected ways and negatively affect certain groups. We saw, for instance, how AI hiring systems might misinterpret pre-interview social media profiling.

Preventing such discrimination requires ethical AI development. This means building systems that are not just accurate but also fair, explainable, and aligned with human supervision. For ethical AI companies, governance cannot be a post-launch add-on; it must proactively guide data selection, model design, and remediation from day one.

Here in Part II, we’ll look at how the use of AI affects society at large in far-reaching ways. We’ll delve into technical considerations around AI bias, the relationship between bias and reinforcement, and how the responsible use of AI—supported by a robust ethical AI framework—can mitigate its negative effects.

The Societal Impact of Biased AI Systems

In both the examples we’ll now look at—examples of how bias in AI systems results in unfairness, negative outcomes, and such—you’ll notice the idea of reinforcement, which we’ll soon explore in detail.

Example 1: Healthcare and Socioeconomic Considerations

You’re probably aware that AI is increasingly being used in healthcare, particularly in diagnosis and treatment recommendations. The primary benefit, of course, is that AI systems can process vast amounts of medical data quickly; the issue of bias rears its head here as well. Consider that an AI system trained on medical records that predominantly represent one demographic may be less accurate in diagnosing or treating conditions for people from other demographics. If the first demographic is “middle-aged white men,” the other demographics could include women as well as ethnic minorities—who have different health risks and respond differently to treatments.

The bias here can result in misdiagnoses or suboptimal care for underrepresented groups; this, in turn, perpetuates health disparities. Studies have shown that some AI diagnostic tools have been less accurate at detecting certain conditions, such as skin cancer, in darker-skinned individuals because the systems were trained predominantly on images of lighter-skinned patients. More pointedly, a 2019 study found that a leading software used to identify high-risk patients for complex health needs consistently assigned lower risk scores to black patients who had the same risk levels as their white counterparts.

The cycle here is clear: AI, which is designed to improve healthcare outcomes, can inadvertently reinforce the inequities in healthcare access and quality that already exist. As we’ll reiterate later, it is vital for healthcare systems—and ethical AI companies—to use datasets that are diverse, representative, and continuously updated so AI systems don’t unintentionally marginalise vulnerable populations.

Example 2: Filter Bubbles and Consumers

The phenomenon of filter bubbles, and its effects, are most easily understood with examples. Consider personalised content recommendations: Algorithms analyse past behaviour and preferences to recommend content that aligns with individual tastes. Positive aspects aside, this can also limit exposure to diverse viewpoints—which ends up reinforcing existing beliefs and interests. Then, think of search engines, which use AI to prioritise results based on previous searches and on stated preferences. This can reinforce filter bubbles: Users are shown content similar to what they’ve previously engaged with—again, limiting access to new perspectives.

The easiest example perhaps comes from one’s choice in YouTube videos:

You believe a certain idea, X.

You notice a YouTube video whose title suggests X is true.

You click and watch it.

The related videos at the end will most likely also suggest X is true. You watch a couple.

YouTube’s algorithm, designed to maximise viewing time, notices your preference for videos supporting X.

It then recommends more such videos. Driven by your natural human tendency to seek confirmation, you’re likely to click on those videos.

Then on, each time you visit YouTube.com, your list of recommended videos—insofar as they involve Idea X at all—will almost all hold that X is true!

We can now understand the effects of filter bubbles in the context of consumer choice:

• Individuals tend to seek out information that confirms existing beliefs. Filter bubbles intensify confirmation bias—a bias in which an AI system mirrors that exact tendency—by repeatedly surfacing content that matches what users already believe, watch, or buy.
• Filter bubbles stifle discovery by making it less likely for users to encounter unfamiliar viewpoints, products, or services.
• They narrow competition by favouring previously engaged content over genuinely new options.
• AI algorithms often prioritise content based on commercial interests, thereby shaping consumer choices in subtle but powerful ways. (Think of Amazon’s “sponsored recommendations,” which—while relevant—appear higher up in the list of what the user sees.)

Reinforcement Reinforces

Coming back to the idea of reinforcement: Reinforcement is often the engine that drives AI bias. AI systems—which are designed to improve personalisation, for instance, and decision-making—have the inherent potential to amplify existing inequalities or pre-existing beliefs, creating a feedback loop that reinforces the current state of affairs rather than challenging or improving them.

Consider our examples above—healthcare and filter bubbles—to see why this is the case.

Many AI systems—especially those used for recommendations or decision-making—employ reinforcement learning. This means they learn by receiving feedback based on their actions. If the system’s actions align with existing biases, it is “rewarded”—such as through higher engagement. This, naturally, reinforces the biases.

In our healthcare example, if an AI system were trained on data where underserved communities have historically received less comprehensive care, it might learn to prioritise resources for wealthier areas, perpetuating the cycle. More pointedly: If the system’s recommendations lead to better outcomes for already privileged groups, its “perception” that its decisions are “correct” is reinforced—even though the recommendations will exacerbate inequities.

Consider the filter-bubble case. If the system recommends content that aligns with a user’s existing views, it receives positive feedback such as clicks and increased engagement. (There is a strong human angle to this: People tend to engage more with content that aligns with their views, as we mentioned.) The AI ends up further prioritising similar content—and it can’t be blamed!

The larger point is that AI doesn’t just replicate existing biases; it amplifies them—which is what makes the problem of bias as difficult as it is. You can imagine that once a bias is embedded in an AI system, the system will continue to replicate that bias over time—which is especially concerning because AI systems operate at a scale and speed that humans cannot match.

…And then there’s the “black box” nature of some AI systems (where no one individual really knows what’s going on inside the system). This makes it difficult to identify and correct biases, and obscures the reinforcement mechanisms at play.

The reinforcement of bias through AI is, ultimately, an ethical challenge—and the phenomenon makes ethical AI development crucial. Consider that one result of bias-reinforcement is that AI can deepen societal divides. The “problematic power” of reinforcement makes it all the more important to mitigate existing biases in AI systems; to break the cycle of reinforcement. This can be done (though it’s easier said!) by using diverse and more representative datasets, developing algorithms that are fair and transparent (so corrective actions can be taken if reinforcement of bias occurs), and implementing mechanisms to detect and correct bias. We’ll soon elaborate upon this.

As food for thought, AI is a mirror that reflects the biases in our world—including social, economic, and political inequalities. As a tool, AI reflects the society that creates it; “rectifying” bias in AI systems means societal changes are needed, too. More plainly, the reinforcement of bias by AI is not a technical flaw but a societal issue.

AI Ethics and Governance: Addressing the Roots of Bias

We’ve spoken about how an AI system can inherit bias from its training data; we’ve reinforced that unless the training data is “perfectly” objective, the AI system will perpetuate stereotypes and existing inequalities. The most significant contributors to bias include:

• Biased Training Data: We’ve explored how AI systems, if trained on biased historical data, replicate the inequities of the past.
• Unrepresentative Data: This refers to the training data only being representative of a certain segment of the population (or other sample). For instance, a facial recognition system trained predominantly on faces of white people will be less accurate in identifying people of colour.
• Algorithmic Design Choices: The manner in which AI models are designed also influences their potential biases. Developers may inadvertently prioritise certain features over others, amplifying bias within the system; for instance, an AI hiring system that prioritises age may discriminate against older candidates.

These issues are not just technical problems; they are challenges that require a generally agreed-upon ethical AI framework.

As a first step, developers—and others involved in the creation of AI systems—need to recognise that training data can be biased or unrepresentative, and that the design of a certain algorithm might be discriminatory.

Addressing these biases requires proactive solutions at every stage of the AI development lifecycle. Bias can be mitigated by using diverse datasets that reflect the full spectrum of society. These can be obtained by actively seeking out data from underrepresented groups and ensuring that demographic diversity—whether by race, gender, age, or other factors—is represented. Second, AI systems should be regularly audited to identify any gaps in the data or disproportionate outcomes affecting specific groups.

Third—and very importantly—developers must prioritise transparency in their models. Ethical AI development entails making the design choices, data sources, and decision-making processes clear and understandable to both internal stakeholders and the public. The ideal situation is one where when an AI system does demonstrate bias, the source of the problem can be traced—whether it is in the training data or the algorithm’s design.

We need to mention fairness-aware algorithms, which developers can incorporate to counteract bias. Such algorithms may prioritise equality alongside accuracy, ensuring that predictions or classifications do not unfairly disadvantage a specific group.

Beyond data, algorithms, and development, there is a need for accountability mechanisms—such as corrective actions or recalibrations to improve fairness— to address discriminatory outcomes. Finally, consider that real-time monitoring can identify biases early and address them quickly—and that continual evaluation can verify that AI systems are fair and unbiased.

AI Ethics and Governance: Who Is Accountable?

To recapitulate the most important ideas we’ve discussed thus far: AI has tremendous potential to enhance the consumer experience—but it can also create filter bubbles, limit exposure to diverse choices, and amplify biases that affect consumer decision-making. The widespread use of AI in commerce has broader implications for society: Discriminatory algorithms can exacerbate existing inequalities and deepen social divisions.

Consider now that as more and more commercial activity begins to depend on AI systems, the ethical concerns become more significant—given that AI-driven decisions influence corporate actions, consumer behaviour, societal structures, and more. In the extreme, the role of AI in shaping public opinion and consumer behaviour raises questions about autonomy and even free will: It is not a stretch to say that consumers who are constantly guided by biased algorithms have lost—to a good extent—the ability to make independent choices.

Questionable ethics. Limited exposure to choices. Involuntary discrimination. Perpetuation of stereotypes. Consumer behaviour shaped by “black-box” algorithms. Reduced autonomy in consumer choice. The loss of free will. If all of these are automated, who is accountable?

At a minimum, accountability cannot disappear into the model, the vendor stack, or the phrase “the system decided.” The organisation deploying the AI remains accountable for the outcome. Within that organisation, responsibility should be assigned across data quality, model supervision, approval of the use case, ongoing monitoring, and remediation when harm occurs.

In other words, a business using AI in high-impact contexts should always be able to answer five questions: Who approved this system, who monitors it, who can pause it, who explains its decisions, and who is responsible when those decisions cause harm.

Towards Ethical AI Development

The issues we just touched upon—limited exposure to choices, involuntary discrimination, reinforcement of stereotypes, diminished consumer autonomy, and the ambiguity of accountability—are profound. Addressing them requires deliberate, proactive measures.

Transparency and Explainability

An essential step towards accountability is to increase transparency in AI systems through ethical AI development. Ethical AI companies must prioritise clear explanations of how their algorithms function, what data they utilise, and how decisions are arrived at. Consumers should be provided with comprehensible information about why they receive certain recommendations or face certain decisions—which can restore autonomy and reduce the impact of filter bubbles. They should have controls over personalisation algorithms—controls that allow them to opt out, or to customise settings. Such transparency also allows external experts and regulators to evaluate the fairness and ethical standards of AI systems.

Regular and Independent Auditing

Towards preventing bias—or at least identifying it so it can be rectified—ethical AI companies should commit to regular audits of their AI systems. Independent, ongoing third-party audits can uncover hidden biases, unrepresentative datasets, or discriminatory outcomes; those issues can and should be addressed before they become embedded in organisational systems.

Inclusive Data Practices

Towards ethical AI development, organisations must actively seek diverse, representative data that reflect real-world populations and scenarios. Incorporating data from groups diverse in terms of ethnicity, gender, and socioeconomic status will help prevent AI systems from perpetuating inequities. Along similar lines, ethical AI companies should promote diversity and inclusivity in their AI development teams to ensure a broad range of perspectives at the development stage.

Fairness-Aware Algorithms and an Ethical AI Framework

By integrating fairness-aware algorithms, ethical guidelines, and anti-discrimination standards into the AI development process—including design and training—organisations can reduce the risk of perpetuating harmful biases. Ethical AI frameworks must become a foundational part of technological design rather than an afterthought.

AI Ethics and Governance

Ambiguity around accountability can be addressed by establishing clear structures for AI governance and ethics. These involve guidelines and standards that encompass the principles of fairness, transparency, and accountability. Organisations that identify as ethical AI companies must clearly define roles and responsibilities such that they can always provide an answer to who is accountable if an AI system makes an outright biased decision. This might involve establishing independent ethical review boards that scrutinise AI systems before they are deployed to ensure they align with societal values. Such boards should be tasked with ongoing AI-system monitoring, updating guidelines for ethical AI frameworks, and managing responses to ethical dilemmas.

Industry-wide Collaboration and Regulation

Finally, ethical AI development requires collaboration across industries and policymakers. Common ethical standards and regulatory frameworks can to a good extent ensure accountability and fairness. Ethical AI companies need to openly share insights and challenges towards contributing to industry-wide improvements in AI ethics and governance.

A Final Word

Ethical AI development is not just a technical challenge—it is a moral imperative.

The prioritisation of AI ethics and governance while promoting transparency and accountability is necessary to harness the transformative potential of AI while mitigating its risks.

Accountability isn’t a question of assigning blame; it’s about committing to a continuous, transparent, and collective effort to ensure AI benefits all of society. 

Finally, moving towards ethical use of AI is not just a technological challenge; it is a societal imperative that requires a collaborative effort involving policymakers, researchers, industry leaders, and the public.

References and Further Reading

• AI in Healthcare (ForeSee Medical)
• AI transparency: What is it and why do we need it? (TechTarget)
• Can Artificial Intelligence (AI) – driven personalization influence customer experiences? (Department of Business Studies, Uppsala University)
• Confirmation bias (Britannica)
• Ethnicity and psychopharmacology (National Library of Medicine)
• FAI: Fairness-Aware Algorithms for Network Analysis (Michigan State University)
• Health Equity and Ethical Considerations in Using Artificial Intelligence in Public Health and Medicine (CDC)
• How to Develop an Effective AI Governance Framework? (Securiti)
• Improving Skin Color Diversity in Cancer Detection: Deep Learning Approach (National Library of Medicine)
• Personalization Vs. Privacy: Balancing Consumer Interests (Forbes)
• Responsible AI: Compliant, ethical and innovative (FRISS)
• Sponsored Products (Amazon Ads)
• Stop Screening Job Candidates’ Social Media (Harvard Business Review)
• Study finds gender and skin-type bias in commercial artificial-intelligence systems (MIT News)
• The potential for artificial intelligence in healthcare (National Library of Medicine)
• There is no such thing as race in health-care algorithms (The Lancet)
• Understanding the AI Auditing Framework (Codewave)
• What is AI bias? (IBM)
• Women Are Still Under-Represented in Medical Research. Here’s Where the Gender Gap Is Most Pronounced (Time)

FAQs