[House Hearing, 117 Congress]
[From the U.S. Government Publishing Office]


                        BEYOND I, ROBOT: ETHICS,
                        ARTIFICIAL INTELLIGENCE,
                          AND THE DIGITAL AGE

=======================================================================

                             VIRTUAL HEARING

                               BEFORE THE

                 TASK FORCE ON ARTIFICIAL INTELLIGENCE

                                 OF THE

                    COMMITTEE ON FINANCIAL SERVICES

                     U.S. HOUSE OF REPRESENTATIVES

                    ONE HUNDRED SEVENTEENTH CONGRESS

                             FIRST SESSION

                               __________

                            OCTOBER 13, 2021

                               __________

       Printed for the use of the Committee on Financial Services

                           Serial No. 117-52
                           
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]

                               __________

                    U.S. GOVERNMENT PUBLISHING OFFICE                    
46-195 PDF                 WASHINGTON : 2022                     
          
-----------------------------------------------------------------------------------   

                 HOUSE COMMITTEE ON FINANCIAL SERVICES

                 MAXINE WATERS, California, Chairwoman

CAROLYN B. MALONEY, New York         PATRICK McHENRY, North Carolina, 
NYDIA M. VELAZQUEZ, New York             Ranking Member
BRAD SHERMAN, California             FRANK D. LUCAS, Oklahoma
GREGORY W. MEEKS, New York           BILL POSEY, Florida
DAVID SCOTT, Georgia                 BLAINE LUETKEMEYER, Missouri
AL GREEN, Texas                      BILL HUIZENGA, Michigan
EMANUEL CLEAVER, Missouri            ANN WAGNER, Missouri
ED PERLMUTTER, Colorado              ANDY BARR, Kentucky
JIM A. HIMES, Connecticut            ROGER WILLIAMS, Texas
BILL FOSTER, Illinois                FRENCH HILL, Arkansas
JOYCE BEATTY, Ohio                   TOM EMMER, Minnesota
JUAN VARGAS, California              LEE M. ZELDIN, New York
JOSH GOTTHEIMER, New Jersey          BARRY LOUDERMILK, Georgia
VICENTE GONZALEZ, Texas              ALEXANDER X. MOONEY, West Virginia
AL LAWSON, Florida                   WARREN DAVIDSON, Ohio
MICHAEL SAN NICOLAS, Guam            TED BUDD, North Carolina
CINDY AXNE, Iowa                     DAVID KUSTOFF, Tennessee
SEAN CASTEN, Illinois                TREY HOLLINGSWORTH, Indiana
AYANNA PRESSLEY, Massachusetts       ANTHONY GONZALEZ, Ohio
RITCHIE TORRES, New York             JOHN ROSE, Tennessee
STEPHEN F. LYNCH, Massachusetts      BRYAN STEIL, Wisconsin
ALMA ADAMS, North Carolina           LANCE GOODEN, Texas
RASHIDA TLAIB, Michigan              WILLIAM TIMMONS, South Carolina
MADELEINE DEAN, Pennsylvania         VAN TAYLOR, Texas
ALEXANDRIA OCASIO-CORTEZ, New York   PETE SESSIONS, Texas
JESUS ``CHUY'' GARCIA, Illinois
SYLVIA GARCIA, Texas
NIKEMA WILLIAMS, Georgia
JAKE AUCHINCLOSS, Massachusetts

                   Charla Ouertatani, Staff Director
                 TASK FORCE ON ARTIFICIAL INTELLIGENCE

                    BILL FOSTER, Illinois, Chairman

BRAD SHERMAN, California             ANTHONY GONZALEZ, Ohio, Ranking 
SEAN CASTEN, Illinois                    Member
AYANNA PRESSLEY, Massachusetts       BARRY LOUDERMILK, Georgia
ALMA ADAMS, North Carolina           TED BUDD, North Carolina
SYLVIA GARCIA, Texas                 TREY HOLLINGSWORTH, Indiana
JAKE AUCHINCLOSS, Massachusetts      VAN TAYLOR, Texas
                           
                           
                           C O N T E N T S

                              ----------                              
                                                                   Page
Hearing held on:
    October 13, 2021.............................................     1
Appendix:
    October 13, 2021.............................................    25

                               WITNESSES
                      Wednesday, October 13, 2021

Broussard, Meredith, Associate Professor, Arthur L. Carter 
  Journalism Institute of New York University....................     4
Cooper, Aaron, Vice President, Global Policy, BSA--The Software 
  Alliance.......................................................    12
King, Meg, Director, Science and Technology Innovation Program, 
  The Wilson Center..............................................     8
Vogel, Miriam, President and CEO, EqualAI........................     6
Yong, Jeffery, Principal Advisor, Financial Stability Institute, 
  Bank for International Settlements.............................    10

                                APPENDIX

Prepared statements:
    Broussard, Meredith..........................................    26
    Cooper, Aaron................................................    33
    King, Meg....................................................    74
    Vogel, Miriam................................................    79
    Yong, Jeffery................................................    88

              Additional Material Submitted for the Record

Garcia, Hon. Sylvia:
    Written responses to questions for the record submitted to 
      Jeffery Yong...............................................    90

 
                        BEYOND I, ROBOT: ETHICS,
                        ARTIFICIAL INTELLIGENCE,
                          AND THE DIGITAL AGE

                              ----------                              


                      Wednesday, October 13, 2021

             U.S. House of Representatives,
             Task Force on Artificial Intelligence,
                           Committee on Financial Services,
                                                   Washington, D.C.
    The task force met, pursuant to notice, at 12 p.m., via 
Webex, Hon. Bill Foster [chairman of the task force] presiding.
    Members present: Representatives Foster, Casten, Pressley, 
Adams, Garcia of Texas, Auchincloss; Gonzalez of Ohio, 
Loudermilk, Budd, and Taylor.
    Chairman Foster. The Task Force on Artificial Intelligence 
will come to order.
    Without objection, the Chair is authorized to declare a 
recess of the task force at any time. Also, without objection, 
members of the full Financial Services Committee who are not 
members of this task force are authorized to participate in 
today's hearing.
    As a reminder, I ask all Members to keep themselves muted 
when they are not being recognized by the Chair. The staff has 
been instructed not to mute Members, except when a Member is 
not being recognized by the Chair and there is inadvertent 
background noise.
    Members are also reminded that they may only participate in 
one remote proceeding at a time. If you are participating 
today, please keep your camera on, and if you choose to attend 
a different remote proceeding, please turn your camera off.
    Today's hearing is entitled, ``Beyond I, Robot: Ethics, 
Artificial Intelligence, and the Digital Age.''
    I now recognize myself for 4 minutes to give an opening 
statement.
    Thank you, everyone, for joining us today at a time when 
the power and perils of artificial intelligence (AI) are very 
much on people's minds. Each generation has its own cautionary 
tales about AI. Recent big-screen adaptations--The Matrix, 
Terminator, and Tron--echo the episodes of the old 1960s Star 
Trek starring William Shatner as Captain James D. Kirk of the 
Starship Enterprise, and those episodes themselves were taken 
from the short stories of Isaac Asimov, Arthur Clarke, and all 
of the old masters of 1950s sci-fi pulp magazines. And 
parenthetically, I should offer our congratulations that today, 
William Shatner was able to boldly go into suborbital near 
space where X-15 pilots have been boldly going since the late 
1950s. But I digress. Asimov's classic, I, Robot, showed us 
what can happen when we deploy technology or AI without fully 
comprehending its consequences.
    There is an ancient joke in AI that I first heard as an 
undergraduate back in the 1970s about an all-powerful AI that 
was given a simple command: Maximize paperclip production. It 
thought about it for a moment and then began killing off all 
humans on Earth because humans interfere with paperclip 
production.
    Now, it may have taken us 50 years, but we are kind of 
there. Facebook's AI was given the simple command, ``maximize 
Facebook's profits,'' whereupon they thought for a moment and 
then began killing off all rational political debate in our 
country because that interferes with Facebook's profits. And 
the situations with social media in Myanmar and around the 
world are even uglier and more deadly.
    In previous hearings of this task force, we have looked at 
the biases and unexpected side effects of using AI in financial 
services and housing. We have also looked at the implications 
of artificial intelligence's voracious appetite for personal 
data and the implications for privacy, at technological 
approaches to maximally preserve privacy while retaining AI's 
effectiveness, and the importance of secure digital identity.
    In this hearing, we are going to take a closer look at the 
frameworks for developing, monitoring, and recognizing AI to 
ensure that the technology we develop and deploy will be of 
overall benefit to society.
    In past hearings, we have examined instances of algorithmic 
bias that have produced discriminatory effects in the lending 
space. We have seen facial recognition technology that is far 
less effective at identifying minorities correctly, despite the 
fact that the developers of these tools did not include a 
discriminatory line of code in their products. So, we clearly 
cannot allow technology to treat humans differently based on 
race and appearance, unless, of course, perhaps we are 
explicitly correcting for past unjust biases, which brings up a 
set of issues that my father struggled with as a civil rights 
lawyer back in the 1950s, and continues with us today. We have 
to understand whether we should hold AI to standards that are 
higher than we would expect of an ordinary human-based 
decision-making process.
    As we start defining frameworks for developing and 
performance-testing AI, it seems possible that we are starting 
to place requirements on AI that are more strict than we would 
ever place on human decision-makers. For example, most of our 
witnesses today have advocated for defining minimum diversity 
standards for the training datasets for AI, but we have never 
considered requiring that a human bank officer would have a 
minimum number of friends of different races or protected 
classes, even though it might arguably result in more fair 
decision-making. And we may already be seeing the positive 
results of holding AI to higher standards than humans with the 
recent reports that fintech apps were apparently more effective 
than human-based banks in issuing Paycheck Protection Program 
(PPP) loans to minority customers.
    As policymakers, we also have to understand to what extent 
we should concentrate on so-called black-box testing that only 
focuses on the inputs and outputs from opaque neural networks 
and other decision-making algorithms, or whether we should 
expect ourselves and the public to receive and to understand a 
detailed explanation of what goes on under the hood. So, there 
is a lot to be examined here. It is my hope that in this 
dialogue, we will discover which frameworks exist and which 
should be created or fleshed out to ensure that AI is working 
effectively and safely for everyone.
    And the Chair will now recognize the ranking member of the 
task force, Mr. Gonzalez of Ohio, for 5 minutes for an opening 
statement.
    Mr. Gonzalez of Ohio. Thank you, Chairman Foster, for your 
leadership on ethics in AI and for convening today's hearing, 
and I also thank our witnesses for being here. It is vital that 
Congress continues to consider how we can best promote 
innovative advancement in the private sector while also 
ensuring that AI is both transparent and ethical. Today's 
hearing provides an opportunity to hear directly from industry 
experts and stakeholders on the importance of this topic.
    A few months ago, the task force held a similar hearing 
examining how human-centered AI can address systemic racism. 
One of our witnesses at that hearing, Professor Rayid Ghani of 
Carnegie Mellon University, testified that algorithms 
themselves are neither inherently biased or unbiased, but work 
by analyzing past data and making generalizations about future 
outcomes. I believe that these discussions on bias and 
algorithms are important to have. We must acknowledge and 
recognize these technologies at times are not perfect due to 
the inherent nature of a technology created by humans. It is 
vital, though, that we do not take steps backwards by 
overregulating this industry, which may have a chilling effect 
on the deployment of these technologies.
    If there are problems with AI and algorithms, we should not 
abandon our push to innovate and move forward. It is through 
further innovation that we are likely going to be able to fix 
these issues and to improve the technology. As Chairman Foster 
recognized, we have seen the benefits in the disbursement of 
PPP loans. I think that is an important thing for us to keep in 
mind as we continue forward.
    We should also continue to work with the experts in 
industry in order to move forward in a bipartisan way that both 
celebrates technical advancements and ensures that there is 
transparency and fairness through the use of artificial 
intelligence. There have been multiple efforts in the 
government and the private sector to address this issue, and we 
have seen tremendous advances not only in AI technology, but in 
efforts to address bias in algorithms internally. There is 
recognition of a business incentive to have transparent 
algorithms that are fair and ethical.
    Beyond the obvious concerns of ethics and transparency, I 
am also looking forward to learning more today from our 
witnesses about ways that we can strengthen data transparency 
for families, and consider reforms that would protect our 
children from being targeted by harmful algorithms. As the 
financial internet and the traditional internet merge--and we 
have seen recently-reported social media companies, like 
TikTok, employing algorithms that promote inappropriate content 
to young users--I think it is extremely troubling and extremely 
timely that we start to discuss these things. An AI-powered 
world where parents have no control over what content or 
products are being fed to their kids, no transparency around 
the algorithms that are funneling the content, and no control 
over the underlying data itself is not an ideal outcome.
    In summary, AI has great promise to innovate industries 
like the financial services sector, but there are still 
opportunities to improve. I look forward to hearing from our 
witnesses today how Congress should be thinking about this 
balance, and I yield back.
    Chairman Foster. Thank you.
    The Chair will now recognize the Chair of the full 
Financial Services Committee, the gentlewoman from California, 
Chairwoman Waters, for 1 minute.
    [No response.]
    Chairman Foster. It is my understanding that she is not 
able to make it right now, so we will move on.
    Today, we welcome the testimony of our distinguished 
witnesses: Ms. Meredith Broussard, an associate professor at 
the Arthur L. Carter Journalism Institute of New York 
University; Ms. Miriam Vogel, the president and CEO of EqualAI; 
Ms. Meg King, the director of the Science and Technology 
Innovation Program at the Wilson Center; Mr. Jeffery Yong, 
principal advisor at the Financial Stability Institute of the 
Bank for International Settlements; and Mr. Aaron Cooper, the 
vice president for global policy at BSA--The Software Alliance.
    Witnesses are reminded that their oral testimony will be 
limited to 5 minutes. You should be able to see a timer on your 
screen which indicates how much time you have left. I would ask 
you to be mindful of the timer, and quickly wrap up your 
testimony once the time has expired, so that we can be 
respectful of both the witnesses' and the members' time.
    And without objection, your written statements will be made 
a part of the record.
    Ms. Broussard, you are now recognized for 5 minutes to give 
an oral presentation of your testimony.

STATEMENT OF MEREDITH BROUSSARD, ASSOCIATE PROFESSOR, ARTHUR L. 
       CARTER JOURNALISM INSTITUTE OF NEW YORK UNIVERSITY

    Ms. Broussard. Thank you. Chairman Foster, members of the 
task force, thank you for hosting this important hearing and 
for giving me the opportunity to testify. My name is Meredith 
Broussard. I am a professor at NYU, the research director at 
the NYU Alliance for Public Interest Technology, and author of 
the book, ``Artificial Unintelligence: How Computers 
Misunderstand the World.'' In my written testimony, I explore a 
practical vision for recognizing AI, and in my short time, I'll 
talk about AI generally as well as discrimination algorithmic 
auditing and regulatory sandboxes.
    The first thing I want to say is that AI is not what we see 
in Hollywood. There is no robot apocalypse coming. There is no 
singularity. We do not need to prepare for artificial general 
intelligence because these things are imaginary. What is real 
is that AI is math, very complicated and beautiful math. 
Machine learning, the most popular kind of AI, is a poorly-
chosen term because it suggests that there is a brain or 
sentience inside the computer. There is not. When we do machine 
learning, we take a large set of historical data and instruct 
the computer to create a model based on patterns and values in 
that dataset. The model can then be used to predict or make 
decisions based on past data. The more data you put in, the 
more precise your predictions will become. However, all 
historical datasets have bias. For example, if you feed in data 
on who has gotten a mortgage in the past in the United States 
and ask the computer to make similar decisions in the future, 
you will get an AI that offers mortgages to more White people 
than people of color.
    AI needs to be regulated because it has all of the flaws of 
any human process, plus some. My own regulatory vision begins 
with frameworks, high-level governance models that guide a 
company's use of AI and data. A company can make sure its 
frameworks are implemented by performing regular algorithmic 
audits, ideally using a regulatory sandbox. The process could 
be monitored by regulators using tools we already have, namely 
compliance processes inside existing regulatory agencies. 
Agencies and companies might decide which AIs need to be 
regulated and monitored by looking at the user and the context. 
Automated license plate readers used at toll booths might be a 
reasonable use of AI. Automated license plate readers used by 
police as dragnet surveillance might be an unreasonable use of 
AI.
    An open secret in the AI world is everyone knows that these 
systems discriminate. Any conversation about a robot apocalypse 
is a deliberate distraction from the harms that AI systems are 
causing today. Right now, AI is preventing people from getting 
mortgages. A recent investigation by The Markup found that 
nationally, loan applicants of color were 40 to 80 percent more 
likely to be turned down by mortgage approval algorithms as 
compared to their White counterparts.
    When the International Baccalaureate used AI to assign 
student grades during the pandemic, high-achieving, low-income 
students received terrible grades, which prevented them from 
getting college credits that would allow them to graduate early 
and incur less student loan debt.
    AI is used to generate secret predictive consumer scores, 
like health risk scores or identity and fraud scores. It is 
likely that Black, Indigenous, and People of Color (BIPOC) 
people are systematically disadvantaged by most of these 
scoring systems. The EU's proposed AI regulation calls for 
categorizing AI into high and low risk, which I think is a good 
strategy. A low-risk use might be using facial recognition to 
unlock your phone. A high-risk use might be the police using 
facial recognition on real-time surveillance video feeds. 
Facial recognition has been shown to consistently misidentify 
people with darker skin; people of color are at a high risk of 
being harmed by facial recognition when it is used in policing. 
In the U.S., we can register and audit high-risk AI to ensure 
that AI is not harming citizens.
    The process for uncovering algorithmic bias is called 
algorithmic auditing. ORCAA, a company I consult with, performs 
bespoke algorithmic audits in context, asking how an algorithm 
might fail and for whom. Audits can show how an algorithm might 
be racist, or sexist, or ableist, or might discriminate 
illegally. Once we identify a problem, it can be addressed, or 
the algorithm can be discarded. There is also software like 
Parity, or Aequitas, or AI Fairness 360, that can evaluate 
algorithms for 1 of 21 known kinds of mathematical fairness.
    I'm enthusiastic about the potential of a regulatory 
sandbox, a protected environment where companies can test their 
algorithms for bias. If and when the bias is discovered, they 
can then address the issue in their code and rerun the test 
until they're in compliance with acceptable thresholds. I'm 
currently working with ORCAA to develop a regulatory sandbox 
prototype. In our version, regulators would also have a limited 
view inside the sandbox to see if the company is auditing their 
algorithms for bias and fixing the problems that they find 
without the companies revealing any trade secrets.
    Thank you for the opportunity to testify today on this 
important topic, and I welcome your questions.
    [The prepared statement of Ms. Broussard can be found on 
page 26 of the appendix.]
    Chairman Foster. Thank you, Ms. Broussard, and I have to 
say I am fascinated with the thought of figuring out for which 
of the 21 definitions of fairness you will be advocating.
    Ms. Vogel, you are now recognized for 5 minutes to give an 
oral presentation of your testimony.

     STATEMENT OF MIRIAM VOGEL, PRESIDENT AND CEO, EQUALAI

    Ms. Vogel. Chairman Foster, Ranking Member Gonzalez, and 
distinguished members of the task force, thank you for 
conducting this important hearing and for the opportunity to 
provide this testimony. My name is Miriam Vogel. I'm president 
and CEO of EqualAI, a nonprofit founded to reduce unconscious 
bias in AI systems. At EqualAI, we are AI net positive. We 
believe AI is and will be a powerful tool to advance our lives, 
economy, and opportunities to thrive, but only if we're 
vigilant to ensure that the AI we use does not perpetuate and 
mass produce historical and new forms of bias and 
discrimination.
    We're at a critical juncture. AI is increasingly becoming 
an important part of our daily lives, but decades of progress 
made and lives lost to promote equal opportunity can be 
unwritten in a few lines of code. And the perpetrators of this 
disparity may not even realize the harm they're causing. For 
instance, we can see our country's long history of housing 
discrimination now replicated at scale in mortgage approval 
algorithms that determine creditworthiness using proxies for 
race and class.
    At EqualAI, we try to help avoid such harms by supporting 
three main stakeholders: companies; policymakers; and lawyers. 
Often, our work involves helping organizations understand they 
are effectively AI companies because they are now using AI in 
pivotal functions. As such, they need an AI governance plan, 
particularly given that with AI, as you know, key assessments 
occur behind the proverbial black box where inputs and 
operations are generally unknown to the end user.
    As discussed in your past hearings, implicit bias 
infiltrates AI in a variety of ways. Our operating thesis is 
that bias can embed in each of the human touch points 
throughout the AI lifecycle, from the ideation phase deciding 
what the problem is you even want to solve with AI, to the 
design, data collection, development, testing, and monitoring 
phases. But we are optimistic and we think each touch point is 
also an opportunity to identify and eliminate harmful biases. 
As such, risk management should occur at each stage of the AI 
lifecycle.
    There are several helpful frameworks to identify and reduce 
harms in the AI systems, including GAO's, GSA's, and the 
important efforts under way at NIST. The EqualAI framework 
offers five pillars to consider when establishing responsible 
governance, including, first, invest in the pipeline. Our basis 
tenet is that AI needs to be created by and for a broader 
cross-section of our population. There are several 
organizations promoting diversity in tech effectively right 
now--AINU, AI4All, and several others--and we need to support 
these efforts.
    Second, hire and promote people with your values. To create 
and sustain a diverse workplace and produce better AI, AI 
programs used in H.R. functions should be checked routinely to 
ensure they're in sync with the values of your organization and 
our country.
    Third, evaluate your data. The more we know about datasets, 
the safer we are as a society. We encourage identifying gaps in 
data so that they can be rectified and, at a minimum, clarified 
for end users.
    Fourth, test your AI. AI should be checked for bias on a 
routine basis. As you know, AI constantly iterates and learns 
new patterns as it is fed new data. On our website, 
EqualAI.org, we offer a checklist to help get you started, and 
we offer additional steps to take in our written testimony. We 
highly recommend as well the use of routine audits.
    And fifth, redefine the team. An often-overlooked 
opportunity to reduce bias in AI is by creating testing teams 
that include those underrepresented in the AI creation and the 
underlying datasets.
    There are numerous ways that Congress can play a key role 
in ensuring more effective, inclusive AI. Several are listed in 
our testimony. A few include, one, Congress can reinforce the 
applicability of laws prohibiting discrimination to AI-
supported determinations. Two, Congress can lead by example, 
create a framework for AI procurement, acquisition, and 
development and ask vendors if they do the same.
    Three, incentivize investment in the future of work. Like 
all transformative technologies, AI will eliminate jobs, but it 
will also open up opportunities. To lead in the AI revolution, 
safeguard our economy, and support greater prosperity among 
more communities, we should re-skill our workforce by 
understanding what jobs are likely to emerge, and offering 
incentives for upscaling and loan forgiveness for those 
committing to a term in public service.
    Finally, we enthusiastically support the bill of rights put 
forward by the White House Office of Science and Technology 
Policy last week to level-set expectations and inform the 
public about their rights.
    In conclusion, we believe we're at a critical juncture to 
ensure that AI is built by and for a broader cross-section of 
our population. It's not only the right thing to do; a strong 
U.S. economy and our leadership depend on it. Thank you for the 
opportunity to testify, and I look forward to your questions.
    [The prepared statement of Ms. Vogel can be found on page 
79 of the appendix.]
    Chairman Foster. Thank you, Ms. Vogel, and I echo your 
enthusiasm for the White House's effort to come up with an AI 
bill of rights, though I don't believe I have seen even a draft 
of it at this point.
    Ms. King, you are now recognized for 5 minutes to give an 
oral presentation of your testimony.

    STATEMENT OF MEG KING, DIRECTOR, SCIENCE AND TECHNOLOGY 
             INNOVATION PROGRAM, THE WILSON CENTER

    Ms. King. Thank you, Chairman Foster, Ranking Member 
Gonzalez, and members of the AI Task Force for inviting me to 
testify today. My name is Meg King. I'm the director of the 
Science and Technology Innovation Program at the Wilson Center, 
a nonpartisan think tank created by Congress nearly 60 years 
ago. My program studies the policy opportunities and challenges 
of emerging technologies, and investigates methods to foster 
more open science and to build serious games. We also offer 
hands-on training programs, called the Technology Labs, to 
Legislative and Executive Branch staff on a variety of issues, 
including artificial intelligence. Next month, we will offer a 
series of individual trainings on AI for Members as well.
    As with any technological evolution, the benefits of AI 
come with associated costs and risks. Focusing only on the 
benefits misses the nuances of the potentials and pitfalls of 
this advance. To help the task force understand the risks to 
any industry and, in particular, the financial services 
industry, I will focus my remarks on the nature of AI 
generally, to understand the environment in which creation is 
occurring.
    Today, there aren't significant incentives for the private 
sector to include ethics directly in the development process. 
At the current pace of advancement, companies cannot afford to 
develop slowly, or a competitor might be able to bring a 
similar product to market faster. Largely due to consumer trust 
concerns, international organizations, regions, and private 
companies have all begun to issue ethical frameworks for AI. 
Most are very vague principles, as you mentioned, Chairman 
Foster, with little guidance as to application.
    Two that this committee should pay close attention to are 
the Organisation for Economic Co-operation and Development 
(OECD), and the European Commission (EC). In addition to their 
principles on AI, the OECD is developing process and technical 
guidelines ranging from pinpointing new research to making 
available software advances which will become part of a 
publicly-available interactive tool for developers and 
policymakers alike. As Ms. Broussard noted, European regulators 
announced a risk-based plan this year to establish transparency 
requirements, including biometric identification and chatbots. 
Chatbots, in particular, are expected to have a significant 
impact on the financial services industry as many companies see 
value in customer service process improvement and the prospect 
of gaining more insight into customer needs in order to sell 
more financial products.
    As regulators ask developers more questions about the 
ethics of their AI systems, they have the potential to slow the 
process, which could cost businesses money. However, if ethical 
concerns are identified too late in the development process, 
companies could face considerable financial loss if not 
addressed properly. No ethical AI framework should be static, 
as AI systems will continue to evolve, as will our interaction 
with them. Key components, however, should be consistent, and 
that, specifically for the financial sector, should include 
explainability, data inputs, testing, and system life cycle. 
Explainable Artificial Intelligence (XAI) is the method to ask 
questions about the outcomes of AI systems and how they achieve 
them. It helps developers and policymakers identify problems 
and failures, possible sources of bias, and helps users access 
explanations. There are a number of techniques available to 
carry out XAI, as well as open source tools, which make these 
techniques more accessible.
    In the financial sector, XAI will become critical as 
predictive models increasingly perform calculations during live 
transactions, for example, to evaluate risk or the opportunity 
of offering a financial product or specific transaction to a 
customer. Establishing a clear process for XAI will be critical 
to address flaws identified in these real-time systems and 
should be an area of focus for the committee.
    Additionally, producing policies on how these systems will 
be used and in what context will be helpful. Without context, 
data pulled from a mix of public/private records can produce 
inaccurate results and discriminate in access to financial 
products. One of the near-term questions this committee should 
ask about systems you will encounter in your oversight is how 
the COVID-19 pandemic experience is factored into these 
systems. One promising possibility to address the data input 
problem might be to synthesize artificial financial data to 
correct for inaccurate or biased historical data. Just today, a 
major tech company announced acquisition of a synthetic data 
startup. Watch this space.
    While quality assurance is part of most development 
processes, there are currently no enforceable standards for 
testing AI systems, and, therefore, testing is uneven at best. 
Additionally, users are far removed from AI system developers. 
Carefully assessing the growing field of Machine Learning 
Operations Tools (MLOps) and machine learning operations and 
identifying ways the committee can participate in that process 
will be useful.
    AI breaks, often in unpredictable ways, at unpredictable 
times. Participants in the Wilson Center's AI Lab have seen AI 
function spectacularly using a deep learning language model to 
produce the first-ever AI-drafted legislation, as well as fail 
when a particular image loaded into a publicly-available 
generative adversarial network produced a distorted picture of 
a monster rather than a human. Lab learners also study why 
accuracy levels matter, as they use a toy supply chain 
optimization model to predict whether and why a package will 
arrive on time and how to improve the prediction by changing 
the variables used, such as product weight and length of 
purchase.
    Beyond mistakes, some AI systems carry out tasks in a way 
humans never would. Many examples exist of scenarios producing 
results developers didn't intend, like a vacuum cleaner 
injecting collected dust so it can collect even more, and a 
racing boat in a digital game looping in place to collect 
points instead of winning the race. Anyone who has played the 
game, ``20 Questions'' understands this problem. Unless you ask 
exactly the right question, you won't get the right answer.
    As more and more AI systems are built and distributed 
widely with varying levels of user expertise, this problem will 
continue. Establishing a framework of ethics for the 
development, distribution, and deployment of AI systems will 
help spot potential problems and provide more trust in them. 
Thank you.
    [The prepared statement of Ms. King can be found on page 74 
of the appendix.]
    Chairman Foster. Thank you, Ms. King.
    Mr. Yong, you are now recognized for 5 minutes to give an 
oral presentation of your testimony.

    STATEMENT OF JEFFERY YONG, PRINCIPAL ADVISOR, FINANCIAL 
    STABILITY INSTITUTE, BANK FOR INTERNATIONAL SETTLEMENTS

    Mr. Yong. Thank you. Good afternoon, Chairman Foster, 
Ranking Member Gonzalez, and distinguished members of the task 
force. My name is Jeffery Yong, and I'm the principal advisor 
at the Financial Stability Institute of the Bank for 
International Settlements, or the BIS. I offer my remarks today 
entirely in my personal capacity based on a publication that I 
co-authored with my colleague, Jermy Prenio, entitled, ``FSI 
Insights No. 35: Humans keeping AI in check--emerging 
regulatory expectations in the financial sector.'' And the 
views expressed in that paper are our own and do not 
necessarily represent those of the BIS, its members, or the 
Basel ommittees. I'm appearing before the task force 
voluntarily. I would like to note that my statements here today 
are similarly my personal views, and they do not represent the 
official views of the BIS, its members, or the Basel 
Committees.
    By way of background, the Financial Stability Institute 
(FSI) is a unit within the BIS with a mandate to support 
implementation of global regulatory standards and sound 
supervisory practices by central banks and financial sectors, 
supervisory and regulatory authorities worldwide. One of the 
ways the FSI carries out this mandate is through its policy 
implementation work which involves publishing FSI Insights 
papers. The papers aim to contribute to international 
discussions on a range of contemporary, regulatory, and 
supervisory policy issues and implementation challenges faced 
by financial sector authorities.
    In preparing FSI Insight No. 35, my co-author and I found 
that regulatory expectations on the use of AI in financial 
services were at a nascent stage. Accordingly, we drafted a 
paper with four key objectives: to identify emerging common 
financial regulatory themes around AI governance; to assess how 
similar or different these common regulatory themes are viewed 
in the context of AI vis-a-vis that of traditional financial 
models; to explore how existing international financial 
regulatory standards may be applied in the context of AI 
governance; and to examine challenges in implementing the 
common regulatory themes.
    To this end, we can select a section of policy documents on 
AI governance issued by financial authorities or groups formed 
by them as well as other cross-industry AI governance guidance 
that applies to the financial sector. In total, we examined 19 
policy documents issued by 16 regional and national authorities 
and 2 international organizations. Most of these documents are 
either discussion papers or high-level principles, which 
underscores the fact that financial regulatory thinking in this 
area is at a very early stage.
    We identified five common themes that recur in policy 
documents that we examined: reliability; accountability; 
transparency; fairness; and ethics.
    On the theme of reliability, emerging supervisory 
expectations for AI and traditional models appear to be 
similar. What seems to be different is that the reliability of 
AI models is viewed from the perspective of avoiding harm to 
data subjects or consumers, for example, through 
discrimination.
    On the theme of accountability, it is acknowledged that 
both traditional and AI models require human intervention. In 
the case of AI, however, this requirement is motivated by the 
need to make sure that decisions based on AI models do not 
result in unfair or unethical outcomes. Moreover, external 
accountability is emphasized in the case of an AI model, so 
that data subjects are aware of AI-driven decisions and have 
channels for recourse and moving on transparency.
    Supervisory expectations related to explainability and 
auditability are similar for AI and traditional models. 
However, expectations or external disclosure are unique to AI 
models. This refers to expectations that firms using AI models 
should make data subjects aware of AI-driven decisions that 
impact them, including how their data is being used.
    On the theme of fairness, there's a distinct and strong 
emphasis in emerging supervisory expectations on this aspect in 
the case of AI models. Fairness is commonly described in the 
documents as avoiding discriminatory outcomes.
    Similarly, on ethics, as a distinct and strong emphasis on 
this aspect of AI models, ethics expectations are broader than 
fairness, and relate to ascertaining that consumers will not be 
exploited or harmed.
    Now, given the similarities of the themes between AI and 
traditional models, existing financial literacy standards that 
govern the use of traditional models may be applied in the 
context of AI. However, there may be scope to do more in 
defining financial regulatory expectations related to fairness 
and ethics. The use of AI in the financial sector presents 
certain challenges, and the key challenge relates to the level 
of complexity and lack of explainability. Given these 
challenges, one way to approach this is to consider a tailored 
and coordinated regulatory policy approach, meaning 
differentiating potential and conduct treatment depending on 
the risk that the AI models pose.
    With that, I conclude. Thank you.
    [The prepared statement of Mr. Yong can be found on page 88 
of the appendix.]
    Chairman Foster. Thank you, Mr. Yong.
    Mr. Cooper, you are now recognized for 5 minutes to give an 
oral presentation of your testimony.

STATEMENT OF AARON COOPER, VICE PRESIDENT, GLOBAL POLICY, BSA--
                     THE SOFTWARE ALLIANCE

    Mr. Cooper. Thank you very much. Good afternoon, Chairman 
Foster, Ranking Member Gonzalez, and members of the AI Task 
Force. My name is Aaron Cooper. I'm vice president of global 
policy for BSA-The Software Alliance. BSA is the leading 
advocate for the global enterprise software industry. Our 
members are at the forefront of developing cutting-edge, data-
driven services that have a significant impact on U.S. job 
creation. I commend the task force for convening today's 
important hearing, and I thank you for the opportunity to 
testify.
    Enterprise software services, including AI, are 
accelerating digital transformation in every sector of the 
economy, and BSA members are on the leading edge, providing 
businesses with the trusted tools they need to leverage the 
benefits of AI. In fact, last year, software supported more 
than 12.5 million jobs outside the tech sector. AI is not just 
about robots, self-driving vehicles, or social media. It's used 
by businesses of all sizes to improve their competitiveness. 
It's the power and industrial design that improves 
manufacturing performance and reduces environmental impact. 
It's the tool that streamlines transportation and logistics 
operations, and that detects cyberattacks and improves H.R. 
operations. In the financial services industry, AI is being 
used to reduce the risk of fraudulent transactions and deliver 
a better customer relations experience.
    While the adoption of AI can unquestionably be a force for 
good, it can also create real risks if not developed and 
deployed responsibly. We commend the task force for its work to 
explore domestic and international AI frameworks because they 
play a critical role in ensuring the responsible use of AI.
    As you explore these issues, we offer our perspective on a 
risk management approach to bias which has been a particular 
focus for BSA, and that we hope will also inform the broader 
conversation. For BSA members, earning trust and confidence in 
AI and other software services they develop is crucial, so 
confronting the risk of bias is a priority. We, therefore, set 
out to develop concrete steps companies can take to guard 
against this. The resulting framework is included in full in my 
written testimony. It is built on three key elements: impact 
assessments; risk mitigation practices; and organizational 
accountability modeled on NIST frameworks, which includes more 
than 50 actionable diagnostic statements for performing impact 
assessments that identify risks of bias and corresponding best 
practices for mitigating those risks.
    Among the unique features of the BSA framework is that it 
recognizes that these steps need to be followed at all stages 
of the AI life cycle: design; development; and deploymentt. 
Also, different businesses will have different roles throughout 
the life cycle, so risk management responsibilities will need 
to be tailored to a company's role. Who's developing the 
algorithm? Who's collecting the data, training the model, and 
ultimately deploying the system? What does that all mean in 
practice?
    A few examples. First, when designing an AI system, 
companies should clearly define the intended use and what the 
system is optimized to predict, identify who may be impacted, 
and, if the risk of bias is present, document efforts to 
mitigate that risk. They should examine data that will be used 
to train the model to ensure that it's representative and not 
tainted by historical biases.
    Second, at the development stage, they should document 
choices made in selecting features for the model and document 
how the model was tested.
    Third, at the deployment phase, they should document the 
process for monitoring the data and model and maintain a 
feedback mechanism to enable consumers to report concerns.
    And to be clear, at every phase, it is important for 
companies to have a team that brings diverse perspectives and 
background, which can help anticipate the needs and concerns of 
people who may be affected by AI in order to identify potential 
sources of bias. Bias is only one of the important ethical 
considerations for responsible AI, but addressing it is 
critical. And the risk management approach we recommend in this 
context can be tailored to address other ethical 
considerations.
    In conclusion, digital transformation across industry 
sectors is creating jobs and improving our lives, but industry, 
civil society, and academia must work together with Congress 
and other policymakers on guidelines and laws which will ensure 
that companies act responsibly in how they develop and deploy 
AI. We appreciate the task force's strong focus on these issues 
and hope that our framework on confronting bias will contribute 
meaningfully to this discussion. Thank you for the opportunity 
to testify, and I look forward to your questions.
    [The prepared statement of Mr. Cooper can be found on page 
33 of the appendix.]
    Chairman Foster. Thank you, Mr. Cooper.
    I will now recognize myself for 5 minutes for questions.
    My first general question is to Ms. King or Mr. Cooper, 
whomever wants to field it. How much should we expect of AI, 
and, in particular, should we be asking more of AI than we do 
of humans? For AI-driven cars, should the standard be that you 
should outperform humans on average or in all circumstances? 
With similar things regarding fairness as well, in general, is 
it reasonable? Are there real dangers in using human-based 
decision-making as the standard of fairness and safety for what 
is acceptable in AI?
    Mr. Cooper. I am happy to jump in. I will give one example 
and a way of thinking about this, that things which are illegal 
in the physical world should be illegal in the digital world 
when we use AI or any other system. In the realm of 
discrimination, for instance, a practice that would be 
discriminatory if a person did it, should still be 
discriminatory and illegal if an AI system does it.
    And I think what we are finding in other areas is that AI 
is increasingly being used, both in everyday features of what 
companies are doing as they go through a digital transformation 
but will also increasingly be used in more high-risk areas. And 
in those situations, we need to make sure that there is a 
proper impact assessment so we know, whether it is related to 
bias or safety or another issue, that companies are thinking 
through what those implications are going to be and taking 
steps to mitigate the risks.
    Ms. Vogel. I am happy to answer if you would like, as well. 
I think the answer is, honestly, we don't know. Certain systems 
are designed very narrowly right now, and that is because AI 
outperforms humans in those systems. But in others, with 
context, with heuristics, the shortcuts that we use as humans 
don't perform well. And as one of the Wilson Center machine-
learning researchers who has written a paper about it reminds 
us regularly, autonomous agents optimize the room, floor, and 
function that we give them. So, until we can improve AI to a 
level where we feel comfortable that moves beyond that narrow 
capability, I don't think we have an answer yet about how to 
think about the consequences, but also the opportunities. They 
are just so varied across so many sectors at this point.
    Chairman Foster. Are there any other comments on the 
deployment decision that has to be made here? You need some 
sort of absolute standard that this is good enough for this 
application, and it is something we are going to have to pace 
because that is probably, at best, the level at which Congress 
will be specific about how these decisions should be set up.
    Another thing that I know we all struggle with is this 
question of black-box testing versus expecting that the public 
should have a detailed understanding of what goes on inside. If 
you look at the trouble that we have had trying to convince 
people to get vaccinated, it is not clear that it helps to tell 
them the details of how the immune system in the human works. 
And, that may make it better. It may make it worse. We had this 
situation very recently where we apparently fired a football 
coach, not, to my knowledge, for mistreating athletes, but for 
what went on in his private decision-making.
    Should we accept or reject algorithms based only on their 
inputs and outputs, or should we actually demand to look inside 
at all of the intermediate levels of the neural network and see 
if there are objectionable racist nodes in them? What is your 
thinking on that, the black box versus detail, and also how to 
convey that to the public? Anyone? Should I just pick someone 
at random?
    Ms. King. I am happy to jump in.
    Chairman Foster. Okay.
    Ms. King. I think it is a great question, and I think that 
the challenge also includes that even if you show the general 
public all of the nodes, it wouldn't necessarily make sense. In 
this case, you wouldn't know which are prioritized, so there is 
a balance to strike. There are intellectual property issues, 
privacy issues, and so forth. So, just opening the box, first 
of all, would be somewhat technically challenging as well as 
legally. Compliance testing, as you say, can be a helpful way 
to demonstrate compliance, safety, and legality. And to the 
extent that more data becomes available, we don't need to 
expect everyone in the general public to understand it. We have 
seen so many cases already where the limited publicly-available 
data has been used for important findings, like with the 
UnitedHealth Care Optum case, where scientists, researchers 
were able to go backwards, look at the algorithms, and identify 
biases in the algorithms.
    Chairman Foster. Thank you. And when you figure all this 
stuff out, let us know.
    The Chair will now recognize the ranking member of the task 
force, Representative Gonzalez of Ohio, for 5 minutes.
    Mr. Gonzalez of Ohio. Thank you, Chairman Foster, and thank 
you to our witnesses. Ms. Vogel, I am going to try to pick up 
where Chairman Foster just left off on compliance testing. Is 
it fair to say, based on the response you gave, that the right 
way to think about this is more to look at the outputs of AI as 
opposed to opening up the hood and trying to understand each 
individual node and network? Is that the right way to think 
about it?
    Ms. Vogel. My view is that it should be a balance. I think 
that, absolutely, the outputs are indicative. They are helpful 
to look at now because so much of the AI is already deployed, 
and so we are not at the design stages for so much of the AI in 
common use, and for that understanding, what the outputs are is 
important and helpful. I think there are elements of what is 
under the hood that would be helpful and important to 
understand, particularly when you are talking about AI used in 
a pivotal sensitive function. So, I don't think it is one or 
the other. I do think it is a balance, but no matter what I 
think, the outputs are very important to be testing and 
watching.
    Mr. Gonzalez of Ohio. No, I appreciate that. I think, as I 
mentioned in my opening statement, it is encouraging that we 
are seeing some AI algorithms produce better results, 
significantly better results in some instances, from a bias 
standpoint. And obviously, the hope is to understand what it is 
that they are doing right and doing more of that or making that 
more transparent, and then helping foster a more collaborative 
innovation environment.
    Ms. King, I want to shift to you, and I want to ask about 
transparency in AI algorithms. Also, as I mentioned in my 
opening statement, the use of algorithms in social media has 
had a detrimental effect on young users, which, as a parent, I 
find extremely problematic. Do you think that more can be done 
to ensure parents have additional transparency about their 
child's data being collected by these apps or their own, and 
how can we strike the right balance and the right line between 
encouraging innovation, managing problematic algorithms, and 
providing data sovereignty to users?
    Ms. King. Thank you, sir. As a parent as well, that is the 
one thing that terrifies me, is my children getting access to 
these capabilities. And unfortunately, I wish there was one 
significant answer that could fix it, but it is going to be a 
constant ever-moving group of things that we have to do. And 
explainability is significant in that problem because, as 
Miriam just said, we have to understand what the outputs are, 
but we have to understand enough about how we are getting there 
to be able to make informed decisions about whether there is 
too much data that is being collected or whether there isn't, 
and there are many ways to do this. One of the most popular is 
this local and interpretable model agnostic explanation. This 
was created by the University of Washington to try to see what 
happens inside, so model agnostic. It should be across models. 
That is one of many ways to do that.
    Another piece to this is, as you just mentioned, that AI 
can be positive and there are some impressive advances 
happening right now in synthetic data that can both hopefully 
correct for some of those historical data biases, but also give 
just a better picture of the people who are going to be 
impacted by the system being created. Now, of course, you have 
to understand what that synthetic data looks like, so you 
probably should have a wide group of interdisciplinary experts 
assessing that to make sure you are not missing something. But 
I think it is a combination of constantly reviewing the 
outcomes, constantly trying to take at least a sample of 
explainability across some of the most important, as Europeans 
are suggesting, high-risk models, and then also assessing kind 
of what are the new technical capabilities we are developing 
now that can help address this problem.
    Mr. Gonzalez of Ohio. Thank you. Shifting to Mr. Cooper for 
a second with my final minute, I want to ask you about BSA's AI 
risk management framework included in your testimony. One 
aspect that seems to be of importance is that a one-size-fits-
all framework will not work for small companies and startups. I 
completely agree. Could you elaborate on why flexibility in any 
framework is important for fostering innovation?
    Mr. Cooper. Sure. Thank you very much. I think it is 
important to have flexibility in a variety of ways of achieving 
a desired outcome for a number of reasons, including that not 
all systems are going to be used for the same purposes. The 
algorithm and the data that is used to determine what shows our 
kids watch or what videos our kids watch online is one form of 
algorithm and one use of AI. But there is also database 
management, and customer relations management tools, and 
farmers who use AI in order to improve crop yield, and one set 
of regulations across-the-board isn't going to be able to be 
flexible enough to address the range of different use cases for 
AI.
    Mr. Gonzalez of Ohio. Yes, thank you. And I also think it 
is almost always the case that the higher the regulatory 
burden, the more you entrench incumbents, and the less 
innovation you have at the startup level as the regulatory 
burden is just too high to even contemplate a startup. So with 
that, I thank the witnesses, and I yield back.
    Chairman Foster. Thank you. The Chair now recognizes Ms. 
Pressley of Massachusetts for 5 minutes.
    Ms. Pressley. Thank you, Chairman Foster, for convening 
this important hearing, and to our witnesses for joining us 
here today. Certainly, systemic racial discrimination is 
widespread in the financial services industry. The damage of 
redlining, banking deserts, and employment discrimination has 
never been fully redressed or repaired in America, and all of 
the data supports those facts. Today, mortgage lenders deny 
Black applicants at a rate 80 percent higher than White 
applicants, and payday lenders continue to target low-income 
people of color, charging 500-percent interest even in the 
midst of a pandemic. Many believe artificial intelligence 
presents an opportunity to make the allocation of credit and 
risk fairer and more inclusive. However, AI technology and 
machine learning can easily go in the other direction, 
exacerbating existing bias, and reinforcing bias credit 
allocation, and making discrimination in lending even harder to 
prove.
    Cases of racial bias in AI are well-documented and have 
impacted everything from mortgage loans and tenant screening to 
student loans. The deciding factor between whether the 
technology has a positive or damaging impact could be its 
developers.
    Ms. Broussard, who is writing the algorithms that are being 
used to make important financial decisions, like 
creditworthiness? Do the teams writing these algorithms 
generally reflect the diversity of people in America?
    Ms. Broussard. Generally, these teams do not represent the 
diversity of people in America. Silicon Valley and its 
developers tend to be very pale, male, and Yale. Compared to 
overall private industry, the EEOC found that the high-tech 
sector employed a larger share of Whites, Asian Americans, and 
men, and a smaller share of African Americans, Hispanics, and 
women.
    Ms. Pressley. Thank you. In fact, in February 2020, the 
Financial Services Committee released a report on the diversity 
of America's largest banks, which found that banks were largely 
undiversified at all levels and departments. Those data points 
you offered there support that.
    Ms. Broussard, one more question, will this lack of 
diversity affect AI used by financial institutions? What is the 
impact?
    Ms. Broussard. Absolutely, yes, there is an impact. The 
problem is that people tend to embed their unconscious biases 
in the technology that they create. When we have a small and 
homogeneous group of people creating AI, that AI then gets the 
collective blind spots of the community of people who are 
creating the algorithms. So, the more diversity you have in the 
room when you are creating algorithms, the better the algorithm 
is going to be for the wide variety of people who live in 
America.
    Ms. Pressley. Thank you, Ms. Broussard. And just to further 
unpack the impact of that on people's lives, there are many 
different facets that AI companies developing these 
technologies really need to consider, from, as we are speaking 
to here, who is developing the algorithms to the AI's impact on 
job loss. A recent report from the World Economic Forum 
predicted that by 2025, the next wave of automation amplified 
by the pandemic will disrupt 85 million jobs globally.
    Ms. Vogel, what role should independent auditors play in 
helping to assess the human cost and the ethical implications 
of AI technologies so that both developers and the public can 
fully understand the ethical impacts these technologies have 
for actual consumers?
    Ms. Vogel. Thank you for that question. It is a really 
important point. We do have this growing body of experts--in 
fact, we have one on this very panel--who do this important 
work of checking in, of taking the temperature and 
understanding where these gaps are. I think it is really 
important that we build our reliance and our infrastructure to 
support more algorithmic auditing because these are the people 
who will tell us if the AI doing what we expect it to. Are we 
discriminating? Are we creating opportunity? For whom will this 
fail, and how do we create more opportunity through our 
algorithms?
    Ms. Pressley. Thank you. I agree. Frequent and independent 
audits are critical. AI-supported recommendations in the 
financial services industry directly impact people's lives and 
economic opportunities, and yet the algorithms used are trained 
on data that is rife with imbalance and discrimination. So as 
we do the work deliberately to enact long-overdue economic 
justice, we can't allow the AI industry to create new problems 
and to compound these already persistent and deeply-embedded 
inequities. Thank you, and I yield back.
    Chairman Foster. Thank you.
    The Chair now recognizes Mr. Loudermilk from Georgia for 5 
minutes.
    Mr. Loudermilk. Thank you, Mr. Chairman. One important 
thing to keep in mind as we discuss AI is the types of bias we 
need to eliminate and the types of bias that we actually want 
to keep. Sometimes, the main purpose of an algorithm is to be 
biased. For example, in loan underwriting, algorithms are 
generally used to distinguish between who can pay back a loan 
and who is not able to pay back a loan. With that in mind, we 
must work toward eliminating the types of bias that have no 
place in our financial system, such as the bias based on race 
or gender or any other factor like that.
    One important way of doing that is when an algorithm is 
being built, there should be a thorough record-keeping of 
everything that is added to the algorithm. That way, if bias is 
suspected, companies and regulators can see everything that 
went into the algorithm and see where the bias may be coming 
from. I think this would help make it where algorithms are not 
a black box and where the outputs cannot be explained, but you 
would have a record where you could see where the problems may 
be.
    Mr. Cooper, your organization's framework for AI best 
practices recommends maintaining records of the data that is 
used to train AI models. I agree with that. Expanding on that, 
do you believe that maintaining thorough records of all of the 
inputs used to build an algorithm can be useful for identifying 
the source of any unwanted bias?
    Mr. Cooper. Thank you very much, Congressman. Yes, I think 
it goes even beyond what the data is. I think that there is a 
whole set of considerations that companies need to go through 
to figure out whether there is a high risk that the AI system, 
as it is intended to be deployed or as it is being deployed, 
may have consequential impacts on people. And the decision-
making about what those risks are and what the right mitigation 
practices are, how the data was tested, what historical biases 
may or may not be present in them, keeping a record of that as 
part of a risk management framework, can be both useful in 
order to make sure that companies are not putting systems out 
into the world or using systems that are going to lead to 
discriminatory results. But it could also be useful, as you 
say, after the fact, if there is a problem, to go back and 
audit and find out why it happened and make sure it doesn't 
happen again.
    Chairman Foster. Representative Loudermilk, I believe you 
are muted.
    Mr. Loudermilk. I don't know why it is cutting off like 
that. Can you hear me now?
    Chairman Foster. Yes, we can.
    Mr. Loudermilk. Okay.
    Chairman Foster. And feel free to exceed your time by 40 
seconds.
    Mr. Loudermilk. Thank you. Mr. Yong, in your testimony, you 
discuss the importance of accountability and transparency in 
AI. Can maintaining records when algorithms are being built 
help achieve those goals?
    Mr. Yong. Yes. Accountability is very important, especially 
when it comes to AI, and without record-keeping, there is no 
transparency. In our testimony, we mentioned that transparency 
is a prerequisite to enabling financial institutions to meet 
the other general AI governance principles. And if the AI model 
is not transparent, then it is very difficult to assess whether 
it is reliable, whether it is sound, and whether there is bias 
involved. So definitely, record-keeping is a prerequisite to 
meeting this accountability and general principles.
    Mr. Loudermilk. Thank you for that. Ms. King, you have 
written that policymakers must govern AI in a way that is 
flexible enough to adapt when technology inevitably changes, 
wand we know that it continually changes in today's 
environment. I agree with that, and I believe that is needed to 
have an environment that fosters innovation. With that in mind, 
how can policymakers ensure that AI governance remains 
flexible, but robust, at the same time?
    Ms. King. Thank you, sir, for that question, and I think it 
is all about having a set of goals that are measurable and 
achievable. One of them is, how can you explain these systems, 
and, again, the complexity here is really because these systems 
cross so many sectors. Yours obviously is financial services, 
so you have some very specific use cases to identify, which is 
helpful, but you need to be able to explain those specific use 
cases. You need to ask a lot of questions, and those questions 
will change, too, but the big ones are why was it developed. 
What are the [inaudible]? How does it possibly fail because it 
is the unexpected failure that is really a lot of the problem 
here.
    And then again, how can we correct those errors and report 
them? So if you can kind of have those four ways of addressing 
this challenge and work with companies, and you work with both 
governments who are buying this and the companies who are 
producing this to have sort of the four methods of regularly 
checking that you are getting, you are producing what you want, 
you are getting what you want out of it, and that it is not 
discriminatory, then I think that is a flexible way to move 
forward.
    And I think the sandbox concept that Ms. Broussard has 
suggested is also very helpful, because while records are great 
and it is easy for us to say, let's keep records here, if you 
have ever taken a look at the code behind some of these systems 
and how often it changes as you shift weights, it gets pretty 
complicated pretty quickly. So, the more you can have these 
kinds of places where companies and organizations can feel safe 
testing is going to be critical going forward well.
    Mr. Loudermilk. Thank you. And, Mr. Chairman, I yield back 
the balance of my time.
    Chairman Foster. Thank you,.
    The Chair now recognizes Representative Casten for 5 
minutes.
    Mr. Casten. Thank you so much. I think Mr. Loudermilk 
really hit it on the head with the transparency question, and I 
want to follow on that, but I want to specifically get to the 
auditability issue, and I think you alluded to this, Ms. King. 
It is one thing to be able to see the code. It is something 
else completely to be able to understand the code. And I say 
this as someone who, before I came to Congress, ran a utility. 
And my biggest risk was predicting revenue--did it vary with 
the weather, did it vary with economic conditions--and I built 
a genetic algorithm that figured all that stuff out. I have no 
idea how it worked, but it was amazingly effective and it made 
our investors much happier because we could predict our 
revenue.
    That is trivial. It is not at all implausible for me to 
imagine that we get to a world where an investment fund has 
figured out from looking at global data that there is about to 
be a massive human rights abuse committed, and it is shorting 
the affected businesses and properties, right? That would be 
deeply unethical, and if we understood it, it would be a 
problem, but it is totally possible that we could never 
actually understand that and saying that is what it is doing.
    So my question is, and I think all of you can answer this, 
but I am going to start with Ms. King, just because I see you 
nodding so vociferously, what is the best regulatory practice 
for ensuring that these algorithms remain auditable, and 
ensuring that they apply to everyone in the system, because 
presumably, as soon as some subset of people agree to have 
auditable algorithms, people who violate that might have an 
investing edge, whether that is a bad actor in our country or a 
foreign actor who wishes us harm. What is that standard, both 
domestically and internationally? How would you recommend we 
think about that?
    Ms. King. Thank you, sir. I will take a quick stab at it, 
and I am going to use a hypothetical because I think it is 
always helpful to have it. Yours is very complicated, and I am 
not going to try and explain your very impressive example. At 
the Wilson Center, in one of our trainings, we use a supply 
chain prediction model. Will it or won't it arrive? Will the 
USPS deliver a package on time? And you have a series of 
variables. You have product wait. You have the month that it 
was ordered. You have things that you probably, as a consumer, 
wouldn't think matter, but about 10 different variables. And as 
you play with the model and you change the variables, you 
change the weights--how much weight do we give to a particular 
variable or not--you understand more why the prediction you get 
comes out, and then you can kind of take that and you can go 
back through the system and check it.
    I would say you need a couple of standards. One is not 
going to work, unfortunately, but a couple of standards that 
have that sort of ability to use a couple of methods, probably 
a model agnostic method, if it is possible, to go back and just 
understand, at least at a strategic level. You may not be able, 
as you know very well, to go and explain the whole thing, but a 
confidence level and then explainability that you could 
achieve. So, you are looking for some sort of confidence trust 
level and some sort of agnostic model verification, and you are 
also looking to make sure as you are going through that 
process, that if you are going to have regulators as part of 
this conversation, you have a number of regulators across 
sectors. As your example points out, you can't just have 
financial services regulators. You are going to have to have 
others from other parts of the government at the table because 
of these unexpected outcomes.
    Mr. Casten. If I could, though, that approach you described 
works where there is a finite number of known inputs. If you 
are using sort of a neural network model that, for all 
practical purposes, has an infinite number of inputs, I don't 
know how you audit that at some level of complexity. To follow 
on from that, and I am sure this varies market to market, is 
there any good analysis? Is there a percentage of algorithmic 
trading or algorithmic investing, wherever it sits, that we 
really don't want to have more than X percent because now the 
algorithms are responding to algorithms? Is there a robust 
mathematical way to think about that? And maybe it is different 
for housing credit decisions than it is for equities 
investments or something else. But is there some robust way to 
think about that so that we don't sort of unwittingly introduce 
too much volatility into the system? And if any of the other 
witnesses want to chime in on this, you would be welcomed for 
your thoughts as well.
    Ms. Vogel. I can speak to auditing algorithms. What we want 
to do is, we don't want to think about auditing all algorithms 
to the same standard. We want to think about auditing 
algorithms in context because the context matters a lot. So, we 
do need to keep track of inputs. We do need explainability. We 
do need to enforce real-world laws inside algorithms. We do you 
need to be aware of bias in, bias out. And so to your point 
about thresholds, the acceptable thresholds would be determined 
based on the context.
    Mr. Casten. I see I am out of time, but I would welcome 
further thoughts offline from any of the witnesses, and I yield 
back.
    Chairman Foster. Thank you.
    The Chair now recognizes Ms. Adams from North Carolina for 
5 minutes.
    Ms. Adams. Thank you, Chairman Foster. And thank you, 
Ranking Member Gonzalez and Chairwoman Waters, for this hearing 
today. And to our witnesses, thank you for your testimony.
    Professor Broussard, in your testimony, you noted that all 
historical data sets have bias, and that AI needs to be 
regulated as soon as possible because it has all of the flaws 
of any human process plus more. You also cite in your testimony 
the potential impact of bias in AI to students and consumers of 
lower socioeconomic status, such as when the International 
Baccalaureate used AI to assign grades to students, to their 
detriment. So, building off of what my colleague, Ms. Pressley, 
discussed, would you tell us more about what happened in these 
scenarios?
    Ms. Broussard. Sure. Thank you for that question. The 
International Baccalaureate (IB) example is a situation where, 
because of the pandemic, the International Baccalaureate exams 
were canceled, and the IB decided to use an algorithm to assign 
imaginary grades to real students, which had disastrous 
consequences, because the inputs to the algorithm were things 
like a school's performance in the past. We know that the 
economic divide is particularly profound when it comes to 
America's schools, and so the students at the poor schools were 
predicted to do poorly, and the students at the rich schools 
were predicted to do well. We have a racial divide there. Who 
are the students at poor schools? They are mostly Black and 
Brown students. Who are the students at rich schools? Well, 
they are mostly White students. So, the algorithm made very 
predictable decisions that disadvantaged Black and Brown and 
poor students. This is what happens most of the time with 
algorithmic decisions.
    Ms. Adams. Would you explain what algorithmic auditing is, 
and how we can encourage public and private entities to adapt 
it as a best practice?
    Ms. Broussard. Thank you. Yes. Algorithmic auditing, as I 
mentioned in my testimony, is something that I do with a 
company called O'Neil Risk Consulting and Algorithmic Auditing, 
Inc. (ORCAA). What we do is we look at an algorithm and we ask, 
who could this algorithm negatively affect, and we look at the 
inputs to the algorithm. We do look at the code. We act as an 
information fiduciary, so we keep everything extremely private. 
We look at the outputs and we do mathematical and statistical 
analysis as necessary in order to figure out what is going on 
in the algorithm. Once you actually figure out where the 
algorithm is going wrong, you can fix it, but in a lot of 
industries now, people are pretending that there is nothing 
wrong. For example, Ms. Vogel mentioned before the Optum case. 
There is also the case of the Apple card, where a man was 
offered a credit limit that was about 10 times higher than his 
wife, even though they shared all of their finances.
    Companies are pretending that they don't collect 
information like race in order to make decisions. But, on the 
other hand, if you are using a factor, like a ZIP Code, that is 
an input to your algorithm, then, actually we have enough 
residential segregation in the United States that if you are 
using a ZIP Code, you are actually using race as a proxy. So, 
there are--
    Ms. Adams. Thank you.
    Ms. Broussard. Thank you.
    Ms. Adams. I want to move on, if I can, quickly.
    Ms. Broussard. Sure.
    Ms. Adams. Ms. Vogel, I was happy to see that part of your 
recommendations related to diversifying the AI field, including 
supporting Historically Black Colleges and Universities 
(HBCUs). Specifically, what shoud Congress be doing to ensure 
that HBCU and Minority Serving Institution (MSI) students are 
able to participate in the AI revolution that is currently 
underway?
    Ms. Vogel. Thank you for that question. We strongly believe 
that we need AI to be built by and for a broader cross-section 
of the population, both so that more can benefit from the AI, 
so that more can benefit from the economic support that comes 
from it, but also so that our AI is better. So, we need to make 
sure that we support HBCUs and MSIs to ensure that their 
students are part of this current AI revolution that is 
undeway. We know that HBCUs produce nearly 20 percent of all 
Black graduates, 25 percent of Black graduates who earned 
degrees in the disciplines of STEM technology, science, 
engineering, and math, and we need to make sure that we have 
all hands on deck. We can't afford to not bring all of these 
students into the AI revolution. Industry is depending on their 
participation.
    Ms. Adams. Thank you, ma'am. I think I am out of time. Mr. 
Chairman, I am going to yield back, but thank you very much for 
your response.
    Chairman Foster. Thank you.
    The Chair now recognizes Mr. Auchincloss of Massachusetts 
for 5 minutes.
    Mr. Auchincloss. Thank you, Mr. Chairman. I would like to 
talk about two specific applications of algorithms that have 
been and are front and center these days, and really invite the 
panel to weigh in on one or both of them. The first is the use 
of algorithms in hiring. A number of organizations, some from 
the center-left, some from the left, and some from the center-
right, have all converged that there are somewhere between 25 
to 30 million ``hidden workers'' in the United States, people 
who could be employed, who, under the right conditions, want to 
be employed. And yet, we are not tapping into their 
productivity, and they are not getting to realize their fullest 
aspirations.
    That is obviously a multifaceted problem, but one element 
of it is algorithms that some of the biggest companies are 
using, something like 75 to 80 percent of Fortune 100s, for 
example, in how they sort resumes that get put forward. They 
are screening out resumes that have discontinuity in 
employment. They are screening out resumes of formerly-
incarcerated individuals. They are screening out resumes that 
don't have a college degree, even for jobs that don't require a 
college degree. I welcome input from the panel on this first 
application, kind of the state of play right now in these 
resume-screening algorithms, and what can be done to improve 
them, and whether they have any role at all going forward?
    Ms. Broussard. I can offer that my colleague, Hilke 
Schellmann, has been writing about these topics, and has done 
some really excellent work in the MIT Technology Review, that 
is an in-depth review on what is going on with hiring 
algorithms.
    Mr. Cooper. Yes. This is one of the reasons why we need a 
risk management framework for when there is going to be an AI 
system that has a highly-consequential impact on somebody's 
lifec, so making sure that there is a thought process that is 
auditable about what the factors are that are considered in 
determining what resume is going to go where is a good example 
of something where we need to make sure that there is a 
thought-through and documented impact assessment, and then 
steps taken to mitigate risk.
    Mr. Auchincloss. And I would just add also that this should 
be a triple-line win for everybody. Companies don't want to be 
screening out high-quality workers for esoteric reasons. They 
are struggling for employees, as we speak. And as a society, we 
want people to be working and contributing, and people 
themselves want meaningful work. So, I would hope that this can 
be an area of actual bipartisan work going forward on how we 
encourage the private sector to be more thoughtful about their 
use of these hiring algorithms and the other elements of this 
challenge of hidden workers.
    The second area that I want to dig into and invite the 
panel to speak on is about what many of us have been reading 
about these last 2 weeks, which is Facebook's algorithm. The 
whistleblower addressing the Senate exposited that while she 
did not think Section 230 should be revised for user-generated 
content, she did think that Facebook's algorithm itself should 
be subject to liability laws. And I would welcome input from 
any of the panelists here about how that might be applicable in 
terms of Facebook, in particular, but really any social media's 
algorithm, whether that should be subject to regulation itself?
    Mr. Cooper. I am happy to jump in again. We don't represent 
Facebook, but I would say that I think it is important to make 
sure that where you have particular high risk in the way an 
algorithm is working--in this case, feeding certain videos or 
certain social media feeds to certain people, particularly 
where it has to do with children--is a high risk, and we need 
to make sure that the decision-making process is appropriate. 
And there is a combination of a regulatory aspect of that and 
also just good practices internally to make sure that there is 
organizational accountability so that when decisions are made, 
that there is somebody at a senior level who signs off on those 
decisions, and that there is documentation of why certain 
choices were made.
    Mr. Auchincloss. Yes. I can see why it would be challenging 
to try to unpick liability for an algorithm that was put into 
place, how you can draw causality directly, and yet part of me 
thinks that we have to answer that question. We have to wrestle 
with that problem because, otherwise, we are going to be in a 
place where I think organizations will be distancing themselves 
from accountability instead of embracing it by being able to 
point towards these black box algorithms and say that they are 
just part of part of their toolkit, and you can never pay cause 
and effect. I think we need to reject that explanation and hold 
companies liable for the algorithms that they choose to use.
    I yield back my time, Mr. Chairman.
    Chairman Foster. Thank you, and I would like to thank our 
witnesses for their testimony today.
    The Chair notes that some Members may have additional 
questions for these witnesses, which they may wish to submit in 
writing. Without objection, the hearing record will remain open 
for 5 legislative days for Members to submit written questions 
to these witnesses and to place their responses in the record. 
Also, without objection, Members will have 5 legislative days 
to submit extraneous materials to the Chair for inclusion in 
the record.
    This hearing is now adjourned.
    [Whereupon, at 1:16 p.m., the hearing was adjourned.]

                            A P P E N D I X

                            October 13, 2021
                            
                           
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]  

                               [all]