IBM
Watson: The Beginning of Cognitive
Computing
Eric
W. Wasiolek
Colorado
Technical University
Introduction
The IBM product and project of
Watson will be presented here. Watson is
an extreme innovation that takes a large step towards making artificial
intelligence viable. Watson is more of a
long-term engineering project which has gone through several iterations and
will continue to develop more fully in the future. Several IBM Watson products have been released
to the market, with applications of the artificial intelligence engine to several
knowledge domains to yield expert systems in those domains. Watson has expert system products in oncology
diagnosis, tax preparation, financial applications (credit analysis), insurance
coverage for medical procedures, and basic research, with planned business
applications, and more applications in healthcare, finance, contact centers,
government, and chemical industries. The
reason Watson was chosen is that it is an innovative project which has a long
and grand future ahead of it.
This paper starts with a general
description of the socio-technical plan for Watson. This is followed by a general description of
Watson which includes a model of Watson's question and answer algorithm. Next, the general features of Watson are
provided, along with some current limitations.
Discussed here are Watson's deep language processing, its machine
learning, and its inference engine. Why
Watson is needed, and its purpose is explained.
The forces that shape Watson, supporting forces and detracting forces
are discussed. Social, technical, and
economic forces among others are considered.
The Method for developing Watson is presented along with the particular
steps used in the Delphi method to arrive at a true socio-technical
innovation. To aid in understanding the
socio-technical plan for Watson are graphical models, which indicate how Watson
works and in particular the question and answer algorithm. The analytical plan, or how Watson is
evaluated is given. Watson's social
impact or anticipated results are considered. Finally, in conclusion, how the Watson
technology is diffused in the community is covered, with examples from its
Oncology Diagnosis, Tax, and Jeopardy expert systems. Finally, areas of future development are
covered.
Socio-Technical Plan for Watson
The
socio-technical plan for Watson is to present a technology that allows humans
to use human language (now English) to communicate with the computer. Questions can be asked in natural language to
Watson, or passages (text) written in English can be presented to it. This is a big socio-technical
innovation. This system will understand
English sentences presented by a user and find answers to the users' questions
and report them back in an English sentence manner. This is quite a bit more sophisticated than
keyword search currently done by sites like Google. Here even English sentences if presented to
Google are analyzed regarding only their keyword constituents, without an
understanding of the sentence, and then web pages that match that keyword or
those keyword combinations are found.
There is some ranking in that the highest matches are presented
first. Watson is much more
sophisticated. It forms an understanding of the meaning of a
sentence before searching for an answer.
More than just answering questions, Watson can use its inference engine
to generate new knowledge about a topic to help a user perform a task. Watson enables superior collaboration between
people and technology in a workplace.
Watson functions as an expert advisor in particular knowledge
domains. These include diagnostic and
treatment answers and advice for oncology doctors, or tax preparation advice
for tax preparers, or credit analysis
and valuation advice for insurance agents and people in the finance industry,
among many other future such collaborations.
General Description of Watson
Watson is a cognitive computing
product. You do not program Watson; you
use human language to interact with it.
It does human type reasoning or inferences using human language, in this case, English (High, 2012). It understands English by using a deep
language processing set of algorithms.
It answers questions, analyzes text, and provides advice. It does English language understanding, it
conducts inferences and can derive new information from existing texts. It uses context to understand and doesn't
just operate literally on language. It continually
improves its performance by using a machine learning algorithm to modify the
knowledge base as it searches for answers to questions or analyzes novel texts.
Figure 1: How Watson Answers a Question. Source:
High, 2012, The Era of Cognitive Systems by IBM Redbooks.
In the next iteration of this paper
in Unit 5, the exact algorithm of how Watson answers a question will be
presented. Here just the general idea is
presented. Watson analyzes the
statement, decomposes it into its key parts and uses contextual information to
understand it. Once there is a deep
understanding of the question hypotheses as to the correct answer are
constructed and compared to information in the knowledge base, also known as the
corpus. Each possible answer is evaluated,
and a confidence score is attached to it.
These scores are ranked, and the answer with the highest score is
presented along with its confidence number.
Scope and Features of Watson
The Watson Cognitive System has many
interesting features. First, it does
Deep Language Processing. Most language
understanding systems to date are shallow language processing. At the simplest level, search engines use
mere combinations of keywords to find relevant web pages. There isn't much understanding here, it's more of a word
match, without any real understanding of the meaning of sentences. Shallow language processing uses specific
well-formed rules based upon keyword combinations to try to understand
language. It does not exhibit general
language understanding. It is not
adaptable. It is precise but not
accurate. Precision is literal
understanding. Non-literal aspects of
language are not understood, like metaphors, idioms, expression, etc...
Accuracy means understanding language when it is imprecise. Most human understanding is accurate but
imprecise, i.e., we understand what someone is saying regardless of the
particular form or syntax and to some extent exact words that are being
used. We understand things that are
non-literal, metaphoric, and expressive.
Deep language understanding attempts to model how humans actually
understand language. Deep language
understanding understands different sentences that have the same meaning or the
same sentence that has different meanings in different contexts. It uses contextual information from a passage
or from the knowledge base to perform language understanding. It provides not a narrow but a broad and
adaptable understanding of language.
Like human language understanding, it is imprecise yet accurate.
A second feature that Watson has is
machine learning. I.e., in each
encounter where it answers a question, it stores the link between the question
and the answer in its knowledge base and hence becomes more skilled at answering
questions over time. In other words, it
learns, and by learning, it continually improves its performance.
A third feature is that Watson has
an inference engine. I.e., it can infer
new information from existing texts by following chains of reasoning. Inference engines have been around for a
while. But natural language inference
engines are somewhat new (Modha et al., 2011). There are close
connections between Philosophy and Artificial Intelligence in Computing. First of all, logic is a subject taught in
Philosophy Departments. Even if you
study symbolic mathematical logic, you do not take the subject in the Math
department; you take it in the
Philosophy department. The reason logic
is a part of Philosophy and not Math is because logic underlies all mathematics
and all language (natural and artificial, i.e., human language and programming
languages). It applies to and underlies
both. When things are general, i.e., are
cross-departmental, they are philosophical.
When I was at Berkeley, to get my Philosophy degree I had to take a year
of symbolic mathematical logic (Fregean Quantificational Predicate
Calculus). One way that an inference
engine works is by syllogisms (invented by Aristotle). A syllogism is a simple line of reasoning
that allows one to proceed from premises to a conclusion. A simple example is "Socrates is a
man," "all men are mortal," (the two premises) therefore (the conclusion) "Socrates is
mortal." Stated symbolically, S is
M; All M are m; therefore, S is m. In symbolic form the syllogism is universal,
i.e., it is always true as long as one preserves the logical form. Therefore, a carp is a fish, all fish have
gills, therefore a carp has gills. This
is the same syllogism or logic. The
premises are based upon evidence. But
the reasoning is unassailable. If it is
true that all fish have gills and a carp is a fish, then by the logical form of
the syllogism, it is also true that carp have gills. You can see how syllogisms can be used to
create chains or lines of reasoning, to take a set of premises and derive
conclusions. All and Some are
existential quantifiers (hence Quantificational predicate calculus). There are many logic laws of inference that
Watson would use: modus ponens, modus
tollens, hypothetical syllogism, disjunctive syllogism, constructive dilemma,
absorption, simplification, conjunction, addition, De Morgan's theorem,
material equivalence, and many others.
There are also logical fallacies which an inference engine would avoid,
but a human may not. This is how Watson
does computerized inference or has reasoning algorithms based upon these
logical inference laws.
As an aside, computers would not
exist without logic. Computers would not
exist with Aristotle who invented logic.
Computers are logical machines both in terms of their hardware and
software. Hardware is based upon Boolean
logic; in fact, for every circuit, there is a Boolean logic equation. Software is based upon logic. Anyone who programs knows that programming is
all logic (not mathematics, logic).
Programs can be written that do mathematical tasks as well as linguistic
tasks. Since logic underlies both
language and math, computer use logic, particularly binary logic. Programming languages have keywords and
syntax. If one makes a syntactical error
in programming, a program doesn't compile.
But one can create a perfectly syntactically correct program which
compiles and yet doesn't work right, doesn't do what it's supposed to do, i.e.,
there are logic errors. Also,
programming languages have logic features, like "if ... then ...
else" and "or" and "and" and "not," etc... Computers wouldn't exist without logic, and
artificial intelligence in particular wouldn't exist without logic and
Philosophy.
There are many cognitive
capabilities that stem from the ability of a program to infer. These include among others the ability of the
program (Watson) to infer new knowledge by running inference algorithms on its
existing and growing knowledge base.
Watson's Current Limitations
There are several limitations to the
current Watson project and products.
First, it only speaks English. It
is a Natural Language Processing and Understanding System, but many of its
features are based on English specific idioms, expression, and oddities. It is perfectly possible to adapt this NLP
technology to other languages, this just hasn't been done yet. The goal was to completely master English
language understanding first.
Watson has many other current
limitations to a full-fledged cognitive system.
These capabilities will be added at some later date. These include the ability to sense and
perceive, the ability to have a dialog or do planning, the ability to have
foresight or do knowledge extrapolation and the creation of new knowledge. These capabilities will be discussed in the
next paper on Watson, which will be described as areas of future research.
Why Watson is Needed: Its Purpose
Greatly needed are computing systems
that can talk to us, that can understand human language and answer questions
and carry out tasks stated in English. This
is true in many domains. And as Watson
can be adapted to answer questions and provide advice in many expert domains,
it can be of great help to doctors doing diagnoses, to insurers or bankers
doing credit analysis or valuation, to people doing their taxes, and to people
doing many other complex tasks that need questions answered or advice.
Greatly needed are computer systems
that can reason and can deduce new information and therefore make discoveries
by inferring from current expert knowledge bases.
Supporting Forces (Amenable
Issues)
A supporting force for Watson is
customer usage. As Watson does machine
learning and improves its performance and answers to questions after each
iteration of a question answer scenario or analysis of text, customer usage is helping Watson improve and become a better
product. This is especially true in
expert knowledge domains. As, for
example, doctors use Watson oncology diagnosis expert systems module, the
performance of this module improves with usage.
I.e., the accuracy of oncology diagnoses and suggested treatments improves
over usage. These improvements can be
incorporated back into the knowledge base of the product for improved
performance in future releases.
Another supporting force is that IBM
simply has the best computer researchers in the world. The talent to develop a highly innovative
artificial intelligence product exists more at IBM than any other company, but
perhaps also at certain universities.
Challenging Forces (Difficult
Aspects)
There are several areas of
difficulty ahead in Watson's development.
Particularly is the ability to continue to generate a general reasoning
machine. The problem with current expert
systems and the current expert systems market is that most expert systems are
highly specific and their expertise is only applicable to a particular
knowledge domain. Watson is trying to do
something revolutionary: to create a general reasoning machine. In adapting Watson to various particular
knowledge domains, as IBM is doing as it releases more and more Watson-based
expert systems, IBM has to be careful to not sacrifice development of the
general reasoning algorithms.
Another area of difficulty may be
the jargon or technical vocabulary in various knowledge domains. IBM has built a deep language processing and
understanding system, but to be effective in many domains, it has to understand
the vocabulary of the particular domain.
Consider something like medical vocabulary in the oncology diagnosis
domain. Clearly, there are many specific
medical and biological terms that need to be understood. However this problem is helped by the fact
that where vocabulary is more precise then accuracy is easier to implement.
Supporting Watson in other languages
to have greater appeal to the global market may involve some difficulty. A deep language understanding system
understand idioms, expressions, and metaphors, and other non-literal aspects of
language. However, these non-literal
aspects of language are highly language specific. French has its own unique idioms,
expressions, and metaphors which are nothing like those of English (i.e., a
literal translation is impossible). This
is true of all languages. Hence, an
attempt to globalize the product by supporting other languages can be done in a
deep language processing system, but it is difficult.
Methods: The Delphi Method
The
team at IBM Watson is clearly using a Delphi method, where the technology is
guided by a group of expert IBM researchers.
As this is highly innovative technology and involves knowledge of
specialized areas like automated reasoning and inferences, machine learning,
and natural language processing and understanding systems, a group of experts
is best able to plan the technological evolution of this product and plan new
features and new spin-off products. For
the core technology the IBM researchers would be the experts, but in the case
of developing expert system modules, experts in a particular knowledge domain
may be utilized as well.
Method Steps
In
the Delphi Group Decision Making Method, a panel of experts on a topic are
assembled and answer questions from a questionnaire in two or more rounds about
Watson. There is a facilitator for the
group. The facilitator takes the input
from the experts each round and provides an anonymous summary of their
technological strategies (Dalkey
and Helmer, 1963). The experts
review the summary and revise their strategies each round. The successive inputs and revisions allow the
group to converge towards a single strategy or similar strategy. The experts may come from inside or outside
the organization, but they are all expected to be knowledgeable in their field
and position and on the topic of the group decision. The process remains anonymous and even who
made or had the most influence on the final decision is kept anonymous (Okoli and Pawlowski, 2004).
The facilitator sends out the questionnaires and collects the responses and
summarizes the responses.
Through
the Delphi method technological forecasting for Watson's future is performed. The experts give their opinions on the
technology and on when the technology is expected to mature and be on the
market. The group essentially is
attempting to forecast the technological future. This group decision-making method is used for
other types of forecasting and decision making as well.
Visual Models
As a visual model, the question and
answer algorithm of IBM Watson is here provided. Watson starts with a question in English as
input. The question is then parsed, and
its major features are extracted from it.
Deep language processing is done at this point. This may involve such tasks as extracting the
spatial and temporal features of a statement to understand it. Watson then generates a set of hypotheses or
possible answers to the question. This
is done by accessing information from the knowledge base. If the question is in an expert domain, this
is the expert domain knowledge base that is consulted. The inference engine or the reasoning
algorithms are used to compare the potential answers to the question. The point is to see which answer is best
inferred from the question. Remember
that the inference engine acts by applying logical syllogisms to the
information in the knowledgebase. The
types of syllogisms, or a subset of them, were discussed in the section above
on the inference engine. The various
reasoning algorithms are then scored or ranked as to how good they are. The scores are weighed against a statistical
model to determine a confidence score (the confidence level in the potential
answer). The answer with the highest
confidence level is the answer given, along with its confidence score.
Figure 2. The Question and Answer Algorithm in IBM
Watson (My Diagram).
Analytical Plan: How is Watson Evaluate
Watson is evaluated according to the
quality of answers and advice it gives, and also how well it understands the
questions that are asked of it. When it
answers a question, is the answer correct?
A good test of Watson's general knowledge was in its Jeopardy Game Show
module. On the Jeopardy game show, Watson demonstrated a superb performance on
answering the Jeopardy general knowledge questions, outscoring all human
contestants. Watson was evaluated on how
well it did in the game, i.e., how many Jeopardy questions Watson got correct,
and finally Watson's overall game score.
More is said on this below in the conclusion to this paper. In more technical tasks, such as in expert
knowledge domains, Watson is evaluated according to how good its answers are or
how good its advice is. In the Oncology
Diagnostics module, for example, does Watson usually or always yield a correct
diagnosis? Is Watson able to generate a
reasonable or best treatment program? Is
Watson correct about its prognosis? In
the Tax module, can Watson answer tax questions? Can Watson answer tricky tax questions that a
human tax preparer may struggle with?
Does Watson demonstrate expert tax law knowledge? Can Watson find deductions and yield a lower
tax bill than a human tax preparer can?
Does Watson warn of possible tax violations in a tax return and
recommend a solution that will avoid an audit?
Over time, does Watson demonstrate expertise in the various new expert
systems modules that will be developed for it?
In general, does Watson demonstrate
a human level of understanding when asked a question? Is Watson able to answer general questions
and demonstrate general understanding?
Is Watson able to precisely answer specific technical questions in an
expert knowledge domain? Is Watson able
to deliver valuable guidance on a subject and to generate useful advice? Are Watson's responses quick, as quick as a
human or expert's responses or quicker?
Anticipated Results: The Social Impact of Change
It is anticipated that Watson and
artificial intelligence engines and expert system modules like Watson will have
a big social impact. With such
technology and products, users can view computers as colleagues or experts that
can guide their work collaboratively.
Users will view computers not as off-putting technological devices that
can only be manipulated effectively by technically knowledgeable individuals,
but instead as knowledge helpers in many domains of knowledge. This includes changes in the expectations of
users about computer technology. We
don't currently expect computer technology to understand our statements in
English, but to at best aid us with keyword matches (ala Google). A computer that can understand what we say,
no matter how we say it, and answer our questions will be viewed as a companion
or helper in our daily tasks. For people
working in particular knowledge domains, the computer may be seen as an expert,
with perhaps more expertise than anyone on staff. There will be a change in the work
environment where collaborative work is done not only with coworkers but also
with an expert computer system. Advice
will be sought not just from surrounding experts but also from an expert system
program.
Conclusion: The Diffusion of IBM Watson in Socio-Technical Environments
Since IBM Watson works in many
knowledge domains, it will here be presented how IBM Watson is diffused within
the different organizations that use the different IBM Watson Expert System
Products. Since a socio-technical system
includes both the hardware and software of the system as well as the physical
environment, e.g. the building, the social environment, i.e., the people, and
the regulatory environment, each of the application areas of IBM Watson will be
considered in how it is used and diffused in all of these components.
Medical Offices Doing
Oncology Diagnosis
With IBM Watson's Oncology Diagnosis
Expert System, the technology is diffused in an office or laboratory doing
oncology diagnosis (cancer diagnosis).
In oncology diagnosis, a doctor
or lab technician needs to start with various biological samples. One may be a biopsy of the tumorous region,
and tests on those cells to determine the nature of the cells and the
disruption that has occurred to them.
Images may be taken of the cells under a microscope. Also, blood samples may be taken from the
patient, and cells in multiple areas of the body may be biopsied and imaged to
determine if cancer has metastasized to
other areas of the body. Hence the
physical environment involves various laboratory instruments to collect the
samples and microscopes and imaging machines to determine the nature of the
cells, as well as the building that houses the laboratories and medical
offices. The social environment includes
the doctor or lab technician collecting the samples and then analyzing them
with various assays, microscopy, and imaging.
The regulatory environment may be any rules or laws that govern proper
medical procedure in such cases. All of
these environments interact with the IBM Watson Oncology Diagnosis Expert
System. Images may be fed into the
expert system for further information about what the condition of the cells are
(done by image analysis software that is part of the expert system). The social interaction with the software
involves the doctors or lab technicians feeding images and information
collected from the lab tests and biopsies into IBM Watson. The physical environment would also involve
communications hardware and software for the lab or doctors office to
communicate with the Watson Oncology Diagnosis Expert System. The laboratory technicians and doctors would
then ask Watson various questions which Watson would answer with a certain
degree of confidence, yielding possible diagnoses, probably rated in ranked
order according to confidence scores.
Watson would also provide advice to the doctors about the diagnosis,
prognosis, and possible regimens of treatment.
The Watson Oncology Diagnosis Expert System was disseminated to numerous
international medical laboratories and doctors offices.
Jeopardy Quiz Show
It is famously known that Watson
beat Jeopardy's top contestants in a televised Jeopardy Quiz Show. The quiz show follows a question and answer
format that Watson is clearly programmed
for. The questions are in English. Since Watson understands English, Watson is
able to understand the questions. Watson
then utilizes its question and answer algorithm, as presented above, to arrive
at the best possible answer to the Jeopardy question in play. In most cases, Watson had the correct answer
and produced it more quickly than any of the contestants. Hence, Watson was able to answer most of the
Jeopardy questions correctly with speed greater than the top contestants and was able to win the quiz show. It is well known that the questions on
Jeopardy are tough and require a broad and deep range of knowledge to succeed
at the game. Watson demonstrated a wide
and deep knowledge in answering the questions.
None of the questions are known in advance, and so the answers could not
be programmed in advance. Each question
was novel and presented in English to each of the contestants. Hence, IBM Watson had to utilize its question
and answer algorithm, its speed of computation, and its vast general knowledge
base to answer the questions more effectively than any of the other
players. It was a true feat of
brilliance and a coup for artificial intelligence.
In this socio-technical system, IBM
Watson provided the hardware and software and communications hardware and
software to the monitor on the quiz show.
Watson was fitted with an input audio mechanism so that it could hear
the questions and a voice synthesizer and speaker so that it could output the
buzzer and answers. The social environment included the host to the show, Alex
Trebec, and the other contestants, as well as the studio audience. The physical environment included the
television studio complete with stage, audience, and apparatae to record and
broadcast the show. The regulatory
environment included whatever the game show rules are as well as any
regulations (according to the FCC) to record and broadcast a television show to
a public audience. This particular
module of Watson does not have a wide distribution and was specifically
designed for the Jeopardy Quiz Show.
H&R Block Tax Preparation
Offices
It is generally known at this point
that H&R Block Tax Offices are now utilizing Watson's Tax Expert
System. Tax law is a particularly good
application for an Expert System. Expert
systems are better at very specific reasoning tasks with well-defined knowledge
domains and have a harder time with less specific tasks or general reasoning as
demonstrated by IBM Watson's Jeopardy engine.
The socio-technical system here includes the social environment, i.e.,
the tax preparer and their clients, the physical environment of the H&R
Block Tax Office, and the computer hardware and software at the office, notably
H&R Block's tax software, as well the hardware and software of the communications
link between the H&R Block computer and program and the IRS and the link to
the hardware and software of IBM Watson's expert system (located off-site but
accessible through a communications link).
In this case, there is a heavy regulatory environment, i.e., the U.S.
Tax Code. The tax preparer may ask
questions to Watson which answers the tax question. The tax preparer may seek tax advice from
Watson. The client's tax forms may be
presented to Watson for Watson to identify any possible regulation violations
or any possible further deductions.
These have been just a few examples
of the dissemination of Watson products to various knowledge domain specific
markets, while many new expert system products are under way increasing the
variety of socio-technical systems that Watson will address.
Areas of Future Research: What is Needed in the Future
Figure 3: Areas for Development of IBM Watson. Source: High, 2012, The Era of Cognitive Systems by
IBM Redbooks.
As a cognitive computing platform,
IBM has made much progress. But for more
robust cognitive computing new
capabilities need to be added to the on-going Watson project. In the diagram above, the darker green boxes
indicate cognitive areas that Watson already addresses. The lighter green boxes indicate future areas
of development for a full functioning cognitive system. As has been presented, Watson is a system
that takes questions, analyzes them along with contextual information, and provides
an answer or a decision. These are
covered in Cognitive Services in the diagram above. Discovery exists to some extent in the
current Watson system, as the system can use machine learning to improve its
performance after each question and answer task. This may involve modifying the knowledge
base. In terms of higher level cognitive
processes, Watson has made some progress in the areas of understanding and
discovery. Clearly, Watson is a Natural
Language Understanding system. Given an
input question or text, Watson creates an internal representation which
indicates the meaning of the sentence, even where idioms, metaphors and
non-literal language is used.
The future of IBM Watson also
includes expansion to many more knowledge domains, i.e., the building of many
more expert systems modules and knowledge bases based upon the inference
engine. Watson has a particularly good
future in acting as a cognitive analytical engine for big data (Chen,
Argentinis, and Weber, 2016).
Final Words
IBM Watson is an extremely
innovative product that is changing the way humans use computers. Watson is actually more of an on-going
engineering project which has undergone many iterations and will undergo many
more. Each rendition and use of Watson
improves its performance, as machine learning is an integral part of its
design. Watson will continue to develop
its general reasoning capabilities and will apply those capabilities to
numerous knowledge domains, making this one of the most remarkable engineering
feats of the century.
References
Chen, Y.,
Argentinis, J. E., & Weber, G. (2016). IBM Watson: how cognitive computing
can be applied to big data challenges in
life sciences research. Clinical Therapeutics, 38(4), 688- 701.
Dalkey, N.,
& Helmer, O. (1963). An experimental application of the Delphi method to
the use of experts. Management Science,
9(3), 458-467.
High, R. (2012).
The era of cognitive systems: An inside look at IBM Watson and how it works. IBM Corporation, Redbooks.
Modha, D. S., Ananthanarayanan,
R., Esser, S. K., Ndirango, A., Sherbondy, A. J., & Singh, R. (2011). Cognitive computing. Communications
of the ACM, 54(8), 62-71.
Okoli, C.,
& Pawlowski, S. D. (2004). The Delphi method as a research tool: an
example, design considerations and
applications. Information & Management, 42(1), 15-29.
No comments:
Post a Comment