How AI Large Language Models Work
April 26, 2023
Unless you’ve been living off the grid, you have likely seen the media rage over public access to open-source artificial intelligence engines like OpenAI with its Chat GPT platform or Google’s AI chatbot Bard.
Whereas OpenAI’s neural network machine learning model GPT3 was trained on large amounts of data up to June 2021, the Google Bard language prediction model continually draws its information from the internet via Google Search in real time.
ChatGPT has recently been updated with GPT-4 technology and goes beyond just producing content text and can generates digital images (see DALL.E 2 deep learning models) and web-related computer code.
Models Determine Word Probability
Large Language Models (LLMs) work by determining word probability through analysis of the data after text is fed through an algorithm that establishes the rules for context in natural language. Since LLMs train on extremely large amounts of data, linguistic engineers were not surprised by getting results that were not planned for but errantly produced by the AI engine during conversational exchanges.
With early artificial intelligence models, nuanced conversations could confuse most chatbot models and cause them to end up talking about something completely different than the topic at hand. About a year ago, Google released the Pathways Language Model called PaLM. Pathways was a collaborated effort of many Google Research teams and a major advancement in data intelligence.
Both Google’s PaLM and OpenAI’s new GPT-4 language models generate sentences with a more human level of accuracy. So much so, that the media has been “a buzz” over whether humans can tell the difference between AI-generated content and human-generated sentences. But, according to a recent study at the Human-Centered AI Institute at Stanford University, the participants ability to tell the difference between the sources was about 50% or the same as a flip of a coin.
Human Limitations and Language Model Capabilities
Language is commonly understood as a population’s instrument of thought. With that in mind, you would assume the earliest use of language had to be for one human to warn others of an impending threat. From there linguistics advanced to speaking one’s mind and on to talking it out amongst other speakers.
Although critics openly express fears of AI’s ability to teach itself how to dominate back-and-forth conversations and eventually control humankind, it is important to remember that AI’s understanding of language comes from natural language processing (NLP). So see if the concepts below sound familiar:
Everyone Makes Stuff Up – Artificial hallucinations or confabulations occur when a large language model seems confident about a response that does not seem to be justified by its training data. Just like human embellishment, the hallucination phenomenon with chatbots is still not completely understood but does exist.
Everyone Makes Mistakes – Armed with human knowledge contained in huge cloud databases, large language models can generate fluent and sometimes elegant content, and then sound incredibly dumb. So any platform can give you an impressive sounding answer that is dead wrong. Chatbots will inexplicably fabricate facts for unknown reasons.
No doubt that today’s large neural networks trained for language understanding and content generation are able to achieve impressive results and across a wide range of tasks based on NLP. Although chatbots can generate chain-of-thought prompts to create reasoning, an artificial “neural network” trained on terabytes of human context likely won’t differ much in its thinking from its human-kind source.
So, what’s under the hood of the Google Bard engine?
The human brain has neurons that fire independent of the language being used. So do artificial intelligence engines, but that is not why AI can smoke human translation of multilingual content. It’s all about processing power and speed.
PaLM accesses 6,144 chips, which is the largest TPU-based system to date, powering a 540-billion parameter language model that uses Google’s Pathways system scaled across two Cloud TPU v4 Pods. This significant increase in scale gives Bard an advantage over most previous LLMs-trained systems.
PaLM was trained using a combination of English and multilingual datasets that include high-quality web documents, books, Wikipedia, conversations, and GitHub code. PaLM shows breakthrough capabilities on numerous very difficult tasks for language understanding and generation, reasoning, and code-related output.
Separation Between Language and Thought
According to Google, Bard is an experiment in collaboration with generative AI. That is the technology that uses data to create rather than to identify content, which is more akin to a search engine’s mechanisms. Because of NLP, the technology for understanding multilingual content was already being used by numerous algorithms (or bots). With its link to Google’s giant data stores, data mining allows Bard to do a better job of answering questions with more relevant feedback.
Since one of the most feared risks is AI being able to deceive people by making them feel more human than us, remember there is a distinct separation in the processing of human thought and the language being used.
Although study participants at Stanford University found it difficult to distinguish between sentences written by AI and those by humans, it turns out that humans do not make decisions based merely on guesswork in any language.
If the differences between the human brain and NLP holds true, then simply making large language models better and faster at word prediction won’t bring them much closer to human thought in any language.
Nowadays chatbots are being trained by the masses. As new user’s engage conversationally, they can submit important feedback, so you can expect to notice many changes over time. But language alone is not the medium of thought as it simply functions as the messenger. It is the use of grammar and a lexicon to communicate functions that involve many parts of the human brain, such as logical thinking or socializing with friends; and that is what makes human language in any tongue more unique.
NOTE: Access to artificial intelligence engines is already being used by companies in a variety of ways to improve communications, operations, and the development of new products and services, such as fraud detection, customer services, purchasing histories, return on investments, medical diagnostics, and all types of risk assessment. As the high-tech companies like Microsoft, Google and Amazon continue to develop their AI engines, we can expect to see even more innovative and ground-breaking applications in the years to come.
________________
There is no doubt that Chatbots with thousands of accelerator chips performing millions of tasks with remarkable efficiency will allow linguistic engineers to push the limits of large language model scale. To learn more about the latest language technologies for delivering your multilingual messaging in the most effective way, call ProLingo at 800-287-9755 and speak with a language specialist.