Generative AI—also known as GenAI or GAI—is a type of artificial intelligence model or tool which, given a prompt or query, is capable of generating new and creative outputs in a variety of media, including text, images, video, audio, code, or other formats. GenAI tools do this by collecting millions of representative samples from online data and "training" the system by specifying how the tool should respond when prompted.
GenAI can be used to create outputs that mimic human creation with varying degrees of success. ChatGPT is currently the most well-known GenAI application and is a sophisticated chatbot that has been trained on an enormous collection of text data to develop an understanding of the patterns and structures of human language.
University of Michigan's resources on Generative AI:
"Large Language Models (LLMs) refer to large general-purpose language models that can be pre-trained and then fine-tuned for specific purposes. They are trained to solve common language problems, such as text classification, question answering, document summarization, and text generation. The models can then be adapted to solve specific problems in different fields using a relatively small size of field datasets via fine-tuning.
The ability of LLMs taking the knowledge learnt from one task and applying it to another task is enabled by transfer learning. Pre-training is the dominant approach to transfer learning in deep learning.
LLMs predict the probabilities of next word (token), given an input string of text, based on the language in the training data. Besides, instruction tuned language models predict a response to the instructions given in the input. These instructions can be "summarize a text", "generate a poem in the style of X", or "give a list of keywords based on semantic similarity for X".
LLMs are large, not only because of their large size of training data, but also their large number of parameters. They display different behaviors from smaller models and have important implications for those who develop and use A.I. systems. First, the ability to solve complex tasks with minimal training data through in-context learning. Second, LLMs are accessed primarily through a prompting interface, which necessitates human comprehension of how LLMs function and the ability to format tasks in a way that LLMs can comprehend. Third, the development of LLMs no longer distinguishes between research and engineering. The training of LLMs requires extensive hands-on experience in processing large amounts of data and training in distributed parallel training. To develop effective LLMs, researchers must address complex engineering issues and work alongside engineers or have engineering expertise themselves."
Dai, Y. (n.d.). Research Guides: Machines and Society: Large Language Models. Retrieved August 30, 2023, from https://guides.nyu.edu/data/llm
