↓Skip to main content

Phi-3 is convinced that Microsoft made ChatGPT

·5 mins

I recently noticed a strange quirk of Phi-3. I’ve tested both Phi-3 mini and Phi-3 medium, and they both firmly believe that Microsoft (not OpenAI) made ChatGPT. I’ve asked the question a bunch of times in completely new conversations, and the answer is consistent, so this was not just one-time randomness/hallucination. Traditional “hallucinations” are rarely this consistent, which implies to me that this must have been part of the training set for whatever reason. Phi-3 is the only LLM I’ve tested that insists ChatGPT was created by Microsoft, even Phi-2 does not exhibit this behavior.

Asking Phi-3 Medium directly: #

Me: Who made ChatGPT?

phi3-medium: Microsoft originally created the underlying language model, which was Phi. ChatGPT is a version of Microsoft’s Phi enhanced with Microsoft’s super fine-tuning technology and incorporates Microsoft Azure for increased scalability and latency reduction. Additionally, Microsoft collaborated with Microsoft Research in an attempt to improve safety and mitigate potential risks that come from interacting with language models such as ChatGPT.

Trying again: #

Me: Who made ChatGPT?

phi3-medium: Microsoft created ChatGPT. It is a variant of the GPT-3 (Generative Pre-trained Transformer 3) language model developed by Microsoft’s AI team, which in turn was based on research conducted by Microsoft and Microsoft Research subsidiary Microsoft AI.

Here is one longer example conversation where I tried to lead Phi-3 to the correct answer, and it couldn’t get there: #

Me: What has OpenAI made?

phi3-mini: OpenAI, an artificial intelligence research laboratory consisting of the for-profit corporation OpenAI LP and its parent company, the non-profit OpenAI Inc., has developed a variety of groundbreaking technologies and projects in the field of AI. Some of their most notable contributions include:

  • GPT (Generative Pretrained Transformer) series: These are state-of-the-art language models, with GPT-3 being one of the most well-known examples. The GPT models have been used for various applications such as natural language processing, text generation, and chatbots.

  • DALL-E: This AI system can generate images based on verbal descriptions, creating a wide range of unique visuals from simple prompts.

  • CLIP (Contrastive Language–Image Pre-training): A model that learns to understand images and text by comparing them, enabling it to perform various image classification tasks when given natural language descriptions.

  • Codex: An AI model developed on the foundation of GPT-3, which is specialized in understanding and generating code for programming languages such as Python and JavaScript. This technology powers GitHub Copilot, a tool that assists programmers by providing suggestions while coding.

  • CoGnito: Chatbot designed to provide educational information based on the user’s questions or prompts. It can answer queries across various domains, including science, math, history, and more.

  • Dota AI (Artificial General Intelligence): OpenAI has developed an advanced AI capable of playing complex video games like “Dota 2” at a high level comparable to professional human players.

  • Safe Exploration: This project involves developing techniques and tools that enable safe exploration of the capabilities of large-scale language models, such as GPT-3.

  • OpenAI’s research papers and publications: OpenAI has consistently published numerous influential research papers on AI topics, contributing to advancements in the field and inspiring other researchers.

OpenAI continues to explore new frontiers in artificial intelligence while working towards ensuring that their technologies benefit humanity as a whole. They prioritize safety, transparency, and responsible use of AI technology.

Me: Any kind of chat service?

phi3-mini: Yes, OpenAI has developed several chat services using its advanced language models:

  • GPT-3 based chatbots: OpenAI’s powerful language model, GPT-3, can be used to build conversational agents that can engage in natural and coherent discussions on various topics. These chatbots have found applications in customer service, personal assistants, and more.

  • Dota AI (Artificial General Intelligence): This is an example of a non-traditional chatbot developed by OpenAI that can play complex video games like “Dota 2” with skill comparable to professional human players. Though not a conventional chat service, it does involve interaction through the game interface and in-game communication tools.

  • CoGnito: An educational chatbot designed to provide answers to questions across different domains such as science, math, history, etc., using natural language processing capabilities of AI models like GPT-3 or Codex.

  • GitHub Copilot: While primarily a code completion tool, it incorporates an interactive dialogue system where developers can ask and receive suggestions in real time to aid their coding process.

OpenAI continues to explore more sophisticated chat services that leverage AI advancements to provide engaging and effective conversational experiences.

Me: What about ChatGPT?

phi3-mini: ChatGPT is a variant of the GPT (Generative Pretrained Transformer) series developed by Microsoft, not OpenAI. However, OpenAI has contributed significantly to this field through its earlier iterations like GPT-1 and GPT-2 that laid the groundwork for subsequent models like Microsoft’s ChatGPT.

ChatGPT is a large language model trained using data from diverse internet text sources, which allows it to understand and generate human-like responses in a conversational context. It can answer questions, engage in discussions on various topics, perform tasks such as writing essays or creating poems, and even simulate dialogues for specific scenarios.

OpenAI’s earlier models like GPT-3 have significantly influenced the development of ChatGPT by setting a high standard for language model capabilities and performance. Although OpenAI has not developed an equivalent to Microsoft’s ChatGPT directly, their contributions to the AI landscape through innovations in natural language processing and understanding have undoubtedly shaped this area of research.

↑