A deep dive into the capabilities of OpenAI's GPT-4o Mini

In May 2024, OpenAI released GPT-4o, and now, just three months later, they're back with another impressive model: GPT-4o Mini. On July 18th, 2024, OpenAI introduced GPT-4o Mini. They are calling it their “most cost-efficient model”! GPT-4o Mini is a compact model that builds on the capabilities of previous models and aims to make advanced AI more accessible and affordable.

GPT-4o Mini currently supports text and vision interactions, with future updates expected to add capabilities for handling images, videos, and audio. In this article, we will explore what GPT-4o Mini is, its standout features, how it can be used, the differences between GPT-4 and GPT-4o Mini, and how it can be used in various computer vision use cases. Let’s dive in and see what GPT-4o Mini has to offer!

Link to this sectionWhat is GPT-4o Mini?#

GPT-4o Mini is the latest addition to OpenAI's lineup of AI models, designed to be more cost-efficient and accessible. It's a multimodal large language model (LLM), which means it can process and generate different types of data, such as text, images, videos, and audio. The model builds on the strengths of previous models like GPT-4 and GPT-4o to offer powerful capabilities in a compact package.

GPT-4o Mini is 60% cheaper than GPT-3.5 Turbo, costing 15 cents per million input tokens (units of text or data the model processes) and 60 cents per million output tokens (units the model generates in response). To put that into perspective, one million tokens is roughly equivalent to processing 2,500 pages of text. With a context window of 128K tokens and the ability to handle up to 16K output tokens per request, GPT-4o Mini is designed to be both efficient and affordable.

Fig 1. GPT-4o Mini is 60% cheaper than GPT-3.5 Turbo.

Link to this sectionKey features of GPT-4o Mini#

GPT-4o Mini supports a range of tasks that make it a great option for various applications. It can be used when running several operations at once, such as calling multiple APIs, dealing with large amounts of data like full code bases or conversation histories, and providing quick, real-time responses in customer support chatbots.

Here are some other key features:

• Updated Knowledge Base: The model contains information up to October 2023.
• Improved Tokenizer: GPT-4o Mini makes processing non-English text more cost-effective.
• Robust Safety Measures: These measures include filtering harmful content and protecting against security issues like prompt injections and system manipulations.

Link to this sectionGetting started with GPT-4o Mini#

You can try using GPT-4o Mini through the ChatGPT interface. It is accessible to Free, Plus, and Team users, replacing GPT-3.5 as shown below. Enterprise users will also gain access soon, in line with OpenAI’s objective of providing AI benefits to all. GPT-4o Mini is also available through the API for developers who want to integrate its capabilities into their applications. At the moment, vision capabilities are accessible only through the API.

Fig 2. Models Options Within ChatGPT.

Link to this sectionThe difference between GPT-4o and GPT-4o Mini#

GPT-4o Mini and GPT-4o both perform impressively across various benchmarks. While GPT-4o generally outperforms GPT-4o Mini, GPT-4o Mini is still a cost-effective solution for everyday tasks. The benchmarks include reasoning tasks, math and coding proficiency, and multimodal reasoning. As shown in the image below, GPT-4o Mini benchmarks quite high when compared to other popular models.

Fig 3. Comparing GPT-4o Mini With Other Popular Models.

Link to this sectionGetting hands-on with GPT-4o and GPT-4o Mini#

An interesting prompt that's been debated online involves popular LLMs comparing decimal numbers incorrectly. When we put GPT-4o and GPT-4o Mini to the test, their reasoning abilities showed clear differences. In the image below, we asked both models which is greater: 9.11 or 9.9, and then had them explain their reasoning.

Fig 4. Testing GPT-4o and GPT-4o Mini.

Both models initially respond incorrectly and claim that 9.11 is greater. However, GPT-4o is able to reason its way to the correct answer and states that 9.9 is greater. It provides a detailed explanation and compares the decimals accurately. In contrast, GPT-4o Mini stubbornly maintains its initial wrong answer despite figuring out the reasoning behind 9.9 being greater correctly.

Both models show strong reasoning skills. GPT-4o's ability to correct itself makes it superior and useful for more complex tasks. GPT-4o Mini, while less adaptable, still offers clear and accurate reasoning for simpler tasks.

Link to this sectionUsing GPT-4o Mini for various computer vision use cases#

If you'd prefer to explore the vision capabilities of GPT-4o Mini without diving into the code, you can easily test the API on the OpenAI Playground. We tried it out ourselves to see how well GPT-4o Mini is able to handle various computer vision related use cases.

Link to this sectionImage classification using GPT-4o Mini#

We asked GPT-4o Mini to classify two images: one of a butterfly and one of a map. The AI model successfully identified the butterfly and the map. This is a fairly simple task given that the images are very different.

Fig 5. Classifying images with the help of GPT-4o Mini.

We went on and ran two more images through the model: one showing a butterfly resting on a plant and another showing a butterfly resting on the ground. The AI did a great job again, correctly spotting the butterfly on the plant and the one on the ground. So, we took it a step further again.

Fig 6. Classifying similar images with the help of GPT-4o Mini.

We then asked GPT-4o Mini to classify two images: one showing a butterfly feeding on the flowers of a Swamp Milkweed and the other showing a butterfly feeding on a Zinnia flower. It's amazing that the model was able to classify a label that is so specific without further fine-tuning. These quick examples show that GPT-4o Mini could possibly be used for image classification tasks without needing custom training.

Fig 7. Classifying detailed images with the help of GPT-4o Mini.

Link to this sectionUnderstanding poses using GPT-4o Mini#

As of now, computer vision tasks like object detection and instance segmentation can't be handled using GPT-4o Mini. GPT-4o struggles for accuracy, but can be used for such tasks. Along these lines, with respect to understanding poses, we can't detect or estimate the pose in the image, but we can classify and understand the pose.

Fig 8. Using GPT-4o Mini to understand the poses in an image.

The image above shows how GPT-4o Mini can classify and understand poses, despite not being able to detect or estimate the precise coordinates of the pose. This can be helpful in different applications. For example, in sports analytics, it can broadly evaluate athletes' movements and help prevent injuries. Similarly, in physical therapy, it can assist in monitoring exercises to make sure the correct movements are made by patients during rehabilitation. Also for surveillance, it can help identify suspicious activities by analyzing general body language. While GPT-4o Mini can't detect specific key points, its ability to classify general poses makes it useful in these and other fields.

Link to this sectionApplications GPT-4o Mini are suitable for#

We've taken a look at what GPT-4o Mini can do. Now, let’s discuss the applications where it’s most optimal to use GPT-4o Mini.

GPT-4o Mini is great for applications that require advanced natural language understanding and need a small computational footprint. It makes it possible to integrate AI into applications where it would normally be too expensive. In fact, a detailed analysis by Artificial Analysis shows that GPT-4o Mini provides high-quality responses at blazing-fast speeds compared to most other models.

Fig 9. Quality Vs. Output Speed of GPT-4o Mini.

Here are some key areas where it could shine in the future:

• Virtual Assistants and Chatbots: GPT-4o Mini can provide quick and smart responses to improve user interactions.
• Educational Tools: The model can be used to build tools to offer personalized tutoring and content generation.
• Productivity Tools: It can improve tasks like summarizing documents, drafting emails, and translating languages to boost efficiency.
• Language Translation: The latest version of GPT can be used to develop translators that provide accurate and real-time language translation for better communication across different languages.

Link to this sectionGPT-4o Mini opens new doors#

GPT-4o Mini is creating new opportunities for the future of multimodal AI. The expense of processing each piece of text or data, known as the cost per token, has decreased substantially - by almost 99% - since 2022, when text-davinci-003, the GPT-3 model, was released. The decrease in cost shows a clear trend towards making advanced AI more affordable. As AI models continue to improve, it's becoming increasingly likely that integrating AI into every app and website will be economically viable!

Want to get hands-on with AI? Visit our GitHub repository to see our innovations and become part of our active community. Find out more about AI applications in manufacturing and agriculture on our solutions pages.