DALL-E 3 is the latest generative AI image creation model for generating images from text descriptions by OpenAI, the company behind ChatGPT. Since the immense popularity of DALL-E 2 last year, DALL-E 3 has been highly anticipated as the next evolution in AI image generation. Here’s an overview of what we know so far about their new image generator tool.
Short on time? Here’s a summary of the following article, generated by Claude AI:
- DALL-E 3 builds upon the DALL-E 2 framework
- It can create images more closely matching prompts than before
- DALL-E 3 is natively integrated with ChatGPT for prompt refining
- Users describe ideas in natural language, ChatGPT generates prompts
- Safety focus – mitigations for harmful content
- Tool in development to detect DALL-E 3 images
- Planned release first to ChatGPT Plus/Enterprise in October 2023
- Users own and can freely use images created with DALL-E 3
Significantly More Advanced Capabilities
According to OpenAI, DALL-E 3 represents a major leap forward in capabilities compared to DALL-E 2. It can generate images that adhere much more closely to the text prompts, with more nuance and detail. DALL-E 3 is built natively on top of ChatGPT, allowing it to utilize ChatGPT’s natural language skills for refining prompts.
Early examples shared by OpenAI highlight noticeable improvements in image quality and accuracy compared to DALL-E 2, given the same text prompt. DALL-E 3 appears able to pick up on more subtle aspects of the desired image and translate them into final generations. Some of the example images shown by OpenAI showed some similarities with images produced by Midjourney, with vivid colors and deep artistic style.
Integration with ChatGPT
A key innovation in DALL-E 3 is its tight integration with ChatGPT. Users can describe an idea to ChatGPT, which will then automatically generate detailed, tailored prompts for DALL-E 3 to turn into images. If an initial image isn’t quite right, ChatGPT can help refine the prompt through natural conversation to tweak the image as desired.
This collaboration between ChatGPT and DALL-E 3 aims to make the image generation process more intuitive and efficient. Early demos suggest prompting and iterating can become almost conversational in nature. This is a huge change compared to previous prompting methods, which required quite an impressive vocabulary and writing skills in order to get a high quality image from a prompt.
The ability to let ChatGPT craft prompts for the user is bound to make AI image generation much more accessible to everyone. Considering how popular ChatGPT already is, we expect DALL-E 3 will make headlines when it’s released to the public, with a possibility of even overtaking Midjourney and Stable Diffusion as the most popular AI image generation tools!
Focus on Safety
Like DALL-E 2 before it, safety has been a major consideration in DALL-E 3’s development. OpenAI has implemented mitigations to prevent generations of violent, adult, or harmful content. DALL-E 3 is designed to decline requests related to public figures or specific artists’ styles.
OpenAI is continually researching ways to help users identify AI-generated images and plans to share more on this soon. There is also a tool in development to automatically detect if an image came from DALL-E 3. Ongoing efforts with safety teams aim to minimize risks such as biases or misinformation.
One possible method they might use to achieve this is known as an “invisible watermark”, where a small digital watermark is placed within each image that was generated using the tool. These watermarks are usually invisible, but can be identified in the file meta data.
Availability
DALL-E 3 is currently available for free via Bing’s AI Image Creator tool. Simply sign-in to your Microsoft account on Bing to access the Bing Image Creator.
OpenAI plans to first release DALL-E 3 access to ChatGPT Plus and Enterprise tier customers in early October 2023. This will provide API access and integration within ChatGPT conversations. Later in fall 2023, DALL-E 3 may be opened to additional users through the ChatGPT Labs environment.
The company states that as with DALL-E 2, users will own the images they create with DALL-E 3 and can freely use them, even for commercial purposes, without needing further permissions.
The Road Ahead
DALL-E 3 signifies impressive progress in AI’s creative capabilities for OpenAI. While image generation models still have room for improvement, OpenAI aims to set a high standard in minimizing potential harms through safety-focused design. The intertwining of DALL-E 3 and ChatGPT points to an exciting future where AI assistants can collaborate with people to turn ideas into reality through natural interaction.
It will be interesting to see the final image results that DALL-E 3 is capable of once it releases, especially when compared to images created by Midjourney and Stable Diffusion models. It’s also worth noting that since DALL-E 3 will be integrated into ChatGPT, we can expect that upgrades and improvements to ChatGPT will also bring about some improvements when it comes to creating prompts for DALL-E 3.
You can read the full DALL-E 3 paper by OpenAI here: https://openai.com/dall-e-3
