Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade from DALL·E 2 to DALL·E 3 #717

Merged
merged 7 commits into from
Feb 21, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Tap into leading cloud-based services like [OpenAI](https://openai.com/), [Micro
* Generate a summary of post content and store it as an excerpt using [OpenAI's ChatGPT API](https://platform.openai.com/docs/guides/chat) and [Google's Gemini API](https://ai.google.dev/docs/gemini_api_overview)
* Generate titles from post content using [OpenAI's ChatGPT API](https://platform.openai.com/docs/guides/chat) and [Google's Gemini API](https://ai.google.dev/docs/gemini_api_overview)
* Expand or condense text content using [OpenAI's ChatGPT API](https://platform.openai.com/docs/guides/chat) and [Google's Gemini API](https://ai.google.dev/docs/gemini_api_overview)
* Generate new images on demand to use in-content or as a featured image using [OpenAI's DALL·E API](https://platform.openai.com/docs/guides/images)
* Generate new images on demand to use in-content or as a featured image using [OpenAI's DALL·E 3 API](https://platform.openai.com/docs/guides/images)
* Generate transcripts of audio files using [OpenAI's Whisper API](https://platform.openai.com/docs/guides/speech-to-text)
* Moderate incoming comments for sensitive content using [OpenAI's Moderation API](https://platform.openai.com/docs/guides/moderation)
* Convert text content into audio and output a "read-to-me" feature on the front-end to play this audio using [Microsoft Azure's Text to Speech API](https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/text-to-speech)
Expand Down Expand Up @@ -397,7 +397,7 @@ Note that [Azure AI Vision](https://docs.microsoft.com/en-us/azure/cognitive-ser

### 2. Configure OpenAI API Keys under Tools > ClassifAI > Image Processing > Image Generation

* Select **OpenAI DALL-E** in the provider dropdown.
* Select **OpenAI DALL·E 3** in the provider dropdown.
* Enter your API Key copied from the above step into the `API Key` field.

### 3. Enable specific Image Processing features
Expand Down
4 changes: 3 additions & 1 deletion includes/Classifai/Features/Feature.php
Original file line number Diff line number Diff line change
Expand Up @@ -1113,11 +1113,13 @@ function ( $role ) {
__( 'Provider', 'classifai' ) => $feature_settings['provider'],
];

if ( method_exists( $provider, 'get_debug_information' ) ) {
if ( $provider && method_exists( $provider, 'get_debug_information' ) ) {
$all_debug_info = array_merge(
$common_debug_info,
$provider->get_debug_information()
);
} else {
$all_debug_info = $common_debug_info;
}

/**
Expand Down
2 changes: 1 addition & 1 deletion includes/Classifai/Features/ImageGeneration.php
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ public function __construct() {

// Contains just the providers this feature supports.
$this->supported_providers = [
DallE::ID => __( 'OpenAI Dall-E', 'classifai' ),
DallE::ID => __( 'OpenAI DALL·E 3', 'classifai' ),
];
}

Expand Down
183 changes: 137 additions & 46 deletions includes/Classifai/Providers/OpenAI/DallE.php
Original file line number Diff line number Diff line change
Expand Up @@ -18,18 +18,18 @@ class DallE extends Provider {
const ID = 'openai_dalle';

/**
* OpenAI DALL·E URL
* OpenAI DALL·E URL.
*
* @var string
*/
protected $dalle_url = 'https://api.openai.com/v1/images/generations';

/**
* Maximum number of characters a prompt can have
* Maximum number of characters a prompt can have.
*
* @var int
*/
public $max_prompt_chars = 1000;
public $max_prompt_chars = 4000;

/**
* OpenAI DALL·E constructor.
Expand Down Expand Up @@ -64,33 +64,53 @@ public function register_endpoints() {
'methods' => WP_REST_Server::READABLE,
'callback' => [ $this->feature_instance, 'rest_endpoint_callback' ],
'args' => [
'prompt' => [
'prompt' => [
'required' => true,
'type' => 'string',
'sanitize_callback' => 'sanitize_text_field',
'validate_callback' => 'rest_validate_request_arg',
'description' => esc_html__( 'Prompt used to generate an image', 'classifai' ),
],
'n' => [
'n' => [
'type' => 'integer',
'minimum' => 1,
'maximum' => 10,
'sanitize_callback' => 'absint',
'validate_callback' => 'rest_validate_request_arg',
'description' => esc_html__( 'Number of images to generate', 'classifai' ),
],
'size' => [
'quality' => [
'type' => 'string',
'enum' => [
'standard',
'hd',
],
'sanitize_callback' => 'sanitize_text_field',
'validate_callback' => 'rest_validate_request_arg',
'description' => esc_html__( 'Quality of generated image', 'classifai' ),
],
'size' => [
'type' => 'string',
'enum' => [
'256x256',
'512x512',
'1024x1024',
'1792x1024',
'1024x1792',
],
'sanitize_callback' => 'sanitize_text_field',
'validate_callback' => 'rest_validate_request_arg',
'description' => esc_html__( 'Size of generated image', 'classifai' ),
],
'format' => [
'style' => [
'type' => 'string',
'enum' => [
'vivid',
'natural',
],
'sanitize_callback' => 'sanitize_text_field',
'validate_callback' => 'rest_validate_request_arg',
'description' => esc_html__( 'Style of generated image', 'classifai' ),
],
'format' => [
'type' => 'string',
'enum' => [
'url',
Expand Down Expand Up @@ -123,7 +143,7 @@ public function render_provider_fields() {
'label_for' => 'api_key',
'input_type' => 'password',
'default_value' => $settings['api_key'],
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID, // Important to add this.
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID,
'description' => sprintf(
wp_kses(
/* translators: %1$s is replaced with the OpenAI sign up URL */
Expand All @@ -141,7 +161,7 @@ public function render_provider_fields() {
);

add_settings_field(
static::ID . 'number_of_images',
static::ID . '_number_of_images',
esc_html__( 'Number of images', 'classifai' ),
[ $this->feature_instance, 'render_select' ],
$this->feature_instance->get_option_name(),
Expand All @@ -152,12 +172,31 @@ public function render_provider_fields() {
'options' => array_combine( range( 1, 10 ), range( 1, 10 ) ),
'default_value' => $settings['number_of_images'],
'description' => __( 'Number of images that will be generated in one request. Note that each image will incur separate costs.', 'classifai' ),
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID, // Important to add this.
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID,
]
);

add_settings_field(
static::ID . 'image_size',
static::ID . '_quality',
esc_html__( 'Image quality', 'classifai' ),
[ $this->feature_instance, 'render_select' ],
$this->feature_instance->get_option_name(),
$this->feature_instance->get_option_name() . '_section',
[
'option_index' => static::ID,
'label_for' => 'quality',
'options' => [
'standard' => __( 'Standard', 'classifai' ),
'hd' => __( 'High Definition', 'classifai' ),
],
'default_value' => $settings['quality'],
'description' => __( 'The quality of the image that will be generated. High Definition creates images with finer details and greater consistency across the image but costs more.', 'classifai' ),
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID,
]
);

add_settings_field(
static::ID . '_image_size',
esc_html__( 'Image size', 'classifai' ),
[ $this->feature_instance, 'render_select' ],
$this->feature_instance->get_option_name(),
Expand All @@ -166,13 +205,32 @@ public function render_provider_fields() {
'option_index' => static::ID,
'label_for' => 'image_size',
'options' => [
'256x256' => '256x256',
'512x512' => '512x512',
'1024x1024' => '1024x1024',
'1792x1024' => '1792x1024',
'1024x1792' => '1024x1792',
],
'default_value' => $settings['image_size'],
'description' => __( 'Size of generated images.', 'classifai' ),
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID, // Important to add this.
'description' => __( 'Size of generated images. Larger sizes cost more.', 'classifai' ),
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID,
]
);

add_settings_field(
static::ID . '_style',
esc_html__( 'Image style', 'classifai' ),
[ $this->feature_instance, 'render_select' ],
$this->feature_instance->get_option_name(),
$this->feature_instance->get_option_name() . '_section',
[
'option_index' => static::ID,
'label_for' => 'style',
'options' => [
'vivid' => __( 'Vivid', 'classifai' ),
'natural' => __( 'Natural', 'classifai' ),
],
'default_value' => $settings['style'],
'description' => __( 'The style of the generated images. Vivid causes more hyper-real and dramatic images. Natural causes more natural, less hyper-real looking images.', 'classifai' ),
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID,
]
);
}
Expand All @@ -194,7 +252,9 @@ public function get_default_provider_settings(): array {
$common_settings,
[
'number_of_images' => 1,
'image_size' => '256x256',
'quality' => 'standard',
'image_size' => '1024x1024',
'style' => 'vivid',
]
);
}
Expand All @@ -217,8 +277,22 @@ public function sanitize_settings( array $new_settings ): array {
if ( $this->feature_instance instanceof ImageGeneration ) {
$new_settings[ static::ID ]['number_of_images'] = absint( $new_settings[ static::ID ]['number_of_images'] ?? $settings[ static::ID ]['number_of_images'] );

if ( in_array( $new_settings[ static::ID ]['image_size'], [ '256x256', '512x512', '1024x1024' ], true ) ) {
$new_settings[ static::ID ]['image_size'] = sanitize_text_field( $new_settings[ static::ID ]['image_size'] ?? $settings[ static::ID ]['image_size'] );
if ( in_array( $new_settings[ static::ID ]['quality'], [ 'standard', 'hd' ], true ) ) {
$new_settings[ static::ID ]['quality'] = sanitize_text_field( $new_settings[ static::ID ]['quality'] );
} else {
$new_settings[ static::ID ]['quality'] = $settings[ static::ID ]['quality'];
}

if ( in_array( $new_settings[ static::ID ]['image_size'], [ '1024x1024', '1792x1024', '1024x1792' ], true ) ) {
$new_settings[ static::ID ]['image_size'] = sanitize_text_field( $new_settings[ static::ID ]['image_size'] );
} else {
$new_settings[ static::ID ]['image_size'] = $settings[ static::ID ]['image_size'];
}

if ( in_array( $new_settings[ static::ID ]['style'], [ 'vivid', 'natural' ], true ) ) {
$new_settings[ static::ID ]['style'] = sanitize_text_field( $new_settings[ static::ID ]['style'] );
} else {
$new_settings[ static::ID ]['style'] = $settings[ static::ID ]['style'];
}
}

Expand Down Expand Up @@ -264,12 +338,19 @@ public function generate_image( string $prompt = '', array $args = [] ) {
$args = wp_parse_args(
array_filter( $args ),
[
'num' => $settings['number_of_images'] ?? 1,
'size' => $settings['image_size'] ?? '1024x1024',
'format' => 'url',
'num' => $settings['number_of_images'] ?? 1,
'quality' => $settings['quality'] ?? 'standard',
'size' => $settings['image_size'] ?? '1024x1024',
'style' => $settings['style'] ?? 'vivid',
'format' => 'url',
]
);

// Force proper image size for those that had been using DALL·E 2 and haven't updated settings.
if ( ! in_array( $args['size'], [ '1024x1024', '1792x1024', '1024x1792' ], true ) ) {
$args['size'] = '1024x1024';
}

if ( ! $image_generation->is_feature_enabled() ) {
return new WP_Error( 'not_enabled', esc_html__( 'Image generation is disabled or OpenAI authentication failed. Please check your settings.', 'classifai' ) );
}
Expand Down Expand Up @@ -307,40 +388,47 @@ public function generate_image( string $prompt = '', array $args = [] ) {
'classifai_dalle_request_body',
[
'prompt' => sanitize_text_field( $prompt ),
'n' => absint( $args['num'] ),
'size' => sanitize_text_field( $args['size'] ),
'model' => 'dall-e-3',
'n' => 1,
'quality' => sanitize_text_field( $args['quality'] ),
'response_format' => sanitize_text_field( $args['format'] ),
'size' => sanitize_text_field( $args['size'] ),
'style' => sanitize_text_field( $args['style'] ),
]
);

// Make our API request.
$response = $request->post(
$this->dalle_url,
[
'body' => wp_json_encode( $body ),
]
);
$responses = [];

set_transient( 'classifai_openai_dalle_latest_response', $response, DAY_IN_SECONDS * 30 );
// DALL·E 3 doesn't support multiple images in a single request so make one request per image.
for ( $i = 0; $i < $args['num']; $i++ ) {
$responses[] = $request->post(
$this->dalle_url,
[
'body' => wp_json_encode( $body ),
]
);
}

set_transient( 'classifai_openai_dalle_latest_response', $responses[ array_key_last( $responses ) ], DAY_IN_SECONDS * 30 );

// Extract out the image response, if it exists.
if ( ! is_wp_error( $response ) && ! empty( $response['data'] ) ) {
$cleaned_response = [];
$cleaned_responses = [];

foreach ( $response['data'] as $data ) {
if ( ! empty( $data[ $args['format'] ] ) ) {
if ( 'url' === $args['format'] ) {
$cleaned_response[] = [ 'url' => esc_url_raw( $data[ $args['format'] ] ) ];
} else {
$cleaned_response[] = [ 'url' => $data[ $args['format'] ] ];
foreach ( $responses as $response ) {
// Extract out the image response, if it exists.
if ( ! is_wp_error( $response ) && ! empty( $response['data'] ) ) {
foreach ( $response['data'] as $data ) {
if ( ! empty( $data[ $args['format'] ] ) ) {
if ( 'url' === $args['format'] ) {
$cleaned_responses[] = [ 'url' => esc_url_raw( $data[ $args['format'] ] ) ];
} else {
$cleaned_responses[] = [ 'url' => $data[ $args['format'] ] ];
}
}
}
}

$response = $cleaned_response;
}

return $response;
return $cleaned_responses;
}

/**
Expand All @@ -354,7 +442,10 @@ public function get_debug_information(): array {
$debug_info = [];

if ( $this->feature_instance instanceof ImageGeneration ) {
$debug_info[ __( 'Number of images', 'classifai' ) ] = $provider_settings['number_of_images'];
$debug_info[ __( 'Number of images', 'classifai' ) ] = $provider_settings['number_of_images'] ?? 1;
$debug_info[ __( 'Quality', 'classifai' ) ] = $provider_settings['quality'] ?? 'standard';
$debug_info[ __( 'Size', 'classifai' ) ] = $provider_settings['image_size'] ?? '1024x1024';
$debug_info[ __( 'Style', 'classifai' ) ] = $provider_settings['style'] ?? 'vivid';
$debug_info[ __( 'Latest response:', 'classifai' ) ] = $this->get_formatted_latest_response( get_transient( 'classifai_openai_dalle_latest_response' ) );
}

Expand Down
2 changes: 1 addition & 1 deletion readme.txt
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Tap into leading cloud-based services like [OpenAI](https://openai.com/), [Micro
* Generate a summary of post content and store it as an excerpt using [OpenAI's ChatGPT API](https://platform.openai.com/docs/guides/chat) and [Google's Gemini API](https://ai.google.dev/docs/gemini_api_overview)
* Generate titles from post content using [OpenAI's ChatGPT API](https://platform.openai.com/docs/guides/chat) and [Google's Gemini API](https://ai.google.dev/docs/gemini_api_overview)
* Expand or condense text content using [OpenAI's ChatGPT API](https://platform.openai.com/docs/guides/chat) and [Google's Gemini API](https://ai.google.dev/docs/gemini_api_overview)
* Generate new images on demand to use in-content or as a featured image using [OpenAI's DALL·E API](https://platform.openai.com/docs/guides/images)
* Generate new images on demand to use in-content or as a featured image using [OpenAI's DALL·E 3 API](https://platform.openai.com/docs/guides/images)
* Generate transcripts of audio files using [OpenAI's Whisper API](https://platform.openai.com/docs/guides/speech-to-text)
* Convert text content into audio and output a "read-to-me" feature on the front-end to play this audio using [Microsoft Azure's Text to Speech API](https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/text-to-speech)
* Classify post content using [IBM Watson's Natural Language Understanding API](https://www.ibm.com/watson/services/natural-language-understanding/) and [OpenAI's Embedding API](https://platform.openai.com/docs/guides/embeddings)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ describe( 'Image Generation (OpenAI DALL·E) Tests', () => {
'/wp-admin/tools.php?page=classifai&tab=image_processing&feature=feature_image_generation'
);
cy.get( '#status' ).check();
cy.get( '#provider' ).select( 'openai_dalle' );
cy.get( '#submit' ).click();
cy.optInAllFeatures();
} );
Expand All @@ -19,13 +20,23 @@ describe( 'Image Generation (OpenAI DALL·E) Tests', () => {
);

cy.get( '#api_key' ).clear().type( 'password' );
cy.get(
'select[name="classifai_feature_image_generation[openai_dalle][number_of_images]"]'
).select( '2' );
cy.get(
'select[name="classifai_feature_image_generation[openai_dalle][quality]"]'
).select( 'hd' );
cy.get(
'select[name="classifai_feature_image_generation[openai_dalle][image_size]"]'
).select( '1024x1792' );
cy.get(
'select[name="classifai_feature_image_generation[openai_dalle][style]"]'
).select( 'natural' );

cy.get( '#status' ).check();
cy.get(
'#classifai_feature_image_generation_roles_administrator'
).check();
cy.get( '#number_of_images' ).select( '2' );
cy.get( '#image_size' ).select( '512x512' );

cy.get( '#submit' ).click();
} );

Expand Down
Loading
Loading