Custom Generator Types

If you want to leverage OpenAI to generate assets within your pipeline, you have two options - Either your own CustomGenerators, or leverage our API layer to make requests directly.

Extending Generators

If you want to extend something that will generate text, make a class that inherits from BasicTextGenerator. Create MyTextGenerator.cs

[CreateAssetMenu(menuName = "MyProject/MyTextGenerator")]
public class MyTextGenerator : BasicTextGeneratorSettings { }

However, we will also want to hook up a custom inspector to generate our output. Create a file named MyTextGenerator_Inspector.cs:

using UnityEditor;
using UnityEditor.UIElements;
using UnityEngine.UIElements;
using UnityEngine;
using System.Collections.Generic;
using System.IO;
using System.Collections.Generic;
using System;
using System.Collections;
using System.Text;
using com.recursiverhapsody;

[CustomEditor(typeof(MyTextGenerator))]
public class MyTextGenerator_Inspector : BasicTextGeneratorSettings_Inspector
{

    public override void OnGenerateClicked()
    {
        var ro = serializedObject.targetObject as MyTextGenerator;
        ro.SendRequest(delegate(ChatResponse result) {
            /// Process ChatResponse....
            loadingInProgress = false;
        }, OnError);
    }
}

The final step will be hooking up the UI layer. Select MyTextGenerator_Inspector in the editor, and drag the appropriate asset from Packages > AIAssetGeneration > Editor > UI. You will have one of three options for UI, TextGeneration, ImageGeneration and MultiImageGeneration. We will choose TextGeneration

From here, you can map the results of ChatResponse for whatever your own means are. You can reference the OpenAI reference for the meaning of these fields.

public class Message
{
    public string role;
    public string content;
}

public class Choice
{
    public int index;
    public Message message;
    public string finish_reason;
}

public class ChatResponse
{
    public string id;
    public int created;
    public List<Choice> choices;
}

API Calls

If you do not want to extend generators, you can manually make API calls. Parameter objects are designed to mirror the OpenAI API in name, default value, and purpose:

To generate an image:

public void SendRequest(Action<ImageResponse> action = null, Action<string> error = null)
{            
    var request = new ImageEditOpenAIRequest(APIKey, new ImageEditParameters () {
        image = ReferenceAsset,
        mask = MaskAsset,
        prompt = Prompt,
        size = size,
        n = NumberOfImages,
    });

    request.SendRequest(action, error);
}

To generate text:

public void SendRequest(Action<ChatResponse> action = null, Action<string> error = null)
{
    var model = CompletionModelNames.MapEnumToName(Model);
    var request = new ChatOpenAIRequest(APIKey, new ChatParameters () {
        model = model,
        messages = new List<Message>() {
            new Message() {
                role = Roles.User,
                content = Prompt,
            }
        }
    });

    request.SendRequest(action, error);
}