ChatGPT o1-preview excels at code generation

“Ho, hum,” I thought in response to the release of the OpenAI o1 models. “Yet another stochastic parrot.” I was wrong. In fact, I tried the o1-preview model on one of my standard programming tasks, and found it to be surprisingly good.

The recently released ChatGPT o1-preview model is an early version of the much-rumored “Strawberry” model. I tested it out by asking it to implement a QuickSort algorithm as a C++ function with integer inputs, a task that has stymied many earlier models, and to write a test suite for the function. Its first response was entirely correct, a big improvement over what I’d seen in the past, and included explanations of what it was doing, but I wasn’t satisfied: I pushed the model for improvements multiple times.

After considerable back and forth, and an actual model crash from which o1-preview eventually recovered, I wound up with a robust, optimized QuickSort function using modern C++, which incorporated randomized pivot selection, gated insertion sorting of small runs, Dutch-flag three-way partitioning to handle duplicates, exception handling, parallelization of large arrays using threads, and a well-factored bank of tests. That result would have been a good day’s work by a smart junior C++ programmer who had read Donald Knuth’s volume on sorting and searching.

All told, ChatGPT o1-preview generated five versions of QuickSort in C++ for me, each increasingly better. Below is the record of my prompts of the model and its responses, including its reasoning steps, code, explanations, and tests.

ChatGPT o1-preview QuickSort: First try

Me: Write a C++ function to implement a QuickSort of integers. Also write a test suite for this function.

ChatGPT: Thought for 16 seconds…

Crafting QuickSort

I’m piecing together a C++ function for QuickSort on integers, including a test suite to ensure its accuracy.

Crafting the solution

OK, let me see. Writing a QuickSort function and a test suite in C++ involves coding the algorithm and testing its accuracy with various inputs.

Assessing QuickSort functionality

I’m digging into the QuickSort function’s performance by testing an empty array, single element, sorted array, reverse sorted array, random array, array with duplicates, and a large random array of 100 numbers.

Setting up and testing

I’m including headers, writing swap and partition functions, and setting up QuickSort. Next, I’ll test it with various array scenarios, including empty, single element, and large arrays.

————

Sure! Below is a C++ implementation of the QuickSort algorithm for sorting integers, along with a comprehensive test suite to verify its correctness.

#include
#include
#include // For rand()
#include // For time()

// Function to swap two integers
void swap(int& a, int& b) {
int temp = a;
a = b;
b = temp;
}

// Partition function used by QuickSort
int partition(int arr[], int low, int high) {
int pivot = arr[high]; // Choosing the last element as the pivot
int i = (low - 1); // Index of the smaller element

for (int j = low; j