Auction price prediction

An auction platform takes in items faster than anyone can value them by hand. The model gives each lot a price from what is already known about it: category, condition, location, weight, make, material, how many items are in the lot, and a handful of engineered features on top.

The first version of this was a hybrid: DistilBERT over the item text, a convolutional path over the photographs, and the structured fields alongside both. It got mean squared error down from 850 to 92, which was the number that made automatic lot bundling worth doing.

It is a different model now, and the interesting part of this project is what changed and why.

From a number to a range

The model no longer predicts a price. It predicts a distribution over a price.

The network ends in two heads rather than one: a mean, and a log variance. They are concatenated into a single tensor so the served signature stays one output, and the whole thing is trained on Gaussian negative log likelihood instead of mean squared error.

mu = y_pred[:, 0]
log_var = tf.clip_by_value(y_pred[:, 1], -4.0, 10.0)
return 0.5 * tf.reduce_mean(log_var + tf.square(y_true - mu) / tf.exp(log_var))

That objective is doing something MSE cannot. Under squared error the model is rewarded for being right on average and has no way to say it does not know. A common item in a familiar category and a strange one-off with a make nobody has seen before get the same confident treatment, and only one of those deserves it.

Under this loss the model has two ways to reduce error: predict better, or admit uncertainty. The variance term is what stops that being a free ride. Widening the interval reduces the squared error term but pays for it directly in the log_var term, so claiming uncertainty everywhere costs more than it saves. The minimum sits where the predicted spread matches the error the model actually makes, which is what makes the resulting interval mean something.

The clip on the log variance is not decoration. Without it, a handful of items the model finds genuinely baffling will drive the variance towards infinity, where the loss is finite and the gradient is not.

MSE stopped being the headline number as a result. Negative log likelihood is not readable as a quality measure, so mean absolute error on the mean head is carried as a monitoring metric alongside it. The loss trains the model, the MAE tells you how it is doing.

Getting the interval into the right space

The target is log-transformed before training, which matters more than it looks.

Prices are multiplicative. The difference between a $20 item and a $40 item is the same kind of difference as between $2,000 and $4,000, and a model trained on raw dollars spends its capacity on the expensive tail while treating everything cheap as approximately zero. Training on log price fixes that, and it also means the variance the model learns is a variance in log space.

So the interval is exponentiated back rather than added in dollars:

sigma = np.sqrt(np.exp(log_var))
prices  = np.expm1(mu)
lower_68 = np.expm1(mu - sigma)
upper_68 = np.expm1(mu + sigma)

The result is asymmetric on purpose. A prediction of $1,250 comes back with a range of roughly $820 to $1,900: less room below than above. That is the correct shape for a price. A symmetric dollar interval would put the lower bound below zero on cheap items and understate how far an expensive one can run.

The spread is also reduced to a single readable confidence score by taking exp(-sigma), which lands in zero to one. A sigma of 0.1 gives about 0.90, a sigma of 0.7 gives 0.50. An operator does not want to reason about log-space standard deviations, and a number that behaves like a percentage is something a user interface can act on.

What got switched off

The image path is gone and the text path is dormant.

DistilBERT is still in the model code, behind a configuration flag, with mean pooling over non-padding tokens rather than the CLS token because the text fields here are short. The two-phase training schedule that goes with it also survives: train the dense head with the encoder frozen, then unfreeze and fine-tune end to end at a much lower rate. Turning it back on is one flag.

Keeping a disabled path warm rather than deleting it is a judgement call and it is not always the right one, since dormant code rots. It is right here because the schedule encodes something that took experimentation to get right, and because the payload contract already carries the text fields.

Backwards compatible on purpose

This is the part I would point at. The model changed shape, gained an output, and changed its loss, and the application consuming it did not have to do anything at all.

The endpoint accepts three payload shapes: a wrapped batch array, a single bare object which is auto-wrapped, and a plain array. Categorical identifiers are accepted as integers, as strings, or as float-formatted strings, and normalised internally, so 1, "1" and "1.0" all mean the same thing. Unknown category values fall back to a default index instead of raising. Unrecognised fields in the payload are ignored rather than rejected. Text fields are still accepted even though nothing currently reads them.

The serving handler goes further and reads both model shapes. If the artifact returns two columns it produces the mean, the interval and the confidence score; if it returns one, it produces the price alone and reports zero confidence. Rolling back to the previous artifact does not require rolling back the handler.

Response shapes are preserved too, down to the nesting the C# client expects.

The reason for all of it is that a model improvement should not become a coordinated release across two teams. The endpoint gained a capability, every existing caller kept working unchanged, and the application can adopt the interval whenever it is ready rather than on my schedule.

Saying when not to trust it

Alongside the prediction, each instance comes back with a list of validation warnings and an unreliable flag: an unfamiliar category value, a numeric feature well outside the range seen in training, text that would have been truncated.

A prediction the model is confident about, made from a category it has never seen, is not the same thing as a prediction it is confident about. The uncertainty head covers what the model knows it does not know. The warnings cover the case where the input itself is off the map, which the variance will not always catch.

Where it is

Deployed on SageMaker behind a TensorFlow inference container, serving the platform’s intake.

The honest state of things is that the consuming application still reads the point estimate and discards the range, because it has not caught up yet. Which means the most useful thing the model now does, telling you when to send a lot to a human instead of pricing it automatically, is sitting in the response unused. The compatibility work above is what makes that fine rather than a problem: the capability is live and waiting, and adopting it is a change on one side only.