Tobacco barn detection from satellite imagery

Tobacco curing barns are small rectangular structures, and where they cluster tells you something useful about land use. Some areas have been surveyed on the ground, which produces a file of coordinates. Most have not, which is the problem: you want to find the barns nobody has walked to.

That is an object detection problem, and object detection is bottlenecked on labelled data. Someone has to draw a box around every instance in every image, and for a thing as small and repetitive as a barn that is thousands of boxes before you learn anything.

This project’s premise was that the survey already contains the labels.

Labels from a survey rather than from a mouse

The ground survey arrives as KML: a placemark per structure, with a name and a latitude and longitude. The pipeline reads it, filters to the tobacco barns, and then does two things with those coordinates.

First it uses them to decide where to look, generating a capture position per barn at a randomised zoom, with up to fifteen metres of jitter applied to the camera position. The jitter matters. If every training image is exactly centred on its barn, the model has an easy shortcut available and will take it, learning “the barn is in the middle of the frame” rather than what a barn looks like.

Second, and this is the actual idea, it uses the same coordinates to write the labels. For each captured image it computes the visible ground extent from the camera altitude and a field-of-view factor, derives metres per pixel, and then projects every known barn into that image’s pixel space. Anything that lands inside the frame becomes a YOLO bounding box. No human draws anything.

The nice property is that it labels barns the operator was not aiming at. If a capture centred on one barn happens to contain three others, all four get labelled, because the pipeline is working from the full survey rather than from what someone noticed while annotating.

The projection is where it gets difficult

Flat-earth trigonometry puts a barn in roughly the right place and not the exact right place, because satellite imagery is not an orthographic projection. There is radial distortion, and there is perspective foreshortening, and both grow with distance from the centre of the frame.

So the projection carries a correction that scales with radial distance from centre, pulling outlying objects back in, with a separate compression factor on the vertical axis. Box sizes are treated the same way: a base physical size in metres converted to pixels, then scaled down slightly with distance from centre because things further out appear smaller, then clamped so no box is absurd.

The honest part is how those constants were arrived at. There is a calibration routine in the code whose instructions are, essentially: generate the labels, look at them against the imagery, and if the boxes drift outward from centre reduce this number, if they drift inward increase it, if the sizes are all wrong adjust the field-of-view factor. It is a human in a loop turning knobs until the overlay looks right.

That works, and it is also the project’s central weakness, which I will come back to.

Two datasets that ask different questions

The training captures are barn-centred by construction: aim at a known barn, jitter slightly, capture. Nearly every image contains a barn near the middle.

Evaluating on data shaped like that would be self-deception, so there is a second generator that tiles a region into a fixed grid at a constant zoom with fifteen percent overlap between neighbours, and labels those tiles from the same survey. A grid tile contains whatever it contains: no barns, one, or several.

That is the shape of the real task. You do not get to point the camera at the answer. You sweep an area and ask what is in it. Building the evaluation set to match the deployment condition rather than the training condition is the methodological decision I would keep from this project.

What the numbers say

They say it did not work well enough.

After 150 epochs on 99 images, the detector reached 58% precision and 39% recall, with mean average precision at IoU 0.5 of 0.42, at the standard confidence threshold. Precision peaked around 63% mid-training and settled below that.

For context on what those mean here: at 58% precision, two out of every five detections are not barns. At 39% recall, the majority of real barns are missed entirely. That is a prototype demonstrating a pipeline, not a system anyone should point at unsurveyed land.

Why it fell short

Three causes, and they compound.

The dataset is tiny. Ninety-nine images is not a detection dataset. The pipeline is capable of generating far more, since it is bounded only by survey size and capture throughput, but this run was not given more.

The label noise is systematic, not random. This is the one that matters most and it follows directly from the calibration-by-eye approach. Random label jitter mostly averages out during training. A projection tuned by hand carries a consistent bias, so every box in the set is wrong in the same direction by a similar amount, and a detector trained on consistently offset boxes learns the offset as if it were signal.

Box sizes are assumed rather than measured. Every barn is labelled at the same base physical footprint, adjusted only for position in frame. Real barns vary, so the model is being taught a size distribution that does not exist, and the IoU-based metrics punish that directly. The gap between mAP at 0.5 and the much lower figure at stricter thresholds is consistent with boxes that are roughly in the right place and the wrong shape.

The fix for all three is the same and it is unglamorous: hand-label a few hundred images properly, use them to measure how far off the projection is instead of eyeballing it, and use the corrected projection to generate the rest. The annotation tooling for that is sitting in the repository and was set up but not carried through.

What I would keep

The idea, which is that a GPS survey and an imagery source are together a labelled dataset generator, and that a small amount of ground truth can be amplified into a large amount of training data. That part is sound and transfers to anything where the objects of interest have been catalogued somewhere: infrastructure, water points, cell towers, informal settlements.

What this run proves is the pipeline end to end and not the detector. Reporting it otherwise would make the next person repeat it.