Skip to content

Build1 publisher3 min readPublished

Three lines of trig let a DynamoDB vector index rank shops by great-circle distance

A dev.to test stores each shop's position as a unit-sphere triple and gets ordering that matches haversine to about half a metre across seven places, while the geohash index it was compared against missed the nearest shop.

The Engineer · Build desk

Illustration accompanying Three lines of trig let a DynamoDB vector index rank shops by great-circle distance

What happened

  • On Serverless Advocate #85, Lee Harding named DynamoDB vector indexes as the AWS service he is most excited about, and said his interest is not embeddings.
  • Harding described a vector index as a general-purpose tool for finding nearest neighbours in any n-dimensional space, which the post then tested on the plainest geography question there is.
  • Because degrees are not a usable space, each shop's latitude and longitude are projected onto the unit sphere with three lines of trig and stored as a list attribute on the item.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • capability A table that already holds coordinates becomes distance-searchable by adding one list attribute and an index over it, with no embedding model and no second datastore in the request path.
  • constraint The geohash design makes recall depend on where the caller is standing: with the cell as the GSI partition key, anything one cell over is unreachable by that single Query.
  • decision Teams now choose between one cheap Query that is exact only inside its own cell and a vector search that ranks the whole table, and that choice is about correctness, not about semantics.

The reason you cannot hand a vector index a latitude and a longitude is that degrees are not a space. Cosine similarity measures the angle your pair makes with the origin, and the origin here is (0, 0). Nothing is measured from there, so two places on opposite sides of the world can score as basically identical [6]. Euclidean is less daft and still wrong. A degree of longitude covers 111 km at the equator and 66 km at Leeds, roughly 59 percent of the equator figure [7][3], so a circle in degree space is an ellipse on the ground [7]. "It's the space that's broken, not the metric," the post wrote [8].

The fix is a projection: x = cos(lat) * cos(lon), y = cos(lat) * sin(lon), z = sin(lat) [9]. The triple goes on the item as an attribute called `position`, an ordinary list of numbers sitting beside the name and the opening hours, because DynamoDB has no vector type [10]. Every shop is then a point on a ball of radius one. The straight-line gap between two such points is 2 * sin(theta / 2), which climbs steadily as theta goes from zero to pi and never doubles back, so ordering by it is the true great-circle ordering [11]. Converting a score back with 2 * 6371 * asin(score / 2) agreed with haversine to about half a metre across the seven places [12]. Both sides of that comparison are spherical formulas using the same 6371 km radius. The half metre is the projection and the reconstruction agreeing with each other, not a check against surveyed ground distance.

The usual DynamoDB answer is a geohash, which interleaves the two coordinates into one string so that nearby places share a leading prefix [13]. Briggate's full hash is `gcwfhct3`. The post takes the first four characters as a cell, keys a GSI on `GSI1PK = GEO#gcwf`, and keeps the whole hash plus the id as the sort key so neighbours inside a cell order by position [14]. One Query answers "what's near me", and the post calls it cheap and exact [15]. The cell is the partition key, so a shop one cell over is not ranked lower, it is in a different partition and the query never goes near it [16].

From the concourse at Leeds station, inside cell `gcwf` [17], that cost the geohash the nearest result. Holbeck sits about a mile south, and the cell boundary runs between it and the station. The query returned four shops and missed the one at 0.92 km. It also included Hyde Park at two and a half times that distance, about 2.3 km, because it happens to share four characters [18][1]. The cell is roughly 23 km by 19 km, some 437 square kilometres [19][2]. The post notes that any query near an edge has the same problem, and most queries are near an edge [19].

The test is also tiny: seven items, one query point, and no figures for query cost or latency [4][17]. For the exact-ordering property the size of the table is beside the point, because monotonicity of the chord is arithmetic [11]. For anything about price or tail latency at a real locator's item count, this is a demonstration and nothing more. Both access patterns are loaded in a console in the post, if you want to run them yourself [20].

What to watch

  • Query cost and latency for a DynamoDB vector index at real locator item counts; the post reports neither.
  • A rerun with the query point in the middle of a cell rather than near its edge, where the geohash should match the projection exactly.
  • Whether the same projection approach appears for the other domains Harding named, such as sensor fusion.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories