Coherent or streaming DMA is not a style choice: three ways ownership gets broken

A walkthrough of the Linux DMA API published on dev.to sets out the contract a driver has to honour, and names the three ways it usually gets broken: the ownership rule of streaming mappings, DMA to memory that is not DMA-able, and cache-line sharing between a DMA buffer and CPU-written fields ^[1]. The reason to care is the failure signature the piece describes: a driver that looks correct, passes a quick test, and corrupts data months later on a different board ^[2].

The API exists because the address a device puts on the bus is not the address the CPU uses, and because on most embedded SoCs the CPU cache is not kept in step with device accesses ^[16]. There are three addresses in play: the CPU virtual address from kmalloc() or ioremap(), the CPU physical address as phys_addr_t, and the dma_addr_t that the device sees, which an IOMMU or host bridge may translate into something completely different ^[3]. On a simple SoC those last two are often the same value, which is precisely why a broken driver runs fine on one board and fails on the next one with an IOMMU or a bus offset ^[4].

The two mapping styles are not interchangeable. dma_alloc_coherent() hands back a CPU pointer and a dma_addr_t for memory both sides can touch without explicit cache maintenance, intended for small long-lived structures allocated at probe ^[5]; the kernel documentation's examples are ring descriptors, adapter mailboxes, and device firmware executed out of main memory, where flushing on every access would be unreasonable ^[7]. Streaming mappings, dma_map_single() and dma_map_sg(), lend an existing buffer to the device for one transfer, with the kernel performing the clean or invalidate at map, sync and unmap time ^[6].

That list of maintenance points is the ownership rule. If cache maintenance happens only at map, sync and unmap, then any CPU read or write in between has no corresponding clean or invalidate behind it, and the two views of the buffer diverge silently ^[14].

The second failure is decided before any transfer starts. Page allocator memory, kmalloc() and kmem_cache_alloc() are DMA-able; vmalloc(), kernel and module image addresses, stack addresses and the return of kmap() are not ^[8]. A DMA buffer on the stack is the classic case: it can appear to work on a coherent x86 test machine and then corrupt neighbouring stack data on an ARM board with non-coherent caches ^[9]. The remedy given is kmalloc(), or embedding the buffer in a device structure with the cache-line isolation the kernel provides ^[10]. Two of the three named causes, non-DMA-able memory and cache-line sharing, are settled at allocation time, before there is any sync call to get wrong ^[15].

Two smaller traps. Declare addressing capability up front, for example dma_set_mask_and_coherent() with DMA_BIT_MASK(32), since the default assumption is 32-bit ^[11]; the documentation notes the call does not fail for masks larger than 32 bits, so "try 64, fall back to 32" is not the recommended pattern and the mask should come from what the hardware actually does ^[12]. And coherent does not mean ordered: the CPU can still reorder stores to coherent memory, with the article's example being a device that must see word0 before word1 is marked valid ^[13].

What to watch in your own tree: buffers on the stack or from vmalloc() passed to a map call ^[8], masks copied from another driver rather than derived from the datasheet ^[12], and CPU touches of a streaming buffer between map and unmap ^[14]. Validation on identity-mapped SoCs will not surface any of it ^[4].