History

swiotlb: fix slot alignment checks

Explicit alignment and page alignment are used only to calculate the stride, not when checking actual slot physical address. Originally, only page alignment was implemented, and that worked, because the whole SWIOTLB is allocated on a page boundary, so aligning the start index was sufficient to ensure a page-aligned slot. When commit 1f221a0d0dbf ("swiotlb: respect min_align_mask") added support for min_align_mask, the index could be incremented in the search loop, potentially finding an unaligned slot if minimum device alignment is between IO_TLB_SIZE and PAGE_SIZE. The bug could go unnoticed, because the slot size is 2 KiB, and the most common page size is 4 KiB, so there is no alignment value in between. IIUC the intention has been to find a slot that conforms to all alignment constraints: device minimum alignment, an explicit alignment (given as function parameter) and optionally page alignment (if allocation size is >= PAGE_SIZE). The most restrictive mask can be trivially computed with logical AND. The rest can stay.

Petr Tesarik committed 3 years ago

Commit 0eee5ae1 parent 39e7d2a1 file · +10 −6Browse files

Unified · Split

kernel/dma/swiotlb.c+10−6
@@ -634,22 +634,26 @@ static int swiotlb_do_find_slots(struct device *dev, int area_index, · 633 unmodified lines
634 BUG_ON(!nslots);634 BUG_ON(!nslots);
635 BUG_ON(area_index >= mem->nareas);635 BUG_ON(area_index >= mem->nareas);
636 636
637 /*
638 * For allocations of PAGE_SIZE or larger only look for page aligned
639 * allocations.
640 */
641 if (alloc_size >= PAGE_SIZE)
642 iotlb_align_mask &= PAGE_MASK;
643 iotlb_align_mask &= alloc_align_mask;
644
637 /*645 /*
638 * For mappings with an alignment requirement don't bother looping to646 * For mappings with an alignment requirement don't bother looping to
639 * unaligned slots once we found an aligned one. For allocations of647 * unaligned slots once we found an aligned one.
640 * PAGE_SIZE or larger only look for page aligned allocations.
641 */648 */
642 stride = (iotlb_align_mask >> IO_TLB_SHIFT) + 1;649 stride = (iotlb_align_mask >> IO_TLB_SHIFT) + 1;
643 if (alloc_size >= PAGE_SIZE)
644 stride = max(stride, stride << (PAGE_SHIFT - IO_TLB_SHIFT));
645 stride = max(stride, (alloc_align_mask >> IO_TLB_SHIFT) + 1);
646 650
647 spin_lock_irqsave(&area->lock, flags);651 spin_lock_irqsave(&area->lock, flags);
648 if (unlikely(nslots > mem->area_nslabs - area->used))652 if (unlikely(nslots > mem->area_nslabs - area->used))
649 goto not_found;653 goto not_found;
650 654
651 slot_base = area_index * mem->area_nslabs;655 slot_base = area_index * mem->area_nslabs;
652 index = wrap_area_index(mem, ALIGN(area->index, stride));656 index = area->index;
653 657
654 for (slots_checked = 0; slots_checked < mem->area_nslabs; ) {658 for (slots_checked = 0; slots_checked < mem->area_nslabs; ) {
655 slot_index = slot_base + index;659 slot_index = slot_base + index;

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t