Thread (19 messages) 19 messages, 4 authors, 2014-03-14
STALE4480d
Revisions (17)
  1. v2 [diff vs current]
  2. v2 [diff vs current]
  3. v2 [diff vs current]
  4. v2 [diff vs current]
  5. v3 [diff vs current]
  6. v3 current
  7. v3 [diff vs current]
  8. v3 [diff vs current]
  9. v3 [diff vs current]
  10. v3 [diff vs current]
  11. v3 [diff vs current]
  12. v3 [diff vs current]
  13. v3 [diff vs current]
  14. v3 [diff vs current]
  15. v3 [diff vs current]
  16. v4 [diff vs current]
  17. v5 [diff vs current]

[RFC PATCH V3 3/6] arm: mm: implement get_user_pages_fast

From: peterz@infradead.org (Peter Zijlstra)
Date: 2014-03-12 14:18:00

On Wed, Mar 12, 2014 at 01:40:20PM +0000, Steve Capper wrote:
+int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
+			  struct page **pages)
+{
+	struct mm_struct *mm = current->mm;
+	unsigned long addr, len, end;
+	unsigned long next, flags;
+	pgd_t *pgdp;
+	int nr = 0;
+
+	start &= PAGE_MASK;
+	addr = start;
+	len = (unsigned long) nr_pages << PAGE_SHIFT;
+	end = start + len;
+
+	if (unlikely(!access_ok(write ? VERIFY_WRITE : VERIFY_READ,
+					start, len)))
+		return 0;
+
+	/*
+	 * Disable interrupts, we use the nested form as we can already
+	 * have interrupts disabled by get_futex_key.
+	 *
+	 * With interrupts disabled, we block page table pages from being
+	 * freed from under us. See mmu_gather_tlb in asm-generic/tlb.h
+	 * for more details.
+	 */
+
+	local_irq_save(flags);
+	pgdp = pgd_offset(mm, addr);
+	do {
+		next = pgd_addr_end(addr, end);
+		if (pgd_none(*pgdp))
+			break;
+		else if (!gup_pud_range(pgdp, addr, next, write, pages, &nr))
+			break;
+	} while (pgdp++, addr = next, addr != end);
+	local_irq_restore(flags);
+
+	return nr;
+}
Since you just went through the trouble of enabling RCU pagetable
freeing, you might also replace these local_irq_save/restore with
rcu_read_{,un}lock().

Typically rcu_read_lock() is faster than disabling interrupts; but I've
no clue about ARM.
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help