Hello!

atomic_inc_and_test is true iff result is zero, so that won't work.

I meant atomic_inc_not_zero(), as Martin noticed.

But the following should work:

	hlist_for_each_entry_rcu(n, tmp, &tbl->hash_buckets[hash_val], hlist) {
		if (dev == n->dev && !memcmp(n->primary_key, pkey, key_len)) {
			if (unlikely(&atomic_inc_return(&n->refcnt) == 1)) {
				neigh_release(n);

I do not think it will work. It has exactly the same race condition.

Yes, atomic_inc_not_zero() is expensive. But it looks like it is the cheapest
variant, which works correctly without more work.

Another variant would be rework use of refcnt. It can be done like rt cache:
when release of the last reference does not mean anything.

Also, probably, it makes sense to add neigh_lookup_light(), which does
not take refcnt, but required to call
neigh_release_light() (which is just rcu_read_unlock_bh()).

Alexey