quoted

The only other kernel version I had available quickly is 3.16 from Debian
Jessie, and that works fine.

Thanks for reporting, I'll have a look.

I suspect this is coming from Keith+Ming changes in
blk_bio_segment_split()...

OK,

I can clearly see that the block layer commitment to respect the
driver virtual boundary was broken in 4.5.

 From the log:
iser: sg[0] dma_addr:0x85FC06000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[1] dma_addr:0x860334000 off:0x0 sz:0x200 dma_len:0x200 <-- gap
iser: sg[2] dma_addr:0x860335000 off:0x0 sz:0x200 dma_len:0x200 <-- gap
iser: sg[3] dma_addr:0x8621EA000 off:0x0 sz:0x200 dma_len:0x200 ...
iser: sg[4] dma_addr:0x8621EB000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[5] dma_addr:0x860384000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[6] dma_addr:0x860385000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[7] dma_addr:0x860316000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[8] dma_addr:0x860317000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[9] dma_addr:0x860294000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[10] dma_addr:0x860295000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[11] dma_addr:0x8609F8000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[12] dma_addr:0x8609F9000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[13] dma_addr:0x8607DA000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[14] dma_addr:0x8607DB000 off:0x0 sz:0x200 dma_len:0x200
iser: sg[15] dma_addr:0x8607D4000 off:0x0 sz:0x200 dma_len:0x200
--

While iser sets the virtual boundary to be 4096, we can
clearly see that each of the SG elements contain a gap
and should not ever see those...

I'm bisecting now, there are a couple of patches from Ming in
the area of the bio splitting code...

CC'ing Ming, Linux-block and Linux-nvme as iser is identical to nvme
wrt the virtual boundary so I think nvme will break as well.

Attaching a small test program I used to force gappy I/O.

$ ./scatter_data -l 64k -n 128 -d <dev>