Boaz Harrosh <boaz <at> plexistor.com> writes:

On 02/26/2016 12:04 PM, Thanumalayan Sankaranarayana Pillai wrote:

quoted

On Thu, Feb 25, 2016 at 10:02 PM, Dan Williams <dan.j.williams <at>

intel.com> wrote:

quoted

quoted

[ adding Thanu ]

quoted

Very few applications actually care about atomic sector writes.
Databases are probably the only class of application that really do
care about both single sector and multi-sector atomic write
behaviour, and many of them can be configured to assume single
sector writes can be torn.

Torn user data writes have always been possible, and so pmem does
not introduce any new semantics that applications have to handle.

I know about BTT and DAX only at a conceptual level and hence do not

understand

quoted

this mailing thread fully. But I can provide examples of important

applications

quoted

expecting atomicity at a 512B or a smaller granularity. Here is a list:

(1) LMDB [1] that Dan mentioned, which expects "linear writes" (i.e., don't
need atomicity, but need the first byte to be written before the second

byte)

quoted

(2) PostgreSQL expects atomicity [2]

(3) SQLite depends on linear writes [3] (we were unable to find these
dependencies during our testing, however). Also, PSOW in SQLite is not

relevant

quoted

to this discussion as I understand it; PSOW deals with corruption of data
*around* the actual written bytes.

(4) We found that ZooKeeper depends on atomicity during our testing, but

we did

quoted

not contact the ZooKeeper developers about this. Some details in our

paper [4].

quoted

It is tempting to assume that applications do not use the concept of disk
sectors and deal with only file-system blocks (which are not atomic in
practice), and take measures to deal with the non-atomic file-system blocks.
But, in reality, applications seem to assume that 512B (more or less)

sectors

quoted

are atomic or linear, and build their consistency mechanisms around that.

This all discussion is a shock to me. where were these guys hiding, under

a rock?

In the NFS world you can get not torn sectors but torn words. You may have
reorder of writes, you may have data holes the all deal. Until you get back
a successful sync nothing is guarantied. It is not only a client
crash but also a network breach, and so on. So you never know what can happen.

So are you saying all these applications do not run on NFS?

Speaking for LMDB: LMDB is entirely dependent on mmap, and the coherence of
a unified buffer cache. None of this is supported on NFS, so NFS has never
been a concern for us. We explicitly document that LMDB cannot be used over NFS.

Speaking more generally, you're talking nonsense. NFS by default transmits
*pages* over UDP - datagrams are all-or-nothing, you can't get torn words.
Likewise, NFS over TCP means individual pages are transmitted with
individual bytes in order within a page.

Thanks
Boaz

quoted

[1] http://www.openldap.org/list~s/openldap-devel/201410/msg00004.html
[2] http://www.postgresql.org/docs/9.5/static/wal-internals.html , "To deal
with the case where pg_control is corrupt" ...
[3] https://www.sqlite.org/atomiccommit.html , "SQLite does always

assume that

quoted

a sector write is linear" ...
[4] http://research.cs.wisc.edu/wind/Publications/alice-osdi14.pdf

Regards,
Thanu
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm <at> lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm

--
  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>