Hi,

On 06/23/2016 04:26 PM, David Sterba wrote:

On Sat, Jun 18, 2016 at 12:01:53AM +0200, Hans van Kranenburg wrote:

quoted

proof-of-concept level scripts to be able to debug my btrfs file
systems, the inevitable happened:

https://github.com/knorrie/python-btrfs/

Currently, the primary goal of this module is to be able to inspect the
internals of an existing filesystem for educational purposes.

Looks great, I like.

Thanks, much appreciated!

Python is better language for prototyping various
things, also more convenient to write one-shot scripts to do specific
rescue tasks.

quoted

The python module acts as a wrapper around the low level kernel calls
and btrfs data structures, presenting them as python objects with
interesting attributes and references to other objects.

I guess it could be extended to read the filesystem objects directly
from the image, so it's not only built around the search ioctl.

quoted

Why?!

* Because it's fun!

Best reason ever!

Moo!

quoted

* Because I get sick of using horrible ducttape regex solutions to
programmatically parse human readable output of external tools to
implement monitoring tools~~!11one.

This is actually the main reason I started it. Or was it being able to 
see where balance got the usage information from...

quoted

* Because I might to be able to help other people that also want to
learn the internals of btrfs.
* Much more...

How?

* It's a python library, so the code will have a decent level of
pythonicity. The ioctl search returns a generator, the search key is a
nice object you can just increment etc...
* This is a work in progress. I can only add support for parts of btrfs
that I understand myself first.

My question is whether it sounds like a good idea to host the python
bindings inside btrfs-progs or not.

Whoa, hold your horses!

I can imagine creating scripts to
craft images for testing or verifying results of fsck, so it would be
more convenient to keep them in one place (but using a git submodule
instead would not be a big deal).

python-btrfs is just a library. You can just clone the git somewhere, 
start hacking on it and put it in your python path, or pip install or 
apt-get install python-btrfs etc...

Programs that use it can be anywhere. What I have in examples/ now 
should probably end up either in howto style documentation or in proper 
tests for the lib (which requires keeping known-state filesystem images 
to test on etc, exploding into a new level of whoa).

I'm mostly interested in the object representation of the filesystem
objects and basic API around that, so if you have other plans with
python-btrfs, we can at least share some of the code. My python-fu is
not that strong so I'd rely on you or others to maintain that if you're
interested.

Ok, I added a little TODO:
https://github.com/knorrie/python-btrfs/blob/master/TODO.md

There's a few things I want to do first at least, which are completing 
to support the full set of data structures using the kernel API, 
finishing to move existing code that I have lying around to use the new 
lib and a bit of extra info to help interested sysadmin/programmers to 
start using the library to peek around in their own filesystem a bit to 
see what's going on.

I haven't looked at how the btrfs-progs treat disks directly, but would 
certainly be interested in learning about that.

 From what you say, I feel that you're thinking about not only looking 
around in filesystems, but having ways to manipulate them?

Can you provide a more concrete example of "craft images for testing" or 
related ideas about how this project could help during regular btrfs 
development?

-- 
Hans van Kranenburg - System / Network Engineer
T +31 (0)10 2760434 | hans.van.kranenburg@mendix.com | www.mendix.com