libjio Programmer's Guide Alberto Bertogli (albertogli@telpin.com.ar) Table of Contents 1 Introduction 2 Definitions 3 The data types 4 The basic functions 5 Advanced functions 5.1 Interaction with reads 5.2 Rollback 5.3 Integrity checking and recovery 5.4 Threads and locking 5.5 Lingering transactions 6 Disk layout 7 Other APIs 7.1 UNIX API 7.2 ANSI C API 8 Compiling and linking 9 Where to go from here 1 Introduction This small document attempts serve as a guide to the programmer who wants to make use of the library. It's not a replacement for the man page or reading the code; but it's a good starting point for everyone who wants to get involved with it. The library is not complex to use at all, and the interfaces were designed to be as intuitive as possible, so the text is structured as a guide to present the reader all the common structures and functions the way they're normally used. 2 Definitions This is a library which provides a journaled transaction-oriented I/O API. You've probably read this a hundred times already in the documents, and if you haven't wondered yet what on earth does this mean you should be reading something else! We say this is a transaction-oriented API because we make transactions the center of our operations, and journaled because we use a journal (which takes the form of a directory with files on it) to guarantee coherency even after a crash at any point. Here we think a transaction as a list of (buffer, length, offset) to be applied to a file. That triple is called an operation, so we can say that a transaction represent an ordered group of operations on the same file. The act of committing a transaction means writing all the elements of the list; and rollbacking means to undo a previous commit, and leave the data just as it was before doing the commit.While all this definitions may seem obvious to some people, it requires special attention because there are a lot of different definitions, and it's not that common to see "transaction" applied to file I/O (it's a term used mostly on database stuff), so it's important to clarify before continuing. It's important to note that the library not only provides a convenient and easy API to perform this kind of operations, but provides a lot of guarantees while doing this. The most relevant and useful is that at any point of time, even if we crash horribly, a transaction will be either fully applied or not applied at all. You should not ever see partial transactions or any kind of data corruption. To achieve this, the library uses what is called a journal, a very vague (and fashionable) term we use to describe a set of auxiliary files that get created to store temporary data at several stages. The proper definition and how we use them is outside the scope of this document, and you as a programmer shouldn't need to deal with it. In case you're curious, it's described in a bit more detail in another text which talks about how the library works internally. Now let's get real. 3 The data types To understand any library, it's essential to be confident in the knowledge of their data structures and how they relate each other. In libjio we have two basic structures which have a very strong relationship, and represent the essential objects we deal with. Note that you normally don't manipulate them directly, because they have their own initializer functions, but they are the building blocks for the rest of the text, which, once this is understood, is obvious and self-evident. The first structure we face is struct jfs, called the file structure, and it represents an open file, just like a regular file descriptor or a FILE *. Then you find struct jtrans, called the transaction structure, which represents a single transaction. You can have as many transactions as you want, and operate on all of them simultaneously without problems; the library is entirely thread safe so there's no need to worry about that. 4 The basic functions Now that we've described our data types, let's see how we can really operate with the library. First of all, as with regular I/O, you need to open your files. This is done with jopen(), which looks a lot like open() but takes a file structure instead of a file descriptor (this will be very common among all the functions), and adds a new parameter jflags that can be used to modify some subtle library behaviour we'll see later, and it's normally not used. We have a happy file structure open now, and the next thing to do would be to create a transaction. This is what jtrans_init() is for: it takes a file structure and a transaction structure and initializes the latter, leaving it ready to use. So we have our transaction, let's add a write operation to it; to do this we use jtrans_add(). We could keep on adding operations to the transaction by keep on calling jtrans_add() as many times as we want. Finally, we decide to apply our transaction to the file, that is, write all the operations we've added. And this is the easiest part: we call jtrans_commit(), and that's it! When we're done using the file, we call jclose(), just like we call close(). Let's put it all together and code a nice "hello world" program (return values are ignored for simplicity): char buf[] = "Hello world!"; struct jfs file; struct jtrans trans; jopen(&file, "filename", O_RDWR | O_CREAT, 0600, 0); jtrans_init(&file, &trans); jtrans_add(&trans, buf, strlen(buf), 0); jtrans_commit(&trans); jclose(&file); As we've seen, we open the file and initialize the structure with jopen() (with the parameter jflags being the last 0)and jtrans_init(), then add an operation with jtrans_add() (the last 0 is the offset, in this case the beginning of the file), commit the transaction with jtrans_commit(), and finally close the file with jclose(). 5 Advanced functions 5.1 Interaction with reads So far we've seen how to use the library to perform writes, but what about reads? The only and main issue with reads is that, because we provide transaction atomicity, a read must never be able to "see" a transaction partially applied. This is achieved internally by using fine-grained file locks; but you shouldn't mind about it if you use the functions the library gives you because they take care of all the locking. This set of functions are very similar to the UNIX ones (read(), readv(), etc.); and in fact are named after them: they're called jread(), jreadv() and jpread(); and have the same parameters except for the first one, which instead of a file descriptor is a file structureIn fact, this set of functions is a part of what is called the "UNIX API", which is described below. . Bear in mind that transactions are only visible by reads after you commit them with jtrans_commit(). 5.2 Rollback There is a very nice and important feature in transactions, that allow them to be "undone", which means that you can undo a transaction and leave the file just as it was the moment before applying it. The action of undoing it is called to rollback, and the function is called jtrans_rollback(), which takes the transaction as the only parameter. Be aware that rollbacking a transaction can be dangerous if you're not careful and cause you a lot of troubles. For instance, consider you have two transactions (let's call them 1 and 2, and assume they were applied in that order) that modify the same offset, and you rollback transaction 1; then 2 would be lost. It is not an dangerous operation itself, but its use requires care and thought. 5.3 Integrity checking and recovery An essential part of the library is taking care of recovering from crashes and be able to assure a file is consistent. When you're working with the file, this is taking care of; but what when you first open it? To answer that question, the library provides you with a function named jfsck(), which checks the integrity of a file and makes sure that everything is consistent. It must be called "offline", that is when you are not actively committing and rollbacking; it is normally done before calling jopen(). Another good practise is call jfsck_cleanup() after calling jfsck() to make sure we're starting up with a fresh clean journal. After both calls, it is safe to assume that the file is and ready to use. You can also do this manually with an utility named jiofsck, which can be used from the shell to perform the checking and cleanup. 5.4 Threads and locking The library is completely safe to use in multithreaded applications; however, there are some very basic and intuitive locking rules you have to bear in mind. Most is fully threadsafe so you don't need to worry about concurrency; in fact, a lot of effort has been put in making paralell operation safe and fast. You need to care only when opening, closing and checking for integrity. In practise, that means that you shouldn't call jopen(), jclose() in paralell with the same jfs structure, or in the middle of an I/O operation, just like you do when using the normal UNIX calls. In the case of jfsck(), you shouldn't invoke it for the same file more than once at the time; while it will cope with that situation, it's not recommended. All other operations (commiting a transaction, rollbacking it, adding operations, etc.) and all the wrappers are safe and don't require any special considerations. 5.5 Lingering transactions If you need to increase performance, you can use lingering transactions. In this mode, transactions take up more disk space but allows you to do the synchronous write only once, making commits much faster. To use them, just add J_LINGER to the jflags parameter in jopen(). It is very wise to call jsync() frequently to avoid using up too much space. 6 Disk layout The library creates a single directory for each file opened, named after it. So if we open a file "output", a directory named ".output.jio" will be created. We call it the journal directory, and it's used internally by the library to save temporary data; you shouldn't modify any of the files that are inside it, or move it while it's in use. It doesn't grow much (it only uses space for transactions that are in the process of committing) and gets automatically cleaned while working with it so you can (and should) ignore it. Besides that, the file you work with has no special modification and is just like any other file, all the internal stuff is kept isolated on the journal directory. 7 Other APIs We're all used to do things our way, and when we learn something new it's often better if it looks alike what we already know. With this in mind, the library comes with two sets of APIs that look a lot like traditional, well known ones. Bear in mind that they are not as powerful as the transaction API that is described above, and they can't provide the same functionality in a lot of cases; however for a lot of common and simple use patterns they are good enough. 7.1 UNIX API There is a set of functions that emulate the UNIX API (read(), write(), and so on) which make each operation a transaction. This can be useful if you don't need to have the full power of the transactions but only to provide guarantees between the different functions. They are a lot like the normal UNIX functions, but instead of getting a file descriptor as their first parameter, they get a file structure. You can check out the manual page to see the details, but they work just like their UNIX version, only that they preserve atomicity and thread-safety within each call. In particular, the group of functions related to reading (which was described above in [sub:Interaction-with-reads]) are extremely useful because they take care of the locking needed for the library proper behaviour. You should use them instead of the regular calls. The full function list is available on the man page and I won't reproduce it here; however the naming is quite simple: just prepend a 'j' to all the names: jread(), jwrite(), etc. 7.2 ANSI C API Besides the UNIX API you can find an ANSI C API, which emulates the traditional fread(), fwrite(), etc. They're still in development and has not been tested carefully, so I won't spend time documenting them. Let me know if you need them. 8 Compiling and linking When you want to use your library, besides including the "libjio.h" header, you have to make sure your application uses the Large File Support ("LFS" from now on), to be able to handle large files properly. This means that you will have to pass some special standard flags to the compiler, so your C library uses the same data types as the library. For instance, on 32-bit platforms (like x86), when using LFS, offsets are usually 64 bits, as opposed to the usual 32. The library is always built with LFS; however, link it against an application without LFS support could lead to serious problems because this kind of size differences and ABI compatibility. The Single Unix Specification standard proposes a simple and practical way to get the flags you need to pass your C compiler to tell you want to compile your application with LFS: use a program called "getconf" which should be called like "getconf LFS_CFLAGS", and it outputs the appropiate parameters. Sadly, not all platforms implement it, so it's also wise to pass " -D_FILE_OFFSET_BITS=64" just in case. In the end, the command line would be something like: gcc `getconf LFS_CFLAGS` -D_FILE_OFFSET_BITS=64 \ app.c -ljio -lpthread -o app If you want more detailed information or examples, you can check out how the library and sample applications get built. 9 Where to go from here If you're still interested in learning more, you can find some small and clean samples are in the "samples" directory (full.c is a simple and complete one), other more advanced examples can be found in the web page, as well as modifications to well known software to make use of the library. For more information about the inner workings of the library, you can read the "libjio" document, and the source code.