Efficient Data Selective Scheme for Parallel and Distributed Environment

Now a Days, many scientific applications spend a significant portion of their execution time in accessing data from files. Various optimization techniques exist to improve data access performance, such as data perfecting and data layout optimization. However, optimization process is usually a difficult task due to the complexity involved in understanding I/O behaviour. Tools that can help simplify the optimization process have a significant importance. In this paper, we introduce a tool, called IOSIG, for providing a better understanding of parallel I/O accesses and information to be used for optimization techniques. The tool enables tracing parallel I/O calls of an application and analysing the collected information to provide a clear understanding of I/O behaviour of the application. We show that performance overheads of the tool in trace collection and analysis are negligible. The analysis step creates I/O signatures that various optimizations can use for improving I/O performance. Input and output signatures are compact, easy-to-understand, and parameterized representations containing data access pattern information such as size, strides between consecutive accesses, repetition, timing, etc. The signatures include local I/O behaviour for each process and global behaviour for an overall application. We illustrate the usage of the IOSIG tool in data perfecting and data layout optimizations.