A Fast and Reliable Real-Time Storage Interface Using File System Watcher

Storing, retrieving and manipulating data is one of the most common operation performed by on a normal computer system or a server machine. Data can be stored in different formats such as plain text, binary or custom formats (e.g. JSON, XML, HTML) and in different storages such as main memory (RAM), physical memory (a file system that interfaces a hard drive or any other physical storage) or on some cloud storage. The fastest way to access data on an average computer system would be to access it from its processors’ registers or dedicated cache memory, but this might not always be possible and such storages generally have a limited capacity. The next fastest way to access data is the main memory of a system. The average time needed to access the memory can vary from 0.5ns (90% of the time) to 10ns (10% of the time) [1]. However, the data stored in the main memory is temporary and is lost as the power supply gets disconnected. The next option, which is present in most systems is to use a file system that would ensure that the data is permanently stored but the time needed to access it would be comparatively longer. In order to create a fast and reliable storage interface, we’ll need to combine the positive points of both the main memory (fastest access time) and file storage system (reliability), with the help of a common file system watcher that keeps track of all the interfaces and this paper exactly discusses a concept based on this idea on an abstract level. The main memory shall be used to access the data and the file system to reliable store it. Whenever the data gets updated on the file system, the file system watcher shall notify all the relevant interfaces and update the main memory. The same notification(s) can then be used to make the storage interface real-time.