Struct Geode 

pub struct Geode {
    pub files_path: PathBuf,
    pub dirs_path: PathBuf,
}

Chunk-based file storage interface.

Fields

files_path: PathBuf

Path to the filesystem directory where file metadata is stored

dirs_path: PathBuf

Path to the filesystem directory where directory metadata is stored

Implementations

impl Geode

pub async fn new(base_path: &PathBuf) -> Result<Self>

Instantiate a new Geode object. base_path defines the root directory where Geode will store its file metadata and chunks.

async fn read_metadata( path: &PathBuf, ) -> Result<(Vec<Hash>, Vec<(PathBuf, u64)>)>

Attempt to read chunk hashes and files metadata from a given metadata path. This works for both file metadata and directory metadata. Returns (chunk hashes, [(file path, file size)]).

pub async fn garbage_collect(&self) -> Result<HashSet<Hash>>

Perform garbage collection over the filesystem hierarchy. Returns a set representing deleted files.

pub async fn chunk_stream( &self, stream: impl AsyncRead + Unpin, ) -> Result<(Hasher, Vec<Hash>)>

Chunk a stream. Returns a hasher (containing the chunk hashes), and the list of chunk hashes.

pub fn sort_files(&self, files: &mut [(PathBuf, u64)])

Sorts files by their PathBuf.

pub fn hash_chunks_metadata(&self, hasher: &mut Hasher, chunk_hashes: &[Hash])

Add chunk hashes to hasher.

pub fn hash_files_metadata( &self, hasher: &mut Hasher, relative_files: &[(PathBuf, u64)], )

Add files metadata to hasher. You must sort the files using sort_files.

pub async fn insert_metadata( &self, hash: &Hash, chunk_hashes: &[Hash], relative_files: &[(PathBuf, u64)], ) -> Result<()>

Create and insert file or directory metadata into Geode. Always overwrites any existing file. Verifies that the metadata is valid. The relative_files slice is empty for files.

pub async fn write_chunk( &self, chunked: &mut ChunkedStorage, stream: impl AsRef<[u8]>, ) -> Result<(Hash, usize)>

Write a single chunk given a stream. The file must be inserted into Geode before calling this method. Always overwrites any existing chunk. Returns the chunk hash and the number of bytes written to the file system.

pub async fn get(&self, hash: &Hash, path: &Path) -> Result<ChunkedStorage>

Fetch file/directory metadata from Geode. Returns ChunkedStorage. Returns an error if the read failed in any way (could also be the file does not exist).

async fn create_chunked_storage( &self, hash: &Hash, path: &Path, chunk_hashes: &[Hash], relative_files: &[(PathBuf, u64)], ) -> Result<ChunkedStorage>

Create a ChunkedStorage from metadata. hash is the hash of the file or directory.

pub async fn get_chunk( &self, chunked: &mut ChunkedStorage, chunk_hash: &Hash, ) -> Result<Vec<u8>>

Fetch a single chunk from Geode. Returns a Vec containing the chunk content if it is found. The returned chunk is NOT verified.

pub async fn read_chunk( &self, stream: impl AsyncRead + Unpin + AsyncSeek, chunk_index: &usize, ) -> Result<Vec<u8>>

Read the file at file_path to get its chunk with index chunk_index. Returns the chunk content in a Vec.

pub fn verify_metadata( &self, hash: &Hash, chunk_hashes: &[Hash], files: &[(PathBuf, u64)], ) -> bool

Verifies that the file hash matches the chunk hashes.

pub fn verify_chunk(&self, chunk_hash: &Hash, chunk_slice: &[u8]) -> bool

Verifies that the chunk hash matches the content.