Using TileDB from Julia? Append columns, reshape


#1

How can I use TileDB from Julia? I’ve searched on Julia observer but haven’t find any package nor port?
Will it be possible from Julia or from JuliaDB?

I’m looking for a fast database to deal with datasets larger than memory (and be able to add new columns on disk and reshape data (from wide to long format). My candidates are hdf5, SciDB, MonetDB, Rasdaman and TileDB, the latter is supposed to be faster.


#2

We don’t have a Julia API for TileDB yet. We are certainly considering adding it in the future, and of course we welcome any contributions. Also reshaping the array domain and changing the data types are all valid features. We will be adding features more aggressively in the coming months as we grow our team.

Regarding your question around databases, unless you need in-database computations, you probably just need a lightweight storage library. So your candidates are TileDB and HDF5 if you are looking for C/C++ libraries, or Zarr which is a nice Python library. If your data is sparse though, TileDB is probably your only option.