dae.backends.storage package
Submodules
dae.backends.storage.filesystem_genotype_storage module
- class dae.backends.storage.filesystem_genotype_storage.FilesystemGenotypeStorage(storage_config, section_id)[source]
Bases:
dae.backends.storage.genotype_storage.GenotypeStorage
A storage that uses the filesystem as its backend.
dae.backends.storage.genotype_storage module
dae.backends.storage.genotype_storage_factory module
dae.backends.storage.impala_genotype_storage module
- class dae.backends.storage.impala_genotype_storage.ImpalaGenotypeStorage(storage_config, section_id)[source]
Bases:
dae.backends.storage.genotype_storage.GenotypeStorage
Defines Apache Impala genotype storage.
- property hdfs_helpers
Create and return an HDFS helpers object.
- hdfs_upload_dataset(study_id, variants_dir, pedigree_file, partition_description)[source]
Upload a variants dir and pedigree file to hdfs.
- property impala_helpers
- impala_import_dataset(study_id, pedigree_hdfs_file, variants_hdfs_dir, partition_description, variants_sample=None, variants_schema=None)[source]
Create pedigree and variant tables for a study.
- impala_load_dataset(study_id, variants_dir, pedigree_file)[source]
Load a study data into impala genotype storage.
- property rsync_helpers
dae.backends.storage.schema2_genotype_storage module
- class dae.backends.storage.schema2_genotype_storage.HdfsStudyLayout(pedigree_file: str, summary_variant_dir: str, family_variant_dir: str, summary_sample: str, family_sample: str, meta_file: str)[source]
Bases:
object
- family_sample: str
- family_variant_dir: str
- meta_file: str
- pedigree_file: str
- summary_sample: str
- summary_variant_dir: str
- class dae.backends.storage.schema2_genotype_storage.Schema2GenotypeStorage(storage_config, section_id)[source]
Bases:
dae.backends.storage.genotype_storage.GenotypeStorage
A genotype storing implementing the new schema2.
- property hdfs_helpers
Return the hdfs helper used to interact with hdfs.
- hdfs_upload_dataset(study_id, variants_dir, pedigree_file, meta_file, partition_description) dae.backends.storage.schema2_genotype_storage.HdfsStudyLayout [source]
Upload local data to hdfs.
- property impala_helpers
- import_dataset(study_id, hdfs_study_layout: dae.backends.storage.schema2_genotype_storage.HdfsStudyLayout, partition_description)[source]
Load a dataset from HDFS into impala.