gms_preprocessing.io package

Submodules

gms_preprocessing.io.input_reader module

GeoMultiSens Input Reader: Universal reader for all kinds of GeoMultiSens intermediate results.

class gms_preprocessing.io.input_reader.DEM_Creator(dem_sensor='SRTM', db_conn='', logger=None)[source]

Bases: object

Creator class for digital elevation models based on ASTER or SRTM.

Parameters

  • dem_sensor – ‘SRTM’ or ‘ASTER’

  • db_conn – database connection string

from_extent(cornerCoords_tgt, prj, tgt_xgsd, tgt_ygsd)[source]

Returns a GeoArray of a DEM according to the given target coordinates

Parameters

  • cornerCoords_tgt – list of target coordinates [[X,Y], [X,Y], …]] (at least 2 coordinates)

  • prj – WKT string of the projection belonging cornerCoords_tgt

  • tgt_xgsd – output X GSD

  • tgt_ygsd – output Y GSD

Returns

DEM GeoArray

get_overlapping_DEM_tiles(tgt_corner_coord_lonlat, timeout_sec=30, use_index_mediator=True)[source]

Get the overlapping DEM tiles for the given extent.

Parameters

  • tgt_corner_coord_lonlat (List[Tuple]) – list of longitude/latitude target coordinates [[X,Y], [X,Y], …]]

  • timeout_sec (int) – database query timeout (seconds)

  • use_index_mediator (bool) – whether to use or not to use the SpatialIndexMediator (default: True)

Return type

list

Returns

list of matching DEM tile scene IDs

gms_preprocessing.io.input_reader.GMSfile2dict(path_GMSfile)[source]

Converts a JSON file (like the GMS file) to a Python dictionary with keys and values.

Parameters

path_GMSfile – absolute path on disk

Returns

the corresponding Python dictionary

gms_preprocessing.io.input_reader.SRF_Reader(GMS_id, no_thermal=None, no_pan=None, v=False)[source]

Read SRF for any sensor and return a dictionary containing band names as keys and SRF numpy arrays as values.

Parameters

  • GMS_id (GMS_identifier) –

  • no_thermal (bool) – whether to exclude thermal bands from the returned bands list (default: CFG.skip_thermal)

  • no_pan (bool) – whether to exclude panchromatic bands from the returned bands list (default: CFG.skip_pan)

  • v (bool) – verbose mode

Return type

collections.OrderedDict

gms_preprocessing.io.input_reader.Solar_Irradiance_reader(resol_nm=None, wvl_min_nm=None, wvl_max_nm=None)[source]

Read the solar irradiance file and return an array of irradiances.

Parameters

  • resol_nm – spectral resolution for returned irradiances [nanometers]

  • wvl_min_nm – minimum wavelength of returned irradiances [nanometers]

  • wvl_max_nm – maximum wavelength of returned irradiances [nanometers]

Returns

gms_preprocessing.io.input_reader.get_list_GMSfiles(dataset_list, target)[source]

Returns a list of absolute paths pointing to gms-files of truely written datasets that fullfill certain criteria.

Parameters

  • dataset_list – [dataset1_dictionary, dataset2_dictionary]

  • target – target GMS processing level

:return [/path/to/gms_file1.gms, /path/to/gms_file1.gms]

gms_preprocessing.io.input_reader.open_specific_file_within_archive(path_archive, matching_expression, read_mode='r')[source]

Finds a specific file within an archive using a given matching expression and returns its content as string.

Parameters

  • path_archive (str) – the file path of the archive

  • matching_expression (str) – the matching expession to find the file within the archive

  • read_mode (str) – the read mode used to open the archive (default: ‘r’)

Return type

(str, str)

Returns

tuple(content_file, filename_file)

gms_preprocessing.io.input_reader.pickle_SRF_DB(L1A_Instances, dir_out)[source]
gms_preprocessing.io.input_reader.read_ENVI_image_data_as_array(path, arr_shape, arr_pos, logger=None, return_meta=False, q=0)[source]

Read ENVI image data as array using a specified read pattern.

Parameters

  • path – <str> Path of the ENVI image or header file

  • arr_shape – <str> ‘cube’,’row’,’col’,’band’,’block’,’pixel’ or ‘custom’

  • arr_pos –

    None, <int> or <list>. The content of the list depends on the chosen arr_shape as follows: - ‘cube’: No array position neccessary. Set arr_pos to None. - ‘row’: single int. -> Row with this index will be read. - ‘col’: single int. -> Column with this index will be read. - ‘band’: single int. -> Band with this index will be read. - ‘block’: row_bounds (2-tuple of ints): (a, b) -> Rows a through b-1 will be read.

    col_bounds (2-tuple of ints): (a, b) -> Columns a through b-1 will be read.

    • ’pixel’: row, col (int): Indices of the row & column for the pixel to be read.

    • ’custom’: row_bounds (2-tuple of ints): (a, b) -> Rows a through b-1 will be read.

      col_bounds (2-tuple of ints): (a, b) -> Columns a through b-1 will be read. bands (list of ints): Optional list of bands to read.

      If not specified, all bands are read.

  • logger – <instance> of logging.logger (optional)

  • return_meta – <bool> whether to return not only raster data but also meta data (optional, default=False)

  • q – <bool> quiet mode (supresses all console or logging output) (optional, default=False)

gms_preprocessing.io.input_reader.read_ENVIfile(path, arr_shape, arr_pos, logger=None, return_meta=False, q=0)[source]
gms_preprocessing.io.input_reader.read_ENVIhdr_to_dict(hdr_path, logger=None)[source]
Return type

dict

gms_preprocessing.io.input_reader.read_mask_subset(path_masks, bandname, logger, subset=None)[source]
gms_preprocessing.io.input_reader.unify_envi_header_keys(header_dict)[source]

Ensures the compatibility of ENVI header keys written by Spectral-Python the code internal attribute names. (ENVI header keys are always lowercase in contrast to the attribute names used in code).

Parameters

header_dict –

gms_preprocessing.io.output_writer module

Output writer: Universal writer for all kinds of GeoMultiSens intermediate results.

gms_preprocessing.io.output_writer.HDR_writer(meta_dic, outpath_hdr, logger=None)[source]
gms_preprocessing.io.output_writer.Tiles_Writer(tileList_or_Array, out_path, out_shape, out_dtype, out_interleave, out_meta=None, arr_pos=None, overwrite=True)[source]

Write tiles to disk using numpy.memmap.

Parameters

  • tileList_or_Array – <list of dicts> each dict has keys ‘row_start’,’row_end’,’col_start’,’col_end’,’data’ <numpy array> representing a subset of a full array. requires arr_pos

  • out_path – <str> path to ENVI header file *.hdr

  • out_shape – <tuple or list> (rows,cols,bands)

  • out_dtype – <object> numpy data type object

  • out_interleave – <str> ‘bsq’,’bil’ or ‘bip’

  • out_meta – <dict> metadata dictionary to be written to header

  • arr_pos – <tuple> ((row_start,row_end),(col_start,col_end))

  • overwrite – <bool>

gms_preprocessing.io.output_writer.add_ENVIclassificationMeta_to_meta(meta, class_names, class_colors=None, data=None)[source]

Prepare ENVI metadata dict. to be written as ENVI classification file by adding custom class names and colors.

Parameters

  • meta (dict) – <dict> ENVI metadata dictionary

  • class_names (list) – <list> of strings with the class names

  • class_colors (Optional[dict]) – <dict> with keys representing class names and values representing 3-tuples with RGB codes

  • data (Optional[ndarray]) – <numpy array> only used to estimate number of classes if class_names is None

Return type

dict

gms_preprocessing.io.output_writer.add_attributes_to_ENVIhdr(attr2add_dict, hdr_path)[source]
gms_preprocessing.io.output_writer.check_header_not_empty(hdr)[source]
gms_preprocessing.io.output_writer.dump_all_SRFs(outpath_dump='/builds/geomultisens/gms_preprocessing/gms_preprocessing/sandbox/out/SRF_DB.pkl', outpath_log='/builds/geomultisens/gms_preprocessing/gms_preprocessing/sandbox/out/SRF_DB.log')[source]
gms_preprocessing.io.output_writer.export_VZA_SZA_SAA_RAA_stats(L1A_object)[source]
gms_preprocessing.io.output_writer.get_dtypeStr(val)[source]
gms_preprocessing.io.output_writer.mask_to_ENVI_Classification(InObj, maskname)[source]
Return type

(np.ndarray, dict, list, list)

gms_preprocessing.io.output_writer.reorder_ENVI_header(path_hdr, tgt_keyOrder)[source]

Reorders the keys of an ENVI header file according to the implemented order. Keys given in the target order list but missing the ENVI fileare skipped. This function is a workaround for envi.write_envi_header of Spectral Python Library that always writes the given metadata dictinary as an unordered dict.

Parameters

  • path_hdr (str) – <str> path of the target ENVI file

  • tgt_keyOrder (list) – <list> list of target keys in the correct order

Return type

None

gms_preprocessing.io.output_writer.set_output_nodataVal(arr_path, nodataVal=None)[source]

Sets the no data value of an already written file

Parameters

  • arr_path –

  • nodataVal –

Returns

gms_preprocessing.io.output_writer.silent_envi_write_image(hdr_file, data, header, **kwargs)[source]

Monkeypatch for spectral.io.envi._write_image in order to silence output stream.

gms_preprocessing.io.output_writer.write_ENVI_compressed(outPath_hdr, ndarray, meta, interleave='bsq')[source]
gms_preprocessing.io.output_writer.write_global_benchmark_output(list__processing_time__all_runs, list__IO_time__all_runs, data_list)[source]
gms_preprocessing.io.output_writer.write_ordered_envi_header(fileName, header_dict, is_library=False)[source]

Monkeypatch for spectral.io.envi.write_envi_header in order to write ordered output headers

gms_preprocessing.io.output_writer.write_shp(path_out, shapely_geom, prj=None, attrDict=None)[source]
gms_preprocessing.io.output_writer.write_shp_OLD(shapely_poly, path_out, prj=None)[source]

Module contents