floodpy.Download.eof package

Submodules

floodpy.Download.eof.asf_client module

Client to get orbit files from ASF.

class floodpy.Download.eof.asf_client.ASFClient(cache_dir: Optional[Union[str, PathLike]] = None, username: str = '', password: str = '')

Bases: object

auth_url = 'https://urs.earthdata.nasa.gov/oauth/authorize?response_type=code&client_id=BO_n7nTIlMljdvU6kRRB3g&redirect_uri=https://auth.asf.alaska.edu/login'
eof_lists = {'precise': None, 'restituted': None}
get_authenticated_session() → Session

Get an authenticated requests.Session using earthdata credentials.

Fuller example here: https://github.com/ASFHyP3/hyp3-sdk/blob/ec72fcdf944d676d5c8c94850d378d3557115ac0/src/hyp3_sdk/util.py#L67C8-L67C8

Returns

Authenticated session

Return type

requests.Session

get_cache_dir()

Find location of directory to store .hgt downloads Assuming linux, uses ~/.cache/sentineleof/

get_download_urls(orbit_dts, missions, orbit_type='precise')

Find the URL for an orbit file covering the specified datetime

Parameters

  • dt (datetime) – requested

  • orbit_dts (list[str] or list[datetime]) – datetime for orbit coverage

  • missions (list[str]) – specify S1A or S1B

Returns

URL for the orbit file

Return type

str

get_full_eof_list(orbit_type='precise', max_dt=None)

Get the list of orbit files from the ASF server.

precise_url = 'https://s1qc.asf.alaska.edu/aux_poeorb/'
res_url = 'https://s1qc.asf.alaska.edu/aux_resorb/'
urls = {'precise': 'https://s1qc.asf.alaska.edu/aux_poeorb/', 'restituted': 'https://s1qc.asf.alaska.edu/aux_resorb/'}
floodpy.Download.eof.asf_client.SIGNUP_URL = 'https://urs.earthdata.nasa.gov/users/new'

Url to prompt user to sign up for NASA Earthdata account.

floodpy.Download.eof.cli module

floodpy.Download.eof.dataspace_client module

Client to get orbit files from dataspace.copernicus.eu .

floodpy.Download.eof.dataspace_client.AUTH_URL = 'https://identity.dataspace.copernicus.eu/auth/realms/CDSE/protocol/openid-connect/token'

Default URL endpoint for performing user authentication with CDSE

floodpy.Download.eof.dataspace_client.DOWNLOAD_URL = 'https://zipper.dataspace.copernicus.eu/odata/v1/Products'

Default URL endpoint for CDSE download REST service

class floodpy.Download.eof.dataspace_client.DataspaceClient(username: str = '', password: str = '')

Bases: object

T0 = datetime.timedelta(seconds=5984, microseconds=571429)
T1 = datetime.timedelta(seconds=60)
download_all(query_results: list[dict], output_directory: Union[str, PathLike], max_workers: int = 3)

Download all the specified orbit products.

query_orbit(t0: datetime, t1: datetime, satellite_id: str, product_type: str = 'AUX_POEORB') → list[dict]
query_orbit_by_dt(orbit_dts, missions, orbit_type: str = 'precise', t0_margin: timedelta = datetime.timedelta(seconds=5984, microseconds=571429), t1_margin: timedelta = datetime.timedelta(seconds=60))

Query the Scihub api for product info for the specified missions/orbit_dts.

Parameters

  • orbit_dts (list[datetime.datetime]) – List of datetimes to query for

  • missions (list[str], choices = {"S1A", "S1B"}) – List of missions to query for. Must be same length as orbit_dts

  • orbit_type (str, choices = {"precise", "restituted"}) – String identifying the type of orbit file to query for.

  • t0_margin (timedelta) – Margin to add to the start time of the orbit file in the query

  • t1_margin (timedelta) – Margin to add to the end time of the orbit file in the query

Returns

list of results from the query

Return type

list[dict]

query_orbit_for_product(product, orbit_type: str = 'precise', t0_margin: timedelta = datetime.timedelta(seconds=5984, microseconds=571429), t1_margin: timedelta = datetime.timedelta(seconds=60))
floodpy.Download.eof.dataspace_client.QUERY_URL = 'https://catalogue.dataspace.copernicus.eu/odata/v1/Products'

Default URL endpoint for the Copernicus Data Space Ecosystem (CDSE) query REST service

floodpy.Download.eof.dataspace_client.SIGNUP_URL = 'https://dataspace.copernicus.eu/'

Url to prompt user to sign up for CDSE account.

floodpy.Download.eof.dataspace_client.download_all(query_results: list[dict], output_directory: Union[str, PathLike], username: str = '', password: str = '', max_workers: int = 3) → list[pathlib.Path]

Download all the specified orbit products.

Parameters

  • query_results (list[dict]) – list of results from the query

  • output_directory (str | Path) – Directory to save the orbit files to.

  • username (str) – CDSE username

  • password (str) – CDSE password

  • max_workers (int, default = 3) – Maximum parallel downloads from CDSE. Note that >4 connections will result in a HTTP 429 Error

floodpy.Download.eof.dataspace_client.download_orbit_file(request_url, output_directory, orbit_file_name, access_token) → Path

Downloads an Orbit file using the provided request URL.

Should contain product ID for the file to download, as obtained from a query result.

The output file is named according to the orbit_file_name parameter, and should correspond to the file name parsed from the query result. The output file is written to the directory indicated by output_directory.

Parameters

  • request_url (str) – The full request URL, which includes the download endpoint, as well as a payload that contains the product ID for the Orbit file to be downloaded.

  • output_directory (str) – The directory to store the downloaded Orbit file to.

  • orbit_file_name (str) – The file name to assign to the Orbit file once downloaded to disk. This should correspond to the file name parsed from a query result.

  • access_token (str) – Access token returned from an authentication request with the provided username and password. Must be provided with all download requests for the download service to respond.

Returns

output_orbit_file_path – The full path to where the resulting Orbit file was downloaded to.

Return type

Path

Raises

requests.HTTPError – If the request fails for any reason (HTTP return code other than 200).

floodpy.Download.eof.dataspace_client.get_access_token(username, password) → Optional[str]

Get an access token for the Copernicus Data Space Ecosystem (CDSE) API.

Code from https://documentation.dataspace.copernicus.eu/APIs/Token.html

floodpy.Download.eof.dataspace_client.query_orbit_file_service(query: str) → list[dict]

Submit a request to the Orbit file query REST service.

Parameters

query (str) – The query for the Orbit files to find, filtered by a time range and mission ID corresponding to the provided SAFE SLC archive file.

Returns

query_results – The list of results from a successful query. Each result should be a Python dictionary containing the details of the orbit file which matched the query.

Return type

list of dict

Raises

requests.HTTPError – If the request fails for any reason (HTTP return code other than 200).

References

1

https://documentation.dataspace.copernicus.eu/APIs/OData.html#query-by-sensing-date

floodpy.Download.eof.download module

Utility for downloading Sentinel precise orbit ephemerides (EOF) files

Example filtering URL: ?validity_start_time=2014-08&page=2

Example EOF: ‘S1A_OPER_AUX_POEORB_OPOD_20140828T122040_V20140806T225944_20140808T005944.EOF’

‘S1A’ : mission id (satellite it applies to) ‘OPER’ : OPER for “Routine Operations” file ‘AUX_POEORB’ : AUX_ for “auxiliary data file”, POEORB=Precise Orbit Ephemerides (POE) Orbit File ‘OPOD’ Site Center of the file originator

‘20140828T122040’ creation date of file ‘V20140806T225944’ Validity start time (when orbit is valid) ‘20140808T005944’ Validity end time

Full EOF sentinel doumentation: https://earth.esa.int/documents/247904/349490/GMES_Sentinels_POD_Service_File_Format_Specification_GMES-GSEG-EOPG-FS-10-0075_Issue1-3.pdf

See parsers for Sentinel file naming description

floodpy.Download.eof.download.download_eofs(orbit_dts=None, missions=None, sentinel_file=None, save_dir='.', orbit_type='precise', force_asf: bool = False, asf_user: str = '', asf_password: str = '', cdse_user: str = '', cdse_password: str = '', max_workers: int = 6) → list[pathlib.Path]

Downloads and saves EOF files for specific dates

Parameters

  • orbit_dts (list[str] or list[datetime.datetime]) – datetime for orbit coverage

  • missions (list[str]) – optional, to specify S1A or S1B No input downloads both, must be same len as orbit_dts

  • sentinel_file (str) – path to Sentinel-1 filename to download one .EOF for

  • save_dir (str) – directory to save the EOF files into

  • orbit_type (str) – precise or restituted

Returns

all filenames of saved orbit files

Return type

list[str]

Raises

ValueError - for missions argument not being one of 'S1A', 'S1B', – having different lengths, or sentinel_file being invalid

floodpy.Download.eof.download.find_current_eofs(cur_path)

Returns a list of SentinelOrbit objects located in cur_path

floodpy.Download.eof.download.find_scenes_to_download(search_path='./', save_dir='./')

Parse the search_path directory for any Sentinel 1 products’ date and mission

floodpy.Download.eof.download.find_unique_safes(search_path)
floodpy.Download.eof.download.main(search_path='.', save_dir='.', sentinel_file=None, mission=None, date=None, orbit_type='precise', force_asf: bool = False, asf_user: str = '', asf_password: str = '', cdse_user: str = '', cdse_password: str = '', max_workers: int = 6)

Function used for entry point to download eofs

floodpy.Download.eof.log module

floodpy.Download.eof.parsing module

Module for parsing the orbit state vectors (OSVs) from the .EOF file

class floodpy.Download.eof.parsing.EOFLinkFinder

Bases: HTMLParser

Finds EOF download links in aux.sentinel1.eo.esa.int page

Example page to search: http://step.esa.int/auxdata/orbits/Sentinel-1/POEORB/S1B/2020/10/

Usage: >>> import requests >>> resp = requests.get(“http://step.esa.int/auxdata/orbits/Sentinel-1/POEORB/S1B/2020/10/”) >>> parser = EOFLinkFinder() >>> parser.feed(resp.text)

handle_starttag(tag, attrs)
floodpy.Download.eof.parsing.parse_orbit(eof_filename, min_time=datetime.datetime(1900, 1, 1, 0, 0), max_time=datetime.datetime(2100, 1, 1, 0, 0), extra_osvs=1)
floodpy.Download.eof.parsing.parse_utc_string(timestring)
floodpy.Download.eof.parsing.secs_since_midnight(dt)
floodpy.Download.eof.parsing.to_datetime(dates, tzinfo=datetime.timezone.utc)

Convert a single (or list of) datetime.date to timezone-aware datetime.datetime

floodpy.Download.eof.parsing.write_orbinfo(orbit_tuples, outname='out.orbtiming')

Write file with orbit states parsed into simpler format

seconds x y z vx vy vz ax ay az

floodpy.Download.eof.products module

Utilities for parsing file names of SAR products for relevant info.

class floodpy.Download.eof.products.Sentinel(filename, **kwargs)

Bases: Base

Sentinel 1 reference: https://sentinel.esa.int/web/sentinel/user-guides/sentinel-1-sar/naming-conventions or https://sentinel.esa.int/documents/247904/349449/Sentinel-1_Product_Specification

Example

S1A_IW_SLC__1SDV_20180408T043025_20180408T043053_021371_024C9B_1B70.zip S1A_IW_RAW__0SSV_20151018T005110_20151018T005142_008200_00B886_61EC.zip

File name format:

MMM_BB_TTTR_LFPP_YYYYMMDDTHHMMSS_YYYYMMDDTHHMMSS_OOOOOO_DDDDDD_CCCC.EEEE

MMM: mission/satellite S1A or S1B BB: Mode/beam identifier. The S1-S6 beams apply to SM products IW,

EW and WV identifiers appply to products from the respective modes.

TTT: Product Type: RAW, SLC, GRD, OCN R: Resolution class: F, H, M, or _ (N/A) L: Processing Level: 0, 1, 2 F: Product class: S (standard), A (annotation, only used internally)

  • we only care about standard

PP: Polarization: SH (single HH), SV (single VV), DH (dual HH+HV), DV (dual VV+VH) Start date + time (date/time separated by T) Stop date + time OOOOOO: absolute orbit number: 000001-999999 DDDDDD: mission data-take identifier: 000001-FFFFFF. CCCC: product unique identifier: hexadecimal string from CRC-16 hashing

the manifest file using CRC-CCITT.

Once unzipped, the folder extension is always “SAFE”

filename
Type

str

FILE_REGEX = re.compile('(?P<mission>S1A|S1B)_(?P<beam>[\\w\\d]{2})_(?P<product_type>[\\w_]{3})(?P<resolution_class>[FHM_])_(?P<product_level>[012])[SA](?P<polarization>[SDHV]{2})_(?P<start_datetime>[T\\d]{15})_(?P<stop_date)
TIME_FMT = '%Y%m%dT%H%M%S'
property absolute_orbit

Absolute orbit of data, included in file name

Example

>>> s = Sentinel('S1A_IW_SLC__1SDV_20180408T043025_20180408T043053_021371_024C9B_1B70')
>>> print(s.absolute_orbit)
21371
property date

shortcut for start_time.date()

Type

Date of acquisition

property level

Alias for product type/level

property mission

Returns satellite/mission of product (S1A/S1B)

Example

>>> s = Sentinel('S1A_IW_SLC__1SDV_20180408T043025_20180408T043053_021371_024C9B_1B70')
>>> print(s.mission)
S1A
property path

Alias for relative orbit number

property polarization

Returns type of polarization of product

Example

>>> s = Sentinel('S1A_IW_SLC__1SDV_20180408T043025_20180408T043053_021371_024C9B_1B70')
>>> print(s.polarization)
DV
property product_type

Returns product type/level

Example

>>> s = Sentinel('S1A_IW_SLC__1SDV_20180408T043025_20180408T043053_021371_024C9B_1B70')
>>> print(s.product_type)
SLC
property product_uid

Unique identifier of product (last 4 of filename)

property relative_orbit

Relative orbit number/ path

Formulas for relative orbit from absolute come from: https://forum.step.esa.int/t/sentinel-1-relative-orbit-from-filename/7042

Example

>>> s = Sentinel('S1A_IW_SLC__1SDV_20180408T043025_20180408T043053_021371_024C9B_1B70')
>>> print(s.relative_orbit)
124
>>> s = Sentinel('S1B_WV_OCN__2SSV_20180522T161319_20180522T164846_011036_014389_67D8')
>>> print(s.relative_orbit)
160
property start_time

Returns start datetime from a sentinel file name

Example

>>> s = Sentinel('S1A_IW_SLC__1SDV_20180408T043025_20180408T043053_021371_024C9B_1B70')
>>> print(s.start_time)
2018-04-08 04:30:25
property stop_time

Returns stop datetime from a sentinel file name

Example

>>> s = Sentinel('S1A_IW_SLC__1SDV_20180408T043025_20180408T043053_021371_024C9B_1B70')
>>> print(s.stop_time)
2018-04-08 04:30:53
class floodpy.Download.eof.products.SentinelOrbit(filename, **kwargs)

Bases: Base

Sentinel 1 orbit reference: https://sentinel.esa.int/documents/247904/351187/GMES_Sentinels_POD_Service_File_Format_Specification

section 2

https://qc.sentinel1.eo.esa.int/doc/api/ https://sentinels.copernicus.eu/documents/247904/3372484/Copernicus-POD-Regular-Service-Review-Jun-Sep-2018.pdf

see here (section 3.6) for differences in orbit accuracy)

Example

S1A_OPER_AUX_PREORB_OPOD_20200325T131800_V20200325T121452_20200325T184952.EOF

The filename must comply with the following pattern:

MMM_CCCC_TTTTTTTTTT_<instance_id>.EOF

MMM = mission, S1A or S1B CCCC = File Class, we only want OPER = routine operational TTTTTTTTTT = File type

= FFFF DDDDDD

FFFF = file category, we want AUX_:auxiliary data files; DDDDDD = Semantic Descriptor most common = POEORB: Precise Orbit Ephemerides (POE) Orbit File

(available after 1-2 weeks)

also, RESORB: Restituted orbit file

(covers 6 hour windows, less accurate, more immediate)

TODO: do I ever want to deal with the AUX antenna files?

<instance id> has a couple: ssss_yyyymmddThhmmsswhere:

ssss is the Site Centre of the file originator (OPOD for S-1 and S-2) and a validity start/stop, same date format

filename
Type

str

FILE_REGEX = '(?P<mission>S1A|S1B)_OPER_AUX_(?P<orbit_type>[\\w_]{6})_OPOD_(?P<created_datetime>[T\\d]{15})_V(?P<start_datetime>[T\\d]{15})_(?P<stop_datetime>[T\\d]{15})'
TIME_FMT = '%Y%m%dT%H%M%S'
property created_time

Returns created datetime from a orbit file name

Example

>>> s = SentinelOrbit('S1A_OPER_AUX_POEORB_OPOD_20200121T120654_V20191231T225942_20200102T005942.EOF')
>>> print(s.created_time)
2020-01-21 12:06:54
property date

shortcut for start_time.date()

Type

Date of acquisition

property mission

Returns satellite/mission of product (S1A/S1B)

Example

>>> s = SentinelOrbit('S1A_OPER_AUX_POEORB_OPOD_20200121T120654_V20191231T225942_20200102T005942.EOF')
>>> print(s.mission)
S1A
property orbit_type

Type of orbit file (e.g precise, restituted)

Example: >>> s = SentinelOrbit(‘S1A_OPER_AUX_POEORB_OPOD_20200121T120654_V20191231T225942_20200102T005942.EOF’) >>> print(s.orbit_type) precise >>> s = SentinelOrbit(‘S1B_OPER_AUX_RESORB_OPOD_20200325T151938_V20200325T112442_20200325T144212.EOF’) >>> print(s.orbit_type) restituted

property start_time

Returns start datetime of an orbit

Example

>>> s = SentinelOrbit('S1A_OPER_AUX_POEORB_OPOD_20200121T120654_V20191231T225942_20200102T005942.EOF')
>>> print(s.start_time)
2019-12-31 22:59:42
property stop_time

Returns stop datetime from a sentinel file name

Example

>>> s = SentinelOrbit('S1A_OPER_AUX_POEORB_OPOD_20200121T120654_V20191231T225942_20200102T005942.EOF')
>>> print(s.stop_time)
2020-01-02 00:59:42

Module contents