Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 45 additions & 17 deletions Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,15 @@ pip install hecdss
### Regular TimeSeries Data
- **Stored object in Python**: Regular Timeseries data is stored as 2 arrays of values and times/interval and startdate.
- **Attributes**: The TimeSeries object has the following attributes:
- `start_date`: The start date of the time series data.
- `times`: The times of the time series data.
- `values`: The values of the time series data.
- `interval`: The interval of the time series data.
- `data_type`: the dss data type ("PER-AVER","PER-CUM", "INST-VAL", "INST-CUM", .. )
- `units`: The units of the time series data.
- `id`: The Path of the time series data.
- `start_date (str)`: The start date of the time series data.
- `times (list[datetime])`: The times of the time series data.
- `values (nd.nparray[float])`: The values of the time series data.
- `quality (list[int])`: The quality flags of the time series data.
- `notes (list[str])`: The notes of the time series data.
- `interval (str)`: The interval of the time series data.
- `data_type (str)`: the dss data type ("PER-AVER","PER-CUM", "INST-VAL", "INST-CUM", .. )
- `units (str)`: The units of the time series data.
- `id (str)`: The Path of the time series data (Format: "/A/B/C/D/E/F/").

```python
# example working with time-series data
Expand All @@ -60,21 +62,22 @@ with HecDss(file_path) as dss:
### Irregular TimeSeries Data
- **Stored object in Python**: Irregular Timeseries data is stored as 2 arrays of values and times.
- **Attributes**: The TimeSeries object has the following attributes:
- `times`: The times of the time series data.
- `values`: The values of the time series data.
- `times (list[datetime])`: The times of the time series data.
- `values (np.ndarray[float])`: The values of the time series data.
- `units`: The units of the time series data.
- `data_type`: the dss data type ('INST-VAL', 'INST-CUM')
- `data`: The time series data.
- `id`: The Path of the time series data.
- `quality (list[int])`: The quality flags of the time series data.
- `notes (list[str])`: The notes of the time series data.
- `data_type (str)`: the dss data type ('INST-VAL', 'INST-CUM')
- `id (str)`: The Path of the time series data (Format: "/A/B/C/D/E/F/").


### Paired Data
- **Stored object in Python**: Paired data stored as aa (x, y) where y could be stored as a 2d numpy matrix.
- **Attributes**: The PairedData object has the following attributes:
- `ordinates`: The x values of the paired data
- `values`: y values of the paired data stored as 2d numpy array.
- `labels`: The labels of the paired data.
- `id`: The Path of the paired data.
- `ordinates (np.ndarray)`: The x values of the paired data
- `values (np.ndarray[float])`: y values of the paired data stored as 2d numpy array.
- `labels (list[str])`: The labels of the paired data.
- `id (str)`: The Path of the paired data.

### Gridded Data
- **Stored object in Python**: A 2d matrix stored as a numpy 2d object.
Expand All @@ -87,6 +90,7 @@ with HecDss(file_path) as dss:
- * Supports storing and reading arrays of integers, floats, or doubles.
- * Arrays are managed with ArrayContainer


```python
# Example working with an array
with HecDss("my-dss-file.dss") as dss:
Expand All @@ -98,8 +102,33 @@ with HecDss(file_path) as dss:
read_array = dss.get(array_ints.id)
```

## CSV Functionality

### Time Series
The `rts.to_csv` and `RegularTimeSeries.read_csv` methods can be used to convert time series data to `.csv` format. This can also be utilized alongside `pandas` methods to convert time series data to pandas DataFrames.

```python
# Round trip CSV example with RegularTimeSeries and Pandas DataFrame
import pandas as pd

file_path: str = "my-dss-file.dss"
with HecDss(file_path) as dss:
data_path: str = "/example/data/////"
rts: RegularTimeSeries = dss.get(data_path)

csv_out: str = "example.csv"
rts.to_csv(csv_out) # RegularTimeSeries -> CSV

df: pd.DataFrame = pd.read_csv(csv_out) # CSV -> DataFrame
df_csv_path: str = "df_example.csv"
# Index needs to be false for round-trip support
df.to_csv(df_csv_path, index=False) # DataFrame -> CSV

with HecDss(file_path) as dss:
RegularTimeSeries.read_csv(df_csv_path) # CSV -> RegularTimeSeries
```

Additionally, `IrregularTimeSeries` and `PairedData` both have equivalent functionality with `.csv` conversion.


### This libray is built using the API for future versions of HEC-DSS
Expand Down Expand Up @@ -153,7 +182,6 @@ These are the driving design ideas and goals of the hec-dss-python project (subj
| hec_dss_native.py | native binding layer | isolate interactions with low level library(if performance is an issue this Ctypes layer can be replaced ) |
| hec_dss.py | Programmer entry point ; Python API | Hides interactions with hec_dss_native, seek to be simple user experience |
|catalog.py|manage list of DSS objects (catalog) | create condensed catalog perspective |
|Pandas_Series_Utilities.py [future](https://github.com/HydrologicEngineeringCenter/hec-dss-python/issues/8) |NumPy/pandas support | provide features such as dataframes, separate from hec-dss.py; can be developed by different/parallel developers |
|Easy to get started |nothing to install, just copy python files and shared library | require minimal privileges to install |


Expand Down
47 changes: 39 additions & 8 deletions src/hecdss/regular_timeseries.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,9 @@ def __init__(self):
self.id = ""
self.location_info = None

def add_data_point(self, date: datetime, value: float, flag: int = None, note: str = None):
def add_data_point(
self, date: datetime, value: float, flag: int = None, note: str = None
):
"""
Adds a data point to the time series.

Expand Down Expand Up @@ -117,6 +119,7 @@ def to_csv(self, file_path: str) -> None:
file_path (str): The path to the .csv file where the data will be exported.
"""
from .dss_csv import timeseries_to_csv

timeseries_to_csv(self, file_path)
print(f"Wrote RegularTimeSeries to .csv file at {file_path}.")

Expand Down Expand Up @@ -184,9 +187,17 @@ def _interval_to_times(self, new_interval):
Args:
new_interval (int): The new interval in seconds.
"""
def is_leap(y): return y % 4 == 0 and y % 100 != 0 or y % 400 == 0
def last_day(y, m): return 31 if m in (1, 3, 5, 7, 8, 10, 12) else 30 if m in (
4, 6, 9, 11) else 29 if is_leap(y) else 28

def is_leap(y):
return y % 4 == 0 and y % 100 != 0 or y % 400 == 0

def last_day(y, m):
return (
31
if m in (1, 3, 5, 7, 8, 10, 12)
else 30 if m in (4, 6, 9, 11) else 29 if is_leap(y) else 28
)

if type(self.start_date) == datetime:
tz = ZoneInfo(self.time_zone_name) if self.time_zone_name else None
first_time = self.start_date.replace(microsecond=0, tzinfo=tz)
Expand Down Expand Up @@ -238,12 +249,16 @@ def _generate_times(self):
"""
Generates times for the time series based on the interval and start date.
"""
if (len(self.times) > 0 and self.start_date == ""):
if len(self.times) > 0 and self.start_date == "":
self.start_date = self.times[0]

x = [self._get_interval_times(), self._get_interval_path(), self._get_interval_interval()]
x = [
self._get_interval_times(),
self._get_interval_path(),
self._get_interval_interval(),
]
x = [i for i in x if i != "empty"]
if (not all(i == x[0] for i in x)):
if not all(i == x[0] for i in x):
raise ValueError("inconsistent interval within arguments")
elif len(x) != 3 and len(x) != 0:
self._interval_to_interval(x[0])
Expand All @@ -262,17 +277,33 @@ def read_csv(file_path: str) -> "RegularTimeSeries":
RegularTimeSeries: A new instance of RegularTimeSeries populated with the data from the .csv file.
"""
from .dss_csv import timeseries_read_csv

return timeseries_read_csv(RegularTimeSeries, file_path)

@staticmethod
def create(values, times=[], quality=[], notes=[], units="", data_type="", interval="", start_date="", time_granularity_seconds=1, julian_base_date=0, time_zone_name="", path=None, location_info=None):
def create(
values,
times=[],
quality=[],
notes=[],
units="",
data_type="",
interval="",
start_date="",
time_granularity_seconds=1,
julian_base_date=0,
time_zone_name="",
path=None,
location_info=None,
):
"""
Creates a new instance of the RegularTimeSeries class with the specified parameters.

Args:
values (list): List of data values.
times (list, optional): List of time values. Defaults to [].
quality (list, optional): List of quality values. Defaults to [].
notes (list[str]): The notes of the time series data.
units (str, optional): Units of the data. Defaults to "".
data_type (str, optional): Type of the data. Defaults to "".
interval (str, optional): Interval of the time series. Defaults to "".
Expand Down
Loading