Data Format Documentation

1. Introduction

This document provides documentation on the various data formats used within the CoE Hidalgo2 research pipeline. Understanding these formats is essential for working with and interpreting the data generated by the pipeline.

2. JSON (JavaScript Object Notation)

JSON is used as the metadata format to provide essential context and information about the research data. It is a lightweight and human-readable data interchange format.

JSON Format Example:

{
  "city_name": "CityName",
  "radius_meters": 1000,
  "creation_date_time": "YYYY-MM-DDTHH:mm:ss",
  "creator_name": "John Doe",
  "gis_resolution": 1.0,
  "building_definition_level": "High",
  "total_buildings": 500,
  "total_building_faces": 2500,
  "has_vegetation": true,
  "has_mountains": false,
  "has_roads": true
}

3. MSH (Gmsh Mesh Format)

MSH files (version 4.1) are used to store mesh data generated using Gmsh, a powerful meshing tool. The MSH format is commonly used in scientific computing for representing complex geometries.

4. CSV (Comma-Separated Values) Files

CSV files are used for storing data related to solar masks and computational results. They are a tabular data format, where values are separated by commas.

The solar mask format is documented xref:.

4.1. Weather time and solar civil-date contract

The weather CSV time column contains Unix timestamps in milliseconds and therefore represents UTC instants. The first weather row is the source of truth for the civil date of the simulation.

KUB exposes reporting timestamps in Unix seconds and advances the FMUs with a relative time in seconds. The civil instant used by every solar calculation is:

civil UTC time = first weather timestamp / 1000 + relative FMI time

Unix timestamps use the Greenwich reference meridian (0 degrees). Geographic longitude is a separate value, positive east. KUB does not apply the host time zone or daylight-saving rules: these have already been resolved when the source date was encoded as a UTC timestamp.

Scenario CSV Time values are relative hours from the same simulation origin. They select set points but do not replace or reinterpret the civil clock carried by the weather data.

This explicit contract is required by simulator datasets starting with v0.9.0. Older FMUs do not expose the civil-time parameters and cannot be mixed with the v0.9.0 runtime contract.

5. FMU (Functional Mock-up Unit) Files

FMU files are used to store building energy models. FMUs are a standard format for packaging model components, making it possible to simulate and exchange models across different simulation tools.

6. Conclusion

This documentation provides an overview of the data formats used within the CoE Hidalgo2 research pipeline. Understanding these formats is crucial for effectively working with and analyzing the data generated during research activities.