Introduction to GeoST#

This introduction covers some of the key concepts and basic features of GeoST to help you get started. GeoST depends on popular data science libraries such as Pandas and GeoPandas, but provides readily available, frequently used selection methods for data held in pandas.DataFrame or geopandas.GeoDataFrame objects. This makes GeoST an easy-to-use option for less experienced Python users, while more experienced users can easily access the underlying DataFrames and apply their own analyses.

GeoST is designed to work with many different kinds of subsurface data. The package is a work in progress, and we aim to support an increasing number of data sources. Below is a list of data sources that are currently supported or planned for future support:

From local files:

  • Tabular data of borehole, CPT, etc. (.parquet, .csv)

  • Geological boreholes xml (BHR-G)

  • Geotechnical boreholes xml (BHR-GT)

  • Pedological boreholes xml (BHR-P)

  • Cone Penetration Test xml/gef (CPT)

  • Pedological soilprofile descriptions xml (SFR)

  • BORIS xml (TNO borehole description software)

  • NLOG boreholes Excel

Directly from the BRO REST-API:

  • BHR-G

  • BHR-GT

  • BHR-P

  • CPT

  • SFR

BRO models:

Planned:

  • BRO/PDOK GeoPackages: BHR-G, BHR-GT, BHR-P, CPT, and SFR

  • Well logs (LAS/ASCII)

  • REGIS II

  • Dino XML geological boreholes

  • BHR-G GEF

  • Soilmap of the Netherlands

GeoST also plans support for several Geophysical data sources such as Seismic, ERT, EM and others.

Concept#

The core data structure is a geost.Collection. This holds all the spatial information of any kind of data source in a “header” attribute (i.e., the header table), and the corresponding data in a “data” attribute (i.e., the data table).

For example, a set of 100 boreholes is held in a Collection, where the “header” contains one row per data entry, so a 100 rows, and provides information about the ID, location, surface level, and depths. The “data” contains the survey information for each described layer. The structure of a Collection is illustrated below.

collection

When working with a Collection, GeoST automatically keeps track of the alignment between the two attributes and ensures that each data entry occurs in both the “header” and “data” attributes. For example, when a user deletes an individual borehole entry from the “header”, the Collection ensures that it is also deleted from the “data”.

Note

User guide: Checkout the Data structures section in the user guide for a more detailed explanation of the GeoST data structures.

The Basics#

Data is usually loaded through various reader functions (see the API reference). For this tutorial, GeoST provides a test set of boreholes in the area of the Utrecht Science Park that can be directly loaded as a Collection. Let’s read the data, print the result to see what it contains, and also plot the locations to get an idea of where we are:

import xyzservices.providers as xyz  # For accessing background map providers

import geost

usp_boreholes = geost.data.boreholes_usp()
print(usp_boreholes)
usp_boreholes.header.explore(
    tiles=xyz.nlmaps.grijs, style_kwds={"weight": 4}
)  # Interactive plot of the borehole locations.

Collection
  header (rows, columns) : (67, 6)
  data (rows, columns)   : (1398, 32)
crs: RD
vertical datum: NAP height
Make this Notebook Trusted to load map: File -> Trust Notebook

As you can see, usp_boreholes is of type Collection. Additionally, it shows the number of rows and columns in the header and data tables, the coordinate reference system, and the number of surveys.

The "header" attribute of a Collection contains information about each borehole, such as its ID, x- and y-coordinates, and other metadata. It also contains geometry objects for each borehole, which enable spatial selections and exports to GIS-supported formats provided by GeoST.

The "header" attribute is a geopandas.GeoDataFrame instance. Let’s see what the attribute looks like by printing it:

usp_boreholes.header.head()  # First five rows of the header data.
nr x y surface end geometry
0 B31H0541 139585 456000 1.20 -9.90 POINT (139585 456000)
1 B31H0611 139600 455060 1.20 -23.00 POINT (139600 455060)
2 B31H0718 139950 455200 1.30 -271.20 POINT (139950 455200)
3 B31H0803 139675 455087 2.16 -4.84 POINT (139675 455087)
4 B31H0806 139684 455384 1.00 -49.50 POINT (139684 455384)

Since the header is a GeoDataFrame, we have direct access to all methods provided by GeoDataFrames. Therefore, the interactive plot of the borehole locations shown above was easily created using the .explore() method. More experienced Python users can use the header to perform any customized operations with GeoDataFrames that they would normally perform.

The other key attribute of a Collection is the "data" attribute, which is a pandas.DataFrame instance. This contains the actual logged data (i.e., layer descriptions) of the boreholes. In this case, the "data" attribute contains layered data because the borehole data are logged in terms of layers (i.e., depth intervals over which properties are the same), with "top" and "bottom" depths. Let’s see what it looks like:

usp_boreholes.data.head()  # First five rows of the data table.
nr x y surface end top bottom lith zm zmk ... cons color lutum_pct plants shells kleibrokjes strat_1975 strat_2003 strat_inter desc
0 B31H0541 139585 456000 1.2 -9.9 0.00 0.20 K NaN NaN ... NaN ON NaN 0 0 0 NaN EC NaN [TEELAARDE#***#****#*] ..........................
1 B31H0541 139585 456000 1.2 -9.9 0.20 0.60 K NaN NaN ... NaN BR NaN 0 0 0 NaN EC NaN [KLEI#***#****#*] grysbruin.
2 B31H0541 139585 456000 1.2 -9.9 0.60 0.95 V NaN NaN ... NaN BR NaN 0 0 0 NaN NI NaN [VEEN#***#****#*] donkerbruin.
3 B31H0541 139585 456000 1.2 -9.9 0.95 2.80 Z NaN ZMFO ... NaN GR NaN 0 0 0 NaN EC NaN [ZAND#***#****#*] FYN TOT matig fyn# iets slib...
4 B31H0541 139585 456000 1.2 -9.9 2.80 4.20 Z NaN ZFC ... NaN BR NaN 0 0 0 NaN BXWI NaN [ZAND#***#****#*] fyn# grysbruin.

5 rows × 32 columns

Also with the "data" attribute, we have direct access to all methods provided by DataFrames. More experienced Python users can therefore use it to perform any customized operations they would normally perform with DataFrames. The "data" attribute of this Collection contains 32 different columns holding the relevant borehole data, describing characteristics such as lithology, sand grain size, plant remains, and others.

Positional reference#

A Collection contains all spatial information about the data, both horizontally and vertically. These attributes can be accessed through the "crs" (coordinate reference system) and "vertical_datum" attributes:

print(usp_boreholes.crs)
print(usp_boreholes.vertical_datum)

EPSG:28992
EPSG:5709

These attributes can be used to reproject the data. For example, we can convert the Dutch “Rijksdriehoekstelsel” coordinates to “WGS84” coordinates. Any reprojection automatically updates the coordinates in the data. Let’s change the coordinate reference system of usp_boreholes and check the "header" again to see this:

usp_boreholes.to_crs(4326)  # Change from RD to WGS 84
print(usp_boreholes.header.head(), usp_boreholes.crs, sep="\n")

         nr         x          y  surface     end                  geometry
0  B31H0541  5.162268  52.092042     1.20   -9.90  POINT (5.16227 52.09204)
1  B31H0611  5.162530  52.083594     1.20  -23.00  POINT (5.16253 52.08359)
2  B31H0718  5.167630  52.084862     1.30 -271.20  POINT (5.16763 52.08486)
3  B31H0803  5.163622  52.083839     2.16   -4.84  POINT (5.16362 52.08384)
4  B31H0806  5.163740  52.086508     1.00  -49.50  POINT (5.16374 52.08651)
EPSG:4326

Note that the coordinates in the "x" and "y" columns have indeed been changed to latitude and longitude coordinates.

Selections and slices#

There are several ways to create subsets of a Collection, such as:

Spatial selections

  • select_within_bbox - Select data points in the Collection within a bounding box.

  • select_with_points - Select data points in the Collection within a specified distance of other point geometries.

  • select_with_lines - Select data points in the Collection within a specified distance of line geometries.

  • select_within_polygons - Select data points in the Collection within polygon geometries.

Conditional selections

  • select_by_values - Select data points in the Collection based on the presence of certain values in one or more data columns.

  • select_by_length - Select data points in the Collection based on length requirements.

  • select_by_depth - Select data points in the Collection based on depth constraints.

Slicing

  • slice_depth_interval - Slice boreholes in the Collection to the specified depth interval.

  • slice_by_values - Slice boreholes in the Collection based on values (e.g., keep only sand layers and remove all others).

We will not go through each of these methods in this quick start, but please see the API Reference for more details.