Introduction to GeoST#

This quick introduction will cover some of the key concepts and basic features of GeoST to help you get started. GeoST depends heavily on popular data science libraries Pandas and GeoPandas but GeoST provides readily available, frequently used selections on data held in DataFrame or GeoDataFrame objects. This makes GeoST an easy to use option for less experienced Python users while more experienced users can easily access the underlying DataFrames and develop their own functionalities.

GeoST is designed to work with many different kinds of subsurface data that is available in The Netherlands. GeoST is a work-in-progress and aims to support an increasing number of data sources. Below is a list of different data sources which are currently supported or will be supported by GeoST in the future:

From local files:

Tabular data of borehole, CPT, etc. (.parquet, .csv)
Geological boreholes xml (BHR-G)
Geotechnical boreholes xml (BHR-GT)
Pedological boreholes xml (BHR-P)
Cone Penetration Test xml/gef (CPT)
Pedological soilprofile descriptions xml (SFR)
BORIS (TNO borehole description software) xml

Directly from the BRO REST-API:

BHR-G
BHR-GT
BHR-P
CPT
SFR

BRO models:

GeoTOP: from local NetCDF or directly via OPeNDAP server

Planned:

BRO/PDOK geopackages: BHR-G, BHR-GT, BHR-P, CPT, SFR
Well logs LAS/ASCII
REGIS II
Dino xml geological boreholes
BHR-G gef
Soilmap of the Netherlands

GeoST also plans support for several Geophysical data sources such as Seismic, ERT, EM and others.

Concept#

At the core, GeoST handles data in a so-called Collection objects which holds all the spatial information of any kind of data source in a “header” attribute (i.e. header table), and the corresponding data in a “data” attribute (i.e. data table). So for example, a set of 100 boreholes is held in a BoreholeCollection where the “header” contains one row per data entry and provides information about the id, location, surface level and depths and the “data” has the information of each described layer. When working with these Collections, GeoST automatically keeps track of the alignment and thus makes sure each data entry occurs in both the “header” and “data” attributes. For example, when a user deletes an individual borehole entry from the “header”, the Collection ensures it is deleted from the “data” as well.

User guide

For a more detailed explanation of the types of GeoST objects for different sources of data, check the Data structures in the user guide.

The Basics#

BoreholeCollection#

Data is usually loaded through various reader functions (see API reference). For this tutorial, GeoST provides a set of readily available boreholes in the area of the Utrecht Science Park which can be directly loaded as a BoreholeCollection. Let’s read the data, print the result to see what it says and also plot the locations to get an idea where we are:

import geost

usp_boreholes = geost.data.boreholes_usp()
print(usp_boreholes)
usp_boreholes.header.explore()  # Interactive plot of the borehole locations.

BoreholeCollection:
# header = 67

Make this Notebook Trusted to load map: File -> Trust Notebook

As you can see it says that ‘usp_boreholes’ is of the type BoreholeCollection. Additionally, it says # header = 67. This means that the collection in total contains 67 boreholes but it also shows the first key attribute of a collection: the “header” attribute.

As said in the previous section, the “header” attribute in a BoreholeCollection contains all the information about each borehole such as the ID, x- and y-coordinates and further metadata. Additionally, it contains geometry objects for each borehole which allows for spatial selections and exports to GIS-supported formats etc. that are provided by GeoST. The header attribute is a Geopandas GeoDataFrame instance. Let’s see what the attribute looks like by printing it:

usp_boreholes.header.head()  # First five rows of the header data.

	nr	x	y	surface	end	geometry
0	B31H0541	139585.0	456000.0	1.20	-9.90	POINT (139585 456000)
1	B31H0611	139600.0	455060.0	1.20	-23.00	POINT (139600 455060)
2	B31H0718	139950.0	455200.0	1.30	-271.20	POINT (139950 455200)
3	B31H0803	139675.0	455087.0	2.16	-4.84	POINT (139675 455087)
4	B31H0806	139684.0	455384.0	1.00	-49.50	POINT (139684 455384)

Since the header is a GeoDataFrame instance, we have direct access to all methods provided by GeoDataFrames. Therefore, the above interactive plot of the borehole locations was easily created using the .explore() method. More experienced Python users can therefore use the header to do any customized operation with GeoDataFrames they would normally do.

The other key attribute of a collection is the “data” attribute which is a Pandas DataFrame instance. This contains the actual logged data (i.e. layer descriptions) of the boreholes. In this case, the “data” attribute contains “layered” data because the borehole data is logged in terms of layers (i.e. depth intervals over which properties are the same) with “top” and “bottom”. Let’s see what it looks like:

usp_boreholes.data.head()  # First five rows of the data table.

	nr	x	y	surface	end	top	bottom	lith	zm	zmk	...	cons	color	lutum_pct	strat_1975	strat_2003	strat_inter	desc
0	B31H0541	139585.0	456000.0	1.2	-9.9	0.00	0.20	K	NaN	None	...	None	ON	NaN	None	EC	NaN	[TEELAARDE#*#*#] ..........................
1	B31H0541	139585.0	456000.0	1.2	-9.9	0.20	0.60	K	NaN	None	...	None	BR	NaN	None	EC	NaN	[KLEI#*#*#] grysbruin.
2	B31H0541	139585.0	456000.0	1.2	-9.9	0.60	0.95	V	NaN	None	...	None	BR	NaN	None	NI	NaN	[VEEN#*#*#] donkerbruin.
3	B31H0541	139585.0	456000.0	1.2	-9.9	0.95	2.80	Z	NaN	ZMFO	...	None	GR	NaN	None	EC	NaN	[ZAND#*#*#] FYN TOT matig fyn# iets slib...
4	B31H0541	139585.0	456000.0	1.2	-9.9	2.80	4.20	Z	NaN	ZFC	...	None	BR	NaN	None	BXWI	NaN	[ZAND#*#*#] fyn# grysbruin.

5 rows × 32 columns

Also with the “data” attribute, we have direct access to all methods provided by DataFrames and more experienced Python users can use it to do any customized operation. The “data” attribute of this collection contains 32 different columns that hold the relevant borehole data and describes characteristics such as lithology, sand grain size, plant remains and others.

Positional reference#

As said, a collection contains all spatial information about the data, both horizontally and vertically. These attributes can be accessed through the “horizontal_reference” and “vertical_reference” attributes:

print(usp_boreholes.horizontal_reference)
print(usp_boreholes.vertical_reference)

EPSG:28992
EPSG:5709

These attributes can be used to reproject the data. For example, changing Dutch “Rijksdriehoekstelsel” coordinates to WGS 84 coordinates or change the vertical reference from Dutch “NAP” to a “Mean Sea Level” plane. Any reprojection automatically updates the coordinates in the data. Let’s change the horizontal reference in “usp_boreholes” and checkout the “header” again to see this:

usp_boreholes.change_horizontal_reference(4326)  # Change from RD to WGS 84
print(usp_boreholes.header, usp_boreholes.horizontal_reference, sep="\n")

          nr         x          y  surface      end                  geometry
 B31H0541  5.162268  52.092042    1.200   -9.900  POINT (5.16227 52.09204)
 B31H0611  5.162530  52.083594    1.200  -23.000  POINT (5.16253 52.08359)
 B31H0718  5.167630  52.084862    1.300 -271.200  POINT (5.16763 52.08486)
 B31H0803  5.163622  52.083839    2.160   -4.840  POINT (5.16362 52.08384)
 B31H0806  5.163740  52.086508    1.000  -49.500  POINT (5.16374 52.08651)
..       ...       ...        ...      ...      ...                       ...
B32C1889  5.181734  52.092078    1.728   -2.772  POINT (5.18173 52.09208)
B32C1890  5.182016  52.090982    3.948    0.248  POINT (5.18202 52.09098)
B32C1891  5.183281  52.092062    2.112   -1.888  POINT (5.18328 52.09206)
B32C1892  5.184245  52.091776    1.769   -2.731  POINT (5.18424 52.09178)
B32C1893  5.186798  52.091987    2.328   -2.172   POINT (5.1868 52.09199)

[67 rows x 6 columns]
EPSG:4326

Note that the coordinates in the “x” and “y” columns have indeed been changed to latitude, longitude coordinates.

Selections and slices#

There are several ways to make subsets of a collection, such as:

Spatial selections

select_within_bbox - Select data points in the collection within a bounding box
select_with_points - Select data points in the collection within distance to other point geometries
select_with_lines - Select data points in the collection within distance from line geometries
select_within_polygons - Select data points in the collection within polygon geometries

Conditional selections

select_by_values - Select data points in the collection based on the presence of certain values in one or more of the data columns
select_by_length - Select data points in the collection based on length requirements
select_by_depth - Select data points in the collection based on depth constraints

Slicing

slice_depth_interval - Slice boreholes in the collection down to the specified depth interval
slice_by_values - Slice boreholes in the collection based on value (e.g. only sand layers, remove others).

We will not go through each of these methods in this quick start but please see the API Reference for more details.