INT

Empowering Visualization

CONTACT US SUPPORT

IVAAP

How IVAAP Maximizes Use of HATEOAS Links

Ever since the concept of web services first gained popularity, developers attempting to use these web services have faced two challenges: The first challenge is finding the right service to use; the second challenge is writing the code to call these services. The goal of this article is to describe how IVAAP uses HATEOAS hypermedia links to address both problems. We’ll also try to highlight other use cases benefiting from the concept of hypermedia applied to microservices.

A Brief Description of HATEOAS

Most microservices developed today (including IVAAP’s) use a REST API. REST stands for “REpresentational State Transfer.” It’s a term coined by Roy Fielding, who is also the inventor of “Hypermedia as the Engine of Application State,” or HATEOAS. It describes REST services that use hypermedia links to describe how they relate to other services.

For example, in IVAAP, the metadata of a seismic dataset is typically accessible through a URL such as this one:

https://…/ivaap/api/ds/geofiles/v1/sources/a8b05811-6409-43bb-8902-c9142ab48cff/seismic/cG9zZWlkb24gdmRzIG4gc2VneS9Qb3NlZGlvbiBkZXB0aC9wc2RuMTFfVGJzZG1GX2Z1bGxfd19BR0NfTm92MTFfdmVsNV9kZXB0aF8zMmJpdHMueGd5

Unlike the JSON content returned by a classic REST service, the JSON content returned by this IVAAP service contains more than just the requested metadata. It also contains a “links” JSON node that leads to additional information about this seismic dataset.

Picture1A sample JSON output for the IVAAP “seismic metadata” service, as shown in Google Chrome Developer Tools


In the example above, there are multiple HATEOAS links. One of them is the “Geometry” link. The purpose of the seismic geometry service is to expose the shape of seismic surveys. The URL of this service is:

https://…/ivaap/api/ds/geofiles/v1/sources/a8b05811-6409-43bb-8902-c9142ab48cff/seismic/cG9zZWlkb24gdmRzIG4gc2VneS9Qb3NlZGlvbiBkZXB0aC9wc2RuMTFfVGJzZG1GX2Z1bGxfd19BR0NfTm92MTFfdmVsNV9kZXB0aF8zMmJpdHMueGd5/geometry

This Geometry service is meant to be used by applications showing seismic datasets on a map. An application leveraging HATEOAS links would typically examine the “links” returned by the “seismic metadata” service to retrieve the URL of the associated “geometry” service. An application not leveraging HATEOAS links would hardcode the logic that /geometry needs to be added to the URL of the “seismic metadata” service to get the same result.

Both approaches are valid, but the HATEOAS approach brings multiple benefits that we are going to detail.

The “Broken Link” Issue Applied to Data

If you surfed in the ’90s, you are certainly familiar with the concept of “broken links.” Back in those days, websites were made of pages maintained by hand, and text on these web pages was often peppered with underlined words (often colored in blue) leading to another page. If the target page was moved, the link would stop working and the web surfer would be greeted by an unhelpful 404 error instead.

The lesson from these early days is that web page URLs are anything but permanent. The initial idea behind HATEOAS links is that this lesson can be applied to web services, too. If an application uses a hard-coded service URL to read data, this application will immediately stop working if that web service is moved. If each microservice describes the URL of related microservices, then the application can just follow URLs instead of using hard-coded versions. The maintenance of URLs becomes a server concern, and no longer a service consumer concern.

The main issue with this concept is that its purported benefits are unproven in the real world. To prove the benefit of this “forward compatibility” approach, you’d have to observe the life-cycle of many microservices (and consumers of these microservices) over a long period of time to determine whether the use of HATEOAS links was worth it. Taking IVAAP as an example, even though the IVAAP microservices use HATEOAS links, changing the URL of an IVAAP REST API is a rare event. One of the reasons is that there is no way to enforce the use of these links on the service consumer side. HATEOAS doesn’t provide a guarantee that no consumer hard-coded any URL. It is even sometimes faster to use hard-coded URLs, for example, to restore dashboards.

The second issue is that the “broken link” issue is a narrow backward compatibility concern, focused on URL changes only. While services may move, their API might also change, and HATEOAS doesn’t provide a way to address the backward compatibility of service APIs.

Discoverability

While HATEOAS links may help address issues associated with changing URLs, HATEOAS links really shine when it comes to discoverability. A component like the IVAAP Data Backend has hundreds of microservices. It doesn’t matter if each one of these services is documented, just finding whether a service exists is a complex task. HATEOAS links clearly indicate all URLs related to the data being accessed, in a consistent manner.

IVAAP is a platform. It was designed so that INT customers can modify the user interface using the IVAAP Client SDK, and we strive to make it as easy as possible. HATEOAS links give contextual documentation of the services that are available for any server-side object that the UI is accessing. As a result, modifying the IVAAP UI doesn’t require client-side developers to discover the server-side REST API before getting started. Developers can be immediately productive.

Testing

Developing a web-based application like IVAAP has a bit of a chicken-and-egg problem. You need to start by developing the data services first, but you don’t really know how well they work until the UI consuming these services is complete.

To get ahead of the game, there are methods to unit test data services, but they are time-consuming to follow. Just building by hand the right URL to test takes time, especially with long URLs. And because data quality varies, bugs might be data-specific and you need to test a bunch of them to make sure your data services are rock solid.

 

Following HATEOAS Links with Postman

Following HATEOAS links with Postman.


A widely used tool to inspect and test web services is Postman. Postman “understands” HATEOAS links and testing your work just consists of following links within Postman, just like you would do with an HTML-based website.

The most common use case of the IVAAP Data Backend SDK is when INT customers write a connector that accesses a proprietary data source. The testing steps of such a connector are typically very fast because they don’t require the UI to be ready. Most bugs can be found immediately, just using Postman.

Going Beyond: Automatic UI Generation

Discoverability and testability are well-known benefits of including HATEOAS links in a REST API. IVAAP also uses HATEOAS links to generate part of its user interface. For example, the tree that is shown to users when they open a well is server-driven, not client-driven. The IVAAP UI parses the HATEOAS links and builds a tree of nodes based on them.

Not all wells have the same details of data. Some wells might only have a location, others might have a trajectory. The presence of relevant HATEOAS links is what gives the UI the information on which data is available for that well. The IVAAP UI doesn’t need to understand what a trajectory is to include a trajectory node under a well node. The tree is generated automatically from HATEOAS links.

The Nodes Under the “AKM-11” Well (left)

The nodes under the “AKM-11” well (left), as listed by the HATEOAS link for that well (right).


Not all HATEOAS links associated with an object are meant to be shown as nodes in the UI. By convention, only the HATEOAS links with the attribute “children” set to true should be shown. Customers who want to customize the nodes shown in the UI don’t need to write client-side code. They have complete control by just plugging their own code into the Data Backend.

The same technique is used to build the UI, allowing users to add datasets to their projects. The Data Backend advertises through HATEOAS links how data from a connector can be browsed, and the UI parses these HATEOAS links to build a matching user interface.

User Interface Generated when Listing Wells in a “mongo” Connector

User interface generated when listing wells in a “mongo” connector.


Each data source has different capabilities, and this is reflected in the user interface. Some data sources might support search by name, paging, or sorting. For example, when search by name is supported on the server side, the IVAAP UI may propose a search box. IVAAP advertises querying capabilities to the user interface by including a “supportedQueries” attribute along its HATEOAS links.

A Sample JSON Output for the IVAAP “connector” Service

A sample JSON output for the IVAAP “connector” service, as shown in Google Chrome Developer Tools.

Likewise, it is sometimes convenient to be able to edit the name of a dataset from the same user interface. Not all data sources support name editing, and it’s only when editing is supported by a connector that a relevant HATEOAS link should be included in the server responses.

A Sample JSON Output for the IVAAP “connector” Service, as Shown in Google Chrome Developer Tools

A sample JSON output for the IVAAP “connector” service, as shown in Google Chrome Developer Tools.

In the response above, not only the Data Backend advertises that data names can be edited, but it also indicates it supports data deletion.

This concept of automated UI generation using HATEOAS links is not a standard use. It requires the addition of attributes that are typically not seen in web services using HATEOAS links. They have powerful tools as they reduce the amount of work on the client side. Actually, the IVAAP REST API is designed to support more complex than the two use cases already mentioned.

A majority of the IVAAP REST services are either services returning a collection of meta-data, or services returning the meta-data of a single dataset. The JSON format of these two types of services is standardized across the IVAAP Data Backend. Because the services provide consistent JSON outputs, it is easy to write a generic UI that will browse through the entire tree of meta-data and even allow editing. In other words, you can write a basic IVAAP client from scratch without much effort on the UI side.

A completely automated UI would not be limited by the search and editing capabilities advertised in HATEOAS links. Each IVAAP REST service comes with its own documentation. This documentation is accessible in the OpenAPI 3.0 format using a standard mechanism.

In this mechanism, if the URL of the service listing wells in a MongoDB database is: https://…/ivaap/api/ds/mongo/v1/sources/a8b05811-6409-43bb-8902-c9142ab48cff/wells/, the URL of its matching OpenAPI documentation would be: https://…/ivaap/api/ds/mongo/v1/sources/a8b05811-6409-43bb-8902-c9142ab48cff/wells/openapispecs

A completely automated UI could expose the search parameters described in this documentation, a bit like SwaggerEditor does. This wouldn’t be limited to search, the same principle can be applied to updating and deleting data.

 

A Form Generated Automatically by SwaggerEditor

A form generated automatically by SwaggerEditor from the OpenAPI specification of the wells service.

 

Going Beyond: Batch Support

Another feature enabled by HATEOAS links is the ability to fetch multiple aspects of a dataset in one HTTP call.

Microservices work best when they do only one thing at a time, but this means the IVAAP client needs to make multiple calls to the Data Backend to restore a dashboard. Currently, Google Chrome only allows up to 6 concurrent HTTP connections per host, sometimes forcing the client to “wait” for the availability of connections. This has a direct impact on the user experience.

To help with this, the IVAAP Data Backend provides a so-called “Batch” REST API to retrieve the content behind multiple URLs in one go. Other servers also have this feature, but what’s different about IVAAP’s Batch API is that it allows developers to leverage HATEOAS links.

For example, if you are building a data map and need to retrieve the metadata of a seismic dataset along with its outline, you would specify to the Batch REST API that you need to retrieve the content behind  https://…/ivaap/api/ds/geofiles/v1/sources/a8b05811-6409-43bb-8902-c9142ab48cff/seismic/cG9zZWlkb24gdmRzIG4gc2VneS9Qb3NlZGlvbiBkZXB0aC9wc2RuMTFfVGJzZG1GX2Z1bGxfd19BR0NfTm92MTFfdmVsNV9kZXB0aF8zMmJpdHMueGd5 as well as the content behind the associated “Geometry” HATEOAS link. This method of fetching multiple aspects of a dataset at once is much more expressive than passing multiple URLs to the Batch REST API.

Something that should be noted when it comes to performance is that we made the HATEOAS links an optional feature of IVAAP. Consumers of the IVAAP Data Backend API who don’t use these links can opt to reduce the size of the JSON payload between the client and the server. The default behavior of the Data Backend is to include HATEOAS links, but the collection services can be called in a way that excludes these links completely or only includes specific, named links.

Conclusion

HATEOAS links have been a part of the IVAAP Data Backend since day one. Over time, we found that they pack much more functionality than we initially thought. All these features have a common goal: facilitating the work of the UI developers and accelerating the delivery of software. While I used examples from IVAAP, the ideas in this article can easily be applied to your own data backend.

For more information or for a free demo of IVAAP, visit int.com/products/ivaap/.

INT and ANPG Improve Collaboration on Oil & Gas Concessions by Creating Virtual Data Room Leveraging IVAAP Technology

The ANPG VDR solution—implemented with a unique partnership between MIAPIA, SATEC and INT—enables companies to remotely assess the potential and requirements for better block concession management.

INT announced today the completion of a Virtual Data Room (VDR) for the Angola’s National Agency for Oil, Gas and Biofuels (Agência Nacional de Petróleo, Gás e Biocombustíveis) (“ANPG”), developed through collaboration with SATEC and MIAPIA.

The ANPG VDR is a cloud platform based on INT’s IVAAP subsurface data visualization platform and implemented with and by SATEC and MIAPIA. The platform allows secure access to E&P data managed by the Gabinete de Archivo de Datos (Data Management Department) of ANPG. The platform restricts user access to specific, predefined data sets.

Some of the VDR’s greatest functionalities are the support for the most important objects and formats in the sector (well logs, 2D and 3D seismic, trajectories, or lithology) through connectors to databases that store E&P information in the most common file formats: LAS, DLIS, SEG-Y. It also allows the visualization and research of these data on a GIS platform and access and preview of technical reports in PDF format.

“ANPG is very pleased with the implementation of the Virtual Data Room solution, a strategic tool for the promotion of Angola’s oil potential and the attraction of foreign investment, as well as the digital transformation of the organization itself,” said Lúmen Sebastião, GAD Director, ANPG.

To date, the VDR supplies E&P data related to offshore blocks, Baixo Congo Basin, and Kwanza Basin. More than 20 VDR sessions have been held, with the participation of 15 companies. Furthermore, ANPG is participating in benchmarking activities, sharing with colleagues from other countries their experience with the VDR solution.

To learn more about the ANPG VDR, visit int.com/anpg-vdr/ or for a preview of IVAAP, visit int.com/ivaap/.

For more information, please contact us at intinfo@int.com.

____________
ABOUT INT
INT software empowers energy companies to visualize their complex data (geoscience, well, surface reservoir, equipment in 2D/3D). INT offers a visualization platform (IVAAP) and libraries (GeoToolkit) that developers can use with their data ecosystem to deliver subsurface solutions (Exploration, Drilling, Production). INT’s powerful HTML5/JavaScript technology can be used for data aggregation, API services, and high-performance visualization of G&G and energy data in a browser. INT simplifies complex subsurface data visualization.

INT, the INT logo, and IVAAP are trademarks of Interactive Network Technologies, Inc., in the United States and/or other countries.

How to Generate Dynamic Forms Using JSON in IVAAP

In late 2020, INT decided to make it easier for IVAAP developers and SDK users to quickly and efficiently create forms and dialogs. After working with the teams to determine the best way, the solution became clear: JSON forms. 

What Are JSON Forms?

So, what is a JSON form? First, JSON is an open standard file format and data interchange format that uses text to store and transmit data objects. This format allows users to create JSON Forms using JSON Schema. JSON Schema is a vocabulary that allows you to annotate and validate JSON documents. The schema itself will annotate the data model. 

As human beings, we can read the schema ourselves and know what properties are included in a data model, what the possible values are, etc. Programmatically with a schema, we can validate our data to make sure that the data model we’re taking as input or providing as output is in the correct shape it needs to be for the purpose it needs to serve.

How Do You Format a JSON Form? 

Formatting or shaping user data is always going to depend on the requirements of what you’re developing. So for an actual example, let’s say for our use-case we have a user, and we want to represent some data about that user with JSON. What’s the correct way to format or shape the user data? 

For our example, let’s say we have two different ways to represent the same basic qualities about a user, with their name and age. The first format just has the name as a single string, with their first and last name and also age as an integer. Using the first example, but this time, the name property itself is an object with a first name property and the last name property, which are both strings. Now with this format, I can write a JSON Schema to precisely define what I’ll consider valid and anything that’s structured any other way is invalid. So you can see that my schema says I’m expecting an object, that object will have a name property and an age property that’s an integer. But with the name, I’m expecting that that property itself should be an object with separate properties for first and last names, that are both string types.

So now you have your concrete data model and the schema that defines which format is valid. The final piece that’s needed to be able to actually use this is a validator. Using your validator, you provide the schema and then pass your data, where it can process it as either valid or invalid and give you an output for you to handle accordingly. This pattern isn’t particular to just JSON either, there are schema specifications and validators, for example, for XML documents.

How Does It Work?

So, with all that covered, you can now put it all together to explain how JSON Forms works and what’s required to make it work.

The package is split into a core library and submodules for each of the three currently-most-popular front-end frameworks — React, Vue, and Angular. These submodules provide a component written natively for each framework. You’ll have a native React or an Angular component that you use just like any other third-party component you might add as a dependency through an npm install.

Since it’s just a typical front-end component, JSON Forms can take props just like any standard component you might write. Now, get the forms rendered by providing a validator class. INT currently uses AJV, which is the most popular open-source validator written in JavaScript. Then comes the actual schemas that we provide. 

There are three pieces of JSON data we can pass into JSON Forms, two that are required and one that is optional: 

  • First is the schema that describes our data model — see it referred to as just “schema.” This should look similar to the example schema previously explained with the name and age of a user.
  • Second is the UI schema — this is where you describe certain properties of the UI for the form.
  • Third, which is optional, you can pass a pre-existing data model as well and this will pre-populate the forms.  

You now have your data schema that also describes your data model. And you have the UI schema — where, like the name suggests, we can define various aspects of the UI of the form. This is where we essentially map the properties in our data schema to the actual form control that’s going to be rendered to the screen. Something that’s not shown here but that you can also define here is some extra options for grouping and layouts. So you can group certain controls together and also define if you want certain controls laid out horizontally or vertically. The data model is just the concrete data that follows the schema laid out by the data schema. With all of that together, JSON Forms can render a form with two controls, already pre-filled with those values.

Another helpful way to conceptualize what JSON Forms does is that it’s basically a way to turn a data model into a set of forms so you can edit or update that data model.  

How Does INT Use JSON Forms?

The last feature that brings it all together and allows you to cleanly integrate JSON Forms into IVAAP is the ability to create your own custom controls on top of JSON Forms.

JSON Forms comes packaged out of the box with its own library of controls. The UI utilizes Google’s Material UI designs. If you are familiar with Material UI, you’ve seen controls like this before. It’s very minimal, but still a very well-functional UI design. This is helpful because if a user just needs a form in their app, all that’s required of them to do is to write up data schemas for their particular model. Then the UI schema lays out the controls based on how the user wants them and they’re good to go.

If you’re developing a large web application that already has a well-defined UI and theming that also deals with a very particular domain that may require more specialized controls, JSON Forms gives you the ability to develop your own custom controls, giving you total control over how you want the look and feel of the controls to be. 

The great thing here is that you can develop these custom controls as components, for any of the three frameworks mentioned. So you can still use the tools and API of what you’re already familiar with to more efficiently add more controls as you need them. You just need to make sure to set it up to accept a few props that JSON Forms will pass down from its root-level component. 

Final Thoughts

While there are other ways to implement forms and dialogs, we found that using JSON Forms – with its ability to dynamically generate dialogs and to create custom controls to meet our feature requirements – to be an impressive tool that improves our developer and user experience.

How Apache SIS Simplifies the Hidden Complexity of Coordinate Systems in IVAAP

See how Apache SIS, with IVAAP, helps support our client’s coordinate systems by using less code.

With the recent release of IVAAP 2.9, now is a good time to reflect on the help we got along the way. One of the components that made IVAAP possible is the Apache SIS library. The goal of this blog article is to bring more visibility to this awesome Java library.

What Is the Apache SIS Library?

Apache SIS is an open-source library written in Java that makes it easier to develop geospatial applications. Many datasets have metadata designating their location on Earth, and these locations are relative to a datum and a map projection method. There are many datums and many map projection methods. Apache SIS facilitates their identification and the accurate conversion of coordinates between them.

What’s a Datum and What’s a Map Projection Method?

Most people are familiar with latitude and longitude coordinates. This geographic coordinate system has been used for maritime and land-based navigation for centuries. Since the late 1800s, the line defining 0º of longitude has been the Prime meridian, crossing the location of the Royal Observatory in Greenwich, England. This meridian defined one axis, from South to North. The equator defined the other axis, from West to East. The origin point of this system on the Earth’s surface is in the Gulf of Guinea, 600 km off the coast of West Africa.

The traditional geographic coordinate system
The traditional geographic coordinate system (Source)

 

Similarly, a datum defines the origin point of the coordinate axes on the Earth’s surface and defines the direction of the axes. To account for the fact that the Earth is not a perfect sphere, a datum also describes the generalized shape of the Earth. For example, WGS 84 (World Geodetic System 1984) is a widely-used global datum based on latitudes and longitudes where the Earth is modeled as an oblate spheroid, centered around its center of mass.

The WGS 84 reference frame. The oblateness of the ellipsoid is exaggerated in this image. (Source)

 

WGS 84 is used by GPS receivers and the longitude 0º of this datum is actually 335 ft east of the Greenwich meridian.

While universal latitude and longitude coordinates are convenient, they are not universally practical because of land masses drift. Satellite measurements show that the location of Houston relative to the WGS 84 datum changes by 1 inch each year. A local datum is a more pragmatic choice than a global datum because distances from a local point of reference are smaller and don’t change over the years when all locations are on the same tectonic plate. A local datum may also align its spheroid to closely fit the Earth’s surface in this particular area.

A map projection method indicates how the Earth’s surface is flattened into a plane in order to make a 2D map. The most widely known projection method was presented by Gerardus Mercator in 1569. This is a global cylindrical projection method. It preserves local directions and shapes but distorts sizes away from the equator.

Cylindrical Projection
An example of global cylindrical projection (Source)

 

In the US, the Lambert Conformal Conic projection has become a standard projection for mapping large areas. This is a projection that requires multiple parameters, defining the longitude and latitude of its center, a distance offset to this center, and the latitude of its southern and northern parallels.

Conical Projection
An example of local conical projection (Source)

 

When a datum and a projection are used together, they define a projected coordinate reference system. While local systems limit distortions, they are only valid in a small area, an area known as the ”area of use” where a minimum level of accuracy is guaranteed.

Select Coordinate System
A screenshot from INTViewer showing the area of use of NAD27 / Wyoming East Central, a derived projected coordinate reference system

 

How Does Apache SIS Help IVAAP?

To show geoscience datasets on one of IVAAP’s 2D map widgets, you need to use a common visualization coordinate reference system.

IVAAP Screenshot
An IVAAP screenshot showing the location of wells on a map

 

This is where Apache SIS helps us: It understands the properties of both the data and visualization systems and is able to convert coordinates between them.

The math to perform these conversions is complex, this is not something you want to implement on your own. It requires specialized skills, both as a programmer and a domain expert. And just beyond the math, the number of datums and projection methods is mind-boggling. Many historical surveys are still in use today. For example, there are two datums used for making horizontal measurements in North America: the North American Datum of 1927 (NAD 27) and the North American Datum of 1983 (NAD 83). The two datums are based on two different ellipsoid models. As a result, the two datums have grid shifts of up to 100 meters, depending on location. IVAAP is able to visualize datasets that used NAD 27 as a reference, and it is Apache SIS that makes it possible to accurately reproject these coordinates into modern coordinate systems, accounting for their respective datum shift.

The datum shift between NAD 27 and NAD 83 (Source)

 

The oil and gas industry is at the origin of some of these local coordinate systems. Many of today’s new oil fields are in remote areas, initially lacking a geographical survey. There is an organization called the “OGP Surveying and Positioning Committee” which keeps track of these coordinate systems. It is colloquially known as “EPSG” for historical reasons. It regularly provides a database of these coordinate systems to all its members. This database is used by IVAAP and Apache SIS provides a simple API to take advantage of it. Each record in this database has a numerical WKID (Well Known ID). To instantiate a projection method or a coordinate system defined in this database, you just need to prefix this id with the “EPSG:” string.

OperationMethod method = getCoordinateOperationFactory().getOperationMethod("EPSG:9807"); // Transverse Mercator method

CoordinateReferenceSystem crs = CRS.forCode("EPSG:32056”);

 

The EPSG database itself is extensive, but it is common for INT customers to use unlisted coordinate reference systems, created for brand new oil fields. In these cases, a WKT (Well Known Text) string can be used instead. This text is a human-readable description of a projection method or coordinate system. The Apache SIS provides a clean API to parse WKTs. It also provides an API for formula-based projection methods that can’t be described by a WKT.

PROJCS["NAD27 / Wyoming East Central",
    GEOGCS["NAD27",
        DATUM["North_American_Datum_1927",
            SPHEROID["Clarke 1866",6378206.4,294.9786982139006,
                AUTHORITY["EPSG","7008"]],
            AUTHORITY["EPSG","6267"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4267"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",40.66666666666666],
    PARAMETER["central_meridian",-107.3333333333333],
    PARAMETER["scale_factor",0.999941177],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",0],
    UNIT["US survey foot",0.3048006096012192,
        AUTHORITY["EPSG","9003"]],
    AXIS["X",EAST],
    AXIS["Y",NORTH],
    AUTHORITY["EPSG","32056"]]

The WKT of NAD27 / Wyoming East Central, with the WKID 32056

Why Did INT Choose Apache SIS Over Other Options?

INT had previous experience using GeoTools. Similarly to Apache SIS, GeoTools is a Java library dedicated to facilitating the implementation of geographical information systems. Being an older library, it goes much further than Apache SIS. For example, one of its components allows the parsing of shape files, something currently outside of the scope of Apache’s library. As a matter of fact, the first versions of IVAAP were using GeoTools for coordinate conversions.

One of the issues we encountered with GeoTools is that it is a library that provides only fine-grained Java conversion APIs. There are several paths to convert coordinates between two systems, and GeoTools allows the developer to choose the best method. Choosing the “best” method without human interaction is complex; it depends on the extent of the data being manipulated and the “area of use” of each coordinate reference system involved. It also depends on the availability of well-known transformation algorithms between datums. In North America, the standard for transformations between datums was formerly known as NADCON. The rest of the world uses a standard known as NTV2. Apache SIS works with both datum shift standards. It may elect to use WGS 84 as a hub when no datum shift is applicable. An algorithm to pick the best method would require a significant amount of code for INT to write and maintain. While Apache SIS allows fine-grained control over the different transformations used when converting from one coordinate reference system into another, it also provides a high-level API to perform this conversion. The picking of the best algorithm is part of the Apache SIS’ implementation. Its high-level Java API that picks a conversion algorithm matches IVAAP’s general use microservice for the same function. To pick the right algorithm, it only takes 3 parameters:

  • A definition of the “from” coordinate system
  • A definition of the “to” coordinate system
  • A description of the “extent” of the coordinates to convert
double x = …
double y = …
GeographicBoundingBox extentInLongLat = …
DirectPosition position = new DirectPosition2D(x, y);
CoordinateReferenceSystem fromCrs = CRS.forCode("EPSG:32056");
CoordinateReferenceSystem toCrs = CRS.forCode("EPSG:3737");
CoordinateReferenceSystem displayOrientedFromCrs = AbstractCRS.castOrCopy(fromCrs).forConvention(AxesConvention.DISPLAY_ORIENTED);
CoordinateReferenceSystem displayOrientedToCrs = AbstractCRS.castOrCopy(toCrs).forConvention(AxesConvention.DISPLAY_ORIENTED);
CoordinateOperation operation = CRS.findOperation(displayOrientedFromCrs, displayOrientedToCrs, extentInLongLat);
MathTransform mathTransform = operation.getMathTransform();
double[] coordinate = mathTransform.transform(position, position).getCoordinate();

Sample code to convert a single x, y position from “NAD27 / Wyoming East Central” to “NAD83 / Wyoming East Central”

We still use GeoTools for other parts, but as a general rule, the Apache SIS Java API tends to be simpler, more modern than GeoTools when it comes to manipulating coordinates and coordinate systems.

After 3 years of use, we are happy with our decision to move to Apache SIS. This library allows us to support more of our customers’ coordinate systems, with less code. We are also planning to use it to interpret the metadata of GeoTIFF files. The support has been excellent. When we needed help, the members of the Apache SIS development team were really keen to help us. This is one of the reasons why INT felt we needed to give back to the open-source community. Being a long-time member of OSDU, INT contributed to OSDU a coordinate conversion library built on top of Apache SIS. This coordinate conversion library converts GeoJSON and trajectory stations between different coordinate reference systems. Users can specify the specific transformation steps that will be used in the conversion process, either through EPSGs or WKTs. Behind the scenes, it’s the Apache SIS’ fine-grained API that is being used.

Improving Seamless Visualization and Embedding Dynamic Visualization with IVAAP 2.9

Companies in the energy industry face several challenges when it comes to streamlining large datasets, finding compatibility among applications, and visualizing different types of data in real-time. The latest release of INT’s IVAAP Data Visualization Platform focuses on solving these challenges by making it more efficient to find and manage subsurface data.

Seamless visualization of domain data goes beyond just accessing the data in the cloud — to maximize your workflow, you must be able to search through data quickly, execute processes and workflows, and visualize the results in one platform. IVAAP’s custom SDK embeds IVAAP’s widgets in users’ applications to add the flexibility to customize the interface, connectors, and a dynamic UX/UI. 

New & Improved Widgets: Diagram, Image, Well Log, and 3D

A new widget has been added to IVAAP’s list of widgets — the diagram widget.  The diagram widget displays SVG data from IVAAP combined with numerical curve values in real-time, supporting animations of your data. The image widget has been improved to support very large TIFFs (Tiled Image Support). Users can visualize their large raster image files by saving them in the cloud and IVAAP creates a single point of access to these files, making compressed rendering images from their original format. 

We also added improvements to other existing widgets. IVAAP’s existing WellLog widget offers Single Data, Multi Data, and Multi Parent modes to display very complex well logs and switch between wells smoothly. The 3D visualization widget’s high performance reservoir visualization is now combined with curser tracking between 3D Basemap and 2D seismic lines and overlay. The basemap feature also supports tops, faults, point sets, and more. 

Empowering Subsurface Workflows

IVAAP has offered the ability to connect to any third-party application for a while, allowing users to integrate in-house workflows and customize visualizations to create a seamless experience. For many companies with data located in more than one repository or cloud database, accessing all of this data through one unique portal streamlines the process and standardizes data access. IVAAP can communicate with external engines, such as ML platforms, processing engines  — Tensorflow and SageMaker— and code can be written in Python and executed through the IVAAP client interface.  

With these new and improved features, IVAAP 2.9 enables easy search and visualization of energy, geophysical, and production data in the cloud. It allows product owners, developers, and architects to build subsurface digital solutions faster without having to start from scratch. 

For more information, please visit www.int.com or contact us at intinfo@int.com.

Visit us online at int.com/ivaap for a preview of IVAAP or for a demo of INT’s other data visualization products.