INT

Empowering Visualization

CONTACT US SUPPORT

Products

How to Generate Dynamic Forms Using JSON in IVAAP

In late 2020, INT decided to make it easier for IVAAP developers and SDK users to quickly and efficiently create forms and dialogs. After working with the teams to determine the best way, the solution became clear: JSON forms. 

What Are JSON Forms?

So, what is a JSON form? First, JSON is an open standard file format and data interchange format that uses text to store and transmit data objects. This format allows users to create JSON Forms using JSON Schema. JSON Schema is a vocabulary that allows you to annotate and validate JSON documents. The schema itself will annotate the data model. 

As human beings, we can read the schema ourselves and know what properties are included in a data model, what the possible values are, etc. Programmatically with a schema, we can validate our data to make sure that the data model we’re taking as input or providing as output is in the correct shape it needs to be for the purpose it needs to serve.

How Do You Format a JSON Form? 

Formatting or shaping user data is always going to depend on the requirements of what you’re developing. So for an actual example, let’s say for our use-case we have a user, and we want to represent some data about that user with JSON. What’s the correct way to format or shape the user data? 

For our example, let’s say we have two different ways to represent the same basic qualities about a user, with their name and age. The first format just has the name as a single string, with their first and last name and also age as an integer. Using the first example, but this time, the name property itself is an object with a first name property and the last name property, which are both strings. Now with this format, I can write a JSON Schema to precisely define what I’ll consider valid and anything that’s structured any other way is invalid. So you can see that my schema says I’m expecting an object, that object will have a name property and an age property that’s an integer. But with the name, I’m expecting that that property itself should be an object with separate properties for first and last names, that are both string types.

So now you have your concrete data model and the schema that defines which format is valid. The final piece that’s needed to be able to actually use this is a validator. Using your validator, you provide the schema and then pass your data, where it can process it as either valid or invalid and give you an output for you to handle accordingly. This pattern isn’t particular to just JSON either, there are schema specifications and validators, for example, for XML documents.

How Does It Work?

So, with all that covered, you can now put it all together to explain how JSON Forms works and what’s required to make it work.

The package is split into a core library and submodules for each of the three currently-most-popular front-end frameworks — React, Vue, and Angular. These submodules provide a component written natively for each framework. You’ll have a native React or an Angular component that you use just like any other third-party component you might add as a dependency through an npm install.

Since it’s just a typical front-end component, JSON Forms can take props just like any standard component you might write. Now, get the forms rendered by providing a validator class. INT currently uses AJV, which is the most popular open-source validator written in JavaScript. Then comes the actual schemas that we provide. 

There are three pieces of JSON data we can pass into JSON Forms, two that are required and one that is optional: 

  • First is the schema that describes our data model — see it referred to as just “schema.” This should look similar to the example schema previously explained with the name and age of a user.
  • Second is the UI schema — this is where you describe certain properties of the UI for the form.
  • Third, which is optional, you can pass a pre-existing data model as well and this will pre-populate the forms.  

You now have your data schema that also describes your data model. And you have the UI schema — where, like the name suggests, we can define various aspects of the UI of the form. This is where we essentially map the properties in our data schema to the actual form control that’s going to be rendered to the screen. Something that’s not shown here but that you can also define here is some extra options for grouping and layouts. So you can group certain controls together and also define if you want certain controls laid out horizontally or vertically. The data model is just the concrete data that follows the schema laid out by the data schema. With all of that together, JSON Forms can render a form with two controls, already pre-filled with those values.

Another helpful way to conceptualize what JSON Forms does is that it’s basically a way to turn a data model into a set of forms so you can edit or update that data model.  

How Does INT Use JSON Forms?

The last feature that brings it all together and allows you to cleanly integrate JSON Forms into IVAAP is the ability to create your own custom controls on top of JSON Forms.

JSON Forms comes packaged out of the box with its own library of controls. The UI utilizes Google’s Material UI designs. If you are familiar with Material UI, you’ve seen controls like this before. It’s very minimal, but still a very well-functional UI design. This is helpful because if a user just needs a form in their app, all that’s required of them to do is to write up data schemas for their particular model. Then the UI schema lays out the controls based on how the user wants them and they’re good to go.

If you’re developing a large web application that already has a well-defined UI and theming that also deals with a very particular domain that may require more specialized controls, JSON Forms gives you the ability to develop your own custom controls, giving you total control over how you want the look and feel of the controls to be. 

The great thing here is that you can develop these custom controls as components, for any of the three frameworks mentioned. So you can still use the tools and API of what you’re already familiar with to more efficiently add more controls as you need them. You just need to make sure to set it up to accept a few props that JSON Forms will pass down from its root-level component. 

Final Thoughts

While there are other ways to implement forms and dialogs, we found that using JSON Forms – with its ability to dynamically generate dialogs and to create custom controls to meet our feature requirements – to be an impressive tool that improves our developer and user experience.

GeoToolkit.JS: New Features and Improvements in Carnac 3D

The Carnac3D module is part of the GeoToolkit.JS library and utilizes WebGL 2.0 technology for 3D rendering on web browsers. For those who are not familiar with it, WebGL doesn’t require an extra plug-in and is built-in for almost all browsers, such as Chrome, Firefox, Edge, and Safari. 

GeoToolkit’s 3D library is extremely useful for developing professional applications. It provides rich oil & gas-oriented components, including seismic, grid surface, well trajectory, well log, heightmap, reservoir, schematics, volume rendering, and a bunch of visualization tools like picking, highlighting, and cursor visuals.

From the relational graph, you can see Carnac3D is integrated with the GeoToolkit base module, and also uses Three.js. Three.js is an open-source library that provides many low-level functionalities like renderer and scene graph. It provides many useful data types that can be used directly in our users’ applications. In GeoToolkit 3.3, we upgraded our internal Three.js to version 126, to maintain compatibility. For users that care about the core implementation of 3D, and maybe want to extend GeoToolkit on their own, WebGL 2.0 is a subset of OpenGL ES 3.0, and the shader version is GLSL 3.0.

carnac

 

In 3.3, we added seismic intersection with other 3D objects and a new grid surface loader called EarthVision loader. We also added a new widget called the projection widget. For minor features, an option was added to display flat shading surfaces, and also an auto-rotating mode to the 3D plot.

For rendering, we improved the accuracy of transparency rendering, reservoir grid performance, gridline, and isoline rendering, and anti-aliasing.

New Feature: Seismic Intersection with Grid Surface

We know that we can add multiple objects to a seismic volume such as grid surface, horizon, and reservoir. They are all intersected with each other. This causes an issue that makes it hard to observe the details with so many objects on the screen. Users must adjust the camera constantly to find the best angle. To resolve this issue, we introduced IntersectionHelper.

 

horizon1

horizon2

The intersection helps us preserve the detail of seismic and objects. We can clearly see its position in inline, xline, and time slice.

IntersectionHelper also supports reservoir intersection. When we visualize a reservoir with seismic, some cells are hidden inside the reservoir. With intersection, we can better observe the properties at intersected planes.

horizon3D

horizon4

horizongif


New Feature: EarthVision Surface Loader and Un-Triangulated Data

EarthVision is a commonly used format in oil and gas applications for loading grid surfaces. It converts the field data to a format that can be recognized by a 3D renderer.  

The format specification is, for each line, it has the following: 

Column index and row index are optional. If they are not provided, the data loader will mark it as un-triangulated data. So in the next step, the SurfaceData class will triangulate vertices using a gridding algorithm. If indices are provided, the SurfaceData has more information, so it can perform a much faster triangulation.

EarthVision loader can load data from either cloud server or local files, as long as it has a proper URL. Then it sends the result to SurfaceData, which is a predefined data class for preparing the data that the GPU can read. We include that and send it to the GridSurface instance. By adding surface instances to the 3D plot, we can finally visualize it.

The specification is: 

xPos, yPos, zPos [, colIndex],[rowIndex]

 

The data loader automatically identifies the type of data and triangulates for users if necessary.

 

carnac2

 

New Feature: Projection Widget

GeoToolkit’s new projection widget can project objects onto its grid planes so that users can better visualize an object’s position in the x/y/z axis. And most importantly, it supports almost all objects in 3D. This widget includes a built-in grid, and it can add projections using only one line of code.

projection2

projection1

 

New Feature: Flat Shading Surface

A new minor feature that has been added to 3.3 is the flat shading surface. 

The first example below shows a surface with smooth shading. While most of the area is super smooth, when it comes to faults or cliffs, it isn’t that good. This is because two adjacent polygons have a huge gap in terms of the surface normal, meaning the transition is not smooth anymore. 

unnamed (1)

 

To solve this uncomfortable artifact, we use flat shading. For flat shading, the lighting is evaluated only once for each polygon, and each polygon only has one color. You can see this in the second screenshot, where faults and cliffs are preserved very well. Here, the lighting is evaluated only once for each triangle so each triangle only has one light level.

 

unnamed (7)

 

Improvement: Accurate Order-Independent Transparency Rendering

One of the most distinct improvements is the switch to accurate transparency rendering. To produce accurate transparency in rasterization is a challenging task, so the usual “go-to “ solution is to render transparent objects from back to front.

But this solution is not perfect, and in many cases, it can result in artifacts. The most obvious case is when we have two or more intersecting objects — none of them is really in front or behind the other, much like in the first picture.

To solve this, GeoToolkit.js now uses Depth-peeling rendering. It provides what is called “Exact Order Independent Transparency,” meaning transparent objects are rendered correctly, without even needing to sort them. The result can be seen in the second picture.

The principle is quite simple to understand: To render each transparent object layer on top of each other, we have to render them one by one, separately and combine the results in the right order.

unnamed (3)

unnamed (4)

 

The final result of a single depth-peeled frame requires multiple rendering passes.

Each pass carefully keeps its color result, but also its depth buffer result, which is a map of the depth of each pixel of the 3D scene.

Now, with that in mind, here’s the process:

First, we render all the opaque objects, since they are opaque, everything behind them is not visible, so we only need to draw them once. This is a big time saver. Then, we render every transparent object, but only keep one fragment per pixel, which is a single object color per pixel. This means if in the same pixel, two or more transparent objects overlap, we only keep the front-most one. Once done, we keep the image result for later, and also update the depth buffer based on the fragment we kept. Then we repeat the last step, we draw every transparent object again, but this time we keep the pixel color only if they are deeper than the previous pass pixel. This depth test ensures that we render a different layer each time. We repeat this step as much as we need since each pass provides one additional transparency layer. And finally, all passes are composed, from back to front, in a final image.

In our case, depth peeling uses five transparent passes by default. It is of course customizable, but this configuration provides a good compromise between transparency depth and performance.

 

unnamed

 

Improvement: Improved Reservoir Grid Performance

In 3.3 we improved the performance so that reservoirs now render between 2x times and 4x times faster than before. Reservoirs are infamous for being slow to render, mainly because of their potentially large number of cells.

Starting with a couple of thousands of cells, most current GPUs hit what we call a vertex bottleneck, meaning there is simply way too much geometry to render, even when using advanced techniques like instanced rendering.

So the only solution to improve performance here is to draw fewer cells. In 3.3, we improve the performance by optimizing the reservoir geometry and the updates to the GPU memory.

First, we do not draw filtered cells that have been hidden by the user through value or position filters. Then, we do not draw cells that are hidden by their surrounding cells; we call them occluded cells.

So on an average reservoir, only 30 to 40% of the cells are actually visible. This means we have to render 60 to 70% less geometry, thus removing a huge load on the main bottleneck in the rendering pipeline.

 Of course, this improvement sounds simple enough, but in practice, there are important points that need to be carefully addressed. First, we must accurately identify the occluded cells, making sure they are hidden by all six of their neighbors. And we need to be able to update the geometry whenever a filter is updated, fast enough so that the user does not notice it.

Improvement: Lines Rendering

Before, lines were showing a lot of step artifacts, and it was difficult to have consistent and good looking lines. The issue is related to aliasing, but the principle is that it is hard to properly draw a line, which is a vector, onto a screen, which is a raster device. It is hard to draw oblique lines with square pixels, as you can see in the picture on the left. To solve this, we have implemented a powerful anti-aliasing technique into the line rendering.

As you can see in the pictures below, the step-looking artifacts are entirely gone in the new version on the right, and the linewidth is much more consistent across all view angles. So this improvement will be mainly visible on the plot grids.

unnamed (5)

unnamed (6)

Improvement: Anti-Aliasing Optimizations

Finally, we have the optimizations to the full scene Anti-Aliasing. GeoToolkit 3.3 introduces new anti-aliasing techniques and strategies, to improve image quality with a limited impact on performance.

Aliasing artifacts are those steps you can see in the left-most picture — instead of a straight line, we see jagged edges. This is because we are trying to represent a line with pixels, which are effectively squares, thus an oblique line will produce steps if nothing is done.

A more mathematical approach to the issue is that we are sampling and displaying a line, which is a function, with a raster monitor, which has pixels. A function can produce an infinite number of positions, but a pixel based monitor can only display a set number of pixels.

And the step issue gets worse as you reduce this number of samples and pixels. Different anti-aliasing techniques provide different results at different costs. 

The simplest ones try to detect and blur the edges of polygons, in a post-processing treatment, while the more advanced ones generally increase the sampling rate, by taking multiple samples per screen pixel, resulting in increased quality, but lower performance.

In 3.3, we introduce two new anti-aliasing techniques that try to cover both ends of the spectrum. 

For low-pixel-density monitors, high-quality anti-aliasing is used, called Super-Resolution Anti-Aliasing. This AA increases the number of samples per pixel, so each pixel is the product of several samples, increasing the image quality while also greatly reducing aliasing. This compensates for low-resolution monitors that naturally produce more visible artifacts because of their reduced pixel density.

For high-pixel-density monitors, however, more performance-friendly anti-aliasing will be used, called FX Anti Aliasing (FXAA). This AA reduces artifacts by detecting edges in the image and thus is much more performance-friendly. This compensates for large resolution monitors, which already have good image quality, but are also much more demanding in terms of performance.

Overall, this approach allows reduced aliasing in the scene, while keeping consistent performance and quality across different monitors. Or to put it more simply, higher quality for small monitors, and better performance for large monitors.

To learn more about GeoToolkit.JS, please visit int.com/products/geotoolkit/or contact us at support@int.com.

How Apache SIS Simplifies the Hidden Complexity of Coordinate Systems in IVAAP

See how Apache SIS, with IVAAP, helps support our client’s coordinate systems by using less code.

With the recent release of IVAAP 2.9, now is a good time to reflect on the help we got along the way. One of the components that made IVAAP possible is the Apache SIS library. The goal of this blog article is to bring more visibility to this awesome Java library.

What Is the Apache SIS Library?

Apache SIS is an open-source library written in Java that makes it easier to develop geospatial applications. Many datasets have metadata designating their location on Earth, and these locations are relative to a datum and a map projection method. There are many datums and many map projection methods. Apache SIS facilitates their identification and the accurate conversion of coordinates between them.

What’s a Datum and What’s a Map Projection Method?

Most people are familiar with latitude and longitude coordinates. This geographic coordinate system has been used for maritime and land-based navigation for centuries. Since the late 1800s, the line defining 0º of longitude has been the Prime meridian, crossing the location of the Royal Observatory in Greenwich, England. This meridian defined one axis, from South to North. The equator defined the other axis, from West to East. The origin point of this system on the Earth’s surface is in the Gulf of Guinea, 600 km off the coast of West Africa.

The traditional geographic coordinate system
The traditional geographic coordinate system (Source)

 

Similarly, a datum defines the origin point of the coordinate axes on the Earth’s surface and defines the direction of the axes. To account for the fact that the Earth is not a perfect sphere, a datum also describes the generalized shape of the Earth. For example, WGS 84 (World Geodetic System 1984) is a widely-used global datum based on latitudes and longitudes where the Earth is modeled as an oblate spheroid, centered around its center of mass.

The WGS 84 reference frame. The oblateness of the ellipsoid is exaggerated in this image. (Source)

 

WGS 84 is used by GPS receivers and the longitude 0º of this datum is actually 335 ft east of the Greenwich meridian.

While universal latitude and longitude coordinates are convenient, they are not universally practical because of land masses drift. Satellite measurements show that the location of Houston relative to the WGS 84 datum changes by 1 inch each year. A local datum is a more pragmatic choice than a global datum because distances from a local point of reference are smaller and don’t change over the years when all locations are on the same tectonic plate. A local datum may also align its spheroid to closely fit the Earth’s surface in this particular area.

A map projection method indicates how the Earth’s surface is flattened into a plane in order to make a 2D map. The most widely known projection method was presented by Gerardus Mercator in 1569. This is a global cylindrical projection method. It preserves local directions and shapes but distorts sizes away from the equator.

Cylindrical Projection
An example of global cylindrical projection (Source)

 

In the US, the Lambert Conformal Conic projection has become a standard projection for mapping large areas. This is a projection that requires multiple parameters, defining the longitude and latitude of its center, a distance offset to this center, and the latitude of its southern and northern parallels.

Conical Projection
An example of local conical projection (Source)

 

When a datum and a projection are used together, they define a projected coordinate reference system. While local systems limit distortions, they are only valid in a small area, an area known as the ”area of use” where a minimum level of accuracy is guaranteed.

Select Coordinate System
A screenshot from INTViewer showing the area of use of NAD27 / Wyoming East Central, a derived projected coordinate reference system

 

How Does Apache SIS Help IVAAP?

To show geoscience datasets on one of IVAAP’s 2D map widgets, you need to use a common visualization coordinate reference system.

IVAAP Screenshot
An IVAAP screenshot showing the location of wells on a map

 

This is where Apache SIS helps us: It understands the properties of both the data and visualization systems and is able to convert coordinates between them.

The math to perform these conversions is complex, this is not something you want to implement on your own. It requires specialized skills, both as a programmer and a domain expert. And just beyond the math, the number of datums and projection methods is mind-boggling. Many historical surveys are still in use today. For example, there are two datums used for making horizontal measurements in North America: the North American Datum of 1927 (NAD 27) and the North American Datum of 1983 (NAD 83). The two datums are based on two different ellipsoid models. As a result, the two datums have grid shifts of up to 100 meters, depending on location. IVAAP is able to visualize datasets that used NAD 27 as a reference, and it is Apache SIS that makes it possible to accurately reproject these coordinates into modern coordinate systems, accounting for their respective datum shift.

The datum shift between NAD 27 and NAD 83 (Source)

 

The oil and gas industry is at the origin of some of these local coordinate systems. Many of today’s new oil fields are in remote areas, initially lacking a geographical survey. There is an organization called the “OGP Surveying and Positioning Committee” which keeps track of these coordinate systems. It is colloquially known as “EPSG” for historical reasons. It regularly provides a database of these coordinate systems to all its members. This database is used by IVAAP and Apache SIS provides a simple API to take advantage of it. Each record in this database has a numerical WKID (Well Known ID). To instantiate a projection method or a coordinate system defined in this database, you just need to prefix this id with the “EPSG:” string.

OperationMethod method = getCoordinateOperationFactory().getOperationMethod("EPSG:9807"); // Transverse Mercator method

CoordinateReferenceSystem crs = CRS.forCode("EPSG:32056”);

 

The EPSG database itself is extensive, but it is common for INT customers to use unlisted coordinate reference systems, created for brand new oil fields. In these cases, a WKT (Well Known Text) string can be used instead. This text is a human-readable description of a projection method or coordinate system. The Apache SIS provides a clean API to parse WKTs. It also provides an API for formula-based projection methods that can’t be described by a WKT.

PROJCS["NAD27 / Wyoming East Central",
    GEOGCS["NAD27",
        DATUM["North_American_Datum_1927",
            SPHEROID["Clarke 1866",6378206.4,294.9786982139006,
                AUTHORITY["EPSG","7008"]],
            AUTHORITY["EPSG","6267"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4267"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",40.66666666666666],
    PARAMETER["central_meridian",-107.3333333333333],
    PARAMETER["scale_factor",0.999941177],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",0],
    UNIT["US survey foot",0.3048006096012192,
        AUTHORITY["EPSG","9003"]],
    AXIS["X",EAST],
    AXIS["Y",NORTH],
    AUTHORITY["EPSG","32056"]]

The WKT of NAD27 / Wyoming East Central, with the WKID 32056

Why Did INT Choose Apache SIS Over Other Options?

INT had previous experience using GeoTools. Similarly to Apache SIS, GeoTools is a Java library dedicated to facilitating the implementation of geographical information systems. Being an older library, it goes much further than Apache SIS. For example, one of its components allows the parsing of shape files, something currently outside of the scope of Apache’s library. As a matter of fact, the first versions of IVAAP were using GeoTools for coordinate conversions.

One of the issues we encountered with GeoTools is that it is a library that provides only fine-grained Java conversion APIs. There are several paths to convert coordinates between two systems, and GeoTools allows the developer to choose the best method. Choosing the “best” method without human interaction is complex; it depends on the extent of the data being manipulated and the “area of use” of each coordinate reference system involved. It also depends on the availability of well-known transformation algorithms between datums. In North America, the standard for transformations between datums was formerly known as NADCON. The rest of the world uses a standard known as NTV2. Apache SIS works with both datum shift standards. It may elect to use WGS 84 as a hub when no datum shift is applicable. An algorithm to pick the best method would require a significant amount of code for INT to write and maintain. While Apache SIS allows fine-grained control over the different transformations used when converting from one coordinate reference system into another, it also provides a high-level API to perform this conversion. The picking of the best algorithm is part of the Apache SIS’ implementation. Its high-level Java API that picks a conversion algorithm matches IVAAP’s general use microservice for the same function. To pick the right algorithm, it only takes 3 parameters:

  • A definition of the “from” coordinate system
  • A definition of the “to” coordinate system
  • A description of the “extent” of the coordinates to convert
double x = …
double y = …
GeographicBoundingBox extentInLongLat = …
DirectPosition position = new DirectPosition2D(x, y);
CoordinateReferenceSystem fromCrs = CRS.forCode("EPSG:32056");
CoordinateReferenceSystem toCrs = CRS.forCode("EPSG:3737");
CoordinateReferenceSystem displayOrientedFromCrs = AbstractCRS.castOrCopy(fromCrs).forConvention(AxesConvention.DISPLAY_ORIENTED);
CoordinateReferenceSystem displayOrientedToCrs = AbstractCRS.castOrCopy(toCrs).forConvention(AxesConvention.DISPLAY_ORIENTED);
CoordinateOperation operation = CRS.findOperation(displayOrientedFromCrs, displayOrientedToCrs, extentInLongLat);
MathTransform mathTransform = operation.getMathTransform();
double[] coordinate = mathTransform.transform(position, position).getCoordinate();

Sample code to convert a single x, y position from “NAD27 / Wyoming East Central” to “NAD83 / Wyoming East Central”

We still use GeoTools for other parts, but as a general rule, the Apache SIS Java API tends to be simpler, more modern than GeoTools when it comes to manipulating coordinates and coordinate systems.

After 3 years of use, we are happy with our decision to move to Apache SIS. This library allows us to support more of our customers’ coordinate systems, with less code. We are also planning to use it to interpret the metadata of GeoTIFF files. The support has been excellent. When we needed help, the members of the Apache SIS development team were really keen to help us. This is one of the reasons why INT felt we needed to give back to the open-source community. Being a long-time member of OSDU, INT contributed to OSDU a coordinate conversion library built on top of Apache SIS. This coordinate conversion library converts GeoJSON and trajectory stations between different coordinate reference systems. Users can specify the specific transformation steps that will be used in the conversion process, either through EPSGs or WKTs. Behind the scenes, it’s the Apache SIS’ fine-grained API that is being used.

Improving Seamless Visualization and Embedding Dynamic Visualization with IVAAP 2.9

Companies in the energy industry face several challenges when it comes to streamlining large datasets, finding compatibility among applications, and visualizing different types of data in real-time. The latest release of INT’s IVAAP Data Visualization Platform focuses on solving these challenges by making it more efficient to find and manage subsurface data.

Seamless visualization of domain data goes beyond just accessing the data in the cloud — to maximize your workflow, you must be able to search through data quickly, execute processes and workflows, and visualize the results in one platform. IVAAP’s custom SDK embeds IVAAP’s widgets in users’ applications to add the flexibility to customize the interface, connectors, and a dynamic UX/UI. 

New & Improved Widgets: Diagram, Image, Well Log, and 3D

A new widget has been added to IVAAP’s list of widgets — the diagram widget.  The diagram widget displays SVG data from IVAAP combined with numerical curve values in real-time, supporting animations of your data. The image widget has been improved to support very large TIFFs (Tiled Image Support). Users can visualize their large raster image files by saving them in the cloud and IVAAP creates a single point of access to these files, making compressed rendering images from their original format. 

We also added improvements to other existing widgets. IVAAP’s existing WellLog widget offers Single Data, Multi Data, and Multi Parent modes to display very complex well logs and switch between wells smoothly. The 3D visualization widget’s high performance reservoir visualization is now combined with curser tracking between 3D Basemap and 2D seismic lines and overlay. The basemap feature also supports tops, faults, point sets, and more. 

Empowering Subsurface Workflows

IVAAP has offered the ability to connect to any third-party application for a while, allowing users to integrate in-house workflows and customize visualizations to create a seamless experience. For many companies with data located in more than one repository or cloud database, accessing all of this data through one unique portal streamlines the process and standardizes data access. IVAAP can communicate with external engines, such as ML platforms, processing engines  — Tensorflow and SageMaker— and code can be written in Python and executed through the IVAAP client interface.  

With these new and improved features, IVAAP 2.9 enables easy search and visualization of energy, geophysical, and production data in the cloud. It allows product owners, developers, and architects to build subsurface digital solutions faster without having to start from scratch. 

For more information, please visit www.int.com or contact us at intinfo@int.com.

Visit us online at int.com/ivaap for a preview of IVAAP or for a demo of INT’s other data visualization products. 

Raster Log and TIFF Rendering in the Cloud with GeoToolkit.JS

A common challenge that our users face is a need to render and display huge images like raster logs and GeoTIFF files. Interactions occur in the client’s server architecture, and while these images can be stored in the cloud or on a server, they are usually very large files, making it difficult to move from the server to the client. It is also time-consuming to read data from the cloud and unpack operations for TIFFs.

When trying to open these files, the user may have to wait up to several minutes to start to receive data from the server. This can lead to a poor user experience because the client doesn’t know if a server is working or if it’s already halted the solution that we applied. So we wanted to improve this for our users.

 

Reducing Calculation Complexity

In order to optimize several areas for the client as well as on the server, we decided to focus on the server data loading from cloud storage and enhance the unpacking of TIFFs. We found a way to minimize the number of requests from client to server and pass more calculations from server to client, reducing the calculation complexity on the server.

So, how did we do it? On the client side, we have a class called tiled shape which requests tiles from the server.

raster 1

For each tile, the client requests its X and Y position, height, and scaling. In the past, when a user would interact with an image and send a request to the server, requesting new tiles, it would be too much for the server to handle, causing even more delays.

To minimize the impact this request has on the server, we decided to check if the  user stops interaction by timer. After any interaction on the client’s side with the image, we start a timer. Before the timer is up, if no requests are sent from the client to the server, this timer finishes quickly and the image loads. This allows us to minimize the number of requests from client to server.

Calculation and Caching

Another feature we optimized is the ability to pass information from the server to plan calculations for the correct tile position and correct tile transformation. By passing the data from server to client, the client calculates the correct position for tile on each side. We also use a cache of tiles on the client side to ensure that we do not request the same tiles. Image cache on a server side is also possible, but after the first request, it is necessary to download the requested image from the cloud, unpack the image, and then start to copy and scale the requested tile to convert this image.

 

Unpacking the Image

On the server side, we implemented TIFF unpacking on a highly performing native implementation. This is a multithreaded and thread safe solution., which allows us to deal with several requests from clients at the same time. As a result, a large TIFF image file can be stored in the cloud without a problem and an unpacking time of less than four seconds compared to the previous solution’s 60 seconds. We also optimized the cloud access speed on the server side by renting a high-performance AWS instance with high-speed access to AWS cloud storage. This allows us to download the whole TIFF image from the cloud in less than three seconds, making the download speeds about 800 megabytes per second.

 

raster2

Faster Rasters

As a result, we spend around three seconds to download the image from cloud storage and about four seconds to unpack the image. So it only takes roughly seven seconds to download, unpack the image, deal with client requests, scale the requested tiles, and send them back to the client as opposed to having to wait several minutes to possibly hours for the same process.  Overall, this process greatly improves the user experience.

The new raster logs and cloud TIFF rendering are just one of the many new features and improvements from the latest GeoToolkit.JS 2021.1 release.

To learn more about GeoToolkit.JS 2021, please visit int.com/products/geotoolkit/or contact us at support@int.com.