INT

Empowering Visualization

CONTACT US SUPPORT

cloud

What Is Kubernetes?: An Introduction and Overview of the Popular Platform

Over the last several years, Kubernetes has been a developing trend in the tech world, gaining popularity as more and more companies begin to take advantage of cloud-based services. As a non-developer living in a developer world, I wanted to understand more about why we develop our software to work with Kubernetes. So I sat down with INT’s Senior Product Manager Steven Reynolds and Senior Architect James Velasco to learn more about Kubernetes and the advantages of working with it.

Christin: So, Steven, tell me — what in the world is Kubernetes??

Steven: Kubernetes is described as an orchestration platform. It’s usually associated with getting big software up and running. For example, with some of our clients, we have our build set up so that all of the software assets are pushed out and deployed using Kubernetes which takes care of monitoring (pods). If a pod fails, Kubernetes will restart it, and it has nice hooks that can spin up new pods if there’s too much work for one pod to handle.

Christin: What are the advantages of using Kubernetes?

James: Traditionally, you have a service provided by a server. If you migrated, changed, or updated the service, there would be a lot of work to change or configure, set up the new server providing it, and with cloud and cloud providers like AWS, they are moving to automating things. So instead of manually going in and doing all of it, you have tools to automate updating an old version, and switching over to the new one automatically.

Christin: The term microservices is used with Kubernetes — can you go into what microservices does?

James: Microservices is an architectural trend happening right now. Instead of having one big server running everything for a web service, you may have that same web service decomposed into little servers that do separate, well defined parts.

Steven: So the idea of Kubernetes and microservice are different concepts, but they’re talked about together. When using a microservice architecture, it’s an advantage, but it’s also a headache to keep track of all the components.

Christin: So when is it best to use Kubernetes?

Steven: We have 15 to 20 services, and it’s a headache to manage, but you can use Kubernetes to help organize it and help it be more manageable for human beings. There are trade-offs because it makes it better to manage, but there are many tradeoffs as in anything else. It’s the engineer’s dilemma.

James: It tends to only be an advantage if you’re managing a lot of services. So if you’re just managing one or two, then it’s iffy. But if you’re managing 10+ or even just 4 or 5 and those 4 or 5 are made up of 10 or 15 subparts, that is when it makes sense. If I divided all my services like that I may have 10 or 20 and then you may have other vendors or services and you can literally have 100 to 200 of these things. You have to swap out in a unique order and all need to come up and down and all need to be monitored, so that’s what Kubernetes helps you do. Kubernetes is also great to scale up or down.

Christin: That’s really cool. So, Steven mentioned that we have built projects to run on Kubernetes. Does that mean IVAAP can be supported on Kubernetes if requested?

James: Kubernetes is one of the environments that IVAAP targets, if a client requests it. It’s important that we do support it because it is one of the modern deployment styles. And it is common that clients use IVAAP running on Kubernetes. Some clients use Azure or AWS so we have to make sure we can run in all different environments.

******

So there you have it. As a growing trend in software development, Kubernetes can be a better way for companies to deploy and maintain software quickly, especially in a cloud environment. For INT, supporting Kubernetes is an important aspect of how we help meet the needs of our many diverse clients. For more information on IVAAP, check out int.com/ivaap or schedule a demo with one of our team.


What Cloud Data Lakes Mean for Geoscience

With the explosion of storage capacity, cloud computing, and bandwidth availability, a trend has emerged in the oil and gas industry over the last few years. Data that was previously aggregated and discarded is now maintained and stored, creating an opportunity for the industry. Coupled with new advancements in machine learning and Al, this data availability is poised to drive more data driven decision making in well planning, drilling & completions, and well operations.

After our conversations with major operators over the last few months, we realized that the concept of a data lake is still pretty new and can be perceived differently. We thought this would be a great opportunity to explain what this technology approach is and its benefits and how major operators can leverage INT’s enterprise data visualization platform to help garner insights from geoscience data in the cloud.

What is a Data Lake?

The idea behind a data lake is that as businesses gather more data, this data cannot be used the same way it has been used in the past. Databases are not a good fit for big data, not just because of the size of that data but also because this data cannot be normalized. A Data Lake is a large storage of raw data where each data point is stored in its native format, along with relevant metadata.

This approach solves a problem that the oil and gas industry has faced for a long time. Despite industry efforts to standardize data formats, these formats are loosely followed and too limited to contain all information related to a particular seismic survey or an oil field. Keeping the data as is makes your job of extracting valuable information difficult. And if you opt to normalize that data, you inevitably lose information. The concept of Data Lake opens the possibility of both keeping your original data while allowing its exploitation.

How do you exploit data from a Data Lake?

When you add files to your Lake, you carry along these files’ relevant metadata. When the right set of metadata is provided, you can search your data from this set. For example, if each document is geo-referenced, you can search for all documents relative to a geographical area.

How is the concept of Data Lake different from a search engine?

Search engines can’t really exploit files in SEG-Y or LAS formats. And even for well-known formats such as PDF, search engines have no awareness of which attributes of a PDF file are important to you. For example, if your metadata indicates that a file documents the characteristics of a well at a particular location, a Data Lake will allow you to find this file just by selecting the right region of interest.

How are cloud providers helping with Data Lakes?

Microsoft Azure, Amazon Web Services, and Google Cloud have developed a wide range of products and components to facilitate the creation and use of data lakes, such as nearly infinite storage, artificial intelligence, and machine learning to provide a seamless way to ingest and consume your data. The technology behind such advanced indexing and analytics cannot be reproduced in-house—you need a world-class partner.

How is INT helping with Data Lakes?

INT helps in two ways: We have unique experience in the industry. Working with so many actors, we have acquired the knowledge required to read multiple data formats, even when these formats are not strictly followed. Our tools facilitate the extraction of the metadata required for the Lake to function as intended, and not as a “swamp.”

And, of course, our visualization technology is what makes it all possible, all from the comfort of your browser. Our IVAAP Enterprise Cloud Viewer allows you to visualize the datasets and documents stored remotely in your Data Lake. You’ll see how you can start from a map and drill down all the way down to the log curves of a well found on that map. In the same screen, you’ll be able to review PDF reports for that well and navigate through the slices of the matching seismic survey as if it was stored locally.

For more information about our enterprise data visualization solutions, visit the IVAAP product page, or contact us.


How to Improve Performance and Reduce Latency of Your Geoscience Data

Storing and accessing large, sometimes sensitive geoscience data is one challenge many top E&P companies face.

Local access is great, but not every user in the world can have local access to the same data. Replication is an option, but with the size of seismic datasets reaching terabytes, this is not practical. In the real world, users only have access to a limited set of local data.

Common Solution Leads to Performance Issues

Many companies store data all over the world. The common infrastructure to allow ubiquitous access to data is to share these files using NFS, a well-known distributed file system protocol used by Linux-based servers.

The issue with NFS is that it is a “chatty” protocol: Many messages are sent back and forth between the client and the server. This is fine when all machines are physically close to each other, but the further away they get, the more latency you introduce. As a result, performance degrades.

NFS is also essentially transparent to the software using it. Some software, like INTViewer, doesn’t “know” that your data is remote, so it can’t optimize its data fetching strategy to the characteristics of your infrastructure. Actually, for seismic data, it assumes that access to individual traces is fast.

A Better Option

This is where INTGeoServer comes into play. Access to data hosted on INTGeoServer—a server with a modern architecture that uses web services to stream geoscience data—is optimized so that there is a limited number of back-and-forth messages. In other words, by installing INTGeoServer next to your data, you make this data accessible from remote places as if it were local.

INTGeoServerNFSGraphic2

To visualize any geoscience file in INTViewer, simply drag and drop that file from the file system to INTViewer’s main window and its content appears automatically. From this experience, it might seem that INTViewer is tied to the file system where it resides, meaning it can only read data from that file system. While this is a common use case, using INTGeoServer removes the requirement to have INTViewer and your data on the same file system.

From an INTViewer user point-of-view, the protocol used to access the data doesn’t change the interaction—the visualizations are the same, the analysis tools work the same way. From a system administrator point of view, however, the burden of maintaining worldwide NFS shares is lifted. And the benefit of accessing that data in larger chunks is that the performance profile improves substantially.

For more information about INTViewer and INTGeoServer, visit the INTViewer product page, or contact us for a free trial.


Accessing INTViewer Remotely

In our blog post on Microsoft Azure, we describe various ways customers can move their data to the cloud. In the configuration where INTViewer is hosted on a remote server and needs to be accessed from a local workstation, a Teradici client is one solution.

These configurations are increasingly popular with our customers. For performance reasons, it makes sense to host INTViewer next to your data. But this data tends to be large and hosted in remote data centers.

Teradici is not the only software we have tested with INTViewer. Other softwares allow such remote access. The minimum sophistication of remote access software depends on what you plan to do with INTViewer. If you only plan to visualize data in 2D, most software will work off the shelf. On Linux, a well-known solution built into the operating system is to use X11 forwarding. On Windows, there are various free software solutions widely available such Microsoft Remote Desktop Connection (bundled with Windows) and VNC.

If you need to display your data in 3D or if you need cross-plotting, these widely available solutions won’t work. In many cases, users will encounter an annoying “Can’t display 3D window” or “Can’t display cross-plot” message. INTViewer uses OpenGL to render cross-plot and 3D visualization, and this technology imposes specific requirements. INTViewer requires OpenGL v3 or greater, and most classic solutions only support OpenGL v2.

As a result, in addition to Teradici, INTViewer has been tested with commercial software such as HP RGS. INTViewer has also been tested with VirtualGL. VirtualGL is open source so there is no cost to download and set up the product. Another product some clients have used is called ThinLinc by Cendio. ThinLinc is not an open-source product, but they offer a limited trial version.

If you need assistance setting up your remote desktop environment, contact us at support@int.com for help.


Microsoft Azure + INTViewer

Yesterday, I was invited to the Microsoft offices in Houston to install a demo of INTViewer. Microsoft advocates for solutions that are hosted on the cloud, and I found that INTViewer fits this model well.

The cloud is not just a set of computing resources hosting web based-applications. Microsoft Azure can also be used to host what we know today as desktop PCs. Constant availability and unlimited storage are just two of the numerous advantages of hosting your PC on the cloud.

To access our “PC in the cloud,” you only need a lightweight appliance hooked to a keyboard, a mouse and a monitor. An example of such appliance is the “Teradici Zero Client.” What’s different between a classic “Remote Desktop Solution” and Teradici’s solution is how rendering is implemented. Network latency can get in the way of the user experience if not done right. Teradici’s network protocol makes remote interaction seamless. This is important for INTViewer: At its core, INTViewer is a visualization application. Users typically have several large monitors to work with their data, which means a lot of pixels to transport.

A key to application performance is to keep your data and your visualization software close to each other. Once you put your geoscience data in Microsoft Azure, INTViewer allows instant visualization of that data. The reason why INTViewer is a good fit for that setup is that it’s a relatively lightweight application. Unlike large interpretation software, you typically only need 1 to 4 GB of memory to run. And it’s a snap to install.

It’s quite a leap to decide to move terabytes of data to Microsoft Azure, and INTViewer is only one factor to that decision. However, there is one immediate use case for Microsoft Azure and INTViewer: INTViewer is a great tool for presentations. The slideshow plugin allows businesses to prepare presentations, each slide showing a particular aspect of a survey. I mocked up in a few minutes a presentation of the well-known Alwyn dataset. This presentation included visualization of a seismic survey and how it relates with wells and the reservoir model. For good measure, I added a spectrum analysis and a numerical view of log curves. The entire data underlying that presentation was only 10GB, easy to copy to Microsoft’s cloud at a moment’s notice.

This opens a new way to use INTViewer. Some of our customers use laptops to showcase acquisition data to potential clients. With a cloud-based solution, the burden of copying the data to each laptop disappears — you only need to copy it once to the cloud.

I was particularly proud of how well the slideshow plugin performed. Navigating through Alwyn became as simple as flipping through Powerpoint slides. But unlike a static presentation, attendees can interact with the data on demand. If a problematic data point is identified, it is easy to zoom, perform a live QA of that data, answer questions, and then move on to the next topic.

Check back soon for more new features and tips on how to use INTViewer or contact us for a demo of INTViewer in the cloud.