ivaap

Nov 06 2020

How to Get the Best Performance out of Your Seismic Web Applications

One of the most challenging data management problems faced in the industry is with seismic files. Some oil and gas companies estimate that they acquire a petabyte of data per day or more. Domain knowledge and specific approaches are required to move, access, and visualize that data.

In this blog post, we will dive deep into the details of modern technology that can be useful to achieve speed up. We will also cover: common challenges around seismic visualization, how INT helps solve these challenges with advanced compression and decompression techniques, how INT uses vectorization to speed up compression, and more.

What Is IVAAP?

IVAAP is a data visualization platform that accelerates the delivery of cloud-enabled geoscience, drilling, and production solutions.

IVAAP Client offers flexible dashboards, 2D & 3D widgets, sessions, and templates
IVAAP Server side connects to multiple data sources, integrates with your workflows, and offers real-time services
IVAAP Admin client manages user access and projects

Server – Client Interaction

Interaction occurs when the client requests a file to display from the server, the server returns the file lists, the user chooses a file to display, and then the server starts sending chunks of data while it displays this data.

Some issues encountered with this scheme include:

Seismic data files are huge in size — they can be hundreds of gigabytes or even terabytes.
Because of the file size, it takes too much time to transfer files via network.
The network can have too much bandwidth.

The goals of this scheme are to:

Speed up file transfer time
Reduce data size for transfer
Add user controls for different network bandwidth

And the solution:

We decided to implement server-side compression and client-side decompression. We also decided to provide the client parameter that we call acceptable error level after the seismic data file compression/decompression process.

By taking a closer look at compression and decompression data, we can see that the original seismic data goes through a set of five transformations — AGC, Normalization, Hear Wavelets, Quantization, and Huffman. As a result of this transformation, we get a compressed file that can be sent to clients via network. And on the client’s side, there is a decompression process that goes in different directions — from inverse Huffman to inverse AGC. This is the way that clients get original data. It does not get precise, original data. But it gets data after the compression and decompression process. That’s why we added an acceptable error level after the compression and decompression process. This is because we have different scenarios where clients don’t always require the full original data with the full level of precision. For example, sometimes the client only needs to review the seismic data. So using this acceptable error level, they can control how much data will be passed by a network and, of course, speed up this process.

The resulting scheme looks like this:

The client requests a file list from the server, the user chooses a file to display, and then the server starts sending the data and compresses it. The server then sends it to the client, the client decompresses, and finally, it displays the data. This is repeated for each tile to display.

So why not use any other existing compression, like GZIP, LZ Deflate, etc.? We tried these compressions, but we found out that this type of compression is not as effective as we’d like it to be on our seismic data.

Server-Side Interaction

The primary objective was to speed up the current implementation of compression and decompression on both the server and client side.

The proposal:

Server-side compression is implemented in Java, so we decided to create C++ implementation of compression sequence and use JNI layer to call native methods. For the client-side decompression, we implemented in JavaScript to create C++ implementation of decompression and use WebAssembly (WASM) proposal for integrating C++ code into JS.
We implemented both compression and decompression algorithms in C++, but after comparing the results and performance of C++ and Java, we discovered that C++ was just 1.5 times faster than “warmed up JVM”. That’s why we decided to move on and apply SIMD instructions for further speedup.

Single Instruction Multiple Data (SIMD)

SIMD architecture performs the same operation on multiple data elements in parallel. For Scalar operation, you have to perform four separate calculations to get the right result. For SIMD operations, you apply one vector value calculation to get the correct result.

SIMD benefits:

Allows processing of several data values with one single instruction.
Much faster computation on predefined computation patterns.

SIMD drawbacks:

SIMD operations cannot be used to process multiple data in different ways.
SIMD operations can only be applied to predefined processing patterns with independent data handling.

Normalization: C++ scalar implementation

Normalization: C++ SIMD SSE implementation

Server-Side Speedup Results

There are different types of speedup for different algorithms:

Normalization is 9 times faster than the scalar C++ version
Haar Wavelets is 6 times faster than the scalar C++ version
Huffman has no performance increase (not vectorizable algorithm)

Overall, the server-side compression performance improvement is around 3 times faster than the Java version. This is applying SIMD C++ code. This was good for us, so we decided to move on to the client-side speedup.

Client-Side Speedup

For the client-side speedup, we implemented decompression algorithms in C++ and used WASM to integrate the C++ code in JavaScript.

WebAssembly

WASM is:

A binary executable format that can run in browsers
A low-level virtual machine
A high-level language compile result

WASM is not:

A programming language
Connected to the web and cannot be run outside the web

Steps to get WASM working:

Compile C/C++ code with Emscripten to obtain a WASM binary
Bind WASM binary to the page using a JavaScript “glue code”
Run app and let the browser instantiate the WASM module, the memory, and the table of references. Once that is done, the WebApp is fully operative.

C++ Code to Integrate (TaperFilter.h/cpp)

Emscripten Bindings

WebAssembly Integration Example

Client-Side Speedup Takeaways:

Emscripten supports the WebAssembly SIMD proposal
Vectorized code will be executed by browsers
The results of vectorization for decompression algorithm are:
- Inv Normalization: 6 times speedup
- Inv Haar Wavelets: 10 times speedup
- Inv Huffman: no performance improvement (not vectorizable)

Overall, the client-side decompression performance improvement with vectorized C++ code was around 6 times faster than the JavaScript version.

For more information on GeoToolkit, please visit int.com/geotoolkit/ or check out our webinar, “How to Get the Best Performance of Your Seismic Web Applications.”

Oct 07 2020

Search, Access, and Visualize OSDU Data with IVAAP

INT and OSDU Collaboration

INT has been working with the Open Subsurface Data Universe™ (OSDU) consortium since the beginning of the OSDU forum in 2019. OSDU is a standard data platform for the oil and gas industry which will reduce silos and put data at the center of the subsurface community. INT made IVAAP available as part of the Demo release of the OSDU platform.

IVAAP: The Premier HTML5 Data Visualization Platform for OSDU

At INT, we worked closely with the OpenSDU team to assist with validating the platform and creating a robust framework.

Developers can deliver cloud-based digital solutions quicker than ever.
Using the powerful OSDU search and delivery API, IVAAP allows users to search, find, display, and analyze data on the fly, including log, trajectories, tops, seismic, horizon, and fault data.
Creating a common environment hosted in the cloud enables oil and gas companies to efficiently access massive amounts of data, reduce data silos, collaborate remotely, implement machine learning (ML), and lower the cost of operations.

Search and interact with different data types.

Access your data in the cloud from anywhere.

Visualize data aggregated from various data sources in one place.

With IVAAP on OSDU R2, you can:

Combine data on the fly from wells, seismic, and many other sources.
Use a microservices architecture, IVAAP supports industry data standards, including OSDU, PPDM, WITSML, and several commercial databases.
Enable user co-visualization.
Combine data types like real-time, historical, 2D/3D seismic, reservoir, and more in your web browser with IVAAP’s one-of-a-kind dashboard capabilities.

Request a demo for IVAAP today

What Is OSDU?

The OSDU Forum was created with the objective of enabling new cloud-native data-driven applications with seamless access to the full range of subsurface and wells data as well as supporting existing applications and data frameworks.

Separating data from applications is the core principle of the OSDU solution. The plan is to achieve this by developing a common data platform with standard public APIs and to involve global cloud hosting vendors to build working implementations that will:

Allow companies to focus on the truly differentiating part of business activities
Reduce the resource and cost burden of independently designing and validating a subsurface data platform
Influence and accelerate key reference components that will be the foundation of the upstream digital transformation.

The OSDU data platform will:

Enable secure, reliable, global, and performant access to all subsurface and wells data
Reduce current data silos to enable transformational workflows
Accelerate the deployment of emerging digital solutions for better decision-making
Create an open, standards-based ecosystem that drives innovation

View the official OSDU brochure.

An Industry Solution

The OSDU Forum will:

Publish a Reference Architecture: a cloud-native subsurface and wells data platform reference architecture with initial implementations by Microsoft^® Azure, Amazon Web Services^® and Google^® GCP, with others expected.
Define Application Standards to ensure applications (microservices), developed by various parties, can run on the same OSDU data platform
Leverage Industry Data Standards for frictionless integration and data access

Business Impact

Reducing data silos: subsurface and wells data integrated and accessible in one data platform
Workflows in the cloud: user workflows seamlessly executed across the OSDU data platform, taking advantage of powerful cloud-native solutions
Access to all metadata: users will have access to powerful search capabilities
Competition and cooperation: software vendors, oil companies, academia, and open community projects can develop software on the OSDU data platform

Scope

Current Scope

Coverage: any organization with a need to store, manage or analyze subsurface and wells data
Portfolios: subsurface including exploration, development, and wells
Support for Data Driven Applications: All data in one Data Platform will enable AI (such as Machine Learning) based applications to access this data

Current Problem Statement

Data is linked to applications: we are not data-driven but applications-driven and, therefore, we will have silos
Data is stored in silos: therefore, there is no lineage between data sources or from wells back to exploration
Metadata is not stored with the data: metadata is information about the data
Limited/no search capabilities: since we have no metadata, we have no real search capabilities
Data set-up: not suitable for data-driven applications

OSDU Scope

All public cloud-based: provides unlimited scalability
All data for the data platform services loaded via the data ingestion services: extract metadata, data quality, etc.
Well-defined (RestFUL) APIs defining the access to the data platform services
Applications: (micro) services-based
Information security: covering all elements with focus on data security
In-country solution: for those countries where we are unable to move data out of the country and where there are no cloud services
Support for legacy applications based on Microsoft^® Windows or Linux^® desktop

What Is the OSDU Value?

The OSDU data platform will:

Revolutionize the industry’s ability to deliver new subsurface capabilities
Create an open, standards-based ecosystem that drives innovation
Accelerate the deployment of emerging digital solutions for better decision-making
Reduce implementation and lifecycle costs
Eliminate current data silos to enable transformational workflows
Provide access to all metadata: users can access their data and applications wherever they are
Create new opportunities for monetizing intellectual property:
- Moving to subscription models for applications or services sold by independent software and services providers
- Opportunity for multiple competitors to sell specialized algorithms or workflows that are easily integrated because of compliance to OSDU standards

Membership

If you’re interested in becoming a member of OSDU today, click here, or for a full list of members, click here.

Ready to Get Started?

Request a demo of the platform chosen to visualize OSDU data.

Request a demo for IVAAP today

Sep 09 2020

INT Adds Client SDK and Improves Seismic and Subsurface Visualization Performance with Latest Release of IVAAP 2.6

This release confirms IVAAP as a leader in the subsurface data visualization space, supporting Data Visualization and Data Management for Subsurface, Exploration, Drilling, or Production Applications in the cloud.

Houston, TX — INT is pleased to announce the newest release of its HTML5 upstream visualization platform, IVAAP™ 2.6, accelerating the pace of our release cycle to respond faster to the growing needs of our clients and partners.

IVAAP now offers a robust client SDK so users can extend the platform to meet their unique business objectives — now users can write new and extend existing widgets; add or remove existing modules; create new data connectors; extend the data model; and more.

IVAAP 2.6 offers several key enhancements that mean better performance and faster data interaction in the cloud (Azure, AWS, Google): more interpretation capabilities with added fault picking in 2D seismic; improved map-based search with multiple criteria for OSDU R2; UI and navigation improvements; support for real-time wireline logging with decreasing depth; and a new connector to the Energy Information Administration (EIA) database.

“For this latest release, we wanted to focus on features and improvements that would complement our partnership with OSDU and enhance our cloud offerings,” said Hugues Thevoux-Chabuel, Vice President, Cloud Solutions at INT. “We gathered feedback from developers and end users to better understand their needs and worked hard to ensure IVAAP exceeds the current and near-future demands of our clients.”

IVAAP is an upstream visualization platform that enables search and visualization of geophysical, petrophysical, and production data in the cloud. It allows product owners, developers, and architects to rapidly build next-level subsurface digital solutions anywhere without having to start from scratch.

Read the press release on PRWeb >

Check out int.com/ivaap for a preview of IVAAP or for a demo of INT’s other data visualization products.

For more information, please visit www.int.com or contact us at intinfo@int.com.

Aug 18 2020

Weatherford, INT Expand Collaboration to Create 2D, 3D Real-Time Well Visualization

INT’s IVAAP framework provides next-level real-time well visualization in 2D and 3D for Weatherford’s Centro digital well delivery software.

HOUSTON, TX – August 18, 2020 – Weatherford International plc announced a strengthened collaboration with upstream data visualization provider INT to provide next-level, real-time well visualization in both 2D and 3D. Weatherford will embed INT’s IVAAP framework into the Weatherford Centro™ digital well delivery software, advancing its data visualization capabilities.

“Weatherford’s selection of the IVAAP framework for their Centro software is extremely exciting for INT,” said Dr. Olivier Lhemann, President of INT, Inc. “Centro is leading the industry-wide digital transformation as part of a new generation of applications built fully in HTML5. It is both highly collaborative and powerful, but still very user-friendly. INT has worked with Weatherford for many years and we are proud to be part of their innovative approach to integrating data and analytics within a single platform.”

Weatherford Centro digital well delivery software offers exceptional workflows that seamlessly integrate every element of an operator’s well data, allowing team members from any global location to access, share, and store all vital project information at any time. Combining Centro’s advanced data visualization capabilities with the IVAAP framework provides next-level real-time well visualization in 2D and 3D.

“Weatherford provides digital solutions that help our customers reduce costs and increase operational efficiency,” said Gustavo Urdaneta, Drilling Software Global Manager at Weatherford. “The proven E&P data visualization and open architecture design was a critical factor in selecting the IVAAP framework to enhance the value of user centric visualizations in our well delivery software.”

Weatherford has been at the forefront of software innovation, especially in drilling and production. Both companies have built a strong relationship by embedding advanced HTML5 domain visualizations libraries and modules from INT in several Weatherford software applications.

Read the full press release on PRWeb.

Visit int.com/ivaap/ to learn more or contact INT at intinfo@int.com.

About INT

INT is a software provider of Data Visualization solutions and platforms used in Upstream E&P and other technical industries. INT Software uses the latest technologies, such as HTML5 and JavaScript, to enable cloud-enabled and mobile-responsive solutions. For more than 25 years, INT visualization libraries, widgets, and frameworks have been used by the leading Oil & Gas and Service companies to empower best-in-class business applications for seismic, geoscience, well intelligence, drilling ops, utilities, manufacturing, and asset management.

About Weatherford

Weatherford is a leading wellbore and production solutions company. Operating in more than 80 countries, the Company answers the challenges of the energy industry with its global talent network of approximately 19,000 team members and 600 locations, which include service, research and development, training, and manufacturing facilities. Visit weatherford.com for more information or connect on LinkedIn, Facebook, Twitter, Instagram, or YouTube.

Jun 11 2020

Optimizing 3D Subsurface Web Applications with GeoToolkit

The WebGL technology that allows 3D rendering on browsers was released in 2009, and yet, 11 years later, 3D is hardly said to be common for web applications in E&P. Despite the advantages of 3D rendering and the trend of migrating applications to web browsers, there are various obstacles that developers have to face and overcome. The steep learning curve, the insufficient cross-platform support, and the difficulty of maintaining make WebGL a beast to work with. In a way, we want to keep the obstacles away from web developers, so INT provides Carnac3D, a subset of GeoToolkit, as an industry-leading graphical product.

Carnac3D is a comprehensive set of graphical tools for data visualization, and it empowers 3D rendering for IVAAP and many other industrial applications. The toolset utilizes WebGL to achieve GPU rendering in web browsers. Meanwhile, it mitigates the drawbacks of the technology and enriches the functionalities specialized for oil and gas.

Getting Started with 3D Rendering Should Not Be Frustrating

The concept of 3D rendering has a lot in common with photography: We need a camera, a scene, and a light source to get started. They are so essential that we can’t take photos without them. In WebGL, it is the same process, except we cannot take that equipment for granted. We had to create a camera, a scene, and a light source ourselves each time we built an application, by playing with the C-like shading language and the low-level graphical APIs. It was a tedious procedure, so we have proposed a solution in Carnac3D. We want our users to have all tools ready when they start to build an application, without having to build tools first.

Having things encapsulated doesn’t mean the user will lose control of the details, because we also provide an intuitive interface. No matter which framework (Angular, React, or Vue) or language (JavaScript or TypeScript) the users choose, they can modify and control the settings for rendering with minimum efforts.

An active community and strong support are things that developers look for when they evaluate and choose among different technologies. INT honors these and has established a developer site with detailed documentation and tutorials. Whenever our users have questions, they can always get help by simply opening a support ticket or logging on to the INT Developer Community.

Rendering E&P Models Is as Simple as Providing Data

From the render engine’s perspective, it only rasterizes primitives like triangles and does not understand industrial models by default. How can our users render complicated industrial models without having to stack triangles? Our work in Carnac3D gets the job done, so the users can focus on what is more important for them.

3D graphics require a lot of math computations to be done. For example, the render engine behind the hood needs to perform numerous matrix operations for simple actions such as translating, scaling, and rotation. During our process of developing, we created many helper functions that tackle 3D math. Our math module also handles the situation when the users need to trim and fit their 3D data.

Typical E&P features in Carnac3D include, but are not limited to, well trajectories, log curves, array logs, seismics, surfaces, volumetric rendering, and reservoirs. In the user’s application, these features are available and ready to use by providing corresponding datasets. Possible sources for the dataset are local files on disk, data in cloud storage like AWS, and different services like INTGeoServer. We also take into account if users have their own data storage.

ReservoirGrid Is an Example of How We Optimize Things

ReservoirGrid is a feature in Carnac3D that allows users to visualize reservoir characterization. For years, web developers in oil and gas industries have worked on increasing the capacity of reservoir rendering while maintaining a desirable performance. Due to the limitation of web browsers, it’s hard for web applications to catch up with desktop counterparts in terms of performance and scalability. ReservoirGrid is our solution that tackles the dilemma.

To break down into details, we shall see a reservoir consists of numerous hexahedral cells. Each cell stores information such as property value (color), IJK index, and custom data. By bundling the cells together, it can display some internal objects like horizons and skeletons. What makes our ReservoirGrid standout if it sounds concise already? The short answer is optimization.

We rely on all possible hardware accelerations that WebGL is capable of doing. For example, we noticed that the outlines of each cell bring extra computing burden, so we eliminated these extra draw calls by dealing with them in shaders. We consider it slow to let the CPU do cell filtering, so we impose filters in shaders to reduce communication costs. We observe that each cell might have different shapes, but it is hexahedral anyway, so we manage the GPU to perform instancing drawing with a single draw call. The above and many other minor optimizations help ReservoirGrid render up to a million cells with an entry-level GPU.

For more information on 3D Rendering or INT’s GeoToolkit, please visit our GeoToolkit page.

Want to know more? Check out our webinar Optimizing 3D Subsurface Web Applications.