One of the unique features of the IVAAP backend SDK is that you can develop your own data connectors and services with the IDE you are already familiar with. The data backbone of IVAAP is meant to be deployed in a cluster made of multiple nodes for scalability and reliability. However, despite the distributed nature of such a deployment, our SDK requires no particular plugin to compile or execute your code. The tools needed to develop a plugin for IVAAP’s backend are identical to the tools you would need to develop classic Java Servlets: a Java SDK (Oracle, OpenJDK), an IDE (Eclipse, NetBeans) and an application server (Tomcat, Glassfish).
From this description, you might think that once you have installed these three components, the installation phase is over. In reality, there are many other tools that are needed on a daily basis to get the job done. I did a brief survey among INT backend developers, and these are some of the tools I found to be commonly installed on their PCs. While some developers may use Linux, for this article we’ll focus on tools found in a Windows environment.
Postman: This is probably the tool backend developers use the most. The IVAAP backend uses a REST API to send data to the front-end, and using Postman is the main way to debug such web calls. The backend’s JSON API uses HATEOAS links, and Postman makes it a breeze to navigate these links. Postman keeps a history of all HTTP calls you tried, measures their performance and is compatible with IVAAP’s default authentication system (OAuth 2.0 bearer tokens).
Google Chrome: This seems obvious since IVAAP is a web-based application. While backend developers don’t work on the user interface, a browser is always needed to validate that the backend and front-end work together well. Google Chrome is preferred because it includes a well-designed “Developer Tools” window allowing an investigation of the HTTP calls made by the HTML5 client. To troubleshoot issues visible on the client, you typically don’t need to know how this client is written, just which HTTP calls it makes. Google Chrome exposes this information very clearly to backend developers, including real-time data passed through web sockets.
Notepad++: The moment you have automated builds, you have configuration files to maintain. While you could use Windows’ built-in Notepad application, Notepad++ is a necessary upgrade: it handles gracefully carriage returns from Linux files and makes it easy to compare the content of two documents. It also provides syntax highlighting, all while keeping Notepad’s advantage against IDEs: it opens text files fast.
Git Extensions: While most of our code will be pushed to our Git repository through the IDE, build configuration files are typically maintained outside of that IDE, requiring additional tools accessible from the file system. Git Extensions provides a graphical interface to Git, and often does a better job than IDEs at showing file revisions.
JProfiler: JProfiler is generally used in two cases: to verify whether the backend has memory leaks, or to detect bottlenecks in the code. It’s one of the rare items in this list of tools that is not free. I found it superior to IDE’s built-in profilers (easier to use and more features), and well worth the purchase.
DbVisualizer: Lots of geoscience data stores are SQL databases. DbVisualizer is versatile—it can connect to all types of database (Oracle RDBMS, Microsoft SQL Server, PostgreSQL, etc.). Writing a connector for a particular database would be impossible without being able to visualize the raw data. You could opt to use the IDE’s built-in SQL tools, but DbVisualizer is also used by project managers (and sometimes clients) to inspect the data you are working with. Using the same software across all stakeholders saves time.
dbKoda: Another popular data repository for geoscience data is MongoDB. To develop IVAAP’s “mongo” connector, developers need a graphical way to inspect data. dbKoda allows you check the syntax and execution of queries, both for reading and writing data. MongoDB Compass Community can be used instead to browse a large number of records at once.
Microsoft Azure Storage Explorer: IVAAP has connectors for Amazon AWS S3, Microsoft Azure Blob Storage and Google Cloud Storage. All these cloud vendors provide a web interface to browse, upload, and download data from the cloud. I prefer to use a desktop application such as Microsoft Azure Storage Explorer when I work with Azure’s Blob Storage. It makes it easy to visualize what’s in a repository and provides a reliable way to upload geoscience datasets. It works for demos as well—when I need to demonstrate an ingestion workflow with INT’s tools, Microsoft Azure Storage Explorer provides a graphical way (“just drag and drop”) to upload data.
Liquid Large File Editor: Notepad++ does a good job in many use cases, but when it comes to inspecting server logs, it chokes at large file sizes. The IVAAP backend outputs a generous amount of logs, very useful for troubleshooting issues. This editor makes it easy to inspect the logs of a server, even after it has been humming along for weeks. It also comes in handy when you need to create small test data files out of large ones.
MobaXterm: The code of IVAAP’s backend offers many deployment options. No two customers use all options, and picking which options to deploy is a key step to any deployment. When things don’t work as planned, you need to know which modules have actually been deployed. You could interrogate the continuous integration system (Jenkins), but the ultimate way to troubleshoot a deployment is to checkout the files directly. With developers working on Windows and deployments on Linux, MobaXterm is a multi-tab user interface to perform SSH against any Linux server that needs inspection.