How to solve system integration tasks for some complicated workflows using the Wave.Integrate tool

For the last 20 years, we implemented hundreds of integrations between different systems: e-commerce platforms, authentication apps, machine learning services, payment gateways, and more. To reduce costs and speed up projects, we developed Wave.Integrate, a flexible platform with wide integration abilities. While a previous article was focused on smaller tasks that were solved by Wave.Integrate,  today let’s refer to more complicated questions. 

Task #1: integration of encrypted files or archives

The Challenge: file archives, especially, in some legacy systems, can be encrypted (for example, using PGP or analogous solutions). Some excel files used in business processes can be also password-protected. To work with these types of data files, it is necessary to perform a series of steps, such as connect to the sFTP server, download a file from a specific folder, decrypt and parse it, and only then pass the extracted data to a target system. In some cases, after the transfer of data, it is necessary to generate the response file, encrypt it and put it back on sFTP for the source system.

Our Solution:  Wave.Integrate supports FTP, sFTP, shared folders file sources, it can read CSV files with or without headers, Excel files (including password-protected ones), it can apply complicated data transformations and data cleaning,  and supports a number of destination adapters for target systems.

Task #2: Integration with manual data approvals based on a user role

The Challenge: in some cases, it is necessary to get manual approval of sensitive data transferred to external systems. For example, to sign a contract. The number and type of approvals may require, for example: 

• To get approval from a single approver

• To get two approvals from any of three approvers

• To get one optional approval from a lower-ranking approver, and one mandatory approval from a higher-ranking approver, etc.

Specific persons who must approve documents may not be ‘hardcoded’ in the workflow, but be selected based on the given user roles. Moreover, the business rules managing the approval process can be changed with time. These requirements depend on a particular company’s workflow. 

Our Solution: we solve such kinds of integrations using Wave.Integrate workflow capabilities with built-in versioning support and also custom extensions implemented to get approval from external approvers. The ability to visually design business processes allowed us to customize the processes involving people who have more knowledge about business than technology. For example in one of the cases, the integration received data from the Ethereum blockchain solution and drove through the approval process using a custom web application with REST API and MS Exchange to collect actual approvals.

Task #3: Integration with Machine Learning and Big Data

The Challenge: some business processes require making decisions based on machine learning methods. For these methods, it is necessary to create mathematical models based on a large amount of collected data. 

This data can be obtained from different systems via different channels like: websites, CRM, data archives, etc. After the data should be cleaned, cured, and transformed in order to get the ML-model of proper quality. The amount of data and the variety of preprocessing algorithms create additional difficulties. 

Our Solution: the whole solution utilized different features of Wave.Integrate. The ETL feature was used for preparing historical data, including loading from different sources, cleaning, and transformation of the data. The Workflow feature was used for automation of the business processes. Special activities were added as plug-ins for interacting with Python code which was used to implement machine learning algorithms.

Task #4: Integration with Exchange

The Challenge: it’s not a secret that companies that work with people and documents (real estate agencies, law firms, etc) have a tight schedule, and it needs to be organized. Exchange and Outlook often are the main tools for doing that. But when it comes to a new application where users need a custom calendar module, they need some sort of synchronization with Exchange to carry over all existing (or relevant) appointments. 

Our Solution: WaveAccess has made it possible by writing a Windows service that syncs calendar items from Exchange to a custom calendar module operating by business rules specified by the client, featuring syncing not only personal appointments, but also Public ones that are available for the whole firm. Appointments created in the Calendar module are also carried over back to Exchange for the purpose of two-way sync.

The service also supports syncing of contacts maintained in the CMS that are a part of the same comprehensive application written for the company. Whenever users add contacts in the CMS, they are automatically carried over to Exchange.

***

Wave.Integrate can be implemented within any business logic connecting more then two systems. Use it for: 

• Automated import/export of data

• Real time bidirectional synchronization

• Data deduplication

• Workflow automation

• Complex integration modules development

Empowered with Wave.Integrate platform, your complex projects will be delivered with improved speed and quality.