How to work with Open Source projects in Microsoft Azure. Part 3

Non-obvious ways to reduce costs in a well-established infrastructure

Companies often face a situation where they receive invoices for services that no one seemed to have ordered. But it turns out that the developer simply forgot to disable some running function. Or, imagine another case: the company launches a large number of virtual machines, and is forced to pay for the increase in consumption. 

Let's consider an approach that will help to avoid such situations, using the example of the replication of machine learning services.

Case: replicating machine learning services

The task of the project was to develop a platform for determining anomalies on satellite images based on Open Source technologies.

The tool was required to allow the operator to upload an image, this image was then analyzed using machine learning and certain anomalies were identified in the landscape.

Therein has lied a big problem: the image size could reach up to 14 GB (tif with information about the terrain, projections and other technical data that were used in the calculations). 

The solution had to be scaled to several regions, while isolating it between them. That is, each operator had to get its own interface, and loading images and receiving a response had to be implemented in real time.

Keeping an expensive virtual machine with a GPU for every region would be prohibitively expensive. Using the documentation and a calculator provided by Azure, WaveAccess calculated the optimal consumption option by forming a queue for operators, and agreed on the response time with business analysts. As a result, the images are processed through Python in one large virtual machine, and the response is returned to the queue, thus ensuring asynchronous processing. 

This in turn ensures that the Azure resources you are using are continuously running. With a large number of users and replicating services to regions, a more efficient use of virtual machines will be obtained during the working day. 

The solution’s key features included:

• Large size images download (6-14 GB)

• Continuous processing of information coming from different regions (training + prediction)

• Jupyter Notebooks support for machine learning model development

As a result, the developed system allows for the processing of data from six regions using only two Reserved Instances.

If necessary, additional capacities are temporarily put into operation to complete user tasks (e.g., for training purposes). If there is a sudden change in some specific feature of the image impacting the calculations accuracy, the system will start additional training of the machine learning model. In turn this has allowed the module to make adjustments to its own characteristics and indicators, and the new uploaded images in the new format have been processed with greater accuracy. 

The project was built using technologies and tools as follows:

• Microsoft Azure

• Azure Kubernetes Service

• Java Spring Framework

• Python, Jupyter Notebooks

• Apache Kafka

• JavaScript, AngularJS, HTML5

• Terraform, Ansible, Docker

Cost reduction strategies 

There are several main options for reducing costs with Microsoft Azure:

1. Reserved virtual machines

• Reservation for 1 and 3 years

• Сost reduction for virtual machines up to 80%

• Possibility to pay in advance or on a monthly basis

2. Shared reserved virtual machines

• Aggregating the load of multiple virtual machines into one reserved one, "as required"

• Open sources provide ready-made scripts for calculating the consumption level and the amount of constantly used resources in Azure subscription

  • Based on this calculation, we can draw a conclusion about the number of reserved machines. This will reduce overall costs by up to 80% and reduce resource management to one virtual machine

3. Connect Azure cloud as a “sandbox”

• Development of new services using Azure capacities

• Test run in Azure

  • In this case, the Azure Kubernetes Service is of particular interest as a cluster unit where you can distribute the load if necessary. Azure services can be used as a “sandbox” for your contractor: you can engage the external team by granting access to your subscription and provisioning the right resources without having to tweak your infrastructure or the exposure of sensitive information to third parties.

• More accurate calculation of the hardware requirements in the data center after load testing

  • Placing a new service in Azure will allow you to quickly launch it and determine the consumption level that is required in the local data center. If you are considering  buying of new equipment, it makes sense to perform load testing and trial operation in Azure, and calculate the costs.

4. Migrating Microsoft solutions to the cloud

• Lowest prices for Windows virtual machines and MS SQL Server

  • Using Windows virtual machines and MS SQL databases in Azure is a more reasonably priced option compared to other clouds and the on-premises data center. The thing is that, in addition to installing this DB, you will also need to maintain the underlying infrastructure itself. Because of this, an allocated budget for a ready-made data center with these out-of-the-box services might be a good compromise, but in this case you lose the option of dynamic configuration, management through Terraform and automated load balancing using proven services.

• Extending VMWare Infrastructure to Azure Cloud

  • You can not only migrate Microsoft 365 solutions or servers, but also connect Azure as a VMWare cluster, thus expanding the virtual park. Which, in turn, will also give you additional capacities.

Change management: how to control business process optimization

Introducing changes is an ongoing process that allows you to constantly strive for the quality of the service that you are developing. We build it along the following line:

1. Planning

2. Development and testing

3. Commissioning

4. Monitoring and telemetry — new planning

Azure provides everything you need to implement any project: both development tools and a unique component set that allow you to finely track various metrics and characteristics.  Also it allows you to compile statistics on services and solution delivery tools with Azure DevOps Services among them.

According to the Accelerate State of DevOps Report by DORA and Google Cloud DevOps & SRE (2019), by implementing Azure DevOps, companies can expect to receive benefits in:

• Release a product or an update 208 times more often

• Deploy 106 times faster after making changes

• Bring the product to market faster

• Increase profits

• Restore the system's performance after a failure 2.6 times faster

• Reduce the likelihood of application failures by 7 times (by introducing tools for monitoring and tracking the state of the infrastructure as a whole)

Today, Azure DevOps Services (Server) provides five tools:

1. Azure Boards

   Enables planning and managing development. An analogue of Jira and YouTrack. It provides a broad array of functions for supporting projects using Agile methodology (Scrum, Kanban)and the ability to customize reports, dashboards, etc.

2. Azure Pipelines

   A service for building, testing and delivering code with the support of CI/CD (both to Azure and to remote virtual servers) and various languages, platforms and deployment environments (Azure, AWS, GCP). Support for monitoring and alert functions ensures that automated code quality and development results checks are performed.

3. Azure Repos

   Open and private code repositories compatible with git and TFS

4. Azure Test Plans

   A set of tools for automated and manual testing of web, mobile and desktop applications. Supports end-to-end tracing, which allows you to determine, down to a line of code, which functional bug is to be attributed to a particular developer. By integrating with Visual Studio and Azure Boards, you can complete a report and submit a task in a few clicks, and this allows you to effectively structure a team's work around the entire line of tools.

5. Azure Artifacts

   Service for storing libraries and distribution packages (Maven, npm, NuGet and Python) — of both your own and upstream repositories.

To dive deeper into this topic, we recommend that you explore:

• Agile Planning and Portfolio Management with Azure Boards

• Embracing Continuous Delivery with Azure Pipelines

• Automating infrastructure deployments in the Cloud with Terraform and Azure Pipelines

• Test Planning and Management with Azure Test Plans

Data center + cloud: why do you need Azure Kubernetes Service and multi-cloud

Some companies are eager to efficiently develop services, but are limited by budget constraints and the timeline. At the same time, they require specific functionality for their developments and other projects, but, unfortunately, are not ready to provide source code and distributions in their environment.

Case: integrated software development environment

Let’s consider a project implemented for an innovative technology company. The services used by the company's employees were located in the data centers, but the management was not ready to let third parties into this environment from a security point of view. It was necessary to create a software development and testing environment isolated from the production environment, for which the WaveAccess team chose Azure.

The key features of the solution offered by WaveAccess involved:

• Organizing the work of several contractors in an integrated development environment.

  • There is an Azure Kubernetes Service cluster, where several environments are set up for different products and different teams: they do not overlap, have access to a limited number of resources in the subscription and develop all the functionality under the contract. Communication was carried out through the ticket system in Jira. Granting access or expanding the cluster in terms of capacity are also quickly issued through a ticket.

• Provisioning the tools for project management, test automation and results delivery

• Limited cloud integration with a private network

  • This turned out to be a truly  important feature of the solution. A multi-cloud installation of Kubernetes provided the ability to move services from development environments to a pre-production environment in a private network. That is, the customer's engineers could easily transfer the settings and reproduce the environment, check the security and code quality. CI/CDs were implemented based on Azure Pipelines. The access to the environment and the located services required additional migration due to the company's security policy.

• Easy transfer of settings and work results from the development environment to production and back

As a result, a Microsoft Azure based development environment was built and integrated with Azure DevOps Server in a private network. Contractors gained access to the environment in Azure Kubernetes Service and projects in Azure DevOps Services.

Kubernetes as a bridge

Azure Kubernetes Service often acts as a “glue”. In several projects, we have come across a situation where a company had both a public network with a Kubernetes based development environment, and a private network where capacities and services were transferred to pre-production operation — after all the work has been completed.

In conclusion, here is a number of strong points that distinguish Kubernetes both as a service and as a connecting link:

• A strong community of developers and administrators

  • Almost any problem can be quickly resolved: many companies are engaged in maintaining and configuring the Kubernetes clusters, finding a contractor for such tasks is not hard at all.

• Azure Kubernetes Service is a provider-managed Kubernetes cluster with unlimited scaling

  • It is handy to deploy an entire cluster in a few clicks, going through the workflow. It does not require delving into virtual machines, configuring the network and uniting servers into a single cluster. The same goes for multi-cluster development and installation: connecting an external cluster does not produce any difficulties.

• Extensive functionality of access control

  • You can ensure the work of several contractors within the same cloud, split them by environments, services and even deployments.

• Kubernetes has become the de facto standard for delivering IT products

• A high degree of automation of resource management operations

  • You can manage both the cluster itself and the network nodes inside it, and the load on them. There are many solutions to enable commissioning under Canary Deployment strategy, Blue Green Deployment or Waterfall —  all of these approaches are available in the Azure DevOps Pipelines.

• Popular Open Source solutions provide ready-made scripts for installation on Kubernetes

  • At least, this is a Standalone installation, and in most cases you can find ready-made cluster solutions with fault tolerance and crash recovery.

• Ready-made solutions for connecting multiple data centers into one cluster

• A global partner network with 24/7 support

Make sure you have browsed through the whole article covering the strategies on how to efficiently work with Open Source projects in Microsoft Azure using the DevOps tools and practices: Part 1 and Part 2.

We create a fully automated solution to allow continuous software integration, deployment, and improved service delivery. Request a quote!