Bitrix24 on-premise clustering: what is it, why it is needed, and how Avivi`s specialists implemented such a project

In the era of Web 2.0, a combination of two methods is usually used to solve performance issues: software improvements and hardware improvements. If the software requires significant resources even after optimization, the traditional solution is to add new hardware capabilities and distribute the load on its individual parts. Clustering in Bitrix24 on-premise is a combination of many technical solutions that allow you to distribute Portal resources to separate servers and ensure their interaction. In this way, it is possible to solve several problems at once:

• Increase service availability (HA — High Available clusters);

• Scale the IT project in conditions of increased load (HP — High Performance clusters);

• Balancing load and data traffic between separate software components;

• Create a complete backup for MySQL.

This is actually a lifesaver for large projects and rapidly evolving systems. Let's first look at the methods that Bitrix24 offers out of the box, and based on that we will demonstrate both their advantages and disadvantages, as well as unexpected problems that can befall those who want to cluster their system.

Web cluster for Bitrix24 on-premise

The "Web-cluster" module contains various technologies that allow you to manage data, its flows, and the processing process. As already mentioned at the beginning, the principles of distribution and strengthening of individual parts of the system are used to overcome overload challenges. The module allows you to manage the cache, databases created in Bitrix24 based on SQL, and the placement of software components on separate servers.

 A total of 5 options and their combinations:

Vertical sharding

Sharding is a method of dividing and storing a single logical set of data in the form of many separate database segments. Because all parts are stored on the same level of the system, this is sometimes called "horizontal sharding", but the first word is simplified for convenience. Another thing is vertical sharding: dividing one database of a web application into two or more parts by extracting separate modules. Web analytics, search, etc. data can be moved to such modules.

MySQL replication and load balancing between servers

The word "replication" got into IT from biology, where it means spontaneous reproduction of DNA molecules by copying. "Web-cluster" allows you to do almost the same thing: the MySQL database is stored on separate nodes of the system and synchronizes common data at the right time. This approach has the following advantages:

• Flexible SQL load balancing;

• Simple administration;

• Cheap, and most importantly, unlimited scalability;

• Backup in real time;

• Does not require customization of the logical part of the web application.

In this method, the "Master"-"Slave" scheme is established, where there is a main server, the information of which is checked by other nodes. We will return to this particular method, as its choice caused certain difficulties for one of our customers.

Distributed data caching

An effective solution used in "Web-cluster" is the use of the Memcached service. In short, caching of data and requests to databases is stored in RAM on separate servers, which increases the speed of information processing. An important feature is virtually unlimited scalability. All that needs to be done to harden the system is to add the next new server from the memcached pool. High performance is provided by the use of a centralized system cache. It also increases the reliability of operation in case of failure of any arbitrary component of the system.

Continuity of sessions between web servers

This method involves saving user sessions in the database. This provides "transparency" of sessions for all web servers of the system: after authorization on any of the servers, the user is considered authorized on the rest of the servers. Conversely, ending a session anywhere causes all current sessions on all available servers to end.

Web server clustering

A comprehensive solution with load balancing on different servers. Under such conditions, the database is stored on a separate powerful server. The system determines the load on each of the available web servers and balances the traffic between them. The server nodes are synchronized automatically, which ensures less than 50% load on each of them. This method is also characterized by an almost unlimited degree of scaling, because an unlimited number of new nodes can be added to increase performance.

Now that the situation with data distribution and the methods of working with it are generally understood, let's consider a specific example of a project implemented by the Avivi development team.

Practical implementation of clustering

Our company's customers from the United States of America have extensive businesses with a large number of online users and a large traffic flow. For work, they chose the Bitrix24 on-premise plan, but over time upgraded it to the Enterprise level. One of the reasons was the need to implement the need for Multiple divisions, but that's another topic to cover on the Avivi blog. Another important reason is to stabilize the work of the Portal with the capabilities of the "Web-cluster" module already with the new tariff.

First of all, customers were inclined to the idea of replication of MySQL databases, however, after analyzing the business, Avivi specialists substantiated the need for web server clustering. But there were also surprises here.

Three separate powerful servers were allocated for the task: one for the MySQL database and two for hosting Bitrix24 files. Note that each of the servers must meet certain requirements for the operation of the Bitrix VM virtual machine. According to the general recommendations from the vendor, the standard scheme "Master"-"Slave" was used, where one of the servers was assigned the status of the main one, and the other — the secondary one.

However, already at the testing stage, a significant problem was revealed. Systematically perceive the "Master" as the main and irrevocable source of information. Thus, if, for example, the user got to the "Slave" server at the time of loading and created a new Entity there in the Bitrix24 CRM, after data synchronization, its data was lost. Reconciliation with the data on the "Master" did not reveal such an entity, and therefore the information was not recorded in the MySQL database. Instead, everything created by the users of the "Master" server is definitely stored and available on the "Slave". A similar situation also occurred with files that users downloaded on a remote server.

The solution was to reset the interaction scheme to "Master"-"Master". As a result, customers received two equal servers with no hierarchy conflict. Under such conditions, any information from both servers is equally synchronized with the database. In the end, everything happens according to the specifics of the clustering of the web server.

Let's finalize

Owners of Bitrix24 on-premise have a unique opportunity to protect their business online from peak loads, increase the level of fault tolerance and balance the work of the business IT structure thanks to the "Web-cluster" module. Among several options, you can choose the most suitable one or use their combination. In any case, clustering requires the involvement of professional developers who will be able not only to implement the project flawlessly, but also to choose the optimal solution and correct critical errors that are not mentioned in the official documentation from the vendor. Avivi developers have relevant qualifications and have extensive experience in servicing on-premise editions and Enterprise solutions. So the Avivi team is ready to help with clustering for your business.