How can a top manager get a dashboard and generate any analytical report for the business?

If I suggest to you how business owners can find out where your data is stored, what it consists of, and how it is used, you will most likely send me get lost at best. But if I suggest increasing control over your business and guaranteed to make more money, you will say “go for it” even before I explain how. Interestingly, in both of these cases, I will offer you the dashboard of the same solution.

Truly promising businesses with a good product (any product) do not receive a lot of money or go bankrupt precisely because of the lack of data or the inability to use it. The longer you are on the market, the wider the range or the more complex the geography of sales, the more related data your business will have. Skillful handling of them will become a reliable basis for scaling, and inept handling can even lead to an unexpected collapse of the entire business due to the accumulation of inconspicuous details. Simple reports will not help here: you must first work on the information competently, and this is a task for data engineering, that is, the Data Engineering direction.

Example of a dashboard for monitoring ticket sales. Image is illustrative.

Depth of reporting

A report is the basis of all understanding of business. Goods sold — a report, an employee worked a day — a work report, the month ended - a comprehensive report on all indicators, and so on and so on. Modern IT products can generate an infinite number of the most diverse reports based on databases, accounting systems, analytics from a website or a social network account.

Most typical reports are not flexible enough and overlook a number of important information. For example, sales reports - they reflect the main things: how much was sold, for what period, in which store, etc. This is enough to understand the progress of business, but not enough to analyze the situation and the reasons for the improvement or deterioration of results. A top manager sees the figure, compares it with the previous one and makes a forecast for the sales plan in the future.

But such reports don't show what time of day shopping activity was highest, where the sold goods went, from which regions orders were the fewest, or how much time was spent on order processing and packaging. All of this is important information for optimizing your business.

Dashboard for an eCommerce project. Image is illustrative.

And when a top manager wants to expand the report with more metrics, or analyze a certain period in some special way, a big problem arises: the result takes a very long time to generate, or the system generally “hangs” for a certain time. It seems like it’s not a problem, but if you try to do this with a database on the server, the entire ecosystem can “collapse”, along with the retail program, CRM, and all the business software that runs there.

That's why it takes time and effort to get in-depth or non-standard reports, especially during a period when the server load is low. There are at least three problems with this approach:

• long and expensive;

• cannot be obtained in real time;

• any changes to the query restart the generation process anew.

But there are ways to get rid of all this and generate any reports you want.

Another database

Simply and essentially, the cause of all problems is that people are “torturing” the wrong database. That is, they demand from the existing database what it was not created for. All your business platforms (online store, CRM, leading program) use transactional databases — those that work to receive information. Created on the basis of Mysql, MS SQL, Oracle, Postgress or other database management systems (DBMS), they are designed to store, structure and process “live” data and at high speed.

Such databases are called transactional precisely because of their ability to quickly process numerous changes to records. The product arrived at the warehouse - the first transaction, the product was sold - the next, the package is ready for shipment - the next, and so on. That is why you can have dozens or hundreds of sellers working on one database at the same time, selling thousands of products, and each request will be executed in a timely manner.

The leading role here is played by the database architecture and its optimization specifically for transactions. Speed is achieved by combining many different components, which store completely different records about products, their quantity, stores, sellers, and everything necessary for the business. Therefore, each individual transaction does not include all the information, say, about the product, but only contains links to database cells where individual characteristics are stored.

Each reporting request forces the transactional database to traverse all record fields one by one, and therefore to sequentially follow all links, which in turn may contain other links. This dramatically increases the load on the system and device memory, increases energy consumption, and lengthens the time it takes to complete the request.

Thus, it is difficult to achieve good results from a large transactional database, because it is simply not designed for this. Of course, the system can build a world of any depth, but at what cost and in how much time? That is why databases of a different type and with a different architecture are used for analysis. These are the so-called Data Warehouses or Data Lakes, which are optimized specifically for the output of information. They work almost on the reverse principle: filling such databases requires a certain amount of time to reformat the volume of information, but they respond to requests as quickly as possible. This approach was first used in scientific research, but its efficiency quickly found application in business.

So we draw conclusions:

• there is a lot of data, a lot of it and it is getting bigger over time;

• reports may lack information to make the right decisions;

• to obtain in-depth reporting, you should have a dedicated database that enables you to process requests for analytical reports effectively.

Transactional Database and Warehouse Architecture Diagram.

Data Engineering

It is impossible to say that Data Engineering is any specific instructions or rules — it is a direction of work that is responsible for collecting, processing, storing and preparing data for business. In the case of obtaining the necessary reports, the main task of a data engineer is to study your existing transactional databases and understand what information is absolutely necessary for business.

And when the information collection routes are defined, the processing mechanisms are programmed, and the information storage location is determined, you have a universal dashboard where you can generate any analytical report. Now all the ways of searching and systematizing data, deep analysis of processes and formation of dependencies are open to you, on the basis of which it is possible to make accurate and effective forecasts. Simply put: you understand where, when and how money comes to you, and therefore you can figure out how to multiply it.

Dashboard for analyzing a marketing campaign. Image is illustrative.

Avivi specialists work fruitfully with many analytical solutions. Let's give a recent example from Apache Spark, which was used on the project in the spring of 2026. Our customers are a large chain of clothing stores, with almost three dozen stores and several warehouses across the country, actively engaged in e-commerce. Every day, a huge flow of transactions passes through the company's databases.

Spark is both a powerful and flexible tool that allows you to systematically update data in the Data Warehouse from transactional databases and use ANSI SQL queries in the dashboard - the fastest of most available today in working with databases. Using Spark has provided the following capabilities:

1. Demand forecast and procurement optimization. Spark analyzes sales for each item and takes into account color, size, etc. This gives an understanding of what is best bought and when, and therefore how many units of goods to deliver before the new season, so that there are no excess leftovers and the necessary sizes are always available;
2. Redistribute products between stores. Real-time analysis monitors stock levels. Spark provides recommendations for even distribution to avoid out-of-stock situations in some stores and overstock in others;
3. Analysis of store and staff efficiency. A clear practice, but with a condition: this happens constantly, not once a month or quarter. Spark allows for timely response, identifying outsiders and promptly taking measures to correct the situation;
4. ABC/XYZ analysis of the assortment in dynamics. Similar to the previous one: analysis of revenue and sales stability is carried out daily. This gives an understanding of which positions should always be held, and which “dead” products on the shelves should be abandoned;
5. Buyer portrait and personalization. It works because customers have a loyalty system implemented. Spark analyzes customer behavior and clusters them by frequency of visits, average check, and other metrics. This allows you to segment customers for marketing campaigns and offer everyone what they are most interested in;
6. Optimizing the production plan for real demand. Spark analyzes sales from previous seasons with the production plan. This shows system errors (and they almost always exist), allows you to produce only what sells, and does not freeze money in illiquid balances;
7. Rapid response to trends. Spark alerts you when it notices “hot items” — items that are in the highest demand after they first appear on shelves. This allows you to quickly order a batch and capitalize on trending customer preferences;
8. Return analysis. Deep analytics of returned goods allows you to find systemic problems: design errors, poor-quality materials, factory defects, etc. Spark does not just record the fact of a return, but looks for the reason to eliminate problems in the future;
9. Pricing and discount management. Analytics analyzes each model according to different metrics. Thanks to the optimal discount curve, it is possible to avoid impulsive decisions regarding the discount and its size (“Let’s give it -30%, because it’s normal…”) and determine the time when it will be most appropriate. Therefore, discounts with Spark really work for Avivi customers, and do not just burn margins.

Importantly, Apache Spark supports Python, which is a perfect tool for the Avivi team to achieve our clients’ business goals. For years, we have been developing solutions of any complexity in this programming language and have extensive expertise. Learn more in the article “Why does Avivi love Python and what is this programming language capable of? ” on our blog.

Business value

Although at its core, a Data Warehouse is a database, from a business point of view this is not essential - first of all, it is a decision-making machine. We have already given examples above from retail and e-commerce in general, where we cited the advantages. However, there are entire industries where there is simply nothing to do without the use of data engineering and analysis. For example, the field of finance in general or fintech in particular. Here the number of transactions is measured in billions, and the volume of data is hundreds of Terabytes per hour.

Time is of the essence in fintech projects, and a delay of even 24 hours is a guaranteed loss. Therefore, credit risk assessment should be done in real time, not based on daily reports. It is simply impossible to obtain such an assessment without Data Warehouse from transactional databases. Another priority is a single customer card. A bank or financial institution knows more about a customer than they know about themselves: transactions, behavior in the application, support requests, products they use. Data Warehouse combines all this into one profile. Personalization, cross-sell, and early detection of customers who are about to leave are based on this.

And since profitability is the main criterion for fintech, another priority concerns this very issue. Top management needs to know the profitability of a specific product, a specific branch, a specific client segment — and have it updated daily, not once a month after two weeks of summing up tables in Excel.

Financial dashboard. Image is illustrative.

Separately, it is worth mentioning Artificial Intelligence. Quite often, Avivi is approached for the implementation of AI in business, and almost always it all starts with the work of data engineers. For some reason, business is in no hurry to streamline work with information, which has long become the main asset of the one who owns it.

As you know, AI uses various models for machine learning, and the more of them there are, the faster and better the artificial intelligence will learn to perform the tasks. This also requires analytical data that allows AI to read your business like an open book, not instructions for a product from an underground workshop. Of course, you can try to train a machine without using Data Engineering, but believe me, under such conditions, AI will only collect garbage for you, not profit.

summarize for whom in business the use of Data Warehouse is a must-have not only for development, but also for competitiveness:

• eCommerce projects with large amounts of data. There are always too many transactions, and even more data that remains unnoticed. Structured data management can increase profits by 23-27% in the first year of use just by rethinking aspects that waste money “nowhere”;

• Retail. Involves the use of accounting programs in many stores, which not only generates an endless flow of transactions, but can also cause chaos. Working with a Data Warehouse minimizes logistical and organizational costs, and studying customer behavior will bring additional revenue;

• Fintech. In principle, it is not possible without analytics, and at the highest level. Here is the main principle: Each report or analytical product must answer a specific question of a specific person who makes a specific decision. If you can’t name this person and this decision, then the analytics are being done for nothing.

Epilogue

One of the best examples of the successful use of a competent analytical approach to business development is the experience of the famous German retailer Lidl. This company was one of the first to realize: money comes to stores with customers and we need to leave it with us, even when people leave. Today, the chain has more than 12,600 stores in 31 countries around the world, and the total profit for 2025 amounted to 167.3 billion euros. Stable annual revenue growth is 5-10 %.

There are many factors behind the undeniable success of retailers, and the analytical approach is at the forefront of them. It is thanks to studying sales from different angles, through a combination of seemingly incompatible metrics, that Lidl's top management was able to track the cause-and-effect actions in visitors' purchases and stimulate them, reduce the percentage of losses from product spoilage on the shelves by accurately determining the time when they need to be delivered, and much more. Except for the use of Data Warehouses, with such a scale of business, it was simply impossible to achieve.

Example of a dashboard for sales funnel analysis. Image is illustrative.

Avivi team understands how to build competent ways of collecting, processing and analyzing information for your business needs. We don’t offer to tell you where the information comes from — we offer to create a dashboard solution for full control over your business and increase profits. Contact us, and our data engineers will show you things in your business that you never thought about.