SEARCH WITHIN CONTENT
Citation Information : Transport Problems. Volume 14, Issue 1, Pages 45-58, DOI: https://doi.org/10.21307/tp.2019.14.1.5
License : (CC-BY-4.0)
Received Date : 19-September-2017 / Accepted: 07-March-2019 / Published Online: 23-April-2019
The work presents a methodology that allows us to evaluate the effectiveness of the system in accordance with accepted standards in terms of the three components of the “passenger–carrier–department” system, taking into account the interests of each side. The given comprehensive effectiveness evaluation of the urban public passenger transport system shows that a new scientific idea consists in forming fundamentally new sets of indicators for each participant of the system (passenger– carrier–department of transport) that correspond to the conditions of functioning of the urban passenger transport system in Russian Federation. The article presents a classification model for the formation of a basic system of indicators and a functional scheme for grouping and ranking them. The effectiveness of the urban public passenger transport system can be considered separately for each participant in the system from its side, in two directions: horizontal (by year, for example, for an enterprise) and vertical (in the current period for all participants).
In accordance with the transport strategy of the Russian Federation, “the priority development of public transport systems involves improving competitiveness, availability, and quality of passenger traffic…”. To solve this problem, “strategies and development concepts in accordance with the achieved results” are proposed. However, a comprehensive evaluation of the effectiveness of the urban public passenger transport system (as the degree of implementation of planned activities and achievement of planned results) is based on methods with low accuracy and reliability. Uniform methodology for the integrated evaluation of the effectiveness of urban public passenger transport system has not been developed yet [9; 13]. The lack of a comprehensiveness of the effectiveness of the urban public passenger transport system is the reason for the untimely management decisions, which negatively affects the efficiency of transportation and the quality of services.
The system of urban public passenger transport, in general, is the area of interaction of the three systems: city (and its government and departments), transport (with transport enterprises), and population (passengers as customers). Thus, new scientifically grounded solutions that take into account the methodological basis for evaluating the effectiveness of complex systems and consider the evaluation from three sides (carriers, passengers, and department of transport) with different sets of indicators are necessary [6; 12].
The evaluation of the results of the urban public passenger transport system is carried out on separate indicators for a certain period, while the changes in the values of these indicators are usually differently directed: the values of some indicators may increase and the values of other indicators may decrease. For example, the most common means of evaluating the effectiveness of enterprises is the analysis of isolated financial indicators by year. The evaluation of the effectiveness is often replaced by the evaluation of the quality of services (a set of system properties that determine its ability to meet needs and assess the level of customer service) and efficiency (degree of results and costs of resources). Therefore, the task of evaluating the effectiveness of functioning of the urban public passenger transport system is solved in a fragmented manner [8, pp. 25-34; 16].
The process of transport service should be viewed as an organized set of interrelated actions that gives a valuable result to the consumer. Thus, we can distinguish between different levels of effectiveness: individual (passenger), organizational (carrier), and process (department of transport). All these levels are interrelated and permeated with processes implemented in the environment. Each of the levels involves the formation of appropriate goals. The passenger wants to get the highest possible quality of services with minimal travel costs (cost, transfers, and physiological condition). The carrier is interested in obtaining the maximum profit (at the same time it is advantageous for him to raise the tariff to a certain value) and minimum investments (balance “tariff quality”) [5; 13]. However, at the same time, he takes into account the balances between “tariff volume of transportation” and “quality-volume transportation.” A carrier often reduces the number of “unprofitable” flights for him during peak periods and on weekends, which negatively affects the quality. But the carrier’s work on the line at such hours and days requires subsidies. At the same time, the department of transport is interested in reducing subsidies, which make up the difference between the fare and the cost of transportation, and in ensuring the maximum quality of passenger transportation, which directly affects the social and economic development of the city. The interests of the administration are to create an environment for successful management at all levels and to achieve the region’s strategic goals towards sustainable development. Despite the difference in the interests of the passenger, the carrier, and the administration of city, there are areas in which interests coincide (regularity, use of capacity, etc.). In conditions of competitive struggle of carriers for ensuring the quality of the transport service and obtaining competitive advantages, the significance of determining the effectiveness of functioning of the urban public passenger transport system and its subsystems is growing. It will help to identify “problem areas” and determine the solutions, contributing to the elimination of factors negatively affecting the transport process [2, pp. 21-45; 7; 8, pp. 30-44].
The methodology for the evaluation of effectiveness of the urban public passenger transport system must meet all the necessary general requirements of evaluation methodology, namely:
be clear, consistent, independent, and reasonable;
contain elements of a dynamic system, taking into account uncertainties;
allow identification and description of the functioning processes;
determine the relationship between the indicators of effectiveness;
include tasks of all types of planning and management;
provide a comprehensive (integrated) evaluation of the results in the shortest possible time;
identify the cause–effect relationships of reduction in effectiveness; and
The concept of quality of transport service is inextricably linked with the demands of its consumer. The process of transport service should be built on the basis of customer requests, while the interests of the carrier that is ready to work in this market should be taken into account.
Theoretical and methodological basis was the works of domestic and foreign scientists devoted to the problems of the evaluation of effectiveness of complex systems.
More than 400 performance indicators are used today. Each indicator is assessed on the basis of its performance category. Indicators could be classified as very important, helpful, or specialized as well as environmental, social, and economic. Therefore, a right balance has to be maintained while selecting indicators. Indicators should be relevant and analytically sound, and corresponding statistical data should be available. These data should comply with certain quality standards. Performance measurement can be carried out from different perspectives: from an enterprise perspective and from a customer perspective. It can take place at different levels: at policy level and at enterprise level. Many current indicators focus on operating efficiency (cost per vehicle-kilometre, etc.) rather than on users (comfort, speed, reliability, etc.). There are also following aspects: ratio (cost per revenue km), indices (frequency, route coverage, and measuring combining capacity) level of service (frequency levels), stand-alone individual quantitative or qualitative measures (ridership), and percentage .
The article  contains a fairly complete description of all existing methods for evaluating the effectiveness of the urban public passenger transport system. The importance-performance analysis (IPA)- and customer satisfaction index (CSI)-based models provide good results but are unable to give the reasons regarding the impact of each factor. Artificial neural network (ANN)-based methods present a better accuracy in analysis, but an obvious drawback of ANN stems from the fact that it fails to yield any direct numerical model as an output.
Authors  decide that the structure equation modeling (SEM) is one of the best modelling approaches in the field of research on service quality measurement. This is because SEM enables understanding the impact of each variable on service quality and customer satisfaction in a more pragmatic manner and, thus, provides an appropriate model for the estimation of each factor score and overall satisfaction in terms of quantitative measurement . Structural equation modeling (SEM) methodology is a powerful multivariate analysis technique in which a set of relationships between observed and unobserved variables are established. It is a relatively new method, the usage of which began in the 1970s (Fornell, 1981) and has been widely applied in various domains of research works, such as psychology, education, social science, economics, and statistics. SEM methodology refers to a series of statistical techniques, such as factor analysis, path analysis, and regression models, that are used to analyze data .
The article  evaluates urban public transport using the data envelopment analysis method (DEA) for three scenarios: infrastructure efficiency, service effectiveness, and efficiency versus effectiveness (equal performance). A few characteristics of DEA are convenient to deal with the inputs and outputs data, for these data can have different units .
The infrastructure efficiency indicator (IEI) is a measure of operational excellence in the resource utilization, while the effectiveness indicator (EI) refers to the use of outputs to achieve the passenger interest. Using the combination of the IEI and EI, it is possible to compare the strategic position of cities regarding the productivity and level of service delivered. The super-efficiency data envelopment analysis (DEA) measures the relative efficiency among comparable units; hence, it is sensitive to extreme values. To overcome the above difficulty the development of the model is an iterative process where the results must be validated at each iteration to reach a reliable model. Nevertheless, the DEA method provides relative evaluation results. Hence, the route with a good evaluation result may still require improvements .
For developing a methodology for the evaluation of the effectiveness of the urban passenger public transport system, we propose to apply the model of indexation (MINOR) and factor analysis. Because it is simple and reliable, it can work with different absolute indicators and it can identify all the factors that increase or decrease the effectiveness in order to decrease their influence. It is necessary to develop the measures for increasing the effectiveness, efficiency, and quality of service. It is based on measuring the proximity of two rank orderings of growth rates (plan and fact) using Spearman and Kendall’s rank correlation coefficients, generalized and positive coefficients of evaluation, and Pareto’s chart from the side of enterprises, passenger, and department of transport [14; 15, pp. 151-158].
In accordance with this model, the entire set of key indicators is converted from absolute values to relative values (growth rates of these indicators). This allows operations for indicators with different units of measure and the estimation of the value of the increment of indicators. The ranking allows us to express the dynamics of the indicators in their mutual relation, to evaluate the property of the system, which cannot be estimated by any of the indicators separately .
Furthermore, according to the rule of the Pareto-analysis “80/20,” the factors that have the most negative impact on effectiveness are identified. For an objective evaluation of the effectiveness of the transport process, the indicators should be complex and multifactorial, including qualitative, economic, and natural criteria, necessarily using international criteria [1; 3].
For the formation of the basic subsets of indicators for the carrier, it is proposed to build on the following levels:
- The “Passenger” is a perspective that evaluates the results of satisfying the interests of consumers (regularity factor, speeds, etc.) and characterizes the quality of services.
- The “Department of transport” is a perspective that evaluates results, which affect the satisfaction of the department of transport (subsidies, traffic volume, etc.), and characterizes the volume of services.
- The “Black box” of enterprise processes is a perspective that evaluates the effectiveness of the processes performed by an organization that has a direct impact on the degree of coherence of processes at the enterprise; these indicators are not of interest to the passenger and the department.
All three groups have intersection areas (Fig. 1).
For the ranking of indicators, the following procedure is suggested:
1) choose the indicators Ni with the planned value and the actual value included in group no. 1, which directly affect the evaluation of the effectiveness on the part of both the passenger and the department of transport, and the enterprise (quality-efficiency-volume);
2) choose the indicators Ni for group no.2, which directly affect the evaluation of effectiveness of both the passenger and the enterprise and are of no interest to the department of transport (quality-efficiency);
3) choose the indicators Ni for group No.3, which directly affect the evaluation of effectiveness from both the department of transport and the enterprise, and are of no interest to the passenger (efficiency-volume);
4) choose the indicators Ni included in group No.4, the “black box” of enterprise, which are important to the enterprise (efficiency).
5) rank the indicators within each group, establish the priority (significance) of indicators.
Systematized indicators of the work of enterprises should be ranked according to their importance, primarily for the passenger, and then for the department of transport, and in the last instance for the enterprise since, in market conditions, effectiveness is not only a good financial result, but is also, first of all, the competitiveness of the enterprise in terms of the quality of services and the amount of subsidies. As a result, an analytical model is formed in the form of a standard rank of indicators.
Sorting indicators with the calculation of correlation coefficients are presented in Table 1.
Calculations of factor analysis are presented in Table 2
Then, we develop preventive and corrective measures and access the cost of improvement measures. This stage includes the choice of the method of calculating the costs for each planned activity, calculating the costs in the amount necessary to carry out all planned activities and assessing each element of costs in order to identify opportunities to reduce them.
To assess the effectiveness of the urban public passenger transport system, the data of the documentation of the Department of Transport and large passenger enterprises of Omsk , the surveys of experts, the heads of the Department of Transport and Enterprises, were used. The set of indicators for evaluating the effectiveness by passengers and by the department of transport is fully with the Social standard for transport services for the population in the Russian Federation  (the first 13 base units) and supplemented with indicators that are also significant from the point of view of the experts. There is an example of the histogram of effectiveness for Passenger Enterprise No. 4 in 2016 (Fig. 2).
The proposed tool for the evaluation of the effectiveness allows us to evaluate the effectiveness of the carrier by years and compare the effectiveness of some carriers for a certain period (in 2016, for example) (Fig. 3).
Therefore, in 2016, Enterprise No. 4 worked with the lowest effectiveness (the only enterprise that showed a negative result in 2016).
The total positive effectiveness (TPE) for all municipal enterprises, 29.6 % in 2016 and 4.4 % in 2017, is shown in Table 3.
The integral coefficient of deviation (ICD) for all municipal enterprises – 0.639 in 2016 and 0.344 in 2017.
The greatest deviations in indicators, which change in one direction or another, are the reason for the decrease in effectiveness, which are observed in Enterprise No. 4. The smallest - in the company “Electric Transport”.
Obviously, for example, the main problems of Passenger Enterprise No. 4 in 2016 are complexities with the average annual cost of fixed assets, inadequate rates of rolling stock output and technical readiness, and a reduction in the length of served routes. The lack of purchases of new rolling stock and insufficient technical readiness led to the main problem of municipal enterprises - aging and deterioration of rolling stock. This, along with the unattractiveness of the profession of the driver of urban public passenger transport, influenced the regularity of traffic. That, in turn, led to the outflow of passengers to commercial rolling stock and gave impetus to the refusal to service part of the routes, which affected the position of municipal transport in the market of services, in general . At the same time, various “chronic” difficulties can be noted in the dynamics of various enterprises.
The total positive effectiveness (TPE) for passenger was 34.6 % in 2017.
The integral coefficient of deviation (ICD) for passenger was 5.194% in 2017.
The passenger of urban public passenger transport faces problems of the system, such as a low share of rolling stock of high ecological classes, a high cost of a ticket, difficulties with the availability of infrastructure and rolling stock for low-mobility groups of population, and a small proportion of vehicles connected to navigation systems with inclusion in the system of notification of arrival.
According to the same methodology, the evaluation of the effectiveness of the urban public passenger transport system from the side of the department of transport is calculated (Tables 6 and 7 and Figs. 7 and 8).
The total positive effectiveness (TPE) for the department of transport was 12.3 % in 2017.
The integral coefficient of deviation (ICD) for the department of transport was 3.645% in 2017.
In 2017, the system works ineffective from the point of view of the Department of Transport. The main problems observed are a low share of rolling stock of high ecological classes, a high cost of a ticket, and difficulties with the availability of rolling stock for the low-mobility groups of the population. Thus, carriers should take into account the environmental friendliness and accessibility of the vehicles for low-mobility population groups (corresponding door width and height of step).
The evaluation of the effectiveness of the urban public passenger transport system allow to see the true causes of reduced effectiveness and develop measures to improve the effectiveness of the system. For assessing the feasibility of measures, we can compare the annual costs of measures with the annual damage in the case of non-compliance with measures. Then, from the values of the coefficients (Table 8), it can be concluded that as a result of the measures taken, the effectiveness of municipal carriers has increased, and the effectiveness of the urban public passenger transport system has also increased from the side of the passenger and the department of transport.
At the same time, the current annual costs of municipal carriers decreased from 2537 million rubles to 2330 million rubles by 207 million rubles or by 8.2%. Revenues increased from 1103 million rubles to 1300 million rubles by 197 million rubles or by 17.9%. The costs of the measures were subsidized – the costs of the Department of Transport increased from 1507 million rubles to 2418 million rubles. The amount of the cost of measures was 910 million rubles or 60.4%. Passenger revenues from the reduction of waiting time and travel time amounted to 2881 million rubles or 2881 rubles per resident.
The proposed solutions can be used both for the evaluation of the effectiveness of the carrier and for the evaluation of the effectiveness of the urban public passenger transport system from the side of the passenger and the department of transport, having determined for them as specific indicators. The authors developed a classification model for determining the set of effectiveness indicators and formed indicators for the evaluation of effectiveness of the system from the side of each participant. The given comprehensive effectiveness evaluation of the urban public passenger transport system shows that a new scientific idea consists in forming fundamentally new sets of indicators for each participant of the system (passenger-carrier-department of transport) that correspond to the conditions of functioning of the urban passenger transport system in Russian Federation.
As a result of calculations by the proposed method, factors that negatively affect the effectiveness of the system from the side of each participant (passenger–department of transport–carrier) are identified. The positive effectiveness of the urban public passenger transport system is considered separately for each participant in the system from its side, in two directions: horizontal (by year, for example, for an enterprise) and vertical (in the current period for all participants). This allows the enterprise and the department of transport to identify deviations in the operation of systems at the processes level, determine the impact of each indicator on system effectiveness, calculate integral indicators that characterize effectiveness, and develop corrective or preventive measures.