مدل آگاهی از شرایط مبتنی بر کیو
Abstract
1- Introduction
2- Bilateral contracts negotiation
3- Proposed methodology
4- Case study
5- Conclusions
References
Abstract
Automated negotiation plays a crucial role in the decision support for bilateral energy transactions. In fact, an adequate analysis of past actions of opposing negotiators can improve the decision-making process of market players, allowing them to choose the most appropriate parties to negotiate with in order to increase their outcomes. This paper proposes a new model to estimate the expected prices that can be achieved in bilateral contracts under a specific context, enabling adequate risk management in the negotiation process. The proposed approach is based on an adaptation of the Q-Learning reinforcement learning algorithm to choose the best scenario (set of forecast contract prices) from a set of possible scenarios that are determined using several forecasting and estimation methods. The learning process assesses the probability of occurrence of each scenario, by comparing each expected scenario with the real scenario. The final chosen scenario is the one that presents the higher expected utility value. Besides, the learning method can determine which is the best scenario for each context, since the behaviour of players can change according to the negotiation environment. Consequently, these conditions influence the final contract price of negotiations. This approach allows the supported player to be prepared for the negotiation scenario that is the most probable to represent a reliable approximation of the actual negotiation environment.
Introduction
The Electricity Markets (EM) restructuring placed several challenges to governments and to the companies that are involved in generation, transmission, and distribution of electrical energy. The privatization of previously state owned companies, the deregulation of privately owned systems, and the internationalization of companies, are some examples of the transformations that have been applied [1]. Environmental concerns related to the use of fossil fuels have led to an increase in renewable energy generation sources. The considerable increase of distributed generation units makes EM more competitive, and consequently encourages a decrease in electricity prices [2,3]. However, some recurrent problems that are being addressed all over the world must be considered, such as the dispatch ability, limitations in the power system network, and the integration and large participation of small producers in the EM, among others [3]. Despite these problems, some global solutions are being adopted, some examples are the case of evolution of European EM. The majority of European countries have joined together into common market operators, resulting in joint regional EM composed of several countries, which supports transactions of huge amounts of electrical energy and allows the efficient use of renewable based generation in places where it exceeds the local needs [4]. Nowadays several market models exist, with a set of complex rules and particular regulations, creating the need to anticipate market behaviour. Some implemented market types have the clearing mechanism based on the optimization of offers, such as most electricity markets in the U.S. [5] and other based on symmetric or asymmetric bids, as is the case of most European countries [4]. However, electricity trade worldwide is also supported by means of bilateral contracts negotiation [6], which are the scope of this study. The common behaviour of market players in contracts negotiation is mainly based on the definition of prices and quantities in energy transactions with each competitor. Hence, relevant information concerning competitors’ history of previous negotiations can be used to improve the decision-making process, considering the characteristics of the moment of negotiation, namely to improve the forecasting of possible contract prices before the negotiation process [6]. It is essential to consider the concept of context awareness, since it influences the prices and quantities of energy to be negotiated. One example is the new ways of participating in EM, such as renewable resources, which has hardly influenced the players’ participation in the negotiation process, due to the dependency of environment factors, such as wind or solar intensity, that influence the final price of electricity. Other examples of contexts are the different types of days such as business days, weekends, holidays, or other days with special situations that affects energy consumption. A unique review of context analysis mechanism of EM negotiating players is presented in [7], which proposed a methodology of analysing the past negotiation context to distinguish days and periods with similar characteristics.