Markov game approach for multi-agent competitive bidding strategies in electricity market

作者：Rashedi Navid; Tajeddini Mohammad Amin^*; Kebriaei Hamed

来源：IET Generation Transmission & Distribution, 2016, 10(15): 3756-3763.

DOI：10.1049/iet-gtd.2016.0075

• power markets
• power system economics
• power system analysis computing
• Markov processes
• game theory
• multi-agent systems
• learning (artificial intelligence)
• decision making
• optimisation
• Markov game model
• multiagent competitive bidding strategies
• competitive electricity market
• optimal bidding strategy
• individual profit maximisation
• straightforward optimisation methods
• supply function equilibrium model
• uniform price market clearing mechanism
• incomplete information game problem

In a competitive electricity market, suppliers seek the optimal bidding strategy, maximising their individual profit. Due to the uncertainties, unknown parameters and dynamics of electricity market, the optimal bidding strategies cannot be attained through straightforward optimisation methods. In this study, the supply function equilibrium model is considered together with uniform price market clearing mechanism. The interaction among suppliers is modelled as an incomplete information game problem and multi-agent reinforcement learning (MARL) is utilised to find the optimal bidding strategy of the suppliers in a non-cooperative game. The proposed framework is an extension, from single agent Markov decision process to Markov game process, which is suitable for studying multi-agent decision making problems in stochastic environments. In this study, a Markov game model of the electricity market is proposed and a new state-action and Markov model of electricity market is proposed to built up the MARL environment. In addition, a learning-based non-cooperative MARL method is utilised for learning the optimal bidding strategies of the suppliers in a day-ahead electricity market. The proposed method is successfully applied to IEEE-30-bus power system. To examine the statistical significance of the results, the T-test method is applied to compare the competitive behaviour of the players in both multi- and single-agent frameworks.

• 出版日期2016-11-17