Blog
Beebop Insights
Keep up to date about the latest news and trends
The flexibility of large scale batteries and Virtual Power Plants (VPPs) is used for near real-time imbalance management and (continuous intra-day) energy trading. The underlying decision making problem can be quite daunting, this is a direct result of its sequential nature, inter-temporal dynamics, uncertainty (and underlying risk), partial observability and non-linearities [when being a price-maker, next to being in a multi-agent setting competing against agents with similar strategies]. In the last years there has been a growing interest in imbalance management and intra-day trading due to the limited market depth of battery-friendly ancillary services and a surge in (announced) battery projects putting pressure on prices, e.g. in the UK the revenue from ancillary services has all but dried up , nowadays any optimizer worth her salt flaunts its multi-market algorithmic trading prowess. We consider 3 paradigms to solve the underlying optimization problem. A first paradigm, which is uncomfortably powerful (if one likes tinkering with algorithms), is that of “ simple ” expert-based rules captured in e.g. a decision tree. A second paradigm followed by many (if not nearly all) algo-traders is that of mathematical programming. Scores of PhD students have mulled over solving near real-time decision making in the context of the problem above w ith multi-stage stochastic programming in all shapes and forms. If the reader would like to catch up on this, an excellent PhD thesis by Priyanka can be found here . In our view, this strong focus on mathematical programming is to a large extent driven by the presence of a fundamental linear structure in the decision making problem (linear cost function, linear dynamics) in combination with easy (and cheap at research level) access to powerful solvers such as CPLEX, Gurobi and the likes. These methods, in our opinion, can get you pretty far but start showing cracks when one is confronted with large fleets of assets (with coupling constraints), a realistic multi-market setting with coupled trading decisions and when one can no longer be considered a price taker and/or one is competing with other agents. To solve some of these challenges, elaborate bi-level optimization schemes have been explored, e.g. by Zugno and Smets , and although powerful concepts, in our view these methods sacrifice (too much) model fidelity on the altar of mathematical programming by shoehorning the decision making problem into a structure that typical mathematical programming approaches can get away with. Currently however, they are the main work horse for most algo-traders. A third paradigm is that of reinforcement learning, a potpourri of concepts, tricks and algorithmic coterie. These ap proaches have in the last years demonstrated remarkable performance, e.g. in the context of nuclear fusion or drug discovery . Reinforcement learning carries a promise that one has to make far less compromises on the model-fidelity compared to mathematical programming. A challenge however is that it is not trivial to obtain stable and reproducible solutions that generalize in an intuitive way over non-observed states. An example of how reinforcement learning can be used for continuous intra-day trading can be found here . Closer to Beebop, Soroush quite recently published the results of close to a year of research on how to get stable and reliable solutions in the context of a battery for imbalance management in a European power system. We consider distributional reinforcement learning in combination with a policy gradient method and careful policy design as an excellent starting point to obtain a high-performance policy without making uncomfortable model assumptions.. At Beebop, we look at these problems from the perspective of large heterogeneous fleets of decentralized assets such as heat pumps, batteries and electric vehicles embedded in a distribution grid, which has (at least) all the complexities of the above. To obtain a truly practical scalable and performant approach, the Beebop team is blending all of the paradigms above, mathematical programming, reinforcement learning and rule-based control into one framework that allows us to harness the flexibility of large heterogeneous assets in a grid-secure, multi-market optimization approach respecting each asset’s constraints and cost.
Crafting Teams and Technology: Empowering Virtual Power Plants In the last 14 years, I have been spending an important part of my time crafting teams, technology and algorithms that bring energy-constrained flexibility to our energy system through virtual power plants. In the coming weeks and months I plan to discuss more about how Beebop’s hybrid algorithms will truly bring scale to Virtual Power Plants. Here however, I want to focus on something quite different but quintessential to deliver high quality flexibility products: team values . Navigating the Terrain: The Essence of Team Values Building virtual power plants is a challenge that requires a balanced team with a set of shared core values. One needs dreamers, opportunity seekers and team players that see a solution for each problem. This needs to be balanced by pragmatists and team members that see a problem in each solution, in my experience once you find a harmony between these treats can you really deliver on innovation. This harmony however can only thrive if the team can have open, direct and respectful conversations on ideas and solutions, play the ball, not the person, this brings me to integrity , a first core value. Balancing Dreamers and Pragmatists: The Key to Innovation A Virtual Power Plant exists and operates in our energy system, and our energy system is arguably one of the most complex engineering feats of mankind. This complexity projects itself on the diversity of stakeholders: energy consumers, transmission system operators, distribution system operators, retailers, device manufacturers, market operators,… All have different (and often conflicting) interests and roles to play. Furthermore a consumer can cast a vote and hence once you dwell into the area of decentralized flexibility and have an impact on the energy bill of consumers, politics becomes part of the game. Innovating in this web of stakeholders creates ripples which lead to inertia, pushbacks and opportunities. It is a tough market to work in. Managing this complexity means that the team requires diverse talent that thrives in such a tough environment, and does not take no for an answer while remaining diplomatic. Integrity and energy system knowledge are key but equally important is the ability to shift gears when the road becomes peculiar and steep, which brings us to tenacity , another of our core values. Integrity in Action: Fostering Respectful Collaboration Forging “practical” decision algorithms to operate over 1e6 energy-constrained assets in real time under uncertainty, embedded in our physical energy system and integrated in flexibility and energy markets is a fun little problem on which hundreds if not thousands of papers have been written. Many papers (including some of my own work) have been sacrificed on the altar of academic fetishes such as mathematical bombast or superficial AI hype opportunism. Equally so, the blanket of pragmatism has suffocated many elegant and truly scalable solutions by adhering to the infamous 20-80 rule (more 5-60 in practice). At Beebop, we have the vision of building deeply practical demand response technology that can handle massive scale, with the understanding that we need both pragmatic and truly disruptive algorithms. Complexity in Focus: Managing Stakeholder Diversity As a consequence, we are building a team with algorithmic dreamers, researchers, practitioners, pragmatists and builders seeking to explore and build VPP technology with the right complexity at the right place*. We hereby combine techniques from operations research such as model-based (based upon physical principles) optimization next to data-driven approaches such as reinforcement learning and structured neural network architectures, but equally so we pragmatically embed rule-based controllers and decision trees or algorithms that learn to interact with consumers. Building such architecture requires a team with an unrelenting drive to deliver quality, on time, never losing track of our vision and impact. This brings me to professional excellence, another core value. Tenacity: Fueling Progress in a Challenging Environment When I got the opportunity to join beebop.ai earlier this year, I saw an opportunity to build a balanced team together with Evelyn Heylen , Julien Leprince , Olivier Deckers , Sandra Trittin and Jan-Willem Rombouts , sharing these core values. I saw an opportunity to do what gives me meaning, an opportunity to contribute to real decarbonisation of society (not just on paper), a key challenge for this and next generations. And here I am. I know where the journey starts, not where it will lead but if you are still reading this, you might be intrigued and maybe you know someone (including yourself) that shares some of the values above and is willing to work with us on building VPP*. If so, send us a mail: beebop@beebop.ai .