How SkyElectric Uses Scylla to Power Its Smart Energy Platform
SkyElectric’s mission is “to provide sustainable energy to all” by revolutionizing how power is delivered for the world — “the great energy challenge of the 21st Century.” SkyElectric takes seriously this mission to bridge the gap in energy usage between the developed and developing nations of the world, and to address climate change. We at ScyllaDB were honored to have Jehannaz Khan, Director of Engineering of SkyElectric, and Meraj Rasool, SkyElectric DevOps Lead, speak at Scylla Summit 2019 about their use case.
The Challenge to Make Clean Energy Universally Available
In Pakistan electricity as a resource is stretched incredibly thin. This nation of 216 million people (about two thirds the population of the U.S.) produces 20 gigawatts (GW) of energy. In comparison, the U.S. produces over 1,072 GW — over 50 times the amount of electricity. Pakistan consistently suffers a 2,000MW shortfall in production behind demand, about 10% of overall production, which leads directly to load shedding — the deliberate shutdown of parts of the energy grid to prevent systemic collapse. Consequently, rural areas suffer up to 11 hours of blackouts daily and urban areas can suffer 8 hours of daily power cuts.
Unreliable energy adversely impacts the commercial and industrial sectors and the Pakistani economy overall. Jehannaz said “growth in GDP is directly linked to energy usage. The more energy you use, the more your GDP grows.” Indeed, an analysis by the U.S. Energy Information Agency (EIA) states the energy crisis costs Pakistan somewhere between 2–3% of GDP.
Pakistan has aggressive national goals to increase renewables use. Whereas wind, solar and biomass presently account for 4% of energy generation, the government seeks to increase that rate to 20–30% by 2030 through incentives as well as reforms such as allowing net-metering.
Pakistan also faces significant effects of climate change and has been described as risking being “ground zero for global warming consequences.” Extreme temperatures can reach over 50º C (122º F), not only stressing the power grid but also increasing mortality rates. So while the government can increase its energy through its abundant fossil fuel reserves (including coal and natural gas), Jehannaz emphasized “we wanted to know how we could provide this energy in a way that didn’t exacerbate climate change.”
To make clean energy universally available by building a distributed and intelligent solar and energy storage grid, managed via the Internet, across the world.
— SkyElectric’s vision and mission
Building a Cloud-Based System for Energy Management
To address these pressing challenges SkyElectric launched the Smart Energy System in February 2017, combining a solar hybrid inverter, grid interconnect and lithium-ion battery solution, “so you never have to worry about the lights going out.” These nanogrids — small power plants in homes and businesses — integrate data services to a cloud platform, including a mobile app so you could check on your power system at any time, and a network operations center (NOC) manned by a customer care team monitoring your system from afar.
An example of statistics from SkyElectric showing that for the monthly period ending 29 October 2019, 570 Megawatt hours (MWh) of their customers’ power was produced by solar, and 422 MWh came from the grid. This accounted for saving over 424 metric tonnes of CO2 emissions.
The Smart Energy System is held together by SmartFlow, an AI-driven energy management solution that controls how it operates. It is fed by historical, current and predicted information on grid availability — trying to anticipate when and if a customer’s power may shut off — plus tariff structures, sunlight, load and storage levels all to ensure that their customers maintain the highest levels of power availability at the lowest rates.
Even as they scaled to their first thousand systems installed in residences and businesses across the country SkyElectric needed to reengineer their back-end systems to deal with the massive amounts of data now under management as well as for capacity to meet their planned expansions. With over 300,000 data points per minute being sent to the cloud, that’s 157 billion data points in a year. They needed to store that data, run fault analysis on it, provide fault rectification, and perform AI algorithms to improve the service.
Evolution of the SkyElectric Cloud
The first prototype (v1) of their back end was built on the Java programming language and the MySQL database in an on-premises environment. Well before they hit their first 50 beta customers they saw problems with scaling. Database write latencies were hitting 2 seconds, and read latencies for analysis of just hours of data could take minutes. Scanning data for even a full day became prohibitive. There was no way to scale to their vision of thousands of systems given this version of their infrastructure. And there was no way to provide the security and peace of mind of their service to their customers if it was going down every few days.
The lesson they had learned was that they needed mutable data with ACID properties (users, systems configurations) as well as, separately, an immutable (read-only) time series data store for their energy and system health metrics.
In a second design (v2) SkyElectric considered MongoDB and Node.js instead. But they abandoned work on this iteration before reaching production.
Their efforts turned to a new design (v3) built with Elixir on top of Scylla and Elasticsearch (NoSQL), as well as PostgreSQL (SQL), deployed to AWS. Their use case focused on performance, scalability, throughput and availability. But also ease-of-use. With those as requirements, they searched into what were the latest databases available. “And that’s when we came across Scylla.”
Though they looked at Cassandra and RiakTS, these did not meet all of their criteria. Cassandra had serious resource issues, according to Meraj. He was also reticent to go into production with a Java-based solution given his past experiences. RiakTS seemed perfect, but Meraj saw it as a major flaw that you cannot update the schema. For example, he said that with every sprint they make minor changes. So this was a requirement of the application.
Meraj familiarized himself with Scylla through videos, benchmarks and other information online. He was greatly relieved to learn that Scylla was written in C++.
SkyElectric has now been in production with Scylla for a year and a half without any major problems. Scylla serves as the primary store for time series data, which has scaled both in request volume and overall data stored by an order of magnitude, now more than one terabyte. Meanwhile average latencies remain low and predictable: 1.4 ms for writes, and submillisecond for reads. “We are very confident we will be able to grow with it to thousands and thousands of systems.”
As a DevOps veteran Meraj also cited the ease of operations that came with Scylla. “I sleep well because of Scylla.” On adding nodes to his expanding cluster, or to replace a failed node, “It was painless. It was very easy to add a node without worrying about my data.” Software updates were also very easy, “Hassle free”.
Meraj noted he was using Scylla Manager to support weekly repairs, but looked forward to the future capabilities for Scylla Manager to also do backups and restores, which he currently does via a Python script to Amazon S3.
“Regarding performance, I will say I have seen an optimal usage of the hardware for Scylla.” For a company trying to maximize power, quite literally, utilizing every core and all the RAM in an instance is very important to Meraj and the team at SkyElectric. “This is a very big ‘yes’ for me.”
He was looking forward to the Change Data Capture (CDC) feature announced at the summit, which is coming up soon in an impending release of Scylla.
Lastly, as an open source user, Meraj was very appreciative of all the help he received through the community via Slack.
https://www.youtube.com/watch?v=8b-1ulck1YY
We look forward to hearing the next chapters of the SkyElectric saga. Would you like to share your own story of the mission-critical work you are doing with Scylla? If so, drop us a line.
